From patchwork Wed Dec 23 10:01:40 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Daniel J Blueman X-Patchwork-Id: 7909851 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Original-To: patchwork-linux-pci@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id A5178BEEE5 for ; Wed, 23 Dec 2015 10:18:21 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 9E21120453 for ; Wed, 23 Dec 2015 10:18:20 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id A89BC20444 for ; Wed, 23 Dec 2015 10:18:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S933814AbbLWKSQ (ORCPT ); Wed, 23 Dec 2015 05:18:16 -0500 Received: from mail-db3on0139.outbound.protection.outlook.com ([157.55.234.139]:49696 "EHLO emea01-db3-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S934024AbbLWKSN (ORCPT ); Wed, 23 Dec 2015 05:18:13 -0500 X-Greylist: delayed 929 seconds by postgrey-1.27 at vger.kernel.org; Wed, 23 Dec 2015 05:18:11 EST DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=numascaleo365.onmicrosoft.com; s=selector1-numascale-com; h=From:To:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=E1zvmT8tCHLG0js1TbLZHCWS5tzp/PdCwgkKkg+zNgs=; b=nBXw9mXRua4enhWpwV3/IIqQg97J/awbkdfafNB+5icd1jFnnbYrYW8K/FDxeBOBexO80uf4a34muFotddln56wL70azpyc2gQ0zaXYZQUJFQHANf0q7/S8A5fnbQkI+ORNRSCcbCRBAim532cmM7bAHOgKQqmdeMWbkp1qKAX0= Authentication-Results: spf=none (sender IP is ) smtp.mailfrom=daniel@numascale.com; Received: from exige.int.numascale.com (175.156.152.184) by AM4PR07MB1428.eurprd07.prod.outlook.com (10.165.248.19) with Microsoft SMTP Server (TLS) id 15.1.361.13; Wed, 23 Dec 2015 10:02:34 +0000 From: Daniel J Blueman To: Bjorn Helgaas , Jeff Kirsher , Jesse Brandeburg , Shannon Nelson , Carolyn Wyborny , Don Skidmore , Bruce Allan , John Ronciak , Mitch Williams CC: Daniel J Blueman , , , , , Steffen Persvold , Jiang Liu , Thomas Gleixner Subject: [PATCH 1/2] PCI: Add mechanism to find topologically near cores Date: Wed, 23 Dec 2015 18:01:40 +0800 Message-ID: <1450864901-16712-1-git-send-email-daniel@numascale.com> X-Mailer: git-send-email 2.5.0 MIME-Version: 1.0 X-Originating-IP: [175.156.152.184] X-ClientProxiedBy: DB4PR05CA0004.eurprd05.prod.outlook.com (25.160.40.14) To AM4PR07MB1428.eurprd07.prod.outlook.com (25.165.248.19) X-Microsoft-Exchange-Diagnostics: 1; AM4PR07MB1428; 2:iilwPHH7pJ4y3HIxHtZS8GYB0FyEZGr3QZu1B6b/8ZMl9pBO6GHsHelWlTKNOsPACzUHXb/UitdyWJu5w0Wym0h7YHeTdDn/VocBf0Z3xH0DlQJ6WPLqRAY/HNK648uvjCCBiBmBJiR7lUg4hTm0pA==; 3:wsEXdl1Je0FSgS/q0TrrF3Pc4K5v0wV1arnUDG3UN03fJydCoJ7qeQyvyI0g1f4oxbLaUEJXvLEoDruZxZ0GWYJxacbetmP9b1dL41Wb7G2YxtRgSc9RK4d0c1QJ16p1; 25:zz61np3WILWkXvfVf7A6B4JineUNeQILx/L0EqZdKurjlUbaCsh5swJ0lVcaKE2Y24CWusLsDjb0tsNIxcIcctfJ+55vq6msLr1Wh1gWxN9RFy5JYQmdoI1gec4PAh0fyipdm2i+dHqb8x9xM1rZjbPXhGBfnDyt04kHgmovVyb2awFKocIaBG4PUVvIFsdUJ1SjdR+bAJU3MgXZyEVYQIVD81lwV6Mlvz+1jNHjBLZ8BprXPSzMD1scbXkX1XlJ9o9M9m2kfNICymZ24+bkew==; 4:doXc2AK/X498vS+y7UdEX1qy6Iz+LiNtyivK28aeGNfLzgq0StN+8P1q6Cq1HZIab0phJpsv+kS1+VZNRYIMYoch3E1RDrtvnIm7Z/EtNMfCZx+gImd2CVSPzFwZeF9lpjYojraxNke9dlJ6D+K4Q/npBLpUUR7mE2Ppbo3ZJY5x5dqDi2uEqDKSBBhFL9+vXLPRdiWMcCO1qWbfXR4EUsRBeI3QoLlkrtnuapH1tW2kMTTUkyeGcVK31LpsVcs1WSgai+G/RXJ6tuAFZQfjT9UESz2lbWUYkJCa7ezKMxVLviT/y8qZJ7IYCn8X5n5wDzGGol2hRMOUp7ZxtZd/+Oo6Ubdiu1FIWzyKiQ851qKSvOMU Pq4IQtFB9qEWl2aW X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AM4PR07MB1428; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(601004)(2401047)(5005006)(520078)(8121501046)(10201501046)(3002001); SRVR:AM4PR07MB1428; BCL:0; PCL:0; RULEID:; SRVR:AM4PR07MB1428; X-Forefront-PRVS: 0799B1B2D7 X-Forefront-Antispam-Report: SFV:NSPM; SFS:(10019020)(6009001)(189002)(199003)(81156007)(86362001)(5003940100001)(48376002)(19580405001)(50986999)(47776003)(66066001)(50226001)(105586002)(87976001)(42186005)(3846002)(19580395003)(1096002)(5004730100002)(92566002)(97736004)(5001770100001)(101416001)(77096005)(50466002)(229853001)(5001960100002)(40100003)(36756003)(5008740100001)(106356001)(122386002)(189998001)(6116002)(33646002)(586003)(37363001); DIR:OUT; SFP:1102; SCL:1; SRVR:AM4PR07MB1428; H:exige.int.numascale.com; FPR:; SPF:None; PTR:InfoNoRecords; MX:1; A:1; LANG:en; Received-SPF: None (protection.outlook.com: numascale.com does not designate permitted sender hosts) X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; AM4PR07MB1428; 23:G93hoeAycQ4dV58smW/p26EqhQ7d3XGg/WIPeIyLp?= =?us-ascii?Q?Lk2TOfFkjKLxh8uPcIeFNjSqX0japs6NjjVPyO6v7LoClNUC3lyv/nOrB7q0?= =?us-ascii?Q?2Hn3/XXhRsLEW47TmLm0PkgIyiKDyGowQ7ofYioILwLXq5h0By6NDOF9zxW9?= =?us-ascii?Q?M5ofSxnBTxtmYSxRBWlxaISpaEeI35pwwjXKmI9eem1uAGH4hWyiCboySZx3?= =?us-ascii?Q?UtBGZ9rH/HBBYBOfgHcJbWmsvIkMzOoV/xRX8MQUYM5qwuRrJB/exPT+NyjU?= =?us-ascii?Q?bhrVjEAkl/g+Gx8LcIzkULFKk2dfLP3tjF/DVgZgiPgw76RS/t+NJQaYZgr4?= =?us-ascii?Q?EArvm457sXv8Ue1xkDUMr2ADulH0KRdaDiBDfMX4i8ljwiE9cmG873C6gHzU?= =?us-ascii?Q?rhrIG3Mv/zTZIPMGX/TzNuIkIVdGL30eNgVxkhh0vCkPEcBiMA/ToAgXp0P/?= =?us-ascii?Q?mY10ElG5wI1+vEVPbhXK0phiBd2DW7oKTTLZ1BwLSq67iqFVXryzAjhm8ZO8?= =?us-ascii?Q?vMES1kAZDec8bwrLNiFganpimDppILWijlvFLfjAsKPc2+XOOkm5iaC1boE6?= =?us-ascii?Q?A8IEoJZxo9To3Eoq1+N496H4rCKrfNPtBAHO0FUvfaUD73GSP7LmbNs3XTWp?= =?us-ascii?Q?lX2Qzeus+y8ldCT5+FESfS6T6AkoAEIqRKlGR4p9MDrPyiWjH0bzSsEO9C1q?= =?us-ascii?Q?sdioyQWDkwqTsOnWPOdDl0pAAdWQ0DSsY53SmorRCGd4ZOkS5iGwD2OhBkG4?= =?us-ascii?Q?hjQHsrmbSlR0U/MBuSSPWkbLKUfofFzgJsxBE/eGuoffYXh+Cddn8GhMc1oz?= =?us-ascii?Q?izHdJo38HVfTdKuaOq5wJRGwSmxIl/IR/+jX5ZWtDrGrO17aOS2dQKgXVR9W?= =?us-ascii?Q?NRIYan58uvlppCT3shciWeoH9TK6+9qUV36BvKwpe0Xh49tnB/Z49ZiTA3Pu?= =?us-ascii?Q?W4YfUIwUi5mvANcYgw7fzr4FaMPy924QYi5Ghn4E3VPuqz1tg4K0xYsKYNei?= =?us-ascii?Q?AK6evC0qsZFXPZnZR9INMGw?= X-Microsoft-Exchange-Diagnostics: 1; AM4PR07MB1428; 5:sTrlc9+ytq7cXylNaEX+NmdTNI21ROoTLmw3xUwqaNA3pgIZhVct536ThEy2zPwnC5RWF2En0+AX+zpz//Z90HnmJlpc6enk4Erh7u2eNzNfqgZ25FjMjTSdlz/3CV991qEioDbSyNseBJVNxq0d1A==; 24:M8Gh4Wz0my6dZjGR7IOf8bwYjUM4tSr4axSqxS70FIOHyTQ+GEmUSnNwKIMqhtgD3hkjpiA8XWoa8cR8X77VGKwtbKzz2+oA1FEju6QCuVg= SpamDiagnosticOutput: 1:23 SpamDiagnosticMetadata: NSPM X-OriginatorOrg: numascale.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Dec 2015 10:02:34.1333 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM4PR07MB1428 Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,RP_MATCHES_RCVD,T_DKIM_INVALID,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Some devices (eg ixgbe) make assumptions about device to core locality when specifying interrupts locality hints and allocate starting from core 0. Moreover, interrupts may not be routable to distant NUMA nodes due to the 8-bit APIC ID space limitations. Provide a mechanism drivers can use to find cores with reasonable locality to a device; use the existing precendent of RECLAIM_DISTANCE (30), wrapping the offset. Signed-off-by: Daniel J Blueman --- drivers/pci/pci.c | 15 +++++++++++++++ include/linux/pci.h | 1 + 2 files changed, 16 insertions(+) -- 2.5.0 -- To unsubscribe from this list: send the line "unsubscribe linux-pci" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index 314db8c..d5535d1 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -4833,6 +4833,22 @@ void __weak pci_fixup_cardbus(struct pci_bus *bus) } EXPORT_SYMBOL(pci_fixup_cardbus); +int cpu_near_dev(const struct pci_dev *pdev, unsigned offset) +{ + /* Start search from node device is on for optimal locality */ + int localnode = pcibus_to_node(pdev->bus); + int cpu = cpumask_first(cpumask_of_node(localnode)); + + while (offset--) { + do { + cpu = (cpu + 1) % nr_cpu_ids; + } while (!cpu_online(cpu) || node_distance(cpu_to_node(cpu), + localnode) > RECLAIM_DISTANCE); + } + + return cpu; +} + static int __init pci_setup(char *str) { while (str) { diff --git a/include/linux/pci.h b/include/linux/pci.h index 6ae25aa..f7491bd 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -842,6 +842,7 @@ void pci_stop_root_bus(struct pci_bus *bus); void pci_remove_root_bus(struct pci_bus *bus); void pci_setup_cardbus(struct pci_bus *bus); void pci_sort_breadthfirst(void); +int cpu_near_dev(const struct pci_dev *pdev, unsigned offset); #define dev_is_pci(d) ((d)->bus == &pci_bus_type) #define dev_is_pf(d) ((dev_is_pci(d) ? to_pci_dev(d)->is_physfn : false)) #define dev_num_vf(d) ((dev_is_pci(d) ? pci_num_vf(to_pci_dev(d)) : 0))