From patchwork Thu Jun 10 16:06:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 12313441 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F2BA6C48BDF for ; Thu, 10 Jun 2021 16:06:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id CF162613E7 for ; Thu, 10 Jun 2021 16:06:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230220AbhFJQIN (ORCPT ); Thu, 10 Jun 2021 12:08:13 -0400 Received: from ale.deltatee.com ([204.191.154.188]:60120 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230134AbhFJQIM (ORCPT ); Thu, 10 Jun 2021 12:08:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:MIME-Version:References:In-Reply-To: Message-Id:Date:Cc:To:From:content-disposition; bh=vFNfL4ZdQLhqnbIZ7A8c4m+bWnFu30CgeGDNcmRJsLw=; b=grnaUqWeLVr5/nldlcUotiB4hi F3GA9IqyKBiLs2+Wvz9vU74yn5TdurnUc+Xx01JkRvw0efOAULxre6qXBugoHxpeQRxfxRSWCCEdr a53xFQVDIHr0I41rj3RtINMBdWmZVoCAY//gbmGd6tC5rQ2i7EtRA71VFIsmhtpHVIubBotlg2jDQ 653M88TCg0TrwO6QMiSmgAyIwvNcN6DfqE1TNcEZNvGgB7DhkC5zaDXn2Y59TkZ7xqcKHgfmIJygs AFnA2immnYjY4NmHWYA0cRsQLpCXnsACH12OsUxfpeR4r7nNR00CbJrqxbNUxB7mb4MMyU9lBUy9z rBDdkYQw==; Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lrNCA-0000Jh-Kn; Thu, 10 Jun 2021 10:06:15 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.92) (envelope-from ) id 1lrNC8-0007Pg-Dz; Thu, 10 Jun 2021 10:06:12 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Stephen Bates , Christoph Hellwig , Dan Williams , Jason Gunthorpe , =?utf-8?q?Christian_K=C3=B6nig?= , John Hubbard , Don Dutile , Logan Gunthorpe Date: Thu, 10 Jun 2021 10:06:04 -0600 Message-Id: <20210610160609.28447-2-logang@deltatee.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210610160609.28447-1-logang@deltatee.com> References: <20210610160609.28447-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, helgaas@kernel.org, sbates@raithlin.com, hch@lst.de, dan.j.williams@intel.com, jgg@ziepe.ca, christian.koenig@amd.com, jhubbard@nvidia.com, ddutile@redhat.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v1 1/6] PCI/P2PDMA: Rename upstream_bridge_distance() and rework documentation X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org The function upstream_bridge_distance() has evolved such that it's name is no longer entirely reflective of what the function does. The function not only calculates the distance between two peers but also calculates how the DMA addresses for those two peers should be mapped. Thus, rename the function to calc_map_type_and_dist() and rework the documentation to better describe the two pieces of information the function returns. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 63 ++++++++++++++++++++++---------------------- 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 196382630363..6f90e9812f6e 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -354,7 +354,7 @@ static bool host_bridge_whitelist(struct pci_dev *a, struct pci_dev *b) } static enum pci_p2pdma_map_type -__upstream_bridge_distance(struct pci_dev *provider, struct pci_dev *client, +__calc_map_type_and_dist(struct pci_dev *provider, struct pci_dev *client, int *dist, bool *acs_redirects, struct seq_buf *acs_list) { struct pci_dev *a = provider, *b = client, *bb; @@ -433,17 +433,18 @@ static unsigned long map_types_idx(struct pci_dev *client) } /* - * Find the distance through the nearest common upstream bridge between - * two PCI devices. + * Calculate the P2PDMA mapping type and distance between two PCI devices. * - * If the two devices are the same device then 0 will be returned. + * If the two devices are the same device then PCI_P2PDMA_MAP_BUS_ADDR + * and a distance of 0 will be returned. * * If there are two virtual functions of the same device behind the same - * bridge port then 2 will be returned (one step down to the PCIe switch, - * then one step back to the same device). + * bridge port then PCI_P2PDMA_MAP_BUS_ADDR and a distance of 2 will be + * returned (one step down to the PCIe switch, then one step back to the + * same device). * * In the case where two devices are connected to the same PCIe switch, the - * value 4 will be returned. This corresponds to the following PCI tree: + * distance of 4 will be returned. This corresponds to the following PCI tree: * * -+ Root Port * \+ Switch Upstream Port @@ -454,31 +455,31 @@ static unsigned long map_types_idx(struct pci_dev *client) * * The distance is 4 because we traverse from Device A through the downstream * port of the switch, to the common upstream port, back up to the second - * downstream port and then to Device B. - * - * Any two devices that cannot communicate using p2pdma will return - * PCI_P2PDMA_MAP_NOT_SUPPORTED. + * downstream port and then to Device B. The mapping type returned will depend + * on the ACS redirection setting of the bridges along the path. If ACS + * redirect is set on any bridge port in the path then the TLPs will go through + * the host bridge. Otherwise PCI_P2PDMA_MAP_BUS_ADDR is returned. * * Any two devices that have a data path that goes through the host bridge - * will consult a whitelist. If the host bridges are on the whitelist, - * this function will return PCI_P2PDMA_MAP_THRU_HOST_BRIDGE. - * - * If either bridge is not on the whitelist this function returns - * PCI_P2PDMA_MAP_NOT_SUPPORTED. + * will consult a whitelist. If the host bridge is in the whitelist, + * this function will return PCI_P2PDMA_MAP_THRU_HOST_BRIDGE with the + * distance set to the number of ports per above. If the device is not + * in the whitelist the type will be returned PCI_P2PDMA_MAP_NOT_SUPPORTED. * - * If a bridge which has any ACS redirection bits set is in the path, - * acs_redirects will be set to true. In this case, a list of all infringing - * bridge addresses will be populated in acs_list (assuming it's non-null) - * for printk purposes. + * If any ACS redirect bits are set, then the acs_redirects boolean will be + * set to true and their pci device name will be appended to the acs_list + * seq_buf. This seq_buf is used to print a warning informing the user + * how to disable ACS using a command line parameter. + * (See calc_map_type_and_dist_warn() below) */ static enum pci_p2pdma_map_type -upstream_bridge_distance(struct pci_dev *provider, struct pci_dev *client, +calc_map_type_and_dist(struct pci_dev *provider, struct pci_dev *client, int *dist, bool *acs_redirects, struct seq_buf *acs_list) { enum pci_p2pdma_map_type map_type; - map_type = __upstream_bridge_distance(provider, client, dist, - acs_redirects, acs_list); + map_type = __calc_map_type_and_dist(provider, client, dist, + acs_redirects, acs_list); if (map_type == PCI_P2PDMA_MAP_THRU_HOST_BRIDGE) { if (!cpu_supports_p2pdma() && @@ -494,8 +495,8 @@ upstream_bridge_distance(struct pci_dev *provider, struct pci_dev *client, } static enum pci_p2pdma_map_type -upstream_bridge_distance_warn(struct pci_dev *provider, struct pci_dev *client, - int *dist) +calc_map_type_and_dist_warn(struct pci_dev *provider, struct pci_dev *client, + int *dist) { struct seq_buf acs_list; bool acs_redirects; @@ -505,8 +506,8 @@ upstream_bridge_distance_warn(struct pci_dev *provider, struct pci_dev *client, if (!acs_list.buffer) return -ENOMEM; - ret = upstream_bridge_distance(provider, client, dist, &acs_redirects, - &acs_list); + ret = calc_map_type_and_dist(provider, client, dist, &acs_redirects, + &acs_list); if (acs_redirects) { pci_warn(client, "ACS redirect is set between the client and provider (%s)\n", pci_name(provider)); @@ -565,11 +566,11 @@ int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients, } if (verbose) - ret = upstream_bridge_distance_warn(provider, - pci_client, &distance); + ret = calc_map_type_and_dist_warn(provider, pci_client, + &distance); else - ret = upstream_bridge_distance(provider, pci_client, - &distance, NULL, NULL); + ret = calc_map_type_and_dist(provider, pci_client, + &distance, NULL, NULL); pci_dev_put(pci_client); From patchwork Thu Jun 10 16:06:05 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 12313439 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B344CC48BD1 for ; Thu, 10 Jun 2021 16:06:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 96676613D8 for ; Thu, 10 Jun 2021 16:06:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230201AbhFJQIN (ORCPT ); Thu, 10 Jun 2021 12:08:13 -0400 Received: from ale.deltatee.com ([204.191.154.188]:60124 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230161AbhFJQIM (ORCPT ); Thu, 10 Jun 2021 12:08:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:MIME-Version:References:In-Reply-To: Message-Id:Date:Cc:To:From:content-disposition; bh=kGb8w7OA+VecwjXRg6RAtUZUrHcBmrTzMPMqQqpjkm8=; b=KHxnMIMpdNc7Po5o4zvLt0xMtY O8CKxzCulcw4C6nUH683CeLFb8/OPy+KDkDPcyv4lZKPe9Ummse/RTBVVjCcfwtGlSpEVmXW9pGhK kvazm4kIWUUX8SuJkmpLi1DWXq7JZkffKjpiAV9MNAUinSyzeAUCy5xQqKcpPFQlViqtNAVe9oWgf Mw/6+ukwawWE66h3aLiHwwmnJCb1O7AyiQquQyZIEIeKKKrSfoFF/tr38wMvawcg8HZKkRLB0El2O VtYjZLWaWwcwi8bk0ixvDdJ479NHzzXIfmG3qD6mTjiMvqptMQ/J+zJyIX/ZgYD/bTuV30KMryaq5 wH1K1iDg==; Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lrNCA-0000Ji-Kn; Thu, 10 Jun 2021 10:06:15 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.92) (envelope-from ) id 1lrNC8-0007Pj-Iq; Thu, 10 Jun 2021 10:06:12 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Stephen Bates , Christoph Hellwig , Dan Williams , Jason Gunthorpe , =?utf-8?q?Christian_K=C3=B6nig?= , John Hubbard , Don Dutile , Logan Gunthorpe Date: Thu, 10 Jun 2021 10:06:05 -0600 Message-Id: <20210610160609.28447-3-logang@deltatee.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210610160609.28447-1-logang@deltatee.com> References: <20210610160609.28447-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, helgaas@kernel.org, sbates@raithlin.com, hch@lst.de, dan.j.williams@intel.com, jgg@ziepe.ca, christian.koenig@amd.com, jhubbard@nvidia.com, ddutile@redhat.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v1 2/6] PCI/P2PDMA: Use a buffer on the stack for collecting the acs list X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org In order to call the calc_map_type_and_dist_warn() function from a dma_map operation, the function must not sleep. The only reason it sleeps is to allocate memory for the seq_buf to print a verbose warning telling the user how to disable ACS for that path. Instead of allocating the memory with kmalloc, allocate it on the stack with a smaller buffer. A 128B buffer is enough to print 10 pci device names. A system with 10 bridge ports between two devices that have ACS enabled would be unusually large, so this should still be a reasonable limit. This also allows cleaning up the awkward (and broken) return with -ENOMEM which contradicts the return type and the caller was not prepared for. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 6f90e9812f6e..3a5fb63c5f2c 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -500,11 +500,10 @@ calc_map_type_and_dist_warn(struct pci_dev *provider, struct pci_dev *client, { struct seq_buf acs_list; bool acs_redirects; + char buf[128]; int ret; - seq_buf_init(&acs_list, kmalloc(PAGE_SIZE, GFP_KERNEL), PAGE_SIZE); - if (!acs_list.buffer) - return -ENOMEM; + seq_buf_init(&acs_list, buf, sizeof(buf)); ret = calc_map_type_and_dist(provider, client, dist, &acs_redirects, &acs_list); @@ -522,8 +521,6 @@ calc_map_type_and_dist_warn(struct pci_dev *provider, struct pci_dev *client, pci_name(provider)); } - kfree(acs_list.buffer); - return ret; } From patchwork Thu Jun 10 16:06:06 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 12313437 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07A36C47094 for ; Thu, 10 Jun 2021 16:06:18 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id DCC9F6128A for ; Thu, 10 Jun 2021 16:06:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230197AbhFJQIN (ORCPT ); Thu, 10 Jun 2021 12:08:13 -0400 Received: from ale.deltatee.com ([204.191.154.188]:60118 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229942AbhFJQIM (ORCPT ); Thu, 10 Jun 2021 12:08:12 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:MIME-Version:References:In-Reply-To: Message-Id:Date:Cc:To:From:content-disposition; bh=RHKX0JViuXrszDPT9ENKjwACXQHnqdf/mlZV6OtxZMk=; b=j/buvVMVxNeZgnUQSBBU3CVZKs a2TBnf6lvkOOlOSwYxa8tgAZ49m5xW7yohaAVCLS3p+UZwoAqoini+WGK44cuPt4nCCOb51zQj/OP uOmEa7fRTTUD8C+RdUUkJkNCYPMGELa1Hn/IrdBBCMk/qhayCw4Cvk5iSoo8HlmbPQExlPw09UucK y5marmFv0XkfVPmXE/g2k6uQ0oBS6BkhL1Lra7PcAww34oPcBGsQ9WEgkBxtbad2Cfq1Zx9KAwM5C Trp9Bq/esNrpxheop98++WSrJ08GNDt+QHpv7uxsERU8pNHVR4BHslCL//iy6UVKp8GgyTKCaHYuF GfNyySyg==; Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lrNCA-0000Jj-Kn; Thu, 10 Jun 2021 10:06:15 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.92) (envelope-from ) id 1lrNC8-0007Pm-Mx; Thu, 10 Jun 2021 10:06:12 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Stephen Bates , Christoph Hellwig , Dan Williams , Jason Gunthorpe , =?utf-8?q?Christian_K=C3=B6nig?= , John Hubbard , Don Dutile , Logan Gunthorpe Date: Thu, 10 Jun 2021 10:06:06 -0600 Message-Id: <20210610160609.28447-4-logang@deltatee.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210610160609.28447-1-logang@deltatee.com> References: <20210610160609.28447-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, helgaas@kernel.org, sbates@raithlin.com, hch@lst.de, dan.j.williams@intel.com, jgg@ziepe.ca, christian.koenig@amd.com, jhubbard@nvidia.com, ddutile@redhat.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v1 3/6] PCI/P2PDMA: Cleanup type for return value of calc_map_type_and_dist() X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org Instead of using an int for the return value of this function use the correct enum pci_p2pdma_map_type. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 3a5fb63c5f2c..09c864f193d2 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -544,11 +544,11 @@ calc_map_type_and_dist_warn(struct pci_dev *provider, struct pci_dev *client, int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients, int num_clients, bool verbose) { + enum pci_p2pdma_map_type map; bool not_supported = false; struct pci_dev *pci_client; int total_dist = 0; - int distance; - int i, ret; + int i, distance; if (num_clients == 0) return -1; @@ -563,15 +563,15 @@ int pci_p2pdma_distance_many(struct pci_dev *provider, struct device **clients, } if (verbose) - ret = calc_map_type_and_dist_warn(provider, pci_client, + map = calc_map_type_and_dist_warn(provider, pci_client, &distance); else - ret = calc_map_type_and_dist(provider, pci_client, + map = calc_map_type_and_dist(provider, pci_client, &distance, NULL, NULL); pci_dev_put(pci_client); - if (ret == PCI_P2PDMA_MAP_NOT_SUPPORTED) + if (map == PCI_P2PDMA_MAP_NOT_SUPPORTED) not_supported = true; if (not_supported && !verbose) From patchwork Thu Jun 10 16:06:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 12313449 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64D30C48BE5 for ; Thu, 10 Jun 2021 16:06:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 5B73A613D0 for ; Thu, 10 Jun 2021 16:06:26 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230377AbhFJQIU (ORCPT ); Thu, 10 Jun 2021 12:08:20 -0400 Received: from ale.deltatee.com ([204.191.154.188]:60186 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230368AbhFJQIR (ORCPT ); Thu, 10 Jun 2021 12:08:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:MIME-Version:References:In-Reply-To: Message-Id:Date:Cc:To:From:content-disposition; bh=QeMDlTK6S7rVNcdHO1iGH78RGAnpL8RDeAFDcSCpHHg=; b=XrSLkbPI2G3LBEDiqZW6cM4HtU R44bH+nZDfJABLhQ+SEQPH9MB2l/2KgjG3CzWglXqHILUL9fD+z0wyFbpJkeXe4JnCa1TLjC7WbQm GDLb+4X106ViW3DRNfnjepTIuEOnj0IwR8pIbjpFD+vmel1jKasORiNygjmOhztQ2F1CTFsuO5qPK 6pn9KgHLH6yV89uRwk/OHSPY3KLT7g9h+oXEAgyb1nUP7MDJbGGD1/eeZ/wB0GyORXI41AFvQRPhG ov7mvzwBZUDruT+zrlyH+48sDTqA+4gIIHnqrZQdn4V8G6FPjiXy0nv+eKbvqTsRop7zjuyBX+MZe kIJT2y4Q==; Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lrNCA-0000Jk-Kn; Thu, 10 Jun 2021 10:06:21 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.92) (envelope-from ) id 1lrNC8-0007Pp-Qy; Thu, 10 Jun 2021 10:06:12 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Stephen Bates , Christoph Hellwig , Dan Williams , Jason Gunthorpe , =?utf-8?q?Christian_K=C3=B6nig?= , John Hubbard , Don Dutile , Logan Gunthorpe Date: Thu, 10 Jun 2021 10:06:07 -0600 Message-Id: <20210610160609.28447-5-logang@deltatee.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210610160609.28447-1-logang@deltatee.com> References: <20210610160609.28447-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, helgaas@kernel.org, sbates@raithlin.com, hch@lst.de, dan.j.williams@intel.com, jgg@ziepe.ca, christian.koenig@amd.com, jhubbard@nvidia.com, ddutile@redhat.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v1 4/6] PCI/P2PDMA: Print a warning if the host bridge is not in the whitelist X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org If the host bridge is not in the whitelist print a warning in the calc_map_type_and_dist_warn() path detailing the vendor and device IDs that would need to be added to the whitelist. Suggested-by: Don Dutile Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 09c864f193d2..2de4a9e2da58 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -309,7 +309,7 @@ static const struct pci_p2pdma_whitelist_entry { }; static bool __host_bridge_whitelist(struct pci_host_bridge *host, - bool same_host_bridge) + bool same_host_bridge, bool warn) { struct pci_dev *root = pci_get_slot(host->bus, PCI_DEVFN(0, 0)); const struct pci_p2pdma_whitelist_entry *entry; @@ -331,6 +331,10 @@ static bool __host_bridge_whitelist(struct pci_host_bridge *host, return true; } + if (warn) + pci_warn(root, "Host bridge not in P2PDMA whitelist: %04x:%04x\n", + vendor, device); + return false; } @@ -338,16 +342,17 @@ static bool __host_bridge_whitelist(struct pci_host_bridge *host, * If we can't find a common upstream bridge take a look at the root * complex and compare it to a whitelist of known good hardware. */ -static bool host_bridge_whitelist(struct pci_dev *a, struct pci_dev *b) +static bool host_bridge_whitelist(struct pci_dev *a, struct pci_dev *b, + bool warn) { struct pci_host_bridge *host_a = pci_find_host_bridge(a->bus); struct pci_host_bridge *host_b = pci_find_host_bridge(b->bus); if (host_a == host_b) - return __host_bridge_whitelist(host_a, true); + return __host_bridge_whitelist(host_a, true, warn); - if (__host_bridge_whitelist(host_a, false) && - __host_bridge_whitelist(host_b, false)) + if (__host_bridge_whitelist(host_a, false, warn) && + __host_bridge_whitelist(host_b, false, warn)) return true; return false; @@ -483,7 +488,7 @@ calc_map_type_and_dist(struct pci_dev *provider, struct pci_dev *client, if (map_type == PCI_P2PDMA_MAP_THRU_HOST_BRIDGE) { if (!cpu_supports_p2pdma() && - !host_bridge_whitelist(provider, client)) + !host_bridge_whitelist(provider, client, acs_redirects)) map_type = PCI_P2PDMA_MAP_NOT_SUPPORTED; } From patchwork Thu Jun 10 16:06:08 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 12313443 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 69D71C48BE6 for ; Thu, 10 Jun 2021 16:06:23 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 538DA613F8 for ; Thu, 10 Jun 2021 16:06:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230344AbhFJQIQ (ORCPT ); Thu, 10 Jun 2021 12:08:16 -0400 Received: from ale.deltatee.com ([204.191.154.188]:60150 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230262AbhFJQIP (ORCPT ); Thu, 10 Jun 2021 12:08:15 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:MIME-Version:References:In-Reply-To: Message-Id:Date:Cc:To:From:content-disposition; bh=ecVQN1myFhNKBV+atAyPDZ6lDFiBiAGm9A4aOtek3/g=; b=O+RUxQEfumKDvQ0B/Ew/JdU8fV gBmRlklFWe0xIg+xEL1m4vstjByraehGmTPmMDiUlcKVCqK6G8hCf+PjrQiKMPsf9JcjH2zhwXsSe ld58irkzM5Vkwnmaf+hvdCHWPBqiJPq06Xz1oITkmaeOPRQ3xraYco0wtXAJJq4XZV5Yx0BTaAwYk W+s1DmIieVPH+bQ+2O4NsAw1GaVX4F+YAlZUKxpvvD2Mo57kMyJycD+HfuMRX87OB+u5nojpcWszA b5uLvU12UnNcc+huth0rX4dAY534WDeUtFrr+geWUMJBVRQWksTNJ9pCRz/lqZGkswmDS5d5b40ar +DpgoGAw==; Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lrNCD-0000Jh-Lb; Thu, 10 Jun 2021 10:06:18 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.92) (envelope-from ) id 1lrNC8-0007Ps-Uk; Thu, 10 Jun 2021 10:06:12 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Stephen Bates , Christoph Hellwig , Dan Williams , Jason Gunthorpe , =?utf-8?q?Christian_K=C3=B6nig?= , John Hubbard , Don Dutile , Logan Gunthorpe Date: Thu, 10 Jun 2021 10:06:08 -0600 Message-Id: <20210610160609.28447-6-logang@deltatee.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210610160609.28447-1-logang@deltatee.com> References: <20210610160609.28447-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, helgaas@kernel.org, sbates@raithlin.com, hch@lst.de, dan.j.williams@intel.com, jgg@ziepe.ca, christian.koenig@amd.com, jhubbard@nvidia.com, ddutile@redhat.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v1 5/6] PCI/P2PDMA: Refactor pci_p2pdma_map_type() to take pagemap and device X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org All callers of pci_p2pdma_map_type() have a struct dev_pgmap and a struct device (of the client doing the DMA transfer). Thus move the conversion to struct pci_devs for the provider and client into this function. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 30 ++++++++++++------------------ 1 file changed, 12 insertions(+), 18 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 2de4a9e2da58..5dc1f9f62a93 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -814,12 +814,20 @@ void pci_p2pmem_publish(struct pci_dev *pdev, bool publish) } EXPORT_SYMBOL_GPL(pci_p2pmem_publish); -static enum pci_p2pdma_map_type pci_p2pdma_map_type(struct pci_dev *provider, - struct pci_dev *client) +static enum pci_p2pdma_map_type pci_p2pdma_map_type(struct dev_pagemap *pgmap, + struct device *dev) { + struct pci_dev *provider = to_p2p_pgmap(pgmap)->provider; + struct pci_dev *client; + if (!provider->p2pdma) return PCI_P2PDMA_MAP_NOT_SUPPORTED; + if (!dev_is_pci(dev)) + return PCI_P2PDMA_MAP_NOT_SUPPORTED; + + client = to_pci_dev(dev); + return xa_to_value(xa_load(&provider->p2pdma->map_types, map_types_idx(client))); } @@ -856,14 +864,8 @@ int pci_p2pdma_map_sg_attrs(struct device *dev, struct scatterlist *sg, { struct pci_p2pdma_pagemap *p2p_pgmap = to_p2p_pgmap(sg_page(sg)->pgmap); - struct pci_dev *client; - - if (WARN_ON_ONCE(!dev_is_pci(dev))) - return 0; - client = to_pci_dev(dev); - - switch (pci_p2pdma_map_type(p2p_pgmap->provider, client)) { + switch (pci_p2pdma_map_type(sg_page(sg)->pgmap, dev)) { case PCI_P2PDMA_MAP_THRU_HOST_BRIDGE: return dma_map_sg_attrs(dev, sg, nents, dir, attrs); case PCI_P2PDMA_MAP_BUS_ADDR: @@ -887,17 +889,9 @@ EXPORT_SYMBOL_GPL(pci_p2pdma_map_sg_attrs); void pci_p2pdma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, int nents, enum dma_data_direction dir, unsigned long attrs) { - struct pci_p2pdma_pagemap *p2p_pgmap = - to_p2p_pgmap(sg_page(sg)->pgmap); enum pci_p2pdma_map_type map_type; - struct pci_dev *client; - - if (WARN_ON_ONCE(!dev_is_pci(dev))) - return; - - client = to_pci_dev(dev); - map_type = pci_p2pdma_map_type(p2p_pgmap->provider, client); + map_type = pci_p2pdma_map_type(sg_page(sg)->pgmap, dev); if (map_type == PCI_P2PDMA_MAP_THRU_HOST_BRIDGE) dma_unmap_sg_attrs(dev, sg, nents, dir, attrs); From patchwork Thu Jun 10 16:06:09 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Logan Gunthorpe X-Patchwork-Id: 12313445 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84327C48BE6 for ; Thu, 10 Jun 2021 16:06:26 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 68BA860FEA for ; Thu, 10 Jun 2021 16:06:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230406AbhFJQIT (ORCPT ); Thu, 10 Jun 2021 12:08:19 -0400 Received: from ale.deltatee.com ([204.191.154.188]:60178 "EHLO ale.deltatee.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230364AbhFJQIR (ORCPT ); Thu, 10 Jun 2021 12:08:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=deltatee.com; s=20200525; h=Subject:MIME-Version:References:In-Reply-To: Message-Id:Date:Cc:To:From:content-disposition; bh=V8wti9Rbd9gM9W3eLoa9VESkzHe3fDUB5qnOwryji5s=; b=EAuVPPWYpAgnjMxfy/hj2I32Vf +JJFwEARwbnRG5My05BB39xig5RZb2TkrRO/y6baz7GKBIBum7fQfTLr2U0FfrsCfGplHwlGLCJNu cUwO6t6TatP3R079/lBIDjf+R4/jlOCfw22/+SNgsAj8wp7/gQcYwcWb6lMZHZrcNtriQ0+b9//cL S8yWhNy9ben89b71mxXeWBNxWLui0q/5CGQPSzNj8GTAcIavnE0PGNiUKLqnvx0mScCGQ7TefpsHy pi2h0NaP3HfO1b6dWH0gNYdVLrD84Z392tJVZGHmqK/Mysyq9GnGH5gBjIHSagY8Za+JD3uwJcgnq 5K0o9iyQ==; Received: from cgy1-donard.priv.deltatee.com ([172.16.1.31]) by ale.deltatee.com with esmtps (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1lrNCD-0000Jj-G9; Thu, 10 Jun 2021 10:06:21 -0600 Received: from gunthorp by cgy1-donard.priv.deltatee.com with local (Exim 4.92) (envelope-from ) id 1lrNC9-0007Pv-20; Thu, 10 Jun 2021 10:06:13 -0600 From: Logan Gunthorpe To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Bjorn Helgaas Cc: Stephen Bates , Christoph Hellwig , Dan Williams , Jason Gunthorpe , =?utf-8?q?Christian_K=C3=B6nig?= , John Hubbard , Don Dutile , Logan Gunthorpe Date: Thu, 10 Jun 2021 10:06:09 -0600 Message-Id: <20210610160609.28447-7-logang@deltatee.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20210610160609.28447-1-logang@deltatee.com> References: <20210610160609.28447-1-logang@deltatee.com> MIME-Version: 1.0 X-SA-Exim-Connect-IP: 172.16.1.31 X-SA-Exim-Rcpt-To: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, helgaas@kernel.org, sbates@raithlin.com, hch@lst.de, dan.j.williams@intel.com, jgg@ziepe.ca, christian.koenig@amd.com, jhubbard@nvidia.com, ddutile@redhat.com, logang@deltatee.com X-SA-Exim-Mail-From: gunthorp@deltatee.com Subject: [PATCH v1 6/6] PCI/P2PDMA: Avoid pci_get_slot() which sleeps X-SA-Exim-Version: 4.2.1 (built Wed, 08 May 2019 21:11:16 +0000) X-SA-Exim-Scanned: Yes (on ale.deltatee.com) Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org In order to use upstream_bridge_distance_warn() from a dma_map function, it must not sleep. However, pci_get_slot() takes the pci_bus_sem so it might sleep. In order to avoid this, try to get the host bridge's device from the first element in the device list. It should be impossible for the host bridge's device to go away while references are held on child devices, so the first element should not be able to change and, thus, this should be safe. Introduce a static function called pci_host_bridge_dev() to obtain the host bridge's root device. Signed-off-by: Logan Gunthorpe --- drivers/pci/p2pdma.c | 34 ++++++++++++++++++++++++++++++++-- 1 file changed, 32 insertions(+), 2 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 5dc1f9f62a93..c7ecd0196102 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -308,10 +308,41 @@ static const struct pci_p2pdma_whitelist_entry { {} }; +/* + * This lookup function tries to find the PCI device corresponding to a given + * host bridge. + * + * It assumes the host bridge device is the first PCI device in the + * bus->devices list and that the devfn is 00.0. These assumptions should hold + * for all the devices in the whitelist above. + * + * This function is equivalent to pci_get_slot(host->bus, 0), however it does + * not take the pci_bus_sem lock seeing __host_bridge_whitelist() must not + * sleep. + * + * For this to be safe, the caller should hold a reference to a device on the + * bridge, which should ensure the host_bridge device will not be freed + * or removed from the head of the devices list. + */ +static struct pci_dev *pci_host_bridge_dev(struct pci_host_bridge *host) +{ + struct pci_dev *root; + + root = list_first_entry_or_null(&host->bus->devices, + struct pci_dev, bus_list); + + if (!root) + return NULL; + if (root->devfn != PCI_DEVFN(0, 0)) + return NULL; + + return root; +} + static bool __host_bridge_whitelist(struct pci_host_bridge *host, bool same_host_bridge, bool warn) { - struct pci_dev *root = pci_get_slot(host->bus, PCI_DEVFN(0, 0)); + struct pci_dev *root = pci_host_bridge_dev(host); const struct pci_p2pdma_whitelist_entry *entry; unsigned short vendor, device; @@ -320,7 +351,6 @@ static bool __host_bridge_whitelist(struct pci_host_bridge *host, vendor = root->vendor; device = root->device; - pci_dev_put(root); for (entry = pci_p2pdma_whitelist; entry->vendor; entry++) { if (vendor != entry->vendor || device != entry->device)