From patchwork Sun Mar 25 10:59:55 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 10306587 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 91B3F600CC for ; Sun, 25 Mar 2018 11:00:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8373829308 for ; Sun, 25 Mar 2018 11:00:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 771E529332; Sun, 25 Mar 2018 11:00:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.1 required=2.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED, DKIM_SIGNED, FREEMAIL_FROM, RCVD_IN_DNSWL_MED, T_DKIM_INVALID autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DB50B2855C for ; Sun, 25 Mar 2018 11:00:22 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 288956E427; Sun, 25 Mar 2018 11:00:18 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from mail-wm0-x241.google.com (mail-wm0-x241.google.com [IPv6:2a00:1450:400c:c09::241]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7247F6E412; Sun, 25 Mar 2018 11:00:05 +0000 (UTC) Received: by mail-wm0-x241.google.com with SMTP id i75so10498199wmf.0; Sun, 25 Mar 2018 04:00:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:subject:date:message-id:in-reply-to:references:mime-version :content-transfer-encoding; bh=y3YXG7RnmvrF+I7znPo7q9CvKyWJNMGhdeWwkSJ7fxU=; b=HPeWiuzzNSAOxbCcSHc5cnARnDzZqLe2IdJFpE60IZ3Q1e4bVY8aVOVrVQne/Ev1eT ZPTsoYE756mMyCysEMfWtG/DxcyYiu6l/plxC64huCI1ESsU3dICsfFEcox7fKT8F0M9 XAwBp/PHBMzaGhErgFL5rju8HyWI6oww0K8yElfkLmy1hgoCanD99cQ6NLf3CrH7VMj/ MpPgdGchglx4V33SjWpSW5Fo+TVyvdZTKQFzF/KMkfsb1eOOmS0SCcpdh0levaWjU/fR iK9aIvzt3P4BAMF0VVSGVkhmNI3JOLjfyoxQcWypysLI2M5U0KSL2LLntY2UcWfVPilQ AFbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=y3YXG7RnmvrF+I7znPo7q9CvKyWJNMGhdeWwkSJ7fxU=; b=TnWmY8TGfzFQKeiWW+e5mmxbE/hCFN/m++qbO5GVcYO+/nOwj2maSAqOCEWANvoeC5 Y3uwn8X/Jf5mOGb1o+MtJy1F1W8y5MC6IcTaczDCQVIFRdxDsAJyMlkG4Cljy+qKtqTI O/pdVDpfJ9P5RGKrrYBXONjnmQLjvE0zDb+cmsrI8VMvI+F+BXZB7e//2xLsHL/JA8KD 7poWuplrS+2VsSmDlRKgE+7/ZgCTUQtJD+1N9c7YSLSSEtCMbyN2UmqlJu+w/GaZqTaU Z6PnAkykuo8cI5TNeEBa8iEoRAn0pX2QVifNAQjlMMdlcKF4j6gieUI9k4HQdIvTFjAe dEhg== X-Gm-Message-State: AElRT7EEPECjJnGQQJNugKXO5LhY0A+dOSMlQQCgCyXoeg3Y8jVAgM7c 0cbiK8h/+hiifhleoqzLgAI= X-Google-Smtp-Source: AG47ELtTajhgf4vVO+MNX1D19whZMQ+jh4WxoGcR+2Pq9cFB8V50wfs+L+ZvzO0SMj/5SuyobjFCMQ== X-Received: by 10.28.183.68 with SMTP id h65mr11067659wmf.35.1521975604007; Sun, 25 Mar 2018 04:00:04 -0700 (PDT) Received: from baker.fritz.box ([2a02:908:1257:4460:80f2:8bcd:4355:2edc]) by smtp.gmail.com with ESMTPSA id z3sm20492230wme.3.2018.03.25.04.00.03 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 25 Mar 2018 04:00:03 -0700 (PDT) From: "=?UTF-8?q?Christian=20K=C3=B6nig?=" X-Google-Original-From: =?UTF-8?q?Christian=20K=C3=B6nig?= To: linaro-mm-sig@lists.linaro.org, linux-media@vger.kernel.org, dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/8] PCI: Add pci_peer_traffic_supported() Date: Sun, 25 Mar 2018 12:59:55 +0200 Message-Id: <20180325110000.2238-3-christian.koenig@amd.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180325110000.2238-1-christian.koenig@amd.com> References: <20180325110000.2238-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP From: "wdavis@nvidia.com" Add checks for topology and ACS configuration to determine whether or not peer traffic should be supported between two PCI devices. Signed-off-by: Will Davis Signed-off-by: Christian König --- drivers/pci/pci.c | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/pci.h | 1 + 2 files changed, 113 insertions(+) diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c index f6a4dd10d9b0..efdca3c9dad1 100644 --- a/drivers/pci/pci.c +++ b/drivers/pci/pci.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -5285,6 +5286,117 @@ void pci_ignore_hotplug(struct pci_dev *dev) } EXPORT_SYMBOL_GPL(pci_ignore_hotplug); +static bool pci_peer_host_traffic_supported(struct pci_host_bridge *host) +{ + struct pci_dev *bridge = pci_get_slot(host->bus, PCI_DEVFN(0, 0)); + unsigned short vendor, device; + + if (!bridge) + return false; + + vendor = bridge->vendor; + device = bridge->device; + pci_dev_put(bridge); + + /* AMD ZEN host bridges can do peer to peer */ + if (vendor == PCI_VENDOR_ID_AMD && device == 0x1450) + return true; + + /* TODO: Extend that to a proper whitelist */ + return false; +} + +bool pci_peer_traffic_supported(struct pci_dev *dev, struct pci_dev *peer) +{ + struct pci_dev *rp_dev, *rp_peer, *common_upstream; + struct pci_host_bridge *peer_host_bridge; + struct pci_host_bridge *dev_host_bridge; + int pos; + u16 cap; + + /* This is tricky enough, focus on PCIe for now */ + if (!pci_is_pcie(dev) || !pci_is_pcie(peer)) + return false; + + /* + * Disallow the peer-to-peer traffic if the devices do not share a + * host bridge. The PCI specifications does not make any guarantees + * about P2P capabilities between devices under separate domains. + * + * PCI Local Bus Specification Revision 3.0, section 3.10: + * "Peer-to-peer transactions crossing multiple host bridges + * PCI host bridges may, but are not required to, support PCI + * peer-to-peer transactions that traverse multiple PCI host + * bridges." + */ + dev_host_bridge = pci_find_host_bridge(dev->bus); + peer_host_bridge = pci_find_host_bridge(peer->bus); + if (dev_host_bridge != peer_host_bridge) + return pci_peer_host_traffic_supported(dev_host_bridge); + + rp_dev = pcie_find_root_port(dev); + rp_peer = pcie_find_root_port(peer); + common_upstream = pci_find_common_upstream_dev(dev, peer); + + /* + * Access Control Services (ACS) Checks + * + * ACS has a capability bit for P2P Request Redirects (RR), + * but unfortunately it doesn't tell us much about the real + * capabilities of the hardware. + * + * PCI Express Base Specification Revision 3.0, section + * 6.12.1.1: + * "ACS P2P Request Redirect: must be implemented by Root + * Ports that support peer-to-peer traffic with other + * Root Ports; [80]" + * but + * "[80] Root Port indication of ACS P2P Request Redirect + * or ACS P2P Completion Redirect support does not imply + * any particular level of peer-to-peer support by the + * Root Complex, or that peer-to-peer traffic is + * supported at all" + */ + + /* + * If ACS is not implemented, we have no idea about P2P + * support. Optimistically allow this if there is a common + * upstream device. + */ + pos = pci_find_ext_capability(rp_dev, PCI_EXT_CAP_ID_ACS); + if (!pos) + return common_upstream != NULL; + + /* + * If the devices are under the same root port and have a common + * upstream device, allow if the root port is further upstream + * from the common upstream device and the common upstream + * device has Upstream Forwarding disabled, or if the root port + * is the common upstream device and ACS is not implemented. + */ + pci_read_config_word(rp_dev, pos + PCI_ACS_CAP, &cap); + if ((rp_dev == rp_peer && common_upstream) && + (((common_upstream != rp_dev) && + !pci_acs_enabled(common_upstream, PCI_ACS_UF)) || + ((common_upstream == rp_dev) && ((cap & PCI_ACS_RR) == 0)))) + return true; + + /* + * If ACS RR is implemented and disabled, allow only if the + * devices are under the same root port. + */ + if (cap & PCI_ACS_RR && !pci_acs_enabled(rp_dev, PCI_ACS_RR)) + return rp_dev == rp_peer; + + /* + * If ACS RR is not implemented, or is implemented and enabled, + * only allow if there's a translation agent enabled to do the + * redirect. + */ + return iommu_present(&pci_bus_type); +} +EXPORT_SYMBOL(pci_peer_traffic_supported); + resource_size_t __weak pcibios_default_alignment(void) { return 0; diff --git a/include/linux/pci.h b/include/linux/pci.h index 0d29f5cdcb04..3c8eaa505991 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -921,6 +921,7 @@ void pci_remove_root_bus(struct pci_bus *bus); void pci_setup_cardbus(struct pci_bus *bus); void pcibios_setup_bridge(struct pci_bus *bus, unsigned long type); void pci_sort_breadthfirst(void); +bool pci_peer_traffic_supported(struct pci_dev *dev, struct pci_dev *peer); #define dev_is_pci(d) ((d)->bus == &pci_bus_type) #define dev_is_pf(d) ((dev_is_pci(d) ? to_pci_dev(d)->is_physfn : false))