From patchwork Thu Aug 30 04:09:13 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581211 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 45DCC920 for ; Thu, 30 Aug 2018 04:12:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 34BE12B8E8 for ; Thu, 30 Aug 2018 04:12:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 28CE82B8EF; Thu, 30 Aug 2018 04:12:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9C8C92B8E8 for ; Thu, 30 Aug 2018 04:12:18 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727848AbeH3IML (ORCPT ); Thu, 30 Aug 2018 04:12:11 -0400 Received: from mga12.intel.com ([192.55.52.136]:20569 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727532AbeH3IML (ORCPT ); Thu, 30 Aug 2018 04:12:11 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:10:58 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959948" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:10:45 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [RFC PATCH v2 01/10] iommu: Add APIs for multiple domains per device Date: Thu, 30 Aug 2018 12:09:13 +0800 Message-Id: <20180830040922.30426-2-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Sharing a physical PCI device in a finer-granularity way is becoming a consensus in the industry. IOMMU vendors are also engaging efforts to support such sharing as well as possible. Among the efforts, the capability of support finer-granularity DMA isolation is a common requirement due to the security consideration. With finer-granularity DMA isolation, all DMA requests out of or to a subset of a physical PCI device can be protected by the IOMMU. As a result, there is a request in software to attach multiple domains to a physical PCI device. One example of such use model is the Intel Scalable IOV [1] [2]. The Intel vt-d 3.0 spec [3] introduces the scalable mode which enables PASID granularity DMA isolation. This adds the APIs to support multiple domains per device. In order to ease the discussions, we call it 'a domain in auxiliary mode' or simply 'auxiliary domain' when multiple domains are attached to a physical device. The APIs includes: * iommu_capable(IOMMU_CAP_AUX_DOMAIN) - Represents the ability of supporting multiple domains per device. * iommu_en(dis)able_aux_domain(struct device *dev) - Enable/disable the multiple domains capability for a device referenced by @dev. * iommu_auxiliary_id(struct iommu_domain *domain) - Return ID used for finer-granularity DMA translation. For the Intel Scalable IOV usage model, this will be a PASID. The device which supports Scalalbe IOV needs to writes this ID to the device register so that DMA requests could be tagged with a right PASID prefix. Many people involved in discussions of this design. They're Kevin Tian Liu Yi L Ashok Raj Sanjay Kumar Alex Williamson Jean-Philippe Brucker and some discussions can be found here [4]. [1] https://software.intel.com/en-us/download/intel-scalable-io-virtualization-technical-specification [2] https://schd.ws/hosted_files/lc32018/00/LC3-SIOV-final.pdf [3] https://software.intel.com/en-us/download/intel-virtualization-technology-for-directed-io-architecture-specification [4] https://lkml.org/lkml/2018/7/26/4 Cc: Ashok Raj Cc: Jacob Pan Cc: Kevin Tian Cc: Liu Yi L Suggested-by: Kevin Tian Signed-off-by: Lu Baolu --- drivers/iommu/iommu.c | 29 +++++++++++++++++++++++++++++ include/linux/iommu.h | 13 +++++++++++++ 2 files changed, 42 insertions(+) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 8c15c5980299..2c6faf417dd5 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2014,3 +2014,32 @@ int iommu_fwspec_add_ids(struct device *dev, u32 *ids, int num_ids) return 0; } EXPORT_SYMBOL_GPL(iommu_fwspec_add_ids); + +int iommu_enable_aux_domain(struct device *dev) +{ + const struct iommu_ops *ops = dev->bus->iommu_ops; + + if (ops && ops->enable_auxd) + return ops->enable_auxd(dev); + + return -EINVAL; +} +EXPORT_SYMBOL_GPL(iommu_enable_aux_domain); + +void iommu_disable_aux_domain(struct device *dev) +{ + const struct iommu_ops *ops = dev->bus->iommu_ops; + + if (ops && ops->disable_auxd) + ops->disable_auxd(dev); +} +EXPORT_SYMBOL_GPL(iommu_disable_aux_domain); + +int iommu_auxiliary_id(struct iommu_domain *domain) +{ + if (domain->ops->auxd_id) + return domain->ops->auxd_id(domain); + + return -EINVAL; +} +EXPORT_SYMBOL_GPL(iommu_auxiliary_id); diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 87994c265bf5..ffd20b315bee 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -101,6 +101,8 @@ enum iommu_cap { transactions */ IOMMU_CAP_INTR_REMAP, /* IOMMU supports interrupt isolation */ IOMMU_CAP_NOEXEC, /* IOMMU_NOEXEC flag */ + IOMMU_CAP_AUX_DOMAIN, /* IOMMU supports multiple domains per + device */ }; /* @@ -185,6 +187,9 @@ struct iommu_resv_region { * @domain_get_windows: Return the number of windows for a domain * @of_xlate: add OF master IDs to iommu grouping * @pgsize_bitmap: bitmap of all possible supported page sizes + * @enable_auxd: enable multiple domains per device support + * @disable_auxd: disable multiple domains per device support + * @auxd_id: return the id of an auxiliary domain */ struct iommu_ops { bool (*capable)(enum iommu_cap); @@ -231,6 +236,10 @@ struct iommu_ops { int (*of_xlate)(struct device *dev, struct of_phandle_args *args); bool (*is_attach_deferred)(struct iommu_domain *domain, struct device *dev); + int (*enable_auxd)(struct device *dev); + void (*disable_auxd)(struct device *dev); + int (*auxd_id)(struct iommu_domain *domain); + unsigned long pgsize_bitmap; }; @@ -400,6 +409,10 @@ void iommu_fwspec_free(struct device *dev); int iommu_fwspec_add_ids(struct device *dev, u32 *ids, int num_ids); const struct iommu_ops *iommu_ops_from_fwnode(struct fwnode_handle *fwnode); +int iommu_enable_aux_domain(struct device *dev); +void iommu_disable_aux_domain(struct device *dev); +int iommu_auxiliary_id(struct iommu_domain *domain); + #else /* CONFIG_IOMMU_API */ struct iommu_ops {}; From patchwork Thu Aug 30 04:09:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581213 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4F7655A4 for ; Thu, 30 Aug 2018 04:12:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3E4912B8E8 for ; Thu, 30 Aug 2018 04:12:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 326842B8EF; Thu, 30 Aug 2018 04:12:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D391D2B8E8 for ; Thu, 30 Aug 2018 04:12:27 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727948AbeH3IMb (ORCPT ); Thu, 30 Aug 2018 04:12:31 -0400 Received: from mga12.intel.com ([192.55.52.136]:20569 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727626AbeH3IML (ORCPT ); Thu, 30 Aug 2018 04:12:11 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:10:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959950" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:10:48 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [RFC PATCH v2 02/10] iommu/vt-d: Add multiple domains per device query Date: Thu, 30 Aug 2018 12:09:14 +0800 Message-Id: <20180830040922.30426-3-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add the response to IOMMU_CAP_AUX_DOMAIN capability query through iommu_capable(). Return true if IOMMUs support the scalable mode, return false otherwise. Cc: Ashok Raj Cc: Jacob Pan Cc: Kevin Tian Cc: Liu Yi L Signed-off-by: Lu Baolu --- drivers/iommu/intel-iommu.c | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 3e49d4029058..891ae70e7bf2 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -5193,12 +5193,39 @@ static phys_addr_t intel_iommu_iova_to_phys(struct iommu_domain *domain, return phys; } +static inline bool scalable_mode_support(void) +{ + struct dmar_drhd_unit *drhd; + struct intel_iommu *iommu; + bool ret = true; + + rcu_read_lock(); + for_each_active_iommu(iommu, drhd) { + if (!sm_supported(iommu)) { + ret = false; + break; + } + } + rcu_read_unlock(); + + return ret; +} + static bool intel_iommu_capable(enum iommu_cap cap) { - if (cap == IOMMU_CAP_CACHE_COHERENCY) + switch (cap) { + case IOMMU_CAP_CACHE_COHERENCY: return domain_update_iommu_snooping(NULL) == 1; - if (cap == IOMMU_CAP_INTR_REMAP) + case IOMMU_CAP_INTR_REMAP: return irq_remapping_enabled == 1; + case IOMMU_CAP_AUX_DOMAIN: + return scalable_mode_support(); + case IOMMU_CAP_NOEXEC: + /* PASSTHROUGH */ + default: + pr_info("Unsupported capability query %d\n", cap); + break; + } return false; } From patchwork Thu Aug 30 04:09:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581207 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 23A835A4 for ; Thu, 30 Aug 2018 04:12:09 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 125CC2B8E8 for ; Thu, 30 Aug 2018 04:12:09 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 04AD52B8EF; Thu, 30 Aug 2018 04:12:09 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B507D2B8E8 for ; Thu, 30 Aug 2018 04:12:08 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727862AbeH3IML (ORCPT ); Thu, 30 Aug 2018 04:12:11 -0400 Received: from mga12.intel.com ([192.55.52.136]:20569 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727852AbeH3IML (ORCPT ); Thu, 30 Aug 2018 04:12:11 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:10:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959952" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:10:51 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu Subject: [RFC PATCH v2 03/10] iommu/amd: Add default branch in amd_iommu_capable() Date: Thu, 30 Aug 2018 12:09:15 +0800 Message-Id: <20180830040922.30426-4-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Otherwise, there will be a build warning: drivers/iommu/amd_iommu.c:3083:2: warning: enumeration value 'IOMMU_CAP_AUX_DOMAIN' not handled in switch [-Wswitch] There is no functional change. Signed-off-by: Lu Baolu --- drivers/iommu/amd_iommu.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/drivers/iommu/amd_iommu.c b/drivers/iommu/amd_iommu.c index 4e04fff23977..237ae6db4cfd 100644 --- a/drivers/iommu/amd_iommu.c +++ b/drivers/iommu/amd_iommu.c @@ -3077,6 +3077,8 @@ static bool amd_iommu_capable(enum iommu_cap cap) return (irq_remapping_enabled == 1); case IOMMU_CAP_NOEXEC: return false; + default: + break; } return false; From patchwork Thu Aug 30 04:09:16 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581209 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A812A920 for ; Thu, 30 Aug 2018 04:12:14 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 98CCA2B8E8 for ; Thu, 30 Aug 2018 04:12:14 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C7332B8EF; Thu, 30 Aug 2018 04:12:14 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2C8F22B8E8 for ; Thu, 30 Aug 2018 04:12:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727881AbeH3IMR (ORCPT ); Thu, 30 Aug 2018 04:12:17 -0400 Received: from mga12.intel.com ([192.55.52.136]:20666 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727627AbeH3IML (ORCPT ); Thu, 30 Aug 2018 04:12:11 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga106.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:10:59 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959954" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:10:54 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [RFC PATCH v2 04/10] iommu/vt-d: Enable/disable multiple domains per device Date: Thu, 30 Aug 2018 12:09:16 +0800 Message-Id: <20180830040922.30426-5-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add iommu ops for enabling and disabling multiple domains for a device. Cc: Ashok Raj Cc: Jacob Pan Cc: Kevin Tian Cc: Liu Yi L Signed-off-by: Lu Baolu --- drivers/iommu/intel-iommu.c | 36 ++++++++++++++++++++++++++++++++++++ include/linux/intel-iommu.h | 1 + 2 files changed, 37 insertions(+) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 891ae70e7bf2..3606d25bc40c 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -2501,6 +2501,7 @@ static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu, info->domain = domain; info->iommu = iommu; info->pasid_table = NULL; + info->auxd_enabled = 0; if (dev && dev_is_pci(dev)) { struct pci_dev *pdev = to_pci_dev(info->dev); @@ -5384,6 +5385,39 @@ struct intel_iommu *intel_svm_device_to_iommu(struct device *dev) } #endif /* CONFIG_INTEL_IOMMU_SVM */ +static int intel_iommu_enable_auxd(struct device *dev) +{ + struct device_domain_info *info; + struct dmar_domain *domain; + unsigned long flags; + + if (!scalable_mode_support()) + return -ENODEV; + + domain = get_valid_domain_for_dev(dev); + if (!domain) + return -ENODEV; + + spin_lock_irqsave(&device_domain_lock, flags); + info = dev->archdata.iommu; + info->auxd_enabled = 1; + spin_unlock_irqrestore(&device_domain_lock, flags); + + return 0; +} + +static void intel_iommu_disable_auxd(struct device *dev) +{ + struct device_domain_info *info; + unsigned long flags; + + spin_lock_irqsave(&device_domain_lock, flags); + info = dev->archdata.iommu; + if (!WARN_ON(!info)) + info->auxd_enabled = 0; + spin_unlock_irqrestore(&device_domain_lock, flags); +} + const struct iommu_ops intel_iommu_ops = { .capable = intel_iommu_capable, .domain_alloc = intel_iommu_domain_alloc, @@ -5398,6 +5432,8 @@ const struct iommu_ops intel_iommu_ops = { .get_resv_regions = intel_iommu_get_resv_regions, .put_resv_regions = intel_iommu_put_resv_regions, .device_group = pci_device_group, + .enable_auxd = intel_iommu_enable_auxd, + .disable_auxd = intel_iommu_disable_auxd, .pgsize_bitmap = INTEL_IOMMU_PGSIZES, }; diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index b34cf8b887a0..15981245796e 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -487,6 +487,7 @@ struct device_domain_info { u8 pri_enabled:1; u8 ats_supported:1; u8 ats_enabled:1; + u8 auxd_enabled:1; /* Multiple domains per device */ u8 ats_qdep; struct device *dev; /* it's NULL for PCIe-to-PCI bridge */ struct intel_iommu *iommu; /* IOMMU used by this device */ From patchwork Thu Aug 30 04:09:17 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581199 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B8D95920 for ; Thu, 30 Aug 2018 04:11:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A9AE12B8E5 for ; Thu, 30 Aug 2018 04:11:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9B9D72B8E8; Thu, 30 Aug 2018 04:11:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 029D02B8E5 for ; Thu, 30 Aug 2018 04:11:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727749AbeH3ILZ (ORCPT ); Thu, 30 Aug 2018 04:11:25 -0400 Received: from mga02.intel.com ([134.134.136.20]:31597 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727694AbeH3ILY (ORCPT ); Thu, 30 Aug 2018 04:11:24 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:11:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959965" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:10:57 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [RFC PATCH v2 05/10] iommu/vt-d: Attach/detach domains in auxiliary mode Date: Thu, 30 Aug 2018 12:09:17 +0800 Message-Id: <20180830040922.30426-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When multiple domains per device has been enabled by the device driver, the device will tag the default PASID for the domain to all DMA traffics out of the subset of this device; and the IOMMU should translate the DMA requests in PASID granularity. This extends the intel_iommu_attach/detach_device() ops to support managing PASID granular translation structures when the device driver has enabled multiple domains per device. Cc: Ashok Raj Cc: Jacob Pan Cc: Kevin Tian Cc: Liu Yi L Signed-off-by: Sanjay Kumar Signed-off-by: Lu Baolu --- drivers/iommu/intel-iommu.c | 132 +++++++++++++++++++++++++++++++++++- include/linux/intel-iommu.h | 10 +++ 2 files changed, 139 insertions(+), 3 deletions(-) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 3606d25bc40c..784bd496f316 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -2502,6 +2502,7 @@ static struct dmar_domain *dmar_insert_one_dev_info(struct intel_iommu *iommu, info->iommu = iommu; info->pasid_table = NULL; info->auxd_enabled = 0; + INIT_LIST_HEAD(&info->auxiliary_domains); if (dev && dev_is_pci(dev)) { struct pci_dev *pdev = to_pci_dev(info->dev); @@ -5036,6 +5037,124 @@ static void intel_iommu_domain_free(struct iommu_domain *domain) domain_exit(to_dmar_domain(domain)); } +/* + * Check whether a @domain will be attached to the @dev in the + * auxiliary mode. + */ +static inline bool +is_device_attach_aux_domain(struct device *dev, struct iommu_domain *domain) +{ + struct device_domain_info *info = dev->archdata.iommu; + + return info && info->auxd_enabled && + domain->type == IOMMU_DOMAIN_UNMANAGED; +} + +static void auxiliary_link_device(struct dmar_domain *domain, + struct device *dev) +{ + struct device_domain_info *info = dev->archdata.iommu; + + assert_spin_locked(&device_domain_lock); + if (WARN_ON(!info)) + return; + + domain->auxd_refcnt++; + list_add(&domain->auxd, &info->auxiliary_domains); +} + +static void auxiliary_unlink_device(struct dmar_domain *domain, + struct device *dev) +{ + struct device_domain_info *info = dev->archdata.iommu; + + assert_spin_locked(&device_domain_lock); + if (WARN_ON(!info)) + return; + + list_del(&domain->auxd); + domain->auxd_refcnt--; + + if (!domain->auxd_refcnt && domain->default_pasid > 0) + intel_pasid_free_id(domain->default_pasid); +} + +static int domain_add_dev_auxd(struct dmar_domain *domain, + struct device *dev) +{ + int ret; + u8 bus, devfn; + unsigned long flags; + struct intel_iommu *iommu; + + iommu = device_to_iommu(dev, &bus, &devfn); + if (!iommu) + return -ENODEV; + + spin_lock_irqsave(&device_domain_lock, flags); + if (domain->default_pasid <= 0) { + domain->default_pasid = intel_pasid_alloc_id(domain, PASID_MIN, + intel_pasid_get_dev_max_id(dev), GFP_ATOMIC); + if (domain->default_pasid < 0) { + pr_err("Can't allocate default pasid\n"); + ret = -ENODEV; + goto pasid_failed; + } + } + + spin_lock(&iommu->lock); + ret = domain_attach_iommu(domain, iommu); + if (ret) + goto attach_failed; + + /* Setup the PASID entry for mediated devices: */ + ret = intel_pasid_setup_second_level(iommu, domain, dev, + domain->default_pasid, false); + if (ret) + goto table_failed; + spin_unlock(&iommu->lock); + + auxiliary_link_device(domain, dev); + + spin_unlock_irqrestore(&device_domain_lock, flags); + + return 0; + +table_failed: + domain_detach_iommu(domain, iommu); +attach_failed: + spin_unlock(&iommu->lock); + if (!domain->auxd_refcnt && domain->default_pasid > 0) + intel_pasid_free_id(domain->default_pasid); +pasid_failed: + spin_unlock_irqrestore(&device_domain_lock, flags); + + return ret; +} + +static void domain_remove_dev_aux(struct dmar_domain *domain, + struct device *dev) +{ + struct device_domain_info *info; + struct intel_iommu *iommu; + unsigned long flags; + + spin_lock_irqsave(&device_domain_lock, flags); + info = dev->archdata.iommu; + iommu = info->iommu; + + intel_pasid_tear_down_second_level(iommu, domain, + dev, domain->default_pasid); + + auxiliary_unlink_device(domain, dev); + + spin_lock(&iommu->lock); + domain_detach_iommu(domain, iommu); + spin_unlock(&iommu->lock); + + spin_unlock_irqrestore(&device_domain_lock, flags); +} + static int intel_iommu_attach_device(struct iommu_domain *domain, struct device *dev) { @@ -5050,7 +5169,8 @@ static int intel_iommu_attach_device(struct iommu_domain *domain, } /* normally dev is not mapped */ - if (unlikely(domain_context_mapped(dev))) { + if (unlikely(domain_context_mapped(dev) && + !is_device_attach_aux_domain(dev, domain))) { struct dmar_domain *old_domain; old_domain = find_domain(dev); @@ -5097,13 +5217,19 @@ static int intel_iommu_attach_device(struct iommu_domain *domain, dmar_domain->agaw--; } - return domain_add_dev_info(dmar_domain, dev); + if (is_device_attach_aux_domain(dev, domain)) + return domain_add_dev_auxd(dmar_domain, dev); + else + return domain_add_dev_info(dmar_domain, dev); } static void intel_iommu_detach_device(struct iommu_domain *domain, struct device *dev) { - dmar_remove_one_dev_info(to_dmar_domain(domain), dev); + if (is_device_attach_aux_domain(dev, domain)) + domain_remove_dev_aux(to_dmar_domain(domain), dev); + else + dmar_remove_one_dev_info(to_dmar_domain(domain), dev); } static int intel_iommu_map(struct iommu_domain *domain, diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h index 15981245796e..cbf3d250b521 100644 --- a/include/linux/intel-iommu.h +++ b/include/linux/intel-iommu.h @@ -408,9 +408,11 @@ struct dmar_domain { /* Domain ids per IOMMU. Use u16 since * domain ids are 16 bit wide according * to VT-d spec, section 9.3 */ + unsigned int auxd_refcnt; /* Refcount of auxiliary attaching */ bool has_iotlb_device; struct list_head devices; /* all devices' list */ + struct list_head auxd; /* link to device's auxiliary list */ struct iova_domain iovad; /* iova's that belong to this domain */ struct dma_pte *pgd; /* virtual address */ @@ -429,6 +431,11 @@ struct dmar_domain { 2 == 1GiB, 3 == 512GiB, 4 == 1TiB */ u64 max_addr; /* maximum mapped address */ + int default_pasid; /* + * The default pasid used for non-SVM + * traffic on mediated devices. + */ + struct iommu_domain domain; /* generic domain data structure for iommu core */ }; @@ -478,6 +485,9 @@ struct device_domain_info { struct list_head link; /* link to domain siblings */ struct list_head global; /* link to global list */ struct list_head table; /* link to pasid table */ + struct list_head auxiliary_domains; /* auxiliary domains + * attached to this device + */ u8 bus; /* PCI bus number */ u8 devfn; /* PCI devfn number */ u16 pfsid; /* SRIOV physical function source ID */ From patchwork Thu Aug 30 04:09:18 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581205 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 180985A4 for ; Thu, 30 Aug 2018 04:12:00 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 084D12B8E8 for ; Thu, 30 Aug 2018 04:12:00 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id F0A882B8EF; Thu, 30 Aug 2018 04:11:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 991782B8E8 for ; Thu, 30 Aug 2018 04:11:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727724AbeH3ILY (ORCPT ); Thu, 30 Aug 2018 04:11:24 -0400 Received: from mga02.intel.com ([134.134.136.20]:31597 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727698AbeH3ILY (ORCPT ); Thu, 30 Aug 2018 04:11:24 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:11:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959972" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:11:01 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [RFC PATCH v2 06/10] iommu/vt-d: Return ID associated with an auxiliary domain Date: Thu, 30 Aug 2018 12:09:18 +0800 Message-Id: <20180830040922.30426-7-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This adds support to return the default pasid associated with an auxiliary domain. The PCI device which is bound with this domain should use this value as the pasid for all DMA requests of the subset of device which is isolated and protected with this domain. Cc: Ashok Raj Cc: Jacob Pan Cc: Kevin Tian Cc: Liu Yi L Signed-off-by: Lu Baolu --- drivers/iommu/intel-iommu.c | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c index 784bd496f316..1ba39f120a38 100644 --- a/drivers/iommu/intel-iommu.c +++ b/drivers/iommu/intel-iommu.c @@ -5544,6 +5544,14 @@ static void intel_iommu_disable_auxd(struct device *dev) spin_unlock_irqrestore(&device_domain_lock, flags); } +static int intel_iommu_auxd_id(struct iommu_domain *domain) +{ + struct dmar_domain *dmar_domain = to_dmar_domain(domain); + + return dmar_domain->default_pasid > 0 ? + dmar_domain->default_pasid : -EINVAL; +} + const struct iommu_ops intel_iommu_ops = { .capable = intel_iommu_capable, .domain_alloc = intel_iommu_domain_alloc, @@ -5560,6 +5568,7 @@ const struct iommu_ops intel_iommu_ops = { .device_group = pci_device_group, .enable_auxd = intel_iommu_enable_auxd, .disable_auxd = intel_iommu_disable_auxd, + .auxd_id = intel_iommu_auxd_id, .pgsize_bitmap = INTEL_IOMMU_PGSIZES, }; From patchwork Thu Aug 30 04:09:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581203 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2E1685A4 for ; Thu, 30 Aug 2018 04:11:44 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1EC0B2B8E5 for ; Thu, 30 Aug 2018 04:11:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 12C9D2B8E8; Thu, 30 Aug 2018 04:11:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 910F22B8E5 for ; Thu, 30 Aug 2018 04:11:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727571AbeH3ILj (ORCPT ); Thu, 30 Aug 2018 04:11:39 -0400 Received: from mga02.intel.com ([134.134.136.20]:31597 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727743AbeH3ILZ (ORCPT ); Thu, 30 Aug 2018 04:11:25 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:11:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959977" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:11:04 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [RFC PATCH v2 07/10] vfio/mdev: Add mediated device domain type Date: Thu, 30 Aug 2018 12:09:19 +0800 Message-Id: <20180830040922.30426-8-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A parent device might create different types of mediated devices. For example, a mediated device could be created by the parent device with full isolation and protection provided by the IOMMU. One usage case could be found on Intel platforms where a mediated device is an assignable subset of a PCI, the DMA requests on behalf of it are all tagged with a PASID. Since IOMMU supports PASID-granular translations (scalable mode in vt-d 3.0), this mediated device could be individually protected and isolated by the IOMMU. This patch defines the domain types of a mediated device and allows the parent driver to specify this attributes when a mediated device is being careated. The following types are defined: * DOMAIN_TYPE_NO_IOMMU - Do not need any IOMMU support. All isolation and protection are handled by the parent device driver through the callbacks with device specific mechanism. * DOMAIN_TYPE_ATTACH_PARENT - IOMMU can isolate and protect this mediated device, and an isolation domain should be attaced to the the parent device. This also reseves a place in mdev private data structure to save the iommu domain, and adds interfaces to store and retrieve the domain. Below APIs are introduced: * mdev_set/get_domain_type(type) - Set or query the domain type of a mediated device. The parent device driver should set the domain type (or keep DOMAIN_TYPE_NO_IOMMU by default) during the mediated device creation. * mdev_set/get_domain(domain) - A iommu domain which has been attached to the parent device in order to protect and isolate the mediated device will be kept in the mdev data structure and could be retrieved later. Cc: Ashok Raj Cc: Jacob Pan Cc: Kevin Tian Cc: Liu Yi L Suggested-by: Kevin Tian Signed-off-by: Lu Baolu --- drivers/vfio/mdev/mdev_core.c | 36 ++++++++++++++++++++++++++++++++ drivers/vfio/mdev/mdev_private.h | 2 ++ include/linux/mdev.h | 26 +++++++++++++++++++++++ 3 files changed, 64 insertions(+) diff --git a/drivers/vfio/mdev/mdev_core.c b/drivers/vfio/mdev/mdev_core.c index 0212f0ee8aea..d45a829c5b11 100644 --- a/drivers/vfio/mdev/mdev_core.c +++ b/drivers/vfio/mdev/mdev_core.c @@ -390,6 +390,42 @@ int mdev_device_remove(struct device *dev, bool force_remove) return 0; } +int mdev_set_domain_type(struct device *dev, enum mdev_domain_type type) +{ + struct mdev_device *mdev = to_mdev_device(dev); + + mdev->domain_type = type; + + return 0; +} +EXPORT_SYMBOL(mdev_set_domain_type); + +enum mdev_domain_type mdev_get_domain_type(struct device *dev) +{ + struct mdev_device *mdev = to_mdev_device(dev); + + return mdev->domain_type; +} +EXPORT_SYMBOL(mdev_get_domain_type); + +int mdev_set_domain(struct device *dev, void *domain) +{ + struct mdev_device *mdev = to_mdev_device(dev); + + mdev->domain = domain; + + return 0; +} +EXPORT_SYMBOL(mdev_set_domain); + +void *mdev_get_domain(struct device *dev) +{ + struct mdev_device *mdev = to_mdev_device(dev); + + return mdev->domain; +} +EXPORT_SYMBOL(mdev_get_domain); + static int __init mdev_init(void) { return mdev_bus_register(); diff --git a/drivers/vfio/mdev/mdev_private.h b/drivers/vfio/mdev/mdev_private.h index b5819b7d7ef7..fd9e33fbd6e5 100644 --- a/drivers/vfio/mdev/mdev_private.h +++ b/drivers/vfio/mdev/mdev_private.h @@ -34,6 +34,8 @@ struct mdev_device { struct list_head next; struct kobject *type_kobj; bool active; + int domain_type; + void *domain; }; #define to_mdev_device(dev) container_of(dev, struct mdev_device, dev) diff --git a/include/linux/mdev.h b/include/linux/mdev.h index b6e048e1045f..3224587bda1e 100644 --- a/include/linux/mdev.h +++ b/include/linux/mdev.h @@ -15,6 +15,32 @@ struct mdev_device; +enum mdev_domain_type { + DOMAIN_TYPE_NO_IOMMU, /* Don't need any IOMMU support. + * All isolation and protection + * are handled by the parent + * device driver with a device + * specific mechanism. + */ + DOMAIN_TYPE_ATTACH_PARENT, /* IOMMU can isolate and protect + * the mdev, and the isolation + * domain should be attaced with + * the parent device. + */ +}; + +/* + * Called by the parent device driver to set the domain type. + * By default, the domain type is set to DOMAIN_TYPE_EXTERNAL. + */ +int mdev_set_domain_type(struct device *dev, enum mdev_domain_type type); + +/* Check the domain type. */ +enum mdev_domain_type mdev_get_domain_type(struct device *dev); + +int mdev_set_domain(struct device *dev, void *domain); +void *mdev_get_domain(struct device *dev); + /** * struct mdev_parent_ops - Structure to be registered for each parent device to * register the device to mdev module. From patchwork Thu Aug 30 04:09:20 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581201 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A27375A4 for ; Thu, 30 Aug 2018 04:11:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8E1602B8E5 for ; Thu, 30 Aug 2018 04:11:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8001A2B8E8; Thu, 30 Aug 2018 04:11:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EF7ED2B8E5 for ; Thu, 30 Aug 2018 04:11:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727765AbeH3ILj (ORCPT ); Thu, 30 Aug 2018 04:11:39 -0400 Received: from mga02.intel.com ([134.134.136.20]:31598 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727720AbeH3ILY (ORCPT ); Thu, 30 Aug 2018 04:11:24 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:11:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959983" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:11:07 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [RFC PATCH v2 08/10] vfio/type1: Add domain at(de)taching group helpers Date: Thu, 30 Aug 2018 12:09:20 +0800 Message-Id: <20180830040922.30426-9-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP If a domain is attaching to a group which includes the mediated devices, it should attach to the mdev parent of each mdev. This adds a helper for attaching domain to group, no matter a PCI physical device or mediated devices which are derived from a PCI physical device. Cc: Ashok Raj Cc: Jacob Pan Cc: Kevin Tian Cc: Liu Yi L Signed-off-by: Lu Baolu --- drivers/vfio/vfio_iommu_type1.c | 77 ++++++++++++++++++++++++++++++--- 1 file changed, 70 insertions(+), 7 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index d9fd3188615d..89e2e6123223 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -91,6 +91,9 @@ struct vfio_dma { struct vfio_group { struct iommu_group *iommu_group; struct list_head next; + bool attach_parent; /* An mdev group with domain + * attached to parent + */ }; /* @@ -1327,6 +1330,66 @@ static bool vfio_iommu_has_sw_msi(struct iommu_group *group, phys_addr_t *base) return ret; } +static int vfio_mdev_set_aux_domain(struct device *dev, + struct iommu_domain *domain) +{ + int (*fn)(struct device *dev, void *domain); + int ret; + + fn = symbol_get(mdev_set_domain); + if (fn) { + ret = fn(dev, domain); + symbol_put(mdev_set_domain); + + return ret; + } + + return -EINVAL; +} + +static int vfio_attach_aux_domain(struct device *dev, void *data) +{ + struct iommu_domain *domain = data; + int ret; + + ret = vfio_mdev_set_aux_domain(dev, domain); + if (ret) + return ret; + + return iommu_attach_device(domain, dev->parent); +} + +static int vfio_detach_aux_domain(struct device *dev, void *data) +{ + struct iommu_domain *domain = data; + + vfio_mdev_set_aux_domain(dev, NULL); + iommu_detach_device(domain, dev->parent); + + return 0; +} + +static int vfio_iommu_attach_group(struct vfio_domain *domain, + struct vfio_group *group) +{ + if (group->attach_parent) + return iommu_group_for_each_dev(group->iommu_group, + domain->domain, + vfio_attach_aux_domain); + else + return iommu_attach_group(domain->domain, group->iommu_group); +} + +static void vfio_iommu_detach_group(struct vfio_domain *domain, + struct vfio_group *group) +{ + if (group->attach_parent) + iommu_group_for_each_dev(group->iommu_group, domain->domain, + vfio_detach_aux_domain); + else + iommu_detach_group(domain->domain, group->iommu_group); +} + static int vfio_iommu_type1_attach_group(void *iommu_data, struct iommu_group *iommu_group) { @@ -1402,7 +1465,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, goto out_domain; } - ret = iommu_attach_group(domain->domain, iommu_group); + ret = vfio_iommu_attach_group(domain, group); if (ret) goto out_domain; @@ -1434,8 +1497,8 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, list_for_each_entry(d, &iommu->domain_list, next) { if (d->domain->ops == domain->domain->ops && d->prot == domain->prot) { - iommu_detach_group(domain->domain, iommu_group); - if (!iommu_attach_group(d->domain, iommu_group)) { + vfio_iommu_detach_group(domain, group); + if (!vfio_iommu_attach_group(d, group)) { list_add(&group->next, &d->group_list); iommu_domain_free(domain->domain); kfree(domain); @@ -1443,7 +1506,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, return 0; } - ret = iommu_attach_group(domain->domain, iommu_group); + ret = vfio_iommu_attach_group(domain, group); if (ret) goto out_domain; } @@ -1469,7 +1532,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, return 0; out_detach: - iommu_detach_group(domain->domain, iommu_group); + vfio_iommu_detach_group(domain, group); out_domain: iommu_domain_free(domain->domain); out_free: @@ -1560,7 +1623,7 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, if (!group) continue; - iommu_detach_group(domain->domain, iommu_group); + vfio_iommu_detach_group(domain, group); list_del(&group->next); kfree(group); /* @@ -1625,7 +1688,7 @@ static void vfio_release_domain(struct vfio_domain *domain, bool external) list_for_each_entry_safe(group, group_tmp, &domain->group_list, next) { if (!external) - iommu_detach_group(domain->domain, group->iommu_group); + vfio_iommu_detach_group(domain, group); list_del(&group->next); kfree(group); } From patchwork Thu Aug 30 04:09:21 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581215 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7DD75920 for ; Thu, 30 Aug 2018 04:12:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6C9CD2B8EF for ; Thu, 30 Aug 2018 04:12:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6045C2B8F2; Thu, 30 Aug 2018 04:12:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 06CB42B8EF for ; Thu, 30 Aug 2018 04:12:32 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727701AbeH3ILX (ORCPT ); Thu, 30 Aug 2018 04:11:23 -0400 Received: from mga02.intel.com ([134.134.136.20]:31597 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727694AbeH3ILX (ORCPT ); Thu, 30 Aug 2018 04:11:23 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:11:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959991" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:11:10 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [RFC PATCH v2 09/10] vfio/type1: Determine domain type of an mdev group Date: Thu, 30 Aug 2018 12:09:21 +0800 Message-Id: <20180830040922.30426-10-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This adds the support to determine the domain type of a group of mediated devices according to the domain type attribute of each device. Cc: Ashok Raj Cc: Jacob Pan Cc: Kevin Tian Cc: Liu Yi L Signed-off-by: Sanjay Kumar Signed-off-by: Lu Baolu --- drivers/vfio/vfio_iommu_type1.c | 47 ++++++++++++++++++++++++++++++--- 1 file changed, 43 insertions(+), 4 deletions(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 89e2e6123223..64bf55b91de1 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -1390,6 +1390,27 @@ static void vfio_iommu_detach_group(struct vfio_domain *domain, iommu_detach_group(domain->domain, group->iommu_group); } +static int vfio_mdev_domain_type(struct device *dev, void *data) +{ + enum mdev_domain_type new, *old = data; + enum mdev_domain_type (*fn)(struct device *dev); + + fn = symbol_get(mdev_get_domain_type); + if (fn) { + new = fn(dev); + symbol_put(mdev_get_domain_type); + + if (*old && *old != new) + return -EINVAL; + + *old = new; + + return 0; + } + + return -EINVAL; +} + static int vfio_iommu_type1_attach_group(void *iommu_data, struct iommu_group *iommu_group) { @@ -1433,9 +1454,19 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, mdev_bus = symbol_get(mdev_bus_type); - if (mdev_bus) { - if ((bus == mdev_bus) && !iommu_present(bus)) { - symbol_put(mdev_bus_type); + if (mdev_bus && bus == mdev_bus) { + enum mdev_domain_type type = 0; + + symbol_put(mdev_bus_type); + + /* Determine the domain type: */ + ret = iommu_group_for_each_dev(iommu_group, &type, + vfio_mdev_domain_type); + if (ret) + goto out_free; + + switch (type) { + case DOMAIN_TYPE_NO_IOMMU: if (!iommu->external_domain) { INIT_LIST_HEAD(&domain->group_list); iommu->external_domain = domain; @@ -1445,11 +1476,19 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, list_add(&group->next, &iommu->external_domain->group_list); mutex_unlock(&iommu->lock); + return 0; + case DOMAIN_TYPE_ATTACH_PARENT: + /* FALLTHROUGH */ + default: + ret = -EINVAL; + goto out_free; } - symbol_put(mdev_bus_type); } + if (mdev_bus) + symbol_put(mdev_bus_type); + domain->domain = iommu_domain_alloc(bus); if (!domain->domain) { ret = -EIO; From patchwork Thu Aug 30 04:09:22 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolu Lu X-Patchwork-Id: 10581197 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 31F365A4 for ; Thu, 30 Aug 2018 04:11:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 213A72B8E5 for ; Thu, 30 Aug 2018 04:11:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 13DA62B8E8; Thu, 30 Aug 2018 04:11:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id AEED92B8E5 for ; Thu, 30 Aug 2018 04:11:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727769AbeH3IL0 (ORCPT ); Thu, 30 Aug 2018 04:11:26 -0400 Received: from mga02.intel.com ([134.134.136.20]:31601 "EHLO mga02.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727765AbeH3IL0 (ORCPT ); Thu, 30 Aug 2018 04:11:26 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 29 Aug 2018 21:11:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.53,306,1531810800"; d="scan'208";a="68959996" Received: from allen-box.sh.intel.com ([10.239.161.122]) by orsmga007.jf.intel.com with ESMTP; 29 Aug 2018 21:11:13 -0700 From: Lu Baolu To: Joerg Roedel , David Woodhouse , Alex Williamson , Kirti Wankhede Cc: ashok.raj@intel.com, sanjay.k.kumar@intel.com, jacob.jun.pan@intel.com, kevin.tian@intel.com, Jean-Philippe Brucker , yi.l.liu@intel.com, yi.y.sun@intel.com, peterx@redhat.com, tiwei.bie@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jacob Pan Subject: [RFC PATCH v2 10/10] vfio/type1: Attach domain for mdev group Date: Thu, 30 Aug 2018 12:09:22 +0800 Message-Id: <20180830040922.30426-11-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180830040922.30426-1-baolu.lu@linux.intel.com> References: <20180830040922.30426-1-baolu.lu@linux.intel.com> Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When attaching domain to a group of mediated devices which all have the domain type attributes set to ATTACH_PARENT, we should attach domain to the parent PCI device instead of mdev device itself. Cc: Ashok Raj Cc: Jacob Pan Cc: Kevin Tian Cc: Liu Yi L Signed-off-by: Sanjay Kumar Signed-off-by: Lu Baolu --- drivers/vfio/vfio_iommu_type1.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 64bf55b91de1..c4231df44304 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -1411,6 +1411,18 @@ static int vfio_mdev_domain_type(struct device *dev, void *data) return -EINVAL; } +static int vfio_parent_bus_type(struct device *dev, void *data) +{ + struct bus_type **bus = data; + + if (*bus && *bus != dev->parent->bus) + return -EINVAL; + + *bus = dev->parent->bus; + + return 0; +} + static int vfio_iommu_type1_attach_group(void *iommu_data, struct iommu_group *iommu_group) { @@ -1458,6 +1470,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, enum mdev_domain_type type = 0; symbol_put(mdev_bus_type); + mdev_bus = NULL; /* Determine the domain type: */ ret = iommu_group_for_each_dev(iommu_group, &type, @@ -1479,7 +1492,14 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, return 0; case DOMAIN_TYPE_ATTACH_PARENT: - /* FALLTHROUGH */ + bus = NULL; + group->attach_parent = true; + /* Set @bus to bus type of the parent: */ + ret = iommu_group_for_each_dev(iommu_group, &bus, + vfio_parent_bus_type); + if (ret) + goto out_free; + break; default: ret = -EINVAL; goto out_free;