From patchwork Mon Jun 10 18:47:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10985295 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D0B914B6 for ; Mon, 10 Jun 2019 18:53:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 77E9828759 for ; Mon, 10 Jun 2019 18:53:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 691C32876E; Mon, 10 Jun 2019 18:53:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B129128759 for ; Mon, 10 Jun 2019 18:53:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8aAJtC2UfQRnMLSi0BNicSGjoOrdf4j2t/DEEp4xius=; b=ZAnDfDU9ytVYQp GRI5Rp7kIHxDyszNCi/TquNk7WeXzt8gympLbQ2ydsGXsLbZQ1N/Fu9eAC76HOzraUWkqbmPtBUD/ 40htfK0d5DdPpVEkXdgrqs/lweMtVin1HIEpSzhtDLHbleQMN3bNJVAxtPktdDH6LyStE2qzlasjj 30Rwie92rKq1jI1+BNaLECvIqTCXoFPz3lVasZTaWHPpXv3yXEnstByvE7WWn5NQpqZyeWWcpTrey ALfAV4gf1diOsvt4xp+RUoPXS4RRLxvK3mAyyKSTovj6t3zCkhzZOEELbJ8UWm/sZ2+EHjcb45K9m J1H5dwYJ2T15+M0gGUIg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPPe-0002vW-Bp; Mon, 10 Jun 2019 18:52:58 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPOc-00028e-R4 for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2019 18:51:58 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 6E83B346; Mon, 10 Jun 2019 11:51:50 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.196.129]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id F3F623F246; Mon, 10 Jun 2019 11:51:48 -0700 (PDT) From: Jean-Philippe Brucker To: will.deacon@arm.com Subject: [PATCH 1/8] iommu: Add I/O ASID allocator Date: Mon, 10 Jun 2019 19:47:07 +0100 Message-Id: <20190610184714.6786-2-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190610184714.6786-1-jean-philippe.brucker@arm.com> References: <20190610184714.6786-1-jean-philippe.brucker@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190610_115154_967054_B8C8C380 X-CRM114-Status: GOOD ( 24.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, jacob.jun.pan@linux.intel.com, joro@8bytes.org, linux-kernel@vger.kernel.org, eric.auger@redhat.com, iommu@lists.linux-foundation.org, robh+dt@kernel.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Some devices might support multiple DMA address spaces, in particular those that have the PCI PASID feature. PASID (Process Address Space ID) allows to share process address spaces with devices (SVA), partition a device into VM-assignable entities (VFIO mdev) or simply provide multiple DMA address space to kernel drivers. Add a global PASID allocator usable by different drivers at the same time. Name it I/O ASID to avoid confusion with ASIDs allocated by arch code, which are usually a separate ID space. The IOASID space is global. Each device can have its own PASID space, but by convention the IOMMU ended up having a global PASID space, so that with SVA, each mm_struct is associated to a single PASID. The allocator is primarily used by IOMMU subsystem but in rare occasions drivers would like to allocate PASIDs for devices that aren't managed by an IOMMU, using the same ID space as IOMMU. Signed-off-by: Jean-Philippe Brucker Signed-off-by: Jacob Pan --- The most recent discussion on this patch was at: https://lkml.kernel.org/lkml/1556922737-76313-4-git-send-email-jacob.jun.pan@linux.intel.com/ I fixed it up a bit following comments in that series, and removed the definitions for the custom allocator for now. There also is a new version that includes the custom allocator into this patch, but is currently missing the RCU fixes, at: https://lore.kernel.org/lkml/1560087862-57608-13-git-send-email-jacob.jun.pan@linux.intel.com/ --- drivers/iommu/Kconfig | 4 ++ drivers/iommu/Makefile | 1 + drivers/iommu/ioasid.c | 150 +++++++++++++++++++++++++++++++++++++++++ include/linux/ioasid.h | 49 ++++++++++++++ 4 files changed, 204 insertions(+) create mode 100644 drivers/iommu/ioasid.c create mode 100644 include/linux/ioasid.h diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index 83664db5221d..9b45f70549a7 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -3,6 +3,10 @@ config IOMMU_IOVA tristate +# The IOASID library may also be used by non-IOMMU_API users +config IOASID + tristate + # IOMMU_API always gets selected by whoever wants it. config IOMMU_API bool diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 8c71a15e986b..0efac6f1ec73 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_IOMMU_DMA) += dma-iommu.o obj-$(CONFIG_IOMMU_IO_PGTABLE) += io-pgtable.o obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) += io-pgtable-arm-v7s.o obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) += io-pgtable-arm.o +obj-$(CONFIG_IOASID) += ioasid.o obj-$(CONFIG_IOMMU_IOVA) += iova.o obj-$(CONFIG_OF_IOMMU) += of_iommu.o obj-$(CONFIG_MSM_IOMMU) += msm_iommu.o diff --git a/drivers/iommu/ioasid.c b/drivers/iommu/ioasid.c new file mode 100644 index 000000000000..bbb771214fa9 --- /dev/null +++ b/drivers/iommu/ioasid.c @@ -0,0 +1,150 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * I/O Address Space ID allocator. There is one global IOASID space, split into + * subsets. Users create a subset with DECLARE_IOASID_SET, then allocate and + * free IOASIDs with ioasid_alloc and ioasid_free. + */ +#include +#include +#include +#include +#include + +struct ioasid_data { + ioasid_t id; + struct ioasid_set *set; + void *private; + struct rcu_head rcu; +}; + +static DEFINE_XARRAY_ALLOC(ioasid_xa); + +/** + * ioasid_set_data - Set private data for an allocated ioasid + * @ioasid: the ID to set data + * @data: the private data + * + * For IOASID that is already allocated, private data can be set + * via this API. Future lookup can be done via ioasid_find. + */ +int ioasid_set_data(ioasid_t ioasid, void *data) +{ + struct ioasid_data *ioasid_data; + int ret = 0; + + xa_lock(&ioasid_xa); + ioasid_data = xa_load(&ioasid_xa, ioasid); + if (ioasid_data) + rcu_assign_pointer(ioasid_data->private, data); + else + ret = -ENOENT; + xa_unlock(&ioasid_xa); + + /* + * Wait for readers to stop accessing the old private data, so the + * caller can free it. + */ + if (!ret) + synchronize_rcu(); + + return ret; +} +EXPORT_SYMBOL_GPL(ioasid_set_data); + +/** + * ioasid_alloc - Allocate an IOASID + * @set: the IOASID set + * @min: the minimum ID (inclusive) + * @max: the maximum ID (inclusive) + * @private: data private to the caller + * + * Allocate an ID between @min and @max. The @private pointer is stored + * internally and can be retrieved with ioasid_find(). + * + * Return: the allocated ID on success, or %INVALID_IOASID on failure. + */ +ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max, + void *private) +{ + u32 id = INVALID_IOASID; + struct ioasid_data *data; + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return INVALID_IOASID; + + data->set = set; + data->private = private; + + if (xa_alloc(&ioasid_xa, &id, data, XA_LIMIT(min, max), GFP_KERNEL)) { + pr_err("Failed to alloc ioasid from %d to %d\n", min, max); + goto exit_free; + } + data->id = id; + +exit_free: + if (id == INVALID_IOASID) { + kfree(data); + return INVALID_IOASID; + } + return id; +} +EXPORT_SYMBOL_GPL(ioasid_alloc); + +/** + * ioasid_free - Free an IOASID + * @ioasid: the ID to remove + */ +void ioasid_free(ioasid_t ioasid) +{ + struct ioasid_data *ioasid_data; + + ioasid_data = xa_erase(&ioasid_xa, ioasid); + + kfree_rcu(ioasid_data, rcu); +} +EXPORT_SYMBOL_GPL(ioasid_free); + +/** + * ioasid_find - Find IOASID data + * @set: the IOASID set + * @ioasid: the IOASID to find + * @getter: function to call on the found object + * + * The optional getter function allows to take a reference to the found object + * under the rcu lock. The function can also check if the object is still valid: + * if @getter returns false, then the object is invalid and NULL is returned. + * + * If the IOASID has been allocated for this set, return the private pointer + * passed to ioasid_alloc. Private data can be NULL if not set. Return an error + * if the IOASID is not found or does not belong to the set. + */ +void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid, + bool (*getter)(void *)) +{ + void *priv = NULL; + struct ioasid_data *ioasid_data; + + rcu_read_lock(); + ioasid_data = xa_load(&ioasid_xa, ioasid); + if (!ioasid_data) { + priv = ERR_PTR(-ENOENT); + goto unlock; + } + if (set && ioasid_data->set != set) { + /* data found but does not belong to the set */ + priv = ERR_PTR(-EACCES); + goto unlock; + } + /* Now IOASID and its set is verified, we can return the private data */ + priv = rcu_dereference(ioasid_data->private); + if (getter && !getter(priv)) + priv = NULL; +unlock: + rcu_read_unlock(); + + return priv; +} +EXPORT_SYMBOL_GPL(ioasid_find); + +MODULE_LICENSE("GPL"); diff --git a/include/linux/ioasid.h b/include/linux/ioasid.h new file mode 100644 index 000000000000..940212422b8f --- /dev/null +++ b/include/linux/ioasid.h @@ -0,0 +1,49 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __LINUX_IOASID_H +#define __LINUX_IOASID_H + +#include + +#define INVALID_IOASID ((ioasid_t)-1) +typedef unsigned int ioasid_t; + +struct ioasid_set { + int dummy; +}; + +#define DECLARE_IOASID_SET(name) struct ioasid_set name = { 0 } + +#if IS_ENABLED(CONFIG_IOASID) +ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max, + void *private); +void ioasid_free(ioasid_t ioasid); + +void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid, + bool (*getter)(void *)); + +int ioasid_set_data(ioasid_t ioasid, void *data); + +#else /* !CONFIG_IOASID */ +static inline ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, + ioasid_t max, void *private) +{ + return INVALID_IOASID; +} + +static inline void ioasid_free(ioasid_t ioasid) +{ +} + +static inline void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid, + bool (*getter)(void *)) +{ + return NULL; +} + +static inline int ioasid_set_data(ioasid_t ioasid, void *data) +{ + return -ENODEV; +} + +#endif /* CONFIG_IOASID */ +#endif /* __LINUX_IOASID_H */ From patchwork Mon Jun 10 18:47:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10985289 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 888F214DB for ; Mon, 10 Jun 2019 18:52:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7547B2876B for ; Mon, 10 Jun 2019 18:52:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 694A628770; Mon, 10 Jun 2019 18:52:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 0D0542876B for ; Mon, 10 Jun 2019 18:52:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=syOZpeKMVRt21MTMzfx3ip9QaXEE6Wa6Y24lEVvsBbY=; b=NywqA6fIRG/0Kq wbNabELKjJ/yQOd7WuUUIFiOE8xU+74+vkkKAceXWxw6orrAfDalW1TeUdEqxajN7jQXx5RIdg9YG rI62TQDc6Khn4ttnT6I5UVMezt2Whdge10EHiJLyEV6KwNYCedf2M8zegrR0htEHeX3FYk41D0IEN HXZ2kX3wluDAFc3g3QEQrHDO3rLSqWMWCbMsfuf4rvJZ8ifLjjHVz2W5W8mGh19zW8yr/0fg5Kjqg 6eUgi5EhXSpccvumltXuVDZjHcyxNyMQQywF8X2mH0qf2Ju/LtmNDzq9hf9HCvnuES1slSZabuhUd lamnjZk2V02Dxks1nXag==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPP1-0002Ns-UI; Mon, 10 Jun 2019 18:52:20 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPOc-00028w-R5 for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2019 18:51:56 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 1C646C15; Mon, 10 Jun 2019 11:51:52 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.196.129]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id A12A93F246; Mon, 10 Jun 2019 11:51:50 -0700 (PDT) From: Jean-Philippe Brucker To: will.deacon@arm.com Subject: [PATCH 2/8] dt-bindings: document PASID property for IOMMU masters Date: Mon, 10 Jun 2019 19:47:08 +0100 Message-Id: <20190610184714.6786-3-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190610184714.6786-1-jean-philippe.brucker@arm.com> References: <20190610184714.6786-1-jean-philippe.brucker@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190610_115154_928547_01413D96 X-CRM114-Status: GOOD ( 11.30 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, jacob.jun.pan@linux.intel.com, joro@8bytes.org, linux-kernel@vger.kernel.org, eric.auger@redhat.com, iommu@lists.linux-foundation.org, robh+dt@kernel.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP On Arm systems, some platform devices behind an SMMU may support the PASID feature, which offers multiple address space. Let the firmware tell us when a device supports PASID. Reviewed-by: Rob Herring Signed-off-by: Jean-Philippe Brucker Reviewed-by: Eric Auger --- Previous discussion on this patch last year: https://patchwork.ozlabs.org/patch/872275/ I split PASID and stall definitions, keeping only PASID here. --- Documentation/devicetree/bindings/iommu/iommu.txt | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/Documentation/devicetree/bindings/iommu/iommu.txt b/Documentation/devicetree/bindings/iommu/iommu.txt index 5a8b4624defc..3c36334e4f94 100644 --- a/Documentation/devicetree/bindings/iommu/iommu.txt +++ b/Documentation/devicetree/bindings/iommu/iommu.txt @@ -86,6 +86,12 @@ have a means to turn off translation. But it is invalid in such cases to disable the IOMMU's device tree node in the first place because it would prevent any driver from properly setting up the translations. +Optional properties: +-------------------- +- pasid-num-bits: Some masters support multiple address spaces for DMA, by + tagging DMA transactions with an address space identifier. By default, + this is 0, which means that the device only has one address space. + Notes: ====== From patchwork Mon Jun 10 18:47:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10985291 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6B3CC1398 for ; Mon, 10 Jun 2019 18:52:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 579AE28759 for ; Mon, 10 Jun 2019 18:52:40 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4B8482876E; Mon, 10 Jun 2019 18:52:40 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id D35C228759 for ; Mon, 10 Jun 2019 18:52:39 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3/OU4rIf6XKPGOjTFklf5K38WZDAFzS33qRwF1VdPds=; b=Tlqxf+6xJ1Cm0D RULFXeCqMol7IUPq6OtRrbv+sw8FstZzqoaKdMVBtSvhnaezKntYlyiHhbDgUCuj2MMFFyyron0o4 CEXC8oLrS9QipAS3lXFq2PikaDEpiq5/iOyxrQ6DdHR+Ng5/VGglsmvbQHoqbU9dAJQ6dHYZpmEvE T2uVHehBxcIA7woLt24hWAn1fWZtY/ziQ6Aes5o+rcSlpChsu7r/Kp/9N807Cffst2zl56+DIrdtB EmAVIlMp6rxJrCJhhahzjRgXPGZAJ7/7SrSVQB/LeCWADI3ty3lxo3mv7sOu564MFrdCC0NPD0M8o hhFESBmz3Zi1W2q88Ntg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPPH-0002fb-0V; Mon, 10 Jun 2019 18:52:35 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPOd-000296-4e for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2019 18:51:57 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id BDCD0C0A; Mon, 10 Jun 2019 11:51:53 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.196.129]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 4E5233F246; Mon, 10 Jun 2019 11:51:52 -0700 (PDT) From: Jean-Philippe Brucker To: will.deacon@arm.com Subject: [PATCH 3/8] iommu/arm-smmu-v3: Support platform SSID Date: Mon, 10 Jun 2019 19:47:09 +0100 Message-Id: <20190610184714.6786-4-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190610184714.6786-1-jean-philippe.brucker@arm.com> References: <20190610184714.6786-1-jean-philippe.brucker@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190610_115155_224013_ABC0F3F2 X-CRM114-Status: GOOD ( 13.64 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, jacob.jun.pan@linux.intel.com, joro@8bytes.org, linux-kernel@vger.kernel.org, eric.auger@redhat.com, iommu@lists.linux-foundation.org, robh+dt@kernel.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP For platform devices that support SubstreamID (SSID), firmware provides the number of supported SSID bits. Restrict it to what the SMMU supports and cache it into master->ssid_bits. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 11 +++++++++++ drivers/iommu/of_iommu.c | 6 +++++- include/linux/iommu.h | 1 + 3 files changed, 17 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 4d5a694f02c2..3254f473e681 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -604,6 +604,7 @@ struct arm_smmu_master { struct list_head domain_head; u32 *sids; unsigned int num_sids; + unsigned int ssid_bits; bool ats_enabled :1; }; @@ -2097,6 +2098,16 @@ static int arm_smmu_add_device(struct device *dev) } } + master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits); + + /* + * If the SMMU doesn't support 2-stage CD, limit the linear + * tables to a reasonable number of contexts, let's say + * 64kB / sizeof(ctx_desc) = 1024 = 2^10 + */ + if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB)) + master->ssid_bits = min(master->ssid_bits, 10U); + group = iommu_group_get_for_dev(dev); if (!IS_ERR(group)) { iommu_group_put(group); diff --git a/drivers/iommu/of_iommu.c b/drivers/iommu/of_iommu.c index f04a6df65eb8..04f4f6b95d82 100644 --- a/drivers/iommu/of_iommu.c +++ b/drivers/iommu/of_iommu.c @@ -206,8 +206,12 @@ const struct iommu_ops *of_iommu_configure(struct device *dev, if (err) break; } - } + fwspec = dev_iommu_fwspec_get(dev); + if (!err && fwspec) + of_property_read_u32(master_np, "pasid-num-bits", + &fwspec->num_pasid_bits); + } /* * Two success conditions can be represented by non-negative err here: diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 519e40fb23ce..b91df613385f 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -536,6 +536,7 @@ struct iommu_fwspec { struct fwnode_handle *iommu_fwnode; void *iommu_priv; u32 flags; + u32 num_pasid_bits; unsigned int num_ids; u32 ids[1]; }; From patchwork Mon Jun 10 18:47:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10985297 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 2C10114B6 for ; Mon, 10 Jun 2019 18:53:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 14C0D2876B for ; Mon, 10 Jun 2019 18:53:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 046A328759; Mon, 10 Jun 2019 18:53:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1142928759 for ; Mon, 10 Jun 2019 18:53:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=J/4CTrUVGxCB1/bxLLSxGSsiT7TdVaDKogo2pbEfeDE=; b=brnY1AVu5oyhJI bNa/vhrjkXV2J8BRC+52MY4itapMm38RzKPo62cYWVrXsFogX/HZk+bTtuvkjWdA7y4+6LA85Mtwd Wm4MUIPMS55E/EJieOxDQw1JNsKOFSIUMnx7cS1a71tP3fhGrB1b30TKyvDOhF7EP2WAExCjVdZFa 1c5tGTG9haLckHbsBLmyZ9tviD2fLCfa+ddic2qQ2Z7MbtIHl4OxEa6BlJ+HeMQ+VGcQdcYqhyvj3 Yc3SqJ1H46G/VfjHIWUS32Jo9bZNMBzjzbSkS50Jn8aIpfJJlg66zwlGeI7fq+KqfU8jUVbUIRG3z Yn1pKBjePHRy1WBvC9pg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPPt-00039u-JU; Mon, 10 Jun 2019 18:53:13 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPOf-0002Ai-7X for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2019 18:51:59 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 87738C1D; Mon, 10 Jun 2019 11:51:55 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.196.129]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id F0EBB3F246; Mon, 10 Jun 2019 11:51:53 -0700 (PDT) From: Jean-Philippe Brucker To: will.deacon@arm.com Subject: [PATCH 4/8] iommu/arm-smmu-v3: Add support for Substream IDs Date: Mon, 10 Jun 2019 19:47:10 +0100 Message-Id: <20190610184714.6786-5-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190610184714.6786-1-jean-philippe.brucker@arm.com> References: <20190610184714.6786-1-jean-philippe.brucker@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190610_115157_363437_762E926C X-CRM114-Status: GOOD ( 23.22 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, jacob.jun.pan@linux.intel.com, joro@8bytes.org, linux-kernel@vger.kernel.org, eric.auger@redhat.com, iommu@lists.linux-foundation.org, robh+dt@kernel.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP At the moment, the SMMUv3 driver implements only one stage-1 or stage-2 page directory per device. However SMMUv3 allows more than one address space for some devices, by providing multiple stage-1 page directories. In addition to the Stream ID (SID), that identifies a device, we can now have Substream IDs (SSID) identifying an address space. In PCIe, SID is called Requester ID (RID) and SSID is called Process Address-Space ID (PASID). Prepare the driver for SSID support, by adding context descriptor tables in STEs (previously a single static context descriptor). A complete stage-1 walk is now performed like this by the SMMU: Stream tables Ctx. tables Page tables +--------+ ,------->+-------+ ,------->+-------+ : : | : : | : : +--------+ | +-------+ | +-------+ SID->| STE |---' SSID->| CD |---' IOVA->| PTE |--> IPA +--------+ +-------+ +-------+ : : : : : : +--------+ +-------+ +-------+ Implement a single level of context descriptor table for now, but as with stream and page tables, an SSID can be split to index multiple levels of tables. In all stream table entries, we set S1DSS=SSID0 mode, making translations without an SSID use context descriptor 0. Although it would be possible by setting S1DSS=BYPASS, we don't currently support SSID when user selects iommu.passthrough. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 238 +++++++++++++++++++++++++++++------- 1 file changed, 192 insertions(+), 46 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 3254f473e681..d90eb604b65d 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -219,6 +219,11 @@ #define STRTAB_STE_0_S1CTXPTR_MASK GENMASK_ULL(51, 6) #define STRTAB_STE_0_S1CDMAX GENMASK_ULL(63, 59) +#define STRTAB_STE_1_S1DSS GENMASK_ULL(1, 0) +#define STRTAB_STE_1_S1DSS_TERMINATE 0x0 +#define STRTAB_STE_1_S1DSS_BYPASS 0x1 +#define STRTAB_STE_1_S1DSS_SSID0 0x2 + #define STRTAB_STE_1_S1C_CACHE_NC 0UL #define STRTAB_STE_1_S1C_CACHE_WBRA 1UL #define STRTAB_STE_1_S1C_CACHE_WT 2UL @@ -305,6 +310,7 @@ #define CMDQ_PREFETCH_1_SIZE GENMASK_ULL(4, 0) #define CMDQ_PREFETCH_1_ADDR_MASK GENMASK_ULL(63, 12) +#define CMDQ_CFGI_0_SSID GENMASK_ULL(31, 12) #define CMDQ_CFGI_0_SID GENMASK_ULL(63, 32) #define CMDQ_CFGI_1_LEAF (1UL << 0) #define CMDQ_CFGI_1_RANGE GENMASK_ULL(4, 0) @@ -421,8 +427,11 @@ struct arm_smmu_cmdq_ent { #define CMDQ_OP_CFGI_STE 0x3 #define CMDQ_OP_CFGI_ALL 0x4 + #define CMDQ_OP_CFGI_CD 0x5 + #define CMDQ_OP_CFGI_CD_ALL 0x6 struct { u32 sid; + u32 ssid; union { bool leaf; u8 span; @@ -506,16 +515,25 @@ struct arm_smmu_strtab_l1_desc { dma_addr_t l2ptr_dma; }; +struct arm_smmu_cd_table { + __le64 *ptr; + dma_addr_t ptr_dma; +}; + +struct arm_smmu_ctx_desc { + u16 asid; + u64 ttbr; + u64 tcr; + u64 mair; +}; + struct arm_smmu_s1_cfg { - __le64 *cdptr; - dma_addr_t cdptr_dma; - - struct arm_smmu_ctx_desc { - u16 asid; - u64 ttbr; - u64 tcr; - u64 mair; - } cd; + u8 s1fmt; + u8 s1cdmax; + struct arm_smmu_cd_table table; + + /* Context descriptor 0, when substreams are disabled or s1dss = 0b10 */ + struct arm_smmu_ctx_desc cd; }; struct arm_smmu_s2_cfg { @@ -811,10 +829,16 @@ static int arm_smmu_cmdq_build_cmd(u64 *cmd, struct arm_smmu_cmdq_ent *ent) cmd[1] |= FIELD_PREP(CMDQ_PREFETCH_1_SIZE, ent->prefetch.size); cmd[1] |= ent->prefetch.addr & CMDQ_PREFETCH_1_ADDR_MASK; break; + case CMDQ_OP_CFGI_CD: + cmd[0] |= FIELD_PREP(CMDQ_CFGI_0_SSID, ent->cfgi.ssid); + /* Fallthrough */ case CMDQ_OP_CFGI_STE: cmd[0] |= FIELD_PREP(CMDQ_CFGI_0_SID, ent->cfgi.sid); cmd[1] |= FIELD_PREP(CMDQ_CFGI_1_LEAF, ent->cfgi.leaf); break; + case CMDQ_OP_CFGI_CD_ALL: + cmd[0] |= FIELD_PREP(CMDQ_CFGI_0_SID, ent->cfgi.sid); + break; case CMDQ_OP_CFGI_ALL: /* Cover the entire SID range */ cmd[1] |= FIELD_PREP(CMDQ_CFGI_1_RANGE, 31); @@ -1045,6 +1069,63 @@ static int arm_smmu_cmdq_issue_sync(struct arm_smmu_device *smmu) } /* Context descriptor manipulation functions */ +static void arm_smmu_sync_cd(struct arm_smmu_domain *smmu_domain, + int ssid, bool leaf) +{ + size_t i; + unsigned long flags; + struct arm_smmu_master *master; + struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_cmdq_ent cmd = { + .opcode = CMDQ_OP_CFGI_CD, + .cfgi = { + .ssid = ssid, + .leaf = leaf, + }, + }; + + spin_lock_irqsave(&smmu_domain->devices_lock, flags); + list_for_each_entry(master, &smmu_domain->devices, domain_head) { + for (i = 0; i < master->num_sids; i++) { + cmd.cfgi.sid = master->sids[i]; + arm_smmu_cmdq_issue_cmd(smmu, &cmd); + } + } + spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); + + arm_smmu_cmdq_issue_sync(smmu); +} + +static int arm_smmu_alloc_cd_leaf_table(struct arm_smmu_device *smmu, + struct arm_smmu_cd_table *table, + size_t num_entries) +{ + size_t size = num_entries * (CTXDESC_CD_DWORDS << 3); + + table->ptr = dmam_alloc_coherent(smmu->dev, size, &table->ptr_dma, + GFP_KERNEL | __GFP_ZERO); + if (!table->ptr) { + dev_warn(smmu->dev, + "failed to allocate context descriptor table\n"); + return -ENOMEM; + } + return 0; +} + +static void arm_smmu_free_cd_leaf_table(struct arm_smmu_device *smmu, + struct arm_smmu_cd_table *table, + size_t num_entries) +{ + size_t size = num_entries * (CTXDESC_CD_DWORDS << 3); + + dmam_free_coherent(smmu->dev, size, table->ptr, table->ptr_dma); +} + +static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_s1_cfg *cfg, u32 ssid) +{ + return cfg->table.ptr + ssid * CTXDESC_CD_DWORDS; +} + static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) { u64 val = 0; @@ -1062,33 +1143,90 @@ static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) return val; } -static void arm_smmu_write_ctx_desc(struct arm_smmu_device *smmu, - struct arm_smmu_s1_cfg *cfg) +static int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, + int ssid, struct arm_smmu_ctx_desc *cd) { u64 val; + bool cd_live; + struct arm_smmu_device *smmu = smmu_domain->smmu; + __le64 *cdptr = arm_smmu_get_cd_ptr(&smmu_domain->s1_cfg, ssid); /* - * We don't need to issue any invalidation here, as we'll invalidate - * the STE when installing the new entry anyway. + * This function handles the following cases: + * + * (1) Install primary CD, for normal DMA traffic (SSID = 0). + * (2) Install a secondary CD, for SID+SSID traffic. + * (3) Update ASID of a CD. Atomically write the first 64 bits of the + * CD, then invalidate the old entry and mappings. + * (4) Remove a secondary CD. */ - val = arm_smmu_cpu_tcr_to_cd(cfg->cd.tcr) | + + if (!cdptr) + return -ENOMEM; + + val = le64_to_cpu(cdptr[0]); + cd_live = !!(val & CTXDESC_CD_0_V); + + if (!cd) { /* (4) */ + cdptr[0] = 0; + } else if (cd_live) { /* (3) */ + val &= ~CTXDESC_CD_0_ASID; + val |= FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid); + + cdptr[0] = cpu_to_le64(val); + /* + * Until CD+TLB invalidation, both ASIDs may be used for tagging + * this substream's traffic + */ + } else { /* (1) and (2) */ + cdptr[1] = cpu_to_le64(cd->ttbr & CTXDESC_CD_1_TTB0_MASK); + cdptr[2] = 0; + cdptr[3] = cpu_to_le64(cd->mair); + + /* + * STE is live, and the SMMU might fetch this CD at any + * time. Ensure that it observes the rest of the CD before we + * enable it. + */ + arm_smmu_sync_cd(smmu_domain, ssid, true); + + val = arm_smmu_cpu_tcr_to_cd(cd->tcr) | #ifdef __BIG_ENDIAN - CTXDESC_CD_0_ENDI | + CTXDESC_CD_0_ENDI | #endif - CTXDESC_CD_0_R | CTXDESC_CD_0_A | CTXDESC_CD_0_ASET | - CTXDESC_CD_0_AA64 | FIELD_PREP(CTXDESC_CD_0_ASID, cfg->cd.asid) | - CTXDESC_CD_0_V; + CTXDESC_CD_0_R | CTXDESC_CD_0_A | CTXDESC_CD_0_ASET | + CTXDESC_CD_0_AA64 | + FIELD_PREP(CTXDESC_CD_0_ASID, cd->asid) | + CTXDESC_CD_0_V; + + /* STALL_MODEL==0b10 && CD.S==0 is ILLEGAL */ + if (smmu->features & ARM_SMMU_FEAT_STALL_FORCE) + val |= CTXDESC_CD_0_S; + + cdptr[0] = cpu_to_le64(val); + } - /* STALL_MODEL==0b10 && CD.S==0 is ILLEGAL */ - if (smmu->features & ARM_SMMU_FEAT_STALL_FORCE) - val |= CTXDESC_CD_0_S; + arm_smmu_sync_cd(smmu_domain, ssid, true); + return 0; +} + +static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master) +{ + struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - cfg->cdptr[0] = cpu_to_le64(val); + cfg->s1fmt = STRTAB_STE_0_S1FMT_LINEAR; + cfg->s1cdmax = master->ssid_bits; + return arm_smmu_alloc_cd_leaf_table(smmu, &cfg->table, 1 << cfg->s1cdmax); +} - val = cfg->cd.ttbr & CTXDESC_CD_1_TTB0_MASK; - cfg->cdptr[1] = cpu_to_le64(val); +static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) +{ + struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - cfg->cdptr[3] = cpu_to_le64(cfg->cd.mair); + arm_smmu_free_cd_leaf_table(smmu, &cfg->table, 1 << cfg->s1cdmax); } /* Stream table manipulation functions */ @@ -1210,6 +1348,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, if (s1_cfg) { BUG_ON(ste_live); dst[1] = cpu_to_le64( + FIELD_PREP(STRTAB_STE_1_S1DSS, STRTAB_STE_1_S1DSS_SSID0) | FIELD_PREP(STRTAB_STE_1_S1CIR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1COR, STRTAB_STE_1_S1C_CACHE_WBRA) | FIELD_PREP(STRTAB_STE_1_S1CSH, ARM_SMMU_SH_ISH) | @@ -1219,8 +1358,10 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); - val |= (s1_cfg->cdptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | - FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS); + val |= (s1_cfg->table.ptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | + FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) | + FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) | + FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt); } if (s2_cfg) { @@ -1674,12 +1815,8 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - if (cfg->cdptr) { - dmam_free_coherent(smmu_domain->smmu->dev, - CTXDESC_CD_DWORDS << 3, - cfg->cdptr, - cfg->cdptr_dma); - + if (cfg->table.ptr) { + arm_smmu_free_cd_tables(smmu_domain); arm_smmu_bitmap_free(smmu->asid_map, cfg->cd.asid); } } else { @@ -1692,6 +1829,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) } static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int ret; @@ -1703,27 +1841,29 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (asid < 0) return asid; - cfg->cdptr = dmam_alloc_coherent(smmu->dev, CTXDESC_CD_DWORDS << 3, - &cfg->cdptr_dma, - GFP_KERNEL | __GFP_ZERO); - if (!cfg->cdptr) { - dev_warn(smmu->dev, "failed to allocate context descriptor\n"); - ret = -ENOMEM; + ret = arm_smmu_alloc_cd_tables(smmu_domain, master); + if (ret) goto out_free_asid; - } cfg->cd.asid = (u16)asid; cfg->cd.ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr[0]; cfg->cd.tcr = pgtbl_cfg->arm_lpae_s1_cfg.tcr; cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair[0]; + + ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &smmu_domain->s1_cfg.cd); + if (ret) + goto out_free_table; return 0; +out_free_table: + arm_smmu_free_cd_tables(smmu_domain); out_free_asid: arm_smmu_bitmap_free(smmu->asid_map, asid); return ret; } static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { int vmid; @@ -1740,7 +1880,8 @@ static int arm_smmu_domain_finalise_s2(struct arm_smmu_domain *smmu_domain, return 0; } -static int arm_smmu_domain_finalise(struct iommu_domain *domain) +static int arm_smmu_domain_finalise(struct iommu_domain *domain, + struct arm_smmu_master *master) { int ret; unsigned long ias, oas; @@ -1748,6 +1889,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain) struct io_pgtable_cfg pgtbl_cfg; struct io_pgtable_ops *pgtbl_ops; int (*finalise_stage_fn)(struct arm_smmu_domain *, + struct arm_smmu_master *, struct io_pgtable_cfg *); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); struct arm_smmu_device *smmu = smmu_domain->smmu; @@ -1804,7 +1946,7 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain) domain->geometry.aperture_end = (1UL << pgtbl_cfg.ias) - 1; domain->geometry.force_aperture = true; - ret = finalise_stage_fn(smmu_domain, &pgtbl_cfg); + ret = finalise_stage_fn(smmu_domain, master, &pgtbl_cfg); if (ret < 0) { free_io_pgtable_ops(pgtbl_ops); return ret; @@ -1932,7 +2074,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (!smmu_domain->smmu) { smmu_domain->smmu = smmu; - ret = arm_smmu_domain_finalise(domain); + ret = arm_smmu_domain_finalise(domain, master); if (ret) { smmu_domain->smmu = NULL; goto out_unlock; @@ -1944,6 +2086,13 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) dev_name(smmu->dev)); ret = -ENXIO; goto out_unlock; + } else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1 && + master->ssid_bits != smmu_domain->s1_cfg.s1cdmax) { + dev_err(dev, + "cannot attach to incompatible domain (%u SSID bits != %u)\n", + smmu_domain->s1_cfg.s1cdmax, master->ssid_bits); + ret = -EINVAL; + goto out_unlock; } master->domain = smmu_domain; @@ -1955,9 +2104,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS) arm_smmu_enable_ats(master); - if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) - arm_smmu_write_ctx_desc(smmu, &smmu_domain->s1_cfg); - arm_smmu_install_ste_for_dev(master); out_unlock: mutex_unlock(&smmu_domain->init_mutex); From patchwork Mon Jun 10 18:47:11 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10985299 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 402A014DB for ; Mon, 10 Jun 2019 18:53:42 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2B7B32876E for ; Mon, 10 Jun 2019 18:53:42 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1F40028787; Mon, 10 Jun 2019 18:53:42 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7A81A2876E for ; Mon, 10 Jun 2019 18:53:41 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Gj5dd3I/wFGaVFF6SRq3fwYZXreL6hVcxbFX5L5zPIU=; b=N6aAfST/VIoAWN G81AemN/Frj/TMs/tLvwVeLsioDFiSwjskUw4KUHh4NRJF3BDOwMQNTCI6rjfs6l4Iuozo/OIC++B 3wSMLe/zoqQHGIcNxNp+uHrVCoosikeg4xqI0nUgRViQQW86EFd133Rp/hK7ymyIv7tc+xnEYDL30 YHFL2BXhhAEJ+YElNhMcenB3isYb4GYtHdPnebJ1DFv4pIf+SWM/6fLTBsgPoaSfPbBLNnkJTVu9m Iag6mvAaUYoVdCc5OFz3kiMyReGx5xUdPxbs+76WVLq/YQdHUAmEi5bhVkbeab8OBoKzPm0p439Gw pUtis9rFsih+lr9u1W0Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPQF-0003Od-9h; Mon, 10 Jun 2019 18:53:35 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPOg-0002B5-6H for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2019 18:52:00 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 35778EBD; Mon, 10 Jun 2019 11:51:57 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.196.129]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id BA39E3F246; Mon, 10 Jun 2019 11:51:55 -0700 (PDT) From: Jean-Philippe Brucker To: will.deacon@arm.com Subject: [PATCH 5/8] iommu/arm-smmu-v3: Add second level of context descriptor table Date: Mon, 10 Jun 2019 19:47:11 +0100 Message-Id: <20190610184714.6786-6-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190610184714.6786-1-jean-philippe.brucker@arm.com> References: <20190610184714.6786-1-jean-philippe.brucker@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190610_115158_395496_407437A5 X-CRM114-Status: GOOD ( 17.23 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, jacob.jun.pan@linux.intel.com, joro@8bytes.org, linux-kernel@vger.kernel.org, eric.auger@redhat.com, iommu@lists.linux-foundation.org, robh+dt@kernel.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The SMMU can support up to 20 bits of SSID. Add a second level of page tables to accommodate this. Devices that support more than 1024 SSIDs now have a table of 1024 L1 entries (8kB), pointing to tables of 1024 context descriptors (64kB), allocated on demand. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 136 +++++++++++++++++++++++++++++++++--- 1 file changed, 128 insertions(+), 8 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index d90eb604b65d..326b71793336 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -216,6 +216,8 @@ #define STRTAB_STE_0_S1FMT GENMASK_ULL(5, 4) #define STRTAB_STE_0_S1FMT_LINEAR 0 +#define STRTAB_STE_0_S1FMT_4K_L2 1 +#define STRTAB_STE_0_S1FMT_64K_L2 2 #define STRTAB_STE_0_S1CTXPTR_MASK GENMASK_ULL(51, 6) #define STRTAB_STE_0_S1CDMAX GENMASK_ULL(63, 59) @@ -255,6 +257,18 @@ #define STRTAB_STE_3_S2TTB_MASK GENMASK_ULL(51, 4) +/* + * Linear: when less than 1024 SSIDs are supported + * 2lvl: at most 1024 L1 entrie, + * 1024 lazy entries per table. + */ +#define CTXDESC_SPLIT 10 +#define CTXDESC_NUM_L2_ENTRIES (1 << CTXDESC_SPLIT) + +#define CTXDESC_L1_DESC_DWORD 1 +#define CTXDESC_L1_DESC_VALID 1 +#define CTXDESC_L1_DESC_L2PTR_MASK GENMASK_ULL(51, 12) + /* Context descriptor (stage-1 only) */ #define CTXDESC_CD_DWORDS 8 #define CTXDESC_CD_0_TCR_T0SZ GENMASK_ULL(5, 0) @@ -530,7 +544,10 @@ struct arm_smmu_ctx_desc { struct arm_smmu_s1_cfg { u8 s1fmt; u8 s1cdmax; - struct arm_smmu_cd_table table; + struct arm_smmu_cd_table *tables; + size_t num_tables; + __le64 *l1ptr; + dma_addr_t l1ptr_dma; /* Context descriptor 0, when substreams are disabled or s1dss = 0b10 */ struct arm_smmu_ctx_desc cd; @@ -1118,12 +1135,51 @@ static void arm_smmu_free_cd_leaf_table(struct arm_smmu_device *smmu, { size_t size = num_entries * (CTXDESC_CD_DWORDS << 3); + if (!table->ptr) + return; dmam_free_coherent(smmu->dev, size, table->ptr, table->ptr_dma); } -static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_s1_cfg *cfg, u32 ssid) +static void arm_smmu_write_cd_l1_desc(__le64 *dst, + struct arm_smmu_cd_table *table) { - return cfg->table.ptr + ssid * CTXDESC_CD_DWORDS; + u64 val = (table->ptr_dma & CTXDESC_L1_DESC_L2PTR_MASK) | + CTXDESC_L1_DESC_VALID; + + *dst = cpu_to_le64(val); +} + +static __le64 *arm_smmu_get_cd_ptr(struct arm_smmu_domain *smmu_domain, + u32 ssid) +{ + unsigned int idx; + struct arm_smmu_cd_table *table; + struct arm_smmu_device *smmu = smmu_domain->smmu; + struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + + if (cfg->s1fmt == STRTAB_STE_0_S1FMT_LINEAR) { + table = &cfg->tables[0]; + idx = ssid; + } else { + idx = ssid >> CTXDESC_SPLIT; + if (idx >= cfg->num_tables) + return NULL; + + table = &cfg->tables[idx]; + if (!table->ptr) { + __le64 *l1ptr = cfg->l1ptr + idx * CTXDESC_L1_DESC_DWORD; + + if (arm_smmu_alloc_cd_leaf_table(smmu, table, + CTXDESC_NUM_L2_ENTRIES)) + return NULL; + + arm_smmu_write_cd_l1_desc(l1ptr, table); + /* An invalid L1 entry is allowed to be cached */ + arm_smmu_sync_cd(smmu_domain, ssid, false); + } + idx = ssid & (CTXDESC_NUM_L2_ENTRIES - 1); + } + return table->ptr + idx * CTXDESC_CD_DWORDS; } static u64 arm_smmu_cpu_tcr_to_cd(u64 tcr) @@ -1149,7 +1205,7 @@ static int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, u64 val; bool cd_live; struct arm_smmu_device *smmu = smmu_domain->smmu; - __le64 *cdptr = arm_smmu_get_cd_ptr(&smmu_domain->s1_cfg, ssid); + __le64 *cdptr = arm_smmu_get_cd_ptr(smmu_domain, ssid); /* * This function handles the following cases: @@ -1213,20 +1269,81 @@ static int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, static int arm_smmu_alloc_cd_tables(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master) { + int ret; + size_t size = 0; + size_t max_contexts, num_leaf_entries; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; cfg->s1fmt = STRTAB_STE_0_S1FMT_LINEAR; cfg->s1cdmax = master->ssid_bits; - return arm_smmu_alloc_cd_leaf_table(smmu, &cfg->table, 1 << cfg->s1cdmax); + + max_contexts = 1 << cfg->s1cdmax; + if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB) || + max_contexts <= CTXDESC_NUM_L2_ENTRIES) { + cfg->s1fmt = STRTAB_STE_0_S1FMT_LINEAR; + cfg->num_tables = 1; + num_leaf_entries = max_contexts; + } else { + cfg->s1fmt = STRTAB_STE_0_S1FMT_64K_L2; + /* + * SSID[S1CDmax-1:10] indexes 1st-level table, SSID[9:0] indexes + * 2nd-level + */ + cfg->num_tables = max_contexts / CTXDESC_NUM_L2_ENTRIES; + + size = cfg->num_tables * (CTXDESC_L1_DESC_DWORD << 3); + cfg->l1ptr = dmam_alloc_coherent(smmu->dev, size, + &cfg->l1ptr_dma, + GFP_KERNEL | __GFP_ZERO); + if (!cfg->l1ptr) { + dev_warn(smmu->dev, "failed to allocate L1 context table\n"); + return -ENOMEM; + } + + num_leaf_entries = CTXDESC_NUM_L2_ENTRIES; + } + + cfg->tables = devm_kzalloc(smmu->dev, sizeof(struct arm_smmu_cd_table) * + cfg->num_tables, GFP_KERNEL); + if (!cfg->tables) + return -ENOMEM; + + ret = arm_smmu_alloc_cd_leaf_table(smmu, &cfg->tables[0], num_leaf_entries); + if (ret) + goto err_free_l1; + + if (cfg->l1ptr) + arm_smmu_write_cd_l1_desc(cfg->l1ptr, &cfg->tables[0]); + + return 0; + +err_free_l1: + if (cfg->l1ptr) + dmam_free_coherent(smmu->dev, size, cfg->l1ptr, cfg->l1ptr_dma); + devm_kfree(smmu->dev, cfg->tables); + return ret; } static void arm_smmu_free_cd_tables(struct arm_smmu_domain *smmu_domain) { + int i; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + size_t num_leaf_entries = 1 << cfg->s1cdmax; + struct arm_smmu_cd_table *table = cfg->tables; + + if (cfg->l1ptr) { + size_t size = cfg->num_tables * (CTXDESC_L1_DESC_DWORD << 3); - arm_smmu_free_cd_leaf_table(smmu, &cfg->table, 1 << cfg->s1cdmax); + dmam_free_coherent(smmu->dev, size, cfg->l1ptr, + cfg->l1ptr_dma); + num_leaf_entries = CTXDESC_NUM_L2_ENTRIES; + } + + for (i = 0; i < cfg->num_tables; i++, table++) + arm_smmu_free_cd_leaf_table(smmu, table, num_leaf_entries); + devm_kfree(smmu->dev, cfg->tables); } /* Stream table manipulation functions */ @@ -1346,6 +1463,9 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, } if (s1_cfg) { + dma_addr_t ptr_dma = s1_cfg->l1ptr ? s1_cfg->l1ptr_dma : + s1_cfg->tables[0].ptr_dma; + BUG_ON(ste_live); dst[1] = cpu_to_le64( FIELD_PREP(STRTAB_STE_1_S1DSS, STRTAB_STE_1_S1DSS_SSID0) | @@ -1358,7 +1478,7 @@ static void arm_smmu_write_strtab_ent(struct arm_smmu_master *master, u32 sid, !(smmu->features & ARM_SMMU_FEAT_STALL_FORCE)) dst[1] |= cpu_to_le64(STRTAB_STE_1_S1STALLD); - val |= (s1_cfg->table.ptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | + val |= (ptr_dma & STRTAB_STE_0_S1CTXPTR_MASK) | FIELD_PREP(STRTAB_STE_0_CFG, STRTAB_STE_0_CFG_S1_TRANS) | FIELD_PREP(STRTAB_STE_0_S1CDMAX, s1_cfg->s1cdmax) | FIELD_PREP(STRTAB_STE_0_S1FMT, s1_cfg->s1fmt); @@ -1815,7 +1935,7 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - if (cfg->table.ptr) { + if (cfg->tables) { arm_smmu_free_cd_tables(smmu_domain); arm_smmu_bitmap_free(smmu->asid_map, cfg->cd.asid); } From patchwork Mon Jun 10 18:47:12 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10985303 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EC74814B6 for ; Mon, 10 Jun 2019 18:53:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D90BC28759 for ; Mon, 10 Jun 2019 18:53:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id CD3642876E; Mon, 10 Jun 2019 18:53:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DA8BF28759 for ; Mon, 10 Jun 2019 18:53:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=dOVUANl3xqviexAC4xcCgkUh2BCIEyGCk8tF0rMWJ6o=; b=dFgJzZTuuIe12h z6/d9nMxM8m2sfjif2Bo+ZIAtKDu95NyI4vpwCZIdk1XWQ8fszMXiqv8LDvjLMwCoYiVdyK4A4jUm 2JpOvRyvg9uF34c17CZacIvExtPObsWoixgoRp7jwKQUiOo1s9/CBxe7PP8bnfD12iXaM1TE44HIt 06yuefcFivy29rC1GKzL08D2XC3mL56/TNyYH7j/w1P9V5/Da7XAaLN0kdnDutfKKxLOyi4icrJNb EPX+9s5DFJX+kUBPjwdLQD4MZNnJazBoZSJLA3Zn5qZ5teb4uxpd9sOr6A1IeF+Bb9gbB60+MoNSo 6gQW3uDyaqYanDGb+/mQ==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPQY-0003hO-Ai; Mon, 10 Jun 2019 18:53:54 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPOh-0002Bw-8g for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2019 18:52:01 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D804ADA7; Mon, 10 Jun 2019 11:51:58 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.196.129]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 68A3C3F246; Mon, 10 Jun 2019 11:51:57 -0700 (PDT) From: Jean-Philippe Brucker To: will.deacon@arm.com Subject: [PATCH 6/8] iommu/arm-smmu-v3: Support auxiliary domains Date: Mon, 10 Jun 2019 19:47:12 +0100 Message-Id: <20190610184714.6786-7-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190610184714.6786-1-jean-philippe.brucker@arm.com> References: <20190610184714.6786-1-jean-philippe.brucker@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190610_115159_429563_7E64CF60 X-CRM114-Status: GOOD ( 24.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, jacob.jun.pan@linux.intel.com, joro@8bytes.org, linux-kernel@vger.kernel.org, eric.auger@redhat.com, iommu@lists.linux-foundation.org, robh+dt@kernel.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP In commit a3a195929d40 ("iommu: Add APIs for multiple domains per device"), the IOMMU API gained the concept of auxiliary domains (AUXD), which allows to control the PASID-tagged address spaces of a device. With AUXD the PASID address space are not shared with the CPU, but are instead modified with iommu_map() and iommu_unmap() calls on auxiliary domains. Add auxiliary domain support to the SMMUv3 driver. Device drivers allocate an unmanaged IOMMU domain with iommu_domain_alloc(), and attach it to the device with iommu_aux_attach_domain(). The AUXD API is fairly permissive, and allows to attach an IOMMU domain in both normal and auxiliary mode at the same time - one device can be attached to the domain normally, and another device can be attached through one of its PASIDs. To avoid excessive complexity in the SMMU implementation we pose some restrictions on supported AUXD usage: * A domain is either in auxiliary mode or normal mode. And that state is sticky. Once detached the domain has to be re-attached in the same mode. * An auxiliary domain can have a single parent domain. Two devices can be attached to the same auxiliary domain only if they are attached to the same parent domain. In practice these shouldn't be problematic, since we have the same kind of restriction on normal domains and users have been able to cope so far: at the moment a domain cannot be attached to two devices behind different SMMUs. When VFIO puts two such devices in the same container, it simply falls back to allocating two separate IOMMU domains. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/Kconfig | 1 + drivers/iommu/arm-smmu-v3.c | 276 +++++++++++++++++++++++++++++++++--- 2 files changed, 260 insertions(+), 17 deletions(-) diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index 9b45f70549a7..d326fef3d3a6 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -393,6 +393,7 @@ config ARM_SMMU_DISABLE_BYPASS_BY_DEFAULT config ARM_SMMU_V3 bool "ARM Ltd. System MMU Version 3 (SMMUv3) Support" depends on ARM64 + select IOASID select IOMMU_API select IOMMU_IO_PGTABLE_LPAE select GENERIC_MSI_IRQ_DOMAIN diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 326b71793336..633d829f246f 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include #include @@ -641,6 +642,7 @@ struct arm_smmu_master { unsigned int num_sids; unsigned int ssid_bits; bool ats_enabled :1; + bool auxd_enabled :1; }; /* SMMU private data for an IOMMU domain */ @@ -666,8 +668,14 @@ struct arm_smmu_domain { struct iommu_domain domain; + /* Unused in aux domains */ struct list_head devices; spinlock_t devices_lock; + + /* Auxiliary domain stuff */ + struct arm_smmu_domain *parent; + ioasid_t ssid; + unsigned long aux_nr_devs; }; struct arm_smmu_option_prop { @@ -675,6 +683,8 @@ struct arm_smmu_option_prop { const char *prop; }; +static DECLARE_IOASID_SET(private_ioasid); + static struct arm_smmu_option_prop arm_smmu_options[] = { { ARM_SMMU_OPT_SKIP_PREFETCH, "hisilicon,broken-prefetch-cmd" }, { ARM_SMMU_OPT_PAGE0_REGS_ONLY, "cavium,cn9900-broken-page1-regspace"}, @@ -696,6 +706,15 @@ static struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) return container_of(dom, struct arm_smmu_domain, domain); } +static struct arm_smmu_master *dev_to_master(struct device *dev) +{ + struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); + + if (!fwspec) + return NULL; + return fwspec->iommu_priv; +} + static void parse_driver_options(struct arm_smmu_device *smmu) { int i = 0; @@ -1776,13 +1795,19 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master, } static int arm_smmu_atc_inv_domain(struct arm_smmu_domain *smmu_domain, - int ssid, unsigned long iova, size_t size) + unsigned long iova, size_t size) { int ret = 0; + unsigned int ssid = 0; unsigned long flags; struct arm_smmu_cmdq_ent cmd; struct arm_smmu_master *master; + if (smmu_domain->parent) { + ssid = smmu_domain->ssid; + smmu_domain = smmu_domain->parent; + } + if (!(smmu_domain->smmu->features & ARM_SMMU_FEAT_ATS)) return 0; @@ -1935,10 +1960,12 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) { struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; - if (cfg->tables) { + if (cfg->tables) arm_smmu_free_cd_tables(smmu_domain); + if (cfg->cd.asid) arm_smmu_bitmap_free(smmu->asid_map, cfg->cd.asid); - } + if (smmu_domain->ssid) + ioasid_free(smmu_domain->ssid); } else { struct arm_smmu_s2_cfg *cfg = &smmu_domain->s2_cfg; if (cfg->vmid) @@ -1948,11 +1975,10 @@ static void arm_smmu_domain_free(struct iommu_domain *domain) kfree(smmu_domain); } -static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, +static int arm_smmu_domain_finalise_cd(struct arm_smmu_domain *smmu_domain, struct arm_smmu_master *master, struct io_pgtable_cfg *pgtbl_cfg) { - int ret; int asid; struct arm_smmu_device *smmu = smmu_domain->smmu; struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; @@ -1961,16 +1987,30 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, if (asid < 0) return asid; - ret = arm_smmu_alloc_cd_tables(smmu_domain, master); - if (ret) - goto out_free_asid; - cfg->cd.asid = (u16)asid; cfg->cd.ttbr = pgtbl_cfg->arm_lpae_s1_cfg.ttbr[0]; cfg->cd.tcr = pgtbl_cfg->arm_lpae_s1_cfg.tcr; cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair[0]; + return 0; +} + +static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, + struct arm_smmu_master *master, + struct io_pgtable_cfg *pgtbl_cfg) +{ + int ret; + struct arm_smmu_s1_cfg *cfg = &smmu_domain->s1_cfg; + struct arm_smmu_device *smmu = smmu_domain->smmu; + + ret = arm_smmu_domain_finalise_cd(smmu_domain, master, pgtbl_cfg); + if (ret) + return ret; + + ret = arm_smmu_alloc_cd_tables(smmu_domain, master); + if (ret) + goto out_free_asid; - ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &smmu_domain->s1_cfg.cd); + ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd); if (ret) goto out_free_table; return 0; @@ -1978,7 +2018,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain, out_free_table: arm_smmu_free_cd_tables(smmu_domain); out_free_asid: - arm_smmu_bitmap_free(smmu->asid_map, asid); + arm_smmu_bitmap_free(smmu->asid_map, cfg->cd.asid); return ret; } @@ -2031,7 +2071,10 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain, ias = min_t(unsigned long, ias, VA_BITS); oas = smmu->ias; fmt = ARM_64_LPAE_S1; - finalise_stage_fn = arm_smmu_domain_finalise_s1; + if (smmu_domain->parent) + finalise_stage_fn = arm_smmu_domain_finalise_cd; + else + finalise_stage_fn = arm_smmu_domain_finalise_s1; break; case ARM_SMMU_DOMAIN_NESTED: case ARM_SMMU_DOMAIN_S2: @@ -2177,15 +2220,13 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) { int ret = 0; unsigned long flags; - struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev); struct arm_smmu_device *smmu; + struct arm_smmu_master *master = dev_to_master(dev); struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); - struct arm_smmu_master *master; - if (!fwspec) + if (!master) return -ENOENT; - master = fwspec->iommu_priv; smmu = master->smmu; arm_smmu_detach_dev(master); @@ -2213,6 +2254,10 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) smmu_domain->s1_cfg.s1cdmax, master->ssid_bits); ret = -EINVAL; goto out_unlock; + } else if (smmu_domain->parent) { + dev_err(dev, "cannot attach auxiliary domain\n"); + ret = -EINVAL; + goto out_unlock; } master->domain = smmu_domain; @@ -2252,7 +2297,7 @@ arm_smmu_unmap(struct iommu_domain *domain, unsigned long iova, size_t size) return 0; ret = ops->unmap(ops, iova, size); - if (ret && arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size)) + if (ret && arm_smmu_atc_inv_domain(smmu_domain, iova, size)) return 0; return ret; @@ -2521,6 +2566,194 @@ static void arm_smmu_put_resv_regions(struct device *dev, kfree(entry); } +static bool arm_smmu_dev_has_feature(struct device *dev, + enum iommu_dev_features feat) +{ + struct arm_smmu_master *master = dev_to_master(dev); + + if (!master) + return false; + + switch (feat) { + case IOMMU_DEV_FEAT_AUX: + return master->ssid_bits != 0; + default: + return false; + } +} + +static bool arm_smmu_dev_feature_enabled(struct device *dev, + enum iommu_dev_features feat) +{ + struct arm_smmu_master *master = dev_to_master(dev); + + if (!master) + return false; + + switch (feat) { + case IOMMU_DEV_FEAT_AUX: + return master->auxd_enabled; + default: + return false; + } +} + +static int arm_smmu_dev_enable_feature(struct device *dev, + enum iommu_dev_features feat) +{ + struct arm_smmu_master *master = dev_to_master(dev); + + if (!arm_smmu_dev_has_feature(dev, feat)) + return -ENODEV; + + if (arm_smmu_dev_feature_enabled(dev, feat)) + return -EBUSY; + + switch (feat) { + case IOMMU_DEV_FEAT_AUX: + master->auxd_enabled = true; + return 0; + default: + return -EINVAL; + } +} + +static int arm_smmu_dev_disable_feature(struct device *dev, + enum iommu_dev_features feat) +{ + struct arm_smmu_master *master = dev_to_master(dev); + + if (!arm_smmu_dev_feature_enabled(dev, feat)) + return -EINVAL; + + switch (feat) { + case IOMMU_DEV_FEAT_AUX: + /* TODO: check if aux domains are still attached? */ + master->auxd_enabled = false; + return 0; + default: + return -EINVAL; + } +} + +static int arm_smmu_aux_attach_dev(struct iommu_domain *domain, struct device *dev) +{ + int ret; + struct iommu_domain *parent_domain; + struct arm_smmu_domain *parent_smmu_domain; + struct arm_smmu_master *master = dev_to_master(dev); + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + + if (!arm_smmu_dev_feature_enabled(dev, IOMMU_DEV_FEAT_AUX)) + return -EINVAL; + + parent_domain = iommu_get_domain_for_dev(dev); + if (!parent_domain) + return -EINVAL; + parent_smmu_domain = to_smmu_domain(parent_domain); + + mutex_lock(&smmu_domain->init_mutex); + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1 || + parent_smmu_domain->stage != ARM_SMMU_DOMAIN_S1) { + ret = -EINVAL; + goto out_unlock; + } else if (smmu_domain->s1_cfg.tables) { + /* Already attached as a normal domain */ + dev_err(dev, "cannot attach domain in auxiliary mode\n"); + ret = -EINVAL; + goto out_unlock; + } else if (!smmu_domain->smmu) { + ioasid_t ssid = ioasid_alloc(&private_ioasid, 1, + (1UL << master->ssid_bits) - 1, + NULL); + if (ssid == INVALID_IOASID) { + ret = -EINVAL; + goto out_unlock; + } + smmu_domain->smmu = master->smmu; + smmu_domain->parent = parent_smmu_domain; + smmu_domain->ssid = ssid; + + ret = arm_smmu_domain_finalise(domain, master); + if (ret) { + smmu_domain->smmu = NULL; + smmu_domain->ssid = 0; + smmu_domain->parent = NULL; + ioasid_free(ssid); + goto out_unlock; + } + } else if (smmu_domain->parent != parent_smmu_domain) { + /* Additional restriction: an aux domain has a single parent */ + dev_err(dev, "cannot attach aux domain with different parent\n"); + ret = -EINVAL; + goto out_unlock; + } + + if (!smmu_domain->aux_nr_devs++) + arm_smmu_write_ctx_desc(parent_smmu_domain, smmu_domain->ssid, + &smmu_domain->s1_cfg.cd); + /* + * Note that all other devices attached to the parent domain can now + * access this context as well. + */ + +out_unlock: + mutex_unlock(&smmu_domain->init_mutex); + return ret; +} + +static void arm_smmu_aux_detach_dev(struct iommu_domain *domain, struct device *dev) +{ + struct iommu_domain *parent_domain; + struct arm_smmu_domain *parent_smmu_domain; + struct arm_smmu_master *master = dev_to_master(dev); + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + + if (!arm_smmu_dev_feature_enabled(dev, IOMMU_DEV_FEAT_AUX)) + return; + + parent_domain = iommu_get_domain_for_dev(dev); + if (!parent_domain) + return; + parent_smmu_domain = to_smmu_domain(parent_domain); + + mutex_lock(&smmu_domain->init_mutex); + if (!smmu_domain->aux_nr_devs) + goto out_unlock; + + if (!--smmu_domain->aux_nr_devs) { + arm_smmu_write_ctx_desc(parent_smmu_domain, smmu_domain->ssid, + NULL); + /* + * TLB doesn't need invalidation since accesses from the device + * can't use this domain's ASID once the CD is clear. + * + * Sadly that doesn't apply to ATCs, which are PASID tagged. + * Invalidate all other devices as well, because even though + * they weren't 'officially' attached to the auxiliary domain, + * they could have formed ATC entries. + */ + arm_smmu_atc_inv_domain(smmu_domain, 0, 0); + } else { + struct arm_smmu_cmdq_ent cmd; + + /* Invalidate only this device's ATC */ + if (master->ats_enabled) { + arm_smmu_atc_inv_to_cmd(smmu_domain->ssid, 0, 0, &cmd); + arm_smmu_atc_inv_master(master, &cmd); + } + } +out_unlock: + mutex_unlock(&smmu_domain->init_mutex); +} + +static int arm_smmu_aux_get_pasid(struct iommu_domain *domain, struct device *dev) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + + return smmu_domain->ssid ?: -EINVAL; +} + static struct iommu_ops arm_smmu_ops = { .capable = arm_smmu_capable, .domain_alloc = arm_smmu_domain_alloc, @@ -2539,6 +2772,13 @@ static struct iommu_ops arm_smmu_ops = { .of_xlate = arm_smmu_of_xlate, .get_resv_regions = arm_smmu_get_resv_regions, .put_resv_regions = arm_smmu_put_resv_regions, + .dev_has_feat = arm_smmu_dev_has_feature, + .dev_feat_enabled = arm_smmu_dev_feature_enabled, + .dev_enable_feat = arm_smmu_dev_enable_feature, + .dev_disable_feat = arm_smmu_dev_disable_feature, + .aux_attach_dev = arm_smmu_aux_attach_dev, + .aux_detach_dev = arm_smmu_aux_detach_dev, + .aux_get_pasid = arm_smmu_aux_get_pasid, .pgsize_bitmap = -1UL, /* Restricted during device attach */ }; @@ -3332,6 +3572,8 @@ static int arm_smmu_device_probe(struct platform_device *pdev) return -ENOMEM; } smmu->dev = dev; + /* Reserve ASID 0 for validity check */ + set_bit(0, smmu->asid_map); if (dev->of_node) { ret = arm_smmu_device_dt_probe(pdev, smmu); From patchwork Mon Jun 10 18:47:13 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10985305 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 532371398 for ; Mon, 10 Jun 2019 18:54:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 40DD128759 for ; Mon, 10 Jun 2019 18:54:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 342172876E; Mon, 10 Jun 2019 18:54:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id CDB1928759 for ; Mon, 10 Jun 2019 18:54:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GcvmuNvI9L37mfzLszNOg2oVw/RYYmksvWH8YOzba5E=; b=AkhJF8/bh4BtZV WNJYo2q3dJEFqcUbmLJ+mMrl3hxGbttorMfRPm15g/JKaqsIAO0+Dr4r1/4nc1O5UakWOApG3l3up Kn04bpHPYPmvaoVc5TMdpalmXhSQcfYoo/mHWqrhi3twcpQGbNwBOfecupneOtDw/l82EPjh5hJwm T8+vRyJZCTGGnbLV9fklxJxjmWpS5FQGmi04oG4zDSAGB2BBIm8GOB5wrj7+A1C5gzv1P27nKHRzr ZxPkql3uKyhIOhyWyBH2Ji+2Yp4h5Dt5RwyHdjhZm7xTNHk4PM+bjijbU61CeHt9xJkmAHta3lDhc sJW1jcp4j3xYras4JF5Q==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPQv-0003z3-So; Mon, 10 Jun 2019 18:54:17 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPOj-0002B5-Dt for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2019 18:52:03 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 854E4ED1; Mon, 10 Jun 2019 11:52:00 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.196.129]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 167603F246; Mon, 10 Jun 2019 11:51:58 -0700 (PDT) From: Jean-Philippe Brucker To: will.deacon@arm.com Subject: [PATCH 7/8] iommu/arm-smmu-v3: Improve add_device() error handling Date: Mon, 10 Jun 2019 19:47:13 +0100 Message-Id: <20190610184714.6786-8-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190610184714.6786-1-jean-philippe.brucker@arm.com> References: <20190610184714.6786-1-jean-philippe.brucker@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190610_115201_553531_01C8C2BC X-CRM114-Status: GOOD ( 13.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, jacob.jun.pan@linux.intel.com, joro@8bytes.org, linux-kernel@vger.kernel.org, eric.auger@redhat.com, iommu@lists.linux-foundation.org, robh+dt@kernel.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Let add_device() clean up behind itself. The iommu_bus_init() function does call remove_device() on error, but other sites (e.g. of_iommu) do not. Don't free level-2 stream tables because we'd have to track if we allocated each of them or if they are used by other endpoints. It's not worth the hassle since they are managed resources. Signed-off-by: Jean-Philippe Brucker Reviewed-by: Eric Auger --- drivers/iommu/arm-smmu-v3.c | 28 +++++++++++++++++++++------- 1 file changed, 21 insertions(+), 7 deletions(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 633d829f246f..972bfb80f964 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -2398,14 +2398,16 @@ static int arm_smmu_add_device(struct device *dev) for (i = 0; i < master->num_sids; i++) { u32 sid = master->sids[i]; - if (!arm_smmu_sid_in_range(smmu, sid)) - return -ERANGE; + if (!arm_smmu_sid_in_range(smmu, sid)) { + ret = -ERANGE; + goto err_free_master; + } /* Ensure l2 strtab is initialised */ if (smmu->features & ARM_SMMU_FEAT_2_LVL_STRTAB) { ret = arm_smmu_init_l2_strtab(smmu, sid); if (ret) - return ret; + goto err_free_master; } } @@ -2419,13 +2421,25 @@ static int arm_smmu_add_device(struct device *dev) if (!(smmu->features & ARM_SMMU_FEAT_2_LVL_CDTAB)) master->ssid_bits = min(master->ssid_bits, 10U); + ret = iommu_device_link(&smmu->iommu, dev); + if (ret) + goto err_free_master; + group = iommu_group_get_for_dev(dev); - if (!IS_ERR(group)) { - iommu_group_put(group); - iommu_device_link(&smmu->iommu, dev); + if (IS_ERR(group)) { + ret = PTR_ERR(group); + goto err_unlink; } - return PTR_ERR_OR_ZERO(group); + iommu_group_put(group); + return 0; + +err_unlink: + iommu_device_unlink(&smmu->iommu, dev); +err_free_master: + kfree(master); + fwspec->iommu_priv = NULL; + return ret; } static void arm_smmu_remove_device(struct device *dev) From patchwork Mon Jun 10 18:47:14 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jean-Philippe Brucker X-Patchwork-Id: 10985309 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D38A1398 for ; Mon, 10 Jun 2019 18:54:47 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 78A4228759 for ; Mon, 10 Jun 2019 18:54:47 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 698872876E; Mon, 10 Jun 2019 18:54:47 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 05F9328759 for ; Mon, 10 Jun 2019 18:54:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Bd8yt1Xxn/9RsBTlNXD2eSi+TOhj6eqLYRRNReEbFXo=; b=RobUuDDEsW5S6R 1ntBtw+WARicACewgcutpu392ydb1lDWGrsKxow21AH/ozfMOJluVF8oWqhwo0srZgFn6hbmVn1cp YYEMcnyvtF9ZrAH4ALTEfsXKTw1m3NSXjvAVJpwUgaf2ihrCK+viksSWfRDP325o8fecaPskR5Ui2 fcEpjXpvn5FbD4rCZcyYiwrV7uWajzxcobpDDamTjnMLYZ4XzIfZqxt4jehc1VsQPEGqQzoj8YjyQ p0VejNJK23gN4OSoLObvkWSo2fgAIJPu6Oz1E3EODl8afowxycHsXI4q5TLqAfm4DfXda4Vx7DF0P jH8CXM1LLryWYJWzFF9A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPRJ-0004Js-EN; Mon, 10 Jun 2019 18:54:41 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.92 #3 (Red Hat Linux)) id 1haPOk-0002Bw-R0 for linux-arm-kernel@lists.infradead.org; Mon, 10 Jun 2019 18:52:04 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 334D6FEC; Mon, 10 Jun 2019 11:52:02 -0700 (PDT) Received: from ostrya.cambridge.arm.com (ostrya.cambridge.arm.com [10.1.196.129]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id B7F193F246; Mon, 10 Jun 2019 11:52:00 -0700 (PDT) From: Jean-Philippe Brucker To: will.deacon@arm.com Subject: [PATCH 8/8] iommu/arm-smmu-v3: Add support for PCI PASID Date: Mon, 10 Jun 2019 19:47:14 +0100 Message-Id: <20190610184714.6786-9-jean-philippe.brucker@arm.com> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190610184714.6786-1-jean-philippe.brucker@arm.com> References: <20190610184714.6786-1-jean-philippe.brucker@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20190610_115203_079261_FE43B787 X-CRM114-Status: GOOD ( 15.14 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: mark.rutland@arm.com, devicetree@vger.kernel.org, jacob.jun.pan@linux.intel.com, joro@8bytes.org, linux-kernel@vger.kernel.org, eric.auger@redhat.com, iommu@lists.linux-foundation.org, robh+dt@kernel.org, robin.murphy@arm.com, linux-arm-kernel@lists.infradead.org Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Enable PASID for PCI devices that support it. Since the SSID tables are allocated by arm_smmu_attach_dev(), PASID has to be enabled early enough. arm_smmu_dev_feature_enable() would be too late, since by that time the main DMA domain has already been attached. Do it in add_device() instead. Signed-off-by: Jean-Philippe Brucker --- drivers/iommu/arm-smmu-v3.c | 51 ++++++++++++++++++++++++++++++++++++- 1 file changed, 50 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu-v3.c b/drivers/iommu/arm-smmu-v3.c index 972bfb80f964..a8a516d9ff10 100644 --- a/drivers/iommu/arm-smmu-v3.c +++ b/drivers/iommu/arm-smmu-v3.c @@ -2197,6 +2197,49 @@ static void arm_smmu_disable_ats(struct arm_smmu_master *master) master->ats_enabled = false; } +static int arm_smmu_enable_pasid(struct arm_smmu_master *master) +{ + int ret; + int features; + int num_pasids; + struct pci_dev *pdev; + + if (!dev_is_pci(master->dev)) + return -ENOSYS; + + pdev = to_pci_dev(master->dev); + + features = pci_pasid_features(pdev); + if (features < 0) + return -ENOSYS; + + num_pasids = pci_max_pasids(pdev); + if (num_pasids <= 0) + return -ENOSYS; + + ret = pci_enable_pasid(pdev, features); + if (!ret) + master->ssid_bits = min_t(u8, ilog2(num_pasids), + master->smmu->ssid_bits); + return ret; +} + +static void arm_smmu_disable_pasid(struct arm_smmu_master *master) +{ + struct pci_dev *pdev; + + if (!dev_is_pci(master->dev)) + return; + + pdev = to_pci_dev(master->dev); + + if (!pdev->pasid_enabled) + return; + + pci_disable_pasid(pdev); + master->ssid_bits = 0; +} + static void arm_smmu_detach_dev(struct arm_smmu_master *master) { unsigned long flags; @@ -2413,6 +2456,9 @@ static int arm_smmu_add_device(struct device *dev) master->ssid_bits = min(smmu->ssid_bits, fwspec->num_pasid_bits); + /* Note that PASID must be enabled before, and disabled after ATS */ + arm_smmu_enable_pasid(master); + /* * If the SMMU doesn't support 2-stage CD, limit the linear * tables to a reasonable number of contexts, let's say @@ -2423,7 +2469,7 @@ static int arm_smmu_add_device(struct device *dev) ret = iommu_device_link(&smmu->iommu, dev); if (ret) - goto err_free_master; + goto err_disable_pasid; group = iommu_group_get_for_dev(dev); if (IS_ERR(group)) { @@ -2436,6 +2482,8 @@ static int arm_smmu_add_device(struct device *dev) err_unlink: iommu_device_unlink(&smmu->iommu, dev); +err_disable_pasid: + arm_smmu_disable_pasid(master); err_free_master: kfree(master); fwspec->iommu_priv = NULL; @@ -2456,6 +2504,7 @@ static void arm_smmu_remove_device(struct device *dev) arm_smmu_detach_dev(master); iommu_group_remove_device(dev); iommu_device_unlink(&smmu->iommu, dev); + arm_smmu_disable_pasid(master); kfree(master); iommu_fwspec_free(dev); }