From patchwork Fri May 18 21:34:49 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 10412307 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 341AC602CB for ; Fri, 18 May 2018 21:36:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 23C7B28980 for ; Fri, 18 May 2018 21:36:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1849828AF6; Fri, 18 May 2018 21:36:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,MAILING_LIST_MULTI autolearn=unavailable version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 71A1128980 for ; Fri, 18 May 2018 21:36:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:Cc:List-Subscribe: List-Help:List-Post:List-Archive:List-Unsubscribe:List-Id:References: In-Reply-To:Message-Id:Date:Subject:To:From:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=TjeAd2T4ClUb/MUSESc0o0WpRHvUi/SXtBWt3c19KzM=; b=OORnm4uzxuV3U5qu6pGcCPUwHt X24aPgn3vbeFd0KWsVzr+LZV9gb8Ic75e+QXogUOyWNXtdSCK8XxcISbL5FtuC5ELF/zZCf//cL0a k10gqX9GH7VivDM5c6gTaK0YmQoEjUx4jGa4nJ62MqhJnh10eigKRggcPomFB4mAW++PaX61PBaAN arCH1yFgtSW+6Y/3qRTJDV0VQzKaxnQhU2FNCoCempPXa1YxAOeAAJvU+Unpijhe7BQC4eFHOWFNZ 6M1iMjWFBS6q1b7bstNl4VqysdrMDtn8sQBjdoI3wyyM9Db8C4Em9TtWoxR5sLORQ/yUK5ZY9cRGX TYhEXlTg==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1fJn2u-0000tN-Cd; Fri, 18 May 2018 21:36:16 +0000 Received: from smtp.codeaurora.org ([198.145.29.96]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1fJn1z-000056-4R for linux-arm-kernel@lists.infradead.org; Fri, 18 May 2018 21:35:22 +0000 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id C10DF60C55; Fri, 18 May 2018 21:35:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1526679312; bh=bD8oGOvRCOSe5ngqqgtZ1qiYfUodAesHIA7DgIP6nMk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=d50E4W8PKV1geJ+/2+54KHJPtJKqtXIpKvSxsMMLE3gVond+byD6/7thx4tg0SjM/ PkhVJzIgxwtrirdYWvGnhKWyVAxx08CLUuRpXQeR6dHrmLz43QppxtGBZcD1m1djTw OjoKHwki2Hqi2rxTj4IrhIjlg7nAsCRX3009AM+U= Received: from jcrouse-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id E20346055C; Fri, 18 May 2018 21:35:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1526679309; bh=bD8oGOvRCOSe5ngqqgtZ1qiYfUodAesHIA7DgIP6nMk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Yl02mIXexaNKggpQD5eX152LrFlToxtwpktZ+ftq9UmoHuN7YaX80gONpGNVtxUan 7VaQJPMplVE/lD530zGw591ufrbKd0I8obFuSR27iJq0ZsN2Vp73zsXVtsNqMI+zQ0 zkmsGPKI5MKT88u4x92q/bA1lpOhWO6iulrJ6HPg= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org E20346055C Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Subject: [PATCH 05/16] iommu: arm-smmu: Add support for private PASIDs Date: Fri, 18 May 2018 15:34:49 -0600 Message-Id: <20180518213500.31595-6-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180518213500.31595-1-jcrouse@codeaurora.org> References: <20180518213500.31595-1-jcrouse@codeaurora.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180518_143519_283011_3235BFEF X-CRM114-Status: GOOD ( 16.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, tfiga@chromium.org, iommu@lists.linux-foundation.org, vivek.gautam@codeaurora.org, linux-arm-kernel@lists.infradead.org MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP Add support for allocating and populating pagetables indexed by private PASIDs. Each new PASID is allocated a pagetable with the same parameters and format as the parent domain. Signed-off-by: Jordan Crouse --- drivers/iommu/arm-smmu.c | 154 +++++++++++++++++++++++++++++++++++-- drivers/iommu/io-pgtable.h | 10 ++- 2 files changed, 155 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index d459909877c3..5c7c135bbb44 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -201,7 +201,6 @@ struct arm_smmu_device { unsigned long va_size; unsigned long ipa_size; unsigned long pa_size; - unsigned long ubs_size; unsigned long pgsize_bitmap; u32 num_global_irqs; @@ -252,6 +251,9 @@ struct arm_smmu_domain { spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ u32 attributes; struct iommu_domain domain; + + spinlock_t pasid_lock; + struct list_head pasid_list; }; struct arm_smmu_option_prop { @@ -259,6 +261,144 @@ struct arm_smmu_option_prop { const char *prop; }; +static struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) +{ + return container_of(dom, struct arm_smmu_domain, domain); +} + +struct arm_smmu_pasid { + struct iommu_domain *domain; + struct io_pgtable_ops *pgtbl_ops; + struct list_head node; + int pasid; +}; + +struct arm_smmu_pasid *arm_smmu_get_pasid(struct arm_smmu_domain *smmu_domain, + int pasid) +{ + struct arm_smmu_pasid *node, *obj = NULL; + + spin_lock(&smmu_domain->pasid_lock); + list_for_each_entry(node, &smmu_domain->pasid_list, node) { + if (node->pasid == pasid) { + obj = node; + break; + } + } + spin_unlock(&smmu_domain->pasid_lock); + + return obj; +} + +static void arm_smmu_mm_detach(struct iommu_domain *domain, struct device *dev, + struct io_mm *io_mm, bool unused) +{ + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_pasid *node, *obj = NULL; + + spin_lock(&smmu_domain->pasid_lock); + list_for_each_entry(node, &smmu_domain->pasid_list, node) { + if (node->pasid == io_mm->pasid) { + obj = node; + list_del(&obj->node); + break; + } + } + spin_unlock(&smmu_domain->pasid_lock); + + if (obj) + free_io_pgtable_ops(obj->pgtbl_ops); + + kfree(obj); +} + +static size_t arm_smmu_sva_unmap(struct iommu_domain *domain, int pasid, + unsigned long iova, size_t size) +{ + struct arm_smmu_pasid *obj = + arm_smmu_get_pasid(to_smmu_domain(domain), pasid); + + if (!obj) + return -ENODEV; + + return obj->pgtbl_ops->unmap(obj->pgtbl_ops, iova, size); +} + + +static int arm_smmu_sva_map(struct iommu_domain *domain, int pasid, + unsigned long iova, phys_addr_t paddr, size_t size, int prot) +{ + struct arm_smmu_pasid *obj = + arm_smmu_get_pasid(to_smmu_domain(domain), pasid); + + if (!obj) + return -ENODEV; + + return obj->pgtbl_ops->map(obj->pgtbl_ops, iova, paddr, size, prot); +} + +static int arm_smmu_mm_attach(struct iommu_domain *domain, struct device *dev, + struct io_mm *io_mm, bool unused) +{ + struct arm_smmu_pasid *obj; + struct io_pgtable_cfg pgtbl_cfg; + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain); + struct arm_smmu_device *smmu = smmu_domain->smmu; + enum io_pgtable_fmt fmt; + unsigned long ias, oas; + + /* Only allow private pasids */ + if (io_mm->type != IO_TYPE_PRIVATE || io_mm->mm) + return -ENODEV; + + /* Only allow pasid backed tables to be created on S1 domains */ + if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1) + return -ENODEV; + + obj = kzalloc(sizeof(*obj), GFP_KERNEL); + if (!obj) + return -ENOMEM; + + /* Get the same exact format as the parent domain */ + ias = smmu->va_size; + oas = smmu->ipa_size; + + if (smmu_domain->cfg.fmt == ARM_SMMU_CTX_FMT_AARCH64) + fmt = ARM_64_LPAE_S1; + else if (smmu_domain->cfg.fmt == ARM_SMMU_CTX_FMT_AARCH32_L) { + fmt = ARM_32_LPAE_S1; + ias = min(ias, 32UL); + oas = min(oas, 40UL); + } else { + fmt = ARM_V7S; + ias = min(ias, 32UL); + oas = min(oas, 32UL); + } + + pgtbl_cfg = (struct io_pgtable_cfg) { + .pgsize_bitmap = smmu->pgsize_bitmap, + .ias = ias, + .oas = oas, + .tlb = NULL, + .iommu_dev = smmu->dev + }; + + obj->pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain); + if (!obj->pgtbl_ops) { + kfree(obj); + return -ENOMEM; + } + + obj->domain = domain; + obj->pasid = io_mm->pasid; + + spin_lock(&smmu_domain->pasid_lock); + list_add_tail(&obj->node, &smmu_domain->pasid_list); + spin_unlock(&smmu_domain->pasid_lock); + + return 0; +} + static atomic_t cavium_smmu_context_count = ATOMIC_INIT(0); static bool using_legacy_binding, using_generic_binding; @@ -268,11 +408,6 @@ static struct arm_smmu_option_prop arm_smmu_options[] = { { 0, NULL}, }; -static struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom) -{ - return container_of(dom, struct arm_smmu_domain, domain); -} - static void parse_driver_options(struct arm_smmu_device *smmu) { int i = 0; @@ -1055,6 +1190,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type) mutex_init(&smmu_domain->init_mutex); spin_lock_init(&smmu_domain->cb_lock); + spin_lock_init(&smmu_domain->pasid_lock); + INIT_LIST_HEAD(&smmu_domain->pasid_list); + return &smmu_domain->domain; } @@ -1694,6 +1832,10 @@ static struct iommu_ops arm_smmu_ops = { .of_xlate = arm_smmu_of_xlate, .get_resv_regions = arm_smmu_get_resv_regions, .put_resv_regions = arm_smmu_put_resv_regions, + .mm_attach = arm_smmu_mm_attach, + .sva_map = arm_smmu_sva_map, + .sva_unmap = arm_smmu_sva_unmap, + .mm_detach = arm_smmu_mm_detach, .pgsize_bitmap = -1UL, /* Restricted during device attach */ }; diff --git a/drivers/iommu/io-pgtable.h b/drivers/iommu/io-pgtable.h index fd9f0fc4eb60..69fcee763446 100644 --- a/drivers/iommu/io-pgtable.h +++ b/drivers/iommu/io-pgtable.h @@ -173,18 +173,22 @@ struct io_pgtable { static inline void io_pgtable_tlb_flush_all(struct io_pgtable *iop) { - iop->cfg.tlb->tlb_flush_all(iop->cookie); + if (iop->cfg.tlb) + iop->cfg.tlb->tlb_flush_all(iop->cookie); } static inline void io_pgtable_tlb_add_flush(struct io_pgtable *iop, unsigned long iova, size_t size, size_t granule, bool leaf) { - iop->cfg.tlb->tlb_add_flush(iova, size, granule, leaf, iop->cookie); + if (iop->cfg.tlb) + iop->cfg.tlb->tlb_add_flush(iova, size, granule, leaf, + iop->cookie); } static inline void io_pgtable_tlb_sync(struct io_pgtable *iop) { - iop->cfg.tlb->tlb_sync(iop->cookie); + if (iop->cfg.tlb) + iop->cfg.tlb->tlb_sync(iop->cookie); } /**