From patchwork Mon Jul 8 18:10:16 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 11035557 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C3F8A14DB for ; Mon, 8 Jul 2019 18:10:28 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B4E58285E0 for ; Mon, 8 Jul 2019 18:10:28 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A90C828610; Mon, 8 Jul 2019 18:10:28 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6203B285E0 for ; Mon, 8 Jul 2019 18:10:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388514AbfGHSK1 (ORCPT ); Mon, 8 Jul 2019 14:10:27 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:53762 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729025AbfGHSK0 (ORCPT ); Mon, 8 Jul 2019 14:10:26 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 6C0CE60FEE; Mon, 8 Jul 2019 18:10:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1562609425; bh=I0b5kpNA6visZ/gWW+mc9BO8uDwDyEu2oJKHrjDh8wg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TIPXG+vU4zIVWPgUws+qo8sFJCJmxQCzmjtMdPdf62G9rnfZca+VhPgMXeolUVAke TICjx5uUZm+lLkQdUrrKlEw6fLUDyEJd2CyQCk4U8wGavaef+Rjh/M5ebK5uLLuo+p kh50cQI0znRp1PkZmfXZOwj5zJw9fzbugsz5Ad+s= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 4679B60E41; Mon, 8 Jul 2019 18:10:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1562609425; bh=I0b5kpNA6visZ/gWW+mc9BO8uDwDyEu2oJKHrjDh8wg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=TIPXG+vU4zIVWPgUws+qo8sFJCJmxQCzmjtMdPdf62G9rnfZca+VhPgMXeolUVAke TICjx5uUZm+lLkQdUrrKlEw6fLUDyEJd2CyQCk4U8wGavaef+Rjh/M5ebK5uLLuo+p kh50cQI0znRp1PkZmfXZOwj5zJw9fzbugsz5Ad+s= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 4679B60E41 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, dianders@chromimum.org, hoegsberg@google.com, baolu.lu@linux.intel.com, iommu@lists.linux-foundation.org, Joerg Roedel , linux-kernel@vger.kernel.org Subject: [PATCH v2 1/3] iommu: Add DOMAIN_ATTR_SPLIT_TABLES Date: Mon, 8 Jul 2019 12:10:16 -0600 Message-Id: <1562609418-25446-2-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1562609418-25446-1-git-send-email-jcrouse@codeaurora.org> References: <1562609418-25446-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a new domain attribute to enable split pagetable support for devices devices that support it. Signed-off-by: Jordan Crouse --- include/linux/iommu.h | 1 + 1 file changed, 1 insertion(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index fdc355c..b06db6c 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -125,6 +125,7 @@ enum iommu_attr { DOMAIN_ATTR_FSL_PAMUV1, DOMAIN_ATTR_NESTING, /* two stages of translation */ DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE, + DOMAIN_ATTR_SPLIT_TABLES, DOMAIN_ATTR_MAX, }; From patchwork Mon Jul 8 18:10:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 11035563 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 081FB14C0 for ; Mon, 8 Jul 2019 18:10:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EB615280B0 for ; Mon, 8 Jul 2019 18:10:44 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id DE8EA28396; Mon, 8 Jul 2019 18:10:44 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id CF738280B0 for ; Mon, 8 Jul 2019 18:10:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390973AbfGHSKb (ORCPT ); Mon, 8 Jul 2019 14:10:31 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:54118 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729025AbfGHSKa (ORCPT ); Mon, 8 Jul 2019 14:10:30 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id 7E76C61112; Mon, 8 Jul 2019 18:10:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1562609429; bh=TS0wlrofdZ9G9964YwYXdJXHhjE2yp06YT1ABAcNFQk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O049BbrJ2w7Z6GVXi+GKsHHhJWkKMyrnrxdzYXD72smjkNUcOtcVN8vqAlL56Hs4Q RYf3ukXUkP05Aufg0D6X6BQlrc4rkRii1HFwH6WdvCrvYsEFqYvI/nrHuv/ahFu9Nr wZGFGfEXr/sJr1VZmKSTPsZJAAAunvy8Viezmzxc= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 5C54D6110F; Mon, 8 Jul 2019 18:10:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1562609426; bh=TS0wlrofdZ9G9964YwYXdJXHhjE2yp06YT1ABAcNFQk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=i5mymvxZOwm5uC6VbezL6pooF1W+9Vvhc44xFXQ34++bURknSuj1+UDh0sbJNAqkR MIj4qfmMF6TfoFEpg26fWsZn7YSeH/Sh8Ko3McpfQllS5iWcF9lxynQ95jR9J1tac3 75X0ozWl82a0alEKxpCJ3atLDgzZ+Jo0fp6yjgLk= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 5C54D6110F Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, dianders@chromimum.org, hoegsberg@google.com, baolu.lu@linux.intel.com, Will Deacon , Zhen Lei , Rob Herring , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Robin Murphy , Joerg Roedel , linux-arm-kernel@lists.infradead.org Subject: [PATCH v2 2/3] iommu/io-pgtable-arm: Add support for AARCH64 split pagetables Date: Mon, 8 Jul 2019 12:10:17 -0600 Message-Id: <1562609418-25446-3-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1562609418-25446-1-git-send-email-jcrouse@codeaurora.org> References: <1562609418-25446-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Add a new sub-format ARM_64_LPAE_SPLIT_S1 to create and set up split pagetables (TTBR0 and TTBR1). The initialization function sets up the correct va_size and sign extension bits and programs the TCR registers. Split pagetable formats use their own own map/unmap wrappers to ensure that the correct pagetable is selected based on the incoming iova but most of the heavy lifting is common to the other formats. Signed-off-by: Jordan Crouse --- drivers/iommu/io-pgtable-arm.c | 261 +++++++++++++++++++++++++++++++++++++---- drivers/iommu/io-pgtable.c | 1 + include/linux/io-pgtable.h | 2 + 3 files changed, 240 insertions(+), 24 deletions(-) diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c index 161a7d56..aec35e5 100644 --- a/drivers/iommu/io-pgtable-arm.c +++ b/drivers/iommu/io-pgtable-arm.c @@ -118,7 +118,12 @@ #define ARM_LPAE_TCR_TG0_64K (1 << 14) #define ARM_LPAE_TCR_TG0_16K (2 << 14) +#define ARM_LPAE_TCR_TG1_4K (0 << 30) +#define ARM_LPAE_TCR_TG1_64K (1 << 30) +#define ARM_LPAE_TCR_TG1_16K (2 << 30) + #define ARM_LPAE_TCR_SH0_SHIFT 12 +#define ARM_LPAE_TCR_SH1_SHIFT 28 #define ARM_LPAE_TCR_SH0_MASK 0x3 #define ARM_LPAE_TCR_SH_NS 0 #define ARM_LPAE_TCR_SH_OS 2 @@ -126,6 +131,8 @@ #define ARM_LPAE_TCR_ORGN0_SHIFT 10 #define ARM_LPAE_TCR_IRGN0_SHIFT 8 +#define ARM_LPAE_TCR_ORGN1_SHIFT 26 +#define ARM_LPAE_TCR_IRGN1_SHIFT 24 #define ARM_LPAE_TCR_RGN_MASK 0x3 #define ARM_LPAE_TCR_RGN_NC 0 #define ARM_LPAE_TCR_RGN_WBWA 1 @@ -136,6 +143,7 @@ #define ARM_LPAE_TCR_SL0_MASK 0x3 #define ARM_LPAE_TCR_T0SZ_SHIFT 0 +#define ARM_LPAE_TCR_T1SZ_SHIFT 16 #define ARM_LPAE_TCR_SZ_MASK 0xf #define ARM_LPAE_TCR_PS_SHIFT 16 @@ -152,6 +160,14 @@ #define ARM_LPAE_TCR_PS_48_BIT 0x5ULL #define ARM_LPAE_TCR_PS_52_BIT 0x6ULL +#define ARM_LPAE_TCR_SEP_SHIFT 47 +#define ARM_LPAE_TCR_SEP_31 (0x0ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_35 (0x1ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_39 (0x2ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_41 (0x3ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_43 (0x4ULL << ARM_LPAE_TCR_SEP_SHIFT) +#define ARM_LPAE_TCR_SEP_UPSTREAM (0x7ULL << ARM_LPAE_TCR_SEP_SHIFT) + #define ARM_LPAE_MAIR_ATTR_SHIFT(n) ((n) << 3) #define ARM_LPAE_MAIR_ATTR_MASK 0xff #define ARM_LPAE_MAIR_ATTR_DEVICE 0x04 @@ -179,11 +195,12 @@ struct arm_lpae_io_pgtable { struct io_pgtable iop; int levels; + u32 sep; size_t pgd_size; unsigned long pg_shift; unsigned long bits_per_level; - void *pgd; + void *pgd[2]; }; typedef u64 arm_lpae_iopte; @@ -426,7 +443,8 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, arm_lpae_iopte pte; if (data->iop.fmt == ARM_64_LPAE_S1 || - data->iop.fmt == ARM_32_LPAE_S1) { + data->iop.fmt == ARM_32_LPAE_S1 || + data->iop.fmt == ARM_64_LPAE_SPLIT_S1) { pte = ARM_LPAE_PTE_nG; if (!(prot & IOMMU_WRITE) && (prot & IOMMU_READ)) pte |= ARM_LPAE_PTE_AP_RDONLY; @@ -470,11 +488,10 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data, return pte; } -static int arm_lpae_map(struct io_pgtable_ops *ops, unsigned long iova, - phys_addr_t paddr, size_t size, int iommu_prot) +static int _arm_lpae_map(struct arm_lpae_io_pgtable *data, unsigned long iova, + phys_addr_t paddr, size_t size, int iommu_prot, + arm_lpae_iopte *ptep) { - struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); - arm_lpae_iopte *ptep = data->pgd; int ret, lvl = ARM_LPAE_START_LVL(data); arm_lpae_iopte prot; @@ -497,12 +514,39 @@ static int arm_lpae_map(struct io_pgtable_ops *ops, unsigned long iova, return ret; } +static int arm_lpae_split_map(struct io_pgtable_ops *ops, unsigned long iova, + phys_addr_t paddr, size_t size, int iommu_prot) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + unsigned long mask = 1UL << data->sep; + arm_lpae_iopte *ptep; + + if (iova & mask) { + ptep = data->pgd[1]; + iova &= (mask - 1); + } else + ptep = data->pgd[0]; + + return _arm_lpae_map(data, iova, paddr, size, iommu_prot, ptep); +} + +static int arm_lpae_map(struct io_pgtable_ops *ops, unsigned long iova, + phys_addr_t paddr, size_t size, int iommu_prot) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + + return _arm_lpae_map(data, iova, paddr, size, iommu_prot, data->pgd[0]); +} + static void __arm_lpae_free_pgtable(struct arm_lpae_io_pgtable *data, int lvl, arm_lpae_iopte *ptep) { arm_lpae_iopte *start, *end; unsigned long table_size; + if (!ptep) + return; + if (lvl == ARM_LPAE_START_LVL(data)) table_size = data->pgd_size; else @@ -532,7 +576,8 @@ static void arm_lpae_free_pgtable(struct io_pgtable *iop) { struct arm_lpae_io_pgtable *data = io_pgtable_to_data(iop); - __arm_lpae_free_pgtable(data, ARM_LPAE_START_LVL(data), data->pgd); + __arm_lpae_free_pgtable(data, ARM_LPAE_START_LVL(data), data->pgd[0]); + __arm_lpae_free_pgtable(data, ARM_LPAE_START_LVL(data), data->pgd[1]); kfree(data); } @@ -643,11 +688,28 @@ static size_t __arm_lpae_unmap(struct arm_lpae_io_pgtable *data, return __arm_lpae_unmap(data, iova, size, lvl + 1, ptep); } +static size_t arm_lpae_split_unmap(struct io_pgtable_ops *ops, + unsigned long iova, size_t size) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + unsigned long mask = 1UL << data->sep; + arm_lpae_iopte *ptep; + int lvl = ARM_LPAE_START_LVL(data); + + if (iova & mask) { + ptep = data->pgd[1]; + iova &= (mask - 1); + } else + ptep = data->pgd[0]; + + return __arm_lpae_unmap(data, iova, size, lvl, ptep); +} + static size_t arm_lpae_unmap(struct io_pgtable_ops *ops, unsigned long iova, size_t size) { struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); - arm_lpae_iopte *ptep = data->pgd; + arm_lpae_iopte *ptep = data->pgd[0]; int lvl = ARM_LPAE_START_LVL(data); if (WARN_ON(iova >= (1ULL << data->iop.cfg.ias))) @@ -656,11 +718,11 @@ static size_t arm_lpae_unmap(struct io_pgtable_ops *ops, unsigned long iova, return __arm_lpae_unmap(data, iova, size, lvl, ptep); } -static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops, - unsigned long iova) +static phys_addr_t _arm_lpae_iova_to_phys(struct arm_lpae_io_pgtable *data, + unsigned long iova, + arm_lpae_iopte *ptep) { - struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); - arm_lpae_iopte pte, *ptep = data->pgd; + arm_lpae_iopte pte; int lvl = ARM_LPAE_START_LVL(data); do { @@ -692,6 +754,31 @@ static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops, return iopte_to_paddr(pte, data) | iova; } + +static phys_addr_t arm_lpae_split_iova_to_phys(struct io_pgtable_ops *ops, + unsigned long iova) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + unsigned long mask = 1UL << data->sep; + arm_lpae_iopte *ptep; + + if (iova & mask) { + ptep = data->pgd[1]; + iova &= (mask - 1); + } else + ptep = data->pgd[0]; + + return _arm_lpae_iova_to_phys(data, iova, ptep); +} + +static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops, + unsigned long iova) +{ + struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops); + + return _arm_lpae_iova_to_phys(data, iova, data->pgd[0]); +} + static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg) { unsigned long granule, page_sizes; @@ -771,6 +858,9 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg) pgd_bits = va_bits - (data->bits_per_level * (data->levels - 1)); data->pgd_size = 1UL << (pgd_bits + ilog2(sizeof(arm_lpae_iopte))); + data->pgd[0] = NULL; + data->pgd[1] = NULL; + data->iop.ops = (struct io_pgtable_ops) { .map = arm_lpae_map, .unmap = arm_lpae_unmap, @@ -780,8 +870,8 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg) return data; } -static struct io_pgtable * -arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) +static struct arm_lpae_io_pgtable * +_arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg) { u64 reg; struct arm_lpae_io_pgtable *data; @@ -845,8 +935,6 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) reg |= (64ULL - cfg->ias) << ARM_LPAE_TCR_T0SZ_SHIFT; - /* Disable speculative walks through TTBR1 */ - reg |= ARM_LPAE_TCR_EPD1; cfg->arm_lpae_s1_cfg.tcr = reg; /* MAIRs */ @@ -863,24 +951,143 @@ arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) cfg->arm_lpae_s1_cfg.mair[1] = 0; /* Looking good; allocate a pgd */ - data->pgd = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); - if (!data->pgd) + data->pgd[0] = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); + if (!data->pgd[0]) goto out_free_data; /* Ensure the empty pgd is visible before any actual TTBR write */ wmb(); /* TTBRs */ - cfg->arm_lpae_s1_cfg.ttbr[0] = virt_to_phys(data->pgd); + cfg->arm_lpae_s1_cfg.ttbr[0] = virt_to_phys(data->pgd[0]); cfg->arm_lpae_s1_cfg.ttbr[1] = 0; + return data; + +out_free_data: + kfree(data); + return NULL; +} + + +/* Allocate split pagetables */ +static struct io_pgtable * +arm_64_lpae_alloc_pgtable_split_s1(struct io_pgtable_cfg *cfg, void *cookie) +{ + u64 reg; + struct arm_lpae_io_pgtable *data; + u32 sep; + + /* Figure out what the sign extension bit should be */ + switch (cfg->ias) { + case 32: + case 36: + case 40: + case 42: + case 44: + sep = cfg->ias - 1; + /* Adjust the address size to account for the extension bit */ + cfg->ias--; + break; + case 48: + /* IAS of 48 is a special case, it has a dedicated bit */ + sep = 48; + break; + default: + return NULL; + } + + data = _arm_64_lpae_alloc_pgtable_s1(cfg); + if (!data) + return NULL; + + /* Add the TTBR1 settings */ + reg = cfg->arm_lpae_s1_cfg.tcr; + + /* TCR */ + reg |= (ARM_LPAE_TCR_SH_IS << ARM_LPAE_TCR_SH1_SHIFT) | + (ARM_LPAE_TCR_RGN_WBWA << ARM_LPAE_TCR_IRGN1_SHIFT) | + (ARM_LPAE_TCR_RGN_WBWA << ARM_LPAE_TCR_ORGN1_SHIFT); + + switch (ARM_LPAE_GRANULE(data)) { + case SZ_4K: + reg |= ARM_LPAE_TCR_TG1_4K; + break; + case SZ_16K: + reg |= ARM_LPAE_TCR_TG1_16K; + break; + case SZ_64K: + reg |= ARM_LPAE_TCR_TG1_64K; + break; + } + + reg |= (64ULL - cfg->ias) << ARM_LPAE_TCR_T1SZ_SHIFT; + + switch (sep) { + case 31: + reg |= ARM_LPAE_TCR_SEP_31; + break; + case 35: + reg |= ARM_LPAE_TCR_SEP_35; + break; + case 39: + reg |= ARM_LPAE_TCR_SEP_39; + break; + case 41: + reg |= ARM_LPAE_TCR_SEP_41; + break; + case 43: + reg |= ARM_LPAE_TCR_SEP_43; + break; + case 48: + reg |= ARM_LPAE_TCR_SEP_UPSTREAM; + break; + } + + cfg->arm_lpae_s1_cfg.tcr = reg; + + /* Allocate the TTBR1 pagetable */ + data->pgd[1] = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); + if (!data->pgd[1]) + goto out_free_data; + + /* Override the data ops with split table specific ops */ + data->iop.ops = (struct io_pgtable_ops) { + .map = arm_lpae_split_map, + .unmap = arm_lpae_split_unmap, + .iova_to_phys = arm_lpae_split_iova_to_phys, + }; + + /* + * remember the sign extension bit, we'll need it later to figure out + * which pagetable to use + */ + data->sep = sep; + + /* Ensure the empty pgd is visible before any actual TTBR write */ + wmb(); + + cfg->arm_lpae_s1_cfg.ttbr[1] = virt_to_phys(data->pgd[1]); return &data->iop; out_free_data: + __arm_lpae_free_pages(data->pgd[0], data->pgd_size, cfg); kfree(data); return NULL; } static struct io_pgtable * +arm_64_lpae_alloc_pgtable_s1(struct io_pgtable_cfg *cfg, void *cookie) +{ + struct arm_lpae_io_pgtable *data; + + data = _arm_64_lpae_alloc_pgtable_s1(cfg); + if (!data) + return NULL; + + return &data->iop; +} + +static struct io_pgtable * arm_64_lpae_alloc_pgtable_s2(struct io_pgtable_cfg *cfg, void *cookie) { u64 reg, sl; @@ -960,15 +1167,15 @@ arm_64_lpae_alloc_pgtable_s2(struct io_pgtable_cfg *cfg, void *cookie) cfg->arm_lpae_s2_cfg.vtcr = reg; /* Allocate pgd pages */ - data->pgd = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); - if (!data->pgd) + data->pgd[0] = __arm_lpae_alloc_pages(data->pgd_size, GFP_KERNEL, cfg); + if (!data->pgd[0]) goto out_free_data; /* Ensure the empty pgd is visible before any actual TTBR write */ wmb(); /* VTTBR */ - cfg->arm_lpae_s2_cfg.vttbr = virt_to_phys(data->pgd); + cfg->arm_lpae_s2_cfg.vttbr = virt_to_phys(data->pgd[0]); return &data->iop; out_free_data: @@ -1041,6 +1248,11 @@ struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns = { .free = arm_lpae_free_pgtable, }; +struct io_pgtable_init_fns io_pgtable_arm_64_lpae_split_s1_init_fns = { + .alloc = arm_64_lpae_alloc_pgtable_split_s1, + .free = arm_lpae_free_pgtable, +}; + struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns = { .alloc = arm_64_lpae_alloc_pgtable_s2, .free = arm_lpae_free_pgtable, @@ -1095,9 +1307,9 @@ static void __init arm_lpae_dump_ops(struct io_pgtable_ops *ops) pr_err("cfg: pgsize_bitmap 0x%lx, ias %u-bit\n", cfg->pgsize_bitmap, cfg->ias); - pr_err("data: %d levels, 0x%zx pgd_size, %lu pg_shift, %lu bits_per_level, pgd @ %p\n", + pr_err("data: %d levels, 0x%zx pgd_size, %lu pg_shift, %lu bits_per_level, pgd @ %p %p\n", data->levels, data->pgd_size, data->pg_shift, - data->bits_per_level, data->pgd); + data->bits_per_level, data->pgd[0], data->pgd[1]); } #define __FAIL(ops, i) ({ \ @@ -1112,6 +1324,7 @@ static int __init arm_lpae_run_tests(struct io_pgtable_cfg *cfg) static const enum io_pgtable_fmt fmts[] = { ARM_64_LPAE_S1, ARM_64_LPAE_S2, + ARM_64_LPAE_SPLIT_S1, }; int i, j; diff --git a/drivers/iommu/io-pgtable.c b/drivers/iommu/io-pgtable.c index ced53e5..ed1985f 100644 --- a/drivers/iommu/io-pgtable.c +++ b/drivers/iommu/io-pgtable.c @@ -20,6 +20,7 @@ io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] = { [ARM_64_LPAE_S1] = &io_pgtable_arm_64_lpae_s1_init_fns, [ARM_64_LPAE_S2] = &io_pgtable_arm_64_lpae_s2_init_fns, [ARM_MALI_LPAE] = &io_pgtable_arm_mali_lpae_init_fns, + [ARM_64_LPAE_SPLIT_S1] = &io_pgtable_arm_64_lpae_split_s1_init_fns, #endif #ifdef CONFIG_IOMMU_IO_PGTABLE_ARMV7S [ARM_V7S] = &io_pgtable_arm_v7s_init_fns, diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h index b5a450a..2ed0e6a 100644 --- a/include/linux/io-pgtable.h +++ b/include/linux/io-pgtable.h @@ -10,6 +10,7 @@ enum io_pgtable_fmt { ARM_32_LPAE_S1, ARM_32_LPAE_S2, ARM_64_LPAE_S1, + ARM_64_LPAE_SPLIT_S1, ARM_64_LPAE_S2, ARM_V7S, ARM_MALI_LPAE, @@ -211,6 +212,7 @@ extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s1_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_32_lpae_s2_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns; +extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_split_s1_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns; extern struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns; From patchwork Mon Jul 8 18:10:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jordan Crouse X-Patchwork-Id: 11035561 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6226F14E5 for ; Mon, 8 Jul 2019 18:10:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5300820572 for ; Mon, 8 Jul 2019 18:10:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 46559280B0; Mon, 8 Jul 2019 18:10:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.7 required=2.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E346B20572 for ; Mon, 8 Jul 2019 18:10:33 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391359AbfGHSKc (ORCPT ); Mon, 8 Jul 2019 14:10:32 -0400 Received: from smtp.codeaurora.org ([198.145.29.96]:54236 "EHLO smtp.codeaurora.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391076AbfGHSKc (ORCPT ); Mon, 8 Jul 2019 14:10:32 -0400 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id DD2F2611D1; Mon, 8 Jul 2019 18:10:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1562609430; bh=xc3K/NrdBEaL95vnh28d+67UATTmmmFkY/nNFHRLfGo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WtnxYmmJfDFFC3ZXL2Jt7pK6XeoHlYEkvYbO/ZegOQMTXUdIz88Q/QhG2UA02bk6O e7BUbq446SPLKxHXsUjNy8rSVjL4nbcKgyzT7aZcPkzgzoa7ME16pMjNHtMyqv/XKu CbNyOsRsGeoUh45EO3aUg1QO9TN1d3Q7EpmTdjLQ= Received: from jcrouse1-lnx.qualcomm.com (i-global254.qualcomm.com [199.106.103.254]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: jcrouse@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id E7D546118F; Mon, 8 Jul 2019 18:10:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1562609428; bh=xc3K/NrdBEaL95vnh28d+67UATTmmmFkY/nNFHRLfGo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=X3CKBpHmq8kNLiWfhNh0vkxDilmJZ/IdNkQuSuV824oCpjlWbnUXvMI4V/68WPG3k O49xsnWLyPK/TIeADf5UiLifp/hxMLuAIkNdTFBnj6XZwnkUIMWbLihOHQqU8EeNrK u+PYnNg/4nckxVTPXIz49sLijvNZrLalnaJuuU4U= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org E7D546118F Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=jcrouse@codeaurora.org From: Jordan Crouse To: freedreno@lists.freedesktop.org Cc: jean-philippe.brucker@arm.com, linux-arm-msm@vger.kernel.org, dianders@chromimum.org, hoegsberg@google.com, baolu.lu@linux.intel.com, Will Deacon , linux-kernel@vger.kernel.org, iommu@lists.linux-foundation.org, Robin Murphy , Joerg Roedel , linux-arm-kernel@lists.infradead.org Subject: [PATCH v2 3/3] iommu/arm-smmu: Add support for DOMAIN_ATTR_SPLIT_TABLES Date: Mon, 8 Jul 2019 12:10:18 -0600 Message-Id: <1562609418-25446-4-git-send-email-jcrouse@codeaurora.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1562609418-25446-1-git-send-email-jcrouse@codeaurora.org> References: <1562609418-25446-1-git-send-email-jcrouse@codeaurora.org> Sender: linux-arm-msm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-arm-msm@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP When DOMAIN_ATTR_SPLIT_TABLES is specified for pass ARM_64_LPAE_SPLIT_S1 to io_pgtable_ops to allocate and initialize TTBR0 and TTBR1 pagetables. v3: Moved all the pagetable specific work into io-pgtable-arm in a previous patch. Signed-off-by: Jordan Crouse --- drivers/iommu/arm-smmu.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/arm-smmu.c b/drivers/iommu/arm-smmu.c index 653b6b3..7a6b4bb 100644 --- a/drivers/iommu/arm-smmu.c +++ b/drivers/iommu/arm-smmu.c @@ -257,6 +257,7 @@ struct arm_smmu_domain { bool non_strict; struct mutex init_mutex; /* Protects smmu pointer */ spinlock_t cb_lock; /* Serialises ATS1* ops and TLB syncs */ + u32 attributes; struct iommu_domain domain; }; @@ -832,7 +833,11 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain, ias = smmu->va_size; oas = smmu->ipa_size; if (cfg->fmt == ARM_SMMU_CTX_FMT_AARCH64) { - fmt = ARM_64_LPAE_S1; + if (smmu_domain->attributes & + (1 << DOMAIN_ATTR_SPLIT_TABLES)) + fmt = ARM_64_LPAE_SPLIT_S1; + else + fmt = ARM_64_LPAE_S1; } else if (cfg->fmt == ARM_SMMU_CTX_FMT_AARCH32_L) { fmt = ARM_32_LPAE_S1; ias = min(ias, 32UL); @@ -1582,6 +1587,10 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain, case DOMAIN_ATTR_NESTING: *(int *)data = (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED); return 0; + case DOMAIN_ATTR_SPLIT_TABLES: + *(int *)data = !!(smmu_domain->attributes & + (1 << DOMAIN_ATTR_SPLIT_TABLES)); + return 0; default: return -ENODEV; } @@ -1622,6 +1631,11 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain, else smmu_domain->stage = ARM_SMMU_DOMAIN_S1; break; + case DOMAIN_ATTR_SPLIT_TABLES: + if (*((int *)data)) + smmu_domain->attributes |= + (1 << DOMAIN_ATTR_SPLIT_TABLES); + break; default: ret = -ENODEV; }