From patchwork Wed Sep 12 10:26:14 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Will Deacon X-Patchwork-Id: 10597333 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9412A14DB for ; Wed, 12 Sep 2018 10:26:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 813CD29A7C for ; Wed, 12 Sep 2018 10:26:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7496629A94; Wed, 12 Sep 2018 10:26:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id DA6A029A7C for ; Wed, 12 Sep 2018 10:26:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3A09F8E0007; Wed, 12 Sep 2018 06:26:05 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 3034D8E0003; Wed, 12 Sep 2018 06:26:05 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0E18A8E0007; Wed, 12 Sep 2018 06:26:05 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-oi0-f70.google.com (mail-oi0-f70.google.com [209.85.218.70]) by kanga.kvack.org (Postfix) with ESMTP id C6AA88E0003 for ; Wed, 12 Sep 2018 06:26:04 -0400 (EDT) Received: by mail-oi0-f70.google.com with SMTP id v4-v6so1793359oix.2 for ; Wed, 12 Sep 2018 03:26:04 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=UQhbSo2CjhJBbAoBN3seYvduCvHh3YrpIQUGvyxA6kg=; b=HeZLmtsPfu+d+wmPbP2oJwynFJwrz9P9+yeNRDLkwepE11xo3yFzKF3+UZhMi2pt1x 8ItGpQ6tUpFqKGcoc6KQygXBsAFVfPRqAJcl6DcEjrHbXX+A+8SdxGxX0l5hUgtn9Wa1 56AndGwUkKnUGT9T2tC6Qa0ABa2fdiFjQbDg25evvdL3I0h2vvEdSc5jRUGZ1gKCjUsU 7FavemNsLcwECNdtNK0NKJ1emQ72D9udtJ+ccMxU7BFGz8830uAhzVbTYl5ggwm+x8mO 1g/ozmjJKLYuc56Bp4y4wW7ocKouiAHqEejXgySAl1kYLrB76hUTf+fSkXJ4VLEiYTi5 43CQ== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of will.deacon@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=will.deacon@arm.com X-Gm-Message-State: APzg51BJ1j449OoLrgzCyaMxivy/p9BupTihsd3Ysdu0IT0tvGtr0B8D kzkJku/m52mzhT1T/56kFaDZqoewIZxbfuknA61ZuYg/IGU17VrZiolshD0LIBU0hb6sE2qJg4Q 6oqKcXO8aTCazazvm/vWkLRNCxCOVAW1ovtFx64Jjm56iygVAfhl9iDsz0pELUiHRdw== X-Received: by 2002:aca:6b15:: with SMTP id g21-v6mr1271093oic.149.1536747964542; Wed, 12 Sep 2018 03:26:04 -0700 (PDT) X-Google-Smtp-Source: ANB0VdbtlFP/iGMGs5OaqHjtjIM3k9Ye9VwE/s/lID9A33o2QEBP70fW8igTW9zrcEg3c5q2vUgD X-Received: by 2002:aca:6b15:: with SMTP id g21-v6mr1271021oic.149.1536747963565; Wed, 12 Sep 2018 03:26:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1536747963; cv=none; d=google.com; s=arc-20160816; b=T2oiFy+k0j96ZuZOXWv920DNHSCHfBy2iKWsG8e9pfE0AmstVd+XGvnTb7YtgQp7lJ 8v8ezqyAHl6h+Zfw3YwDlBCvsP8/t/RjbzFNcLOpJASUgQ2I+GNOIDh6nJxf6/D26/ST eYU0D02cDbC6Q8khvCYJZgDXOkJuvhi6jbiiTKtW3RcEBTdRcRmOtARd5t5yXeZtgH2X as4vHsIzMiGw9DVxu1xRDr3ljTfrj0pvm4oA6WJNJ8mhfQeqiSLjiZ0KgK8/WLCV3eGF fCzmZf2YlNvbLdSVsae86bHQtT+GEUdrUKqTRmRoOpGQkEPPZWes7gX9EJ6CJ0PWgygb JuEw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=UQhbSo2CjhJBbAoBN3seYvduCvHh3YrpIQUGvyxA6kg=; b=xCAHyhjOccouttIOJKqFCLW44MbwGslzf+rcPQDjnoAarMC9cJkG5qgKSnohaFzr4v zxqZYNU+NhIpbwB9iz6lxGWr40ca+V7B82myCX6Il/9UxFQccesiKMDgYHwkc8QsKQVF U1cTetNB/lVHZF+eHNXPFaiQCfKaXZhKxd91mev0kaXwdEYfByO76P7C7WWKM+P6nrwJ xsQFPG0IsgyC1sFjeD7TSDREhfW3/rC4sZ422JGdtvQHaLKAEhZchz91TQZXforIB2Nu wqWNzn3KbDosgQ2eO3uoVz8KUwBn2SwSlPNehEpsvBF3G9kespnPqheD4TipdluobTT4 gW/A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of will.deacon@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=will.deacon@arm.com Received: from foss.arm.com (usa-sjc-mx-foss1.foss.arm.com. [217.140.101.70]) by mx.google.com with ESMTP id u185-v6si421964oib.207.2018.09.12.03.26.03 for ; Wed, 12 Sep 2018 03:26:03 -0700 (PDT) Received-SPF: pass (google.com: domain of will.deacon@arm.com designates 217.140.101.70 as permitted sender) client-ip=217.140.101.70; Authentication-Results: mx.google.com; spf=pass (google.com: domain of will.deacon@arm.com designates 217.140.101.70 as permitted sender) smtp.mailfrom=will.deacon@arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id C15401688; Wed, 12 Sep 2018 03:26:02 -0700 (PDT) Received: from edgewater-inn.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.72.51.249]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 936BE3F614; Wed, 12 Sep 2018 03:26:02 -0700 (PDT) Received: by edgewater-inn.cambridge.arm.com (Postfix, from userid 1000) id C26BA1AE357F; Wed, 12 Sep 2018 11:26:18 +0100 (BST) From: Will Deacon To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: cpandya@codeaurora.org, toshi.kani@hpe.com, tglx@linutronix.de, mhocko@suse.com, akpm@linux-foundation.org, Will Deacon Subject: [PATCH 5/5] lib/ioremap: Ensure break-before-make is used for huge p4d mappings Date: Wed, 12 Sep 2018 11:26:14 +0100 Message-Id: <1536747974-25875-6-git-send-email-will.deacon@arm.com> X-Mailer: git-send-email 2.1.4 In-Reply-To: <1536747974-25875-1-git-send-email-will.deacon@arm.com> References: <1536747974-25875-1-git-send-email-will.deacon@arm.com> X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP Whilst no architectures actually enable support for huge p4d mappings in the vmap area, the code that is implemented should be using break-before-make, as we do for pud and pmd huge entries. Cc: Chintan Pandya Cc: Toshi Kani Cc: Thomas Gleixner Cc: Michal Hocko Cc: Andrew Morton Signed-off-by: Will Deacon Reviewed-by: Toshi Kani --- arch/arm64/mm/mmu.c | 5 +++++ arch/x86/mm/pgtable.c | 8 ++++++++ include/asm-generic/pgtable.h | 5 +++++ lib/ioremap.c | 27 +++++++++++++++++++++------ 4 files changed, 39 insertions(+), 6 deletions(-) diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 0dcb3354d6dd..58776b90dd2a 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -1024,3 +1024,8 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) pmd_free(NULL, table); return 1; } + +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; /* Don't attempt a block mapping */ +} diff --git a/arch/x86/mm/pgtable.c b/arch/x86/mm/pgtable.c index b4919c44a194..c6094997d060 100644 --- a/arch/x86/mm/pgtable.c +++ b/arch/x86/mm/pgtable.c @@ -779,6 +779,14 @@ int pmd_clear_huge(pmd_t *pmd) return 0; } +/* + * Until we support 512GB pages, skip them in the vmap area. + */ +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} + #ifdef CONFIG_X86_64 /** * pud_free_pmd_page - Clear pud entry and free pmd page. diff --git a/include/asm-generic/pgtable.h b/include/asm-generic/pgtable.h index 88ebc6102c7c..4297a2519ebf 100644 --- a/include/asm-generic/pgtable.h +++ b/include/asm-generic/pgtable.h @@ -1019,6 +1019,7 @@ int pud_set_huge(pud_t *pud, phys_addr_t addr, pgprot_t prot); int pmd_set_huge(pmd_t *pmd, phys_addr_t addr, pgprot_t prot); int pud_clear_huge(pud_t *pud); int pmd_clear_huge(pmd_t *pmd); +int p4d_free_pud_page(p4d_t *p4d, unsigned long addr); int pud_free_pmd_page(pud_t *pud, unsigned long addr); int pmd_free_pte_page(pmd_t *pmd, unsigned long addr); #else /* !CONFIG_HAVE_ARCH_HUGE_VMAP */ @@ -1046,6 +1047,10 @@ static inline int pmd_clear_huge(pmd_t *pmd) { return 0; } +static inline int p4d_free_pud_page(p4d_t *p4d, unsigned long addr) +{ + return 0; +} static inline int pud_free_pmd_page(pud_t *pud, unsigned long addr) { return 0; diff --git a/lib/ioremap.c b/lib/ioremap.c index fc834a59c90c..49d2e23dad2e 100644 --- a/lib/ioremap.c +++ b/lib/ioremap.c @@ -156,6 +156,25 @@ static inline int ioremap_pud_range(p4d_t *p4d, unsigned long addr, return 0; } +static int ioremap_try_huge_p4d(p4d_t *p4d, unsigned long addr, + unsigned long end, phys_addr_t phys_addr, + pgprot_t prot) +{ + if (!ioremap_p4d_enabled()) + return 0; + + if ((end - addr) != P4D_SIZE) + return 0; + + if (!IS_ALIGNED(phys_addr, P4D_SIZE)) + return 0; + + if (p4d_present(*p4d) && !p4d_free_pud_page(p4d, addr)) + return 0; + + return p4d_set_huge(p4d, phys_addr, prot); +} + static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, phys_addr_t phys_addr, pgprot_t prot) { @@ -168,12 +187,8 @@ static inline int ioremap_p4d_range(pgd_t *pgd, unsigned long addr, do { next = p4d_addr_end(addr, end); - if (ioremap_p4d_enabled() && - ((next - addr) == P4D_SIZE) && - IS_ALIGNED(phys_addr, P4D_SIZE)) { - if (p4d_set_huge(p4d, phys_addr, prot)) - continue; - } + if (ioremap_try_huge_p4d(p4d, addr, next, phys_addr, prot)) + continue; if (ioremap_pud_range(p4d, addr, next, phys_addr, prot)) return -ENOMEM;