From patchwork Tue Dec 19 17:50:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jisheng Zhang X-Patchwork-Id: 13498797 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 021F6C46CCD for ; Tue, 19 Dec 2023 18:03:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 72C448D0006; Tue, 19 Dec 2023 13:03:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6DE308D0001; Tue, 19 Dec 2023 13:03:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C3218D0006; Tue, 19 Dec 2023 13:03:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 397FA6B0078 for ; Tue, 19 Dec 2023 13:03:31 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 088B7A208F for ; Tue, 19 Dec 2023 18:03:31 +0000 (UTC) X-FDA: 81584340222.29.7E31D3D Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf27.hostedemail.com (Postfix) with ESMTP id EE8FC40013 for ; Tue, 19 Dec 2023 18:03:28 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NZ+HplTE; spf=pass (imf27.hostedemail.com: domain of jszhang@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=jszhang@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1703009009; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RQg7suMb/9Jnz+wAQ9WdVfRyNeBfCryQ9+DO8mQ1BQw=; b=G7+u82//Di0GJdjLjttmZjACjBe5IWNEBssUOjIGWxTFLPy1S1FeKcPzeT59syA/SyVs+W 4ToQWwMY/9NkIDaYonVsxTpPTEuhTxyg74Ui0nzP5SCxPLon4z2W5trUAJuE8WEhsUHo1b MOBn1L0b273kD5RTTZnfPfzD1n/TAm4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1703009009; a=rsa-sha256; cv=none; b=sQYQA4GaQHlKdSQH3a1f3PQ6oejvRewyqSrqco1GOwBGCcljIwFo3b/ZTmN2V7VNABeCf4 +CVJvivBiq55vucuNn+BsELjpBH5aRX5bQx4hXWB2rhnPRfyxCByWZ70Tp9DZn8X5rI+qD idDWfHkL6KdqmltsSxJcc8wuJy4vs3Q= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=NZ+HplTE; spf=pass (imf27.hostedemail.com: domain of jszhang@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=jszhang@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 03F7BCE1AA9; Tue, 19 Dec 2023 18:03:26 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D62CDC433C9; Tue, 19 Dec 2023 18:03:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1703009005; bh=39JAuH598M6uo4CmVtLFeafqNHIttPmqjwfQDvfhOjs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=NZ+HplTEbztnxahsANz3uYQlJxV91QZtNCH9i3pK9wbRFkjwTt4VzJXgiEfG1XQyS BXqJkmD+ZuHI3pJbq7hpWh4FazAdi4Y3yKeIPf67IpkIahu9t2PTGodx/3z22CFrV7 m+cNydw6UBPt2db6BACnS7usZ86CqOzEjfRnK1vUXTwLBuz8h9WArwgkfMCsadNvpZ ADKvoV3VZLzOxA8w0qKo27CtkfmEvEQi0IGRe7dYSov3cTESJzZ5pYlOHNLP8W51Mp QrorLyrRGypL6qE5j1GbuY+013Qauw6noSlblzldFoZ80Yl2nqBBjJIFW2VycqEPjS p8/QIHPpMR03g== From: Jisheng Zhang To: Paul Walmsley , Palmer Dabbelt , Albert Ou , Will Deacon , "Aneesh Kumar K . V" , Andrew Morton , Nick Piggin , Peter Zijlstra Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 1/4] riscv: tlb: fix __p*d_free_tlb() Date: Wed, 20 Dec 2023 01:50:43 +0800 Message-Id: <20231219175046.2496-2-jszhang@kernel.org> X-Mailer: git-send-email 2.40.0 In-Reply-To: <20231219175046.2496-1-jszhang@kernel.org> References: <20231219175046.2496-1-jszhang@kernel.org> MIME-Version: 1.0 X-Stat-Signature: f5em9x4s5pr8ezest8mc4d13ahro36ee X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: EE8FC40013 X-Rspam-User: X-HE-Tag: 1703009008-604613 X-HE-Meta: U2FsdGVkX19PrHbHdgzhro9bhGbD9OwqnMQN3jQhdHDw5NzI5DctBVGyVJyYD8nThm+DeJXsqybx1+r4692lfEiX0nMa1Ktepb0nl1LJoDmMy/yuQW0nEkIVQ2OFrFQYeiaXyhEU9Bz1298ddEvYVPW4sqoGqtD3pFkiNzcbHgiuurOQb3EqRlkra8S7gn07gGG4a2C20YG8zgq6aQsh+0lLvxY8xmS/CFeMGyVSgDYAO2s8x3XW7zu8setCdL/v+UawibrmZBv7RRwqD4Y0eTYVm6d/MOhAlA8CVZKoQT1i1lUiYwCgGIxzmbkf+aHmG/QSZI5IG83tGHpAadEEunv76ECfsj5cAG9JdwHwK5FMzJ2mVUp1kt6iAXk/34Rx2ZHltD629bxwYrkzcQToM3L2JZy0E/QqSh0FSWA80HffILLD5ZueSWfmHwRYqF8xZAfCWsii17nPgLxs9ZPaFvD3SOuAnKNsO6mAKodo2jxKXCQp/T/xkVyzeiUQW9TnErUQILFV1zODi30VerDws9lVhy+SSxkfPucvHlX63FqgjTfGCMul265aEOr30SB+veb4LZpeI04wUwnctOG73j8d7ZnANRaq6CPMl1SziIFWfkxSzTHu/YMBeL+DIqLyDen3JuSEAN5HoaJdYu9rntaNl71OVyT6tWciLihswwqvJ2dgKfV2LXHzB9Ds4kU6z2Fkyo84JjnPChPnmHdTU8ssQfsf5dKsRk0J1RkpMAzhKWR25VuoWD9hJ3AlzxUWLh1IEU9daexI3kgeVh7ruYRnUwufhAVyE8xrHLkoRxKacD8RKl6p/+Y5jcSqqgsEBv4ICR6a7faOqqU8aQe4Xjxqu4lf/QqiIFAoccsbDD3PlcDLxDLI2IDsAYogGCgdIwFO2jlze1VrnCSGqwiVfnwJWcsKAr9AFVfyP8sViAbdedhtVZp95fHZOcekm792i2Dk6ck+NHySgbnuaMi PhcER3Bq a6noK X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: If non-leaf PTEs I.E pmd, pud or p4d is modified, a sfence.vma is a must for safe, imagine if an implementation caches the non-leaf translation in TLB, although I didn't meet this HW so far, but it's possible in theory. Signed-off-by: Jisheng Zhang Reviewed-by: Alexandre Ghiti --- arch/riscv/include/asm/pgalloc.h | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/arch/riscv/include/asm/pgalloc.h b/arch/riscv/include/asm/pgalloc.h index d169a4f41a2e..a12fb83fa1f5 100644 --- a/arch/riscv/include/asm/pgalloc.h +++ b/arch/riscv/include/asm/pgalloc.h @@ -95,7 +95,13 @@ static inline void pud_free(struct mm_struct *mm, pud_t *pud) __pud_free(mm, pud); } -#define __pud_free_tlb(tlb, pud, addr) pud_free((tlb)->mm, pud) +#define __pud_free_tlb(tlb, pud, addr) \ +do { \ + if (pgtable_l4_enabled) { \ + pagetable_pud_dtor(virt_to_ptdesc(pud)); \ + tlb_remove_page_ptdesc((tlb), virt_to_ptdesc(pud)); \ + } \ +} while (0) #define p4d_alloc_one p4d_alloc_one static inline p4d_t *p4d_alloc_one(struct mm_struct *mm, unsigned long addr) @@ -124,7 +130,11 @@ static inline void p4d_free(struct mm_struct *mm, p4d_t *p4d) __p4d_free(mm, p4d); } -#define __p4d_free_tlb(tlb, p4d, addr) p4d_free((tlb)->mm, p4d) +#define __p4d_free_tlb(tlb, p4d, addr) \ +do { \ + if (pgtable_l5_enabled) \ + tlb_remove_page_ptdesc((tlb), virt_to_ptdesc(p4d)); \ +} while (0) #endif /* __PAGETABLE_PMD_FOLDED */ static inline void sync_kernel_mappings(pgd_t *pgd) @@ -149,7 +159,11 @@ static inline pgd_t *pgd_alloc(struct mm_struct *mm) #ifndef __PAGETABLE_PMD_FOLDED -#define __pmd_free_tlb(tlb, pmd, addr) pmd_free((tlb)->mm, pmd) +#define __pmd_free_tlb(tlb, pmd, addr) \ +do { \ + pagetable_pmd_dtor(virt_to_ptdesc(pmd)); \ + tlb_remove_page_ptdesc((tlb), virt_to_ptdesc(pmd)); \ +} while (0) #endif /* __PAGETABLE_PMD_FOLDED */