From patchwork Sat Sep 9 20:16:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13378329 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0AA76EE14AA for ; Sat, 9 Sep 2023 20:17:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=blfOTHTIA7+Hxm5wKYQ5cq1zyuDsj07Xs7OMLTST+eY=; b=sqCvgAgtLDkNjz aaUyXJ6fqffOPIeK64gWYdtcSOv2IVqDrwmKRq9NSuqW+kzqTj9GVTK7XhI5WuhsK/9hmDhcseT6c YI4NrWnC5W3lfP1A1XPFg9xZmeeWp/arO1sfZZoco0OuuaKClE9e8buVlCTGNAQraB3D+CenoXaZz yN5vMPydu6FMhpmh2Qjs5y2mZIDPxf6hdtb8nHzHUZ5HIRnXpxCZsRN+Nhqu/C29Xn7VxzcVjTNK9 JlUXT5CFZZLp6b6j45okUSmilCTyyxbOD2w29D4AHU39ejzhbvKrgT4DUMQY4SCHL896AhnuxDAvx Xz8cpCzvn4vUTWbx0bug==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Oc-00FknI-1A; Sat, 09 Sep 2023 20:17:34 +0000 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Oa-00Fkmm-0z for linux-riscv@lists.infradead.org; Sat, 09 Sep 2023 20:17:33 +0000 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.west.internal (Postfix) with ESMTP id 53337320090E; Sat, 9 Sep 2023 16:17:31 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Sat, 09 Sep 2023 16:17:31 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sholland.org; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1694290650; x= 1694377050; bh=7NXnmFYFmr6Y8hhtV6LxTtgTAxVz99Y/He8EwBOMTPo=; b=v MGC/yqEpNIdjgRMas1jOpQX+RGuER3w1a9LX7DRtbX3YU/zuvaZQ6lNCqgC5uN3u +WX7QnEdXIsb6GU2zk0Fq6WqIhS9vF4GMjrvaQthRv21wcMpyMuogSqMZpRH65A4 ikNYD/4Ya6H3nKwcWxOFq2m2+CIOhkHPIOMUEMAaQaEQzmBDjk3eYR2Cm9lxdwAM Bf0Y4WrP+rxWWli4eqa0N2OMzHBwgJIIb06Z4fHo0Wjt0pjAhXAclM5363cBvuIJ dVIyEreE8ynOQTiWVp73D9Vlp/vxjklgg+hjVQKJNpBN1zOzdmXUfuj28u3zFde6 0ARU84BJ1h6BjreTvI7hw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1694290650; x= 1694377050; bh=7NXnmFYFmr6Y8hhtV6LxTtgTAxVz99Y/He8EwBOMTPo=; b=V /6PtmbbrFsuIPtof2CD8rYJbN8SW8zFJs5B3F1yVaeB/EqxlRcGpakegQduTcXFf wSIkvVrmCEmsjz84lWiez9t8xrXMOpE+l7K5d3o8XLvRcEmDlcZKlRmF+IfGUFtT QDFb0L7Pm0baBvnwqSvUn5jhoeJjGS6ifeXG1Am8mpFsnshwMkgfruTNH/7INHdC hSgmCO+CZdE0Mvud4b5FpStlebpmjkVmy4FMOZtL0k2pgsXO1EjAXQDSHEtgAzNv P5xxWSFJfWyEDjXDo+0/euINs2raDRN16uSY1G907a004+eyFYLMVowcmZMDaK7m sUNogkSv3rR3eNzMLMXRw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudehledgudeghecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefurghm uhgvlhcujfholhhlrghnugcuoehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhgqeenuc ggtffrrghtthgvrhhnpedukeetueduhedtleetvefguddvvdejhfefudelgfduveeggeeh gfdufeeitdevteenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhg X-ME-Proxy: Feedback-ID: i0ad843c9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 9 Sep 2023 16:17:30 -0400 (EDT) From: Samuel Holland To: Palmer Dabbelt , Alexandre Ghiti , linux-riscv@lists.infradead.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Samuel Holland Subject: [PATCH 1/7] riscv: Apply SiFive CIP-1200 workaround to single-ASID sfence.vma Date: Sat, 9 Sep 2023 15:16:29 -0500 Message-ID: <20230909201727.10909-2-samuel@sholland.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230909201727.10909-1-samuel@sholland.org> References: <20230909201727.10909-1-samuel@sholland.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230909_131732_372726_DFA01693 X-CRM114-Status: GOOD ( 11.31 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org commit 3f1e782998cd ("riscv: add ASID-based tlbflushing methods") added calls to the sfence.vma instruction with rs2 != x0. These single-ASID instruction variants are also affected by SiFive errata CIP-1200. Until now, the errata workaround was not needed for the single-ASID sfence.vma variants, because they were only used when the ASID allocator was enabled, and the affected SiFive platforms do not support multiple ASIDs. However, we are going to start using those sfence.vma variants regardless of ASID support, so now we need alternatives covering them. Signed-off-by: Samuel Holland --- arch/riscv/include/asm/errata_list.h | 12 +++++++++++- arch/riscv/include/asm/tlbflush.h | 13 ++++++++++++- arch/riscv/mm/tlbflush.c | 17 ----------------- 3 files changed, 23 insertions(+), 19 deletions(-) diff --git a/arch/riscv/include/asm/errata_list.h b/arch/riscv/include/asm/errata_list.h index fb1a810f3d8c..6e416297847c 100644 --- a/arch/riscv/include/asm/errata_list.h +++ b/arch/riscv/include/asm/errata_list.h @@ -39,11 +39,21 @@ ALTERNATIVE(__stringify(RISCV_PTR do_page_fault), \ CONFIG_ERRATA_SIFIVE_CIP_453) #else /* !__ASSEMBLY__ */ -#define ALT_FLUSH_TLB_PAGE(x) \ +#define ALT_SFENCE_VMA_ASID(asid) \ +asm(ALTERNATIVE("sfence.vma x0, %0", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (asid) : "memory") + +#define ALT_SFENCE_VMA_ADDR(addr) \ asm(ALTERNATIVE("sfence.vma %0", "sfence.vma", SIFIVE_VENDOR_ID, \ ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ : : "r" (addr) : "memory") +#define ALT_SFENCE_VMA_ADDR_ASID(addr, asid) \ +asm(ALTERNATIVE("sfence.vma %0, %1", "sfence.vma", SIFIVE_VENDOR_ID, \ + ERRATA_SIFIVE_CIP_1200, CONFIG_ERRATA_SIFIVE_CIP_1200) \ + : : "r" (addr), "r" (asid) : "memory") + /* * _val is marked as "will be overwritten", so need to set it to 0 * in the default case. diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index a09196f8de68..e2960c9c16e8 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -19,10 +19,21 @@ static inline void local_flush_tlb_all(void) __asm__ __volatile__ ("sfence.vma" : : : "memory"); } +static inline void local_flush_tlb_all_asid(unsigned long asid) +{ + ALT_SFENCE_VMA_ASID(asid); +} + /* Flush one page from local TLB */ static inline void local_flush_tlb_page(unsigned long addr) { - ALT_FLUSH_TLB_PAGE(__asm__ __volatile__ ("sfence.vma %0" : : "r" (addr) : "memory")); + ALT_SFENCE_VMA_ADDR(addr); +} + +static inline void local_flush_tlb_page_asid(unsigned long addr, + unsigned long asid) +{ + ALT_SFENCE_VMA_ADDR_ASID(addr, asid); } #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 77be59aadc73..115d617d16c9 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -6,23 +6,6 @@ #include #include -static inline void local_flush_tlb_all_asid(unsigned long asid) -{ - __asm__ __volatile__ ("sfence.vma x0, %0" - : - : "r" (asid) - : "memory"); -} - -static inline void local_flush_tlb_page_asid(unsigned long addr, - unsigned long asid) -{ - __asm__ __volatile__ ("sfence.vma %0, %1" - : - : "r" (addr), "r" (asid) - : "memory"); -} - static inline void local_flush_tlb_range(unsigned long start, unsigned long size, unsigned long stride) { From patchwork Sat Sep 9 20:16:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13378331 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 60FB7EEB580 for ; Sat, 9 Sep 2023 20:17:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=GDhvqzPJf9MBumZ2xJVBRFYnUwXr1LQ+6sYjqGznVMM=; b=IBdhHnKC+UB46s 6Motk0mF4sYRPCza+u+jzoi83ueihvd4lbewWFeSxwQfFo/UsBX8uGwTxQmX858RDJc5pmNu9XP/V k2UNFWWySwau4p0+Vf32J/ac/gSaKNVgWkonJOjkZMM4i0YBoIveIt5sZ6IjxTih3P824QWuMDVkf zo6e+18wYAXjyrVWe0jeJO80ULRFC4H6CxPAf1FHpWsqYnZNMJdNFRNG6uAU/490yitS+7vGh1pTl 8Z1mbr4dlWhWewgTJwBXW2p83CKGsoXIf0268AAyRPdP7kQEp68/QF50oLV34LmlH8bFCfPE8E6b7 ft2C6qYkxz78RsSB+VrQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Og-00FkpP-0I; Sat, 09 Sep 2023 20:17:38 +0000 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Oc-00FknH-1q for linux-riscv@lists.infradead.org; Sat, 09 Sep 2023 20:17:35 +0000 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id 8DF853200805; Sat, 9 Sep 2023 16:17:33 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Sat, 09 Sep 2023 16:17:34 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sholland.org; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1694290653; x= 1694377053; bh=9o/8XILCtdG4mXKommKnttfmvY+ZRjJhj3Lk+06yNt8=; b=o JKP/ccSB9pnH/lIQMaUwNjeMx4b0yKYX1T3YcLti7UShQ/qMN4NIAZHuIgnVE/0M jJ/Xf5DbTmoSXWsC3bgwqsNScEBW2b42gSESxEFIEwPt95leJISLNrcMcdx+bvIS PaqmQNrLlYZ3C6XRNZgaGpadzDgk4FsdQ/QGH3Ls9GLLcrFqTC3pqVOjJkZNjSm/ 2c03LXAroxyjMhbXuX8RO9u3jBqh46QtS6s4J4RQL/y8c+36SLen/y5+rHfG8anL GaJyzx1yFDuafoUFgAdVFBklTwVO8T7ljdOrm6ymmiK7qAd3UNTT2Oq+YmS2IwZ3 FiX024vNMz5HVfNifNulw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1694290653; x= 1694377053; bh=9o/8XILCtdG4mXKommKnttfmvY+ZRjJhj3Lk+06yNt8=; b=O GuyNno8goMCmg0N6xXxvylW9yZdKkK0dhtII1fAqnR8EPO8WRSg3xTHFltHJfOtG xklZ7h25LOpDZ8Pdd8yEzVIDGiE3WlKqXZd4c4Mt8CeMOlJwiCEL8wMA0Wp++nB3 diyEu5p5YyJ684tWQ9kj5y93p3dX0+k5sGlNSnYnhdJiVs3btWIxt2NtVHX9kr0F 2MoIc3nT/Xaw7CbuTs8r5qeYH4iqxzJPyJjEZ45DgeEjeO5kY5E4xTs769xHSoPW r0qjpQK4YP7Pap2cj9ZTX2Ilzc09jZ269qNpRnrJgWgERWivIQrT7Em1Q85/Duuh DLCNFDBeLW4Gh3hKxfihw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudehledgudegiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefurghm uhgvlhcujfholhhlrghnugcuoehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhgqeenuc ggtffrrghtthgvrhhnpedukeetueduhedtleetvefguddvvdejhfefudelgfduveeggeeh gfdufeeitdevteenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhg X-ME-Proxy: Feedback-ID: i0ad843c9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 9 Sep 2023 16:17:32 -0400 (EDT) From: Samuel Holland To: Palmer Dabbelt , Alexandre Ghiti , linux-riscv@lists.infradead.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Samuel Holland Subject: [PATCH 2/7] riscv: mm: Introduce cntx2asid/cntx2version helper macros Date: Sat, 9 Sep 2023 15:16:30 -0500 Message-ID: <20230909201727.10909-3-samuel@sholland.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230909201727.10909-1-samuel@sholland.org> References: <20230909201727.10909-1-samuel@sholland.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230909_131734_636974_5756D5FA X-CRM114-Status: GOOD ( 13.71 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org When using the ASID allocator, the MM context ID contains two values: the ASID in the lower bits, and the allocator version number in the remaining bits. Use macros to make this separation more obvious. Signed-off-by: Samuel Holland --- arch/riscv/include/asm/mmu.h | 3 +++ arch/riscv/mm/context.c | 12 ++++++------ arch/riscv/mm/tlbflush.c | 2 +- 3 files changed, 10 insertions(+), 7 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index 355504b37f8e..a550fbf770be 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,6 +26,9 @@ typedef struct { #endif } mm_context_t; +#define cntx2asid(cntx) ((cntx) & asid_mask) +#define cntx2version(cntx) ((cntx) & ~asid_mask) + void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); #endif /* __ASSEMBLY__ */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 217fd4de6134..43d005f63253 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -81,7 +81,7 @@ static void __flush_context(void) if (cntx == 0) cntx = per_cpu(reserved_context, i); - __set_bit(cntx & asid_mask, context_asid_map); + __set_bit(cntx2asid(cntx), context_asid_map); per_cpu(reserved_context, i) = cntx; } @@ -102,7 +102,7 @@ static unsigned long __new_context(struct mm_struct *mm) lockdep_assert_held(&context_lock); if (cntx != 0) { - unsigned long newcntx = ver | (cntx & asid_mask); + unsigned long newcntx = ver | cntx2asid(cntx); /* * If our current CONTEXT was active during a rollover, we @@ -115,7 +115,7 @@ static unsigned long __new_context(struct mm_struct *mm) * We had a valid CONTEXT in a previous life, so try to * re-use it if possible. */ - if (!__test_and_set_bit(cntx & asid_mask, context_asid_map)) + if (!__test_and_set_bit(cntx2asid(cntx), context_asid_map)) return newcntx; } @@ -168,7 +168,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) */ old_active_cntx = atomic_long_read(&per_cpu(active_context, cpu)); if (old_active_cntx && - ((cntx & ~asid_mask) == atomic_long_read(¤t_version)) && + (cntx2version(cntx) == atomic_long_read(¤t_version)) && atomic_long_cmpxchg_relaxed(&per_cpu(active_context, cpu), old_active_cntx, cntx)) goto switch_mm_fast; @@ -177,7 +177,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) /* Check that our ASID belongs to the current_version. */ cntx = atomic_long_read(&mm->context.id); - if ((cntx & ~asid_mask) != atomic_long_read(¤t_version)) { + if (cntx2version(cntx) != atomic_long_read(¤t_version)) { cntx = __new_context(mm); atomic_long_set(&mm->context.id, cntx); } @@ -191,7 +191,7 @@ static void set_mm_asid(struct mm_struct *mm, unsigned int cpu) switch_mm_fast: csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | - ((cntx & asid_mask) << SATP_ASID_SHIFT) | + (cntx2asid(cntx) << SATP_ASID_SHIFT) | satp_mode); if (need_flush_tlb) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 115d617d16c9..54c3e70ccd81 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -73,7 +73,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, /* check if the tlbflush needs to be sent to other CPUs */ broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; if (static_branch_unlikely(&use_asid_allocator)) { - unsigned long asid = atomic_long_read(&mm->context.id) & asid_mask; + unsigned long asid = cntx2asid(atomic_long_read(&mm->context.id)); if (broadcast) { if (riscv_use_ipi_for_rfence()) { From patchwork Sat Sep 9 20:16:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13378330 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id E0BC7EEB581 for ; Sat, 9 Sep 2023 20:17:44 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UowThSLSp9fPB84OB1bAjlqo7CW6Bawy1Nc9UN64oNE=; b=RBZo6ky9mC5rsJ WudPCDda04YKReirB9HqIoVsa+utQkUQCkxtZRwgP6GIiSSlwKHfqt3ANowvqrpnbQyPmrGokc60b 1hOHO6B0NlAsGenqUoCfOyx0mumT2AZWovKtwtRfS0djUp8H9Ud9dt4SgTj6nwdfI5c1418R32ftm H5G0LeU2T5bngd6hSBF0/a0XzTH0FSXJ8jKLS36900Ut7EuJSckSrLTYL3igSBs3OSHm41NSrRwIw OyWMYch1uvf/zi5sMD2aIFryifDKCAAxX8ddknBJgT4ooT8wtBmYbAuDoWDu7uifqE4YZK5HMtJN0 p8BT0DjNVrpWTngMh0FQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Oh-00Fkqt-2R; Sat, 09 Sep 2023 20:17:39 +0000 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Oe-00Fkow-2n for linux-riscv@lists.infradead.org; Sat, 09 Sep 2023 20:17:38 +0000 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id DBC32320090E; Sat, 9 Sep 2023 16:17:35 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Sat, 09 Sep 2023 16:17:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sholland.org; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1694290655; x= 1694377055; bh=u+R27fQSLabNs20twvWYROSpY1egJDSSJ6jgm8OSGJ4=; b=r E1qegEdZ8OH3AdJXRXbJ/3z8lj+sZPmBGODILRCdn0bGZqET9HOPF/BLhEBOPWJE UB3KaCLKmFAO7lp8BrUybwd5JEtyvovEXTUm/VmGk42tvImQ/h1auBQeBtI4ZrhT aDxPkcjwdxS/Vx/1vXS24IxBeW3VY7w+Pir/r9JlSvgmzjjel8UR77T7I/WxZ5Qy WrsZdAs+ZtD9Fx0mBj+WYsGrqz2QPzxb4fBSdpKTbxGaX3m+avHeZx/UaSJ8ifMk VNKZKqZ4blwejANEGo7AjjYSBsVnQrjxPdzkX67JnSHX3Bmx4vGTU9gt/tIia9MH rdbMvtWfLFRnhZcL2DsnQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1694290655; x= 1694377055; bh=u+R27fQSLabNs20twvWYROSpY1egJDSSJ6jgm8OSGJ4=; b=t 3ES3w7Sl5IIg4IQUHFQ3gL5Ri8nvST3n+1Ut3ygOL3N7iqih9+T6xLJwRv5Wjyjs wOqv7gl99t7WiQf7lzZ5RWiiA5iUoIK5P6RX2tIHpx/nyo4TMG+QSDWnJeJEABei JGombEGeOD8hwPZglBCGnCYNSGzA+F0isXNVK+jhUOz/VZLGQpBIpp7Dr5Dz6/A3 gz5mpQIc2RCL634Iz+BnysxVcZmKDJyr3B8pNnuP5JA65ewDofu+t8isLrYema1z HMbTfqUFykfl/O7GcJVhKD53ou+LN47Z5/r9H3gRi5/sutIF1k/lOm3kTXKS7tRB b0j2F6Y/wxhM0WBRYj0Kg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudehledgudegiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefurghm uhgvlhcujfholhhlrghnugcuoehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhgqeenuc ggtffrrghtthgvrhhnpedukeetueduhedtleetvefguddvvdejhfefudelgfduveeggeeh gfdufeeitdevteenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhg X-ME-Proxy: Feedback-ID: i0ad843c9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 9 Sep 2023 16:17:34 -0400 (EDT) From: Samuel Holland To: Palmer Dabbelt , Alexandre Ghiti , linux-riscv@lists.infradead.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Samuel Holland Subject: [PATCH 3/7] riscv: mm: Use a fixed layout for the MM context ID Date: Sat, 9 Sep 2023 15:16:31 -0500 Message-ID: <20230909201727.10909-4-samuel@sholland.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230909201727.10909-1-samuel@sholland.org> References: <20230909201727.10909-1-samuel@sholland.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230909_131736_929888_04AF9115 X-CRM114-Status: GOOD ( 11.19 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Currently, the size of the ASID field in the MM context ID dynamically depends on the number of hardware-supported ASID bits. This requires reading a global variable to extract either field from the context ID. Instead, allocate the maximum possible number of bits to the ASID field, so the layout of the context ID is known at compile-time. Signed-off-by: Samuel Holland --- arch/riscv/include/asm/mmu.h | 4 ++-- arch/riscv/include/asm/tlbflush.h | 2 -- arch/riscv/mm/context.c | 6 ++---- 3 files changed, 4 insertions(+), 8 deletions(-) diff --git a/arch/riscv/include/asm/mmu.h b/arch/riscv/include/asm/mmu.h index a550fbf770be..dc0273f7905f 100644 --- a/arch/riscv/include/asm/mmu.h +++ b/arch/riscv/include/asm/mmu.h @@ -26,8 +26,8 @@ typedef struct { #endif } mm_context_t; -#define cntx2asid(cntx) ((cntx) & asid_mask) -#define cntx2version(cntx) ((cntx) & ~asid_mask) +#define cntx2asid(cntx) ((cntx) & SATP_ASID_MASK) +#define cntx2version(cntx) ((cntx) & ~SATP_ASID_MASK) void __init create_pgd_mapping(pgd_t *pgdp, uintptr_t va, phys_addr_t pa, phys_addr_t sz, pgprot_t prot); diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index e2960c9c16e8..e55831edfc19 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -12,8 +12,6 @@ #include #ifdef CONFIG_MMU -extern unsigned long asid_mask; - static inline void local_flush_tlb_all(void) { __asm__ __volatile__ ("sfence.vma" : : : "memory"); diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 43d005f63253..b5170ac1b742 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -22,7 +22,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); static unsigned long asid_bits; static unsigned long num_asids; -unsigned long asid_mask; static atomic_long_t current_version; @@ -128,7 +127,7 @@ static unsigned long __new_context(struct mm_struct *mm) goto set_asid; /* We're out of ASIDs, so increment current_version */ - ver = atomic_long_add_return_relaxed(num_asids, ¤t_version); + ver = atomic_long_add_return_relaxed(BIT(SATP_ASID_BITS), ¤t_version); /* Flush everything */ __flush_context(); @@ -247,7 +246,6 @@ static int __init asids_init(void) /* Pre-compute ASID details */ if (asid_bits) { num_asids = 1 << asid_bits; - asid_mask = num_asids - 1; } /* @@ -255,7 +253,7 @@ static int __init asids_init(void) * at-least twice more than CPUs */ if (num_asids > (2 * num_possible_cpus())) { - atomic_long_set(¤t_version, num_asids); + atomic_long_set(¤t_version, BIT(SATP_ASID_BITS)); context_asid_map = bitmap_zalloc(num_asids, GFP_KERNEL); if (!context_asid_map) From patchwork Sat Sep 9 20:16:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13378333 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 71A2FEE57CC for ; Sat, 9 Sep 2023 20:17:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wZiPvZBmRV32oEmRUW40iqumqSFKdPSlxnSh+MCQl2g=; b=TAFOsF8ChvdjrK RGhKxPdbnuCK6xFq7PefWjoviekUFkbU1xcQ8SR7ZfYsgXb46FJm5C9PIM7TuqfvLlv1KhUCN/+AL SUq4uRvgOdl30hdZTsePCv9azNVfTiD9Y2nLnB9kkrHq+m6JlINqxJ6TbYUOSYymTPCbgrvGmhsXL e+kQ8tjNSzCbbMc8jkUQwX3clhN+QIYEma9T9NnKc2LmGdLqkdh5i7pwzTnn6TPgcw6cEN/m1MrP6 kpcwfVn5DyvrmB+6UqemhXUQu8ixRpnrOa0651wwIVorAO3LKzKRbIWc6aIrtakPk6EjlG37Fs6Cu htbbNEKx2nge+DyUw/vw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Op-00Fky5-03; Sat, 09 Sep 2023 20:17:47 +0000 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Oh-00FkqA-0W for linux-riscv@lists.infradead.org; Sat, 09 Sep 2023 20:17:41 +0000 Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailout.west.internal (Postfix) with ESMTP id 2D9BD3200805; Sat, 9 Sep 2023 16:17:38 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Sat, 09 Sep 2023 16:17:38 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sholland.org; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1694290657; x= 1694377057; bh=rUsqFVJjBVyLyCtvleKeuxMZhdTqpYL3EtsdFwHrfmk=; b=E N8Sx3QN/zwyT1uB2+ogFARNEBx5avj520tUSxakXtHkIRaKHXquCPWKjhWIzDZgG srtx3VyL9+CDt3cqFSDUG6MQF2i/sOudw4fvnTOxSxeyya6GP/oPZW2IYxCwRut2 40mmFiiqU6NNduAoRnoXLlYRKOSSRHcwtZ6GXPO3ZkS6e9fZ27yZrvOyB0euLz9y kj88QUz7fOXwIZUbxic69m6Kh2dLZ/WpeigR/X4t3feBkqX7h24Rk/4bXQEfm40M fQdX4a9Nz3LeVejqFaHuJXTUPRCg2thjwWhWK1mK2V6IgHXJdCUprlREPHV1mf+b tDiB8MPwA70ma9zm4JwUQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1694290657; x= 1694377057; bh=rUsqFVJjBVyLyCtvleKeuxMZhdTqpYL3EtsdFwHrfmk=; b=l R+Q/fYfNiHJVIzgyH8iPibJTTy6BLgjJwAJX9wbPM0FZjI/tTm4mQMH5s2MYqzG7 CoypenZlZDIJwwfOBTr01Buows1SWTkPrwjGvr26szHSjHxGbO+6QUhMwxCDrjqy X013ycgQWlbyxULqmNrU3Xfy+mJZL3U876cdL6urH/vRtDyoMEk99b46BbaZTwPE FPfJ+CPX3k0+3F9zmkQop8AxbAbmC2TrHdCRcFf3vELyKw6fJ4BqmE60GJNZdLXQ slewUndvm4x5TzH5Y1HEHVR6aif3RLAIEw78+QsrAXYpDFfgIf7KUkBJT8Srh0vt LdKCPxnON/hgTnyTp6hFQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudehledgudeghecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefurghm uhgvlhcujfholhhlrghnugcuoehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhgqeenuc ggtffrrghtthgvrhhnpedukeetueduhedtleetvefguddvvdejhfefudelgfduveeggeeh gfdufeeitdevteenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhg X-ME-Proxy: Feedback-ID: i0ad843c9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 9 Sep 2023 16:17:37 -0400 (EDT) From: Samuel Holland To: Palmer Dabbelt , Alexandre Ghiti , linux-riscv@lists.infradead.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Samuel Holland Subject: [PATCH 4/7] riscv: mm: Make asid_bits a local variable Date: Sat, 9 Sep 2023 15:16:32 -0500 Message-ID: <20230909201727.10909-5-samuel@sholland.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230909201727.10909-1-samuel@sholland.org> References: <20230909201727.10909-1-samuel@sholland.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230909_131739_228803_8559B23F X-CRM114-Status: UNSURE ( 7.73 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This variable is only used inside asids_init(). Signed-off-by: Samuel Holland --- arch/riscv/mm/context.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index b5170ac1b742..43a8bc2d5af4 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -20,7 +20,6 @@ DEFINE_STATIC_KEY_FALSE(use_asid_allocator); -static unsigned long asid_bits; static unsigned long num_asids; static atomic_long_t current_version; @@ -226,7 +225,7 @@ static inline void set_mm(struct mm_struct *prev, static int __init asids_init(void) { - unsigned long old; + unsigned long asid_bits, old; /* Figure-out number of ASID bits in HW */ old = csr_read(CSR_SATP); From patchwork Sat Sep 9 20:16:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13378332 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 3C713EE14AA for ; Sat, 9 Sep 2023 20:17:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SPcQBLt/HJwMxjpTpNVUr8zRoOYEZtoDJrHpOuIriC8=; b=OwH3haQJsLf4Dm b8NeIo6MxX4JWlSMnu9bKO6GQgeNuMsox4/q4QYVWFJ7F/uAVSav9MPYv/e2GNuJFDiX1ZNgT3bD3 72EwiTXUTqOoaI7/F3Wap2K2DZDJtAc4qM0j+PpTxApvrHUzQ0HmqKg8PyR+HyupDvJE8SXatWTuO z4LGcxLsXx3DkkzhWnmMT+mgeIP3uFk2no4Fk7jomb96/Vn5ESweQuFu821oCCzNMbS7dzUWkPBbp hTsAlMcM06zwGW95PILQQltyk462ZE0lG8qwPNHFfQDpSxnwIw94Bqo8J+Qd4tEVb2JVDMlBLNDmt g5O0atvmvAp7124rKoLA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Op-00Fkyc-2P; Sat, 09 Sep 2023 20:17:47 +0000 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Oj-00Fkrx-1h for linux-riscv@lists.infradead.org; Sat, 09 Sep 2023 20:17:43 +0000 Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.west.internal (Postfix) with ESMTP id 68E34320090E; Sat, 9 Sep 2023 16:17:40 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute1.internal (MEProxy); Sat, 09 Sep 2023 16:17:40 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sholland.org; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1694290659; x= 1694377059; bh=HyKF8F/MhKP+/+HfUHmiXOs6fZCNU+XAaKpYQdP8b5o=; b=J kDfNIKb5OoXzQdKkTgNCXeFNXCpiqW915uClF5iObI79vo7f9KW1Oh4HGJX2fdDc FPCHVatGnaFaAlAgyLyGtZzbsLIaQWHH7iPoANcK791yNmKKu5RVi6V/QynxemzS 9monY3DvscjJUv3HIne/CVYPeBIsoH6jlmGSlsqD26JgXo+SAFgzQmy36f3E2OU2 LQ9cK5jE9nFkIoViqQQHHBUq4luDwnBWdHCU9UzEAdtgKI1W82x3CkbZo1jLqHIY PKbuXclW1at0SLW9cBQEpGMPFwBs7o1D0p7KCJ5P8elIVm3US6fNhwdD3dssW/7e LqEL3+hsdeSyUd3ulfZKQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1694290659; x= 1694377059; bh=HyKF8F/MhKP+/+HfUHmiXOs6fZCNU+XAaKpYQdP8b5o=; b=N GLiUn6J/oadG0HJX2fg9RyrdTv40GynBxcSwJzMBTpgX9CssXqBIHR3YkQQTcvPM f/zVsR8nU1eDqhte1SDK47QoMsAmiFKU3cbnTCOxlMf85tywSHvFcicsnb90A/lX AmMRLVKdOTJy2Q1bbjaDYxbrZs+RWZVuhripR6HiIDThEFArWjRWs49+0fMS+/5J BIBL1TxmT25fKm4qStV573bJdKmt3adGucn1QNZm2giqSiQ/m58+wtk5AdPw4x+H B8ky8qdfvjGGulnOCMfE6kJLFdsjLxoBSd77bsbHctJlyNembZZg+ep7W6Kf2lpj X93SgqnkADSG/m/0yvoHQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudehledgudegiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefurghm uhgvlhcujfholhhlrghnugcuoehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhgqeenuc ggtffrrghtthgvrhhnpedukeetueduhedtleetvefguddvvdejhfefudelgfduveeggeeh gfdufeeitdevteenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfh hrohhmpehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhg X-ME-Proxy: Feedback-ID: i0ad843c9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 9 Sep 2023 16:17:39 -0400 (EDT) From: Samuel Holland To: Palmer Dabbelt , Alexandre Ghiti , linux-riscv@lists.infradead.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Samuel Holland Subject: [PATCH 5/7] riscv: mm: Preserve global TLB entries when switching contexts Date: Sat, 9 Sep 2023 15:16:33 -0500 Message-ID: <20230909201727.10909-6-samuel@sholland.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230909201727.10909-1-samuel@sholland.org> References: <20230909201727.10909-1-samuel@sholland.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230909_131741_607730_40F2787E X-CRM114-Status: GOOD ( 13.49 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org If the CPU does not support multiple ASIDs, all MM contexts use ASID 0. In this case, it is still beneficial to flush the TLB by ASID, as the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. This optimization is recommended by the RISC-V privileged specification: If the implementation does not provide ASIDs, or software chooses to always use ASID 0, then after every satp write, software should execute SFENCE.VMA with rs1=x0. In the common case that no global translations have been modified, rs2 should be set to a register other than x0 but which contains the value zero, so that global translations are not flushed. It is not possible to apply this optimization when using the ASID allocator, because that code must flush the TLB for all ASIDs at once when incrementing the version number. Signed-off-by: Samuel Holland --- arch/riscv/mm/context.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 43a8bc2d5af4..3ca9b653df7d 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -200,7 +200,7 @@ static void set_mm_noasid(struct mm_struct *mm) { /* Switch the page table and blindly nuke entire local TLB */ csr_write(CSR_SATP, virt_to_pfn(mm->pgd) | satp_mode); - local_flush_tlb_all(); + local_flush_tlb_all_asid(0); } static inline void set_mm(struct mm_struct *prev, From patchwork Sat Sep 9 20:16:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13378335 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0210FEE14C3 for ; Sat, 9 Sep 2023 20:17:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1HWfo0SJRskoTmjuHFnlCXmFzgOh+/H54sG1vdYVGIk=; b=QuVYj5dOZq7HTi 2G//K0ZU1njHsxLEgh2jFzjU10Gw5PCH5Yu0n0oDeuEXXfKDdsXf6n36RRLTJP27qMIMNyi74OF2J GODL4lq4xVTITzhhi0IAV98CKamt6Mic4mlWC+whUmMtpyZKciii3/K7tBpzxO7mCq899TlsWn4Bv 6PJk+5lrDoTwhTmQC90UGI9AwZ/8zXRn/1lFWBnRwZVFYjerHaYyzg2nStRRwQasiN1utCtzBJrX1 bLQd4j53KCQ7HIn1uIUt/IZ9hEUssHMwYBq74LdinsPe7B80fgcxs4ozXpDBO0apLVJdgKgLF0RpB NiVIvvEypB4ssK1I7deA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Os-00Fl2Z-2p; Sat, 09 Sep 2023 20:17:50 +0000 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Ol-00Fktp-27 for linux-riscv@lists.infradead.org; Sat, 09 Sep 2023 20:17:45 +0000 Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailout.west.internal (Postfix) with ESMTP id A23CD3200805; Sat, 9 Sep 2023 16:17:42 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute6.internal (MEProxy); Sat, 09 Sep 2023 16:17:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sholland.org; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1694290662; x= 1694377062; bh=vYgeei/BDcJ6bUB2DAGppW0ND32ofla8o2DCj1NxYPE=; b=K u0dd0RZHMaDqMns7iRitHO88BySItmXc1V/+Vb/OkmR3KGcO1DvaRAtq3OLHETze naLsbxraep105inN1mW8i3aVWoN2CHwYKGEBnzeDVwtB/CY825a4Z6xplHeKufSB j5nFyWllPSvSBWdcnnya2PQa8PQnYYZV8XJ5rJi+x/EQ026aTvbwXVV8VdsVpeGh K6pNLRO3moFbl1eU9gKfEAOCfyD/8MKFiMMKp6OiMj6Z78ZchXerkP5VT2pQbY1v Kah1XXspL8hkSGENcggXYwqoYZTnzsPBZM5B7wK0UGfXDstVHgvLEPFUuxcvWJ4X I7OZP8RqM2VGeFapjE9CA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1694290662; x= 1694377062; bh=vYgeei/BDcJ6bUB2DAGppW0ND32ofla8o2DCj1NxYPE=; b=J 6plQnuSCnXtX5IBcgF5KYZGLQQG5MhLcW0wynftAZ0np5C/3SOw1d4GtGQhHZoeA RUCoGTMdxPBabmUAs8z9fzJypFl0W39j4LSRzE/kj+YtSJ+y2/NJxCWOMlC+OiBh E06QkMy8GJfKVsumpoaI812sFBrDYMDbN7JgsvEuVRLEJ8/imdo2h3gHSfEr1mtS U5OdXOVeblb2nv0zlpwDBRQC818RhI7+SAipU2efh9earF6Cmks7jZbChNmnwBYD Vd/3hzPGOd+tI1EvQdpIoT/1VmuvUeJxT7/xF3v23ep25CoStRL7VhCvoOIJo699 CEAt1LarUDmRPpTBESL8w== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudehledgudegiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefurghm uhgvlhcujfholhhlrghnugcuoehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhgqeenuc ggtffrrghtthgvrhhnpedukeetueduhedtleetvefguddvvdejhfefudelgfduveeggeeh gfdufeeitdevteenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfh hrohhmpehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhg X-ME-Proxy: Feedback-ID: i0ad843c9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 9 Sep 2023 16:17:41 -0400 (EDT) From: Samuel Holland To: Palmer Dabbelt , Alexandre Ghiti , linux-riscv@lists.infradead.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Samuel Holland Subject: [PATCH 6/7] riscv: mm: Always flush a single MM context by ASID Date: Sat, 9 Sep 2023 15:16:34 -0500 Message-ID: <20230909201727.10909-7-samuel@sholland.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230909201727.10909-1-samuel@sholland.org> References: <20230909201727.10909-1-samuel@sholland.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230909_131743_777474_E2497DF9 X-CRM114-Status: GOOD ( 14.48 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Even if ASIDs are not supported, using the single-ASID variant of the sfence.vma instruction preserves TLB entries for global (kernel) pages. So it is always most efficient to use the single-ASID code path. Signed-off-by: Samuel Holland --- arch/riscv/include/asm/mmu_context.h | 2 - arch/riscv/include/asm/tlbflush.h | 11 +++-- arch/riscv/mm/context.c | 3 +- arch/riscv/mm/tlbflush.c | 68 ++++++---------------------- 4 files changed, 24 insertions(+), 60 deletions(-) diff --git a/arch/riscv/include/asm/mmu_context.h b/arch/riscv/include/asm/mmu_context.h index 7030837adc1a..b0659413a080 100644 --- a/arch/riscv/include/asm/mmu_context.h +++ b/arch/riscv/include/asm/mmu_context.h @@ -33,8 +33,6 @@ static inline int init_new_context(struct task_struct *tsk, return 0; } -DECLARE_STATIC_KEY_FALSE(use_asid_allocator); - #include #endif /* _ASM_RISCV_MMU_CONTEXT_H */ diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index e55831edfc19..ba27cf68b170 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -54,13 +54,18 @@ void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, #define flush_tlb_all() local_flush_tlb_all() #define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) +static inline void flush_tlb_mm(struct mm_struct *mm) +{ + unsigned long asid = cntx2asid(atomic_long_read(&mm->context.id)); + + local_flush_tlb_all_asid(asid); +} + static inline void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end) { - local_flush_tlb_all(); + flush_tlb_mm(vma->vm_mm); } - -#define flush_tlb_mm(mm) flush_tlb_all() #endif /* !CONFIG_SMP || !CONFIG_MMU */ /* Flush a range of kernel pages */ diff --git a/arch/riscv/mm/context.c b/arch/riscv/mm/context.c index 3ca9b653df7d..20057085ab8a 100644 --- a/arch/riscv/mm/context.c +++ b/arch/riscv/mm/context.c @@ -18,8 +18,7 @@ #ifdef CONFIG_MMU -DEFINE_STATIC_KEY_FALSE(use_asid_allocator); - +static DEFINE_STATIC_KEY_FALSE(use_asid_allocator); static unsigned long num_asids; static atomic_long_t current_version; diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 54c3e70ccd81..56c2d40681a2 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -6,15 +6,6 @@ #include #include -static inline void local_flush_tlb_range(unsigned long start, - unsigned long size, unsigned long stride) -{ - if (size <= stride) - local_flush_tlb_page(start); - else - local_flush_tlb_all(); -} - static inline void local_flush_tlb_range_asid(unsigned long start, unsigned long size, unsigned long stride, unsigned long asid) { @@ -51,62 +42,33 @@ static void __ipi_flush_tlb_range_asid(void *info) local_flush_tlb_range_asid(d->start, d->size, d->stride, d->asid); } -static void __ipi_flush_tlb_range(void *info) -{ - struct flush_tlb_range_data *d = info; - - local_flush_tlb_range(d->start, d->size, d->stride); -} - static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, unsigned long size, unsigned long stride) { + unsigned long asid = cntx2asid(atomic_long_read(&mm->context.id)); struct flush_tlb_range_data ftd; struct cpumask *cmask = mm_cpumask(mm); unsigned int cpuid; - bool broadcast; if (cpumask_empty(cmask)) return; cpuid = get_cpu(); /* check if the tlbflush needs to be sent to other CPUs */ - broadcast = cpumask_any_but(cmask, cpuid) < nr_cpu_ids; - if (static_branch_unlikely(&use_asid_allocator)) { - unsigned long asid = cntx2asid(atomic_long_read(&mm->context.id)); - - if (broadcast) { - if (riscv_use_ipi_for_rfence()) { - ftd.asid = asid; - ftd.start = start; - ftd.size = size; - ftd.stride = stride; - on_each_cpu_mask(cmask, - __ipi_flush_tlb_range_asid, - &ftd, 1); - } else - sbi_remote_sfence_vma_asid(cmask, - start, size, asid); - } else { - local_flush_tlb_range_asid(start, size, stride, asid); - } - } else { - if (broadcast) { - if (riscv_use_ipi_for_rfence()) { - ftd.asid = 0; - ftd.start = start; - ftd.size = size; - ftd.stride = stride; - on_each_cpu_mask(cmask, - __ipi_flush_tlb_range, - &ftd, 1); - } else - sbi_remote_sfence_vma(cmask, start, size); - } else { - local_flush_tlb_range(start, size, stride); - } - } - + if (cpumask_any_but(cmask, cpuid) < nr_cpu_ids) { + if (riscv_use_ipi_for_rfence()) { + ftd.asid = asid; + ftd.start = start; + ftd.size = size; + ftd.stride = stride; + on_each_cpu_mask(cmask, + __ipi_flush_tlb_range_asid, + &ftd, 1); + } else + sbi_remote_sfence_vma_asid(cmask, + start, size, asid); + } else + local_flush_tlb_range_asid(start, size, stride, asid); put_cpu(); } From patchwork Sat Sep 9 20:16:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Samuel Holland X-Patchwork-Id: 13378334 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 44B2AEE57CC for ; Sat, 9 Sep 2023 20:17:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=J9tl2dkAd2KJ4mjIvQH7L9dv0NC+/MYmn7ieoOaaNAA=; b=ZlEg3Vw6Aps48Q y0p43RHG0VVZ06UL/lUIVddZ4xqUQdlt9sau5CRgIRMxWcyhQn1+sZZjK6s1pQM1Ldn5W0+/LOR8C Jm2dEsg+UqedUYqPF658uGvZRiPCp757ZdOqQA09eJQyEmeOdCMVpsvQ3IV5BjeE/PBBw+AEt82mo J1gCPCZrp4cxNAu4BQ7qznItZFQeGf0ye/ia+iOx7wZu04ix99fLhX392qBZSzpPSa1fffYCgvdl3 bcZ4VFnZ1ndDIuknkGwxhWsyrIsEK7oR1siaczIlVNig8pGh0gXCMCZlTrhFNG3SUTCTpywltIAXh O4FmHKLrxsNL6Fe0WpGg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qf4Ov-00Fl5h-26; Sat, 09 Sep 2023 20:17:53 +0000 Received: from wout4-smtp.messagingengine.com ([64.147.123.20]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1qf4On-00FkwV-2s for linux-riscv@lists.infradead.org; Sat, 09 Sep 2023 20:17:47 +0000 Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailout.west.internal (Postfix) with ESMTP id DF3C4320090E; Sat, 9 Sep 2023 16:17:44 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute3.internal (MEProxy); Sat, 09 Sep 2023 16:17:45 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sholland.org; h= cc:cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm2; t=1694290664; x= 1694377064; bh=a7RQskjaETnUB7Hp/T92jpvvET5ms5jF4Eb+hZaDEyI=; b=J 8U6+zD6CiKtlWqKDAskkYFhuDtgIckiPB/7lksw946WKmDxpWUgYdbbEDm0RclGl vDY74H5Yr4DdxccX57XaVRToiLWYDmlTbO0luKq3/cIrsBDx9pxmD+O04EwVAJXy Z21t93Ly8W6i0Ahhc5B5/eFQ7QBQT3UpVbqQK0MNoer5pr3h4xjtVgL60X6P4i1H ivUx277V8FX//1JY4wlhJ2ZwxJLn1p4QtQQP8W7r9gDllhtb0BH9vrjQ8IHf9k7B RZYPilLZGpkEPRe42T2nppigHi+ANLbEsRK64ycR8vSZGjBdFJ9mIfhFOWLTATBk aUBt2R3LPMlA1XLxVcrZQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to:x-me-proxy:x-me-proxy :x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=1694290664; x= 1694377064; bh=a7RQskjaETnUB7Hp/T92jpvvET5ms5jF4Eb+hZaDEyI=; b=b M27lb4EWA9BmB3m23H9K7njzrwz1bmIeiC5n4OWhUv2bwrE3fqlPBd/kAKAq3I9T e5xwcraAUx0Gc+NwOYzt8ojKyH3/XxxiNv1oVipM8loatwq/R2656NXKOYmYKmNi O21F3BqBHB2BTq1h+mMfnaq1iypcjKCMUukjOpNMPVr32ts1eBC4LMBohZWu+Oj7 XIThvzOMEr/coKW8CccbNQsAKKcGQYFdbNpx0mDpETtmLShxDxs8IRZt1mJDC6tC FWSYgAAhfceLtWGniOn7cqvbhIGsHYCwX7xj3I/kK4U/xdM8i/oeAhkk4N5YvzQH ckFKYFs9xZRm+KPQViSKw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudehledgudegiecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkffojghfggfgsedtkeertdertddtnecuhfhrohhmpefurghm uhgvlhcujfholhhlrghnugcuoehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhgqeenuc ggtffrrghtthgvrhhnpedukeetueduhedtleetvefguddvvdejhfefudelgfduveeggeeh gfdufeeitdevteenucevlhhushhtvghrufhiiigvpedunecurfgrrhgrmhepmhgrihhlfh hrohhmpehsrghmuhgvlhesshhhohhllhgrnhgurdhorhhg X-ME-Proxy: Feedback-ID: i0ad843c9:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat, 9 Sep 2023 16:17:43 -0400 (EDT) From: Samuel Holland To: Palmer Dabbelt , Alexandre Ghiti , linux-riscv@lists.infradead.org Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Samuel Holland Subject: [PATCH 7/7] riscv: mm: Combine the SMP and non-SMP TLB flushing code Date: Sat, 9 Sep 2023 15:16:35 -0500 Message-ID: <20230909201727.10909-8-samuel@sholland.org> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230909201727.10909-1-samuel@sholland.org> References: <20230909201727.10909-1-samuel@sholland.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230909_131746_038988_7631C411 X-CRM114-Status: GOOD ( 11.82 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This allows non-SMP configurations to take advantage of improvements to the code in tlbflush.c, such as support for huge pages and flushing multiple-page ranges. Signed-off-by: Samuel Holland --- arch/riscv/include/asm/tlbflush.h | 31 ++++++++----------------------- arch/riscv/mm/Makefile | 5 +---- arch/riscv/mm/tlbflush.c | 7 ++++++- 3 files changed, 15 insertions(+), 28 deletions(-) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index ba27cf68b170..a947ae3afd28 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -33,13 +33,12 @@ static inline void local_flush_tlb_page_asid(unsigned long addr, { ALT_SFENCE_VMA_ADDR_ASID(addr, asid); } -#else /* CONFIG_MMU */ -#define local_flush_tlb_all() do { } while (0) -#define local_flush_tlb_page(addr) do { } while (0) -#endif /* CONFIG_MMU */ -#if defined(CONFIG_SMP) && defined(CONFIG_MMU) +#ifdef CONFIG_SMP void flush_tlb_all(void); +#else +#define flush_tlb_all() local_flush_tlb_all() +#endif void flush_tlb_mm(struct mm_struct *mm); void flush_tlb_page(struct vm_area_struct *vma, unsigned long addr); void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, @@ -49,24 +48,10 @@ void flush_tlb_range(struct vm_area_struct *vma, unsigned long start, void flush_pmd_tlb_range(struct vm_area_struct *vma, unsigned long start, unsigned long end); #endif -#else /* CONFIG_SMP && CONFIG_MMU */ - -#define flush_tlb_all() local_flush_tlb_all() -#define flush_tlb_page(vma, addr) local_flush_tlb_page(addr) - -static inline void flush_tlb_mm(struct mm_struct *mm) -{ - unsigned long asid = cntx2asid(atomic_long_read(&mm->context.id)); - - local_flush_tlb_all_asid(asid); -} - -static inline void flush_tlb_range(struct vm_area_struct *vma, - unsigned long start, unsigned long end) -{ - flush_tlb_mm(vma->vm_mm); -} -#endif /* !CONFIG_SMP || !CONFIG_MMU */ +#else /* CONFIG_MMU */ +#define local_flush_tlb_all() do { } while (0) +#define local_flush_tlb_page(addr) do { } while (0) +#endif /* CONFIG_MMU */ /* Flush a range of kernel pages */ static inline void flush_tlb_kernel_range(unsigned long start, diff --git a/arch/riscv/mm/Makefile b/arch/riscv/mm/Makefile index 9c454f90fd3d..64f901674e35 100644 --- a/arch/riscv/mm/Makefile +++ b/arch/riscv/mm/Makefile @@ -13,15 +13,12 @@ endif KCOV_INSTRUMENT_init.o := n obj-y += init.o -obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o +obj-$(CONFIG_MMU) += extable.o fault.o pageattr.o tlbflush.o obj-y += cacheflush.o obj-y += context.o obj-y += pgtable.o obj-y += pmem.o -ifeq ($(CONFIG_MMU),y) -obj-$(CONFIG_SMP) += tlbflush.o -endif obj-$(CONFIG_HUGETLB_PAGE) += hugetlbpage.o obj-$(CONFIG_PTDUMP_CORE) += ptdump.o obj-$(CONFIG_KASAN) += kasan_init.o diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 56c2d40681a2..587b3bb084b2 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -15,6 +15,7 @@ static inline void local_flush_tlb_range_asid(unsigned long start, local_flush_tlb_all_asid(asid); } +#ifdef CONFIG_SMP static void __ipi_flush_tlb_all(void *info) { local_flush_tlb_all(); @@ -41,12 +42,12 @@ static void __ipi_flush_tlb_range_asid(void *info) local_flush_tlb_range_asid(d->start, d->size, d->stride, d->asid); } +#endif static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, unsigned long size, unsigned long stride) { unsigned long asid = cntx2asid(atomic_long_read(&mm->context.id)); - struct flush_tlb_range_data ftd; struct cpumask *cmask = mm_cpumask(mm); unsigned int cpuid; @@ -54,9 +55,12 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, return; cpuid = get_cpu(); +#ifdef CONFIG_SMP /* check if the tlbflush needs to be sent to other CPUs */ if (cpumask_any_but(cmask, cpuid) < nr_cpu_ids) { if (riscv_use_ipi_for_rfence()) { + struct flush_tlb_range_data ftd; + ftd.asid = asid; ftd.start = start; ftd.size = size; @@ -68,6 +72,7 @@ static void __flush_tlb_range(struct mm_struct *mm, unsigned long start, sbi_remote_sfence_vma_asid(cmask, start, size, asid); } else +#endif local_flush_tlb_range_asid(start, size, stride, asid); put_cpu(); }