From patchwork Wed Feb 26 12:03:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13992196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0DF83C021BF for ; Wed, 26 Feb 2025 12:03:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7273128001A; Wed, 26 Feb 2025 07:03:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D7D128001E; Wed, 26 Feb 2025 07:03:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5500928001A; Wed, 26 Feb 2025 07:03:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2EE1728001E for ; Wed, 26 Feb 2025 07:03:50 -0500 (EST) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id EF8E952502 for ; Wed, 26 Feb 2025 12:03:49 +0000 (UTC) X-FDA: 83161961778.24.4D25C12 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf27.hostedemail.com (Postfix) with ESMTP id EB4F940008 for ; Wed, 26 Feb 2025 12:03:47 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=none; spf=pass (imf27.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740571428; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=G30xGqy1s/coEL/U1j52C+U7PjQw0ciNR8FppuYW4fo=; b=LNl4AiiT7+4Ie2txyI7ImSI66g/8N4CgehROBN8fUikrpUT2bZ8rc/0idvKbyAOrWtTbh0 Lzw0/5OWlYojHh3CuEA+5L9yCK12YdqKPTtfQWpvZvGl5cIusFU2DZmhPEooeAf6qEWuA7 7BAFtOl+xfmWuuZ1O88wnPtpIfJxhkk= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=none; spf=pass (imf27.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740571428; a=rsa-sha256; cv=none; b=esdVuiXsowSrWuwfMogvWDnRm6FMCY/CqR0jHUh60/KruOZD9Qmx8FiIVve+ma7edCYknH 5A+24oFTRo8z9oYAYXSOAHHYmd/ly18z6xVOaHSCTjTo0q86XvaFlJWnk66ZuGL4pcfLlm fzN/vSI4Dga5216lYtUnBbkkCwERKeM= X-AuditID: a67dfc5b-3e1ff7000001d7ae-00-67bf0322bf9e From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, rjgolo@gmail.com Subject: [RFC PATCH v12 based on v6.14-rc4 03/25] riscv/tlb: add APIs manipulating tlb batch's arch data Date: Wed, 26 Feb 2025 21:03:14 +0900 Message-Id: <20250226120336.29565-3-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250226120336.29565-1-byungchul@sk.com> References: <20250226113024.GA1935@system.software.com> <20250226120336.29565-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrMLMWRmVeSWpSXmKPExsXC9ZZnka4S8/50g46pkhZz1q9hs/i84R+b xdf1v5gtnn7qY7G4vGsOm8W9Nf9ZLc7vWstqsWPpPiaLSwcWMFkc7z3AZDH/3mc2i82bpjJb HJ8yldHi9485bA58Ht9b+1g8ds66y+6xYFOpx+YVWh6bVnWyeWz6NInd4925c+weJ2b8ZvF4 v+8qm8fWX3YejVOvsXl83iQXwBPFZZOSmpNZllqkb5fAlbFj7V2WgmtCFU8e/2NqYHzF38XI ySEhYCIxYdMHNhh78YnTYDabgLrEjRs/mUFsEQEziYOtf9i7GLk4mAWWMUnsPdEAViQskCHR f2I/O4jNIqAqMbF1L2sXIwcHr4CpxPcN5hAz5SVWbzgANocTaM6nacfAWoUEkiV2/v7DBDJT QuA+m8TzvvesEA2SEgdX3GCZwMi7gJFhFaNQZl5ZbmJmjoleRmVeZoVecn7uJkZgUC+r/RO9 g/HTheBDjAIcjEo8vA/O7E0XYk0sK67MPcQowcGsJMLLmbknXYg3JbGyKrUoP76oNCe1+BCj NAeLkjiv0bfyFCGB9MSS1OzU1ILUIpgsEwenVANj/Hzjbvdwn806nk28n7711UZUe6T+yX6x 1U3zvFFDU/UMPt9os5JAy4g/JeULc3/L/FJdcfjPBRPh97u+bF150p9D7dhxhTO2eoyZ6w57 zLeUyHV+LrRj8kGrg8u282yt8PSdId+u5Cs0fV+u4UfG1zlir16omG/9+d/WmDW4XO5krN+D z1JKLMUZiYZazEXFiQCHrFxAZgIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrFLMWRmVeSWpSXmKPExsXC5WfdrKvEvD/d4M4KAYs569ewWXze8I/N 4uv6X8wWTz/1sVgcnnuS1eLyrjlsFvfW/Ge1OL9rLavFjqX7mCwuHVjAZHG89wCTxfx7n9ks Nm+aymxxfMpURovfP+awOfB7fG/tY/HYOesuu8eCTaUem1doeWxa1cnmsenTJHaPd+fOsXuc mPGbxeP9vqtsHotffGDy2PrLzqNx6jU2j8+b5AJ4o7hsUlJzMstSi/TtErgydqy9y1JwTaji yeN/TA2Mr/i7GDk5JARMJBafOM0GYrMJqEvcuPGTGcQWETCTONj6h72LkYuDWWAZk8TeEw1g RcICGRL9J/azg9gsAqoSE1v3snYxcnDwCphKfN9gDjFTXmL1hgNgcziB5nyadgysVUggWWLn 7z9MExi5FjAyrGIUycwry03MzDHVK87OqMzLrNBLzs/dxAgM0WW1fybuYPxy2f0QowAHoxIP 74Mze9OFWBPLiitzDzFKcDArifByZu5JF+JNSaysSi3Kjy8qzUktPsQozcGiJM7rFZ6aICSQ nliSmp2aWpBaBJNl4uCUamC8b7pDp87Q/FDLVWEh/bvTGf4f+7Buus8uhoOLSqabT5zR8Ell TuDG8pSY6LMFwbziKy99neTONN9DR3t69BOxd3Zsz+yk1t2R5eS167+5xPCdXavwoX2fPDme WK3Y95iheek0j/7c2L9GW7qfzii4V8gsF/KX805rTmbHIs+qIrYIyR/HVxgpsRRnJBpqMRcV JwIAI6pn4k0CAAA= X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: EB4F940008 X-Stat-Signature: jj98fjrxtty9rfhj1n5em1drc7azyd4j X-HE-Tag: 1740571427-458276 X-HE-Meta: U2FsdGVkX1+DnxQ1OHju2HMAKrpesPdJ4mr3BSOXre5yxlfZcgu6UuGvdviVKHYbjOrG4Txn19zaZm5VrKgGT3Z9BilFpHcaKN8I8LwKEDnbnxHFO9INjmsMMRMX3SW9Fwx6heVmp3JCGfS2a0mc0wDpZ1RHIzOknUECH0z4mKa2ty8JB+Y6OHMYgZPYs7VLV7YfQ+LWKfbFo49hyxPJE7J0yD4P2gdcF5ufSqA3jv1NFbt0PyZJshdOw4EwYhaQMYU7NxE6/5dyRHq3owIt/2a4lWkmvzGp5IH49eM1UTSA00nZH7GedrhplybbSb9pDts9ymaoIFilHInJOOB7s6en0DCXW+RSu1buATd4VpSfBAENqWDb6A4+KH3SmS4O2M2aCyGIWglOSa+YtJBSknF8AbY6uqNZnI8YnbHkegIq94A0bsqXMZaLIAY4Dh3NrAVRwzwOqIfBYhQG1lCvidZg+oM6KGJJmWW/RrMjFSxq6NolJX9dZ+ZSM45fnii9ojxpe6P4NE7veWQPKCW5mVXOVoJ4CpCYrMYhCUu86099uWxhr00hcYU55wHMiBsJu5JEDj5Lyy8NfRB7ALMWahomM9Jfbo549KJdQIZfVhsbEF6lrAmt6UcQRdcMEhhc2k1R+SrcLNLtAs0i5K/svej2rcT37vQdXe/CU/kMEp0YDE//h0AATffymCWxJ8nm5Cag+BfEd0c5nc8Cnu4jjp2oJBhAt83k01FJnO/8SKkgTOTdrm8Mk66cfTErvB0c9jILaZ7S/Bbhwj+pCPBP2EeIxkz2r0o3r9fMiIbkh6GXj+cB/tdwgIQHy2LgQVHPYptnA21NFJv/Z9L47hJ4gwnmJ4qe07x22B0hh9G4v2nuIpUWFXOMlYXN3MvIw7oJhr0J07u56KCA5UzZy+xb2Oa6J7Hdh7aSFrU+4kLP1XrNRMbfI1nB2544yXqv80ppZ8TqHVQRCbmZJ7Tu1rv HUp7ucqU 5TrY5cdaCYKTsB79zdaHaQw/KZcdMBcSDaQLw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A new mechanism, LUF(Lazy Unmap Flush), defers tlb flush until folios that have been unmapped and freed, eventually get allocated again. It's safe for folios that had been mapped read only and were unmapped, since the contents of the folios don't change while staying in pcp or buddy so we can still read the data through the stale tlb entries. This is a preparation for the mechanism that needs to recognize read-only tlb entries by separating tlb batch arch data into two, one is for read-only entries and the other is for writable ones, and merging those two when needed. It also optimizes tlb shootdown by skipping CPUs that have already performed tlb flush needed since. To support it, added APIs manipulating arch data for riscv. Signed-off-by: Byungchul Park --- arch/riscv/include/asm/tlbflush.h | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index 72e5599349529..1dc7d30273d59 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -8,6 +8,7 @@ #define _ASM_RISCV_TLBFLUSH_H #include +#include #include #include @@ -65,6 +66,33 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); +static inline void arch_tlbbatch_clear(struct arch_tlbflush_unmap_batch *batch) +{ + cpumask_clear(&batch->cpumask); + +} + +static inline void arch_tlbbatch_fold(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + cpumask_or(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); + +} + +static inline bool arch_tlbbatch_need_fold(struct arch_tlbflush_unmap_batch *batch, + struct mm_struct *mm) +{ + return !cpumask_subset(mm_cpumask(mm), &batch->cpumask); + +} + +static inline bool arch_tlbbatch_done(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + return !cpumask_andnot(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); + +} + extern unsigned long tlb_flush_all_threshold; #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0)