From patchwork Wed Feb 26 12:01:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13992172 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2C90EC021BF for ; Wed, 26 Feb 2025 12:01:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D9BC8280015; Wed, 26 Feb 2025 07:01:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D163B28001C; Wed, 26 Feb 2025 07:01:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 96F6028001B; Wed, 26 Feb 2025 07:01:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id EB637280015 for ; Wed, 26 Feb 2025 07:01:48 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D560A1613EC for ; Wed, 26 Feb 2025 12:01:46 +0000 (UTC) X-FDA: 83161956612.28.4818732 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf13.hostedemail.com (Postfix) with ESMTP id 7FB9120042 for ; Wed, 26 Feb 2025 12:01:44 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740571305; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=VwiS9ZEcAts7a8IbXYyi30Hso5bXjL0vgcDTbz5F65Y=; b=caM5jOLo2zN5jhLqr2XNhCj8tuXnSBEn3jxPGKEcHH5naIx4zf6iXIXEpZbEa+wwodrJAL xZ4WmPf/nkYEP5bggLrveVNxg0ZHOoAaMCAEuOcCjwgDtn7pdJ+FXG5eSJFR0m6mctpEj9 vqVvJcJj2V1gBhLIrzDR7jsu+TtvQAw= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=none; spf=pass (imf13.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740571305; a=rsa-sha256; cv=none; b=thQcY7a7K4bbEFIf1I10ID4Q3SBeYi/SY7hkKZj4/DUyvvUZqW108UUSV/81FyEN6lpZaD mQvOAbVRKEtYBdSRZozqHFzgpMcLH3mm+H21lsN5Gw4YwUVc8Q1jny2mtbq4B7QH4cfrR0 CN2yAg9plOr5U+1e41R4xq6aHcs4le4= X-AuditID: a67dfc5b-3e1ff7000001d7ae-d1-67bf02a6bbce From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, rjgolo@gmail.com Subject: [RFC PATCH v12 based on mm-unstable as of Feb 21, 2025 03/25] riscv/tlb: add APIs manipulating tlb batch's arch data Date: Wed, 26 Feb 2025 21:01:10 +0900 Message-Id: <20250226120132.28469-3-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250226120132.28469-1-byungchul@sk.com> References: <20250226113342.GB1935@system.software.com> <20250226120132.28469-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrCLMWRmVeSWpSXmKPExsXC9ZZnke4ypv3pBr8uSFrMWb+GzeLzhn9s Fl/X/2K2ePqpj8Xi8q45bBb31vxntTi/ay2rxY6l+5gsLh1YwGRxvPcAk8X8e5/ZLDZvmsps cXzKVEaL3z/msDnweXxv7WPx2DnrLrvHgk2lHptXaHlsWtXJ5rHp0yR2j3fnzrF7nJjxm8Xj /b6rbB5bf9l5NE69xubxeZNcAE8Ul01Kak5mWWqRvl0CV8bO1RMYC64JVSzduoipgfEVfxcj J4eEgInEv3cvGWHsZ3ves4PYbALqEjdu/GQGsUUEzCQOtv4BinNxMAssY5LYe6KBDSQhLFAn sXvjY1YQm0VAVeLN6c1gcV4BU4n+U49ZIIbKS6zecABsECfQoH+7f4MtEBJIlmhZ/5sFZKiE wG02iaY/i6EaJCUOrrjBMoGRdwEjwypGocy8stzEzBwTvYzKvMwKveT83E2MwLBeVvsnegfj pwvBhxgFOBiVeHgfnNmbLsSaWFZcmXuIUYKDWUmElzNzT7oQb0piZVVqUX58UWlOavEhRmkO FiVxXqNv5SlCAumJJanZqakFqUUwWSYOTqkGRt22/ftbTKxcGS9PbhBdtaKhPNREYsaG/8yn XTVPcZQ9DuluKovnUF6utEK9K8y+a8NuxbebP2Zqmx8WWDJd49aZ/3cfOrPKWbVUiB1c/T7D 0IS9MfhW3+oM/5CLLz+19m6JzeGtEtD6uML36+8fLgc/PZ6obie+2O7uhAcXdtmGPHIt8rhy WomlOCPRUIu5qDgRAFq0F5lnAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrNLMWRmVeSWpSXmKPExsXC5WfdrLuMaX+6wbMDfBZz1q9hs/i84R+b xdf1v5gtnn7qY7E4PPckq8XlXXPYLO6t+c9qcX7XWlaLHUv3MVlcOrCAyeJ47wEmi/n3PrNZ bN40ldni+JSpjBa/f8xhc+D3+N7ax+Kxc9Zddo8Fm0o9Nq/Q8ti0qpPNY9OnSewe786dY/c4 MeM3i8f7fVfZPBa/+MDksfWXnUfj1GtsHp83yQXwRnHZpKTmZJalFunbJXBl7Fw9gbHgmlDF 0q2LmBoYX/F3MXJySAiYSDzb854dxGYTUJe4ceMnM4gtImAmcbD1D1Cci4NZYBmTxN4TDWwg CWGBOondGx+zgtgsAqoSb05vBovzCphK9J96zAIxVF5i9YYDYIM4gQb92/0bbIGQQLJEy/rf LBMYuRYwMqxiFMnMK8tNzMwx1SvOzqjMy6zQS87P3cQIDNJltX8m7mD8ctn9EKMAB6MSD++D M3vThVgTy4orcw8xSnAwK4nwcmbuSRfiTUmsrEotyo8vKs1JLT7EKM3BoiTO6xWemiAkkJ5Y kpqdmlqQWgSTZeLglGpgZDaarrH8x1nXjrBzh1j523t3bN2yYur6m4t62OVecSxhFUxuudg5 j7X3dqFD20HfrIofHKxv0+eE8evw+zN+kDj7jUFQXW7GwSWmIl86Vyb0xAd2/mQRM1zzbruN tvrhOv1/hf5TDDw0k3dv28WlKKwTNaHcJCJslqPtRv/QjZlVp9U2XSlXYinOSDTUYi4qTgQA k/WaE04CAAA= X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 7FB9120042 X-Stat-Signature: uepapp96fun3x7z4dxyg9jstrbqdnxoc X-HE-Tag: 1740571304-202506 X-HE-Meta: U2FsdGVkX1+gIGMvuX/4q69IIjM+uFZdo+JZLGUt7sKWPmZ+g/YYrNXwY8kzvhhheM2Z3+j6fj/r5eDRUNUVHPemtBVLlkVFvKog/0mOctvu/tLQoLersm98yRSnYvc25DtgEn7Kayr6jZIpqTS+pIfTrsTbJ+zw1XGk9jZrHOK7sbr5TQxO9l6LFYcP0/kH/TcmJjVprtrqUhGnMT5IOyZxzEjDZEJRPWQgDDhGHbx5q77DL/FxU/O9RHFPEc7Y5gsPYE7kPA9CgfDc2C9xruGKYS8rv0hleE3hAA1EYvjcBomkBRB6sSh/LTOb7/B9R1az93RqPLdH4az+VOIdTssTZn8uGP13tOZ3wsmA9g+OZ6El5MhFCM8jGgEyexlQ81YMD9eE/8YCjA1dfvxPTgmaJSlB99+GP5l06lk5Fpfjzh3wCXzP0D6/qDksjm+SK4ztNv45B215zBY6ocbRaB4+2XESH+XkKTZWC9RU85DOREcw7z3W1Fm1rZuWRtalP+lnvkhsRuJOXAbEYiuRje96LkpAwLQ6w5LjY9KErZ61TUU+hx/SyW2eQKxGEZQIyyeMLc5+ZUvgtinNRKgbdrP2HUizq0k/f9l/NmvV4UqxPsIq19TDw47H6f29mCAxdwwhaVuVwy64adDorUfjkuOySZWIEdacH3/OraeoHyhp1yAhIXRD2Bf0X74i7Z9gBZxPctSE3DZN65Sc2yvDG5k2iYvxMmpdRHHvanWOjzqCiZvB68KfU7KIQvqh4YTm/Xap14Wid9ETChlO1TBRJGJcvUiI0dTbW8fMToKU+EKRn92WROzDny74qaZxT5kJR26+z79j/nqJDXgp9cFY/6rZZ/JRSSM6SAocDqlt8y2BjU8f6ugTRt8R4++VVPDRTBvFW7CyyXHhrdmYKB4ErwRAwp2ycE4rnJpXVDuarUTCEiY12PIkctMroOV5athL1mkjUMyIT5zprhZYgQ9 GYPN+hIi BTLRM4pfHwBlsVuITD8zzunGXucEJSDwD9r0r X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A new mechanism, LUF(Lazy Unmap Flush), defers tlb flush until folios that have been unmapped and freed, eventually get allocated again. It's safe for folios that had been mapped read only and were unmapped, since the contents of the folios don't change while staying in pcp or buddy so we can still read the data through the stale tlb entries. This is a preparation for the mechanism that needs to recognize read-only tlb entries by separating tlb batch arch data into two, one is for read-only entries and the other is for writable ones, and merging those two when needed. It also optimizes tlb shootdown by skipping CPUs that have already performed tlb flush needed since. To support it, added APIs manipulating arch data for riscv. Signed-off-by: Byungchul Park --- arch/riscv/include/asm/tlbflush.h | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/arch/riscv/include/asm/tlbflush.h b/arch/riscv/include/asm/tlbflush.h index ce0dd0fed7646..cecd8e7e2a3bd 100644 --- a/arch/riscv/include/asm/tlbflush.h +++ b/arch/riscv/include/asm/tlbflush.h @@ -8,6 +8,7 @@ #define _ASM_RISCV_TLBFLUSH_H #include +#include #include #include @@ -64,6 +65,33 @@ void arch_tlbbatch_add_pending(struct arch_tlbflush_unmap_batch *batch, void arch_flush_tlb_batched_pending(struct mm_struct *mm); void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); +static inline void arch_tlbbatch_clear(struct arch_tlbflush_unmap_batch *batch) +{ + cpumask_clear(&batch->cpumask); + +} + +static inline void arch_tlbbatch_fold(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + cpumask_or(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); + +} + +static inline bool arch_tlbbatch_need_fold(struct arch_tlbflush_unmap_batch *batch, + struct mm_struct *mm) +{ + return !cpumask_subset(mm_cpumask(mm), &batch->cpumask); + +} + +static inline bool arch_tlbbatch_done(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + return !cpumask_andnot(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); + +} + extern unsigned long tlb_flush_all_threshold; #else /* CONFIG_MMU */ #define local_flush_tlb_all() do { } while (0)