From patchwork Wed Feb 26 12:03:18 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13992199 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59FB7C021BF for ; Wed, 26 Feb 2025 12:04:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 617C6280035; Wed, 26 Feb 2025 07:03:52 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C51E280024; Wed, 26 Feb 2025 07:03:52 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3879A280034; Wed, 26 Feb 2025 07:03:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 120C7280024 for ; Wed, 26 Feb 2025 07:03:52 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C544EA0B31 for ; Wed, 26 Feb 2025 12:03:51 +0000 (UTC) X-FDA: 83161961862.03.A01B3EF Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf11.hostedemail.com (Postfix) with ESMTP id EB4254000F for ; Wed, 26 Feb 2025 12:03:49 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740571430; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=hxCrz9G+d7At/E4HL/oRu9TFVWuCM/ndy7Fucb7/da8=; b=wv4wsni9SbwGkhfJuDXdHrQeCCp/L8CNtlbvp/d3n5WPQ/TDyhXd8zp/gDIcMiy86WVRIr CxFJjqgxV/9oS31DpZtujhKSnmY7ORIHslDkj7ibW/mOY2QsLFLGu9COjFtBSJuaKFRkCA DcFtkCBqHcgg0v2TG31rLLzAROmvtd4= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1740571430; a=rsa-sha256; cv=none; b=bTQTn2Z+oDMWWcx+Nj3bWLk+qn3uQT2+apKg89cj1sVMafkP1EwE1GUevv6Ve6dY1fU+zx NS/RA0f5i/B29unhjaYzpjc1jAusAQPH47/20oVRKBedvs5tmQ6yBfUoPnrf2q+yTSr42L 7R43a4dvJrh5t+HLACRMIqmM6lv9bhw= X-AuditID: a67dfc5b-3e1ff7000001d7ae-14-67bf032243b6 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, rjgolo@gmail.com Subject: [RFC PATCH v12 based on v6.14-rc4 07/25] mm: introduce luf_ugen to be used as a global timestamp Date: Wed, 26 Feb 2025 21:03:18 +0900 Message-Id: <20250226120336.29565-7-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20250226120336.29565-1-byungchul@sk.com> References: <20250226113024.GA1935@system.software.com> <20250226120336.29565-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrELMWRmVeSWpSXmKPExsXC9ZZnoa4S8/50g55+DYs569ewWXze8I/N 4uv6X8wWTz/1sVhc3jWHzeLemv+sFud3rWW12LF0H5PFpQMLmCyO9x5gsph/7zObxeZNU5kt jk+Zymjx+8ccNgc+j++tfSweO2fdZfdYsKnUY/MKLY9NqzrZPDZ9msTu8e7cOXaPEzN+s3i8 33eVzWPrLzuPxqnX2Dw+b5IL4InisklJzcksSy3St0vgyphw4yVTwTvRinszt7I3MN4Q7GLk 5JAQMJE4NecpSxcjB5i9/3Y1SJhNQF3ixo2fzCC2iICZxMHWP+xdjFwczALLmCT2nmhgA0kI C2RKvOvewQ5iswioSpy8f5YdZA6vgKnExK2SEOPlJVZvOAA2hxNozqdpx8BahQSSJXb+/sME MlNC4DGbxNTWXWwQDZISB1fcYJnAyLuAkWEVo1BmXlluYmaOiV5GZV5mhV5yfu4mRmBIL6v9 E72D8dOF4EOMAhyMSjy8D87sTRdiTSwrrsw9xCjBwawkwsuZuSddiDclsbIqtSg/vqg0J7X4 EKM0B4uSOK/Rt/IUIYH0xJLU7NTUgtQimCwTB6dUA+Oahx8ZVkjfP87m9y1v38kXD7bw3Gy/ rmDtPnvejXMfeMtWqfoIvlnZemk73/eJ3uvntWwT6d/t9TWm6syeisqvKt/XGrQHLj1bdMen 7kTHfG23LcuK9G1EFFwP+ei6lnZ+e1ayOf7xqoOdr8Quz4lMaZyjsf9HQoTZch+zV/78Xy71 7u9kviSpxFKckWioxVxUnAgA1zaIxGUCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrJLMWRmVeSWpSXmKPExsXC5WfdrKvEvD/dYPYyeYs569ewWXze8I/N 4uv6X8wWTz/1sVgcnnuS1eLyrjlsFvfW/Ge1OL9rLavFjqX7mCwuHVjAZHG89wCTxfx7n9ks Nm+aymxxfMpURovfP+awOfB7fG/tY/HYOesuu8eCTaUem1doeWxa1cnmsenTJHaPd+fOsXuc mPGbxeP9vqtsHotffGDy2PrLzqNx6jU2j8+b5AJ4o7hsUlJzMstSi/TtErgyJtx4yVTwTrTi 3syt7A2MNwS7GDk4JARMJPbfru5i5ORgE1CXuHHjJzOILSJgJnGw9Q97FyMXB7PAMiaJvSca 2EASwgKZEu+6d7CD2CwCqhIn759lB5nDK2AqMXGrJEhYQkBeYvWGA2BzOIHmfJp2DKxVSCBZ YufvP0wTGLkWMDKsYhTJzCvLTczMMdUrzs6ozMus0EvOz93ECAzQZbV/Ju5g/HLZ/RCjAAej Eg/vgzN704VYE8uKK3MPMUpwMCuJ8HJm7kkX4k1JrKxKLcqPLyrNSS0+xCjNwaIkzusVnpog JJCeWJKanZpakFoEk2Xi4JRqYKw5f6xrl+mxn51MLNXMB6+b73u+Te+MwY/F2pKOi5bnz798 MyvNfoLhtPia193Tzs0SrP6YebGw0t3wZm764Wu/n2wNVTopOeMay4vTSn2fKzcvFnvh+C7l wI52WUbtBaFPArZkuinznXnhLy751laTSX7XoVgX3/xn2k19IUnsXdsX+tjzVCmxFGckGmox FxUnAgD9F61dTAIAAA== X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: EB4254000F X-Stat-Signature: e4y75otqkmfc54oyq9hdser5h54q7je5 X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740571429-18902 X-HE-Meta: U2FsdGVkX18NXlmnn1SvdIfslQJUWEF7WZ3EREXJhnVT0KMzpYLVeTlQ+0aXQBDNXuQe2UAXq3jpC/VhZ1sx4dnssuvtpeXDTmSBnZPPmQuwWFVTZ4UxrxmItwpaYAJlKhOlTvIxxdfO8NLruq29N2JxnMC6fUfxXHsPcxqozgZy+STszlMgsFP1JY9VYYl/p9+n+XeaKfqpAF1mDJxFUGePMPa1wMkDI2oMaQBc9+cVR2orf467VAtUBApu2PI3P8+B919Y6HPnZNOJesCDMlaWexIN+PPRzIzeeCra9+UGsubIRZmLRvZp7WR2YnM+i1ZaAJycV4ZRQNA0k7b8xSafV7m17iVjvqI+BCACIe9LLD+jxGpEwY9t4AR86EcVEGucbKF9e0tL0pecXyjIFc8WcmehK7eq9B69yNNwhdm5D4/h0qWQi/5jGp/Wxie9kFMz4PsvPfnqRs5bAHNnu+46rww6g4t+8Er9Js599iwOW5DOdQqOAZ0FrIm/mDjdJevrSvlTnK2+viYfHqMcgEY/K1ew0rvvAETI6JB+En/Srt9aHUv4xrHOEMYTxAZHIwzVXlf5zGt0EAoDVskDIO5c0p0hgd0QNqLh8fTbMzrKMuYXi70WHZ+1YMWOwRHEdJjID4QWCH5jjXDcJQtGFuQjOFh120Aq5C9W8Yh4Hc0Off9mfTHWGpI9oWDIyn84idTjNgqGfPVcwxRy54+fj1Urpnzobfj/CLUt6PQKOX3u9VkevUulebOIXTZvoLgCf05jxtWF4YO3VtrFUNL9Bpg+TeJe6y2q4Ue8NlRCjtkshDen3zMXorkS+arXwlwPxCR6LikgtNiT8lzD9iP624RKcjRoR5NATNu3QRww3ZyJyzOWf70k8OzrhgM1jmw3o6VIMITqEKZQdyZ6utraivorUoeBi7LByfec6LqDeQIPyhYGunn2XSoAbeZ1f2FM4EpX2MbgcgblGla+EBe LmLjya67 DAJ/83ELK0WjOMbwjaPcj4Kqz3OPxN+dE1YD2C6p65TkAb87ryefiLk9d6Ts34ZBk++9rR8Pg4bDWrC8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Functionally, no change. This is a preparation for luf mechanism that needs to evaluate the temporal sequence of events to determine whether tlb flush required has been done on each CPU. To achieve that, this patch introduced a generation number, luf_ugen, and a few APIs manipulating the number. It's worth noting the number is designed to wraparound so care must be taken when using it. Signed-off-by: Byungchul Park --- include/linux/mm.h | 34 ++++++++++++++++++++++++++++++++++ mm/rmap.c | 22 ++++++++++++++++++++++ 2 files changed, 56 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 7b1068ddcbb70..8c3481402d8cb 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -4155,4 +4155,38 @@ int arch_get_shadow_stack_status(struct task_struct *t, unsigned long __user *st int arch_set_shadow_stack_status(struct task_struct *t, unsigned long status); int arch_lock_shadow_stack_status(struct task_struct *t, unsigned long status); +#if defined(CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH) +/* + * luf_ugen will start with 2 so that 1 can be regarded as a passed one. + */ +#define LUF_UGEN_INIT 2 + +static inline bool ugen_before(unsigned long a, unsigned long b) +{ + /* + * Consider wraparound. + */ + return (long)(a - b) < 0; +} + +static inline unsigned long next_ugen(unsigned long ugen) +{ + if (ugen + 1) + return ugen + 1; + /* + * Avoid invalid ugen, zero. + */ + return ugen + 2; +} + +static inline unsigned long prev_ugen(unsigned long ugen) +{ + if (ugen - 1) + return ugen - 1; + /* + * Avoid invalid ugen, zero. + */ + return ugen - 2; +} +#endif #endif /* _LINUX_MM_H */ diff --git a/mm/rmap.c b/mm/rmap.c index 2de01de164ef0..ed345503e4f88 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -634,6 +634,28 @@ struct anon_vma *folio_lock_anon_vma_read(const struct folio *folio, } #ifdef CONFIG_ARCH_WANT_BATCHED_UNMAP_TLB_FLUSH + +/* + * This generation number is primarily used as a global timestamp to + * determine whether tlb flush required has been done on each CPU. The + * function, ugen_before(), should be used to evaluate the temporal + * sequence of events because the number is designed to wraparound. + */ +static atomic_long_t __maybe_unused luf_ugen = ATOMIC_LONG_INIT(LUF_UGEN_INIT); + +/* + * Don't return invalid luf_ugen, zero. + */ +static unsigned long __maybe_unused new_luf_ugen(void) +{ + unsigned long ugen = atomic_long_inc_return(&luf_ugen); + + if (!ugen) + ugen = atomic_long_inc_return(&luf_ugen); + + return ugen; +} + /* * Flush TLB entries for recently unmapped pages from remote CPUs. It is * important if a PTE was dirty when it was unmapped that it's flushed