From patchwork Wed Apr 17 07:18:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13632930 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1102BC4345F for ; Wed, 17 Apr 2024 07:19:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A127C6B008A; Wed, 17 Apr 2024 03:19:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9B2416B0093; Wed, 17 Apr 2024 03:19:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7408C6B008A; Wed, 17 Apr 2024 03:19:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 3983B6B008A for ; Wed, 17 Apr 2024 03:19:07 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C2D6640D6D for ; Wed, 17 Apr 2024 07:19:06 +0000 (UTC) X-FDA: 82018172292.11.BFB187F Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf16.hostedemail.com (Postfix) with ESMTP id F246B180015 for ; Wed, 17 Apr 2024 07:19:03 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf16.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713338345; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=DXRIJdBTI4JMeLiUqF/DwOHmi4232WnymtijAP7MU4U=; b=lF6hKy1OlqMvWXBwG01elxoHGygiT8W8JbpT34A7Ip3QnJgLvv13KpL568hFtiEYTFraTq ZcZBVhD3CTDu5TMujahmdKdxl1iyW5hNk25Do6zuKLIAHE6dcRPif5VFzWGyj/gQs/JjeP 06/uptAdzRue/qClI9dQKQkl1sdFMjg= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf16.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713338345; a=rsa-sha256; cv=none; b=vine2tI1gVyuH+OO/jPqhL0QGy9dS7/mDTGFh8/FpjApDeGeOqJuEhDwHgdq/Pp+wuChkx tH/bzIR+YrmrfBKkoDeH6ekWgJdfHOSHdywNpchEIJxWiA5f5KO0CrvmbcAS7mnkbzbYab nYVrtETRTSuZQw/CtR3xGNExzrG56PE= X-AuditID: a67dfc5b-d6dff70000001748-92-661f77e2e210 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [PATCH v9 1/8] x86/tlb: add APIs manipulating tlb batch's arch data Date: Wed, 17 Apr 2024 16:18:40 +0900 Message-Id: <20240417071847.29584-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240417071847.29584-1-byungchul@sk.com> References: <20240417071847.29584-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrGLMWRmVeSWpSXmKPExsXC9ZZnke6jcvk0g2eL+S3mrF/DZvF5wz82 ixcb2hktvq7/xWzx9FMfi8XlXXPYLO6t+c9qcX7XWlaLHUv3MVlcOrCAyeJ47wEmi/n3PrNZ bN40ldni+JSpjBa/fwAVn5w1mcVBwON7ax+Lx85Zd9k9Fmwq9di8Qstj8Z6XTB6bVnWyeWz6 NInd4925c+weJ2b8ZvGYdzLQ4/2+q2weW3/ZeTROvcbm8XmTXABfFJdNSmpOZllqkb5dAldG w70XLAVr+Cp6P81naWCcydPFyMkhIWAi8W72FGYY++nLfjCbTUBd4saNn2C2iICZxMHWP+wg NrPAXSaJA/1sILawgI/EqZl9YDaLgKpE7/9zYPW8AqYS8xfeZ4KYKS+xesMBsDgn0JzNzWvA 6oWAar6f+AhkcwHVvGaTWPu7hRGiQVLi4IobLBMYeRcwMqxiFMrMK8tNzMwx0cuozMus0EvO z93ECAz9ZbV/oncwfroQfIhRgINRiYfXIEouTYg1say4MvcQowQHs5IIb4uwbJoQb0piZVVq UX58UWlOavEhRmkOFiVxXqNv5SlCAumJJanZqakFqUUwWSYOTqkGRgu7/neSpzNfWF7d2Lk2 dC/DHIZZX3gaOacKqexSjTioVPPO7oNb4kpmpoPPZPmPhSwKClt2N5NzcYILa1bwnk1JvQnR ovdur2zmZ3YW+8EqyfmLceL7ULPpwtsPb77gFuQe0RvINuHedual/xecen1k2qELr+ISdhfI VCxmWxztHG+ykee+jxJLcUaioRZzUXEiANheHqx5AgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrLLMWRmVeSWpSXmKPExsXC5WfdrPuoXD7NYNZCFos569ewWXze8I/N 4sWGdkaLr+t/MVs8/dTHYnF47klWi8u75rBZ3Fvzn9Xi/K61rBY7lu5jsrh0YAGTxfHeA0wW 8+99ZrPYvGkqs8XxKVMZLX7/ACo+OWsyi4Ogx/fWPhaPnbPusnss2FTqsXmFlsfiPS+ZPDat 6mTz2PRpErvHu3Pn2D1OzPjN4jHvZKDH+31X2TwWv/jA5LH1l51H49RrbB6fN8kF8Edx2aSk 5mSWpRbp2yVwZTTce8FSsIavovfTfJYGxpk8XYycHBICJhJPX/Yzg9hsAuoSN278BLNFBMwk Drb+YQexmQXuMkkc6GcDsYUFfCROzewDs1kEVCV6/58Dq+cVMJWYv/A+E8RMeYnVGw6AxTmB 5mxuXgNWLwRU8/3ER7YJjFwLGBlWMYpk5pXlJmbmmOoVZ2dU5mVW6CXn525iBAbysto/E3cw frnsfohRgINRiYfXIEouTYg1say4MvcQowQHs5IIb4uwbJoQb0piZVVqUX58UWlOavEhRmkO FiVxXq/w1AQhgfTEktTs1NSC1CKYLBMHp1QDo8PJ4HcKjJfDDhsqXSyp/FZhuOwrY1C3surV C1NU/jpKbZdVnZJidkB0wrQXtVX+rya5Lb7RO5H/0cov/KffX2D235J4t9zZU7Rh326/o0ur 8n85+M4Rs3q2LVbPe1Po81+dnhMeLlivkiIV/kV4u2/R/oVCWvcvtzp7tGo4nZmkYnDi+/q6 d0osxRmJhlrMRcWJALm+JTdgAgAA X-CFilter-Loop: Reflected X-Stat-Signature: tc7obmce6wo1s5m9yjqha4sahmg9fjmq X-Rspamd-Queue-Id: F246B180015 X-Rspamd-Server: rspam02 X-Rspam-User: X-HE-Tag: 1713338343-895941 X-HE-Meta: U2FsdGVkX1/d54r4CaYxj2Nsshl3zXYOGVhUuxTaNVSyLhWYHqK6dd0hxiS4ukRK7Pyq58SwKPGD2jhgdDt5l5i7Re0CC9jgBLfBO8TRbLAT7htD0+46OA2GIdgK/JLKaKz4wPv2tPcH+yX5h3qY9mu2M/fyfbj0X4cdIWpVkS/SWFejFq+V25WnmZ0ibUrTA7WTtjg+wDQfapyJjCFBuGDWyF3gfnSEraLLv7cP382/viJHGv6sESTvvOWZrAr4EQaQcPkd/NRbTDha1TC0LCbh5hfZh3N2ZlOxAN15t9lT/0/k0xoxHKnv0O/PipQL4tUOpZWrLIf9t+CqZjxkZAzP3D2y87ToBGsMQ6vmwEHFQ82Mf+gW2L9sDZZ0G0Eshjm5sGesOq8MkEoZvT0DlY1hJSV0xHH9jwvuUhg+g0NW8Up75vpO1OslMRcKr2MLxhEAOUAUWF40s+srFHxg47AQ3BYQdLt6XI5Zv2rU4D6OaOnKTqUID0H7DOXwm7HcAT+IWonq9n3aWXjV7gxwR2+RxHyRZl4ry3VmInVgBI16E9aXskPKJozZ/S1klukhslAXcKEUYEVEFV5ThBUOOiA6ptFPc6CfXtBmeSw27iKvbVZvFOxmtVfof2LsaSwIU/kSEDQTssSxBAxrlz69zwtYE3YfZ3x36iAOnbUnXrdMLk9amGptHnbpAPE86CC2eb8mQLZWLmhr/QLFtVkfRx7NlQ5pCXdk2ziY0pf6fNRji5yRTByzQzwF1Kyotf96nB9eQvenne4FX83B7U/c4Hrx2YG7ovOCvU/xngOKyM9VXVNVmult9YD5r3x6hJa9oaTaJ9N64U2VdTcu4cA8YSd7NQ48aW/QXLtW3EXqlO+kuvcvdIprIR7Cz6q8tTcsRIJrLhMnGVBi0Ks7JcZhyZ8n2ve4A/DnZFflcBWP3QXL6h2Xp9n0gQEeY83GoRJLAbgp3jE2bjzEvyHg33h 2O0ZwXyZ wNOFGjsvB6/+4e5dhmkBOvMXTvA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation for migrc mechanism that needs to recognize read-only tlb entries during migration by separating tlb batch arch data into two, one is for read-only entries and the other is for writable ones, and merging those two when needed. Migrc also needs to optimize tlb shootdown by skipping CPUs that have already performed tlb flush needed for a while. To support it, added APIs manipulating arch data for x86. Signed-off-by: Byungchul Park --- arch/x86/include/asm/tlbflush.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 25726893c6f4..a14f77c5cdde 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -293,6 +294,23 @@ static inline void arch_flush_tlb_batched_pending(struct mm_struct *mm) extern void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); +static inline void arch_tlbbatch_clear(struct arch_tlbflush_unmap_batch *batch) +{ + cpumask_clear(&batch->cpumask); +} + +static inline void arch_tlbbatch_fold(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + cpumask_or(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + +static inline bool arch_tlbbatch_done(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + return !cpumask_andnot(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + static inline bool pte_flags_need_flush(unsigned long oldflags, unsigned long newflags, bool ignore_access)