From patchwork Thu Apr 18 06:15:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13634189 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CC8FC4345F for ; Thu, 18 Apr 2024 06:15:56 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6DDCF6B007B; Thu, 18 Apr 2024 02:15:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 667306B0082; Thu, 18 Apr 2024 02:15:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4DEA46B0087; Thu, 18 Apr 2024 02:15:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 3026D6B0082 for ; Thu, 18 Apr 2024 02:15:55 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E3B781C02A2 for ; Thu, 18 Apr 2024 06:15:54 +0000 (UTC) X-FDA: 82021641828.28.CE0F0CE Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf05.hostedemail.com (Postfix) with ESMTP id 23E21100004 for ; Thu, 18 Apr 2024 06:15:50 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf05.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1713420953; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=DXRIJdBTI4JMeLiUqF/DwOHmi4232WnymtijAP7MU4U=; b=B3w+FELTaXwsfnyUCoB6pXkclkeKtY+fw+8tyZsegQfcnNxBQuBHZSc2Xl+SFNq14v2uN7 SDOooYsGGkl9o6DzoJ0JCIYHM6Td1fEpSsWpr2hvSNyD886XiFI13gRitzULT/h4yZSEAt D+3AH82DXduUKiHxXQU5q45DnaOzGFY= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf05.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1713420953; a=rsa-sha256; cv=none; b=cTon9kJk4z2IlZrdJO3K3abiHZpeiBD4qjUg0Hpzqmhzg/GLJ5kJoSYZMYO0ebmZfZSRbR K6D/CYg8oH+CFIq/nyCSj96qTlu+S5YgdAV7jIOhhvagCtoGgrkboVxZSxvFT0pF3NtawB SQ3pvHMnIkF89zI+YX3cz8WdqehwNy4= X-AuditID: a67dfc5b-d6dff70000001748-fd-6620ba924102 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [PATCH v9 rebase on mm-unstable 1/8] x86/tlb: add APIs manipulating tlb batch's arch data Date: Thu, 18 Apr 2024 15:15:29 +0900 Message-Id: <20240418061536.11645-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240418061536.11645-1-byungchul@sk.com> References: <20240418061536.11645-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrCLMWRmVeSWpSXmKPExsXC9ZZnke6kXQppBi2zjS3mrF/DZvF5wz82 ixcb2hktvq7/xWzx9FMfi8XlXXPYLO6t+c9qcX7XWlaLHUv3MVlcOrCAyeJ47wEmi/n3PrNZ bN40ldni+JSpjBa/fwAVn5w1mcVBwON7ax+Lx85Zd9k9Fmwq9di8Qstj8Z6XTB6bVnWyeWz6 NInd4925c+weJ2b8ZvGYdzLQ4/2+q2weW3/ZeTROvcbm8XmTXABfFJdNSmpOZllqkb5dAldG w70XLAVr+Cp6P81naWCcydPFyMEhIWAisXuxShcjJ5h5/FETE4jNJqAucePGT2YQW0TATOJg 6x92EJtZ4C6TxIF+NhBbWCBJ4uP+GWA2i4CqxKOHXawgNq+AqcTSnd2MEDPlJVZvOAA2hxNo Tv/7Q4wga4WAai78Deli5AIqec8m8epUKxNEvaTEwRU3WCYw8i5gZFjFKJSZV5abmJljopdR mZdZoZecn7uJERj2y2r/RO9g/HQh+BCjAAejEg/vyQPyaUKsiWXFlbmHGCU4mJVEeFuEZdOE eFMSK6tSi/Lji0pzUosPMUpzsCiJ8xp9K08REkhPLEnNTk0tSC2CyTJxcEo1MFqb/PT/tpbn 3aTwWxt1nfR13k+2FJSUv/im4+WhY5eUFp0UeMYVnOjg/c65nn+i+umv/J+37n/x483RF/oX P/0UCVWV/vcurv68zwnzqoJ35pKhjmIJ85Xk9KOc9X5u8ZSZxa1/w+jU2VeJnQpXThvee3BF k1vpCeOxNSdDGKIKE9y1tbeqiSqxFGckGmoxFxUnAgDQigswdwIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrDLMWRmVeSWpSXmKPExsXC5WfdrDtpl0KawZRfGhZz1q9hs/i84R+b xYsN7YwWX9f/YrZ4+qmPxeLw3JOsFpd3zWGzuLfmP6vF+V1rWS12LN3HZHHpwAImi+O9B5gs 5t/7zGaxedNUZovjU6YyWvz+AVR8ctZkFgdBj++tfSweO2fdZfdYsKnUY/MKLY/Fe14yeWxa 1cnmsenTJHaPd+fOsXucmPGbxWPeyUCP9/uusnksfvGByWPrLzuPxqnX2Dw+b5IL4I/isklJ zcksSy3St0vgymi494KlYA1fRe+n+SwNjDN5uhg5OSQETCSOP2piArHZBNQlbtz4yQxiiwiY SRxs/cMOYjML3GWSONDPBmILCyRJfNw/A8xmEVCVePSwixXE5hUwlVi6s5sRYqa8xOoNB8Dm cALN6X9/CCjOwSEEVHPhb8gERq4FjAyrGEUy88pyEzNzTPWKszMq8zIr9JLzczcxAsN4We2f iTsYv1x2P8QowMGoxMN74oB8mhBrYllxZe4hRgkOZiUR3hZh2TQh3pTEyqrUovz4otKc1OJD jNIcLErivF7hqQlCAumJJanZqakFqUUwWSYOTqkGxoVt8vsezf/IU1n9/PvbOR9t2n9fmGn/ TD+yb6VQSPA1wyWcZp5SnRtL3z1YE5jHp7yUY4eehY6k6N7XebFz9/F9OzFrqb+kGHf1u4Wn Gv2yfjdqyk38tXg9u8QrEfaNVxwZi8J8pj53dm5NmpTDcVF9/r7KW5On+/TtPuLJKPxhy9uq 3GdP9ZRYijMSDbWYi4oTARgcRMNfAgAA X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 23E21100004 X-Stat-Signature: d8ghrbfq7f99a9b5t4w3mukpshsx9xgr X-HE-Tag: 1713420950-418932 X-HE-Meta: U2FsdGVkX1/V/W+z6/QCbht70Jd7BkUyNVuikci/WponGL9GseeKUSc4P5WZEuLlByoFVOyQMNCD30IEiAq6gSVzeWormVpM1F7t/KIhBUQGyHo8XUKLugde0TfDFo2OKu1AcVXY60+24fQWy3lmpC0B4SetRqrj7+TSMGOE/JVpB1GnsVhDF6polvm4QMbAPzylr24nv6pysKQzAb7JeT8gxBEW5VGmU5XESbKeQYytWr+p62OdXBS//mWXYNtIQ5RvE7Czolm6P6RY8ReaFf8F2pxBQtSkerNh939eWSjaYg/Ybi7JvgWQ+PJGHq19eRVr4fcWHVfMonSzCOqrTHoOC6jGEfP9lhgfbv3xBexsVLPDJDUpNKbqARMdQdbb145RDRCzKRyWrBHL7yWd3QkM3UhJwbfKRtcbwz28LxOqQJMU/mI9WFi1RyScgYy3A1j1cBHsSn2Gcka3YRxiDZZasWEmQAXvM+YLJ2eKYbMEtjELdmFeErbQZJ2hqLZLytvAEGt1Fe3k4Hjw+lnkkjbtG+dfHnL/cr8rXJfAH8swlNBcrSM3lGMAIAr4YtlCNVZ8OUVIpi13BJEpGdChkY+LS6IA3KTuStnY0pRHjDnkhOZ54L1PN2yCNdhe2hnhLtfj3doWMN1rKSb1hEObEow2CIQJK9aDEi6Mdwotk/txhaRIIpLG+ThMkoXAetq+B0vY76F0RnMbWWNgCN4dEBHdxZoSWljbGwO869eTF77GW7Cve5zUmnwS82oseaPr5IMS8XTqx3LWm+6+gQQmlh20objO2wAjyWVE0JmLnOm4VzFIyb1TL82E89d8GmGSjRrDnvru3uV9j2aNnX9GkQ7+azw4HhuDF7O3Wik7ZNc1RQud4RFnUPe6TCzXi9CgcIsBUDsYs/FYuBADTXRS58Xaj4DsDTK9VHl6Tm0aOaHho+ARwNH55D0s13GGwOYscJW5WnCUplH0tVWqGL6 cw1vuZo1 6TgkceQ+R1g+mTsCy7MO2TUsXgA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation for migrc mechanism that needs to recognize read-only tlb entries during migration by separating tlb batch arch data into two, one is for read-only entries and the other is for writable ones, and merging those two when needed. Migrc also needs to optimize tlb shootdown by skipping CPUs that have already performed tlb flush needed for a while. To support it, added APIs manipulating arch data for x86. Signed-off-by: Byungchul Park --- arch/x86/include/asm/tlbflush.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 25726893c6f4..a14f77c5cdde 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -293,6 +294,23 @@ static inline void arch_flush_tlb_batched_pending(struct mm_struct *mm) extern void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); +static inline void arch_tlbbatch_clear(struct arch_tlbflush_unmap_batch *batch) +{ + cpumask_clear(&batch->cpumask); +} + +static inline void arch_tlbbatch_fold(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + cpumask_or(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + +static inline bool arch_tlbbatch_done(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + return !cpumask_andnot(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + static inline bool pte_flags_need_flush(unsigned long oldflags, unsigned long newflags, bool ignore_access)