From patchwork Mon Feb 26 03:06:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13571186 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D6D78C54798 for ; Mon, 26 Feb 2024 03:06:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EA4C16B0143; Sun, 25 Feb 2024 22:06:31 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D82AC6B0146; Sun, 25 Feb 2024 22:06:31 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC81C6B0145; Sun, 25 Feb 2024 22:06:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 983436B0143 for ; Sun, 25 Feb 2024 22:06:31 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 68E37A07D9 for ; Mon, 26 Feb 2024 03:06:31 +0000 (UTC) X-FDA: 81832466982.02.CAE71CC Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf05.hostedemail.com (Postfix) with ESMTP id 45AFA10000A for ; Mon, 26 Feb 2024 03:06:28 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708916789; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=eRD1/8o+qtEOTAKMY7ucioDJJtMI3+WpTL+ASvHX4To=; b=GVnlKyC9+hdR5dbMeaVG+LiNIgAVWbma9qZYIdIe6YH0gnoml6RdsamFKP98hhu7V34j/K uA+c4JHm4m9Z30n3uNPwkh2Y2qMaOoYRS4oNBRqBRg/ZzNrs6Hci05r+KHOKG/czRQHfj8 BHLQz9zPWGMYoXhd1Cs899t6J159F68= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708916789; a=rsa-sha256; cv=none; b=WZvkpo9cgarXy8T0oeuxk6PK1/pDrNjUKfUv8kaR4QBFq/UDDzEtaj2l9INQgr1cr6psWl Rk3tptYPMbp7UT1AT2k+bf+xLN7P+tjyKTwXjO+7Rz+MGT40BrCTC/mtKXiHQBohboZnHp hyEy0VNRY2moCWDrOEeRX1TDPi8Nmk8= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none X-AuditID: a67dfc5b-d6dff70000001748-27-65dc002fde00 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [RESEND PATCH v8 1/8] x86/tlb: Add APIs manipulating tlb batch's arch data Date: Mon, 26 Feb 2024 12:06:06 +0900 Message-Id: <20240226030613.22366-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240226030613.22366-1-byungchul@sk.com> References: <20240226030613.22366-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKLMWRmVeSWpSXmKPExsXC9ZZnka4+w51UgxdrDS3mrF/DZvF5wz82 ixcb2hktvq7/xWzx9FMfi8XlXXPYLO6t+c9qcX7XWlaLHUv3MVlcOrCAyeJ47wEmi/n3PrNZ bN40ldni+JSpjBa/fwAVn5w1mcVBwON7ax+Lx85Zd9k9Fmwq9di8Qstj8Z6XTB6bVnWyeWz6 NInd4925c+weJ2b8ZvGYdzLQ4/2+q2weW3/ZeTROvcbm8XmTXABfFJdNSmpOZllqkb5dAlfG 7dsvmQpW8VUs+dHO1sA4jaeLkZNDQsBEYvfzC0ww9tqN61lBbDYBdYkbN34yg9giAmYSB1v/ sIPYzAJ3mSQO9LOB2MICwRIvLqwCq2ERUJXYf+QMI4jNK2Aqsf7lfKiZ8hKrNxwAquHg4ASa 8/6/IUhYCKjkb+9VoHIuoJLPbBJP3k9lhaiXlDi44gbLBEbeBYwMqxiFMvPKchMzc0z0Mirz Miv0kvNzNzECA39Z7Z/oHYyfLgQfYhTgYFTi4V3w4XaqEGtiWXFl7iFGCQ5mJRHecJmbqUK8 KYmVValF+fFFpTmpxYcYpTlYlMR5jb6VpwgJpCeWpGanphakFsFkmTg4pRoYuVz6bB9Jzriy X8VMq9yz6u+fJIHjl14utgvvvFLz+vG1NI32YqXOFV2vJ50/LLDgY+mTsIK2WZfiXQ9Ylqms VXUwXRJdkpDhGPPLS7Y//Rjzt6/fqvSSU7dythmc7Z21dKH+x6TGK6rHZ768+PTCrp+Xt1SI 7mS0ckp/fjTwS0lLMWtOttYjJZbijERDLeai4kQANQUZ3XgCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrDLMWRmVeSWpSXmKPExsXC5WfdrKvPcCfV4EKrusWc9WvYLD5v+Mdm 8WJDO6PF1/W/mC2efupjsTg89ySrxeVdc9gs7q35z2pxftdaVosdS/cxWVw6sIDJ4njvASaL +fc+s1ls3jSV2eL4lKmMFr9/ABWfnDWZxUHQ43trH4vHzll32T0WbCr12LxCy2PxnpdMHptW dbJ5bPo0id3j3blz7B4nZvxm8Zh3MtDj/b6rbB6LX3xg8tj6y86jceo1No/Pm+QC+KO4bFJS czLLUov07RK4Mm7ffslUsIqvYsmPdrYGxmk8XYycHBICJhJrN65nBbHZBNQlbtz4yQxiiwiY SRxs/cMOYjML3GWSONDPBmILCwRLvLiwCqyGRUBVYv+RM4wgNq+AqcT6l/OZIGbKS6zecACo hoODE2jO+/+GIGEhoJK/vVcZJzByLWBkWMUokplXlpuYmWOqV5ydUZmXWaGXnJ+7iREYxstq /0zcwfjlsvshRgEORiUe3gUfbqcKsSaWFVfmHmKU4GBWEuENl7mZKsSbklhZlVqUH19UmpNa fIhRmoNFSZzXKzw1QUggPbEkNTs1tSC1CCbLxMEp1cA475ylarTBfS6DxKkLHoZMUPNqZ6s3 WRz5f/LmNdb/N0pH8RdsrHrgPX93wpIshQkpPFs3NSscv3LOdKJmk32G8v6Ie4zpVY93Fs+p VWzeqnklMXD2Go0du2Zfr86Snpe0Lv3CTd5bydZLFm+Ry/BRTzY/NyfwyaOXi75ZlzX8/qAU UhH7KyVciaU4I9FQi7moOBEA6KTWhl8CAAA= X-CFilter-Loop: Reflected X-Stat-Signature: ynb8q6bkhe6itwg6u5imyk7n7hzuk7po X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 45AFA10000A X-Rspam-User: X-HE-Tag: 1708916788-95388 X-HE-Meta: U2FsdGVkX1+9h2s+SCgwMg/vHXxzcQgU1uVJt+0vsQt7LsC9X7i7KxiCA1tkKEAjt9Xq/VF0fgWUBX27WMw0bvF9y1IO9QBni/N0Q6bbzJuS3enunTNU7nWLK7WPHtODP05AgR5UhQQUm9BiyJpGnctAbakkxFL10Wsh7rtbc0SA0/iYwS8UE+16PQXDIuF48oEl4F8RUgUyR7qGyJg8w0Km6tGcOxFuSC5VN3ssvEyRMXXfCr/P7YtAHi1NX9b976O1C6ZLOPlW/cssds2/w/8eDcdS5IBEAZlhEQG++pktQ/ri00GfVutwSrQV/kHAiM6RTbM4S1GnY7WJ3xk0VQjOYJTKPwKWPKUUNK96t/Mtfor92GBkj5Y2v4FsvG/sBvej6PV8vq5aGcz64yqPaIXf6Q4g+qSMYCn1MbSKFM/zJMTc8WDRMH7fUBcCOQ2Ac2yaM3CG4x5yVTFnvRDcbh094Yg2nlpvoNhKZUGJlFvsQNfVqWJMmzIMjnVkX73o4fyzJtR+y0clpUQ5Twu0Fw/2tr9Y/HaDkABOTl0UtCpv9erbfV7evlrO/z5FuJZg+/bLFMFHfsUuTsalkaCUpjKP5LvnhfMTsyFpC0uJ0q4ZF1KSRjvw4v0Ub8mzFGr96fAzvm7mMpvkm4k4ncr7ThFjSv9lyCCwk9J/Sgw2yekQ+nqSxNbEitj0zOhaACLGfym9DBkRvlZSAz+LSolINpTRHSnRWsSMLfj67YRODG+5w2i9keH9Z1D5zq5v/B87irwrS4ovg2/RTwdju/ETiw2OivsTsqx82pbADCn1tpeNw209i1HAHxMtr5F5IpcvjNVM8b/cL2wMsglGVXHQO4TFuFx4MF0pZUNuCe+Z0MNCnz3e0TobfyVDaE4rlF5fcNc8Ndkmvky3uUm0MBplgGVm327NB4tRjjYTubdNWMnPSns4dioovFhAuaCxYjXwqPwa2A6uNGQTgD+iWEw M/Q0yUmw RAC6nelzrcBj34qrqKYWxwRWprA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation for migrc mechanism that needs to recognize read-only TLB entries during batched TLB flush by separating tlb batch's arch data into two, one is for read-only entries and the other is for writable ones, and merging those two when needed. Migrc also needs to optimize CPUs to flush by clearing ones that have already performed TLB flush needed. To support it, added APIs manipulating arch data for x86. Signed-off-by: Byungchul Park --- arch/x86/include/asm/tlbflush.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 25726893c6f4..fa7e16dbeb44 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -293,6 +294,23 @@ static inline void arch_flush_tlb_batched_pending(struct mm_struct *mm) extern void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); +static inline void arch_tlbbatch_clear(struct arch_tlbflush_unmap_batch *batch) +{ + cpumask_clear(&batch->cpumask); +} + +static inline void arch_tlbbatch_fold(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + cpumask_or(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + +static inline bool arch_tlbbatch_done(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + return cpumask_andnot(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + static inline bool pte_flags_need_flush(unsigned long oldflags, unsigned long newflags, bool ignore_access)