From patchwork Thu Feb 8 06:26:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13549355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1C73EC4829C for ; Thu, 8 Feb 2024 06:26:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E54CD6B009A; Thu, 8 Feb 2024 01:26:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DDCA86B009B; Thu, 8 Feb 2024 01:26:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C33016B009F; Thu, 8 Feb 2024 01:26:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B036B6B009A for ; Thu, 8 Feb 2024 01:26:26 -0500 (EST) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 924571C1212 for ; Thu, 8 Feb 2024 06:26:26 +0000 (UTC) X-FDA: 81767652372.27.6539787 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf02.hostedemail.com (Postfix) with ESMTP id A08838000B for ; Thu, 8 Feb 2024 06:26:24 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf02.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707373585; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=eRD1/8o+qtEOTAKMY7ucioDJJtMI3+WpTL+ASvHX4To=; b=xDSzn/yWmDZQlmV11FTR72zRs0SUZgwjbt09KEr6RCGJIWZofNU3OrdG1SVM369AldxpQM mWShdgRx/zZ935/Yhek0ODnq8ia6zb7PxojO0AMTjwRYK7+ZACGWLTuypKiAG327gIG52d Ge+eLTs+Gsh3w+iAFcA14YS8EPwFUTs= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf02.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707373585; a=rsa-sha256; cv=none; b=2gIVKiQAhIwsOuLuh8SA4qMjQLl/JtM5nONOAdDkfzAaJu+tmt4/XRhoHwjJ1czq0pdjNl DUfTx6C2AU+2o2iZRJR1yeOW5myKFO4EWvkEfLdtwZHr1VFx4IGGspl5JHehY1owSzNd3+ a3R4CEJCMF5Qj3eECv3L0lsz/jgwm4g= X-AuditID: a67dfc5b-d85ff70000001748-74-65c4740a3ee5 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, namit@vmware.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [RESEND PATCH v7 1/8] x86/tlb: Add APIs manipulating tlb batch's arch data Date: Thu, 8 Feb 2024 15:26:01 +0900 Message-Id: <20240208062608.44351-2-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240208062608.44351-1-byungchul@sk.com> References: <20240208062608.44351-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFvrILMWRmVeSWpSXmKPExsXC9ZZnoS53yZFUg23r2CzmrF/DZvF5wz82 ixcb2hktvq7/xWzx9FMfi8XlXXPYLO6t+c9qcX7XWlaLHUv3MVlcOrCAyeL6roeMFsd7DzBZ zL/3mc1i86apzBbHp0xltPj9A6jj5KzJLA6CHt9b+1g8ds66y+6xYFOpx+YVWh6L97xk8ti0 qpPNY9OnSewe786dY/c4MeM3i8e8k4Ee7/ddZfPY+svOo3HqNTaPz5vkPN7Nf8sWwB/FZZOS mpNZllqkb5fAlXH79kumglV8FUt+tLM1ME7j6WLk5JAQMJGYuHUdM4y97csFRhCbTUBd4saN n2BxEQEziYOtf9i7GLk4mAU+Mkms/t7BApIQFgiWuL7qExuIzSKgKrF5zx8mEJtXwFRi55T9 LBBD5SVWbzgANogTaNDObadYQWwhoJoXsyezgQyVEGhnl1g57wYjRIOkxMEVN1gmMPIuYGRY xSiUmVeWm5iZY6KXUZmXWaGXnJ+7iREYEctq/0TvYPx0IfgQowAHoxIP74nyw6lCrIllxZW5 hxglOJiVRHjNdhxIFeJNSaysSi3Kjy8qzUktPsQozcGiJM5r9K08RUggPbEkNTs1tSC1CCbL xMEp1cAYub/cWXVhd03Z9S05PF7Xdm3dYl1vu2DF2RQjw4scU14tneW8if863wu9GczeKjN+ +aVN4VOseqx8V0WQ31JV0Puyxo2LHOy9T+6/e3dQfVV95bJ2+wdJH07Ybn3pwKLpVvFeuvNM dujHoEmHD6r9Dcq4J/CA6VdausbTXccb+fSjWQoW6lxXYinOSDTUYi4qTgQAo4YPLYQCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrBLMWRmVeSWpSXmKPExsXC5WfdrMtVciTVoPW6qcWc9WvYLD5v+Mdm 8WJDO6PF1/W/mC2efupjsTg89ySrxeVdc9gs7q35z2pxftdaVosdS/cxWVw6sIDJ4vquh4wW x3sPMFnMv/eZzWLzpqnMFsenTGW0+P0DqOPkrMksDkIe31v7WDx2zrrL7rFgU6nH5hVaHov3 vGTy2LSqk81j06dJ7B7vzp1j9zgx4zeLx7yTgR7v911l81j84gOTx9Zfdh6NU6+xeXzeJOfx bv5btgCBKC6blNSczLLUIn27BK6M27dfMhWs4qtY8qOdrYFxGk8XIyeHhICJxLYvFxhBbDYB dYkbN34yg9giAmYSB1v/sHcxcnEwC3xkklj9vYMFJCEsECxxfdUnNhCbRUBVYvOeP0wgNq+A qcTOKftZIIbKS6zecABsECfQoJ3bTrGC2EJANS9mT2abwMi1gJFhFaNIZl5ZbmJmjqlecXZG ZV5mhV5yfu4mRmB4L6v9M3EH45fL7ocYBTgYlXh4T5QfThViTSwrrsw9xCjBwawkwmu240Cq EG9KYmVValF+fFFpTmrxIUZpDhYlcV6v8NQEIYH0xJLU7NTUgtQimCwTB6dUAyN76BPlyY/y tq+8Iv+2ci1/eYl90+u8zrdxLNd364df1HT9bzPRaeGy75+m/nJ49pt3JXe9XwK/8srQ1TNW uZ0/YX70wHrVR2o93nc25UhOXFWtsKj6z8FJEcGTbu70XDtJwMpI6+aOxQ2/31o+f/LgxWvW h1u+bn6iONPhb9/6rc92GdWqnJifrsRSnJFoqMVcVJwIAFYAZeNrAgAA X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: A08838000B X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 4zx5egr7eo6ik5pi6tq74cqfhguyqt4b X-HE-Tag: 1707373584-815228 X-HE-Meta: U2FsdGVkX1/rb5kod265hQb/jytkZeZAkIHqFY9qUWc+5BXsRX7JsVlkEqyZtGp/X/nehKhsINbB1/uko32+YUw2JOQlhpPUNY3FOOaKinjjKUJl62N/KzepbH9KSXZbVrMfX8j2xsDpfcE2aAWgsyswhOcLedRqN7N0lNtxjAkIVhBsq57yy24RKSVSsrXb3AeUnBxf7dUH0tc++v66i5pkEKfKct/WXPOcgcTXF6SgFZpjI7pINBk/uj+6AcfO+1C36G3RdzcWkpaMFqjvtkiWzepq3q3Io6ZiHooNA20AtGA6GxmTSF1q4o9kbKo3MzPRdXVdcAxv45VY39BHm0Vu6jHUDGrq1gqOoI6w8RNW/txRWlXReeJyj10c0U8Ggn0KiDqLjnJ/QoUQRWF7pOthg2fLa9ENBUF6chtg07/ZzFewu37o8xVaPVFtqq7pUK0JAOLxjbpQeYwoUoJhK2hAqfKB+UUcbJbj7wIv3iq48p4xMu94HDZzzBWrkC5YxN+OfwfeAnoSsIRZLbDq9OrkhQ26znGtmzwGhdvfBABYDbQDho2slPCkRriSgosR4zUunxugBV0rJoaU5/KWvNBvoj5vcdy8qrfUwLIM37ZNb3htIMQGUNuER497M+337Ol5TbAGkDMx6/qT0RRd6SleJFlhZpo68/jwOeYfhJ75akVM0yHkdquWcwsGqmfCRBwkfkpBPJQUKkjT5M7fqLddreBZwdp1uHoo8wyOTKPkS2zutsommjNTBy+hjkcohAh0sXS15NzL+ZLyCtWaYVgfoXlHf2eWw57p3XVPdOqycgzMWQRIoeqUaJqdJQhxYQu+WnQujiEnZuYmJdj9nwAFx+LucUBXjdlBZ3sMN/CNGesALaVk5aBIhezzojAXlUtjBE9az3ufnBXVJbKgdmVTO+fnRS2jgJdL1LaIcjqyyDHgqB6tdPdGvuum6ax9FcTuXLa4mtnEgT8e5m8 9UWgFvgw RqrjVw533bSga/UBfu5/xqYpsIw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This is a preparation for migrc mechanism that needs to recognize read-only TLB entries during batched TLB flush by separating tlb batch's arch data into two, one is for read-only entries and the other is for writable ones, and merging those two when needed. Migrc also needs to optimize CPUs to flush by clearing ones that have already performed TLB flush needed. To support it, added APIs manipulating arch data for x86. Signed-off-by: Byungchul Park --- arch/x86/include/asm/tlbflush.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h index 25726893c6f4..fa7e16dbeb44 100644 --- a/arch/x86/include/asm/tlbflush.h +++ b/arch/x86/include/asm/tlbflush.h @@ -5,6 +5,7 @@ #include #include #include +#include #include #include @@ -293,6 +294,23 @@ static inline void arch_flush_tlb_batched_pending(struct mm_struct *mm) extern void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch); +static inline void arch_tlbbatch_clear(struct arch_tlbflush_unmap_batch *batch) +{ + cpumask_clear(&batch->cpumask); +} + +static inline void arch_tlbbatch_fold(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + cpumask_or(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + +static inline bool arch_tlbbatch_done(struct arch_tlbflush_unmap_batch *bdst, + struct arch_tlbflush_unmap_batch *bsrc) +{ + return cpumask_andnot(&bdst->cpumask, &bdst->cpumask, &bsrc->cpumask); +} + static inline bool pte_flags_need_flush(unsigned long oldflags, unsigned long newflags, bool ignore_access)