From patchwork Mon May 20 02:17:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13668000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 283EDC25B7A for ; Mon, 20 May 2024 02:18:01 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2431F6B0098; Sun, 19 May 2024 22:17:53 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EA5F76B0096; Sun, 19 May 2024 22:17:52 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B221A6B0099; Sun, 19 May 2024 22:17:52 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 7F0DD6B0098 for ; Sun, 19 May 2024 22:17:52 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 3C9CB80B44 for ; Mon, 20 May 2024 02:17:52 +0000 (UTC) X-FDA: 82137163584.27.09E56B4 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf14.hostedemail.com (Postfix) with ESMTP id 6715D10000B for ; Mon, 20 May 2024 02:17:50 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1716171470; a=rsa-sha256; cv=none; b=u8jgLaSXhNA2YW/fG8FFS6vMIoF5lTKThMhKMo0ys72Mj80wV/Uu4wulNlQVTI0xr6h/Xz YLovdBEA2+nYpUT+QN4jFcillQcGpMsG0yFETLnOcDiuVsVaJ8TRh2Q5mOoxwBkUTVqiCJ qipq5GkXaIL4Au/kbF1wJgisMq1AGq8= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=none; spf=pass (imf14.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1716171470; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=evCmjUh3KdBYN1DfbQBG2ARQXc+DAdC3ZcSVsI5RRA8=; b=wEB8cTRHbpDnBGxXWAjiPHp5Nh3R9M337RW+Jm9JwDvPy4gOvcz/gSNeH1s97xU1NXZDQJ OuorfsrPlQpjZdex4g/EnCd3XH0kxCs7zBaG6K9Hl75mh9pfj6PXfmwYrx6w/1vEmV2AIF 0vzTzX97cx5cxKyU2euSE498d4pKaH8= X-AuditID: a67dfc5b-d6dff70000001748-a9-664ab2c8cf5b From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [RESEND PATCH v10 04/12] x86/tlb, riscv/tlb, mm/rmap: separate arch_tlbbatch_clear() out of arch_tlbbatch_flush() Date: Mon, 20 May 2024 11:17:26 +0900 Message-Id: <20240520021734.21527-5-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240520021734.21527-1-byungchul@sk.com> References: <20240520021734.21527-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrKLMWRmVeSWpSXmKPExsXC9ZZnke7JTV5pBtceMFrMWb+GzeLzhn9s Fi82tDNafF3/i9ni6ac+FovLu+awWdxb85/V4vyutawWO5buY7K4dGABk8Xx3gNMFvPvfWaz 2LxpKrPF8SlTGS1+/wAqPjlrMouDgMf31j4Wj52z7rJ7LNhU6rF5hZbH4j0vmTw2repk89j0 aRK7x7tz59g9Tsz4zeIx72Sgx/t9V9k8tv6y82iceo3N4/MmuQC+KC6blNSczLLUIn27BK6M R78msBfc56tYM3EPawPjVJ4uRg4OCQETifmzmbsYOcHM1U/bwGw2AXWJGzd+gtkiAmYSB1v/ sIPYzAJ3mSQO9LOB2MICVRLtSx+C2SwCqhK3ts1lAbF5BUwl2hYvYYeYKS+xesMBsDmcQHNm rNoJViMEVPPj6FegXi6gmtdsEnc+NEI1SEocXHGDZQIj7wJGhlWMQpl5ZbmJmTkmehmVeZkV esn5uZsYgYG/rPZP9A7GTxeCDzEKcDAq8fDueOSZJsSaWFZcmXuIUYKDWUmEd9MWoBBvSmJl VWpRfnxRaU5q8SFGaQ4WJXFeo2/lKUIC6YklqdmpqQWpRTBZJg5OqQZGrlVnlrBvObjA6OIp Rxb9RZv3CcseuXN9SflEBX1nrcj02Ubz9ogv3tf/4ZiNsaK7V6CoXZ5w/5KjaoflfhhOWbyq X7vU//gbIz+JjwLmipHlOd+vbm9S8E08pBlWOUHJeftDgUuBnk+217OJW/8SUjok3vHqudVH oa+rbt/f79o+vXSGh/9vJZbijERDLeai4kQAji8YPXgCAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrLLMWRmVeSWpSXmKPExsXC5WfdrHtik1eawbGNJhZz1q9hs/i84R+b xYsN7YwWX9f/YrZ4+qmPxeLw3JOsFpd3zWGzuLfmP6vF+V1rWS12LN3HZHHpwAImi+O9B5gs 5t/7zGaxedNUZovjU6YyWvz+AVR8ctZkFgdBj++tfSweO2fdZfdYsKnUY/MKLY/Fe14yeWxa 1cnmsenTJHaPd+fOsXucmPGbxWPeyUCP9/uusnksfvGByWPrLzuPxqnX2Dw+b5IL4I/isklJ zcksSy3St0vgynj0awJ7wX2+ijUT97A2ME7l6WLk5JAQMJFY/bSNGcRmE1CXuHHjJ5gtImAm cbD1DzuIzSxwl0niQD8biC0sUCXRvvQhmM0ioCpxa9tcFhCbV8BUom3xEnaImfISqzccAJvD CTRnxqqdYDVCQDU/jn5lm8DItYCRYRWjSGZeWW5iZo6pXnF2RmVeZoVecn7uJkZgIC+r/TNx B+OXy+6HGAU4GJV4eDfc9kwTYk0sK67MPcQowcGsJMK7aQtQiDclsbIqtSg/vqg0J7X4EKM0 B4uSOK9XeGqCkEB6YklqdmpqQWoRTJaJg1OqgTHBpXxGoe7uK9OeLaxQj5H9fHSLnp5wUDw/ V7B0KGvrlQv2XiFJtlOmPI1vOFT7rCfzp09goYvc0cov2pYPtm2zymE/7ZRWauEbExzzy+35 6qZ1p/491JlZKdzQEFxnMf1ZzbF3rr43XaZef3C9v3ThlugzwRdE08+dNOjMDbykZjb9Ztdb XiWW4oxEQy3mouJEADDRNYVgAgAA X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 6715D10000B X-Stat-Signature: th9g3pxigt8oh7j885phyo7rcpy4p481 X-HE-Tag: 1716171470-33925 X-HE-Meta: U2FsdGVkX18Dr1EYPS6dlKE8F2OYTfKEPhx4uZWuy0UBcoBrj9arxFvQgVfabzvfxuZE49HMrL2a8HzNQaHcBKucXwvpo660d+awqwV8OIi9iSJQ0igtrBzH7GTeKFxBXb7tZadMkccBfWU6Y07C6V05+j/ULLYqUwau/Q4txgfYclYxkG3iPM1B00DFXzPBPFln1aZhPNGtFLCs/KtvkAgsFYxGNvuSO2+7u/NTL3y/m4Tg6rFLy9HRZocbXlpoA2CrKwNee8bejSFokYA3VwLRBt+Ck9axjusAvl0m7hPHZ4jgdfQEAyEBHWqZSF5NJji5xp07X7Q/oETEPSV+9NjINGIA+WTt/zrc00N1bzbl/lu42nxQm9NkqBc6S2F4l7TnXiWlKvoBQAOU01SsKUXZaPX8lNjFCHdnoika97PWnrUkmmTDn8kjCBvH5HEKfTVq8la35hA0xKbDl046SDjGLhrHY66LL1gTVOIVyPEh7za0e56VbyZQPbRE8R2tuBBk5a0CDWsHN1s90NzHYC9fNc499UBjSLtyn0nIIzla5gAXCyB5qw5BE46k+i64kkGPtLBlHCyPpgQzcElIvCMvkjf4Eur3xEUEKBpuQtK2BGMz8Skvl8psHsK9Q/Tb9+uwT9BiYUHznTTc8dti8eeTX7vg8BARsuY6X4hXzPFLMlaIldXNJWZ0mIA0Rx7HGz29pg1YwuzrCFJL4yEOtQLgOq2B45VYyXAAmkek3aKC/u1VMlE6PRoUtiKJO1JZEODzjfjLYfmGirVXl9R83KUU8z2pfZ1btyPd1ZB2QiX8RB/a6G63Tgwd32hJW2zm2+XXoQ5IF9/P3t9wvFSwK4dGXFPgWXDCxAvXJ4yhEOY6nh6thFReg2GMLATCTjLxgmsyswEuuH1FhKaVhDP/yrEZM499jlEyIlpDzETL9UiqaModWQNteYrHuDKaHjI4+IlgtbNgTUU6Be5OSkD 8tbxIDZ8 0yEGBYcj7y0N0fauh5gH17IsntZYGSrdrxNRsV88n9RvYJ2t5vcnpa5Ek2YfXf38YPrKoE2sKVh9hWu4wGH+GT/BcXQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A new mechanism, LUF(Lazy Unmap Flush), defers tlb flush until folios that have been unmapped and freed, eventually get allocated again. It's safe for folios that had been mapped read only and were unmapped, since the contents of the folios don't change while staying in pcp or buddy so we can still read the data through the stale tlb entries. This is a preparation for the mechanism that requires to avoid redundant tlb flush by manipulating tlb batch's arch data. To achieve that, we need to separate the part clearing the tlb batch's arch data out of arch_tlbbatch_flush(). Signed-off-by: Byungchul Park --- arch/riscv/mm/tlbflush.c | 1 - arch/x86/mm/tlb.c | 2 -- mm/rmap.c | 1 + 3 files changed, 1 insertion(+), 3 deletions(-) diff --git a/arch/riscv/mm/tlbflush.c b/arch/riscv/mm/tlbflush.c index 07d743f87b3f..9cbd27148357 100644 --- a/arch/riscv/mm/tlbflush.c +++ b/arch/riscv/mm/tlbflush.c @@ -234,5 +234,4 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) { __flush_tlb_range(&batch->cpumask, FLUSH_TLB_NO_ASID, 0, FLUSH_TLB_MAX_SIZE, PAGE_SIZE); - cpumask_clear(&batch->cpumask); } diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c index 44ac64f3a047..24bce69222cd 100644 --- a/arch/x86/mm/tlb.c +++ b/arch/x86/mm/tlb.c @@ -1265,8 +1265,6 @@ void arch_tlbbatch_flush(struct arch_tlbflush_unmap_batch *batch) local_irq_enable(); } - cpumask_clear(&batch->cpumask); - put_flush_tlb_info(); put_cpu(); } diff --git a/mm/rmap.c b/mm/rmap.c index 2608c40dffad..cf8a99a49aef 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -649,6 +649,7 @@ void try_to_unmap_flush(void) return; arch_tlbbatch_flush(&tlb_ubc->arch); + arch_tlbbatch_clear(&tlb_ubc->arch); tlb_ubc->flush_required = false; tlb_ubc->writable = false; }