From patchwork Fri May 10 06:52:01 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13660938 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0B959C25B10 for ; Fri, 10 May 2024 06:52:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5D1346B009B; Fri, 10 May 2024 02:52:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 52DD16B009F; Fri, 10 May 2024 02:52:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 177E06B009D; Fri, 10 May 2024 02:52:27 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id ED6A96B009B for ; Fri, 10 May 2024 02:52:26 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B555E1C1E9B for ; Fri, 10 May 2024 06:52:26 +0000 (UTC) X-FDA: 82101567492.21.F5BD7A4 Received: from invmail4.hynix.com (exvmail4.hynix.com [166.125.252.92]) by imf10.hostedemail.com (Postfix) with ESMTP id CDB89C0003 for ; Fri, 10 May 2024 06:52:24 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf10.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1715323945; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:in-reply-to: references:references; bh=ZKx3F49B6gn8AWD8yrAfoz7FZCy6q02Z9LdSwJIimPw=; b=73SxNU0DJ1q1XoJhbqBSwFy2x8ArT+Q4cTL3taYJwSPr9u4kaO22Xfmg/OgeOVVU1JSCQH T3R+kxY14Yu+nV12fJn6X7kX9EoMsxCOx5WHel3nJdntieoHXM5ZNnO2a37bmEC74wLNQF qiAQTpwtFMONW+8PfKPG9csbNGh6pqw= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf10.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1715323945; a=rsa-sha256; cv=none; b=uPF72QizmnXlyNj4PjVlYHGftScJqe1wta2ICcik8GtYt40pJ6LwqeI9Lecu/msTc4p0Qq lJ7KHYWQ0cUPJnbU432WBI+4vanZnJZwbrCAd2jUzCtkxY6MCFf5qDIJKc4NdlTOTool66 wVe8FSuGWnsOe88q+pvCXkQ6uzM6Bvo= X-AuditID: a67dfc5b-d6dff70000001748-e2-663dc4213de4 From: Byungchul Park To: linux-kernel@vger.kernel.org, linux-mm@kvack.org Cc: kernel_team@skhynix.com, akpm@linux-foundation.org, ying.huang@intel.com, vernhao@tencent.com, mgorman@techsingularity.net, hughd@google.com, willy@infradead.org, david@redhat.com, peterz@infradead.org, luto@kernel.org, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, rjgolo@gmail.com Subject: [PATCH v10 07/12] mm: add a parameter, unmap generation number, to free_unref_folios() Date: Fri, 10 May 2024 15:52:01 +0900 Message-Id: <20240510065206.76078-8-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20240510065206.76078-1-byungchul@sk.com> References: <20240510065206.76078-1-byungchul@sk.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFnrCLMWRmVeSWpSXmKPExsXC9ZZnka7iEds0gz8v1S3mrF/DZvF5wz82 ixcb2hktvq7/xWzx9FMfi8XlXXPYLO6t+c9qcX7XWlaLHUv3MVlcOrCAyeJ47wEmi/n3PrNZ bN40ldni+JSpjBa/fwAVn5w1mcVBwON7ax+Lx85Zd9k9Fmwq9di8Qstj8Z6XTB6bVnWyeWz6 NInd4925c+weJ2b8ZvGYdzLQ4/2+q2weW3/ZeTROvcbm8XmTXABfFJdNSmpOZllqkb5dAlfG pm8T2AruqlV03fFvYNwp38XIySEhYCLR9OYNI4zduOYlE4jNJqAucePGT2YQW0TATOJg6x92 EJtZ4C6TxIF+NhBbWCBeYu25DWBxFgFViZN7p7J0MXJw8AqYSvw4rAkxUl5i9YYDYGM4gcZ8 WDYDbLwQUEnHg4lAa7mAat6zSfy7sJcJokFS4uCKGywTGHkXMDKsYhTKzCvLTczMMdHLqMzL rNBLzs/dxAgM+2W1f6J3MH66EHyIUYCDUYmHd8dmmzQh1sSy4srcQ4wSHMxKIrxVNdZpQrwp iZVVqUX58UWlOanFhxilOViUxHmNvpWnCAmkJ5akZqemFqQWwWSZODilGhhbmLIy56/pml1h klWpybhetmq1VL30mpuecbt7lzwN3dpdeuWbZnbpyswgfpfIdxtFhL9o5S9NfP/v9sykSUoz FU8m2jdGLvoktfm8RtvJmzoNXyc0leu8q3qV83OHjfmUy027BWrm25u+vrv83bme12cWsdsH /nXaEaKwcpoLT4ne3WIBnjdKLMUZiYZazEXFiQArqVPSdwIAAA== X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFjrNLMWRmVeSWpSXmKPExsXC5WfdrKt4xDbNYP5cOYs569ewWXze8I/N 4sWGdkaLr+t/MVs8/dTHYnF47klWi8u75rBZ3Fvzn9Xi/K61rBY7lu5jsrh0YAGTxfHeA0wW 8+99ZrPYvGkqs8XxKVMZLX7/ACo+OWsyi4Ogx/fWPhaPnbPusnss2FTqsXmFlsfiPS+ZPDat 6mTz2PRpErvHu3Pn2D1OzPjN4jHvZKDH+31X2TwWv/jA5LH1l51H49RrbB6fN8kF8Edx2aSk 5mSWpRbp2yVwZWz6NoGt4K5aRdcd/wbGnfJdjJwcEgImEo1rXjKB2GwC6hI3bvxkBrFFBMwk Drb+YQexmQXuMkkc6GcDsYUF4iXWntsAFmcRUJU4uXcqSxcjBwevgKnEj8OaECPlJVZvOAA2 hhNozIdlM8DGCwGVdDyYyDiBkWsBI8MqRpHMvLLcxMwcU73i7IzKvMwKveT83E2MwCBeVvtn 4g7GL5fdDzEKcDAq8fDu2GyTJsSaWFZcmXuIUYKDWUmEt6rGOk2INyWxsiq1KD++qDQntfgQ ozQHi5I4r1d4aoKQQHpiSWp2ampBahFMlomDU6qBsf7FjoWTtidEVrX2HOhfUaE1h+frgeUy /65/v2G/ZFfFwdtPPrmu/sCTI3j2+cP9ueyK3Nxmpb3L6pX89wYda+Osc40veJQupFaU1Wkb eDylkuv8kVXiXTWzE2QtbPle+BX4q4VFLfrr8PhC+ucJ/yOT6z6yWh21yw++J7SkKOjPjjDB pQ93K7EUZyQaajEXFScCAFRkCp5eAgAA X-CFilter-Loop: Reflected X-Stat-Signature: yw4oyrujiiytktfy8x71k33k4613geai X-Rspamd-Queue-Id: CDB89C0003 X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1715323944-329584 X-HE-Meta: U2FsdGVkX1+AoeOrnNPtYR6i+xuDQHMFLkc4cmix0NCwdWW36E5/WIoAmY4WwKb8+qviNQ0ymn/yWerujGi1wil7c+Q6jc45qDs1ukcnGsVQik1bnmhTPsawTnupDsbsn1Qts8Rm3nZWYKK4VggHYaKog8eh8pEGdLLxU90lZj1cJT9Glw4rfkj+74oHw6m8pYwrgNpFhjdb16YbdXZ2kjGYDIETvS//AmevghxzbwxFVtBQjPSMRUDZEi+Ybw20fa4o3k6dfiW1KBxLengh+skHCM4og6aka+5twMRZbhr1XcFRBbgCS5LCdrdeIyJKh3/lj4fhLblVu0S3Q1LizzSS8kwrmZxWD4uNUg8pTlFYTi8vaEs4qLfeNIzBJfiZ8dmuEdE6X90M4m3bo2aaPCb39yCil+dgLPtsGRqpkRp6H6y4I6/PG2BKNBUp32w5p5c3twB83C988MNFVoL1hZ3CDzWXaojfW4LuQF9BaX57/Npu273n0L1T8ivbeNJZYTSGreCNOOH22fqOIIanJAuHGPWDEzY9ducGlUBA9Kem8oPm4/PkRntd9VA8wXF2uf2kFBw5INj05y2xRbtyZvKdOJT0iq9Eea4sfz2LNV7AMRfIm9IuniPCT8ikuyjxEzo+w5cI4pnH95MT+8rmtUrIBb/eE8Swy42sIDPpY0vmxB+R3uOqdKq9Dr/srE7UjuRtyn1tKjHmAiJAVwfUdRXHcNNguU0jUkPD2DjyZcdqRfXaviDqIM9GFWgAlkWSt69iTHrI3vsoKGLLcUDc3VN+eTjyG/C/vob+gYjgid2pGTgZlPKEsm96/E5wSoWiJUGKbH4BVcBLk8RVzDVnT3/jv3COUVnFldRzyxJ571xGib8vSU71V+pJCMWbpLTcBv08mp0d30soOLzgOtkGwh0rvtVNdNQIAgnD7Ikw+dDlGdGDhNLtikuINNVYaWmujw9nSqoGwvkjjnTxxuI vWFaCjNh MG8ojZqbDkFP+ejt4WsRiU33CgkhHGgFyj+ws X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Unmap generation number is used by luf mechanism to track need of tlb flush for each page residing in pcp or buddy. The number should be delivered to pcp or buddy via free_unref_folios() that is for releasing folios that have been unmapped during reclaim in shrink_folio_list(). Signed-off-by: Byungchul Park --- mm/internal.h | 2 +- mm/page_alloc.c | 10 +++++----- mm/swap.c | 6 +++--- mm/vmscan.c | 8 ++++---- 4 files changed, 13 insertions(+), 13 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 332662047c17..0d4c74e76de6 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -639,7 +639,7 @@ extern bool free_pages_prepare(struct page *page, unsigned int order); extern int user_min_free_kbytes; void free_unref_page(struct page *page, unsigned int order, unsigned short int ugen); -void free_unref_folios(struct folio_batch *fbatch); +void free_unref_folios(struct folio_batch *fbatch, unsigned short int ugen); extern void zone_pcp_reset(struct zone *zone); extern void zone_pcp_disable(struct zone *zone); diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 2cd278c207d1..63f14305f4de 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2706,7 +2706,7 @@ void free_unref_page(struct page *page, unsigned int order, /* * Free a batch of folios */ -void free_unref_folios(struct folio_batch *folios) +void free_unref_folios(struct folio_batch *folios, unsigned short int ugen) { unsigned long __maybe_unused UP_flags; struct per_cpu_pages *pcp = NULL; @@ -2729,7 +2729,7 @@ void free_unref_folios(struct folio_batch *folios) */ if (!pcp_allowed_order(order)) { free_one_page(folio_zone(folio), &folio->page, - pfn, order, FPI_NONE, 0); + pfn, order, FPI_NONE, ugen); continue; } folio->private = (void *)(unsigned long)order; @@ -2765,7 +2765,7 @@ void free_unref_folios(struct folio_batch *folios) */ if (is_migrate_isolate(migratetype)) { free_one_page(zone, &folio->page, pfn, - order, FPI_NONE, 0); + order, FPI_NONE, ugen); continue; } @@ -2778,7 +2778,7 @@ void free_unref_folios(struct folio_batch *folios) if (unlikely(!pcp)) { pcp_trylock_finish(UP_flags); free_one_page(zone, &folio->page, pfn, - order, FPI_NONE, 0); + order, FPI_NONE, ugen); continue; } locked_zone = zone; @@ -2793,7 +2793,7 @@ void free_unref_folios(struct folio_batch *folios) trace_mm_page_free_batched(&folio->page); free_unref_page_commit(zone, pcp, &folio->page, migratetype, - order, 0); + order, ugen); } if (pcp) { diff --git a/mm/swap.c b/mm/swap.c index 0fc5a5e8457f..1937ac937b8f 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -163,11 +163,11 @@ void put_pages_list(struct list_head *pages) /* LRU flag must be clear because it's passed using the lru */ if (folio_batch_add(&fbatch, folio) > 0) continue; - free_unref_folios(&fbatch); + free_unref_folios(&fbatch, 0); } if (fbatch.nr) - free_unref_folios(&fbatch); + free_unref_folios(&fbatch, 0); INIT_LIST_HEAD(pages); } EXPORT_SYMBOL(put_pages_list); @@ -1029,7 +1029,7 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs) folios->nr = j; mem_cgroup_uncharge_folios(folios); - free_unref_folios(folios); + free_unref_folios(folios, 0); } EXPORT_SYMBOL(folios_put_refs); diff --git a/mm/vmscan.c b/mm/vmscan.c index 49bd94423961..bb0ff11f9ec9 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1460,7 +1460,7 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, if (folio_batch_add(&free_folios, folio) == 0) { mem_cgroup_uncharge_folios(&free_folios); try_to_unmap_flush(); - free_unref_folios(&free_folios); + free_unref_folios(&free_folios, 0); } continue; @@ -1527,7 +1527,7 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, mem_cgroup_uncharge_folios(&free_folios); try_to_unmap_flush(); - free_unref_folios(&free_folios); + free_unref_folios(&free_folios, 0); list_splice(&ret_folios, folio_list); count_vm_events(PGACTIVATE, pgactivate); @@ -1869,7 +1869,7 @@ static unsigned int move_folios_to_lru(struct lruvec *lruvec, if (folio_batch_add(&free_folios, folio) == 0) { spin_unlock_irq(&lruvec->lru_lock); mem_cgroup_uncharge_folios(&free_folios); - free_unref_folios(&free_folios); + free_unref_folios(&free_folios, 0); spin_lock_irq(&lruvec->lru_lock); } @@ -1891,7 +1891,7 @@ static unsigned int move_folios_to_lru(struct lruvec *lruvec, if (free_folios.nr) { spin_unlock_irq(&lruvec->lru_lock); mem_cgroup_uncharge_folios(&free_folios); - free_unref_folios(&free_folios); + free_unref_folios(&free_folios, 0); spin_lock_irq(&lruvec->lru_lock); }