From patchwork Tue Apr 9 09:28:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13622060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB162CD1284 for ; Tue, 9 Apr 2024 09:25:23 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79B776B008C; Tue, 9 Apr 2024 05:25:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 721246B0092; Tue, 9 Apr 2024 05:25:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FF8B6B0093; Tue, 9 Apr 2024 05:25:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 209146B008C for ; Tue, 9 Apr 2024 05:25:23 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id D4491802BF for ; Tue, 9 Apr 2024 09:25:22 +0000 (UTC) X-FDA: 81989460084.10.3B8F2BA Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf24.hostedemail.com (Postfix) with ESMTP id 89E9118001F for ; Tue, 9 Apr 2024 09:25:20 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DRFcqFN9; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712654721; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qZVGZwY9DtxMHMolatICyCd8jVx09nsfE645jS/598A=; b=yOsXliOE+z7NNdpmPhLKdEY6VsYkUI3kDU0Rif0RiVLghGHdMM7DZ9zEDzczePE+CgLz9N OYUzccKAnvzH62YvMhOdvX7AXQajy8tQwhGbXD7RERiVcdy5AHAWoJOBockApH2LW52+rn +3s1WrL+b5NxKGAvzddltO9uTinQThI= ARC-Authentication-Results: i=1; imf24.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=DRFcqFN9; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf24.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712654721; a=rsa-sha256; cv=none; b=YFSXmjQ9MILSrvqvfr3Kc/UjBFWcULLRb3gpsS0gpb5LAhinLgrzY/I9BqauR+y8z2ubYV msNZi7iEpUseyV9HbYPqDukpPrNcY+9Lpz1AOPnVMSDXd55g+2etQFNeulChufXXs0aDb6 CP8eqozoeXKI9cMlCafXFWx2Datgn+c= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id CCCF8CE1DF4; Tue, 9 Apr 2024 09:25:17 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F0EFEC43390; Tue, 9 Apr 2024 09:25:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712654717; bh=jy6PhQURlzWnemsEhuISNb873RtM8zjTy52Z9fJnaQo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=DRFcqFN9/XtdCn1Sbvo2Ddn+Ka5gBb/M2Y/Q0SD9k4n7tUVbGSJveLmsl9M8J9cDy 20+EmBJsTN2SUT/vt59KF9BxtksTeVWOSZs3E6BIXoMeScBM1ZfqUx8Fq783/yp8m2 u2oEPEGPPrgCI+XsJXITtUUH8VIM0MlCkXPcy/2EKPzxthn2kzLzMcE5U/a8+L1EYa 91TJ5ltdVOR3EDcHvQdeGI9BJJ5T96tgJZTnePSgKHwFYMpzrqD4vZmzICwhNWKQZk liyNGwApsCciLgWPQ6zoLlKRFvNaQytldVXlzCIsJC8KqkYZJEqkChrbG38FjG9//8 /dOweJU2fevgQ== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Alex Shi (tencent)" , Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , David Hildenbrand Subject: [PATCH v4 1/9] mm/ksm: add ksm_get_folio Date: Tue, 9 Apr 2024 17:28:16 +0800 Message-ID: <20240409092826.1733637-2-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240409092826.1733637-1-alexs@kernel.org> References: <20240409092826.1733637-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 89E9118001F X-Stat-Signature: 5t6drey9ubd6ntmwme9q1to7juj3fmt4 X-Rspam-User: X-HE-Tag: 1712654720-16501 X-HE-Meta: U2FsdGVkX1+VBEwuXR7J6T8yXVD6Tcpk1H2bUHvJQAyvO4GhxFBad7sYoiSayEJEqD3ovUvbaWPv80S2RAyRY8seDmx1tp5IR/9R0Aya0JwhEyYCjbkNhkjoalgkg8m67E/u1NdEQvtyHdtyHvVxJ17KxPV8omuo+l3uEE9Op8FFOMv/8JT6K9CeyfIbIa3vg1l80/VHvLAcHbsYAQVrflHtVsRgSZM5jlRzJCOikIG9yVWbHLFehdFtauk9iOp58aQtWBYFOYh9i0Vi7DmLnKa2WkwbKK9XaqinkORadr/HSAww2c5a6J6/qlDHm3HDUWix0ZcFX7zSC/PDiDcpN84EbGPwReFMSJkAELGIuymoywsZPVaUh/yk3hJwtaGMKBa3WcCC2gKs5JqKDYkNbEgIFwAFq5QaR1YveAhbxWJSIbUnpvWKfX0hyimBqydmOy2nYCgQc9u2dq5NP3DEwiVVuA4Ut+0v8SDNI3nX2mLNrXjDHf9SWwbhNe2T2NsbbrP4a3LH56glinI98PfWw0YdEItiD9cyFHG4Z92It6laV/ZhHpj+p48MTa4BLY9Uif99kHZso/9Mf4El+MhOTKpKEOeiFuH6cAPcmyFStNMDPbjZyCn5ida9U8yrbpT88wA26xC62tNpzfEE6Bu/095oShzzMC/+L/JTzrTO3XAuGVIo9XPUeN/r7rCceHt3iSD+xjueRBIGIrPL8BvfwdSMxEHzCFCDLPkFZIAAfvQ7apoDiDRWFUKyPmBvcfzyXQzxdCyL+ApYS9l4MWjqS8ROkXB7UpofX/P2odIzxniZ/4Mc2Z7JgyKZ+wlNMckpszhW+lzbf8j9JB7VfDBegGxoxGe2QSd+x9llzBHxOB6cx0ssPxuJSOMz68DwI1C0NFM7Pave5fLFHoEfZsOOjVNCGz5qG6d/LKayTneHTck8pTbhD4t0fwqlGC0QyEhr1VQvVf+BNV8htqcsbGW PVQ+jv16 aybpp6LMeQ9AaeYpRnohCOs7Fy4TeH7hnVn0XuWlBIhTSXeOcSgSN8Z6Mqm1SusRB1JMHzTkrUjbsokHHEaDQ8Zg0yKTts+l6mfXz397+Fhpe/6MgsmMNZnHsgnpgJ9gyE7DDaxgBUkvceeQxmFAfEROIq3kHU7UcyNroIFeFD4pro9Tpi+c7TM3NHxqK1t1nRPp+i58mMNtrKnLqgGvv7R8mA+n1gJyAneNYCY54tNYjSC92nqZMSsXofu3OKdQhkQIFJmvvkjFKIQ3oQbcmhDK80/qx4zdmWK+uqWKm8RQnbUs/jMUpUmTZW3at/4rT6knL84SdZ0dwlDBG2Yve3BDgHq04I3Ceh3aYqn4TvWsLaZI+5aDlIvY94WoOYL2NDiv2i6YN1MK/TqApdHoWRKm0oyue86PAfakVHhacufy/ExC9wKHUysi28v/a+MnrqYGZpC+kay8fde/Qpb634SLfR1JRC6zn0UR72KvETCZJVUEccc/Oo1Ybt5+ZzzQiN1OIv1oHjYLFe/GJuiS+c5v2vEoqzPuycX1wJn76Fz+194duFFn5xAoV0w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" The ksm only contains single pages, so we could add a new func ksm_get_folio for get_ksm_page to use folio instead of pages to save a couple of compound_head calls. After all caller replaced, get_ksm_page will be removed. Signed-off-by: Alex Shi (tencent) To: Andrew Morton Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 40 ++++++++++++++++++++++++---------------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 8c001819cf10..ac080235b002 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -915,10 +915,10 @@ enum get_ksm_page_flags { * a page to put something that might look like our key in page->mapping. * is on its way to being freed; but it is an anomaly to bear in mind. */ -static struct page *get_ksm_page(struct ksm_stable_node *stable_node, +static struct folio *ksm_get_folio(struct ksm_stable_node *stable_node, enum get_ksm_page_flags flags) { - struct page *page; + struct folio *folio; void *expected_mapping; unsigned long kpfn; @@ -926,8 +926,8 @@ static struct page *get_ksm_page(struct ksm_stable_node *stable_node, PAGE_MAPPING_KSM); again: kpfn = READ_ONCE(stable_node->kpfn); /* Address dependency. */ - page = pfn_to_page(kpfn); - if (READ_ONCE(page->mapping) != expected_mapping) + folio = pfn_folio(kpfn); + if (READ_ONCE(folio->mapping) != expected_mapping) goto stale; /* @@ -940,41 +940,41 @@ static struct page *get_ksm_page(struct ksm_stable_node *stable_node, * in folio_migrate_mapping(), it might still be our page, * in which case it's essential to keep the node. */ - while (!get_page_unless_zero(page)) { + while (!folio_try_get(folio)) { /* * Another check for page->mapping != expected_mapping would * work here too. We have chosen the !PageSwapCache test to * optimize the common case, when the page is or is about to * be freed: PageSwapCache is cleared (under spin_lock_irq) * in the ref_freeze section of __remove_mapping(); but Anon - * page->mapping reset to NULL later, in free_pages_prepare(). + * folio->mapping reset to NULL later, in free_pages_prepare(). */ - if (!PageSwapCache(page)) + if (!folio_test_swapcache(folio)) goto stale; cpu_relax(); } - if (READ_ONCE(page->mapping) != expected_mapping) { - put_page(page); + if (READ_ONCE(folio->mapping) != expected_mapping) { + folio_put(folio); goto stale; } if (flags == GET_KSM_PAGE_TRYLOCK) { - if (!trylock_page(page)) { - put_page(page); + if (!folio_trylock(folio)) { + folio_put(folio); return ERR_PTR(-EBUSY); } } else if (flags == GET_KSM_PAGE_LOCK) - lock_page(page); + folio_lock(folio); if (flags != GET_KSM_PAGE_NOLOCK) { - if (READ_ONCE(page->mapping) != expected_mapping) { - unlock_page(page); - put_page(page); + if (READ_ONCE(folio->mapping) != expected_mapping) { + folio_unlock(folio); + folio_put(folio); goto stale; } } - return page; + return folio; stale: /* @@ -990,6 +990,14 @@ static struct page *get_ksm_page(struct ksm_stable_node *stable_node, return NULL; } +static struct page *get_ksm_page(struct ksm_stable_node *stable_node, + enum get_ksm_page_flags flags) +{ + struct folio *folio = ksm_get_folio(stable_node, flags); + + return &folio->page; +} + /* * Removing rmap_item from stable or unstable tree. * This function will clean the information from the stable/unstable tree. From patchwork Tue Apr 9 09:28:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13622061 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BE10FC67861 for ; Tue, 9 Apr 2024 09:25:25 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 725336B0093; Tue, 9 Apr 2024 05:25:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 639AC6B0096; Tue, 9 Apr 2024 05:25:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 43DCB6B0095; Tue, 9 Apr 2024 05:25:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1B07F6B0092 for ; Tue, 9 Apr 2024 05:25:24 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id BDBDCC02DC for ; Tue, 9 Apr 2024 09:25:23 +0000 (UTC) X-FDA: 81989460126.12.CA5944B Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf17.hostedemail.com (Postfix) with ESMTP id 2175C40016 for ; Tue, 9 Apr 2024 09:25:20 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="a9lLnck/"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712654721; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=MqIPFBc7nbtVV5KfBtO4U0LYKxmmn3vpAgivZBxfJEY=; b=3kloUFdZulN7ltNhlYN6XsoFOKAX7yi1pW2NRcpv3xWG2hgpKPqUO9+msNQKwKWaljrqj/ 5RSfp7qZue8vk5SXJ6c0Apm0COuH0RL7+PoDcfQmt2T+U2HLjmf/VPmF+Yy0tkiiI7CT0H yxIVRPPyfhh0GJ7jLTePnoVTR7Cb43s= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="a9lLnck/"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712654721; a=rsa-sha256; cv=none; b=QDugc/DRUR9BsAO5xOSjm7ajUk1FJHZLDxGOlCDdttb+OtUL9QCGMdLT6NMnrz5S2GHy1I VI+rWvn4j4ch7pmN09jrLVnr2FSIiGHqS/t/OPmL0p+jEDC/M/4F+dVY8Njsr5kgFk9kot /AvtCED6ws66U44CsGikeAi4FGlt7Rs= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 06BC761654; Tue, 9 Apr 2024 09:25:20 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AF8F0C433B2; Tue, 9 Apr 2024 09:25:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712654719; bh=IVhAzlnf/FbFRG9PEnvDGIGtbMw8j3OJiDaE0yo5KvA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=a9lLnck/EFYB51Ch65extYs7iz20UjcwqKAleLCh5etxv9i7HDqZ82ImyRakGRseu Pdlzye6w9+SFe5lw0GRFLowvE3JZCKwRDpuDXnI423xC/rQVfyxivcxkonCU2YVLTZ 2qe9JsD1kGKPTfSBViXa6XOWcf8pGLz+iiw/9OhSA8ppiaIwqsb6LY/eLX9h8GRGMZ ouoAZrQy0bfnG8+DQJTie69CTdAr7lBswCvZf2lf+z10wMqKOIUAzDDDoQwAq59hxC xcaQY3pEymd7znTG4lG1ZvZvSfQkCx6XZZhrKLUNpbrA93wB6X1eTHGuwUr363y/W0 UnqUoiDYz6M2A== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Alex Shi (tencent)" , Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , David Hildenbrand Subject: [PATCH v4 2/9] mm/ksm: use folio in remove_rmap_item_from_tree Date: Tue, 9 Apr 2024 17:28:17 +0800 Message-ID: <20240409092826.1733637-3-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240409092826.1733637-1-alexs@kernel.org> References: <20240409092826.1733637-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 2175C40016 X-Stat-Signature: b1squif7cem7us59hmz1a83mhohuyq4p X-HE-Tag: 1712654720-495888 X-HE-Meta: U2FsdGVkX18hlUDv/JdE3DtTWva9tz8+42d3llhPq7v6eheHNLc0Bo6gZyRjiceoLJJyy5XI443h6X41P8htjYxu1cmqXnxBYKOryLXWSMvv+H7SSONsQIdQRaww+ZxnGrE7m4aksmjx2OZfw31QCG5PUsqr+WGTl9CSNjRXG6xottaGsU92M90YaiEOgKxnWaJE2XfLzpkh0BzRpCiJCa+5nTcEHiVp5VXMPM12jyqo/QWUfNob8ZRBcIMDynPXpas6bF6cgVzQjmh9z4UVuCP1eK6CgcI3wdhVXKKc6cVEJGiRXYsXTWCfSMffV42BCalRo7tv/Bqw6Jn0joBvPjAW8poNHMyMVRujumgjsU6DlG/q20qIpyoBPst4X+ZO+i4tAKuRdMiXQFlQj34Q+DT9qrkhSm89A0KQbJvxqR1MUP63AaIejLlxtGxuwBC01jvvKzVDhVBJzYIafotsHaqj0POC2mljTkB/ALebwpcBvVvvZzLfq/dKSdiRLWSuS5OJ/LOhp85EKjk5yWbtlgoGuyCv8s01YiMNPoHnHUGO+R4ww+ah9jT8NCifNsbKorXpt+fDF4jpYOw5ParteYMpJXyKz7dDDIyMiHFOm4Bw9tGDyvaX9WnOpzZrQFLc9gE1stCe16FQ4vfENvLSNHxjl8Q53wGyMyG0fx1M4kKTErQsT5+XBobJyWrbWpCLF/LpBAvmnDAR8sLlePz+qWW5y/KoP5204XnEWG10x2gpgikbe8qP9sHhaoCxs1E0ohJzJFYiK0cTCrOTYufr/fbhOzh3yDfLehuifEA9Ws4jMexA6/hg2AqCdq/lxeOUHXzRl5JO1zTowb9DigZbYGyx0NDWkEh5lQNcve/6uzD0FkbFN1ykm5w8o061JX1ij9jeqHhvpgIVX9W5Wy1ka+UZTi68j/FMZAUv4eKlWSTR6IMolPnFyMehVmYwziBbkQBz1RXCc/BKE460pCA F5euameb 5fxCGZhArBthsAh/PcauDzKRU/07LhxbVmhcmiZUM2VjxNiKT8peZ3tenZjIL8roc4Oj/SP8gLZet/jEV+5nbXTdV0lpo6h7UmA6XyoyJexDM8VElrJXvq++HJoQkdcv24SVfABLlPYIQOztVRMfPEQoGKqZwaBEHmgRjEziHm3i6jDSFwYSM5fESjcVhaoHmajx0w46a0jZdhpmhVJJ2o/n0m6EKCZvQvIWK2YuNkdadJzM9h7daYQeB/5jLr9Px9VEG+KnmFraCyzVfRRY6qUvwnB5yOYJyUKbAJiim2h2fGcTjcNaOBWjB1iVnJ6mZvaWJVsRgSUBSjkooO+Su77Q7h0JLvGed3NuRDKSh31YvFz9C+8E5BNr5EUXR+DcpzsB2FjdPAR9JVFi3W56na7xuAFKVrV67F8dOH9pywNac8b8u03hdiTzXbJiblsb+aILIAADlT/gcHSwfy07jlyDyyBdAyWomoCDatI3f2VFBfP4m7PFdTUotnnPqS6GvvrbP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" To save 2 compound_head calls. Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index ac080235b002..ea3dabf71e47 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1006,16 +1006,16 @@ static void remove_rmap_item_from_tree(struct ksm_rmap_item *rmap_item) { if (rmap_item->address & STABLE_FLAG) { struct ksm_stable_node *stable_node; - struct page *page; + struct folio *folio; stable_node = rmap_item->head; - page = get_ksm_page(stable_node, GET_KSM_PAGE_LOCK); - if (!page) + folio = ksm_get_folio(stable_node, GET_KSM_PAGE_LOCK); + if (!folio) goto out; hlist_del(&rmap_item->hlist); - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); if (!hlist_empty(&stable_node->hlist)) ksm_pages_sharing--; From patchwork Tue Apr 9 09:28:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13622062 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B7B73CD1297 for ; Tue, 9 Apr 2024 09:25:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4DC616B0096; Tue, 9 Apr 2024 05:25:28 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4642A6B0098; Tue, 9 Apr 2024 05:25:28 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 303C56B0099; Tue, 9 Apr 2024 05:25:28 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 04BE66B0096 for ; Tue, 9 Apr 2024 05:25:27 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id CDD27C02F5 for ; Tue, 9 Apr 2024 09:25:27 +0000 (UTC) X-FDA: 81989460294.19.3837401 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf09.hostedemail.com (Postfix) with ESMTP id BBCBF14000C for ; Tue, 9 Apr 2024 09:25:25 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cHW5B7MG; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712654726; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=jtkdWevEu3psN02cr3rISCmZfj2it3cnlRAxu8k3HO8=; b=S4xEAjN4n3JD8hT2qx5LgKhJU9ZDqxttvFvUqJau56e5NZgP9lhzs302IGIkUSZJdtvlnR 4VA5lytgyXVgb8uIo09/ZWvSUVU9fswA8YEUhv+UwlPGLOv2TIGf6GvZAXPkNnyliWvzBA gzzQe3VCgr3F5A5SEUiACdPXrRn/QYI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712654726; a=rsa-sha256; cv=none; b=1FX/XHxmt6Q7ohrwc7Pzb9YE7YWf/0I2eU6Z/inHoac0JDpVFHKRQ2rPTBHNeAexquTJuu iKD+frSxipIRvqhU7smFlcs7dx/21UScVF2i5ITEv4nTKL+x6LHUiQkveE5hgdotdKC+Uw sXOu3f3mTZoMv+qMZUg670c5/dSAbdA= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=cHW5B7MG; spf=pass (imf09.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id D9A9FCE1DF8; Tue, 9 Apr 2024 09:25:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 66ADAC433B2; Tue, 9 Apr 2024 09:25:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712654722; bh=h5mTUeVzSSO40P/dqunWDfKwvbrZ6ROf/OJsSV/+/Tc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=cHW5B7MG3rTIH2E0eZCzzevv5RY2jP8XktDsl8WLKbwItLlisGdKj492muL5NV4mm av+RdTMeEMaMoDsAGubCuRkp+hARu4OGfmFTc85AP7jc7sAxULjkUcM7mdviojnabW 1H9tHT4Y8NVSSOFaPSkMeCychOjudYI9JrQaQ3D6p9Q5kdA3d7F2Jw0WxPEVL771VY CIJC3gE5IJ3yIRUl1QFORzLz2EPuU1CDn00wT4HkyOjJ3kdBDjeOyZ8OPD/tcw9G9I JZWAIQHvQRvw/67Tmt0tYZYZtispL4c2DpDAYtoivh92pGDWgK0hFcj8zklPe3q91p IPnjllxt9U90w== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Alex Shi (tencent)" , Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , David Hildenbrand Subject: [PATCH v4 3/9] mm/ksm: add folio_set_stable_node Date: Tue, 9 Apr 2024 17:28:18 +0800 Message-ID: <20240409092826.1733637-4-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240409092826.1733637-1-alexs@kernel.org> References: <20240409092826.1733637-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: BBCBF14000C X-Rspam-User: X-Stat-Signature: j57y7xfpx395or7d17faya5y5u7zkqha X-Rspamd-Server: rspam03 X-HE-Tag: 1712654725-438422 X-HE-Meta: U2FsdGVkX18MY4BuZf6SiykxpN0ANsivHRZg+FcMoYDGPvWG4m72H+bWrccFozhviR3ZBAtHK292CTPiA9nz0hHONsXZ8IiVaEbDNp8lKs/EzpTLkwzKAcExsdfxioo9Nods4FMMxoSpy+zZMezjni90FWj9TILD43a+McBzKg6nUyaBH4E8/MrziLj4wuH++Z5deA7p6SPQmbjHxzD/e8f/+yl3+g2LVhBJ37lttPOXrXh1N2NUZxLQONPE/94vnFU7PmNj/xNSs4esAFwo2nhXOBQwcd3qpSoLIg2UP7/MebQSIAlwm7xu3Fe3DTDfAbBNTJTXHsMJ+EXqQDB/2hRKcLChcgCNeQeZf4bgYsOsycOK9erGXEEbrQtxmbi/yseR25zILpRW8B6j7l07VyEtrU29N1Y45On5thcSkBhlasDT1LDwH9y++Q/x76Ha0vP0wn8UW6gwZl78t7NHfjTwNRoNsRv09+YWPPyusdpFMHhPBFBB+fwHsSVf6wy7CLHRbBSSPu2firp2UdU+9N0p4TzhyCZly2TGv9hM19Ew5cQenyMEjQ6HZ5pUL3yChkC2C0x1omnlEjsSBOiO9QG/sdvCzO1tlk3/LX0jLcgUR+OR1zRiJuP1DcY7W2biMa/en8PrpfpgBbQqChRy93J5xm7llWADGlonLGmJRZ7eZsw+9NiksX9jvgAhjN7QYBltbL/K9cbHWbpH5JyYhvO4f6djZioVTlL/pyxXSaDtSsr077jNf0n2VWCwjneO8ENjDWm1ttyZZvUvtWgTFRZbGWeyUVIqV6NifB35WQ319Lad4rEobdBNf6ipTAF+NTdODpmxwf3oFPA7xMP88s/WH7CIGPQ9gPvStnwe59SrAXefAKOHJol1iHSx5RdrzG/MVdHlYtbEY5248qLFNRrezJARGsKWgGF/v2pVG7Q4a34z73qBdvxU8IwRt3oTSMZqO820dsTmbbeoQR+ 21U+kzny 6BrQ8/QnJLzD85CleiKHrFMAfuE/CjIDf8IObwAlNpUPDCUp3I9NWwLQ0rj1wPIFiv5ZXVEM2+Vt5LkHLyaE+iejR6QkX44+9sYdv7ivOjsuL+KasU3l0l/01yp7687p1ujWllbFqwldpQrUgIQksEoEhv4DfE6VKzFwp70oP0O1nKVFwPHJDT4n5NxwHpKNC+jlugXkRBFMOntw5Vhwba7gLaAKYMBTWsHbmaDB6xXDu2Ss019h0Kd4/KBYWM2c6XZy0AB3qiN7/pojYxqimYiAgFBS3fJ+6i0oTNmNSefdeXyeqpYforHsL+dawudZgHdbub6FIwnNpAHHkZq8mour1aoXn2UMYdP3aNKJUAmPT54OVzAfDcdjfjqRhslWhn0wVY7vkGnDF//iSZ9Na24CKo9LNSifQs5XmcEjsaglzvrL0TayrCkN4kMVi2FH5e3GukZfbIXCy86pkYyj7OPsAg9Sl/HkwvL3/eZaC4QCBVYX8dHQ5WA5R5lOeNsrYXEOi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Turn set_page_stable_node() into a wrapper folio_set_stable_node, and then use it to replace the former. we will merge them together after all place converted to folio. Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/mm/ksm.c b/mm/ksm.c index ea3dabf71e47..c9b7c5701f22 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1109,6 +1109,12 @@ static inline void set_page_stable_node(struct page *page, page->mapping = (void *)((unsigned long)stable_node | PAGE_MAPPING_KSM); } +static inline void folio_set_stable_node(struct folio *folio, + struct ksm_stable_node *stable_node) +{ + set_page_stable_node(&folio->page, stable_node); +} + #ifdef CONFIG_SYSFS /* * Only called through the sysfs control interface: @@ -3241,7 +3247,7 @@ void folio_migrate_ksm(struct folio *newfolio, struct folio *folio) * has gone stale (or that folio_test_swapcache has been cleared). */ smp_wmb(); - set_page_stable_node(&folio->page, NULL); + folio_set_stable_node(folio, NULL); } } #endif /* CONFIG_MIGRATION */ From patchwork Tue Apr 9 09:28:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13622064 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id EE185CD1297 for ; Tue, 9 Apr 2024 09:25:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6FC606B0099; Tue, 9 Apr 2024 05:25:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 65DBB6B009C; Tue, 9 Apr 2024 05:25:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 417436B009A; Tue, 9 Apr 2024 05:25:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0E4746B009B for ; Tue, 9 Apr 2024 05:25:31 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 9CF11A0794 for ; Tue, 9 Apr 2024 09:25:30 +0000 (UTC) X-FDA: 81989460420.15.2882FF3 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf22.hostedemail.com (Postfix) with ESMTP id 94577C0006 for ; Tue, 9 Apr 2024 09:25:28 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=p5Bnq1HO; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712654729; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6QulrtiHM3qMf89NRSaECQx6TgoGBwLTjWk8wB72ft8=; b=XmiccHuHdU2Ys/11N5+Dqvdn+Vo54Fj0AnHxKPlIgR8hsGlOBEDbEEvWOT2hQJkCxJLCPM 5N/pIa51WDScR31B7apohk+9qfE11NPd2sdwZ7VyuuiagFOPxblQpDyXms8bs1/skSImHF o/ok8ZQUYvijLp+bwaGak1rjuy4tZA0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=p5Bnq1HO; spf=pass (imf22.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712654729; a=rsa-sha256; cv=none; b=QIXDfpGPo6oIodO+iCLpa+3zPuvGG7HxSIyvV1NehwoXk2N9OgwZXa315IiPMbw5RhKiwo XT8SQxHq83FXACdfNWETAeXtlDHOIWUQKJjh53PvLJnRIgcUKaZxbINVrk8Tp6TE6eNE7f BZTPKKTEip8uBlZAViAGbgvUJE68BRM= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id ECF6ECE1DFF; Tue, 9 Apr 2024 09:25:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 215C9C433F1; Tue, 9 Apr 2024 09:25:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712654725; bh=75r9vBcHOLP/QZTkSWPs35KE2gnggcdp1NXQWqEu+94=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=p5Bnq1HOAUtLmDUOgf30B1us3yYOvoqfbsqdgljPW7tCMagB+HFvKmmnfPuHCqaAA WPbEjJPAWmlTdry8oDNy9VfvJlw3qvFvbmUGQul6e8oP+RnraHBE2TakqzFIhvQuIq gphN4ixndTfyy2XxZ/Ih4+dcjLFX157nljTFeGMVlsqBhy/vxZP0XlmVcr8GIPuJKa g7j8plpLvXti8Y8cI7oq9RyAyMvD0AqSfkFNlBH7UdUQtfuiXx2IH96mDn4IDRkCZo dWGzI82T70q/SMpExS+ufP8Za7G3Te0vL34DVbmTxufVb5E77tIJ5+arJk5Z4eqxrd Gymd/Ri0z08AA== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Alex Shi (tencent)" , Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , David Hildenbrand Subject: [PATCH v4 4/9] mm/ksm: use folio in remove_stable_node Date: Tue, 9 Apr 2024 17:28:19 +0800 Message-ID: <20240409092826.1733637-5-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240409092826.1733637-1-alexs@kernel.org> References: <20240409092826.1733637-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 94577C0006 X-Rspam-User: X-Stat-Signature: ajbt5yxak9t553f4f41jojugmcy1sxyg X-Rspamd-Server: rspam01 X-HE-Tag: 1712654728-969281 X-HE-Meta: U2FsdGVkX19ujgzY03Z+Ug+CVn6nf1Nqi1ockrICOiIp6ajOlOuVh3Q94KsZ11wA8pP24ZAtnBnwV640U1DzUGFB1kP0F7U1zaSiWvwLy8tZ9sC6SeB6h7MTbtoQIRKgZUywQVGTULdVH3uo8OR4aJx3tNCmD7flSZqjrqp34T3WBL9HfZC0AxZ/kGmG1C/EUWG1yfWAFeCLTg0yaRDhrh1lH8gezFxcOUIKJQRFcVVqL+rCsSJR4iHGPhP21+GLeI/QqGRIsRnMbIBaVCzQijKrtZU/fOh8vNe7Eui2CCpiUVmpXP8LnjkBgB65qUHsvbmKZvneo+v6w1cpzPNTnIrJaLTcvEvo4vEOOfs8CGG6LjgU6RH6nFydqVQYk2xfb78/Cfff3rILvXiXJChbjjuWWfw4KXn4f8vMuAA31NxO1KNsJGavA+IbfkBmcxqxsGyftoIQUxcFLbrO46PC4wVXge44pElehQ4kDLZskpC+dDzpUzx0eTR5mGVxrj/uLWF9DkpMMo89nx1WjM687ikC0JDE/0HCbWz/ZbANziA51Wf27er5Tmhp6mrvJeH0JnlcICofCRDFiMmyPOwcnYD8VdAD7DxhmraPwOQKfeWgmn1Z+MRd47MVrHP5XqTkfUSGss8clxX0hxBCXRCuosPtCRVcjHTdKLWg4mxHrQtUclOLMnWFtrhNwxP2dkr1Fld3EyAXaepl1WWAwhxqazH9uT2GkYRh/bNR4U2IWKBF/NRGQ7I7ExCvKjVeoiNfQO/KAFFSo6J96fYFFKFpjM9WBG75MTTYfotXXJKpaGWijE35b3FaKrwDwM5Acphgt/EMN/Ms8fexOz/mmegszzrS03v5OnjfuPMzWwQ4j67HQI8TW2xVWQixh/IbM9T56uHDuRCQbqsoGuL40EDi0PCQpFIkHF0E/vc6jeR6OIQSuiCfdgMLdO2wjgdQOpHzRrB0u83Ld68WJJsTz0Q 04j1lsOB TWkjEmsHrbEA9s1PaYa+HZrj22mL0a+cY6gv6dFCQohYc3Gmr8kkQ9co09oyCqUF+C9Q1gCamgZ75SpICrruzlCaEK8b6NwaY9s/JPSvMI2KmxHk88NVHyXIrQu9jXQtclYvKWOGQDja95cbkY4HeUMzWGje8izN/njNYL2SR4stW/x3NUUfx3E7mBwLkb6xEUFsYTQZUEycb4ujEhZiXfQOTom3rieDydmiFjAyXyGpFB5Sx0uCzXfCHhvqkGr8HEYBgL2anBu0f07rEJ+8LdIwNviliPKBjMRrKG1leLpdu1Jc2PqYFTMj9jHLiSrJ9JepcuAE3BoatJxLDNQY5enX2xhjDgHXvjy28ILjpqRk2MsVfDhGo177NPKK/NtNyusvra7pI2fY+EsxgqBOrU86Xs9Q4/ylkeOq9pO7Om8xtX+ruiEHBSINCeJFWrUNv2rxF9+3XBBQtxPTtRJZtEOgMakEvKntLce5lyQHxn8NPzPwwaljWv9JjdZowXvIjNDjy X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" pages in stable tree are all single normal page, so uses ksm_get_folio() and folio_set_stable_node(), also saves 3 calls to compound_head(). Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index c9b7c5701f22..b6ee2bc7646f 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1121,11 +1121,11 @@ static inline void folio_set_stable_node(struct folio *folio, */ static int remove_stable_node(struct ksm_stable_node *stable_node) { - struct page *page; + struct folio *folio; int err; - page = get_ksm_page(stable_node, GET_KSM_PAGE_LOCK); - if (!page) { + folio = ksm_get_folio(stable_node, GET_KSM_PAGE_LOCK); + if (!folio) { /* * get_ksm_page did remove_node_from_stable_tree itself. */ @@ -1138,22 +1138,22 @@ static int remove_stable_node(struct ksm_stable_node *stable_node) * merge_across_nodes/max_page_sharing be switched. */ err = -EBUSY; - if (!page_mapped(page)) { + if (!folio_mapped(folio)) { /* * The stable node did not yet appear stale to get_ksm_page(), - * since that allows for an unmapped ksm page to be recognized + * since that allows for an unmapped ksm folio to be recognized * right up until it is freed; but the node is safe to remove. - * This page might be in an LRU cache waiting to be freed, - * or it might be PageSwapCache (perhaps under writeback), + * This folio might be in an LRU cache waiting to be freed, + * or it might be in the swapcache (perhaps under writeback), * or it might have been removed from swapcache a moment ago. */ - set_page_stable_node(page, NULL); + folio_set_stable_node(folio, NULL); remove_node_from_stable_tree(stable_node); err = 0; } - unlock_page(page); - put_page(page); + folio_unlock(folio); + folio_put(folio); return err; } From patchwork Tue Apr 9 09:28:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13622063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2EDCCD1284 for ; Tue, 9 Apr 2024 09:25:31 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 41C866B009B; Tue, 9 Apr 2024 05:25:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 377F76B009D; Tue, 9 Apr 2024 05:25:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 155676B009C; Tue, 9 Apr 2024 05:25:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id DE2FE6B0099 for ; Tue, 9 Apr 2024 05:25:30 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id A9577402D2 for ; Tue, 9 Apr 2024 09:25:30 +0000 (UTC) X-FDA: 81989460420.09.F4FFA07 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf08.hostedemail.com (Postfix) with ESMTP id 1037616000D for ; Tue, 9 Apr 2024 09:25:28 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="K/9py+Ni"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712654729; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=IbRO76iTSzroHX7fl9UkQl3yuoVMKDt79AtbotqUFNU=; b=QavtyyLCckZYO/TbN/Dm1KFeBgEH2iYJIiu8EZrR740HBIzPaxtYQbuagkQK2q2SA98oX0 geaVkAXkW+CxjvtxdQWAsHGnuIK6NQq5CSMYOFdVWntUFyjGkFr0kOE79ecR5Dfp0PJ4kG logNuOTAw5JnG1lH1EmQUYo6xxoNUUQ= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="K/9py+Ni"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf08.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712654729; a=rsa-sha256; cv=none; b=rghqukqehBUsi0OtdheNGV4zPbQP7JoMSZ52x2rZDAZZU8zKyeth9qHD/aHsLK3+NurY5o DMUBAz8MyI9W9bA3mCHoQe2+ixffsAzHyae68InzzhhUB3J6qfGTEelAsAGLxu3NBXM2E9 eQ1a1dq9o50WArk1WWJ0cZEc4zVvG7Y= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 29F0261684; Tue, 9 Apr 2024 09:25:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CD7A1C433B1; Tue, 9 Apr 2024 09:25:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712654727; bh=ZY9sin7F1FOM2/QeBR1n457pb/jjou4lTyaBx1XPflg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=K/9py+NidPXCVPUMGd0kufG/y63MT2AfemIKhalAx4FloWPEwHbZK2/NodbgYlfam zo+IBb/3dmx60VAHPosB7YEpJzjE4f6byWsF3cqBNDQTCYzHNlz1HVqKnDT2HCh1lK cUy1Qp/Nk+bw0qgcno/YuSUreXEUWW5SOgssY+aVr+ovZlF1o6DrO62VsIqPaDSqIL ttYkVNFKBA5orHBOuPQ5xlqvdb3bamLKbDz+es7APokW2gfKAb7pAAom0ncqZU5hyo ppGJR9OmDqADNk7NeIrzMkHFXmhTBU3CmUQdRp16HYybhutxFGJOQXHy2+KNamaACi QX7pHZuJY4FPg== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Alex Shi (tencent)" , Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , David Hildenbrand Subject: [PATCH v4 5/9] mm/ksm: use folio in stable_node_dup Date: Tue, 9 Apr 2024 17:28:20 +0800 Message-ID: <20240409092826.1733637-6-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240409092826.1733637-1-alexs@kernel.org> References: <20240409092826.1733637-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: yc8p19c8opumjoxfc3ugr4me15ech73s X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 1037616000D X-HE-Tag: 1712654728-516354 X-HE-Meta: U2FsdGVkX1+NyPDgA2RMiUjn5VOYayZQX5H5zUfTdlwhahC0Fd8xmql4sIW8wqjDirpdRHfYsL64L4zhqj4lzIu6cipDzr28gZfQaKFMuT4oqcvr9DElg35Zp/gqgzLQQYykgYDTEdInU/LxxaHVEte6aOaOXUtTuZrqP7X66BtJENSvucLnm5tz7ce2BE4zMdW5ygBgtz/F9wiLfzMzdngmWEIXrT2GUSJ+fOdTKRdxrRw70Rac+0Et1TRckEQYwiFJ8MXj4CLViF56rSRSblZiyPEU4jiguzmBq2+DL4hcF1QtBjgh6qsSDeXMAFCwwi6nYfHJ+d4KOBliziimzdP76GOhiRmhy/r5w1jahtxMbNncTtpsrQTjfVL28/Yq+/13mTSsRA0UU8+Tq1ozHTO3CLWqG9oyHpwsf5N0S2STSqscpql4FlavlbrEAv+fXbmQiPnD21mOALMqY7D4dANxTKfnxqqDfb40GpdaLnB9W+CddANRLuQL9wHgwtxHUKM5J+RWN25zYIU8GPhX1vQIyUnghXfind1h3H6RBplYhkcBzoHwALwLZE0/80zLU0Rn3ovKwh2APYcOnDbTiZhfjAC25nm4FuwOCioFKEPrSj3vBgznANpP7icpfHsEIdraZ03n1akN4/U2PAVtrVsL9wN5NgFQIkWsLlCKy2MTH1fDLRXf9/r/9OA6O1ncKoZ9qj/WlaEueX8rjbgkM7s9EiYYx8jMvs7lGTNqtCHH4hyrIWm/CjDjtoLlDueBE/xEGnM73xxBiKkmpoXY2f9kUfkp/U8RBia2+locxqhrhtCDkRag2d6ch3+Her4hcmlRixgnNPMjPn/IcyFayhMUqZT4QCMkTHsZeWHmuEqaYLq/7gutzmQ+wbvinCIl3PkOT4yhLcg/uqaBUYCB/LPiZHvAYQLCI0zu80ynxAJHFcnJqzQThF2Z08QxP0ewxd4u3D9gE6MS2IJLTgy 0RpzfinD /yNWAH3DkCkvAOnn4DFCwgxC8buVHwkj0QMFMcqLn03ird955ZHumpbZD8byWMjktJgf5/BIYm+KSGfzrTcebj0jT1XueZ7rMoHGXWB7KOGELKNV1jbEH5FcRTu24HCGo2CufhXewYExsSnIS9GJA/HjksbViW2nCKhkLmVj78f/umV5sCYFMkb1DvYS47+1F4klM0EnlR4cxHLtI5D9Ur5WVKuVnyiUJQWEyQewQNyCSjC5R2CRO1yDnKofHo53O7g6kZ/RoQy96QrGdLb2JWJi3lm4L9a+y0/PzPq2AcOBMG2qSyZgvFNj93vF8S//k9TmjYucNciYqvJQz0IAn5Q5fZ6y/U+3RjQYCv+JRPRI42A0AMjbVfo9yE8pQTXjD3EU37Y2Qt6OHA4bDNMwbbvIMrEBowifdS2hyo2OEDaQHskidJRBLXWrlmRYxQcKjZdLoKO3NruQXA1IMidioaQ/KR5/PJfBNtr5imT643HjW1ACX/0l1G1er1q4BKhXwXp+m X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Use ksm_get_folio() and save 2 compound_head calls. Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index b6ee2bc7646f..aa80fbf3a8e0 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1638,7 +1638,7 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, { struct ksm_stable_node *dup, *found = NULL, *stable_node = *_stable_node; struct hlist_node *hlist_safe; - struct page *_tree_page, *tree_page = NULL; + struct folio *folio, *tree_folio = NULL; int nr = 0; int found_rmap_hlist_len; @@ -1663,18 +1663,18 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, * stable_node parameter itself will be freed from * under us if it returns NULL. */ - _tree_page = get_ksm_page(dup, GET_KSM_PAGE_NOLOCK); - if (!_tree_page) + folio = ksm_get_folio(dup, GET_KSM_PAGE_NOLOCK); + if (!folio) continue; nr += 1; if (is_page_sharing_candidate(dup)) { if (!found || dup->rmap_hlist_len > found_rmap_hlist_len) { if (found) - put_page(tree_page); + folio_put(tree_folio); found = dup; found_rmap_hlist_len = found->rmap_hlist_len; - tree_page = _tree_page; + tree_folio = folio; /* skip put_page for found dup */ if (!prune_stale_stable_nodes) @@ -1682,7 +1682,7 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, continue; } } - put_page(_tree_page); + folio_put(folio); } if (found) { @@ -1747,7 +1747,7 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, } *_stable_node_dup = found; - return tree_page; + return &tree_folio->page; } static struct ksm_stable_node *stable_node_dup_any(struct ksm_stable_node *stable_node, From patchwork Tue Apr 9 09:28:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13622065 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 983DBCD1284 for ; Tue, 9 Apr 2024 09:25:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E2AC96B009C; Tue, 9 Apr 2024 05:25:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D636C6B009D; Tue, 9 Apr 2024 05:25:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B426E6B009E; Tue, 9 Apr 2024 05:25:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 872D96B009C for ; Tue, 9 Apr 2024 05:25:33 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4E73AA08E5 for ; Tue, 9 Apr 2024 09:25:33 +0000 (UTC) X-FDA: 81989460546.28.D0D9CC4 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf29.hostedemail.com (Postfix) with ESMTP id B9EDC120002 for ; Tue, 9 Apr 2024 09:25:31 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Gic+mMF2; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712654731; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=pcZg1c5VCocn3Bzg9z8DaXxlAUkYxtwMcy/PnEDZf40=; b=qx3+zc8W6Bl8r7NxYhKMojilGShG8tiIsC+VjKcaQhbHZSoR1sLKHpgkBZdEB0rd3axcKI afQcKVaneOhSyY3ehh8/yBVWyN5ut0qXUT43iZmLEmsOPZURPWuGWxTsCsba5S+lqbgUf1 fnsorQSGxZjaKbSe9YmGXEfFemqq6gU= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=Gic+mMF2; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf29.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712654731; a=rsa-sha256; cv=none; b=1RL0uowmh2B46JsuoBY00F1hWuOqNYUOIOeEfLPG6/4vHejTplBDqu260PB3pu4wMMP3FQ FCINpz7noZPgvynZiq5Hmcq7nM/3avR6SU8/OWGjaIJcs9qh0d+zrJEKAUviXRCHRg3LQv jWxeLhuYqWKfp4WPL5zf4fEXDrSRaO0= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id EF0D361683; Tue, 9 Apr 2024 09:25:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 86B23C433F1; Tue, 9 Apr 2024 09:25:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712654730; bh=dJ5DRDYCIT+gow/J5pRXi3ey1ZJ/tDnBvGy0lEgeqkQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Gic+mMF2M3oHglqJidfptsSOTIB5WDDSmFo1b/2IAIsAgt8Oz447ET1mybxFY1INy BgMffSGcxZEVDBTTmnHAZjUoxh3ayyAO9oI5fOFrMQIfWYjbl1kbAUnVI3tgxFAA1t WX6cYtXy20umo9u/uX+OznEJ9GemIxIa3Z1iYuqiNvdEQpTVsR0UbaCYphrDbKI2+k HCvjTjAm2AEyo67B3Vrn9+TIOQ6OXfMPFuEwHROErNbWDFkU7d6feG8hxPQ45Y5gti H8yg8goUkxXPDhDblpLIV2sIpGyk3ZceEplmGuo4cF1SvDwiALUVKeZYYN2/DUah6u aHaCRd7rZJIlQ== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Alex Shi (tencent)" , Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , David Hildenbrand Subject: [PATCH v4 6/9] mm/ksm: use ksm_get_folio in scan_get_next_rmap_item Date: Tue, 9 Apr 2024 17:28:21 +0800 Message-ID: <20240409092826.1733637-7-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240409092826.1733637-1-alexs@kernel.org> References: <20240409092826.1733637-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: B9EDC120002 X-Stat-Signature: sbbuscqb3m3yayckbitobt8yzmpjwcms X-HE-Tag: 1712654731-566154 X-HE-Meta: U2FsdGVkX1+vrkTYSTs4/TS07B2JXuQYD6vgaty+9rX28B8SJPKs9oML66JpRi0icy3e73qiC1P8fto7M3Sw9uCXGh3PFvpiSTtrJdwXobv2wJWcMxsv2Keb4ZomUX4P9vw3Nwm7Cgd3WeAShKX/4cIBeDy032bZUyl4iNMDHrRxZcuRS0wxE/47kZWkVl7tg4exULs/C+uq7nG0jeZlC98Ie4xdtmVNgFf+1wUHrMyyaiBDOAyCJXTB8fBpWlaJsvi8G5fpWWBvstEH3pGrbpP0wGS1PMS9NMXUt9u+FXbntqaN1cAoIxDkh37TSUXQ40Iws9mSrtv7gq+6UKieIrtb84Yjt9fXA1erM1MUcCkfguwmMSFdPXt703oI2nEVjmeT58ISWGu2rleIxxpQDVj1+BZAL5GMbash/zjGzG46IDZbgyYoTy65x51xHJlGvUl/lfa+QKtr5NKMkbB8o5EtGad6fcrtBggFg/SaW/8Sm/eqPovrJHGEgunV5IfRfrL+AJhetQpZ4Z4a4MNMKNMDA2Lx2tZPIrzS1bqAsl6W5NzLV4Jyt8pwkoQQv/Kr3RnE3QlLv1YGkMYgkzJ9Y71CtCHzWdD++DHdhyhHi3g0MMOHfWnsrESoJcJ4dm4oxeDVPmmEwVMticJEyOROEJCknq6I5VgcBA71ToXivBxvgmZ4bDrLwLMKGZ+LkkdrWHGvoA41CWZQsJOCpZ2kQ0am9gS3QLAlioXZY3D0scX2mg2RoZbsdV02QKxoP7xNzvKbhdYLiEhqqcJruQrjO16Pg28vYG0swhBfaiI+xXodldzvEplzhE5+AXnu1zHiimty4rAq2NUzgnEPtG12vOiUY3kdrpPcCLJKyaqulfz1sjuCcz8B49wJaRpOpZNrqnEuXA9FwCnvF+IXC7guTf14hafTwDNUpIz7UMxKRgqz+biGrJkuHF7BPjMAu6M+Qt2iJOatRwrQ6l5ifVx C6aWA34I qUQSdNyjPjPbkFJpaGFdhkfrM570Jtmv/rQR+MpnT6q68nivpr021AL6/TMXXAMWgFwlNJ8Lol1qoeKs7CEijitP5bkpgtkYz5L40WjQoM7tlrtfp1v42gnKHC5rx8ciLiUMq2B5YUhFEipwArwCvTLPOaeCgZCrSsTEb4KPNPRQGONesD4rdjiQvkxtJG+l9E2PvBHIep0L6qpu2YtTrVxPlMwQzDtaB4L6BLUj17dWAn6uu5P2M+KGI+0mfD04ZAcaJ1R9FCsV4d+e0WOMWkoeHdHN1QevWGHe/O4ToHc2ih8ZgF4lFAPFnatisKJM7XXTqmqOiUpao5KHK4D44tjffro/BKX7lwCTa4CDPuPnLiz+GvjKr+Q4GUHNGTvOxl+caGMTwyCoI7dcOJjQsq69/yLXJKODKTIT4zCquL2YKvKJFzb9sZmcZEoWlS4Nic4c5VB4cNiS0EwdS3NPumWq3RFkcdX1ykYOnf5Dlw2IyMC3j+IwBtF6x0mVywCgIjv2x X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Save a compound calls. Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index aa80fbf3a8e0..95a487a21eed 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -2611,14 +2611,14 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page) */ if (!ksm_merge_across_nodes) { struct ksm_stable_node *stable_node, *next; - struct page *page; + struct folio *folio; list_for_each_entry_safe(stable_node, next, &migrate_nodes, list) { - page = get_ksm_page(stable_node, - GET_KSM_PAGE_NOLOCK); - if (page) - put_page(page); + folio = ksm_get_folio(stable_node, + GET_KSM_PAGE_NOLOCK); + if (folio) + folio_put(folio); cond_resched(); } } From patchwork Tue Apr 9 09:28:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13622066 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2BAECCD1297 for ; Tue, 9 Apr 2024 09:25:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 725556B009E; Tue, 9 Apr 2024 05:25:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 637F46B009F; Tue, 9 Apr 2024 05:25:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4B1696B00A0; Tue, 9 Apr 2024 05:25:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 217E96B009E for ; Tue, 9 Apr 2024 05:25:36 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E3A2080328 for ; Tue, 9 Apr 2024 09:25:35 +0000 (UTC) X-FDA: 81989460630.11.BB9A00F Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf14.hostedemail.com (Postfix) with ESMTP id 4394A100002 for ; Tue, 9 Apr 2024 09:25:34 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eXbjGhv4; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf14.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712654734; a=rsa-sha256; cv=none; b=SsKPbelV71sycoIQ9j0V+MhHru4tvFSSqwu1bymye5Z7fLW3LJkQ4BwMwD3E701qy89s4Y 8FfCVCl0h2BgqD34fZ+WZLLggcVCqcmVmVrYnc6oaMjP6yClogWgobPbfibm5UQpq4YSp4 0PCAjPDlwppOds0vsY0cyWM7Ek6/LlM= ARC-Authentication-Results: i=1; imf14.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=eXbjGhv4; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf14.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712654734; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=mxoFzUrGhXaDOktVgy7CK9L3BNl27BfH9gualQu8PvY=; b=1jUE0rY/Catj5PPUGjtJsKSwSw/mGJ63I0US6NoWLFLeQXqs7vcfytr0JDwQeS/Ekz/dXK +1vgehd2IADogm+KieM3o2EX9hJX4Tl9MQHS3VR5tgV+fBzKj7WBDSFsa+WgCnKCmw5vyt aLQErWxAAuhPCuJPY/Lhb+Jr+EM/G9w= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 8A38061655; Tue, 9 Apr 2024 09:25:33 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 4145FC433F1; Tue, 9 Apr 2024 09:25:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712654733; bh=kQN6FVtHG3hCQ1muL4e6/z5sNInnsnuhNQpT8qXQDKE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=eXbjGhv4j5QR/j+cdEHV+7W1koJXqWz0Rb1a9u1mejXGAL6dCIqJ9poZ+7mJMyPUQ MehaKYskrhsNlDJYo7S+wZ7vfMjpdOoT6Pq4ShQfZjkvD0QIDex94hl7XsYYf2E9YJ j57kxizvU1otNbZOiGJrxz3y4Q3QnaD794ZR/3Jn8aGO/lzghT3lAFBA1h8WHB26mj VZFsfFvXHGVlVtDCsaupYUkBkOCzGZwUDifeCCNVvwdKV/sRVIA8vQZsnCorU89iyF u9xuYV0nQrm7N4XH8chMU/kdGbvnuUZCWJxqY/AqNRoCE5Rn/gVb0ViQXJgFTgAhdF 6x1JKxmdZ52rA== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Alex Shi (tencent)" , Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , David Hildenbrand Subject: [PATCH v4 7/9] mm/ksm: use folio in write_protect_page Date: Tue, 9 Apr 2024 17:28:22 +0800 Message-ID: <20240409092826.1733637-8-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240409092826.1733637-1-alexs@kernel.org> References: <20240409092826.1733637-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 4394A100002 X-Stat-Signature: dgcpufmuqdnbpj1prh6pgui8na7t1xoq X-HE-Tag: 1712654734-288319 X-HE-Meta: U2FsdGVkX1/N9Bnxer9Zx7wGQ7L7eAw3fJ61vt3osgEGPvdSImqaj8o7BeMYjSNXwi+HfmlHRmucYHcXZNw7vBiT0OPiM6WtStXkg3lWPdI23tEpdcuH0RB8AOWBpFMGhAJW+T94CxGJoXmzUBAtFIE1fLQE+xua5sAkvsToQcJPcrxRxWduE6zct2nrEYMlB1aedWdqfkKVqdbv8jFLJ8gUepx0xJf0wiQ5BsgplVVqUpJnjbm0F6I3OW0/3ngFVhNN081eGGRru4kZ1BGVNSsOVZvABFO9HgbqI5OSIveQhl2QMfTcnj5JbYJgO7EO5LcMQNEcplKfhv4MytGSCgDOS42MGs52+Ntt5tYO8NNKZ07RbTQfyUppJ7TJfeG/Xb247yO7VThvMRdaczIp3AA1sEgTTCi1OS5jB01nJdgvtBh/Qg0n0dEPa59hRIORC0WQrqqc+7DT818v9zBBrIvcbSqe6R+PFn0uPTaOAJU6AaYF+Yy9IPe6IW9gTxLvkahZWMIiHwu4znAc6Z4qiDW5n+3pIM1deLo5+IvbHdqmAYYkpuqm2nsK4Wf2etXb2nmneFZZaf1lqLqvs7Lc+Za+OlOBTFGAeMpVMRo7im3h11eUvuICqy1gV63DB9RnqzsVeqpRwHCeC4uhCpVFC5qCZvkrvEpBXInGUhnxOSovFI0xKELQoKTf2FlCHp3qOrzHnViyt+XYVh66IYgyiTIVHoXz6OWDRmQJCrV+jkrLNGWRj5ealY6qSkUwo3bWlT18ExExpNOy4BPQW2+/LPXxOX5e+Ph70RQuqv9v1VahwsweDyO4oJJRBue6BnTjgh7KFx22KZ6xcEFTpM2qsAY6nFX/aNiVD+lCuPul6WbY1MJ/JMXfY7wGVCGZSai/pjsF3F8TW/wNmUFAIx0f9/FEkn0yDKWEEzC3EsBWJutfY9dZ3ozNrP1yXh0avyEuSvB+qlGCOGoZUvRqHnp waEVox3U QIlNNnss6zFvrXpVRu2F1g6fweFGeGJx7B+D4DuIxFMuox7y/7gs94dYX39sz/IdBWpCVkYD+VuhZLgTtJDsLJ/L9rlAYhmo8oF9niP0JILw0K3jXDmj95OGQVnZ5zsbw/iww87dSWQkMlGd+l6ojVB0bq2Q4CchI8OAeoMEJNM9bCiRjr6AQ6HtfjdU9VRF5VUg93qX56akJpIizx9MZujDmSMgLx6G2TGhmTYwZhNviOQfCpVaZNoOJZfa/tQYmJDPI5gQcx7Ldn30pJMghL90Pxvqeidtb187Y0aG0G581ZtMU1yMDCWh8oi+KSpxUdyrcButg03csjMBuvAK4UMGs6Mmuh4hn/Ukav38Dovx8gDzQWr53/p/Sja++7Rt5GC3itelZG5uSDMHVR3il+H+0D27vRQb0beWvLR4NkF3HdpJxB51qoPE/twK3vh+6dSgU83ZyvJVAdD4hchgXO1digwLPLwZXtDvLFWc+WAFB70VuYti5bmvjk5SnMIkvm/mu X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Compound page is checked and skipped before write_protect_page() called, use folio to save a few compound_head checking. Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 95a487a21eed..776094fab34e 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1289,23 +1289,24 @@ static u32 calc_checksum(struct page *page) return checksum; } -static int write_protect_page(struct vm_area_struct *vma, struct page *page, +static int write_protect_page(struct vm_area_struct *vma, struct folio *folio, pte_t *orig_pte) { struct mm_struct *mm = vma->vm_mm; - DEFINE_PAGE_VMA_WALK(pvmw, page, vma, 0, 0); + DEFINE_FOLIO_VMA_WALK(pvmw, folio, vma, 0, 0); int swapped; int err = -EFAULT; struct mmu_notifier_range range; bool anon_exclusive; pte_t entry; - pvmw.address = page_address_in_vma(page, vma); + if (WARN_ON_ONCE(folio_test_large(folio))) + return err; + + pvmw.address = page_address_in_vma(&folio->page, vma); if (pvmw.address == -EFAULT) goto out; - BUG_ON(PageTransCompound(page)); - mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, mm, pvmw.address, pvmw.address + PAGE_SIZE); mmu_notifier_invalidate_range_start(&range); @@ -1315,12 +1316,12 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, if (WARN_ONCE(!pvmw.pte, "Unexpected PMD mapping?")) goto out_unlock; - anon_exclusive = PageAnonExclusive(page); + anon_exclusive = PageAnonExclusive(&folio->page); entry = ptep_get(pvmw.pte); if (pte_write(entry) || pte_dirty(entry) || anon_exclusive || mm_tlb_flush_pending(mm)) { - swapped = PageSwapCache(page); - flush_cache_page(vma, pvmw.address, page_to_pfn(page)); + swapped = folio_test_swapcache(folio); + flush_cache_page(vma, pvmw.address, folio_pfn(folio)); /* * Ok this is tricky, when get_user_pages_fast() run it doesn't * take any lock, therefore the check that we are going to make @@ -1340,20 +1341,20 @@ static int write_protect_page(struct vm_area_struct *vma, struct page *page, * Check that no O_DIRECT or similar I/O is in progress on the * page */ - if (page_mapcount(page) + 1 + swapped != page_count(page)) { + if (folio_mapcount(folio) + 1 + swapped != folio_ref_count(folio)) { set_pte_at(mm, pvmw.address, pvmw.pte, entry); goto out_unlock; } /* See folio_try_share_anon_rmap_pte(): clear PTE first. */ if (anon_exclusive && - folio_try_share_anon_rmap_pte(page_folio(page), page)) { + folio_try_share_anon_rmap_pte(folio, &folio->page)) { set_pte_at(mm, pvmw.address, pvmw.pte, entry); goto out_unlock; } if (pte_dirty(entry)) - set_page_dirty(page); + folio_mark_dirty(folio); entry = pte_mkclean(entry); if (pte_write(entry)) @@ -1519,7 +1520,7 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, * ptes are necessarily already write-protected. But in either * case, we need to lock and check page_count is not raised. */ - if (write_protect_page(vma, page, &orig_pte) == 0) { + if (write_protect_page(vma, page_folio(page), &orig_pte) == 0) { if (!kpage) { /* * While we hold page lock, upgrade page from From patchwork Tue Apr 9 09:28:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13622068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id ED3FFCD1297 for ; Tue, 9 Apr 2024 09:25:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3EF1A6B00A2; Tue, 9 Apr 2024 05:25:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 351836B00A4; Tue, 9 Apr 2024 05:25:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F38586B00A5; Tue, 9 Apr 2024 05:25:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id BD75A6B00A2 for ; Tue, 9 Apr 2024 05:25:42 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 87853A0311 for ; Tue, 9 Apr 2024 09:25:42 +0000 (UTC) X-FDA: 81989460924.11.32FDCF5 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf26.hostedemail.com (Postfix) with ESMTP id 35072140017 for ; Tue, 9 Apr 2024 09:25:39 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=STnJqxeB; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712654740; a=rsa-sha256; cv=none; b=0g5rIILCM4AOCjW09xi2ZTBXxLyO3kPCuCgwTXHSValCn6DzNcJP+sXZZ6LXvffGwHWbTE fMiuFyhQz4jZtiqAj61D9Vl4Sb4hprhNAeUk98hzAT2+eGoZqKc0uMOlBSpUzTTexyI7Yg qxv+TtiEvR5W+xxeN2ILYIsnpr7K0FQ= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=STnJqxeB; spf=pass (imf26.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712654740; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=qm/f8fsMYkhJdBntKClcDD3+KwLUGf2B4OJOAbWSh2s=; b=jc/dwkNRugSZnOmpnp+Zppb3caUBnIn4ZlHqrfHDTKUn+ZOSHQ+rCXSzKkdNIJm7y/a9pu Al5tV7ZpAMbHOimecmKMx/Y/Bn0pChbBUs9GaXtWf7dNxd+l1NxM4OLC+o2YmIH0VcFzyx sU1FRbuqcBomX3B7gnFretQyZbwj22g= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 13ADECE1E01; Tue, 9 Apr 2024 09:25:37 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id EC10FC433F1; Tue, 9 Apr 2024 09:25:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712654736; bh=ToFHqwNrZLOSNZtndww4urHDf8n4ZJAaJsmGP6gBu+o=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=STnJqxeBDxOdiZmfXD/9sLEdlt0U3v7CRWglZW6qzBAOMoHwZrcT26c9bEmFiJPLx zoQ+FHn06k9q5TdOaYntTZKQ4Pz3OyMXILEMazpCrv4Zp07kLjFOEQ+tDOdK3aEhha ZOyPX/PpWbYPXDfxjLTu9CQbUN/YXCF52ZVPPszcx0he7nP462ZCgPv6746CSAiMV0 h7Hg/7siOaF8wh48RC9oVHWEnUie/uTV9J2w8skYN058rAJh9LeffLrRnXrfHS0/Hk aJ0MVU5ugd8ZZGIbT5Y5f1xclzXTOcBEJy0epduhoEbQv2ZHHLH8OeVDuGy7Drmv/S aKlNDyZC6Hf0A== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Alex Shi (tencent)" , Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , David Hildenbrand Subject: [PATCH v4 8/9] mm/ksm: Convert chain series funcs and replace get_ksm_page Date: Tue, 9 Apr 2024 17:28:23 +0800 Message-ID: <20240409092826.1733637-9-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240409092826.1733637-1-alexs@kernel.org> References: <20240409092826.1733637-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 35072140017 X-Stat-Signature: hp9qd9s5zf9h631bz1zj7ow9f367cp8f X-Rspam-User: X-HE-Tag: 1712654739-210506 X-HE-Meta: U2FsdGVkX1+t/yDPWjES7TF3ENY1UnWaCqwhJiVv8qAAHdpc3El0iyDPdjCZ+veGmo0DjkvXfWMx3LSrqFoGvJAUhXYUPp3jrDbn4iwQWIXssExsN4iIbzrhJPvsRcrD+rlvMGl3gk3BQhI+VHhkh5fEWrT6953ILCYopKJiYmojUFxc2/V6qAd9gLGwOpqeAQtkYsKAr6F/UB30Z46n9BXsLA12aCp7v5ynbz92/PjBd3bRQV5SxIP2u1KyPYOcbk9qc8FNoONKTUDGCzsE/y+MQlKrFhUDrfxbNIIwYUV/R7YClaoGRygG8ZGhevsfhBlu44nntKJbCZS1z9RuFAku6xpsQlwyIpx38TOWeWg4QNjOHneuwbO3VFi9u003rF5z1t8PvYL/UBFEtQtTAoFxcuP8qL1PrlH+kriFcQUW6702Oc2x/25dDU9SmmMa/2URZuxdbJkAObQNDLPDiH8bfuP/DwzUIJPGzeW0z4SrVDh0TV4iWRY0lQo0BQF8jBV9cbWQFicdo26wNJoldUDpsVFShBfv/nU6rL7aV0sMx0nfSN4NyBO0nBFyWU8YvhRcSHH1IZNlRGu2oTV6inFxsdJkbDdTihqydsgRpdTPVPpB3pLeK2q1/tZY2FZr+mZWAYdA2gzPMgRchBN7/EtalO0Tt9QSEkh6etJt77nItlDiV0o5dR/g663qPpPstOyHg4lydvAcxhFpiVQN8sRqjpJggGZG+mrtDK1/OG1+L6oD6vQ6H77U+bctsIJsef0GGrtPk7o0tTlHx4uR23xqRLxMHzz2vuPP6bVeLovS4yt8dY1kjpjbnWxmiaat6JunzyRpRwWQ2lUAccZ6u2rAkb1QdKsvxlTb3VwzDB2lWtld4n4jqe2HrUpPLmjBGNfrDbX3CTtpS3c8IdZu5xzo5x6cfQVFoMlIMI/3P0bpNvJanFZz+DPoFpdn3Gs+Q49/beh99WXEv+O60ff fLG3DsvX TA9543DNy5mUFl8Fv48t/FFxrxRl6lRfYAdts3rwXizXJS0e+L+5cgqdRAGfFK4ceuCGQeFP2xEWl3jy+8u2MXbGK0/SJ6nW081CzfoqCR7qqgl3x9ivmL6maaL5grGuH7WSVs52SE+ar29d0oC/CUMnc6TkEUKd0MLaF4N+Fe0d6jMhsFIONwLNWdA+f2mFRVOevakuhd2WEoumskR0g2NUQJy3pbrByTNP7FQVBoAG8cLPfDjKPSX9Rjl55WOJpKUVllqsYFJAQm6MDglKThop67s+6+ZeqkaDRJnLu7MdcLi0jWEMQgHJmdDnbfBRyVgzMqHYLcHm5Jwso3VD/2mq4l9f/FRarZdX3old9X/t+SDqjYVa9PVD4FPDYF34ftMiAdMFrK2DwT6HI4iiIOc6VOBFyIDHi8s48iZsVeYw5ZsvleaFJY/a6MKLKl0UPbhH6qriuc+Mmo5uJlV7DS+e+7FNifWDNU+ZVgl2/X97N57pQftY4qXOyAImwwU5qNpdC X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" In ksm stable tree all page are single, let's convert them to use and folios as well as stable_tree_insert/stable_tree_search funcs. And replace get_ksm_page() by ksm_get_folio() since there is no more needs. It could save a few compound_head calls. Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand Signed-off-by: David Hildenbrand --- mm/ksm.c | 150 +++++++++++++++++++++++++-------------------------- mm/migrate.c | 2 +- 2 files changed, 73 insertions(+), 79 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 776094fab34e..a705ed16aa1b 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -890,14 +890,14 @@ static void remove_node_from_stable_tree(struct ksm_stable_node *stable_node) free_stable_node(stable_node); } -enum get_ksm_page_flags { +enum ksm_get_folio_flags { GET_KSM_PAGE_NOLOCK, GET_KSM_PAGE_LOCK, GET_KSM_PAGE_TRYLOCK }; /* - * get_ksm_page: checks if the page indicated by the stable node + * ksm_get_folio: checks if the page indicated by the stable node * is still its ksm page, despite having held no reference to it. * In which case we can trust the content of the page, and it * returns the gotten page; but if the page has now been zapped, @@ -916,7 +916,7 @@ enum get_ksm_page_flags { * is on its way to being freed; but it is an anomaly to bear in mind. */ static struct folio *ksm_get_folio(struct ksm_stable_node *stable_node, - enum get_ksm_page_flags flags) + enum ksm_get_folio_flags flags) { struct folio *folio; void *expected_mapping; @@ -990,14 +990,6 @@ static struct folio *ksm_get_folio(struct ksm_stable_node *stable_node, return NULL; } -static struct page *get_ksm_page(struct ksm_stable_node *stable_node, - enum get_ksm_page_flags flags) -{ - struct folio *folio = ksm_get_folio(stable_node, flags); - - return &folio->page; -} - /* * Removing rmap_item from stable or unstable tree. * This function will clean the information from the stable/unstable tree. @@ -1127,7 +1119,7 @@ static int remove_stable_node(struct ksm_stable_node *stable_node) folio = ksm_get_folio(stable_node, GET_KSM_PAGE_LOCK); if (!folio) { /* - * get_ksm_page did remove_node_from_stable_tree itself. + * ksm_get_folio did remove_node_from_stable_tree itself. */ return 0; } @@ -1140,7 +1132,7 @@ static int remove_stable_node(struct ksm_stable_node *stable_node) err = -EBUSY; if (!folio_mapped(folio)) { /* - * The stable node did not yet appear stale to get_ksm_page(), + * The stable node did not yet appear stale to ksm_get_folio(), * since that allows for an unmapped ksm folio to be recognized * right up until it is freed; but the node is safe to remove. * This folio might be in an LRU cache waiting to be freed, @@ -1632,10 +1624,10 @@ bool is_page_sharing_candidate(struct ksm_stable_node *stable_node) return __is_page_sharing_candidate(stable_node, 0); } -static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, - struct ksm_stable_node **_stable_node, - struct rb_root *root, - bool prune_stale_stable_nodes) +static struct folio *stable_node_dup(struct ksm_stable_node **_stable_node_dup, + struct ksm_stable_node **_stable_node, + struct rb_root *root, + bool prune_stale_stable_nodes) { struct ksm_stable_node *dup, *found = NULL, *stable_node = *_stable_node; struct hlist_node *hlist_safe; @@ -1658,7 +1650,7 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, * We must walk all stable_node_dup to prune the stale * stable nodes during lookup. * - * get_ksm_page can drop the nodes from the + * ksm_get_folio can drop the nodes from the * stable_node->hlist if they point to freed pages * (that's why we do a _safe walk). The "dup" * stable_node parameter itself will be freed from @@ -1748,7 +1740,7 @@ static struct page *stable_node_dup(struct ksm_stable_node **_stable_node_dup, } *_stable_node_dup = found; - return &tree_folio->page; + return tree_folio; } static struct ksm_stable_node *stable_node_dup_any(struct ksm_stable_node *stable_node, @@ -1765,7 +1757,7 @@ static struct ksm_stable_node *stable_node_dup_any(struct ksm_stable_node *stabl } /* - * Like for get_ksm_page, this function can free the *_stable_node and + * Like for ksm_get_folio, this function can free the *_stable_node and * *_stable_node_dup if the returned tree_page is NULL. * * It can also free and overwrite *_stable_node with the found @@ -1778,16 +1770,16 @@ static struct ksm_stable_node *stable_node_dup_any(struct ksm_stable_node *stabl * function and will be overwritten in all cases, the caller doesn't * need to initialize it. */ -static struct page *__stable_node_chain(struct ksm_stable_node **_stable_node_dup, - struct ksm_stable_node **_stable_node, - struct rb_root *root, - bool prune_stale_stable_nodes) +static struct folio *__stable_node_chain(struct ksm_stable_node **_stable_node_dup, + struct ksm_stable_node **_stable_node, + struct rb_root *root, + bool prune_stale_stable_nodes) { struct ksm_stable_node *stable_node = *_stable_node; if (!is_stable_node_chain(stable_node)) { if (is_page_sharing_candidate(stable_node)) { *_stable_node_dup = stable_node; - return get_ksm_page(stable_node, GET_KSM_PAGE_NOLOCK); + return ksm_get_folio(stable_node, GET_KSM_PAGE_NOLOCK); } /* * _stable_node_dup set to NULL means the stable_node @@ -1800,24 +1792,24 @@ static struct page *__stable_node_chain(struct ksm_stable_node **_stable_node_du prune_stale_stable_nodes); } -static __always_inline struct page *chain_prune(struct ksm_stable_node **s_n_d, - struct ksm_stable_node **s_n, - struct rb_root *root) +static __always_inline struct folio *chain_prune(struct ksm_stable_node **s_n_d, + struct ksm_stable_node **s_n, + struct rb_root *root) { return __stable_node_chain(s_n_d, s_n, root, true); } -static __always_inline struct page *chain(struct ksm_stable_node **s_n_d, - struct ksm_stable_node *s_n, - struct rb_root *root) +static __always_inline struct folio *chain(struct ksm_stable_node **s_n_d, + struct ksm_stable_node *s_n, + struct rb_root *root) { struct ksm_stable_node *old_stable_node = s_n; - struct page *tree_page; + struct folio *tree_folio; - tree_page = __stable_node_chain(s_n_d, &s_n, root, false); + tree_folio = __stable_node_chain(s_n_d, &s_n, root, false); /* not pruning dups so s_n cannot have changed */ VM_BUG_ON(s_n != old_stable_node); - return tree_page; + return tree_folio; } /* @@ -1829,7 +1821,7 @@ static __always_inline struct page *chain(struct ksm_stable_node **s_n_d, * This function returns the stable tree node of identical content if found, * NULL otherwise. */ -static struct page *stable_tree_search(struct page *page) +static void *stable_tree_search(struct page *page) { int nid; struct rb_root *root; @@ -1837,28 +1829,30 @@ static struct page *stable_tree_search(struct page *page) struct rb_node *parent; struct ksm_stable_node *stable_node, *stable_node_dup, *stable_node_any; struct ksm_stable_node *page_node; + struct folio *folio; - page_node = page_stable_node(page); + folio = page_folio(page); + page_node = folio_stable_node(folio); if (page_node && page_node->head != &migrate_nodes) { /* ksm page forked */ - get_page(page); - return page; + folio_get(folio); + return folio; } - nid = get_kpfn_nid(page_to_pfn(page)); + nid = get_kpfn_nid(folio_pfn(folio)); root = root_stable_tree + nid; again: new = &root->rb_node; parent = NULL; while (*new) { - struct page *tree_page; + struct folio *tree_folio; int ret; cond_resched(); stable_node = rb_entry(*new, struct ksm_stable_node, node); stable_node_any = NULL; - tree_page = chain_prune(&stable_node_dup, &stable_node, root); + tree_folio = chain_prune(&stable_node_dup, &stable_node, root); /* * NOTE: stable_node may have been freed by * chain_prune() if the returned stable_node_dup is @@ -1892,14 +1886,14 @@ static struct page *stable_tree_search(struct page *page) * write protected at all times. Any will work * fine to continue the walk. */ - tree_page = get_ksm_page(stable_node_any, - GET_KSM_PAGE_NOLOCK); + tree_folio = ksm_get_folio(stable_node_any, + GET_KSM_PAGE_NOLOCK); } VM_BUG_ON(!stable_node_dup ^ !!stable_node_any); - if (!tree_page) { + if (!tree_folio) { /* * If we walked over a stale stable_node, - * get_ksm_page() will call rb_erase() and it + * ksm_get_folio() will call rb_erase() and it * may rebalance the tree from under us. So * restart the search from scratch. Returning * NULL would be safe too, but we'd generate @@ -1909,8 +1903,8 @@ static struct page *stable_tree_search(struct page *page) goto again; } - ret = memcmp_pages(page, tree_page); - put_page(tree_page); + ret = memcmp_pages(page, &tree_folio->page); + folio_put(tree_folio); parent = *new; if (ret < 0) @@ -1953,26 +1947,26 @@ static struct page *stable_tree_search(struct page *page) * It would be more elegant to return stable_node * than kpage, but that involves more changes. */ - tree_page = get_ksm_page(stable_node_dup, - GET_KSM_PAGE_TRYLOCK); + tree_folio = ksm_get_folio(stable_node_dup, + GET_KSM_PAGE_TRYLOCK); - if (PTR_ERR(tree_page) == -EBUSY) + if (PTR_ERR(tree_folio) == -EBUSY) return ERR_PTR(-EBUSY); - if (unlikely(!tree_page)) + if (unlikely(!tree_folio)) /* * The tree may have been rebalanced, * so re-evaluate parent and new. */ goto again; - unlock_page(tree_page); + folio_unlock(tree_folio); if (get_kpfn_nid(stable_node_dup->kpfn) != NUMA(stable_node_dup->nid)) { - put_page(tree_page); + folio_put(tree_folio); goto replace; } - return tree_page; + return tree_folio; } } @@ -1985,8 +1979,8 @@ static struct page *stable_tree_search(struct page *page) rb_insert_color(&page_node->node, root); out: if (is_page_sharing_candidate(page_node)) { - get_page(page); - return page; + folio_get(folio); + return folio; } else return NULL; @@ -2011,12 +2005,12 @@ static struct page *stable_tree_search(struct page *page) &page_node->node, root); if (is_page_sharing_candidate(page_node)) - get_page(page); + folio_get(folio); else - page = NULL; + folio = NULL; } else { rb_erase(&stable_node_dup->node, root); - page = NULL; + folio = NULL; } } else { VM_BUG_ON(!is_stable_node_chain(stable_node)); @@ -2027,16 +2021,16 @@ static struct page *stable_tree_search(struct page *page) DO_NUMA(page_node->nid = nid); stable_node_chain_add_dup(page_node, stable_node); if (is_page_sharing_candidate(page_node)) - get_page(page); + folio_get(folio); else - page = NULL; + folio = NULL; } else { - page = NULL; + folio = NULL; } } stable_node_dup->head = &migrate_nodes; list_add(&stable_node_dup->list, stable_node_dup->head); - return page; + return folio; chain_append: /* stable_node_dup could be null if it reached the limit */ @@ -2079,7 +2073,7 @@ static struct page *stable_tree_search(struct page *page) * This function returns the stable tree node just allocated on success, * NULL otherwise. */ -static struct ksm_stable_node *stable_tree_insert(struct page *kpage) +static struct ksm_stable_node *stable_tree_insert(struct folio *kfolio) { int nid; unsigned long kpfn; @@ -2089,7 +2083,7 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) struct ksm_stable_node *stable_node, *stable_node_dup, *stable_node_any; bool need_chain = false; - kpfn = page_to_pfn(kpage); + kpfn = folio_pfn(kfolio); nid = get_kpfn_nid(kpfn); root = root_stable_tree + nid; again: @@ -2097,13 +2091,13 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) new = &root->rb_node; while (*new) { - struct page *tree_page; + struct folio *tree_folio; int ret; cond_resched(); stable_node = rb_entry(*new, struct ksm_stable_node, node); stable_node_any = NULL; - tree_page = chain(&stable_node_dup, stable_node, root); + tree_folio = chain(&stable_node_dup, stable_node, root); if (!stable_node_dup) { /* * Either all stable_node dups were full in @@ -2125,14 +2119,14 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) * write protected at all times. Any will work * fine to continue the walk. */ - tree_page = get_ksm_page(stable_node_any, - GET_KSM_PAGE_NOLOCK); + tree_folio = ksm_get_folio(stable_node_any, + GET_KSM_PAGE_NOLOCK); } VM_BUG_ON(!stable_node_dup ^ !!stable_node_any); - if (!tree_page) { + if (!tree_folio) { /* * If we walked over a stale stable_node, - * get_ksm_page() will call rb_erase() and it + * ksm_get_folio() will call rb_erase() and it * may rebalance the tree from under us. So * restart the search from scratch. Returning * NULL would be safe too, but we'd generate @@ -2142,8 +2136,8 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) goto again; } - ret = memcmp_pages(kpage, tree_page); - put_page(tree_page); + ret = memcmp_pages(&kfolio->page, &tree_folio->page); + folio_put(tree_folio); parent = *new; if (ret < 0) @@ -2162,7 +2156,7 @@ static struct ksm_stable_node *stable_tree_insert(struct page *kpage) INIT_HLIST_HEAD(&stable_node_dup->hlist); stable_node_dup->kpfn = kpfn; - set_page_stable_node(kpage, stable_node_dup); + folio_set_stable_node(kfolio, stable_node_dup); stable_node_dup->rmap_hlist_len = 0; DO_NUMA(stable_node_dup->nid = nid); if (!need_chain) { @@ -2440,7 +2434,7 @@ static void cmp_and_merge_page(struct page *page, struct ksm_rmap_item *rmap_ite * node in the stable tree and add both rmap_items. */ lock_page(kpage); - stable_node = stable_tree_insert(kpage); + stable_node = stable_tree_insert(page_folio(kpage)); if (stable_node) { stable_tree_append(tree_rmap_item, stable_node, false); @@ -3244,7 +3238,7 @@ void folio_migrate_ksm(struct folio *newfolio, struct folio *folio) /* * newfolio->mapping was set in advance; now we need smp_wmb() * to make sure that the new stable_node->kpfn is visible - * to get_ksm_page() before it can see that folio->mapping + * to ksm_get_folio() before it can see that folio->mapping * has gone stale (or that folio_test_swapcache has been cleared). */ smp_wmb(); @@ -3271,7 +3265,7 @@ static bool stable_node_dup_remove_range(struct ksm_stable_node *stable_node, if (stable_node->kpfn >= start_pfn && stable_node->kpfn < end_pfn) { /* - * Don't get_ksm_page, page has already gone: + * Don't ksm_get_folio, page has already gone: * which is why we keep kpfn instead of page* */ remove_node_from_stable_tree(stable_node); @@ -3359,7 +3353,7 @@ static int ksm_memory_callback(struct notifier_block *self, * Most of the work is done by page migration; but there might * be a few stable_nodes left over, still pointing to struct * pages which have been offlined: prune those from the tree, - * otherwise get_ksm_page() might later try to access a + * otherwise ksm_get_folio() might later try to access a * non-existent struct page. */ ksm_check_stable_tree(mn->start_pfn, diff --git a/mm/migrate.c b/mm/migrate.c index 73a052a382f1..9f0494fd902c 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -616,7 +616,7 @@ void folio_migrate_flags(struct folio *newfolio, struct folio *folio) folio_migrate_ksm(newfolio, folio); /* * Please do not reorder this without considering how mm/ksm.c's - * get_ksm_page() depends upon ksm_migrate_page() and PageSwapCache(). + * ksm_get_folio() depends upon ksm_migrate_page() and PageSwapCache(). */ if (folio_test_swapcache(folio)) folio_clear_swapcache(folio); From patchwork Tue Apr 9 09:28:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13622067 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C704CCD1284 for ; Tue, 9 Apr 2024 09:25:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 54FFA6B00A1; Tue, 9 Apr 2024 05:25:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4D9AF6B00A2; Tue, 9 Apr 2024 05:25:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 32B066B00A3; Tue, 9 Apr 2024 05:25:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 0D1816B00A1 for ; Tue, 9 Apr 2024 05:25:42 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id BE3271402E5 for ; Tue, 9 Apr 2024 09:25:41 +0000 (UTC) X-FDA: 81989460882.26.D7AEF74 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf13.hostedemail.com (Postfix) with ESMTP id 200582001F for ; Tue, 9 Apr 2024 09:25:39 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WssjhCxI; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf13.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712654740; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=768Emqci/FTw1ok7jT9pkAzjbRBjqS5i3RDOq4D3A08=; b=bXAifRb5Z1Bwac3hk/lFxDSF6z0dQn2JS8gUdJYOFUko3cqResVMHYRgDaVODBJKJNserr 6bWdzNXUAET8rFZkk7e7DXB1U91DlfDKXuBKbby2fxXgdoJSXl9zsoI/LjJm1zpwrRwKpl TVlyM7k1q0rWsAGFB+z/9GNKZYcJGa4= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=WssjhCxI; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf13.hostedemail.com: domain of alexs@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=alexs@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712654740; a=rsa-sha256; cv=none; b=hQXtkNMTtWUCl92IPCsoxZoJPL1/2WJ5a7QcahSbPi0IjPgqZSv86UGINJ8xB5gqxOENpu VYgpXare1e3MApRyZe+2Z7MgRaG0oHVnhrpBzleEPUNd8o5qOcWj3YL1YFhSk7bVKHqeTN CS/T9hrPpZauzMkrJL+ra61HBPAlLPg= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 5881461639; Tue, 9 Apr 2024 09:25:39 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D9AEBC43390; Tue, 9 Apr 2024 09:25:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712654739; bh=MGM8cnYOc2xUfzqiaHDKMMd1O2BaUHSekMXG68vTT9g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=WssjhCxIFV/yIm+DWR08ghSxRH0lwhxnaD1vS2T9rvquOXkj3qGcbCZmtRVCMugbx IpWPk8pGp412gnIoQ31Hh+Nj/nr0kRrJQXzBuprxRcKWtx46uGS3zwlAVGzzAnLZOa 52pTuciC59Q7qvfrK/iyC4ieJiOEUd4NAdFCWYHbq3mk8cYivLFFarlmlmJQ3ANn4L vJKWBxpUoAfA+1TuCAgUsamwLOWgCzpz3n7rfQU0g534XfH/f/OuaqzHk9rvsWc4Go 6wBLSxMwmhQEKLpYL8rYKa8MXVHlvfAV/2jf/n0c8537CHdVO8iTIdYeth+9BA3DNQ yLoovR8/YKs9Q== From: alexs@kernel.org To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: "Alex Shi (tencent)" , Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , David Hildenbrand Subject: [PATCH v4 9/9] mm/ksm: replace set_page_stable_node by folio_set_stable_node Date: Tue, 9 Apr 2024 17:28:24 +0800 Message-ID: <20240409092826.1733637-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240409092826.1733637-1-alexs@kernel.org> References: <20240409092826.1733637-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 200582001F X-Stat-Signature: pp17tquqnhwgnfpbzmcrjbd8sxyyz1jz X-HE-Tag: 1712654739-423739 X-HE-Meta: U2FsdGVkX1+jFsGgdr1Hn2rmC+05UxdbHzHpVdx1gmrq+kKivznvjTEMDVTm573kASyi+5owQaf8LIZb28R6kuzpD91j7a1AqmSl3409lB23ydXSAGLy96TKCiJxeR9cTWy2ZQBLTtgb0X1Acu0da79baAiEWHSjaOlFgvaCWMfrJdZloRBjD1VWfE+KG+/eJAin/r/yyhrFYZavR8VdNNHCayyGTh8O5gyIkaafCuIvtsjKzfpBKhhCPrLxfXy2A5/jYrVEnrXvaBMEEU3CoYU6ztE9RbSNzRnKGUAqrd8kQ7adu5PjcYycMkRLk1pf3xfYg1qS0KLiqFjVkjbPtMOObwz/bHDWDvdZMVNgECAqkq0jlccLP3P9KfO3DsSiQV5NxSYNE47XKnvsENzzOw2omch1ZeB9PMIqqQziykCO/UkSXoHAD5RlxWBUNwF5nfRiNFiIsDAulEv1pzbUImT0sS4aiPYeFuLNSf4hAVlpc49avPwjhTUgbxfgd3M64JiDSZjRf6LDJ7Izn/hW5mOd2C5s6ttimlFTu4dUYB0c3B7rP+zI+PQTkwuKNAynp8MuYXQAfLIvICj9Ss5QNayrDgifmu9vqn8sbCj8Q37yB4sfUQIpXZ3aSGqx1gtYRSD1v5udlZoZAnZMFtDu56oGETprqbm9W16piM+EyXzqf/ha+2BdXzcFNiaCM4bKZRpkagxaopWL5Fj5NDrTP7LYdnnd0ORc3JDsaf92esflnZWH1mDjIiAhTaj3CZMbdclr8Sz5DBrKndDvLUpfpp3CxpD8b0AugkbiXTu/AA9f6YNs89xoUQL8WyZdca9lKH/FiDIUyAVRyb1tg21EAZtglxluFhhMtvmSOI8+xxi8maee4RlreslFyyOkcWiododcFqYXcstWheedVyMY2oHxSwArS3bkk16AfZpNWEg+6egkagTslpWvwBxDNP+3x0hSkc1V9TNorh/AoYa lE51YAwl dPfQk2Nklh7J+D5VhYW8/W/65ojuDlaYQhYDh/Xofff3IVTRrooMuGgji4Xa+7CdUoOOtvIiRZPgklMNUhFSqiiTbCK85c6506IbmSPLfoGI8+sYq4kp75BpYsYcpN3K2WSgHzlgYEhTiWAlmwHsLrco5+0s3tQvtP3yTzOlwdDDx9YznC3FZWMgwVKJV3uZYSB47zRlsZOMpQES42fWVAzBYOM8ARE17tTBWKYIsY0COVOV5Is7Sn5osIgADj/B7/TQw/Dx2hZBaSWUGk++pzSNnupjFb+kvogyOH0A2/yLsAHfnbvUViQMvEW3aSWAOJi1VLW/VQu5sAbuo9sVlEafjcmibTvA2999bWeIzrDVXVB07yb+z87gK6eFOcxKvHq7V6qFuyrTqganpWhN2ICly8DL4lnKzyyzPjNMiQyZQqW14vRn5lQ9UAnixZFieeVOexWOGYj4SgRHruouQ38ZVI7nOZHfMIwZnAZu1TkxQk8dgpjiCVe3sP7VQBK4E/UJp X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Only single page could be reached where we set stable node after write protect, so use folio converted func to replace page's. And remove the unused func set_page_stable_node(). Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright Reviewed-by: David Hildenbrand --- mm/ksm.c | 12 +++--------- 1 file changed, 3 insertions(+), 9 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index a705ed16aa1b..0d703c3da9d8 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1094,17 +1094,11 @@ static inline struct ksm_stable_node *page_stable_node(struct page *page) return folio_stable_node(page_folio(page)); } -static inline void set_page_stable_node(struct page *page, - struct ksm_stable_node *stable_node) -{ - VM_BUG_ON_PAGE(PageAnon(page) && PageAnonExclusive(page), page); - page->mapping = (void *)((unsigned long)stable_node | PAGE_MAPPING_KSM); -} - static inline void folio_set_stable_node(struct folio *folio, struct ksm_stable_node *stable_node) { - set_page_stable_node(&folio->page, stable_node); + VM_WARN_ON_FOLIO(folio_test_anon(folio) && PageAnonExclusive(&folio->page), folio); + folio->mapping = (void *)((unsigned long)stable_node | PAGE_MAPPING_KSM); } #ifdef CONFIG_SYSFS @@ -1519,7 +1513,7 @@ static int try_to_merge_one_page(struct vm_area_struct *vma, * PageAnon+anon_vma to PageKsm+NULL stable_node: * stable_tree_insert() will update stable_node. */ - set_page_stable_node(page, NULL); + folio_set_stable_node(page_folio(page), NULL); mark_page_accessed(page); /* * Page reclaim just frees a clean page with no dirty