From patchwork Wed Mar 20 07:40:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: alexs@kernel.org X-Patchwork-Id: 13597423 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B1FCC54E67 for ; Wed, 20 Mar 2024 07:38:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 102358D0002; Wed, 20 Mar 2024 03:38:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 0B1396B00A5; Wed, 20 Mar 2024 03:38:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E96E38D0002; Wed, 20 Mar 2024 03:38:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id DA6AD6B00A4 for ; Wed, 20 Mar 2024 03:38:13 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id AE4741C0EAF for ; Wed, 20 Mar 2024 07:38:13 +0000 (UTC) X-FDA: 81916614066.25.35603F8 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf17.hostedemail.com (Postfix) with ESMTP id 7E3AD4000A for ; Wed, 20 Mar 2024 07:38:11 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QnyMHcy7; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710920292; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=EfnUB+zh6Jkuo8KxC5rPnusQaj/Ro7vi77DeDl1FMzg=; b=drD3w/W4muRlzd3+yeHeySN4dm1oZXp7CRX/jxfE+xHupc4eavmi+/XFGtzQTiwp82N/XA +7KORpfoLMGIh7/hFLKuI9/V1OraswWHZvz3tnrQYW2WgqwGNWDbIVb3xIcQID0MmzKriS 6uOKotWe0K7R8GiAPJ/lAs3lN1rK8FE= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=QnyMHcy7; spf=pass (imf17.hostedemail.com: domain of alexs@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=alexs@kernel.org; dmarc=pass (policy=none) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710920292; a=rsa-sha256; cv=none; b=CPujvBNFXX1tEryIvRkSWSD0d6EB6jtQ33OaFXXgDkMzwv8T1RwxR0FZB16EHBnqXYuMoU BVRJjQjPs722kWjAZGpE4K1afD8atbZexd/OGML2Gi8indYvqVZ+q/xnH8LsRlyARZCI1B jilLy5JkmvtgflOjiQo9GKXI/7XVf1g= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id D79C2CE109F; Wed, 20 Mar 2024 07:38:05 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id F12D5C43390; Wed, 20 Mar 2024 07:38:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1710920285; bh=tjVPu2LfV6/oJL+/cGFCPeE7wstp4Ud62/5nB+tu7Fo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QnyMHcy70fzp8mjnSBi4o1+7TO69JWuiRoHjNyhFVoYTtNUcOygYYs1aYzSwHZ4tP B1BEQoxvGpCI5WgyIKpDPphqywUn0C0GwKZ9osVoCfI7FwSszybl7JM+X7kspzNj1U ZaUyDOXUqJMrYPlAP3tjgfC0glaUofbiUSDuv3ZHGJJXB3urIxTdY8dqSOJpLF7Q4J +0xpRLKV7XWf7OvDi6wMx13J5ewLV5vUggPB+1o1apGUrBvZIVUI7PM1LA3+YdCGy4 16PM0s7aJeQoqB4eV7jgHGwGxedDMFcl/prNrGkefrYvjHT9OTNC9gi84ELn/jrimA l05LCkNlt1caA== From: alexs@kernel.org To: Izik Eidus , Matthew Wilcox , Andrea Arcangeli , Hugh Dickins , Chris Wright , kasong@tencent.com, Andrew Morton , linux-mm@kvack.org (open list:MEMORY MANAGEMENT), linux-kernel@vger.kernel.org (open list) Cc: linux-kernel@vger.kernel.org, "Alex Shi (tencent)" Subject: [PATCH 09/11] mm/ksm: Convert stable_tree_search to use folio Date: Wed, 20 Mar 2024 15:40:45 +0800 Message-ID: <20240320074049.4130552-10-alexs@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240320074049.4130552-1-alexs@kernel.org> References: <20240320074049.4130552-1-alexs@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 7E3AD4000A X-Rspam-User: X-Stat-Signature: e3qb1gzi9o6fedn9x3wkapz5a8h9et3h X-Rspamd-Server: rspam01 X-HE-Tag: 1710920291-288500 X-HE-Meta: U2FsdGVkX19H6D9GdabaxWqzn1UjJ37paX1Z/yrH+uU5AtJYv0/4mlPXypeOykX6HMifPZJgr4Edq62+1/V72+pHxG4Soothh9+SRwboDNvcTIMbwRI6GLXIS7R7NfxLzt4cA87KUzHPXJjpwzt4XiWvgh6Yq1J6ZrgNhz7/bZorCxS8pFxM/uOqHK7sTdeX7WjmK211j0JhA8Q1S36TCiN5tZxGcxm55nDKW3do1di4CnifP8BIq3L57XdnBQrqpv36Wjv8pbOEEVUJI/GxTW3/9uiSq7yoTAxvo6C23FYRvVIlWetkY59z8pIW2VInNGf0C2hirDEjOA0wi3xucxzcTFrUMHG/vrIZiJzPKWxcXDFUOZhSJCIZkbFmILgKrmGrxkn7M7a6VZXPmRIcr76xepD4uvOaKNRGfvrGjEyyDY7AZn5yV70cXU+ub1uOruWK029UpCV/SK7U4AXUTYMpZ/zcS8nbqw0skZnB0z4Qg7B5Nsum3u3iO2UxTU2P4eTWOIBQbf+bauqE+D+k+L4Pl5dXSMTU5v6WPRJ4hft6arzqMhCMdRh1EAObYs/HbdfegOVCOIUZx07EpTdZEa5P2jaYHnsQJHlKo0uIVCV6g9qH87sdVDM2828CSC09A0ZVZrnszBjO0kuPqs4/A9AP5a88m3Rk3rk28gIJ2t4BnMGwMrv6fwI00e+yj4ImLJKUCFkSQ7fH/pmVxlQYg4+1WehpuWKtCegFgqyaQqap8CA6YeYicNosnZy6RPOhsGkvjou8ZgqhOjU2ABjAeGCb8pxfLGq5F7htXMnDswEe249lWRRSaaNF1nezRSqqPAzpU1Jq8yj1/mJ3ty6khbpMKX4f2lEFp/z/8q5INvijDPODABcdRWDTpJjquMZZLelSDM2ue1eicIgfA467ePtbbvu58LNeGoMUwlv8tpsv/xywcQ7wEM/yJjbMas1octs8+mQ76KXH0soo379 JQxw5jc2 V+WC8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Alex Shi (tencent)" Although, the func may pass a tail page to check its contents, but only single page exist in KSM stable tree, so we still can use folio in stable_tree_search() to save a few compound_head calls. Signed-off-by: Alex Shi (tencent) Cc: Izik Eidus Cc: Matthew Wilcox Cc: Andrea Arcangeli Cc: Hugh Dickins Cc: Chris Wright --- mm/ksm.c | 60 +++++++++++++++++++++++++++++--------------------------- 1 file changed, 31 insertions(+), 29 deletions(-) diff --git a/mm/ksm.c b/mm/ksm.c index 71d1a52f344d..75401b3bae5c 100644 --- a/mm/ksm.c +++ b/mm/ksm.c @@ -1812,7 +1812,7 @@ static __always_inline void *chain(struct ksm_stable_node **s_n_d, * This function returns the stable tree node of identical content if found, * NULL otherwise. */ -static struct page *stable_tree_search(struct page *page) +static void *stable_tree_search(struct page *page) { int nid; struct rb_root *root; @@ -1820,28 +1820,30 @@ static struct page *stable_tree_search(struct page *page) struct rb_node *parent; struct ksm_stable_node *stable_node, *stable_node_dup, *stable_node_any; struct ksm_stable_node *page_node; + struct folio *folio; - page_node = page_stable_node(page); + folio = (struct folio *)page; + page_node = folio_stable_node(folio); if (page_node && page_node->head != &migrate_nodes) { /* ksm page forked */ - get_page(page); - return page; + folio_get(folio); + return folio; } - nid = get_kpfn_nid(page_to_pfn(page)); + nid = get_kpfn_nid(folio_pfn(folio)); root = root_stable_tree + nid; again: new = &root->rb_node; parent = NULL; while (*new) { - struct page *tree_page; + struct folio *tree_folio; int ret; cond_resched(); stable_node = rb_entry(*new, struct ksm_stable_node, node); stable_node_any = NULL; - tree_page = chain_prune(&stable_node_dup, &stable_node, root); + tree_folio = chain_prune(&stable_node_dup, &stable_node, root); /* * NOTE: stable_node may have been freed by * chain_prune() if the returned stable_node_dup is @@ -1875,11 +1877,11 @@ static struct page *stable_tree_search(struct page *page) * write protected at all times. Any will work * fine to continue the walk. */ - tree_page = get_ksm_page(stable_node_any, - GET_KSM_PAGE_NOLOCK); + tree_folio = get_ksm_page(stable_node_any, + GET_KSM_PAGE_NOLOCK); } VM_BUG_ON(!stable_node_dup ^ !!stable_node_any); - if (!tree_page) { + if (!tree_folio) { /* * If we walked over a stale stable_node, * get_ksm_page() will call rb_erase() and it @@ -1892,8 +1894,8 @@ static struct page *stable_tree_search(struct page *page) goto again; } - ret = memcmp_pages(page, tree_page); - put_page(tree_page); + ret = memcmp_pages(&folio->page, &tree_folio->page); + folio_put(tree_folio); parent = *new; if (ret < 0) @@ -1936,26 +1938,26 @@ static struct page *stable_tree_search(struct page *page) * It would be more elegant to return stable_node * than kpage, but that involves more changes. */ - tree_page = get_ksm_page(stable_node_dup, - GET_KSM_PAGE_TRYLOCK); + tree_folio = get_ksm_page(stable_node_dup, + GET_KSM_PAGE_TRYLOCK); - if (PTR_ERR(tree_page) == -EBUSY) + if (PTR_ERR(tree_folio) == -EBUSY) return ERR_PTR(-EBUSY); - if (unlikely(!tree_page)) + if (unlikely(!tree_folio)) /* * The tree may have been rebalanced, * so re-evaluate parent and new. */ goto again; - unlock_page(tree_page); + folio_unlock(tree_folio); if (get_kpfn_nid(stable_node_dup->kpfn) != NUMA(stable_node_dup->nid)) { - put_page(tree_page); + folio_put(tree_folio); goto replace; } - return tree_page; + return tree_folio; } } @@ -1968,8 +1970,8 @@ static struct page *stable_tree_search(struct page *page) rb_insert_color(&page_node->node, root); out: if (is_page_sharing_candidate(page_node)) { - get_page(page); - return page; + folio_get(folio); + return folio; } else return NULL; @@ -1994,12 +1996,12 @@ static struct page *stable_tree_search(struct page *page) &page_node->node, root); if (is_page_sharing_candidate(page_node)) - get_page(page); + folio_get(folio); else - page = NULL; + folio = NULL; } else { rb_erase(&stable_node_dup->node, root); - page = NULL; + folio = NULL; } } else { VM_BUG_ON(!is_stable_node_chain(stable_node)); @@ -2010,16 +2012,16 @@ static struct page *stable_tree_search(struct page *page) DO_NUMA(page_node->nid = nid); stable_node_chain_add_dup(page_node, stable_node); if (is_page_sharing_candidate(page_node)) - get_page(page); + folio_get(folio); else - page = NULL; + folio = NULL; } else { - page = NULL; + folio = NULL; } } stable_node_dup->head = &migrate_nodes; list_add(&stable_node_dup->list, stable_node_dup->head); - return page; + return folio; chain_append: /* stable_node_dup could be null if it reached the limit */ @@ -2109,7 +2111,7 @@ static struct ksm_stable_node *stable_tree_insert(struct folio *kfolio) * fine to continue the walk. */ tree_folio = get_ksm_page(stable_node_any, - GET_KSM_PAGE_NOLOCK); + GET_KSM_PAGE_NOLOCK); } VM_BUG_ON(!stable_node_dup ^ !!stable_node_any); if (!tree_folio) {