From patchwork Fri Apr 26 19:02:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13645265 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBBFEC4345F for ; Fri, 26 Apr 2024 19:03:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5793D6B009A; Fri, 26 Apr 2024 15:03:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5286D6B009B; Fri, 26 Apr 2024 15:03:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3CA596B009C; Fri, 26 Apr 2024 15:03:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 1D5026B009A for ; Fri, 26 Apr 2024 15:03:04 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C8CFBA22AA for ; Fri, 26 Apr 2024 19:03:03 +0000 (UTC) X-FDA: 82052605446.25.395AB17 Received: from wfout3-smtp.messagingengine.com (wfout3-smtp.messagingengine.com [64.147.123.146]) by imf06.hostedemail.com (Postfix) with ESMTP id 9672318002E for ; Fri, 26 Apr 2024 19:03:00 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=sent.com header.s=fm3 header.b=NLkCziXg; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=PwaakadR; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf06.hostedemail.com: domain of zi.yan@sent.com designates 64.147.123.146 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714158181; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=u3Bdoo77C06eP9MyEhIJVQSYXIKHj1xxukvATG+3jJo=; b=CPwCEGmCcfoVQxV0NBhSFJmVWFpkb/FGoKWOEkf1+F29/+hs24lqgvzMyTe9xAbHbZGuf+ opTk3nHFDHn46TRsgC7GHcBVGcgdqg6GbiQASJ0wNPB4/mdPodNwOkPO98D0e5nOW8a8iM p1k/jT9u3VL2Ej0YB3zV1FQGtj2Y7Ik= ARC-Authentication-Results: i=1; imf06.hostedemail.com; dkim=pass header.d=sent.com header.s=fm3 header.b=NLkCziXg; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=PwaakadR; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf06.hostedemail.com: domain of zi.yan@sent.com designates 64.147.123.146 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714158181; a=rsa-sha256; cv=none; b=qdLdEQIVxqGZFPVWHa/zBThKjJ9yL8QVqwlU1EzC7HyDWi4oFlbEzG+sOTtxo6fLsy/xOv SEZw6YU5BC7I9i3J26+Jvrv5Y5aMM/8lVPy7vT6bUwQ5o2H/e24dd7zkykIUQKX/A/VEPE BiHANr/R+P/ozJswBNPd1juN8UoUVmU= Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfout.west.internal (Postfix) with ESMTP id A0BF31C0015C; Fri, 26 Apr 2024 15:02:57 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute4.internal (MEProxy); Fri, 26 Apr 2024 15:02:58 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:message-id:mime-version:reply-to:reply-to:subject :subject:to:to; s=fm3; t=1714158177; x=1714244577; bh=u3Bdoo77C0 6eP9MyEhIJVQSYXIKHj1xxukvATG+3jJo=; b=NLkCziXgYCGcc0B84MgzWd7zIW PpOJMhuiqf9N0PAKh1HHU42kUEZHVKTCMIFXtUZkctPYN0fT0rTE2pclov9JZuBT /8Rk6eYDBgVH4/1aeIibBhXJrzuWpVNOTxPqshaPyXOoDuPfBCdNOL9fCsvqklWg 0K9cg6sy1WiN7jLLqG33rpRad+PABBZFRS6z31aUf62B+hQLUhKdlKkWWDEUJdk3 WlesY5D3T6mhynXMd2g+eM8mrAv7KkCthWnKeVn6SvJ3Z+M7Dctbd0iyugXCN2Zo qBtWbmyTCvbSMIgaaqM/REMwXFi3Nf1VvJV9/54kVNjt/1tzM6gjjjHxHL2A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:message-id:mime-version:reply-to:reply-to:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1714158177; x=1714244577; bh=u3Bdoo77C06eP 9MyEhIJVQSYXIKHj1xxukvATG+3jJo=; b=PwaakadRBqQi1SnA5G0sIKeUcPRcZ V/2UxUgFrxpU1U9Pd5J6mMOIfGyB4Ul/RVtdP6AAoQvjB2bO5g7kX0GkzDypB+0g fEUIiQ65lEGni40J5MVc6+X4YrEXKLZ/4/PZtz7IGJ30gozXILF5E/kyZL2g+w2c Gs2Mr9resHVQuE3Cz8TzahqeiRGIlM8AOtQFbYM0R8ZFibHUrc5AMGKeZ5rOS0Vm bmKOn3HRQYAo73kfURMOCfDcD3hf7g+M6DvwI6h/NyCH4PNnNEXzGpcmt4mOBIc+ Kf0b1AbNT/8EHvlEK1g6vtNWmMJc8qrwTL0+uJDCv2da/wEx8Vd7dxmYA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudelledgudefvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkfforhgggfestdekredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepffejue eifeejudejudfgiedtfeeltdfgueffhfdulefgtdekteekfeejheekhedtnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 26 Apr 2024 15:02:56 -0400 (EDT) From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: Zi Yan , "Matthew Wilcox (Oracle)" , Yang Shi , Ryan Roberts , Barry Song <21cnbao@gmail.com>, David Hildenbrand , Lance Yang , linux-kernel@vger.kernel.org Subject: [PATCH v5] mm/rmap: do not add fully unmapped large folio to deferred split list Date: Fri, 26 Apr 2024 15:02:53 -0400 Message-ID: <20240426190253.541419-1-zi.yan@sent.com> X-Mailer: git-send-email 2.43.0 Reply-To: Zi Yan MIME-Version: 1.0 X-Rspamd-Server: rspam01 X-Stat-Signature: r5cqzfjordx1trbokdzb6w8twabb9681 X-Rspam-User: X-Rspamd-Queue-Id: 9672318002E X-HE-Tag: 1714158180-191042 X-HE-Meta: U2FsdGVkX19owyiU4Bn1tOaRub+hjzcJWsbRyiH7forUNXrVAslCzH1Xpa+EkrCNj5du/OCOjrGjKqsckA8mE/apcMa0UoaOMVujCoxLhlx8bCcF55Dz5H7daeRHU2l+UYiQ46FyKZJqh0qtQr946uAgGx/aYOuyegOYtnnjFfYrj5XfFzktBlB9822OJIDcRGTKAMC2/Olrup72wUhuDd9CBZOxPHEq+4Ixu0BoeqguMDrZqVdFQgO67blJotuuCUOx9j+k5qF4hGJEPhL9EMmabH/9uilB3N5AjrS1E1BTbve3eCVw7/S8VUNAYJsPHP0jckRSY7mAppffCzzZRMwNbgOi1fg3fa46XnrMETzFY0W26TG7a48JK/LFtzKrB5biDX9448JvlU8VfIy60SodtdDnOaiNv5mWUtJOWCQEGxuD0y4QhBUtv6TiePRqyx5uhhSUH6gqJcDf9gcpXYSjOGbS/WHwuPSVbdl31nGNW+CqcYsB/1EL9F5E/EK1hwV1s6/rUo6TKeuUrCiIMCMqnoGsucjDGVYs58jZETNAkyhab9Ze29WzC/VvMxdX6rLd20NX4sSg9OqZ0zVQanm8caWpAdPPy+uXxGJUCt2brphqcZPiTzjdEfIXFVyOqe6sKiWmACmY93ni07dF5A8YYVNTiYyF2+PT3AHgmh16qG51ZyZVk/wkcCLMGFU+CzNIYP1PK5PzY+bacox1ALagoNXTmTmOH3RdshFMlAeOxR9F5+dTS+itj7/Pn6e8ZdqtoKNtqW+f2yHqfKl3N9ZgZgt45AJIsZuxAoNv/mFDedvnBeG82woqhla/o81QmWxD0ohoHcAd1PjFq8jA55uRybaln+i9qxPUA+LEZsnNfawnqN3cBeMduDP3F+r4U5DC03sAwO8qSZSInOhGe4ok4KnL4mHc1TGwVmno2qn6AhFwCMVtCcffUMFCPuWw5tHV466T3UsHt2663eZ tks8Zhou vGVisUq59SjmqiBgph9C1P6RczNl8cxsas7ErgBmiYr0QIoc9UgJ25KL8YdR7OhLOhXQfkDZTX7YKy/oulhqOvjbtHD+cmJ5gLMvacF/b0jHJnQQh/PhHdUepDxhS7YETuGpbV0y7LsrkxYe4QfIr7T5pfiligH7tY6szBg7D3z5Y/08EO6V0Y8jefJCNXLMG76EXzVjiw8SgR0rDBJ9CUdt4jvITbNWuInRnkY8kK6DSNp9jLiG+Q+HdTJ6LuHWvFbhRDpjraszn4uj8Z36i5WSRFBGeiSn9PTQPft9tJ7ASpyygDhELaesZK1zHERL+Mqhyi6Palcml4MxsnZunI8gihrTY6o+wSN2EMQwl/2u+Zxf2uD8TUKas09O789dto/vFC96NuDSoGITaA0bvbt1eRHMMEHes7BQT0G9fCpHzZ1IsxlHQYfDkDli40+z5wIn5pHakbaWqwNVbo5Dfhr2rhA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Zi Yan In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary. For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue. However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise. Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration. To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too. Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable(). Suggested-by: David Hildenbrand Signed-off-by: Zi Yan Reviewed-by: David Hildenbrand Reviewed-by: Yang Shi Reviewed-by: Lance Yang --- mm/rmap.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) base-commit: 3dba658670af22074cc6f26dc92efe0013ac3359 diff --git a/mm/rmap.c b/mm/rmap.c index 2608c40dffad..a9bd64ebdd9a 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1495,6 +1495,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, { atomic_t *mapped = &folio->_nr_pages_mapped; int last, nr = 0, nr_pmdmapped = 0; + bool partially_mapped = false; enum node_stat_item idx; __folio_rmap_sanity_checks(folio, page, nr_pages, level); @@ -1515,6 +1516,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, nr++; } } while (page++, --nr_pages > 0); + + partially_mapped = !!nr && !!atomic_read(mapped); break; case RMAP_LEVEL_PMD: atomic_dec(&folio->_large_mapcount); @@ -1532,6 +1535,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, nr = 0; } } + + partially_mapped = nr < nr_pmdmapped; break; } @@ -1553,9 +1558,10 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, * page of the folio is unmapped and at least one page * is still mapped. */ - if (folio_test_large(folio) && folio_test_anon(folio)) - if (level == RMAP_LEVEL_PTE || nr < nr_pmdmapped) - deferred_split_folio(folio); + if (folio_test_anon(folio) && + list_empty(&folio->_deferred_list) && + partially_mapped) + deferred_split_folio(folio); } /*