From patchwork Thu May 2 13:28:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13651726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 64458C4345F for ; Thu, 2 May 2024 13:29:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ED4416B009A; Thu, 2 May 2024 09:29:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id E84C16B009B; Thu, 2 May 2024 09:29:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D26176B009C; Thu, 2 May 2024 09:29:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B4B776B009A for ; Thu, 2 May 2024 09:29:34 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 6BAC580855 for ; Thu, 2 May 2024 13:29:34 +0000 (UTC) X-FDA: 82073537868.21.2A4B16D Received: from wfout2-smtp.messagingengine.com (wfout2-smtp.messagingengine.com [64.147.123.145]) by imf27.hostedemail.com (Postfix) with ESMTP id 1F1F240013 for ; Thu, 2 May 2024 13:29:30 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=sent.com header.s=fm3 header.b=TQy9HzT3; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="W0/TJGy/"; spf=pass (imf27.hostedemail.com: domain of zi.yan@sent.com designates 64.147.123.145 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714656571; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=TQdtp9gQ6BLDd977+sCpOX4jKxONWblKEhEd4WgTehk=; b=X5aZ1EOAUnu1Qi3TRmBT9bOhxGs++/AkCOxQeoHGgQ6C7DO9yEK5QOXLYU8If0rg0l9ICx QlGY8lu0rhX0+U7R7cywokzVEfW+CEzFxK255Csxr8p/28990rwwynvpc5AFOJnvOBTxU7 AXzPgHEfNPC2uL/u3YW4fIyGv5m/2yE= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=sent.com header.s=fm3 header.b=TQy9HzT3; dkim=pass header.d=messagingengine.com header.s=fm3 header.b="W0/TJGy/"; spf=pass (imf27.hostedemail.com: domain of zi.yan@sent.com designates 64.147.123.145 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714656571; a=rsa-sha256; cv=none; b=7abhR0SGaY9I3zqnk1u/drV86y2O0BcfE4fPzUfMgqj3YrRZHb09MV7pjiBpoYGAksh8UY nMuOUDVMPxT57ttvVEewgx4e8oC1lgE1inxu5soQ/WDG7+3Gpw8uoHyN/0VQPPNp/9qCge sa41fc+QYZNt+Xh7JEiFx9pdloNqeRI= Received: from compute6.internal (compute6.nyi.internal [10.202.2.47]) by mailfout.west.internal (Postfix) with ESMTP id D627C1C0013B; Thu, 2 May 2024 09:29:25 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute6.internal (MEProxy); Thu, 02 May 2024 09:29:26 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:message-id:mime-version:reply-to:reply-to:subject :subject:to:to; s=fm3; t=1714656565; x=1714742965; bh=TQdtp9gQ6B LDd977+sCpOX4jKxONWblKEhEd4WgTehk=; b=TQy9HzT3sNGOc9O+5R3BXvhfWm mp8PPjtg43ZF4gJ/rP1agU+usQmXl2wHiuW1J7X14hARufAR8WPzoR5HlN/5G8uo vokgdxFjRXJZOC4xmHzV09EB7WKSV0yhnikYp59D08hWl8rKur9TsKE9aKiUgKPj 3BoWFXtagEputhToRTM/1FjlT2Lp6ysyFFuzJw7WjxU0qQyOvMABPTnDHtTUK0e9 kFuxyB0oau+fPYgRAkVKgNM69TJITc+OwVIcfHx6R3Jj9Zn2m65K8b1w3Pes7K/7 RoxZJERX2SXi1s9ur4rxqSxhKq9X2i1LKxIOkzhbznttJskF6CikwvFFTMMQ== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:message-id:mime-version:reply-to:reply-to:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1714656565; x=1714742965; bh=TQdtp9gQ6BLDd 977+sCpOX4jKxONWblKEhEd4WgTehk=; b=W0/TJGy/zhCXq4uG8tdXhnZ9ObBAc pWpimSkr58lUKNgVCj6Kq6vmEm/Qj0+O2v/xSR+otlLGhQzkwlFiVRJR6JNTI/3t QONuGLHOkWt09if4FXHmdBf9tBfnm9uvfvpjIQivEZnPpjw0G6sXTIdWOgMo7fvs ZWJe/eCc5G0ZeS4rgpSK3VUBCX0qs30DMMheyzeaD1DZpTUlnvxkuuuU03kqYv9o e9575nfcFYzBbzjKPEXfmV1pAAbNL55ZlsnaZiP86NFQHdLjeNgCUithbhffZ13j oZhQy0I5tiszOesIA0Wk/0xOq613CPqvkQ64IsopGHgxw2XqPmuJvM6Qw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrvddukedgieehucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofhrggfgsedtkeertdertddtnecuhfhrohhmpegkihcujggr nhcuoeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeffjeeuie efjedujedugfeitdefledtgfeufffhudelgfdtkeetkeefjeehkeehtdenucevlhhushht vghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnh htrdgtohhm X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 2 May 2024 09:29:24 -0400 (EDT) From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: Zi Yan , "Matthew Wilcox (Oracle)" , Yang Shi , Ryan Roberts , David Hildenbrand , Lance Yang , Alexander Gordeev , linux-kernel@vger.kernel.org, Barry Song Subject: [PATCH v6] mm/rmap: do not add fully unmapped large folio to deferred split list Date: Thu, 2 May 2024 09:28:51 -0400 Message-ID: <20240502132852.862138-1-zi.yan@sent.com> X-Mailer: git-send-email 2.43.0 Reply-To: Zi Yan MIME-Version: 1.0 X-Rspamd-Queue-Id: 1F1F240013 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: 93xi1fbnkings7gbbi8m31w6m8yiju1j X-HE-Tag: 1714656570-595775 X-HE-Meta: U2FsdGVkX1+TeKfi2xN+ef3kazh7+7jw/XnheSfokCytJQLb7CsMCl0HAfmr1R700vg4EizcE0tDBgtO9tXaksjEPd+WNs7LCV8JGHSL6Si38cUmR+7Lgejln1PhplqnaunC3m71Vijv0DxufMxi/+yR4rO4nViYCoO9nQr1XbJle8vpRNHTJvhY0mIy2baMDGtPytSjPJMZebiimL/wi//sXnGtAxMNb99JtBq70xNxVJbDUZZlctiePnkxUYZkHpycVQ7q6S462N8fYv5pLyS3693FIQIAmVIGcSZulWiqakeuOKjzIHLPVivAH8IX+btpUusNYThdRCG5vcebQnCGvZsLtaaf74xXFFNy0ekcvQpPIPe3VUicMf7ocNlQ/s1PsV1o1l1WtC0HQfrLTdlZHAfaq0wIT9twVjwu7fgbO3NJrsjz5+UjRAaXAuh5pP4bx3Z4fwcQv5h+MMbiDbZ0ghnCLezxBSZ/ipVzpYo4kFVkJOYt2WTgdFqHSW9YGwFLQ428RWNWjs6OvxEMM2xE384DJxLv212GUWjWjgIHrT4+8Y3VDRQZ8gL1lGbxY10ZYEre13CPMnSp6TtRArP1zRABlM5yCQfgYGGPR7xJ8wA7L0Vfn7/J6vv98trFdvErnn40iEN8nMSA6JkFiCpvJAefN6bk2GYSvenXDDirljc6z4K5AtYMDsax+tlm/qTqpKqow6mC6fVIuNjA34FTBGeC8fAkmd4k1cARUY+aGOk+LlMKWi9fiFTiViOB+DlwKWSRudWrjc+gY2PmMEa7gYwbiuENRS0FT6Y5OAKuCNTS0YyJ9tPOHuPJD9cnSZoKAGfxbs2uUKKZhq22L+VDRc8ZmF2w5Cx1r0VzWXBZM1cQEnl9HpcqVbLmUx3ObhhLydJ/CIREGJXXve2Pa9QWyPBGVV5MxeiPs9z/IWStVA7bA75nsboCNltXfwa2ktTumoVwDFXydmlyTAw pjz/ATTH IrjhtQFg1BwLwdMHYKw/3pVH8S3KONN/AC38ATMB96mtjDm/veKLqywwwi5xZqutvERNik6xLpmoPvkurfE1DpSC5FUF2fV5Iz+KznBNteBCPEABpiTF6cwxaKzIYmpprUTXz9hGaugYt2NAJco/PF8u8HkQrAtVGTKokn48cxISRpgVIEIlwa8GLFziDGaL7akSKn5l4RGB5Xn6E/QlTPWSuthxvItMwF/LsYvTy949f2a2kvvzIUXdRb7myYSqFXIGhzt1UlyIxUJw+sZ7VJeAiPm6nE1Nh80z0WewmqZYY/q26RVFB0VeAFfEswdZ6RLFaBHLzmwSZHUAWx95lNHAorWpAZSsPPaBLJymk/eNtZ9S8XohpDEOyXCPUJ1sSUz2ZIjzh4fFnXvOE4WYICHSnmE0OXrBz7fU5shNo1RGl4M0JWzUBbKVeIvq3JwCc8qRVvWlYpmxvaMYCgphIzWMIj7uJWX/5gVlMzeSiTI436qRUmUfxLfoooapGYiLsTY8+pmskGArr3BsADJ9vzN0pcGHB0I9tp3bSWH/3monxURc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Zi Yan In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary. For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue. However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise. Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration. To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too. Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable(). Signed-off-by: Zi Yan Suggested-by: David Hildenbrand Reviewed-by: Yang Shi Reviewed-by: David Hildenbrand Reviewed-by: Barry Song Reviewed-by: Lance Yang --- mm/rmap.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 7cdb53380199..d5af687e2ad8 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1494,6 +1494,7 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, { atomic_t *mapped = &folio->_nr_pages_mapped; int last, nr = 0, nr_pmdmapped = 0; + bool partially_mapped = false; enum node_stat_item idx; __folio_rmap_sanity_checks(folio, page, nr_pages, level); @@ -1514,6 +1515,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, nr++; } } while (page++, --nr_pages > 0); + + partially_mapped = nr && atomic_read(mapped); break; case RMAP_LEVEL_PMD: atomic_dec(&folio->_large_mapcount); @@ -1531,6 +1534,8 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, nr = 0; } } + + partially_mapped = nr < nr_pmdmapped; break; } @@ -1551,10 +1556,12 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, * Queue anon large folio for deferred split if at least one * page of the folio is unmapped and at least one page * is still mapped. + * + * Check partially_mapped first to ensure it is a large folio. */ - if (folio_test_large(folio) && folio_test_anon(folio)) - if (level == RMAP_LEVEL_PTE || nr < nr_pmdmapped) - deferred_split_folio(folio); + if (folio_test_anon(folio) && partially_mapped && + list_empty(&folio->_deferred_list)) + deferred_split_folio(folio); } /*