From patchwork Thu Apr 25 21:11:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13643758 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 53BEEC4345F for ; Thu, 25 Apr 2024 21:11:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D17EA6B009C; Thu, 25 Apr 2024 17:11:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CC8036B009D; Thu, 25 Apr 2024 17:11:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B677E6B009E; Thu, 25 Apr 2024 17:11:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 99D0A6B009C for ; Thu, 25 Apr 2024 17:11:47 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 3DC321C0814 for ; Thu, 25 Apr 2024 21:11:47 +0000 (UTC) X-FDA: 82049301054.19.64A637A Received: from wfhigh8-smtp.messagingengine.com (wfhigh8-smtp.messagingengine.com [64.147.123.159]) by imf27.hostedemail.com (Postfix) with ESMTP id 2B12540011 for ; Thu, 25 Apr 2024 21:11:44 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=sent.com header.s=fm3 header.b=Qz4mbo7F; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=E6SjiYRr; spf=pass (imf27.hostedemail.com: domain of zi.yan@sent.com designates 64.147.123.159 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714079505; a=rsa-sha256; cv=none; b=K8Wsw6sKOW6NXm5N2WveobXKdi2UF/0qKTqJ32i7+lUQGjJJL1ckvp5xQhXv9gECiUszz1 nUnvyOZXQEbF4gTNBof8wBNUBQam19XfcmkA/cYFvaxiCbNtUrYWXCTpwpg/L4Dqmi17Pq yng/QX5yJsBkjU/ClAf08gWZEVsIxTU= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=sent.com header.s=fm3 header.b=Qz4mbo7F; dkim=pass header.d=messagingengine.com header.s=fm3 header.b=E6SjiYRr; spf=pass (imf27.hostedemail.com: domain of zi.yan@sent.com designates 64.147.123.159 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714079505; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=iGwfXom4e7yCPiqRGg7li8EHLijwecEGUquGPFC+5WI=; b=knlDWQNw2a+eit5nhiw15YAuPDHhO128sP6yMXAJkHE0zA2Ge0I2O7NG3rIMTnh6g9s5Vr fKTgcQ78n7MB8jH6+Or1iA7fX1zMECeJAi5IDS89PFb7cFW0Xx4MiRMtdY+bljGnTwZUeb kcjx+EmnLnRYHXUXFJJnvUmikx4buys= Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfhigh.west.internal (Postfix) with ESMTP id 1C804180010A; Thu, 25 Apr 2024 17:11:41 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute2.internal (MEProxy); Thu, 25 Apr 2024 17:11:42 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:message-id:mime-version:reply-to:reply-to:subject :subject:to:to; s=fm3; t=1714079500; x=1714165900; bh=iGwfXom4e7 yCPiqRGg7li8EHLijwecEGUquGPFC+5WI=; b=Qz4mbo7FRm3b3kQ6kdzfBqIyIB ZUSMsl012WSYgD353Z6ow2o+nGTIfmDa/Kpnj1jrdVNIBZW7cOzc5rFk4OLQd9Kc sSdw10UyXSDdqm3qpJV3eloNEmOJC1Jzu7ic51yxZiTK2ccMisHf5O0Z8KFI+Wzz l4TwGnAyLXx77B6eTgABlblfbwCdFH6dbWhdbsl/Lg9CDnmJwqQXIBqD5VSneYuk opmuH6RxlTO+/snV4v4HvkdE7t0YutQMeY0gUg1vxCUFADgcwRnLbtFynKDP1OyL znKzrn8Ac1mCsD2ZGGu/BcVH7OR1jq9AO5LlJLJT3KSTmAGkpGU3UPdBQ87Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:message-id:mime-version:reply-to:reply-to:subject :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender :x-sasl-enc; s=fm3; t=1714079500; x=1714165900; bh=iGwfXom4e7yCP iqRGg7li8EHLijwecEGUquGPFC+5WI=; b=E6SjiYRrFB34PrHMNhJYrxrZXCMFj JeG1NC9YAjvWrJmrlxkMwUGpaBEjHEyYdOHJ37Mj2GKEmd0jgYBglVtZRSf9GRdy 6uO0QxGQLxuGA9T82gythsIbEtuLfwfGQM1ZUFt3EWfdvp8chADb5rLjTp+AK8bD Z+pBm1kHZ5z5v5dx119Ph4qKixzha5rZvN4ltGwJqEI6ivWKQgCPLJwCZpufGoN4 X6j+cxfviwe95dWJ+S2vni6W4rnZgJPiEioAppz5CSlM2++rtJh41L+b1yH5NuYJ Y8QnIWoz+IxfzX22nFmx5Ttf5fGN2zhvFOeh+IhY1NJWLCS8zngpz5xLg== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrudeljedgudehlecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkfforhgggfestdekredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepffejue eifeejudejudfgiedtfeeltdfgueffhfdulefgtdekteekfeejheekhedtnecuvehluhhs thgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepiihirdihrghnsehsvg hnthdrtghomh X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Thu, 25 Apr 2024 17:11:39 -0400 (EDT) From: Zi Yan To: Andrew Morton , linux-mm@kvack.org Cc: Zi Yan , "Matthew Wilcox (Oracle)" , Yang Shi , Ryan Roberts , Barry Song <21cnbao@gmail.com>, David Hildenbrand , Lance Yang , linux-kernel@vger.kernel.org Subject: [PATCH v4] mm/rmap: do not add fully unmapped large folio to deferred split list Date: Thu, 25 Apr 2024 17:11:36 -0400 Message-ID: <20240425211136.486184-1-zi.yan@sent.com> X-Mailer: git-send-email 2.43.0 Reply-To: Zi Yan MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 2B12540011 X-Stat-Signature: 4ud6z9bbo67ncsu6mxf5zwbykjak14wo X-Rspam-User: X-HE-Tag: 1714079504-329429 X-HE-Meta: U2FsdGVkX19Hny37B+979XPUHl0fxOUfyKUIu6cqyRKNlARPVWeyXPGj4G1QqbeH7Zyd6Ph14jZtY4dXPpWi6Wpv+oGLj49EweyCTj0S8g4gFIGtyh2kBek8IfeWbtuO7jdjQI/UZQq8348n4YvQpNKB1EGTUbkgjXwXzY0rqOmzh/MvniHB/GxiO/X6/1QHb6a1vMXP30ZNJ814jVPTMcen5JgF2vKxKEc6UNIyFPI36tRNJwDO5lB/copqSUj1L/3WSOO2qGF5SmDTAONcVNNboBaGM1wiU348dFnNv0qTwi/KrOBNi2qG1cjBsxDA1vyEfZ0n6aAkddhBe9BSqtW3m/W4qKllw0p8GZVaaCXUqmFIPp/+1JyV0ZRNs6AWus+gYz9+kyiuWWBU26LLH+vSKZqU5pUKokKX8oQWJfQnBz7OnWyhGUEjrIBMwXdNl1Wfk9OwFewskfhkQ+UKDRnHZ/iP64vLBpRKgrxdb1tfvbyjSAYXBXqA/0WpRjLdGrveReIliFhuxp/YZYLB2viyZJQ+D1d6zf4fBB4bmLMTggxsmDZDU/bjCBld7MqjCBUbPWXHImin5MWsI0Lzt4vmD7ZihTWG6JpmgstE69sRHobA93VOS5r4j434MHLL+sTpm/psefl+NGWzcXiuLtjATmaxz4NsULPWG3TJsSfvPgqijwnLzohkaE0kYwLCYa1flhlHQr8N9nNlt3Y+X72oWgfF7Q2+rENtbCyJfeNfsafG5BPXqoszVFAWHSfcKUqI2GK1EzK8Abr60TckKA9Dz8yAjdHSeWUS0BI2SXaM7PI89CXQr8JpMf8hbUU1+rgTpF2KCrJZv7tEYbRV21Jy8ZQpOPxYCjIna+9qOuTlnpxsLlrsPzm67on1Ec/GnHljFyhvnTZCagSTgDTfmgyqjYBNnMUcVSN1pYxcOXKhuY1rI2VZ5D5slY+eT1XqAzliuggUdzuG/g9nRjR F84KE4l2 9ZZvxRoveamnBxLx0hysep47jCr49RfYaULmLukrul0+fhDGuK/m1bsoD/5KnCp06NLVHBu44jvvwr7UjDDYYoBt3nouAeukVfNtgkCVnIYo8TpvvNhSMsCdTQuT60ceOqJvCFP1U+C5xiAkLHDcUU9Jj8Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Zi Yan In __folio_remove_rmap(), a large folio is added to deferred split list if any page in a folio loses its final mapping. But it is possible that the folio is fully unmapped and adding it to deferred split list is unnecessary. For PMD-mapped THPs, that was not really an issue, because removing the last PMD mapping in the absence of PTE mappings would not have added the folio to the deferred split queue. However, for PTE-mapped THPs, which are now more prominent due to mTHP, they are always added to the deferred split queue. One side effect is that the THP_DEFERRED_SPLIT_PAGE stat for a PTE-mapped folio can be unintentionally increased, making it look like there are many partially mapped folios -- although the whole folio is fully unmapped stepwise. Core-mm now tries batch-unmapping consecutive PTEs of PTE-mapped THPs where possible starting from commit b06dc281aa99 ("mm/rmap: introduce folio_remove_rmap_[pte|ptes|pmd]()"). When it happens, a whole PTE-mapped folio is unmapped in one go and can avoid being added to deferred split list, reducing the THP_DEFERRED_SPLIT_PAGE noise. But there will still be noise when we cannot batch-unmap a complete PTE-mapped folio in one go -- or where this type of batching is not implemented yet, e.g., migration. To avoid the unnecessary addition, folio->_nr_pages_mapped is checked to tell if the whole folio is unmapped. If the folio is already on deferred split list, it will be skipped, too. Note: commit 98046944a159 ("mm: huge_memory: add the missing folio_test_pmd_mappable() for THP split statistics") tried to exclude mTHP deferred split stats from THP_DEFERRED_SPLIT_PAGE, but it does not fix the above issue. A fully unmapped PTE-mapped order-9 THP was still added to deferred split list and counted as THP_DEFERRED_SPLIT_PAGE, since nr is 512 (non zero), level is RMAP_LEVEL_PTE, and inside deferred_split_folio() the order-9 folio is folio_test_pmd_mappable(). Signed-off-by: Zi Yan Reviewed-by: Yang Shi Reviewed-by: David Hildenbrand --- mm/rmap.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) base-commit: 66313c66dd90e8711a8b63fc047ddfc69c53636a diff --git a/mm/rmap.c b/mm/rmap.c index a7913a454028..220ad8a83589 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1553,9 +1553,11 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, * page of the folio is unmapped and at least one page * is still mapped. */ - if (folio_test_large(folio) && folio_test_anon(folio)) - if (level == RMAP_LEVEL_PTE || nr < nr_pmdmapped) - deferred_split_folio(folio); + if (folio_test_large(folio) && folio_test_anon(folio) && + list_empty(&folio->_deferred_list) && + ((level == RMAP_LEVEL_PTE && atomic_read(mapped)) || + (level == RMAP_LEVEL_PMD && nr < nr_pmdmapped))) + deferred_split_folio(folio); } /*