From patchwork Fri Mar 22 19:33:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13600414 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29B71C54E71 for ; Fri, 22 Mar 2024 19:33:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 791896B0082; Fri, 22 Mar 2024 15:33:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 740E36B0083; Fri, 22 Mar 2024 15:33:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5E1486B0087; Fri, 22 Mar 2024 15:33:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 4B57B6B0082 for ; Fri, 22 Mar 2024 15:33:20 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id C3D4E404E5 for ; Fri, 22 Mar 2024 19:33:19 +0000 (UTC) X-FDA: 81925673718.15.9D5BFBB Received: from fout3-smtp.messagingengine.com (fout3-smtp.messagingengine.com [103.168.172.146]) by imf21.hostedemail.com (Postfix) with ESMTP id 90E801C001C for ; Fri, 22 Mar 2024 19:33:17 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=sent.com header.s=fm2 header.b="J jKgd0L"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=FgVjeW9o; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf21.hostedemail.com: domain of zi.yan@sent.com designates 103.168.172.146 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711135997; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Z4qwXiUVtIySYeeeDHMpOYt0KW6NAQHnh4gQJgkXoJ4=; b=qdgCNTRDp/v9huu/C38e8p2Iyuyrc1QTiUS6NHapfkuSLteZ4Y9SnsiViaUeopOR48LHdw PcMpZ58ZpeAzBSOoIyWp5WhNrcX+9gEe+S/TUflGRKUTz0YQbwkVWuDo0E3Afs8NNVivAh ECtEHBKOTsOVK4f0S9DAjFq1PA5VHlk= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=sent.com header.s=fm2 header.b="J jKgd0L"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=FgVjeW9o; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf21.hostedemail.com: domain of zi.yan@sent.com designates 103.168.172.146 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711135997; a=rsa-sha256; cv=none; b=VYo8ovRJoVRcN+I/3wyIHGvSZJt5UPeOIuJ9+a21Y7lTVn71rUOO1tAGxaWL7dUWqd+PrM zvIPwRf0wpkbTRKnrkOrQvDzr0i9hxPWls3p70fR9J5WrEjMQ2+JB90rZ4u63wD1GhRtM4 NR+hibbA30QhNXMbfE5zgAYReES+H3I= Received: from compute4.internal (compute4.nyi.internal [10.202.2.44]) by mailfout.nyi.internal (Postfix) with ESMTP id D261C13800C8; Fri, 22 Mar 2024 15:33:16 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute4.internal (MEProxy); Fri, 22 Mar 2024 15:33:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:message-id:mime-version:reply-to :reply-to:subject:subject:to:to; s=fm2; t=1711135996; x= 1711222396; bh=Z4qwXiUVtIySYeeeDHMpOYt0KW6NAQHnh4gQJgkXoJ4=; b=J jKgd0LDJusRF0knrNa/Y3vIZsqA0w4UKM57xw1itDNsuPwP6aPkIFBrl7zrN7il4 bNqLYIutwLOFKI8x7Z2EVdRDCXvqszrdQXGpUVWZ6Iv8YlGoPUzT1sKnrXJ9KaCj P7xoO+4KFQRE8QfWfptIi+uFGoXs6HCqYO+iL5kBZoThkCK+JYlI/3a4V70zYoyl PNIfqjrGonZNaaSulKPstQXQ0kqlKk9qUV7ao8E/1qHHKdFooQbyobSKo2d2RGwT YlhMGPd6YGLCvLGgvCnU/XzaaKALJLmjjDnu6YSX2E4bxiPJNIDQ9y0ohhU5yKgr ZTl9Nobi4OpJZPh8lBk2Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:message-id:mime-version:reply-to:reply-to :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1711135996; x=1711222396; bh=Z 4qwXiUVtIySYeeeDHMpOYt0KW6NAQHnh4gQJgkXoJ4=; b=FgVjeW9osSLt3u5fV U6dMfnZUjQK+MnEXnWIY45U5l8YS29AcPodXP4MYaLWP8CVZLqDuteQoGN/+MbIA MkQy5yj7FDEKPB/ZXjekF4JOSLp/b6BRbaHY954hleESgi8exrQdLUvTDBrlLp9F RFixnQTy3RkYjUMz5WIev1vLCMHtvAUyKuzayP44l5r3zvEtQ2tfY8xhFBSItMkV fsnu3VCOV7c1ZZYqLMiTBfGM3ilqXSWaVvvFOV6wm40REDF2Nr18VHvd8VeR+ihA ZulIXqjVdExBpdXcVrA50n49XB9EaBl/fqmHAnznBINyaSP3xKdv2uuDsk9H3+O/ PA3wA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddtvddgiedvucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofhrgggtgfesthekredtredtjeenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepteduve ehteehheeiteeihfejveejledtgfdvieeuiedutefftdevtdfhteevtdffnecuffhomhgr ihhnpehkvghrnhgvlhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnhhtrdgtohhm X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Fri, 22 Mar 2024 15:33:15 -0400 (EDT) From: Zi Yan To: linux-mm@kvack.org Cc: Zi Yan , Andrew Morton , "Matthew Wilcox (Oracle)" , Yang Shi , Huang Ying , "Kirill A . Shutemov" , Ryan Roberts , Baolin Wang , "Yin, Fengwei" , SeongJae Park , linux-kernel@vger.kernel.org Subject: [PATCH v5] mm/migrate: split source folio if it is on deferred split list Date: Fri, 22 Mar 2024 15:33:04 -0400 Message-ID: <20240322193304.522496-1-zi.yan@sent.com> X-Mailer: git-send-email 2.43.0 Reply-To: Zi Yan MIME-Version: 1.0 X-Rspamd-Queue-Id: 90E801C001C X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: h6gqjzsycqhox4mxhi1ra9ec5r87by5h X-HE-Tag: 1711135997-372177 X-HE-Meta: U2FsdGVkX1+xRjXvgl9lsvfFojy3lZhTWaGbypCjcfYFCSywWihGn+4pRRFsaR6u8k98kzc+6W+MhBaba7wgndq960lUfYbkbQcRlZFNe7sDef4xQaeUHV98sROMQUosNrJurKdyXwtZb9SvXHBWQ9jO+Fy0cYgcrTv6MqLi3AkQDiWf/niipa+yZlCMbnva4Ja5rN7K6EzZh5sMugMR7hYv4UO9MqIX2XykyYsfi7irzGKcQTpLCYYpTHzzdWsqnCNfM/2QL9V5b5PFOLKFhWNYoQ6V39jgXCmwmS+BOnBVjiKX4dLQ7OCLuOVrgCP2DFo9XM1lGugyhV72/5oVLkgDkm25o404LUiP3Xof+g4J1TFjx/pZkKy9lV7Q8NSo6Jc4u9XR+rh45RW1CfLHyFImsU2OkLSHIbFkpi+nFCUfqxBut3Q5DWOAwOXDnQUzwYSjDhxhB9V2jjnDK8BaAS0w3hJI/nlp3o23PKIsfvdUHnmAO/iZKSOaxykS3es4nZv0V1hvc1TPM3Xhrxa4xAi45JeKA9QslYu2q6CIZhaqp33OuQyTDsFz56llz4PRwcm3yUGU9KBVUz/O2p1yyzCFJI68zc578R00bRoHtE/ix4H1UCy7/uociDzF16cXkGJGnQUemJvPuV8uz/dEDHVjTBEa5SXN31xT0UyhqhHSwVxCM3irzI+7HQnVCUpu3cMpI3CBBWExrejZPq/55/xNfQvKUDZBPCOG2cVm5MLsjGTj78Iy5u+oZCaLpsWeZCMPQWZw4UkiuunIt6zTWkPslo+RPT0F8GTo9tp+z1Met8ubRKJpnQwKINe90xQac1v2XH9ZhmM2vY8BtkbYZPEd/lPiezcovV4LQGI+OnuqpmYELm34VkfB4R2imRFOj3d75RI0Y0yIy4+iekmRh1oAkHsZohXL7kxq4sVXwd2AYYx7JCNz6ZJ4FGrlxKImEUmfRV0sXxgdxRzWMNA +VP6dfJm QJsi8X6SFB+UIDkyTzHeJKqu+2tGLmvJS7ju2bD6gsl57dRoqQ882zZGUyRH+ROjVDsXlFALyGPAoY08mlsBU0j/6Zs76s9HJDbONBjVSiXx3WCvFw/KwiJq5drxsCe1dF1xoD5nlTJIOAoyD8DrCX3n4gllLns/85p1/hlM2PGy02vbU4tP+IXl0SWGdKWAtquCXx1tE0FLD2dHL7Md6VFcnCPkKTtp+NrSkMRZAFMekmzgBlPOQWbJlUdkBioIks5i/SHur2muDHiYaZy6ku8ZLKY1bQ4YNTiiP/QWhy3I7YayxdbGwOxpJ3ysNn5UK5cSpN9oWqChBOqHl9LVeIpTkAP2bCjuG+s9Ilz3dndoeGPf0MHb7QvgdpBl/pKIfvk6Cgfvf3WGVngBF9s1rnMnsIkvWZeIKMQ/bk6/hvYrOF4ewF4cbnpME4KIPGmkND0UnxZubnZ9Hznms1q1U9PKSjuMgjzyELj5p1ff6e8E1UidQdURITEBl2YuRi3mS/QBezzz1Ud9lkm0RH3Ce7CKIj1TdKm3ovcyb X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Zi Yan If the source folio is on deferred split list, it is likely some subpages are not used. Split it before migration to avoid migrating unused subpages. Commit 616b8371539a6 ("mm: thp: enable thp migration in generic path") did not check if a THP is on deferred split list before migration, thus, the destination THP is never put on deferred split list even if the source THP might be. The opportunity of reclaiming free pages in a partially mapped THP during deferred list scanning is lost, but no other harmful consequence is present[1]. From v4: 1. Simplify _deferred_list check without locking and do not count as migration failures. (per Matthew Wilcox) From v3: 1. Guarded deferred list code behind CONFIG_TRANSPARENT_HUGEPAGE to avoid compilation error (per SeongJae Park). From v2: 1. Split the source folio instead of migrating it (per Matthew Wilcox)[2]. From v1: 1. Used dst to get correct deferred split list after migration (per Ryan Roberts). [1]: https://lore.kernel.org/linux-mm/03CE3A00-917C-48CC-8E1C-6A98713C817C@nvidia.com/ [2]: https://lore.kernel.org/linux-mm/Ze_P6xagdTbcu1Kz@casper.infradead.org/ Fixes: 616b8371539a ("mm: thp: enable thp migration in generic path") Signed-off-by: Zi Yan --- mm/migrate.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) base-commit: 08a487ab26d541a3bd0adaee144f684b724d233b diff --git a/mm/migrate.c b/mm/migrate.c index ab9856f5931b..6bd9319624a3 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1652,6 +1652,29 @@ static int migrate_pages_batch(struct list_head *from, cond_resched(); + /* + * The rare folio on the deferred split list should + * be split now. It should not count as a failure. + * Only check it without removing it from the list. + * Since the folio can be on deferred_split_scan() + * local list and removing it can cause the local list + * corruption. Folio split process below can handle it + * with the help of folio_ref_freeze(). + * + * nr_pages > 2 is needed to avoid checking order-1 + * page cache folios. They exist, in contrast to + * non-existent order-1 anonymous folios, and do not + * use _deferred_list. + */ + if (nr_pages > 2 && + !list_empty(&folio->_deferred_list)) { + if (try_split_folio(folio, from) == 0) { + stats->nr_thp_split += is_thp; + stats->nr_split++; + continue; + } + } + /* * Large folio migration might be unsupported or * the allocation might be failed so we should retry