From patchwork Tue Mar 26 15:00:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13604346 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8FD9CC54E67 for ; Tue, 26 Mar 2024 15:00:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 003316B00A2; Tue, 26 Mar 2024 11:00:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EF46B6B00A3; Tue, 26 Mar 2024 11:00:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D48C76B00A5; Tue, 26 Mar 2024 11:00:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B3E276B00A2 for ; Tue, 26 Mar 2024 11:00:41 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 75708C0CDA for ; Tue, 26 Mar 2024 15:00:41 +0000 (UTC) X-FDA: 81939501882.18.BAC89FD Received: from wfout3-smtp.messagingengine.com (wfout3-smtp.messagingengine.com [64.147.123.146]) by imf21.hostedemail.com (Postfix) with ESMTP id 24EDD1C0034 for ; Tue, 26 Mar 2024 15:00:37 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=sent.com header.s=fm2 header.b="h lzN8L8"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=N3dxK1hb; spf=pass (imf21.hostedemail.com: domain of zi.yan@sent.com designates 64.147.123.146 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711465238; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=K/wy8Tr0L+Nsny9bf7oP+FXiEoeQqvQFzQlzCUqbxGA=; b=ZSdB5a6DSVXe5Qe/3Zj5sqIODI+NGl/kcCsynMiZMuOlUEhVp90vZP8St22AmiIrcrOCFN 3oRZSAgIJ23iFxlVR2DZEvyF63jzL+gADJW5sR0MdLE3W+CM6bvuo/2oivCSnQ2Ir7vHpq 0Nk4DieIIOHHsRckGaPpAq+dPwTR+7U= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=sent.com header.s=fm2 header.b="h lzN8L8"; dkim=pass header.d=messagingengine.com header.s=fm2 header.b=N3dxK1hb; spf=pass (imf21.hostedemail.com: domain of zi.yan@sent.com designates 64.147.123.146 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711465238; a=rsa-sha256; cv=none; b=P5LtOYMXJWj7Vma3RkaVdfcuVOFZrUp81+/0Gr3zD2rwuJUx7mXVJHn+B8BqgCIIhOVdBh xXuVVxNiXx7Z9WhHGo4J5ftBheZ/OddFMp1hzfzLld6v2KiUUchHgQhkKYhGy1EH46XOLQ cZmwY209thBem2mBauugx/VfqlyeEQk= Received: from compute2.internal (compute2.nyi.internal [10.202.2.46]) by mailfout.west.internal (Postfix) with ESMTP id 552081C0008E; Tue, 26 Mar 2024 11:00:35 -0400 (EDT) Received: from mailfrontend1 ([10.202.2.162]) by compute2.internal (MEProxy); Tue, 26 Mar 2024 11:00:36 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:message-id:mime-version:reply-to :reply-to:subject:subject:to:to; s=fm2; t=1711465234; x= 1711551634; bh=K/wy8Tr0L+Nsny9bf7oP+FXiEoeQqvQFzQlzCUqbxGA=; b=h lzN8L8MgVeC1huObqTMQm4/JkVu+6/ioqXutkfnME+OCbS8EhTU3hhGAW0AHAD7o gEqTUhe68Vfn2iSv+T3/xMxWPMighv7viBL4jKnwD/vAL2n0egzt2s0RscmeuFAR UaNprhjpQP96boS6nxC2YaMdizwJ23xBttznLb6QJW2CFTFhWcAUSVwFd36k6xP5 Sj17Fm2rQO2eGNYhpKThf0dGP1zvR6Oo3e+YbdUCNG62bB9unvJt1nrwcpt3pSVG 32RtdPHsrmszt31LfTXFv5k0Fog2l3mmkbwcLs8O7D/PsMwJhy0/fpEJz0LsIn+u 7aR+84OEZ9CP3HJh6Sd7Q== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:message-id:mime-version:reply-to:reply-to :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm2; t=1711465234; x=1711551634; bh=K /wy8Tr0L+Nsny9bf7oP+FXiEoeQqvQFzQlzCUqbxGA=; b=N3dxK1hbhVHxdQTM1 b80EL1pK16h2vM7WZTFAYX/XNGqHNLxyb0ydQPoZytSRV31Hp8b4Ci7YiuwdShUM Pp80f93T+j3TVSviYsV6qWU8PrNGfyU2wLxh4E8Co2gKdOu8o2LbR5blmfHuF4qS RCZR0LZ8UfVJ5rOIQ8I/UA+/G7L4a3J6HxAMYipZY+bOtFTW1wvV0xlNoDQK7cUE tdrqA0KzdnMhz0VtJXMhFfQN9A2LKrRdia4uTtB+yV7IwhAc3yl84eC4en//+oyQ dHpiA83OejYUQ1mnjGMSfqTlWgTef+UDrRxG5pPGlvZnqGpoUP9bz9hcBzYRAPQG WbShQ== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledruddufedgjeduucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne cujfgurhephffvvefufffkofhrgggtgfesthekredtredtjeenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepteduve ehteehheeiteeihfejveejledtgfdvieeuiedutefftdevtdfhteevtdffnecuffhomhgr ihhnpehkvghrnhgvlhdrohhrghenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmh epmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnhhtrdgtohhm X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 26 Mar 2024 11:00:33 -0400 (EDT) From: Zi Yan To: linux-mm@kvack.org Cc: Zi Yan , Andrew Morton , "Matthew Wilcox (Oracle)" , Yang Shi , Huang Ying , "Kirill A . Shutemov" , Ryan Roberts , Baolin Wang , "Yin, Fengwei" , SeongJae Park , linux-kernel@vger.kernel.org Subject: [PATCH v6] mm/migrate: split source folio if it is on deferred split list Date: Tue, 26 Mar 2024 11:00:31 -0400 Message-ID: <20240326150031.569387-1-zi.yan@sent.com> X-Mailer: git-send-email 2.43.0 Reply-To: Zi Yan MIME-Version: 1.0 X-Rspamd-Queue-Id: 24EDD1C0034 X-Rspam-User: X-Stat-Signature: myi5wzm57i6ugshbytqhdnafjxrbm916 X-Rspamd-Server: rspam01 X-HE-Tag: 1711465237-501471 X-HE-Meta: U2FsdGVkX1+JVsgPxiAUgSh+JfeBwyeqiGq7bPpk/d+dsaugiFeYKy7P+gvcnyNzJpTwxswhjwuMYlrJo+Lthn8A2gulqsvBn0LX1NDj791/EO6c5tW+5IFMc0RudrsuUQviiPDR5j96fmk/EJrVSqGQ9PqoSR/dLFMZuy7k9cFC5BjMTXckZ3AA8h59ATZ7k0JthP2QmYpQr+t9SvRr2zhEqodXX4lONPtJRHYR/TPmiAYb6tkwyOF9ui1HC6XU2p9iiTVtFXowcbttXwhREMBwObO+uhL6tSDnf5yGfHw4qvQHv7BI5TUbl1DB8cPDu06NISb4v2balIIrt2Xu8zPn8zRkMWFd2t4G4exVAggthnEJBc2WjB62wQn9+jOaOnVF0byoZjxdxMLASU0pS4Ctj7EbKR6E1dEorqivSr+u50p0s0XldaW+r1yO7UA0++jhfhKCdkBLhXn4W9mxJljNZbal89hPovBhpcJu6iB7QA2WwPGd2PTq7a+FyN6iebB2Zcomr8VmzdvRdpTiSVpT5L4KuiTUnQdNjJZlIAqdDGdAQdeNsxIIPI8p9o+dHY9ZcsJMAyN0fasvqC5fcOLuKlkinQi1aBbzqbNi+3aoODcjQBUCCCtEPEuN1ddi86cSvBoBHw5ZsPnbDRyYW1zLMn3+/CoO67894zing/ihPJyw76kpRzbI6TAcL8xWAg2BKPtosRbch1ttpBNJkOiykhNZPovqoXKXM6xJy194DQxngiiscgupM+/qGFhAcnj8hWOozCu48bE92dguOTtBm0NjcV5b074UnqPlYiwi7UdLQv0H/zb3glK3AB4SDLFyQlIgBJE6RY/IP/80xA+y88U1xu6fcsFAIUjkkYVPrFZZGcIefFIHMgi7opTDq4c/LyndE4n18XxwTEED71SRLxuYksKMAFrizLFg2/6q5IxPaHPPLOglKiPdFevk52C8bHo7ynh8KlRhTLp Jr22awvQ DQOM3H95LpVFwmGw0AA1AzyMhA1yE8QMf/DclHq6jaZHdPzlhgQ7oibAHJuC5NQfL0EL9aMpMc1e3SHMBuH7oRRwoIcLMdUDFCKCPbkV9HN6pkPQ2W/K8Rf1NMcu5V0RAfaQQtpUVPGEHeRiQiW9ZaNKAWzg0ieVDtQSRvHPzbhazpFbq4Gvdv1ZZBe76BD7JFGravl+yvZRWayxj6oMzPTDYEzY2eEjZGZiXjH55cw862ydLuaMv+JOaUeKdSwIKqhy5j00jPMm6XeL9Pdjio+yfbhisZlS298g02tPorR1Pc3/RkfkAt5kr0VFuX/PcFqXNzHqP9cw16vAkambaDx5qx02mJ6DA8RMExk1E3qCKrYdOfrMVOph7ciW3wZEIGhc9JthNm62TI7G1bVT1uQ9dCgeqEyxR8/v5w8oNyRhNCZgkbVSkV+YLX/pNKhvILj4gA3HqA3pk7opH3HG4tajYqamOqEH2Nq4S8nJQ6ns9l0aRfDjkYTDkUC3OMqTZpMb3I4re5dI7gnLCvRkhycjRSSc98cxAjG1CYI+scroGsNO6+v2HKlyH5ATD+1kszzVE/12M7XCqpZpGFhXjJUn24iRDVLMoz+fn X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Zi Yan If the source folio is on deferred split list, it is likely some subpages are not used. Split it before migration to avoid migrating unused subpages. Commit 616b8371539a6 ("mm: thp: enable thp migration in generic path") did not check if a THP is on deferred split list before migration, thus, the destination THP is never put on deferred split list even if the source THP might be. The opportunity of reclaiming free pages in a partially mapped THP during deferred list scanning is lost, but no other harmful consequence is present[1]. From v5: 1. Fixed an error in migrate_misplaced_folio() reported by Baolin Wang[3]. From v4: 1. Simplify _deferred_list check without locking and do not count as migration failures. (per Matthew Wilcox) From v3: 1. Guarded deferred list code behind CONFIG_TRANSPARENT_HUGEPAGE to avoid compilation error (per SeongJae Park). From v2: 1. Split the source folio instead of migrating it (per Matthew Wilcox)[2]. From v1: 1. Used dst to get correct deferred split list after migration (per Ryan Roberts). [1]: https://lore.kernel.org/linux-mm/03CE3A00-917C-48CC-8E1C-6A98713C817C@nvidia.com/ [2]: https://lore.kernel.org/linux-mm/Ze_P6xagdTbcu1Kz@casper.infradead.org/ [3]: https://lore.kernel.org/linux-mm/df9a644c-a007-46ac-98e3-61d4014fcfff@linux.alibaba.com/ Fixes: 616b8371539a ("mm: thp: enable thp migration in generic path") Signed-off-by: Zi Yan Tested-by: Baolin Wang Reviewed-by: Baolin Wang Acked-by: David Hildenbrand --- mm/migrate.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/mm/migrate.c b/mm/migrate.c index 1dbe5bd927de..a31aa75d223d 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1652,6 +1652,29 @@ static int migrate_pages_batch(struct list_head *from, cond_resched(); + /* + * The rare folio on the deferred split list should + * be split now. It should not count as a failure. + * Only check it without removing it from the list. + * Since the folio can be on deferred_split_scan() + * local list and removing it can cause the local list + * corruption. Folio split process below can handle it + * with the help of folio_ref_freeze(). + * + * nr_pages > 2 is needed to avoid checking order-1 + * page cache folios. They exist, in contrast to + * non-existent order-1 anonymous folios, and do not + * use _deferred_list. + */ + if (nr_pages > 2 && + !list_empty(&folio->_deferred_list)) { + if (try_split_folio(folio, split_folios) == 0) { + stats->nr_thp_split += is_thp; + stats->nr_split++; + continue; + } + } + /* * Large folio migration might be unsupported or * the allocation might be failed so we should retry