From patchwork Tue Sep 12 16:28:11 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13381931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7C68CA0EEB for ; Tue, 12 Sep 2023 16:28:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 411B96B012B; Tue, 12 Sep 2023 12:28:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C2006B012C; Tue, 12 Sep 2023 12:28:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 28A326B012D; Tue, 12 Sep 2023 12:28:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 1A5FB6B012B for ; Tue, 12 Sep 2023 12:28:36 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 8CA7D407DD for ; Tue, 12 Sep 2023 16:28:35 +0000 (UTC) X-FDA: 81228478590.10.98CDB3A Received: from out2-smtp.messagingengine.com (out2-smtp.messagingengine.com [66.111.4.26]) by imf19.hostedemail.com (Postfix) with ESMTP id 6A8C51A001D for ; Tue, 12 Sep 2023 16:28:33 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=sent.com header.s=fm1 header.b=yPxIE70a; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="M3g/WjBO"; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf19.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694536113; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=5t1ioQ/11KphbqzXH7SCYin9Bo5ArvfTJo+4zOQ+tUo=; b=tjhaqILdRASMfbE+IfSnbUth2WsPP/MzoELBf3zzfiNdjErAeyuxmIKgtd586HAJ+W75IN peLd1aANaAOHBotxchqWAWHUJvT/vvB7Lj5q+86BpdwItn3jzdkzekeWgJ5wfYAH51b9cQ Hxk1Uqco9XZEHx/YOmlzwCl7AcZCGE8= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=sent.com header.s=fm1 header.b=yPxIE70a; dkim=pass header.d=messagingengine.com header.s=fm1 header.b="M3g/WjBO"; dmarc=pass (policy=none) header.from=sent.com; spf=pass (imf19.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.26 as permitted sender) smtp.mailfrom=zi.yan@sent.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694536113; a=rsa-sha256; cv=none; b=sJGkufe59EqBAk9PwyxmM+sVph23CRq7pRvK62SQMpPL4FLNjKL1nvQX/w9LsqrfgTnaqK 0I8BWpxpL66DEquq5UWoyKrdda4ZUavPsXB8noYR1PIRxykzZvnSqDcpGMipuV4DANZvr6 T1UzPCaA23Db4GsEHSS+J1/AP2c4nZ0= Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id 8F5BD5C020E; Tue, 12 Sep 2023 12:28:32 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Tue, 12 Sep 2023 12:28:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:date:date:from:from :in-reply-to:message-id:mime-version:reply-to:reply-to:sender :subject:subject:to:to; s=fm1; t=1694536112; x=1694622512; bh=5t 1ioQ/11KphbqzXH7SCYin9Bo5ArvfTJo+4zOQ+tUo=; b=yPxIE70aHJyQBG+Cz7 r1pUm8bA7QZYChGe1yn5GyPCFUsWuCw8vTMgKJmPWCfBljM6LYkcMmbYqPU3v8W1 McvyoyJHw0npjr6fTKnw0sr9TRsbFbyXlgtwcNGKqStMbESs4XwQQ3D38KA/YrbS KgFz6ASTZTmJ2PXCS9mzxogr12O+bxO3FvfD89UhJc33NzdB6GfVoNxgLx0OMEIF 69l0gbGaX8ewEojdkEH+NzlFLMBXBGOZWc5wFtP3yZ0i7P9AgnvXf9zqCsLe03PD JLnF3CucRaTXVr0Otp/U/SQ0xXWS/FxTmkwt6s/fXU9X18/LyA1VsRX/BMDCe1oU PYvw== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:date:date:feedback-id:feedback-id:from:from :in-reply-to:message-id:mime-version:reply-to:reply-to:sender :subject:subject:to:to:x-me-proxy:x-me-proxy:x-me-sender :x-me-sender:x-sasl-enc; s=fm1; t=1694536112; x=1694622512; bh=5 t1ioQ/11KphbqzXH7SCYin9Bo5ArvfTJo+4zOQ+tUo=; b=M3g/WjBOWlHh3VsRR 8pDFfruMUOVnACEwfCjQWnnoCfpyyBSB4X5h+mfZkTim3kdPSAQySOdddZl8FbLN QQI+5pCliiBxHNtQFzOJvD1abGwpoVL7Q5LwiM8lByN4baeMu8Znx+Z616roydOz TG6AW7aQ3ZmDGMhu80YKMaSdz9wjkzd6Sni1cbTmQXqB6OzL8YEVG9KNPyxFPjBK crlzO7e5y5aLclGd/KAstXTbSsN+rkdJXFEJze9ClEiTpRx4VoHxOmJMczbncWj7 6czUo77It5UygNrbHbwUf3QkLnQePOc/mK/rS9TCCPdF2ps+xfUy0ju0mwyZanQ+ mf9gw== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedviedrudeiiedguddtvdcutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefhvfevufffkfforhgggfestdhqredtredttdenucfhrhhomhepkghiucgj rghnuceoiihirdihrghnsehsvghnthdrtghomheqnecuggftrfgrthhtvghrnhepudevud egfeffgffhteehjeeuheejueelvdehhfekhfduieeggfduvdevkeevieevnecuffhomhgr ihhnpehkvghrnhgvlhdrohhrghdptghomhhprggtthhiohhnrdhmmhdpshhplhhithdrmh hmpdhprghgvghsrdhmmhdpfhholhhiohhsrdhmmhenucevlhhushhtvghrufhiiigvpedt necurfgrrhgrmhepmhgrihhlfhhrohhmpeiiihdrhigrnhesshgvnhhtrdgtohhm X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Tue, 12 Sep 2023 12:28:29 -0400 (EDT) From: Zi Yan To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Zi Yan , Ryan Roberts , Andrew Morton , "Matthew Wilcox (Oracle)" , David Hildenbrand , "Yin, Fengwei" , Yu Zhao , Vlastimil Babka , Johannes Weiner , Baolin Wang , Kemeng Shi , Mel Gorman , Rohan Puri , Mcgrof Chamberlain , Adam Manzanares , John Hubbard Subject: [RFC PATCH 0/4] Enable >0 order folio memory compaction Date: Tue, 12 Sep 2023 12:28:11 -0400 Message-Id: <20230912162815.440749-1-zi.yan@sent.com> X-Mailer: git-send-email 2.40.1 Reply-To: Zi Yan MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 6A8C51A001D X-Stat-Signature: sgm1rftauwcricex1x8mez1w7gpxshyz X-Rspam-User: X-HE-Tag: 1694536113-872094 X-HE-Meta: U2FsdGVkX1+IlI37oHxMjbVJio49eHvPzXFYt7V7aK06Q7JE9hKn9MXgj2IISCx2iICh/aC6hBi0kFol09NPzonuQa3K4hknM1R8q88ff2T6wujRP6vyNKtGo+GY6HGcqn43ajr5iTyi9Q7MR/Af/F1zvlvE/gRZkavIOrTBpqMruLyLVoCF+hD/robUXVPkcEqvae7KsRRNg95twquxT7dRL8UDEseWFUx/lTvddjx1j7tMG2NP9Fjg0PCiIg4dbA4xV8ynQ+yrLXnLEJJ/QnHpwctdJP0iuoa7wwAcvF/IGMAs66fPHETwmgFej67/BWuR+p7i6TP9ETcTSlF8eJCE8KSkVWRKHIkR4m6L40o4b/r4zBRLR3mLkbiNTCbDKwErVBUKvPw+AeJf979vqLswFpE2B15x3vKgOxTEPHsLC1YRBlkMA4xVB7J6OLO0bBI4jBkeUaQDmymvAtfrUYVE1fpt36hApzDgjbyUFGGHVOjvXweacZIq6OOEOO9n2kNHDWjhqsJQO3CQKu7Fh2f5xOrVe9anEV3mlCuwcTTWCtB/Hcl7iv8EPApmI5jfawaoLBWJqzXEvqbkNkyMd8k265jp954uXb6C6ESkUM/Jqbruq7/BoGIT3anXcveQoIJ7cSAlCVO/wzIUYcfLDxj3jmeCLXoNrjwnuUe2AyKxpndLx7eM855fMmNroXvTQQNeOS1WQYiVo3BOmwQU2RLrgl520aIVGYIq3ru2t93TokcZm65DcEk0bB9584nRlEEqC4eCW016WRjS5aMJVe9KTsZuHmS+noS7Z1NbSZ5qHLWOCYObU0OAO21UkfcqIrN2N5a3iq0GcoFmwBNCfF8/823j6q0Mqlhp2Dd4gIQSi6ageh+VFnJqYZP9MBlidcFipjWa6Dm4pAYkurg2OqtSLGFsmYQAhivgXmhoChIGW1N8nJVPQcDdwNA8rQWE4hxCsgx2qbbrOD3qt1N FYIcL2MB c9uUu3hzNo2MGtBlBOPRX89svXBRVnedzZ7CX53+r94ctT5jZXYbHKuljDKYhOyhc+9CfcVy+S+Co7lRFnWW42Ha0JFbG2XeSajOAhlpNHGRmPGHlBfBQrLzZ8xCjhTgl5qPExl5hFRhQs790a2W/SUMALQUJ22WcppPx2qHH6/C0UuCzH32/+qupyKJf+VAc79ZS4D7/vthKlMMHx2kIN1XPVgX3+25gyJX+8uMUzwsFhdhLJT6w4QlfJmj+/CNjc2x0SlPXcqWpd38dn+cl2VnjkuKFpQ44CY0wRzuGxQ94XfBYPyRrzlWIT3+GWzoa/WSiuq2yXsZB4f0MImsl/DsUmEn3ObUi+YClV6guvaIK88Ee1tsVVr/g1ie2U0NDz/g8LOU8kX8CzYpQiYk0GTfOthspD/BzEmPiyYru3XOW77RfRj5HzxAGGuBUMf/naSzf48CRSP7GplnvH3aGiv3MMq6/AcTMD4dLD30sMRQl26vwmjqgO+5963g1NujxMkpXySuvzr5sYIAmlPn/lcmCaJ213BJi6l9yL7augdvt8vObGGARYIC5HbYdt/E8XnNUrixVR7rGX4lxVyajtms4QkektPD1pG+lbi676HWdMGYHfMVViGVbBzp+2JCt1QEg X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Zi Yan Hi all, This patchset enables >0 order folio memory compaction, which is one of the prerequisitions for large folio support[1]. It is on top of mm-everything-2023-09-11-22-56. Overview === To support >0 order folio compaction, the patchset changes how free pages used for migration are kept during compaction. Free pages used to be split into order-0 pages that are post allocation processed (i.e., PageBuddy flag cleared, page order stored in page->private is zeroed, and page reference is set to 1). Now all free pages are kept in a MAX_ORDER+1 array of page lists based on their order without post allocation process. When migrate_pages() asks for a new page, one of the free pages, based on the requested page order, is then processed and given out. Optimizations === 1. Free page split is added to increase migration success rate in case a source page does not have a matched free page in the free page lists. Free page merge is possible but not implemented, since existing PFN-based buddy page merge algorithm requires the identification of buddy pages, but free pages kept for memory compaction cannot have PageBuddy set to avoid confusing other PFN scanners. 2. Sort source pages in ascending order before migration is added to reduce free page split. Otherwise, high order free pages might be prematurely split, causing undesired high order folio migration failures. TODOs === 1. Refactor free page post allocation and free page preparation code so that compaction_alloc() and compaction_free() can call functions instead of hard coding. 2. One possible optimization is to allow migrate_pages() to continue even if get_new_folio() returns a NULL. In general, that means there is not enough memory. But in >0 order folio compaction case, that means there is no suitable free page at source page order. It might be better to skip that page and finish the rest of migration to achieve a better compaction result. 3. Another possible optimization is to enable free page merge. It is possible that a to-be-migrated page causes free page split then fails to migrate eventually. We would lose a high order free page without free page merge function. But a way of identifying free pages for memory compaction is needed to reuse existing PFN-based buddy page merge. 4. The implemented >0 order folio compaction algorithm is quite naive and does not consider all possible situations. A better algorithm can improve compaction success rate. Feel free to give comments and ask questions. Thanks. [1] https://lore.kernel.org/linux-mm/f8d47176-03a8-99bf-a813-b5942830fd73@arm.com/ Zi Yan (4): mm/compaction: add support for >0 order folio memory compaction. mm/compaction: optimize >0 order folio compaction with free page split. mm/compaction: optimize >0 order folio compaction by sorting source pages. mm/compaction: enable compacting >0 order folios. mm/compaction.c | 205 +++++++++++++++++++++++++++++++++++++++--------- mm/internal.h | 7 +- 2 files changed, 176 insertions(+), 36 deletions(-)