From patchwork Wed Jan 25 13:44:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 13115653 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1F01CC27C76 for ; Wed, 25 Jan 2023 13:45:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B91C46B0078; Wed, 25 Jan 2023 08:45:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B41D66B007E; Wed, 25 Jan 2023 08:45:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A30AD6B0080; Wed, 25 Jan 2023 08:45:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 9362C6B0078 for ; Wed, 25 Jan 2023 08:45:32 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 6781914040D for ; Wed, 25 Jan 2023 13:45:32 +0000 (UTC) X-FDA: 80393443704.21.6834D6B Received: from outbound-smtp58.blacknight.com (outbound-smtp58.blacknight.com [46.22.136.242]) by imf23.hostedemail.com (Postfix) with ESMTP id 5807A140018 for ; Wed, 25 Jan 2023 13:45:30 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.136.242 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1674654330; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=dBusSDQMc1Av9eoZM4nbPmSEyQIlVUn7Q/Wo0slBnwQ=; b=uiya4Jfzh9iacfUfduNRhpJVgdVI2dRW3RxcZOdJCt3bdJe8pB8SofTvWsQ/Xk+3CSMtGp CZq9WRsara8A1OSVIvyO+4zzxQ5n0nRDD2E0zqEWn9EiR7oEXP3pezlqKWwdteJ9noDwnb ARTeIMiXVEx6qNL5g2BMG8OTeE58S1M= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=none; dmarc=none; spf=pass (imf23.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.136.242 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1674654330; a=rsa-sha256; cv=none; b=eLixT969VQa5dZDfD9MQobGWHln9TVmxHBDybrZAODF7OAL8C2JGmt7Na0BU/6yygdIEa1 1G8L9+RL3FaCY+2jBCDLwZIT87MgB4LhrC6ikPJIC1PAuFcHI+NT/CPLzLOtt9WIpVi76b zIQw+jomON4pY8kr/gL/WouygjF6VEg= Received: from mail.blacknight.com (pemlinmail05.blacknight.ie [81.17.254.26]) by outbound-smtp58.blacknight.com (Postfix) with ESMTPS id 05CB2FAB23 for ; Wed, 25 Jan 2023 13:45:29 +0000 (GMT) Received: (qmail 22502 invoked from network); 25 Jan 2023 13:45:28 -0000 Received: from unknown (HELO morpheus.112glenside.lan) (mgorman@techsingularity.net@[84.203.198.246]) by 81.17.254.9 with ESMTPA; 25 Jan 2023 13:45:28 -0000 From: Mel Gorman To: Vlastimil Babka Cc: Andrew Morton , Jiri Slaby , Maxim Levitsky , Michal Hocko , Pedro Falcato , Paolo Bonzini , Chuyi Zhou , Linux-MM , LKML , Mel Gorman Subject: [PATCH 4/4] mm, compaction: Finish pageblocks on complete migration failure Date: Wed, 25 Jan 2023 13:44:34 +0000 Message-Id: <20230125134434.18017-5-mgorman@techsingularity.net> X-Mailer: git-send-email 2.35.3 In-Reply-To: <20230125134434.18017-1-mgorman@techsingularity.net> References: <20230125134434.18017-1-mgorman@techsingularity.net> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 5807A140018 X-Stat-Signature: 6hwocbnnxjiwyr1k356ir9h1fn1x4j6u X-Rspam-User: X-HE-Tag: 1674654330-203337 X-HE-Meta: U2FsdGVkX18tmmndoJ1tPr9Cl2gJQ0uJ37kLNWV/zgZM3LeRZbmVKjB/ynjPtnJHkIQSKfeXnBHZCVBJdquQrsQz5//+zpIIt8EfHsoPPhtv3a3odFoXTdZpe35FeujcUvU0oaNRMiDd/wfKnR81Aqy0GzYsckRrOYKBpteJ53WA9E9PBDdQh5KD11VC/cKxF4ySsGllu57ojk27MDD2leE5Ve6TZnPUUlwoz7fTB/fQJih/7AtEiXEVUDYpyNL+eyJ9ENTdbzYzOla1h+vFkxzJbF6ddMelSYBPPpec/6mawG7SZtWCt+TRJTf+cj1ei0Nzvgpe07J7AZtnmPqNSZnj/mj5DVHVw+a6MHs7dLWv6bFAd3FMHOx//Wa7poAGfJEEJH9LvCQqOBjj6Z2/LvQc8Nvh+jKng6hEAE1hiAQyX+1fx/CcmLRO5hSFqe5Zt/YHsSbAxeB/+ZYibCB/0Ws2J+Gn8wc6j/xsu9l+nkraauAGZMQxcEkaamSrNIKPtM8/QisOFq5ZC4CiO8DyWvkuKgwhEmeLAb2Jv4cckQDYMUZK5aTjZMyH7rK90zobtjK7ixlYkNcVBON6h4HpU3L7anDHcSa4OKJnGjFJKopn1jo3ODDQJTHHMAAyBNTjlS1dKp75TjDyHZ4azDGyCNZzr0FBjgXUrfRoHGKGRL1Pghl3kjMUYcNU7I+YK0tfEkZjheuPPNoH4DO2fFFVi+VN/20cUeNnns8hgYpnG2QCpjhBvYGbzP3GsrdI2SRb1W4OYwddme5Xlww4R1Vl8O4K/x4vdr5RufFIHRR37twUfsqGEMvrncovWRBcvCN6PdTyg2bJC/S07WvfY1E0c1ZQIZguhOHPul5R/pfjCHon3rlk5Gjw38o287mXMzAqWj06XlIQerafDEY1V3BgU6YSqgIqbMHWSh2j+vZ97qY2ePEV0BfEs+66bvD9PUAv2OENV61pmR/O26o/1GK 34p9sZw6 Ntipo+3yC6n0TAmeFJxMTZvmJpsVstmnmH6lOQSi4ixSclLb09YnvF2CKliafgIovZpM8MgVO8TweRaM2fXXM6erA5J1T8s0wnOgR2KPj3JD5F/e1RmfVtZS9E5M95vnSdwnGuyV5qth+H2A7XKtzVUF3BoJtd6TG53DWMz6z+37WbiRtgYE7rEj+fLn52mQB0hF4tn/z4XGQ2g3iqJXF0wyidR5kzvL+9iGOq1K8TjxoZyNWl5nWRjOEV1ikdTayJMAcKpfJsO8F+bgdoROFWcxj502yTbyPRT1tS5MTnvIQEralCivSh5C6nQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Commit 7efc3b726103 ("mm/compaction: fix set skip in fast_find_migrateblock") address an issue where a pageblock selected by fast_find_migrateblock() was ignored. Unfortunately, the same fix resulted in numerous reports of khugepaged or kcompactd stalling for long periods of time or consuming 100% of CPU. Tracing showed that there was a lot of rescanning between a small subset of pageblocks because the conditions for marking the block skip are not met. The scan is not reaching the end of the pageblock because enough pages were isolated but none were migrated successfully. Eventually it circles back to the same block. Pageblock skip tracking tries to minimise both latency and excessive scanning but tracking exactly when a block is fully scanned requires an excessive amount of state. This patch forcibly rescans a pageblock when all isolated pages fail to migrate even though it could be for transient reasons such as page writeback or page dirty. This will sometimes migrate too many pages but pageblocks will be marked skip and forward progress will be made. "Usemen" from the mmtests configuration workload-usemem-stress-numa-compact was used to stress compaction. The compaction trace events were recorded using a 6.2-rc5 kernel that includes commit 7efc3b726103 and count of unique ranges were measured. The top 5 ranges were 3076 range=(0x10ca00-0x10cc00) 3076 range=(0x110a00-0x110c00) 3098 range=(0x13b600-0x13b800) 3104 range=(0x141c00-0x141e00) 11424 range=(0x11b600-0x11b800) While this workload is very different than what the bugs reported, the pattern of the same subset of blocks being repeatedly scanned is observed. At one point, *only* the range range=(0x11b600 ~ 0x11b800) was scanned for 2 seconds. 14 seconds passed between the first migration-related event and the last. With the series applied including this patch, the top 5 ranges were 1 range=(0x11607e-0x116200) 1 range=(0x116200-0x116278) 1 range=(0x116278-0x116400) 1 range=(0x116400-0x116424) 1 range=(0x116424-0x116600) Only unique ranges were scanned and the time between the first migration-related event was 0.11 milliseconds. Fixes: 7efc3b726103 ("mm/compaction: fix set skip in fast_find_migrateblock") Signed-off-by: Mel Gorman --- mm/compaction.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/mm/compaction.c b/mm/compaction.c index 4b3a0238879c..937ec2f05f2c 100644 --- a/mm/compaction.c +++ b/mm/compaction.c @@ -2394,6 +2394,7 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) cc->finish_pageblock = true; } +rescan: switch (isolate_migratepages(cc)) { case ISOLATE_ABORT: ret = COMPACT_CONTENDED; @@ -2436,15 +2437,28 @@ compact_zone(struct compact_control *cc, struct capture_control *capc) goto out; } /* - * We failed to migrate at least one page in the current - * order-aligned block, so skip the rest of it. + * If an ASYNC or SYNC_LIGHT fails to migrate a page + * within the current order-aligned block, scan the + * remainder of the pageblock. This will mark the + * pageblock "skip" to avoid rescanning in the near + * future. This will isolate more pages than necessary + * for the request but avoid loops due to + * fast_find_migrateblock revisiting blocks that were + * recently partially scanned. */ - if (cc->direct_compaction && - (cc->mode == MIGRATE_ASYNC)) { - cc->migrate_pfn = block_end_pfn( - cc->migrate_pfn - 1, cc->order); - /* Draining pcplists is useless in this case */ - last_migrated_pfn = 0; + if (cc->direct_compaction && !cc->finish_pageblock && + (cc->mode < MIGRATE_SYNC)) { + cc->finish_pageblock = true; + + /* + * Draining pcplists does not help THP if + * any page failed to migrate. Even after + * drain, the pageblock will not be free. + */ + if (cc->order == COMPACTION_HPAGE_ORDER) + last_migrated_pfn = 0; + + goto rescan; } }