From patchwork Fri Apr 14 14:14:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mel Gorman X-Patchwork-Id: 13211582 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1993BC77B6E for ; Fri, 14 Apr 2023 14:14:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7B3E96B0071; Fri, 14 Apr 2023 10:14:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 763CD6B0072; Fri, 14 Apr 2023 10:14:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 652C1900002; Fri, 14 Apr 2023 10:14:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 557266B0071 for ; Fri, 14 Apr 2023 10:14:36 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 044DC1A0324 for ; Fri, 14 Apr 2023 14:14:35 +0000 (UTC) X-FDA: 80680192152.28.F62C54A Received: from outbound-smtp36.blacknight.com (outbound-smtp36.blacknight.com [46.22.139.219]) by imf26.hostedemail.com (Postfix) with ESMTP id 30B3514001C for ; Fri, 14 Apr 2023 14:14:32 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.139.219 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681481673; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references; bh=bqNnFZYcSPLDWtpTY6WXVnE/MjuzD/klvxGDSHz0WYI=; b=3zu1TLhRTjB9G1+QeBq36Y/SAiqnmLQLky+a4cvGxJf4IE/OAVSgfZdVwkM6C85y5lOfJ2 byDHynC0qbnphKXYva8fTLJ9hc2MeXq95DNcZ7OVz5khDChhj4kVodYeekF0GlEf6u74lB pAQDgNDct7RBR8ULxaalvxmd+DGqx3w= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of mgorman@techsingularity.net designates 46.22.139.219 as permitted sender) smtp.mailfrom=mgorman@techsingularity.net; dmarc=none ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681481673; a=rsa-sha256; cv=none; b=NZWxZrmNqWddDVUXr3/xbqkTwBXtn+ulTycfMQa6H7vgDje//8uQgx9VOr4fI4EvC5VcHl cp55ZWsc/Dl4iodJqUYgsQSQQkv8dSOBJ5zZ1Rdpmm27kgAbI4Mp8lkIcwjA/cmxtpErNL OMBtj1tmvWw8qmbPgPC+j7v69XYeaOI= Received: from mail.blacknight.com (pemlinmail01.blacknight.ie [81.17.254.10]) by outbound-smtp36.blacknight.com (Postfix) with ESMTPS id 8843A20A4 for ; Fri, 14 Apr 2023 15:14:31 +0100 (IST) Received: (qmail 24320 invoked from network); 14 Apr 2023 14:14:31 -0000 Received: from unknown (HELO techsingularity.net) (mgorman@techsingularity.net@[84.203.21.103]) by 81.17.254.9 with ESMTPSA (AES256-SHA encrypted, authenticated); 14 Apr 2023 14:14:31 -0000 Date: Fri, 14 Apr 2023 15:14:29 +0100 From: Mel Gorman To: Andrew Morton Cc: Vlastimil Babka , Michal Hocko , Oscar Salvador , Yuanxi Liu , David Hildenbrand , Matthew Wilcox , Linux-MM , LKML Subject: [PATCH] mm: page_alloc: Skip regions with hugetlbfs pages when allocating 1G pages Message-ID: <20230414141429.pwgieuwluxwez3rj@techsingularity.net> MIME-Version: 1.0 Content-Disposition: inline X-Stat-Signature: pg7m5g6wqwyfwitibuz4u5gb8tbbkqka X-Rspam-User: X-Rspamd-Queue-Id: 30B3514001C X-Rspamd-Server: rspam06 X-HE-Tag: 1681481672-41834 X-HE-Meta: U2FsdGVkX19ThsZHIV+UOMIUCoIMGsayNDokbBJMg5uVlNSOawmWdBGXizjHFcj1T5jHzUEVYDSWjKjt6hJZm1++RcEPRQyl+jJlFhk4Rbifodv5YxpmLpplT2p4U+MRE0JxgFUmmJHTyfmAV8/XXYqOSPAaUvm/3vav29ctJi5S/SkKI7VhYiP+TgsV8CkMJhR2vJZmNcOiNYFgty7eu/AUz9ecExQqRCX9/dxewBKb5K8EmN6/qbbJVn++bRJazF66f6Q2rRpGLqWGOC65Yp96Fns3rkyio2jqEeGf+Lwfg8TNQBgu/tMZWw8Ad9e2gkf+CujxCdojVt9wISHihvMvtKB8dgC0To3Ts+KIpFORnqIMODPgS3ccYWtwR2OHIHjMSKL2gj7NDSpQUs365FehPnK6qK7iSG0NV+NMwP1WNV/J9EKoUEOW4n/UfCmNh8xfJfv7ROP059vq7ODzt+ggWRy+GlXpggNXIdoHdRQRA5tyIeKFL3kbNyj1E7RSBVqVzdTRweOtK+uoHJq/kCixtyuisNqgoCuhepaXsnT0LPXPc+8RTb/tZUcdS8yGW8Fpm0dos6+40QL6Z4O17W7bVGNaG5U7X7XmMOc1ZqikiK8KUcyHgWHFcuMKYHQbjcG9sdbsiJ/Rv2pZUi2HVCApgCOP2Wjx9dMLjAtwk8VFicrawS1+ubxy2D9ymnoQlZMa3o05/2sVdj8dMyhA8F6zFlPfTooDu3QtPcxursBU3NM9+Vp+yl0XDqs+s0FRfQd+q0NNHLaVSGF8msTjcKecYEUnbwM/rvYb1cJBIRPzCOu7VksuEiedpqN0UYKoQG5TIpXtprZEFUEIp3v4vhC9jX3GztXgBRDCmlg15bhJ6u1fWEPo3M8KxhPS4krQRHuk/8+gVP026zKozsjixVSsMz1EnTsjMHdqy/jreo+4reGqOMkSjlFsnLaaIgB94YTMFkml0GRZYgAs/9w FtA5rK++ DN/XJY4myYIraj0At8hZLk7KrSfyGpZrQNA43oK71zjusxjkOG1PE0V6hQdudEdujWg3pS2nZESXVyHcCgObeNe6aNt/Hfbbp/Q2VeUR9Tdzd861Jp/huNOJmjm+55LNnST3d1aNNe3xl+G4+BOTarbzb7VHcdV65LD6tGsQBHiVbqSwoy+itORjaFHURZSOFQLf3XCw+Y+FBxOmnu99XyYmwmbCSduE0Lp5esTk+4wPV+eNTtYeJEmSpvgG3+3YqqMUgJ2jIFEvdFrDwRgOCQWxzqA4iLzbxAoJdVk6Tk91r9hPbU9d4ex95jsNGL4r4f7admaomhDS2JEFx49bY+UgUCdmzLguWNcizso5u1Ov8mzL9fLTUVWslRELAo7yDJnwK2Y6aKfyVcIjS+GyCayjKv67yaSavtcZG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A bug was reported by Yuanxi Liu where allocating 1G pages at runtime is taking an excessive amount of time for large amounts of memory. Further testing allocating huge pages that the cost is linear i.e. if allocating 1G pages in batches of 10 then the time to allocate nr_hugepages from 10->20->30->etc increases linearly even though 10 pages are allocated at each step. Profiles indicated that much of the time is spent checking the validity within already existing huge pages and then attempting a migration that fails after isolating the range, draining pages and a whole lot of other useless work. Commit eb14d4eefdc4 ("mm,page_alloc: drop unnecessary checks from pfn_range_valid_contig") removed two checks, one which ignored huge pages for contiguous allocations as huge pages can sometimes migrate. While there may be value on migrating a 2M page to satisfy a 1G allocation, it's potentially expensive if the 1G allocation fails and it's pointless to try moving a 1G page for a new 1G allocation or scan the tail pages for valid PFNs. Reintroduce the PageHuge check and assume any contiguous region with hugetlbfs pages is unsuitable for a new 1G allocation. The hpagealloc test allocates huge pages in batches and reports the average latency per page over time. This test happens just after boot when fragmentation is not an issue. Units are in milliseconds. hpagealloc 6.3.0-rc6 6.3.0-rc6 6.3.0-rc6 vanilla hugeallocrevert-v1r1 hugeallocsimple-v1r2 Min Latency 26.42 ( 0.00%) 5.07 ( 80.82%) 18.94 ( 28.30%) 1st-qrtle Latency 356.61 ( 0.00%) 5.34 ( 98.50%) 19.85 ( 94.43%) 2nd-qrtle Latency 697.26 ( 0.00%) 5.47 ( 99.22%) 20.44 ( 97.07%) 3rd-qrtle Latency 972.94 ( 0.00%) 5.50 ( 99.43%) 20.81 ( 97.86%) Max-1 Latency 26.42 ( 0.00%) 5.07 ( 80.82%) 18.94 ( 28.30%) Max-5 Latency 82.14 ( 0.00%) 5.11 ( 93.78%) 19.31 ( 76.49%) Max-10 Latency 150.54 ( 0.00%) 5.20 ( 96.55%) 19.43 ( 87.09%) Max-90 Latency 1164.45 ( 0.00%) 5.53 ( 99.52%) 20.97 ( 98.20%) Max-95 Latency 1223.06 ( 0.00%) 5.55 ( 99.55%) 21.06 ( 98.28%) Max-99 Latency 1278.67 ( 0.00%) 5.57 ( 99.56%) 22.56 ( 98.24%) Max Latency 1310.90 ( 0.00%) 8.06 ( 99.39%) 26.62 ( 97.97%) Amean Latency 678.36 ( 0.00%) 5.44 * 99.20%* 20.44 * 96.99%* 6.3.0-rc6 6.3.0-rc6 6.3.0-rc6 vanilla revert-v1 hugeallocfix-v2 Duration User 0.28 0.27 0.30 Duration System 808.66 17.77 35.99 Duration Elapsed 830.87 18.08 36.33 The vanilla kernel is poor, taking up to 1.3 second to allocate a huge page and almost 10 minutes in total to run the test. Reverting the problematic commit reduces it to 8ms at worst and the patch takes 26ms. This patch fixes the main issue with skipping huge pages but leaves the page_count() out because a page with an elevated count potentially can migrate. BugLink: https://bugzilla.kernel.org/show_bug.cgi?id=217022 Fixes: eb14d4eefdc4 ("mm,page_alloc: drop unnecessary checks from pfn_range_valid_contig") Reported-by: Yuanxi Liu Signed-off-by: Mel Gorman Acked-by: Vlastimil Babka Reviewed-by: David Hildenbrand Acked-by: Michal Hocko Reviewed-by: Oscar Salvador --- mm/page_alloc.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 7136c36c5d01..b47f520c3051 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -9450,6 +9450,9 @@ static bool pfn_range_valid_contig(struct zone *z, unsigned long start_pfn, if (PageReserved(page)) return false; + + if (PageHuge(page)) + return false; } return true; }