From patchwork Tue Apr 18 19:12:55 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13216096 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7C9FC77B78 for ; Tue, 18 Apr 2023 19:13:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1BC5C900007; Tue, 18 Apr 2023 15:13:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 16C1B900003; Tue, 18 Apr 2023 15:13:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F281B900007; Tue, 18 Apr 2023 15:13:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id D90F0900003 for ; Tue, 18 Apr 2023 15:13:31 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id A994F1A0443 for ; Tue, 18 Apr 2023 19:13:31 +0000 (UTC) X-FDA: 80695460622.16.F29141B Received: from mail-qv1-f46.google.com (mail-qv1-f46.google.com [209.85.219.46]) by imf10.hostedemail.com (Postfix) with ESMTP id AA70FC0028 for ; Tue, 18 Apr 2023 19:13:29 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=duezlBdf; spf=pass (imf10.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.46 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681845209; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+Fg7akH6DcyqQI9O6hhnfSSKo6TLLXotHGx/ELhTsVs=; b=3JJ8VKfOAabhfioHB6GoSODWHIo/WdFZLqVWXxoisat1oWAXxcUQLCCrl7cUgaPuKB1cDb zmoOaqZXeTmteY/bOK5TtTDCYBNB6TPNPPltzL1NiIryH45+5nwhbtRnPbVW36JoWwgQ9p D1N/hF40YTav+5TC9Ygzm2GL9tPGjX0= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=duezlBdf; spf=pass (imf10.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.46 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681845209; a=rsa-sha256; cv=none; b=LfI6Xl6Qe4dDpZaQVeNoJEbVgO3Rh2c2K4YjxdaYxzcUAXlJTw5yo/eAJ09NA7iaggPKpv pDCToDOmtsqX+0hVP2GDb2fEVqppqisgCqS+CKohdsFXvdB2EZmgQ3bbeiDi7YxDZdpLDA 2Pp35lTYyRt9WTa0k1iue+jYBakBJPI= Received: by mail-qv1-f46.google.com with SMTP id h14so18464044qvr.7 for ; Tue, 18 Apr 2023 12:13:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1681845208; x=1684437208; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=+Fg7akH6DcyqQI9O6hhnfSSKo6TLLXotHGx/ELhTsVs=; b=duezlBdfFiEByRb2UhLuZ0Hbbz17B9Qr/emODLH8mZ2CGMjptqHGa11W+QzdSjQ/7F NrpNC/GaqkJEOfmZT8IQhhmClT+39ZedrNCcu8ABafH+q5j+YOBKEgdjdxPGtGKwAbo+ P2TEzDioWeN451luG7HEJwPMt/rOzqsoER6GOlwhz2qDDjhKRGqWoYpeyDSp91a0bKPf 25r1qZ+NsVNcaUBJQLFKjSOaPj15VqfhoKvKXLa7AyFqKWsmjO1Nkx11EK8FkuJho1GV PlZj02YYGkSrKmCNJW7AgCuXuYglqYWYf5+G3gp8PSNESKrtKbSuyF4B4khcnFe0QCvs SdMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681845208; x=1684437208; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=+Fg7akH6DcyqQI9O6hhnfSSKo6TLLXotHGx/ELhTsVs=; b=EftrhfBEmnUb3+DfInkgFj8096rTWzVttHonaA6BIjNb7ywLfMa9OY82j7INp78LdI nse33bTzF5EaLeY54SL8UxMbm20XosM6iAHPfISbSPXaHWgZp5zjf/R2VjeLmxePHQtj QRnhg87p0cChWNhxOo7+oaBVIHjpyQpnh7YZelYi2/ArkOZuKM0GrtwREmv1olbvge6p WyRGEjGJzRDizwO1KTaNK63H6cQwJ7QsxqXNFg0FlkdTgE6Q+fboRqKv/tkV3NbvkcI0 u+eQ7DHM+Jx81saYSz8zTVqSFcPfqaMSA/dMnSUC7rVqLf/VbVEmBiH141R7jcbkBKlZ JXYg== X-Gm-Message-State: AAQBX9cEGN2vbUrnGJmYy/+iu2hk7/htSfpSxgtv3UhBtU+H1nLIOsle aW7uGOPuHuh9uxPMi1HANTyXX5nn0o56RXi59wc= X-Google-Smtp-Source: AKy350aBP+4ihxJVgdmom0s8xvFh5tp4l/UPrJZnbm1VerN0bSQuYJNYtHePgve1c8xb8nyNL5IrOA== X-Received: by 2002:a05:6214:252a:b0:5ef:8004:e0b4 with SMTP id gg10-20020a056214252a00b005ef8004e0b4mr11443725qvb.48.1681845208456; Tue, 18 Apr 2023 12:13:28 -0700 (PDT) Received: from localhost ([2620:10d:c091:400::5:e646]) by smtp.gmail.com with ESMTPSA id c2-20020a0ceb42000000b005dd8b9345e3sm3924742qvq.123.2023.04.18.12.13.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 12:13:28 -0700 (PDT) From: Johannes Weiner To: linux-mm@kvack.org Cc: Kaiyang Zhao , Mel Gorman , Vlastimil Babka , David Rientjes , linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [RFC PATCH 08/26] mm: page_alloc: claim blocks during compaction capturing Date: Tue, 18 Apr 2023 15:12:55 -0400 Message-Id: <20230418191313.268131-9-hannes@cmpxchg.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230418191313.268131-1-hannes@cmpxchg.org> References: <20230418191313.268131-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: AA70FC0028 X-Stat-Signature: mqmjgfa87crcbibxcgmx61azoww44gup X-Rspam-User: X-HE-Tag: 1681845209-747621 X-HE-Meta: U2FsdGVkX1+qQSVByXHwnGYQG9wCMqw7anTz14Bg8s+ekKzj077WLK6yAJgfwX3npwCWQnEeSrbAqO8QheZeA2tLFxVuTXmch0t4nqE9K3ev6gCJiLvI7NHYQZvolQFKbJYi4NSA2ADl1OAZU9/+uE1ndn3/UE2gusqronD3j+G3OaEl1H6MJ1yBOhkA63h8OUFClKVyPiYGT96ZUky4tYahB8LTjCq+mBPQ+2gvFc7YL5OxG5Or88CDZZ9OsGW90Ga59j/iayItqFfgEnfbqUHGTYyyEQs6XHX4AHjqU/zAlSPBEo6odRDFzULlguJJFfgJf88SBD/X5ScXE8JnGZpWS0AcIbFGde4ud1204Ls4tgVSW8wKfc9n+1bos9ktwyCelUggEF87NL9XKMpK2sLkzaqNKYGxwMQ4CbrQBhrUTq6W/K2cxbYAfhD7N0+SA9J0ftd3Vmt/gveYIEyZmlLgGe4tBS8zgaexocAde1QD6JoIF9VyPNU7PurOnLMUpa1sUtkrXtI0Y/46wg3qi4640LlIBAse7lVNbfgdeq+zObnKUkMONC053zCg3yW958Mw4mVI4w8Y0lwCr0sDo6b7SK5wiYvlzGgtw6QVz3ZxC5GqnUr6S5zih35/4NEvoyhaUzEZHOtKMuAPPYdVEunUQsLCjJDlJf5JurJpT7JAI+aU63FIo6KrVZz1jn2sVv9SDsf4xxQ7NxaBFCV7Pd3bhhGu0D0eITqUXxwV5Jhp4JTIT3CcE5JqoITsY62XHsn97d1DhPcMnIlVqz1HJ8DCNgH2pEOnK4ukRn6RBNJSH7ZWD2RPJOz1haVoQnCk4hinR5y6JJf0iqrJc3b6T70yL0Ah8Ar5pODwd6mlvkQ3l8orKAOGtxKwXXrHtccJJYKtS8Jogx5RUvSwL1I1ZbkfGqHIuwfoQdQwMSzTgfZCosZLyWwGdrvlAt4ciWc4745F4CAn31STcyxNu4k rVem4Doy Hmdtp/HRJaf8akmP4Xuhespg9oMjKVlfveUUXjxyTt5wJh8MlCFXjf9prCMSJa7X+/5HD1WqHvHG9Yzk2HuNDea1TfAGC1Mjp9ewbw38PnUQTqE0SY0FJfcjrCqB0jjgaMqi6XWJnVoUbHoJhfM8hnBox5xUH9G20MoYhd8ljB58hWJ3dr9U5r+ZgIavsDiXuARnWYNAvcIbAjJEiGHvZbhhrF7AhtlUZeglUL7K18qCgGvCkvtyvi4+rQlf28X5fzRY1KJt/YwlVCt9Q6j7mUL/KxVEBOkJMK89UtTpBYHpBHo24eG5+qJKe2pT29A+lmuTvpfPouET/+TXBv6oTKkt3/+VFnAoWgLDuhNGxUJJ4BN0qGDMgnpFncvwvVV4Ba/bv2Ewdevd7BzLFXMpjip6kxlVT+Ssq/ZqtGiePFT7xduzQfGBXbepO80hMef650/VmIpfeLm0H+B/RYCqJr1A2tf1Qt1W8MnxFLU+hQXNO9YzPAe9Nxx7cpqrXHUkZmPgUECmnmajzWdmuSDzmJF+vOQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When capturing a whole block, update the migratetype accordingly. For example, a THP allocation might capture an unmovable block. If the THP gets split and partially freed later, the remainder should group up with movable allocations. Signed-off-by: Johannes Weiner --- mm/internal.h | 1 + mm/page_alloc.c | 42 ++++++++++++++++++++++++------------------ 2 files changed, 25 insertions(+), 18 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index 024affd4e4b5..39f65a463631 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -432,6 +432,7 @@ struct compact_control { */ struct capture_control { struct compact_control *cc; + int migratetype; struct page *page; }; diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 4d20513c83be..8e5996f8b4b4 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -615,6 +615,17 @@ void set_pageblock_migratetype(struct page *page, int migratetype) page_to_pfn(page), MIGRATETYPE_MASK); } +static void change_pageblock_range(struct page *pageblock_page, + int start_order, int migratetype) +{ + int nr_pageblocks = 1 << (start_order - pageblock_order); + + while (nr_pageblocks--) { + set_pageblock_migratetype(pageblock_page, migratetype); + pageblock_page += pageblock_nr_pages; + } +} + #ifdef CONFIG_DEBUG_VM static int page_outside_zone_boundaries(struct zone *zone, struct page *page) { @@ -962,14 +973,19 @@ compaction_capture(struct capture_control *capc, struct page *page, is_migrate_isolate(migratetype)) return false; - /* - * Do not let lower order allocations pollute a movable pageblock. - * This might let an unmovable request use a reclaimable pageblock - * and vice-versa but no more than normal fallback logic which can - * have trouble finding a high-order free page. - */ - if (order < pageblock_order && migratetype == MIGRATE_MOVABLE) + if (order >= pageblock_order) { + migratetype = capc->migratetype; + change_pageblock_range(page, order, migratetype); + } else if (migratetype == MIGRATE_MOVABLE) { + /* + * Do not let lower order allocations pollute a + * movable pageblock. This might let an unmovable + * request use a reclaimable pageblock and vice-versa + * but no more than normal fallback logic which can + * have trouble finding a high-order free page. + */ return false; + } capc->page = page; return true; @@ -2674,17 +2690,6 @@ int move_freepages_block(struct zone *zone, struct page *page, old_mt, new_mt, num_movable); } -static void change_pageblock_range(struct page *pageblock_page, - int start_order, int migratetype) -{ - int nr_pageblocks = 1 << (start_order - pageblock_order); - - while (nr_pageblocks--) { - set_pageblock_migratetype(pageblock_page, migratetype); - pageblock_page += pageblock_nr_pages; - } -} - /* * When we are falling back to another migratetype during allocation, try to * steal extra free pages from the same pageblocks to satisfy further @@ -4481,6 +4486,7 @@ __alloc_pages_direct_compact(gfp_t gfp_mask, unsigned int order, unsigned long pflags; unsigned int noreclaim_flag; struct capture_control capc = { + .migratetype = ac->migratetype, .page = NULL, };