From patchwork Wed Mar 6 04:08:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13583365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CE5CDC5475B for ; Wed, 6 Mar 2024 04:16:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AA7E56B0098; Tue, 5 Mar 2024 23:16:34 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A5BAE6B0096; Tue, 5 Mar 2024 23:16:34 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AB756B0098; Tue, 5 Mar 2024 23:16:34 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 741036B0095 for ; Tue, 5 Mar 2024 23:16:34 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4B6A0120A59 for ; Wed, 6 Mar 2024 04:16:34 +0000 (UTC) X-FDA: 81865302708.21.1950B60 Received: from mail-qv1-f45.google.com (mail-qv1-f45.google.com [209.85.219.45]) by imf01.hostedemail.com (Postfix) with ESMTP id A3FD040005 for ; Wed, 6 Mar 2024 04:16:32 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=lHU5s+e8; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf01.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.45 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709698592; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=m/ZzL/TH/R4lBA3ZDd+dHH0WpYTGgWIE5ifE2nwqWl8=; b=OeqzewUpp1+n8szMFihrG7n534vqoPyHAJscCfeZ80sCwLTfbeFMDk8O8hZBjlvs7wD5Wu UUhDdU6FJQ627RTiXdcsG1PJwrCDJD/Wi9DSfGs20OYmm92Q2Z+XjEd3Et9RE//MH6Sgm6 AhoCNmTRSFdH8ISzE54/Ww+lpgBCSIg= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=lHU5s+e8; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf01.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.45 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709698592; a=rsa-sha256; cv=none; b=mWQ4lnp87ItjL1w/02uUaGv5S1qH7CB/PxP8qhXKDL359dyf/DSDzIiAD4m8uLhXSO/887 Ie/yIcbccuREdV0Mk+GOPKrSkfjMKf6/XMxC4kOEEpEAL29JEQE47pOMOG2ZQnMirvQNB1 odSPA9IZN2//Za/peaXWRgzunBqqQ+0= Received: by mail-qv1-f45.google.com with SMTP id 6a1803df08f44-6902947c507so8763046d6.3 for ; Tue, 05 Mar 2024 20:16:32 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1709698592; x=1710303392; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=m/ZzL/TH/R4lBA3ZDd+dHH0WpYTGgWIE5ifE2nwqWl8=; b=lHU5s+e8URHvQUxdPvjXJIEYrFoO4XLJyY7N3yu8V1jTk98j+rqKqZk/tKRbrF8OO2 wRZAu08vs4nCnGn7FJOninP4m6h9qUF5fR5FkpIrLCb/ErpXp2YxJBhM/8ofT75tC9Dz mqQEqnZjcr8Sqfk4So1Mjfacv8C0Utvww2XMFSBzQHIoCHQrzipzCUfEf7Ubft6bG5vc 0C+W+cULZtSbHoAz3UQLiu+DfgTwK9BleN39Rjjlo9XxElKkkU/NsirqjS0zOM/7xNoi WVRonbzkPnwdBdVpAEVXvoxrlg3w4RbHwJMlh0iFCtDtK/q/w4wQTLeJ+ZXEtdqoLWEr vLaw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709698592; x=1710303392; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=m/ZzL/TH/R4lBA3ZDd+dHH0WpYTGgWIE5ifE2nwqWl8=; b=HJzb18iP466SSvOA53XnmPXTvPBHygW9J5b4pvxOiJfQkUhVhvz+t+07GMogb7Banh 9DUY591/l9kENXRVqdnQ8fAAi1GiyN+254rfrYi1BGKk8Q+1sq6eYoV09T9VWb74j0lu 12yyYDG6IRUXNUSBZFFGdtNZSpck5Ggv3POaEe4j3Lv3liF9Gg5Zq+bLJ0+BKiHabtJ7 vwu22W0lFd3HgcBGEU/0igD+ZamDkd14GtB9UrHFK1Gp0EJTIlB5uSTbW4mSycicDfeY hvSHhrOzTFCOomBrYCkuH0hoDC/bE2Vj0rY7OEXcHrV0vtHurBnyUrls7/SjMZQzTPe0 XHyQ== X-Forwarded-Encrypted: i=1; AJvYcCWTQCgtuOEnNWwH66oOPI6R1fMPT/r+HYs0lW7cH+CDuVuZmdneg4CvOR5akCtNiLH5taih+dsOgAkcKh5aSfRXSXM= X-Gm-Message-State: AOJu0YxksOK09RXAMjcYAIdr3RPpG4W8bHWz3weaWTXHzoUAh5nhF5Ls W+qJ/Gi5aF2SOZDCWqymSUqBcputWBks1/dfnYicY/eS8LdU1efgGNZBg441gjY= X-Google-Smtp-Source: AGHT+IEhLmA5Z/jbA3zn2+GD6DrmyUDwK6kcHB1ddo4FwuhIkm61lmFRuGfA57LnzXsN6BEZH/RY/w== X-Received: by 2002:ad4:4d06:0:b0:690:6ce7:432e with SMTP id l6-20020ad44d06000000b006906ce7432emr4033712qvl.9.1709698591864; Tue, 05 Mar 2024 20:16:31 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id pd6-20020a056214490600b006907801a000sm2714835qvb.26.2024.03.05.20.16.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Mar 2024 20:16:31 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Vlastimil Babka , Mel Gorman , Zi Yan , Mike Kravetz , "Huang, Ying" , David Hildenbrand , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 08/10] mm: page_alloc: set migratetype inside move_freepages() Date: Tue, 5 Mar 2024 23:08:39 -0500 Message-ID: <20240306041526.892167-9-hannes@cmpxchg.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240306041526.892167-1-hannes@cmpxchg.org> References: <20240306041526.892167-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: jogkpy9fqynjxnird799afnmbbrm6j8a X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: A3FD040005 X-HE-Tag: 1709698592-504796 X-HE-Meta: U2FsdGVkX180dEwVFidXYVAsKTugm8rXV9Fjdpxng9zQIqqlEiKqIkmMdxw7XCzaeSAekSonsb7tTVaA5f9KSxRxebacCHu9IDoKoRCE5cqdGFXnhqxELHkHPt3RhCzmZwHpxjFY3gxfM7gHJ0v1zC+mN4gY7gKhVxIVCuULuWryi5iE9p2YbSV1ZySeaPejcf/2aDcHpGcS3NnXLrJIC2UrN095GfWajOf8xGJybugpcUCkvDkf6b6IHvrjB/6hHkMdrdeVNbVChzGGqCxxzWKBnccY8rEBlIj9wZYdjnOSMFmbZjye1YMXz4ZNFiM5Up5m5fgm+ypAo/rZdJ0b4aY9Iwg+oanlrzAEKlevc5ejDLK4cwcop4mzfl5Ky+bTmAFKf5l228P98Uv2KhaFzISBqXogWMaylX61dZIilurItxSDG/+m2FK2v2ns++S8HxmkMhMR5/xtDlI1iX0AmGOJurkvjXxyseLf9IkUKI4wgq2B6Vsc01WiFbKpRbdOUu5+UQv6QesXZH+HoBuTrMDMLAH8XKc/BaHTTh5n7cwhIwuPlkvVGVA+fcZs2m2quLwZoYgeRSemXVRJUDRmQ8ZvarDWtXZT9zeXtO33YKPwvbrivRWS3CSgm8t3mLNVu6H2IECpkqzFVD860CaLTk9krzOD+s+Dw4Bp2o27I+klA4Besq+uN2QfZn/IWwV1BdV73JDc0Iy0AwSPOLssVPgU0AGqfwRTcpnJOioH5ovZ2VOD/boqPk8vuW1ds44SGptIGEhSPmcw7MhiFyeXRjHaICsU5DCe/wX/OaE7wTQxhlkmm5jD6rHbwf//9y2FgN/amArc0BAnUEaSnv1BnVX5e2zNrHYSfW5ySQZP+Et6lfpARp/fCBj+7AL/hu03RbAZMUsiGyxFiHQ++9CVkkBz4GfetJe6BKDe+t3KiZXYCnfz789o6M8+7nENzI/H++v7YA1pfniCS93nvDX fJDgGqV1 OOqmI1W808Oe0Wumjfvo0z49V3BznP2U2mSInHUQGSn0IiY7KFuap5Beno6Y9r9pCAaW3ci9FDKeqBPKj56VxcEeuelDjv5y+Kf6hxIDcfgqmfXvqlRdXp4JofV6VQuIwTJco4QbMAMWNA6BQK+JiDXjzmx1zKh+Q/cUkmiXtSUqlvd/NnXUM34lD7YobpAqvQg6rXVzdFRuFeus7szezRvNpSUkXUtwJ065O X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Zi Yan This avoids changing migratetype after move_freepages() or move_freepages_block(), which is error prone. It also prepares for upcoming changes to fix move_freepages() not moving free pages partially in the range. Signed-off-by: Zi Yan Signed-off-by: Johannes Weiner --- mm/page_alloc.c | 27 +++++++++++++-------------- mm/page_isolation.c | 7 +++---- 2 files changed, 16 insertions(+), 18 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 82e6c4068647..a057b82c4f1d 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1581,9 +1581,8 @@ static inline struct page *__rmqueue_cma_fallback(struct zone *zone, #endif /* - * Move the free pages in a range to the freelist tail of the requested type. - * Note that start_page and end_pages are not aligned on a pageblock - * boundary. If alignment is required, use move_freepages_block() + * Change the type of a block and move all its free pages to that + * type's freelist. */ static int move_freepages(struct zone *zone, unsigned long start_pfn, unsigned long end_pfn, int migratetype) @@ -1593,6 +1592,9 @@ static int move_freepages(struct zone *zone, unsigned long start_pfn, unsigned int order; int pages_moved = 0; + VM_WARN_ON(start_pfn & (pageblock_nr_pages - 1)); + VM_WARN_ON(start_pfn + pageblock_nr_pages - 1 != end_pfn); + for (pfn = start_pfn; pfn <= end_pfn;) { page = pfn_to_page(pfn); if (!PageBuddy(page)) { @@ -1610,6 +1612,8 @@ static int move_freepages(struct zone *zone, unsigned long start_pfn, pages_moved += 1 << order; } + set_pageblock_migratetype(pfn_to_page(start_pfn), migratetype); + return pages_moved; } @@ -1837,7 +1841,6 @@ steal_suitable_fallback(struct zone *zone, struct page *page, if (free_pages + alike_pages >= (1 << (pageblock_order-1)) || page_group_by_mobility_disabled) { move_freepages(zone, start_pfn, end_pfn, start_type); - set_pageblock_migratetype(page, start_type); return __rmqueue_smallest(zone, order, start_type); } @@ -1911,12 +1914,10 @@ static void reserve_highatomic_pageblock(struct page *page, struct zone *zone) /* Yoink! */ mt = get_pageblock_migratetype(page); /* Only reserve normal pageblocks (i.e., they can merge with others) */ - if (migratetype_is_mergeable(mt)) { - if (move_freepages_block(zone, page, MIGRATE_HIGHATOMIC) != -1) { - set_pageblock_migratetype(page, MIGRATE_HIGHATOMIC); + if (migratetype_is_mergeable(mt)) + if (move_freepages_block(zone, page, + MIGRATE_HIGHATOMIC) != -1) zone->nr_reserved_highatomic += pageblock_nr_pages; - } - } out_unlock: spin_unlock_irqrestore(&zone->lock, flags); @@ -1995,7 +1996,6 @@ static bool unreserve_highatomic_pageblock(const struct alloc_context *ac, * not fail on zone boundaries. */ WARN_ON_ONCE(ret == -1); - set_pageblock_migratetype(page, ac->migratetype); if (ret > 0) { spin_unlock_irqrestore(&zone->lock, flags); return ret; @@ -2698,10 +2698,9 @@ int __isolate_free_page(struct page *page, unsigned int order) * Only change normal pageblocks (i.e., they can merge * with others) */ - if (migratetype_is_mergeable(mt) && - move_freepages_block(zone, page, - MIGRATE_MOVABLE) != -1) - set_pageblock_migratetype(page, MIGRATE_MOVABLE); + if (migratetype_is_mergeable(mt)) + move_freepages_block(zone, page, + MIGRATE_MOVABLE); } } diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 71539d7b96cf..f84f0981b2df 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -188,7 +188,6 @@ static int set_migratetype_isolate(struct page *page, int migratetype, int isol_ return -EBUSY; } __mod_zone_freepage_state(zone, -nr_pages, mt); - set_pageblock_migratetype(page, MIGRATE_ISOLATE); zone->nr_isolate_pageblock++; spin_unlock_irqrestore(&zone->lock, flags); return 0; @@ -262,10 +261,10 @@ static void unset_migratetype_isolate(struct page *page, int migratetype) */ WARN_ON_ONCE(nr_pages == -1); __mod_zone_freepage_state(zone, nr_pages, migratetype); - } - set_pageblock_migratetype(page, migratetype); - if (isolated_page) + } else { + set_pageblock_migratetype(page, migratetype); __putback_isolated_page(page, order, migratetype); + } zone->nr_isolate_pageblock--; out: spin_unlock_irqrestore(&zone->lock, flags);