From patchwork Wed Mar 20 18:02:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13598068 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 80875C54E58 for ; Wed, 20 Mar 2024 18:05:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D90F6B009C; Wed, 20 Mar 2024 14:05:06 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 65EAE6B009D; Wed, 20 Mar 2024 14:05:06 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4899E6B009E; Wed, 20 Mar 2024 14:05:06 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2C4DD6B009C for ; Wed, 20 Mar 2024 14:05:06 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id D7BA5160CCF for ; Wed, 20 Mar 2024 18:05:05 +0000 (UTC) X-FDA: 81918193770.23.C9A1D2C Received: from mail-qk1-f178.google.com (mail-qk1-f178.google.com [209.85.222.178]) by imf02.hostedemail.com (Postfix) with ESMTP id A7E388000A for ; Wed, 20 Mar 2024 18:05:03 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=RFoJQvs9; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf02.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.178 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710957903; a=rsa-sha256; cv=none; b=Yic2SJ7DYQcT82KHf2fw5omP5/alCMCNKQX69P0JfMFprrSk4fdxK1HCr+TGHdp/OM2iod wjootHXUHnZtVgtCltALyt+FzGnOz6jgk59nlMHO2xey/+DwnSZZDOz/42jkjRsu+HMEC8 Dfrd2DCjUoFskpWy+Zrza8/IswOuqdc= ARC-Authentication-Results: i=1; imf02.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=RFoJQvs9; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf02.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.178 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710957903; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/agc7Iq+rpxvE93VG+YAHYepTCDVe1OJjvDT1Xeceos=; b=8dudNTd51ypoxaoUC4oDQr1GkQrpHPU9KGh2mxF9PoLrk7NeT0cQpYeIVb2BoJgyFO7fRi LDGbibnJ5A3zhOwKBkFmI2UMY1A26dal/e7iXHoiC0rI6rBHfxVSceuC+9qDdlgV2aAiWT Jbz+dW5Oie0d0P9hyX/JEGhKuXicYvo= Received: by mail-qk1-f178.google.com with SMTP id af79cd13be357-789e6e1d96eso10281385a.3 for ; Wed, 20 Mar 2024 11:05:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1710957903; x=1711562703; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/agc7Iq+rpxvE93VG+YAHYepTCDVe1OJjvDT1Xeceos=; b=RFoJQvs96cN8iGNFpu16jpiuSovmrMryRLJOigOAYa3HFib5Rl3IStk7i8xeuL3t2E ABWpDtrngIZ/9VlR7fyHFug9ei9k8EpZuA2aE9xMxHHcW8UrILqdRP1uSlZHaZp7oRq3 0hp3wBm/uf2xD2z0VBf/be/qVJD1xaiAn77a3Ibk63nc1kbcaBBDq/aD5s0xMyhuisyy Dy7RkrGmuli+zrmF9ZsCnqbDWQyFkhM4jEwGlgg0JPlyLgrltTmg8CP9BDcUC5WWWx/s aWnDzroF6TAodLFJj8eiSqGYWkbrVZuJnAXGTNEryEVXy15B9hz7x6c3S0M7Pw/qy2rT cOeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710957903; x=1711562703; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/agc7Iq+rpxvE93VG+YAHYepTCDVe1OJjvDT1Xeceos=; b=jPIGEj8wSTFiCRn1S4D/hMS6LcdVl+5Q5x1NbNyRw8WCG1KnecUnbCePcfnigEE1C3 LqsGd56slhQx7A2KiX7QHaNVtNfgEBb86D0tR5mzrqrlsVusxfRtWYgDY+126+wYYf2v dgd591lFMxzX+kvzU8YfWvET47NiDJ3hMM+Oisa9kSVsrtmJVgWdBA5AAwoUYp+lQEvK /52pgAKVXVFUGMy7PlNcwc4AM/GvAYHFAA90aie6p5A1Xh4523ZVhRaQM5NhVp9CMIBa qvxwRlghwNy7LMutm3PSPdvwvtvWLhrxI7KSmRNTws3HCXUGy5GvTkY66vfj9nBR6k/2 tyPw== X-Forwarded-Encrypted: i=1; AJvYcCXjJWy/Kv+AJCNkvbHjJzbRP96W529hIDgYB3xsBqGWJthh2FVLzOAPWEo1g16qEbAAVtXH4ON89/cdDP1FdiuZF/c= X-Gm-Message-State: AOJu0YzWugWsjVCegeiRE7jm40Ywtqgo18kT/+qMmCZRIenirc+0QH8z euizg+vbJdIXWODMGiUp2n4k9LN/rSJl0UQ3qofwRNQFrBx8ahBKeznWde8HIX8= X-Google-Smtp-Source: AGHT+IHVDZsbT9cqnC3HSTs2AyL3KxvP5tWB415HoSJ1bTxAaB/NmMu1L5tzkR6Fc/n91vn14sa1gw== X-Received: by 2002:a37:db11:0:b0:78a:86:534f with SMTP id e17-20020a37db11000000b0078a0086534fmr2673755qki.76.1710957902810; Wed, 20 Mar 2024 11:05:02 -0700 (PDT) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id w8-20020a05620a094800b0078a210372fasm900228qkw.86.2024.03.20.11.05.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Mar 2024 11:05:02 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Vlastimil Babka , Mel Gorman , Zi Yan , "Huang, Ying" , David Hildenbrand , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 08/10] mm: page_alloc: set migratetype inside move_freepages() Date: Wed, 20 Mar 2024 14:02:13 -0400 Message-ID: <20240320180429.678181-9-hannes@cmpxchg.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240320180429.678181-1-hannes@cmpxchg.org> References: <20240320180429.678181-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: A7E388000A X-Stat-Signature: ygagc1sfte3n9c4pkcbqiuwaupttyhhc X-HE-Tag: 1710957903-508470 X-HE-Meta: U2FsdGVkX18yMiUoHHI70q9owJE1bBA2WUYNLszssfKB3kN3geuYx0cdwzSwemUkEa7FpDT5HKWwqmT2QMyeXRuBgHwkh9mYJtrlxMJgItgNIuHJDy9fZRB3wQTulPpf+NPPLmvbus4gU7/063WHTqQ44qqh4p6LdhNYI5d5bn2reU0l8FXXny0dVlkS+mGInHHJHuqckXS5UM4UvZ+jcglQwPM/5Sr0l1ORE2Ar15bVPCLi+zqNo8470yDW/eGZHbPooC6I736Js7YXiK1ivQxG2MP+uY6xgrVyVnmo7C7HopBxwQyks50cm80Ri1HtV950DPNXjI6H/Cy+eVBkYEeXiI79VnQGbSjDRVEy6oMBStjAutsI9JcFj3mxdfFLFuacUfR0WStrh6WCS1sp9f2N2AWmPnkCvcmyAeEHW3c722ok5bX+4FoBi5fD0DPQ/NK91BBQUaSucxqgCZDex0oVmCF7KeTv0W6YLR4ldQsSwhP2yLmkIBiYYwFUAR9NesosiPmbCVP+dVP07zbGOHadkz6GPJwIk6j914W7rEaxpU79WU5iegXjaY4OipeFGJ2BNg7xPd2U9tL7v4YFoCn23zBjureUPMCzxJKrvluep1cd/xVosFsHbIS+XPglSlpA8NUwq/FLLgjYudWils492GrJVY3iGF3UYzklSEnwl89usW0X7BUG7BisXfxdjYJoeAy03BKCROXSyq++5X9DWejaBfDvxDeOIOPwxi5zWaRW+aaqpLvFGMuKeOmrs1j+rLdlpn3a8T5ZstfdkkyvinK+Fp2lbxYl12v3JI7IjqXkPL/oRPi96QM8SBrCc7ZqCxY/Lx2ifjvb1gAAMYzTpEv9o1js2acjNunZZ3G1wFB50RjGqk5V56JT3ZzsNz3R12nW9XfqRtzN2sH36RL9VLyTLykbXJpbVPeP+/C/UWBrvnlg/x+LomNQoDmtYG/aRGMM7VoijsuAUqR qQE1qOQB +vYbMmEwqPQ8XEhInX/B+xtfBpGkEKucwSHfer0UwlyFo6DoUZQFWCdHmy1p8eLod7WoRVl54jg9YNPv6kyaFxigZF3fd5/wkyuKez2fcpa1Ug3UgZjUoeCqas0p6ec6KebBCBzguTu8W3SUrtf4iDlgNFSyUFUOuHqRGsOFAozOwM/OnCehy+002Ozpd/6lEWTR3oJPlOkTzsh/W/LoHJVDZBN4l7w3Vy1bW6q97O/BBD5QcsInhuVgUVS7OtMAaP5YanKwtGVCE5KttrXNlCPZ2KTRo1YtnPeIerKwmpbnCRxQysVU6TKMIOQ2d/IdeXPYTLEVOhaOFEy+BEea+hQIAQvCtp5d70qAnZMc2M1gdB9NDQld3JCb1RCnbh/fgBIFR+MQloXuxkdSUYxPNYHRHYhC6lvNi/dAxCjqq949urHGVadO0zuqHbUpAxDF86RzVZX2VeKoUhtDo6K/Kf19ejqiNEIkVVT8shkrUsi3HPl0ARFs8dZr6ZM6XHVVKGg+kcC/Y8w3xdwU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Zi Yan This avoids changing migratetype after move_freepages() or move_freepages_block(), which is error prone. It also prepares for upcoming changes to fix move_freepages() not moving free pages partially in the range. Signed-off-by: Zi Yan Signed-off-by: Johannes Weiner Reviewed-by: Vlastimil Babka --- mm/page_alloc.c | 27 +++++++++++++-------------- mm/page_isolation.c | 7 +++---- 2 files changed, 16 insertions(+), 18 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 3f65b565eaad..d687f27d891f 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1581,9 +1581,8 @@ static inline struct page *__rmqueue_cma_fallback(struct zone *zone, #endif /* - * Move the free pages in a range to the freelist tail of the requested type. - * Note that start_page and end_pages are not aligned on a pageblock - * boundary. If alignment is required, use move_freepages_block() + * Change the type of a block and move all its free pages to that + * type's freelist. */ static int move_freepages(struct zone *zone, unsigned long start_pfn, unsigned long end_pfn, int migratetype) @@ -1593,6 +1592,9 @@ static int move_freepages(struct zone *zone, unsigned long start_pfn, unsigned int order; int pages_moved = 0; + VM_WARN_ON(start_pfn & (pageblock_nr_pages - 1)); + VM_WARN_ON(start_pfn + pageblock_nr_pages - 1 != end_pfn); + for (pfn = start_pfn; pfn <= end_pfn;) { page = pfn_to_page(pfn); if (!PageBuddy(page)) { @@ -1610,6 +1612,8 @@ static int move_freepages(struct zone *zone, unsigned long start_pfn, pages_moved += 1 << order; } + set_pageblock_migratetype(pfn_to_page(start_pfn), migratetype); + return pages_moved; } @@ -1837,7 +1841,6 @@ steal_suitable_fallback(struct zone *zone, struct page *page, if (free_pages + alike_pages >= (1 << (pageblock_order-1)) || page_group_by_mobility_disabled) { move_freepages(zone, start_pfn, end_pfn, start_type); - set_pageblock_migratetype(page, start_type); return __rmqueue_smallest(zone, order, start_type); } @@ -1911,12 +1914,10 @@ static void reserve_highatomic_pageblock(struct page *page, struct zone *zone) /* Yoink! */ mt = get_pageblock_migratetype(page); /* Only reserve normal pageblocks (i.e., they can merge with others) */ - if (migratetype_is_mergeable(mt)) { - if (move_freepages_block(zone, page, MIGRATE_HIGHATOMIC) != -1) { - set_pageblock_migratetype(page, MIGRATE_HIGHATOMIC); + if (migratetype_is_mergeable(mt)) + if (move_freepages_block(zone, page, + MIGRATE_HIGHATOMIC) != -1) zone->nr_reserved_highatomic += pageblock_nr_pages; - } - } out_unlock: spin_unlock_irqrestore(&zone->lock, flags); @@ -1995,7 +1996,6 @@ static bool unreserve_highatomic_pageblock(const struct alloc_context *ac, * not fail on zone boundaries. */ WARN_ON_ONCE(ret == -1); - set_pageblock_migratetype(page, ac->migratetype); if (ret > 0) { spin_unlock_irqrestore(&zone->lock, flags); return ret; @@ -2711,10 +2711,9 @@ int __isolate_free_page(struct page *page, unsigned int order) * Only change normal pageblocks (i.e., they can merge * with others) */ - if (migratetype_is_mergeable(mt) && - move_freepages_block(zone, page, - MIGRATE_MOVABLE) != -1) - set_pageblock_migratetype(page, MIGRATE_MOVABLE); + if (migratetype_is_mergeable(mt)) + move_freepages_block(zone, page, + MIGRATE_MOVABLE); } } diff --git a/mm/page_isolation.c b/mm/page_isolation.c index 71539d7b96cf..f84f0981b2df 100644 --- a/mm/page_isolation.c +++ b/mm/page_isolation.c @@ -188,7 +188,6 @@ static int set_migratetype_isolate(struct page *page, int migratetype, int isol_ return -EBUSY; } __mod_zone_freepage_state(zone, -nr_pages, mt); - set_pageblock_migratetype(page, MIGRATE_ISOLATE); zone->nr_isolate_pageblock++; spin_unlock_irqrestore(&zone->lock, flags); return 0; @@ -262,10 +261,10 @@ static void unset_migratetype_isolate(struct page *page, int migratetype) */ WARN_ON_ONCE(nr_pages == -1); __mod_zone_freepage_state(zone, nr_pages, migratetype); - } - set_pageblock_migratetype(page, migratetype); - if (isolated_page) + } else { + set_pageblock_migratetype(page, migratetype); __putback_isolated_page(page, order, migratetype); + } zone->nr_isolate_pageblock--; out: spin_unlock_irqrestore(&zone->lock, flags);