From patchwork Mon Sep 11 19:41:43 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13379607 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E68BEEE57DF for ; Mon, 11 Sep 2023 19:50:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 26C616B02DD; Mon, 11 Sep 2023 15:50:38 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 1F7106B02DE; Mon, 11 Sep 2023 15:50:38 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F3BE56B02DF; Mon, 11 Sep 2023 15:50:37 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id E60966B02DD for ; Mon, 11 Sep 2023 15:50:37 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B890BB3BA5 for ; Mon, 11 Sep 2023 19:50:37 +0000 (UTC) X-FDA: 81225358914.27.DA11875 Received: from mail-oo1-f48.google.com (mail-oo1-f48.google.com [209.85.161.48]) by imf08.hostedemail.com (Postfix) with ESMTP id EB602160018 for ; Mon, 11 Sep 2023 19:50:35 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=DquP0kvo; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf08.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.161.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694461836; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sIntV6SfyJxxBgJqngsAK3yqYWZ9fE7nMi1YvaBUo2Y=; b=lPVXS7aaWE1Ob5lNg4Vi9gR1PCitrw0Iwhcfq17+tzms5uxGZrN6+S2M11N9DjzagjmRHR rh3+Yjn6QoXo2vdIFDC80cCLWXSq3/AVubZ5MVVDOxd62epiLxIvoQHQrxcXim5stA794/ txC2YKXKS5++j0dcuc6wlyWVha6g4TY= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=DquP0kvo; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf08.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.161.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1694461836; a=rsa-sha256; cv=none; b=As3JxekurEjgOo5iu5K4RQ6FkBokSh4/y4huigMZmgn4AyGdLd22kRBW5m+Ka9ICXcvO33 XXsMAllKLhr5KV316RhcDBs4v2wCP33aTVz1Y5YiQv2CY6V6CBzNvoTbHpVDKhdSbuf3Oz as+eqWWgsfggA2588LT+ZgmmCBatJ7E= Received: by mail-oo1-f48.google.com with SMTP id 006d021491bc7-5733bcf6eb6so2713619eaf.0 for ; Mon, 11 Sep 2023 12:50:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1694461835; x=1695066635; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=sIntV6SfyJxxBgJqngsAK3yqYWZ9fE7nMi1YvaBUo2Y=; b=DquP0kvokh2rP06FkTSMOlfmpMAm58upc8aK1izs2JOxsWTALmzssSm4iiF6pVhaq8 bABGZy1LNRS4fMDSuETB7Jsp3Kka/q4eJ2MkoNXFUlvQYwK8haZjBKbPspsEmS7pEVhv lUkX/wIyUf0yPXZAlCNWthillYrShDoU1hmN4J+ZKFEeFyarezOf4V8dnhbeiggq2fTs qq4uAJ044omxDmaNkoQr5AC3LAtJXmYJAMPWaK2T+xg9jmh5CeHhSv6wwjVG2yXJwzG7 PKdtSwoqZOMufq8a+u1pviIGV56p5+gaYRSGC+4nPoCiZz3JzBrhHcSt0tFo7rihQsWz RtCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1694461835; x=1695066635; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=sIntV6SfyJxxBgJqngsAK3yqYWZ9fE7nMi1YvaBUo2Y=; b=idmgUOzNbyjmHl+7Z+ziuxdKrVj4ZQVn9fyNPGY4bq+QShlRVRyz+Qe+xi4bqeqnGj L/+HIkPVSqCihXQKj881ew698VKzJs61MWTcBwNawJsR3wcQLjobfIVFyi7+rKHNKJd6 qYjGBIpsjpYVnZWw7rQUEitEX2NWly2+lCUaRjuekYHWgZeAL3cydPZGN7fF8J1tylnW ZUNPcXO7FrrdRV+1PMaZI3v1lXEfX/LLUuPKxCk/2uQzhZOhAuNxz7R03AysYlEsQdxH l+S2s8JjCszm9kaiUj7pTUZjyHG75ul3v6aROsn0WBsOlG8wnQiFj8f1pUzRf14hDST6 0lxQ== X-Gm-Message-State: AOJu0Yw65ffXi7g4xZa8ezdsF/jqjW0BTmWTaNRegtgbi9o9Ycqbnd0h xhQ//igdAN8pY4pcNW4FdEMKPA== X-Google-Smtp-Source: AGHT+IGS06A0iPrm3hhJt1coz/NmHegf9pi5FM+C3kEjocCFV6oSc5waC969xFTQcEM5YgwdIe4bDQ== X-Received: by 2002:a05:6358:6f92:b0:130:faea:a81f with SMTP id s18-20020a0563586f9200b00130faeaa81fmr5835243rwn.28.1694461834812; Mon, 11 Sep 2023 12:50:34 -0700 (PDT) Received: from localhost (2603-7000-0c01-2716-3012-16a2-6bc2-2937.res6.spectrum.com. [2603:7000:c01:2716:3012:16a2:6bc2:2937]) by smtp.gmail.com with ESMTPSA id j20-20020a0cf514000000b0064b502fdeecsm3129162qvm.68.2023.09.11.12.50.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 11 Sep 2023 12:50:34 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Vlastimil Babka , Mel Gorman , Miaohe Lin , Kefeng Wang , Zi Yan , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/6] mm: page_alloc: fix up block types when merging compatible blocks Date: Mon, 11 Sep 2023 15:41:43 -0400 Message-ID: <20230911195023.247694-3-hannes@cmpxchg.org> X-Mailer: git-send-email 2.42.0 In-Reply-To: <20230911195023.247694-1-hannes@cmpxchg.org> References: <20230911195023.247694-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: EB602160018 X-Stat-Signature: y7uodkayfqt917xy61u9ajdsa58rghuc X-Rspam-User: X-HE-Tag: 1694461835-419931 X-HE-Meta: U2FsdGVkX1/z05b6CLSZtxgs6FnaDBNZSl7NwA7jVh7wuQqIjOohCXP5b34/5Rihfv+BTBV1AZgKuqAs0K6mGGUfKvD2XFv1vZcckFyOi3wCRZRgS25GTU2isO98t6Q8WHJeT9qDCaKB5IilqgIGvHFoVnqcE9C/P9ialumAA5DzHKXOdJtOhTkRS0immfuWfC5QDuMhrpzIL0ReoCp17/xxLsFdRLLEeZamFLeSc9nkbCoecJ8dv1/1Up5fXVSW1EtG9UPEURyQx2/JyZTDqlZB3/3tjSAr01z54M2RP1SJAZxDLHDQn2pP4TB8+RbQhQkJrymI+r5cK/lldpvIZAyyPALHX5kraJJNk/MpGhoCt1Dd4bbKkQ5c7q/Vi+L2nWqi83mEaxAh/XCJzphYpmlV6cbd1rtsbagrK8v2o7oPpVoXCCaGpccNopwyAd72kpeAhPDvGa+MxHW+4BWhiQfKC50BnfYU0irREe1k/S6O5wbkWKPD6v6YyZR0e2QOV98EG5mwBEJnuDykW5NaxBfBAcKIFABSHDDF6e1x4+QUVQa1l7KwDPLuoZ8M5p64THeFG3ZypVtH0si2z2x1noNKQQUEW3N24n79d6VvkttkSjPgoU/1WOH2WYjn4eHLecrgB09S6SuYhUzssOluURxaHdBqJYoYFfH1Q8qppLexQeRRZ8PYO37HgT9uvDJqZPzKNeKZv8YMn7tK07p82wgCEhWwGi2vnl10RDa1/F6yuZLb6VwE5LIPDWr66aMOfxmYdWSq1rSB/9K6sKZVuZ1l8soNnyQ94WO8/Ia3HdhlrRUo5x9hYSox6Mxp/jl6dao9M1Njpi9uTZLhm9GtU1EgDoHWDDZvb8OlUZ4gNCJeUhhdA+O43I0cxhcAgXr53erJswFCc1U8VW5Tbkud4PoBWTaPcfcmrgCjhIbepEXlcszLynH2skoAEkt+XKDla8HRixj4FkNAMJwNmF2 2TKGjO67 PleQi8my7bAYWdp6DrWmdpvPlAE4mv5s8uijR9odDYG7cj3JVAgPzw4AS0JP11xK5lExuv1qBOuuRaTQNk6arssuYxIvscNm0r32Gx3gvIFBcCgINd72WpfGGuw4rQWGwI/MN0YzxNGvCgOnor1Obj7Nn0Y6FaaPRPL2QW/7KpMfb1PwpTpi/0eLL8ZBGecoVaY/wSYlFcJ1fDcxX7K4KHEFwTmfoDUdeT+1khO2QMEMXuyGuHUnfAbeU9HWfsA0APHcVzdp3I+p2TZngH7RMQWGq2XjuuWQ76cBlNjpBaQjlu5RLUUVhmrOqTzHa4LhUiglntp1Uy4LvkX8LMJkPEeBhFTLsPfdSWxQ4LRijwM/SoBYX4pB0psA36+XXSqIJpd8pbl8mhSzIi+OtHze3IasrsVidGi+Mn7P+mDqP4OwPfZQI0RB7bv1kmvieAu+zNOfEbq2GrWh5/IZ8HAsht+zG2KoE1mCcctBoXbmanN33IWk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The buddy allocator coalesces compatible blocks during freeing, but it doesn't update the types of the subblocks to match. When an allocation later breaks the chunk down again, its pieces will be put on freelists of the wrong type. This encourages incompatible page mixing (ask for one type, get another), and thus long-term fragmentation. Update the subblocks when merging a larger chunk, such that a later expand() will maintain freelist type hygiene. v2: - remove spurious change_pageblock_range() move (Zi Yan) Signed-off-by: Johannes Weiner Reviewed-by: Vlastimil Babka Acked-by: Mel Gorman --- mm/page_alloc.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e3f1c777feed..3db405414174 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -783,10 +783,17 @@ static inline void __free_one_page(struct page *page, */ int buddy_mt = get_pfnblock_migratetype(buddy, buddy_pfn); - if (migratetype != buddy_mt - && (!migratetype_is_mergeable(migratetype) || - !migratetype_is_mergeable(buddy_mt))) - goto done_merging; + if (migratetype != buddy_mt) { + if (!migratetype_is_mergeable(migratetype) || + !migratetype_is_mergeable(buddy_mt)) + goto done_merging; + /* + * Match buddy type. This ensures that + * an expand() down the line puts the + * sub-blocks on the right freelists. + */ + set_pageblock_migratetype(buddy, migratetype); + } } /*