From patchwork Wed Mar 20 18:02:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13598063 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 349EBC6FD1F for ; Wed, 20 Mar 2024 18:05:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DD8BD6B008A; Wed, 20 Mar 2024 14:04:58 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D62386B008C; Wed, 20 Mar 2024 14:04:58 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B90136B0093; Wed, 20 Mar 2024 14:04:58 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9F2CD6B008A for ; Wed, 20 Mar 2024 14:04:58 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 4FC89120320 for ; Wed, 20 Mar 2024 18:04:58 +0000 (UTC) X-FDA: 81918193476.18.F7F1BE3 Received: from mail-ot1-f48.google.com (mail-ot1-f48.google.com [209.85.210.48]) by imf17.hostedemail.com (Postfix) with ESMTP id 638B640016 for ; Wed, 20 Mar 2024 18:04:56 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=Uowj1WeR; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf17.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.210.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710957896; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=T7r0X9mNHsEvKFNd3QjZ02+rYxlZ2ijHgMC19/7k7mQ=; b=rEQjsrQBW1RmZwmEyVaVK3/VoPDn1Iirf1NlF4pElX+2QXaTwefjvTQaULGdjNcdpZRb4b tx4xJpU3/WSjdYp4P5phrAYCNreJg0YRz0U7iSGvwpTYtlncsHe0ZTdhuFT6JpLkp/azeY mfyL0UKxBo2GP47p9hyI+ne5Yghj2Mo= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=Uowj1WeR; dmarc=pass (policy=none) header.from=cmpxchg.org; spf=pass (imf17.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.210.48 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710957896; a=rsa-sha256; cv=none; b=YDcJ4Q0fg4RdjwBSzA0Ty6av+uxgTMENYcim3AdXoULuRq7MTs9qxMPPeRI3IuGfC7wfuN kJMiu8Dqkugy/KW7Cy5U3YuEjzaWTcSwJAI9n8VqClVEsnv6Kv9FV4YhDsmGtQ+s7LPQrl gnt762IS7+Enh2UG0oGuAoU1eaE+ReE= Received: by mail-ot1-f48.google.com with SMTP id 46e09a7af769-6dc8b280155so49627a34.0 for ; Wed, 20 Mar 2024 11:04:56 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1710957895; x=1711562695; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=T7r0X9mNHsEvKFNd3QjZ02+rYxlZ2ijHgMC19/7k7mQ=; b=Uowj1WeR3XfZ0EIFx9/OTSpAnIg5vISCYyckl+5aW75IhIfIfU2yOQiTAztoAkuANH bsVS/74uX2Onjn4m1hMnAwAoasRDJ/z3IFUuuunMxICX4vfIphYLV9thSXQMNQbuqNVq WBnggfwVqLWHgGpfW5Qw0i0lgb+S7khpmGnT1GUCiLbIXm5FSPipJ1EvwvCo/6xMT44+ 6Pqia8jv50TP5xxjd6DeUIaEzTuM6xH3PEUeY4qucYRMVw2CAoGhhAVZZrfhAg+CoGHC 878CYNGMrlC3Eg64nQVYT6aazMShZ9zCiC2S/M63W0aRcASgxOHeoNND7RYTkoxvfYHP Q3ig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1710957895; x=1711562695; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=T7r0X9mNHsEvKFNd3QjZ02+rYxlZ2ijHgMC19/7k7mQ=; b=B6xbYgAMptSM5pLkae8l2jsHI8dNX0HhFSSikKNmTkiu+ntL3lJzXZu5ZnuJpkf/Mx nC7nQ9t/XqXmdrgeipIK8XxOc8uhxwoo5q5UvKnqPYv68OdeDUn40H4T6m5f2/BdR/Rn 3fkGUGC8bwUJtZFQ0Md7eHQ0eDc6AX7v3r+EzeEiBg7K8iCY4JLD82K27atcILqb7Kiv FE5iX4q30lq+1R2sUaSImKzCsyyGQNXTCP/Ln5NPu33wVpZS6xGG5tR17I3nTM1uT0H4 KRHv8dNxm6UBr0u14RcTtKUuUbHWQlaMzF0x6JRa1l3wXZeEUamPSDfgHGnQ64cAWNFd RkCw== X-Forwarded-Encrypted: i=1; AJvYcCXod7EEIVE63eRpvXdGRGLHFU7NKqmnkSfW2sCtxf4eJqIRLyRFhDd4fSAqf/EKn9HlGdvuXeh+Dztef5lho5Pd1o4= X-Gm-Message-State: AOJu0YxTdD36wy0Z9XNlxQXurneOJ/U7XYaBanY076LTcHMQNe7TQ9de mEi336YPfh/f9zLC8f93dgpb2WfWfpKlp+3Mn4elaLrjBKJcGR8lHw/h+ykOBVc= X-Google-Smtp-Source: AGHT+IEjrZQWghTRcTpqpNffkDCKSOVahuvg3qnwvR8ayQXYPdrYk9RXL0q0Eb9YG2QM0kUquwP9xQ== X-Received: by 2002:a05:6808:130d:b0:3c3:8339:6ef6 with SMTP id y13-20020a056808130d00b003c383396ef6mr3008346oiv.45.1710957895377; Wed, 20 Mar 2024 11:04:55 -0700 (PDT) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id fg14-20020a05622a580e00b00430bddc75a5sm5411969qtb.23.2024.03.20.11.04.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Mar 2024 11:04:55 -0700 (PDT) From: Johannes Weiner To: Andrew Morton Cc: Vlastimil Babka , Mel Gorman , Zi Yan , "Huang, Ying" , David Hildenbrand , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 03/10] mm: page_alloc: fix up block types when merging compatible blocks Date: Wed, 20 Mar 2024 14:02:08 -0400 Message-ID: <20240320180429.678181-4-hannes@cmpxchg.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240320180429.678181-1-hannes@cmpxchg.org> References: <20240320180429.678181-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 638B640016 X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: mataxcm9nc1thdesheo3qdib8jyqz7xz X-HE-Tag: 1710957896-322565 X-HE-Meta: U2FsdGVkX1/ORuFJiadvEpULZTk0yce3etOtIJvbI/ErRRI2JnS1JqbEdSDDTckvR7JZKYtZuMxfP4zhoT9W/rlXQT8q05BXLggLlq9W6F3tDLhNH5gdlxCqylqgTxsDWvpehqEutARRxgE5fgnfUR/1mjjSpBjDO0YmxcHNJZvRsXMZRT+kuWyBudbwevZR5bKOenVnuuR/8scnjdfpBFOo+fjjmivSbMk33GBj6EYX2LViyCqA5fH4kUUGclOgbpOwdmDSGTHrzghVhRYqy3znO5dEFs3wqju5W1Qj93+0oOaV0IohMg8UKGCm4lhtBZCojlSAmmiQJSsOG8EGQAyySUypd7a96OUKDC7IDU50wRPp/C89Ckp4GdN0A5hlXAx6HWJTWUt9+rNFphcat5Jq6Xz++zCX7nsFVn025vAHe+bBGnYRuXt+K33qMYAnsoAd9AfNHHYVSkH30FHpDKCPwDreMA8NmMvBoCk7Xo2B7P4/cuF5LEdP7jl0eFM6uxM4fQBi8nxeMlOrr9tghUkX+RoLDUcnTSpcj+5vTdQ5A8XROkfz6YeKMs2rrWUwcSgo1IVPDQBPWAnn4AUlSGOJFfRNwJy8t3VfNUIDvL6fij+Oe+eAmxiaqExycvM3A2FIaSPmHxIk+yULI3xV9/R2d5+PhBkC6WVJoWKVqjiW3b+NWMDdEOPiNmtJ1Y+VgqDwcxbnEIiOAltswmHXyT4C7Nk+29JCGs/qEtmWvgmyoTFzygD6PMw9pxrUtqoVSqG9tXGScDEC/efSqt7Kv33wAsUJQFYN1YM3Uo3TeehWdrI29Esb112vNrWLFyKQhIDbtP1ZBK61fX6MtT/WW3fuevqlUaASSrBCJpp9oULChuF7hly7zM5+b9+7CIwprV+D+qkiz3uxtZv8eetANxe7wuqq8PQAmco/mthUVcRfAo1pifQ4kcZGVWacagr8fPP7ZNUheAxVts5zIGc yAfyIl8W nKvrf2MYvIWcDPcY6Yg9NyEbsWbTChxAtfLrkoi8iOpYW69+blKWCVDkmXm1sx0x80jmUQFoemzr/+orKjnEy+68JP7RPw1p+ICgBwUqfTrUTCD+FN27NwlHtU8A1c55/Ge+W7ESEKDKMP6LCRkc1bBv1L8owIXUUxVKcL4JOQ3EGbjpHnVxSEfEo92GrlUgwK840Cs4ATEBNXOKbLD7sl+9jBxhUjD83PmxlzfQi6QYOu2uaDdWOqXYVHKF4VUbKio8iXJnmkI/LC1+LyC/4fkP556WB+8UJvDRJmDuyANTCRxQe5gPGoPDJG7MysosLwIDp9ndUT4jF/5Up8ZHBU6cs55SxHmCBi/hr0pP/YQSCVQZm4L6CRLzuK8MopWVjtoVM952kksOqMWGoUF8/4IDgGY5wsjTIWnMoLlMhqkqNv2bfovTMLlLerO6h7iUgdvmnWqiLc0cs3RRLnvZL+NEXeoIQmy0alj1tdRzvfT/PdGwmxsTTNjxLQBoKcSrmma2rKppVyP+28TzCopb2H0xSmZToIwjmQkbNpRyNlW6UWR5valnYLTm6oUM2wJC7nGxlWdcMkzqfSxpPa4F8v/2k2w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The buddy allocator coalesces compatible blocks during freeing, but it doesn't update the types of the subblocks to match. When an allocation later breaks the chunk down again, its pieces will be put on freelists of the wrong type. This encourages incompatible page mixing (ask for one type, get another), and thus long-term fragmentation. Update the subblocks when merging a larger chunk, such that a later expand() will maintain freelist type hygiene. v2: - remove spurious change_pageblock_range() move (Zi Yan) Reviewed-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: Mel Gorman Tested-by: "Huang, Ying" Signed-off-by: Johannes Weiner --- mm/page_alloc.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 994e4f790e92..4529893d9f04 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -785,10 +785,17 @@ static inline void __free_one_page(struct page *page, */ int buddy_mt = get_pfnblock_migratetype(buddy, buddy_pfn); - if (migratetype != buddy_mt - && (!migratetype_is_mergeable(migratetype) || - !migratetype_is_mergeable(buddy_mt))) - goto done_merging; + if (migratetype != buddy_mt) { + if (!migratetype_is_mergeable(migratetype) || + !migratetype_is_mergeable(buddy_mt)) + goto done_merging; + /* + * Match buddy type. This ensures that + * an expand() down the line puts the + * sub-blocks on the right freelists. + */ + set_pageblock_migratetype(buddy, migratetype); + } } /*