From patchwork Wed Mar 6 04:08:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13583360 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B325DC5475B for ; Wed, 6 Mar 2024 04:16:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C071E6B0082; Tue, 5 Mar 2024 23:16:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BBA646B0083; Tue, 5 Mar 2024 23:16:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A30856B0085; Tue, 5 Mar 2024 23:16:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 8F3DA6B0082 for ; Tue, 5 Mar 2024 23:16:26 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 5CA8EA0757 for ; Wed, 6 Mar 2024 04:16:26 +0000 (UTC) X-FDA: 81865302372.22.641325A Received: from mail-qk1-f173.google.com (mail-qk1-f173.google.com [209.85.222.173]) by imf30.hostedemail.com (Postfix) with ESMTP id 916DE8000B for ; Wed, 6 Mar 2024 04:16:24 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=Bzt0QzLr; spf=pass (imf30.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.173 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1709698584; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/m+XHRukptduyyFcyZXFpaIgdTql7GFreCc0MXKSlXc=; b=eY9ybyGcVSQIgmhEByoTPK3rhXH1KLE5RC9PJN1KsPnoVUVxKw//vlXtczITc+uQpPXeJo L0nhnbZ0NgPkZu+soLit7u2ajZqErx53ge9jlIUV8zMDxMl/ABaOo3Fln+Dc/FsRg0UTK7 57VbZIC6Gjtsf90pNaMd41lEBZfxwOE= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1709698584; a=rsa-sha256; cv=none; b=PWj/OylpmpQ8Xd/U9OX10Ow0J3lpKVNyL+wZmpUjazPUh57ri4MWL5Gd2X+XrEikZhPzLr 5fI8OHBdYV3Tl+pl04Ch+f22wQ9SuHQ+05gk7jp+pi0jp8+inFxHEpfnj6mKzmknEwGmJn w4mUZNTdo9AZGyPLWSHxipeAUYt8WpI= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=cmpxchg-org.20230601.gappssmtp.com header.s=20230601 header.b=Bzt0QzLr; spf=pass (imf30.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.222.173 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org Received: by mail-qk1-f173.google.com with SMTP id af79cd13be357-788237c4536so24002685a.2 for ; Tue, 05 Mar 2024 20:16:24 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20230601.gappssmtp.com; s=20230601; t=1709698584; x=1710303384; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=/m+XHRukptduyyFcyZXFpaIgdTql7GFreCc0MXKSlXc=; b=Bzt0QzLr88FMXNbrfHYFAvMyaCRklToxAqQGm507wyBS7mcWqduB8DRusLOGqdy4fC 6TcieHqn/LFuB5JCfKZeydJiib0DoVBbBTD+OgqehvYesBneIWv8VCUS3ZnG7ynkgw9S knFZBwf72xtYXvBE8z32P2xe4wrZ8X8ou/G7tC5yKOth9w59Wv+xqXHGXY66hcVv2749 74szddC9saeprc6/dqy1dt5i7C6Rh20gyoPtSH2Mk9Yah3zAvqVfjcKkKdnp0wc8hAwB RVyxN+11nnWFAe2ariR5hi2rq1bIjcXLUVQq+DQlWzRBJbLCJBiBqzdIxS4emz7Hwflo KrzQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1709698584; x=1710303384; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/m+XHRukptduyyFcyZXFpaIgdTql7GFreCc0MXKSlXc=; b=qJMJfj8t6ydmVCd+PJAvNCyrCiACwm7nmsr8PY6VgS864YBLhEhL22B1Cabgmr6pbN wmI4TSyXzqACQThkCimaUiwO9d/SYiFl+YsdwAm1lEkZUI6xJG1aHgyY8kuFzGcarq4j H0yVWCWSfFc+mg1uhZ9nR0dAx/QoU6X38RR9DwkwBwqNrM7svG0L2dMP20NBRvouuIOH mXWxmpjllpnWanPrzNiE5mdBNZzlhSkUH8OZ9aUgFf+BIXit3UhPqcSMC221mH9z5nLO J9SF1atMRc8WKVqj6jbGhDsR+zhl52kW5qqKjwY+QIuiHRtX6GsTmFiBnU/6tkYKbJjF S1pg== X-Forwarded-Encrypted: i=1; AJvYcCVcNhk1aSlUwJ+j1gHkKd+ku7188IB/timq2mnW59eJJyR1PsMwArRT19nVXGz77ISEq6tcTH4PLUBabp+Ei4vRlNE= X-Gm-Message-State: AOJu0YyOXUe4uAh0qkjhbbNcoIBKbXzuExkCHJZ4+3z0elRqmyq2dLP9 5nGIfISvvvO6Z6VZK0Upt6ke4IJi4nBQHdhbNGqTZ07BVuNxrBIXZ/RVP5/5uTs= X-Google-Smtp-Source: AGHT+IEXUAmf9zcOiQ21viaXa9n/2vz9wsETICwKDGogRNNcrHNmI77n49+NVvK4GXUrVLm+lFphsA== X-Received: by 2002:a05:620a:1272:b0:787:ed4f:a091 with SMTP id b18-20020a05620a127200b00787ed4fa091mr3769084qkl.72.1709698583788; Tue, 05 Mar 2024 20:16:23 -0800 (PST) Received: from localhost (2603-7000-0c01-2716-da5e-d3ff-fee7-26e7.res6.spectrum.com. [2603:7000:c01:2716:da5e:d3ff:fee7:26e7]) by smtp.gmail.com with ESMTPSA id e14-20020a05620a208e00b00788323a0c14sm1716137qka.39.2024.03.05.20.16.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 05 Mar 2024 20:16:22 -0800 (PST) From: Johannes Weiner To: Andrew Morton Cc: Vlastimil Babka , Mel Gorman , Zi Yan , Mike Kravetz , "Huang, Ying" , David Hildenbrand , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 03/10] mm: page_alloc: fix up block types when merging compatible blocks Date: Tue, 5 Mar 2024 23:08:34 -0500 Message-ID: <20240306041526.892167-4-hannes@cmpxchg.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240306041526.892167-1-hannes@cmpxchg.org> References: <20240306041526.892167-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 916DE8000B X-Rspam-User: X-Stat-Signature: mrgk44tyk5mm4ezf78nrcm1u1bzroeqm X-Rspamd-Server: rspam03 X-HE-Tag: 1709698584-589029 X-HE-Meta: U2FsdGVkX18Qc6Y1ehpQZNVOn4vnIaGWA0mtV362sABx+zydaBJo5d8MODmJ9aXX+4aThXF+594EZT6v1up1DgRwNj+W9k7qKWYYWOcBWKKQfdpIJKiyLLAk03nVBrycAgpazelnjOApuNv/7JD6OLHP8caQc1ugq7BgV5eLT6mHxq8DK8Y7xy2+NZPZrJ4HCO9cl9Tv9FUZo1q5RV2N6ShegRLeEtrOWOqArLbrsoXpjx5mlILbcz8nvk4O86suJevTnSnkuNOntrOowLUK3oxtiv2DpOaQ+NW19RnTvhAZ1Ol0kdNJz5sPLClVOwcIzLMc/YfSWEc7nAkpsUBWHruz5vIWsqtGrdx+ew5LhWng8ht9ZmxdDUEaWtK2wnJ8QH23Sh9VDKmKWhjerdh3GTCILevma2MtHluKkeNUiZ35nRTJXDuFf9/1zFD/Ur7RFbbtYvv7rfGJ22+HxG1xNnHZk5+ff16vwy13E6Wf9yFdy1Op/eQIQ9xrKO2iRxvDGFH5bQSZXRvj87dbEzsJWzXdOWYcGbiJACvW8Q0+7SiEv5yre7GnUuohUaFRoTFAcG32OqbhErWGe189+LJhuhiI/hYUITbW2wAIdz69Eymr9GFttmgFmzFfG7+c+8k8IXdi04ydXuNXqd8kwuyW69co1oPq7YZSa+n3nyYSIZHZwfgzgBDBQAJxL6Cs5hGY6o/43oroGj86zqvBZjtj3fUQXVEnivxGWMvIPFR9sCK2wiQZ74ez2XceI9ZU9tyllKn5RfOZxFosFbS7H/92m17NhsHcJt/BsM3ni55fVm1chCaoWvHUhApBoicNV41xtzJ1fRiuj8EWBMBtyA73WAc0pagbAAXc0AqkTeX4XJAjxb0vgBfhYtmxXVBhIH6+m2ZFI6sY2Gnvy3CPUk0sGbejzPCeCH/MVuakqysw+sstwqcOQBrX6QiSXYifGLJCFKpmZURMWQORGYauCJn KTjdStIh vGxPutqDCu/WM7T1ANgulDmGQbRz9DVRcHvbc2PSp1AM1GSXAOHu3FFnoQXhtlKeBRqI+OTm6ZOggfycQvl2to6sVQ10mBv3tGUYpFIwJJ2kIf6lLWZhB4YzpMB4oiBuLMuXSAIvgjlwfJzWsJqwClV5a+CN95V7RRy8A36f7wTLu8lrP5yn7niiWIE6EWQbNX/JKEtcQU7C1Avyn2TS00E9WE2TCP/Okbs6Cj2U9USFn1AkPxkZTd9GrWiENceDHT4s8kVs+4LrAx80AHkCWW8zpYPPJvVKXcDdP+Dmc6xKlOZSct2indTFpvmo4D7z3XhnRqMY1g5rB6LqkEPE7HfPeC/fYjBp5qIqE795q6E2L3RBp9s1MNFVDgCRVhOtxct4ZbFIrYVSFyLOdu10zOaZXXMb3KAE68HbCDV+cuhyN5eZ3hYXzFz9t1SRtrOvtiqt6COmP9cMSznBo2qZRWKpcEBYPAMb+5ALzyit+orYw+69B6Pa6cU66aLUl/1C3qvW2HpSYtp47U0ywBxgFdUeIRCYmFIVXp0SCzC4JoB0Q0SSma/EEFfgxjWjFr8ZdtuNhTeBfYFaJr6zoS9s23IkYcQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The buddy allocator coalesces compatible blocks during freeing, but it doesn't update the types of the subblocks to match. When an allocation later breaks the chunk down again, its pieces will be put on freelists of the wrong type. This encourages incompatible page mixing (ask for one type, get another), and thus long-term fragmentation. Update the subblocks when merging a larger chunk, such that a later expand() will maintain freelist type hygiene. v2: - remove spurious change_pageblock_range() move (Zi Yan) Reviewed-by: Zi Yan Reviewed-by: Vlastimil Babka Acked-by: Mel Gorman Tested-by: "Huang, Ying" Signed-off-by: Johannes Weiner --- mm/page_alloc.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 88f4e08726ee..3f9ea4c392d5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -785,10 +785,17 @@ static inline void __free_one_page(struct page *page, */ int buddy_mt = get_pfnblock_migratetype(buddy, buddy_pfn); - if (migratetype != buddy_mt - && (!migratetype_is_mergeable(migratetype) || - !migratetype_is_mergeable(buddy_mt))) - goto done_merging; + if (migratetype != buddy_mt) { + if (!migratetype_is_mergeable(migratetype) || + !migratetype_is_mergeable(buddy_mt)) + goto done_merging; + /* + * Match buddy type. This ensures that + * an expand() down the line puts the + * sub-blocks on the right freelists. + */ + set_pageblock_migratetype(buddy, migratetype); + } } /*