From patchwork Tue Apr 18 19:12:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Johannes Weiner X-Patchwork-Id: 13216098 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 474ABC77B78 for ; Tue, 18 Apr 2023 19:13:40 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE08F900003; Tue, 18 Apr 2023 15:13:33 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CB908280001; Tue, 18 Apr 2023 15:13:33 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B3158900009; Tue, 18 Apr 2023 15:13:33 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 9AD15900003 for ; Tue, 18 Apr 2023 15:13:33 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6B845AB76E for ; Tue, 18 Apr 2023 19:13:33 +0000 (UTC) X-FDA: 80695460706.27.73BC47A Received: from mail-qv1-f50.google.com (mail-qv1-f50.google.com [209.85.219.50]) by imf30.hostedemail.com (Postfix) with ESMTP id 9F5738000E for ; Tue, 18 Apr 2023 19:13:31 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=wn0fopvG; spf=pass (imf30.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.50 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681845211; a=rsa-sha256; cv=none; b=bw5mY9VnVmzTBWWO2ZzoYtk1rpP8crhWr4t4LwM/W0d7uFpx4F8p6rHSwdiD8nNKjPbFWX gkFZd91s97z7poJsozWMRwBeNfNCLZoHIJVwYkRKyDbF0KB8gp7Z9Z7poXVQCGaD33nXIq h1JTpK431fECFHkZpq5AhoR2Mcxxi50= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=cmpxchg-org.20221208.gappssmtp.com header.s=20221208 header.b=wn0fopvG; spf=pass (imf30.hostedemail.com: domain of hannes@cmpxchg.org designates 209.85.219.50 as permitted sender) smtp.mailfrom=hannes@cmpxchg.org; dmarc=pass (policy=none) header.from=cmpxchg.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681845211; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W40OmA9FJpZ9KM8b83NoKn4oLbbGXwdSocdEiJSdSOk=; b=RcCCF77TIflkhch4JxvqFMvE5XkD5iVauySX6kSmxFjrLl0JeMudMQv6njFkbKESXOJ8oV IBEImHhvPlhwtUcK8h8NUunK5jUuE7A531aDpdfJruW/Nchwe01sFJypXALMBm5inGJKyb p+VgzGDlF4jnL46EvjfI4lyZYlNqUOs= Received: by mail-qv1-f50.google.com with SMTP id qh25so19655948qvb.1 for ; Tue, 18 Apr 2023 12:13:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=cmpxchg-org.20221208.gappssmtp.com; s=20221208; t=1681845210; x=1684437210; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=W40OmA9FJpZ9KM8b83NoKn4oLbbGXwdSocdEiJSdSOk=; b=wn0fopvGsbR3i6PvjhCvPLQ1N5JWQrSjYBsgbrLuF6738z7mGxzLK6h6jbm6MmLsmJ pF8n/O/stCPiQEof58dQGBFCvd4zD48BHZayas7tdYm+iNpqPr5K40ekJd/L+fqSbnWx dF76tgKVdT587CaFRK8QZjlJ0JQitDXtdsiSxO7LH78byqCngXvJYUfGQXVnbLa3i4e/ Kwi9t5fN+cwPv3JGfkRVdEAeRey7ve3x28vGryxxL8HGqMYxVgzfDZIdBf6OLivjGG2u 1lZUigMFLGf6JwZyC57zeYywVprLLuC3qi6qYNpVO4TuaZDQqSJWnTgv3OsLRhreN2Jg PPfg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1681845210; x=1684437210; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=W40OmA9FJpZ9KM8b83NoKn4oLbbGXwdSocdEiJSdSOk=; b=AoAx6+Vfo5khwdXhZbLsEuZ08F933iOG4o8y+GSfKux52bN9FAZhZHwjcgpMqgxUHs SLVbnsjfrX0/Cq8R0OEN1GYcJIqBEyv3400fAJyvcnI3Qc/l7Oem68rz/UVrRzou33zm mL8lH3ZKJA/kdZWXYs99KMBuSjWg/3PUfyHwsSse3CsN0R0lXTzNs4oZ1ZUY9ryKvq8G wB+P1bPZIVbvqIrK2fthvxUMm2NNFVdKIi4YPZ9Z0975UD7Ig3UcK5jNh13KYWwNkhfk ndPPUmJczuhF3wRnxGl6mTB+5vh1mMD+wtryl7rknw4MYztG07fnp66zQKLsFwAy7dQK HzPw== X-Gm-Message-State: AAQBX9f1e2qIDWOb1Qv44685xkB5JqZdKGNSdavchsR+8Q4jrFO5ts0k RGUuvXvqEB4gixhPHCJz/j2A810LqanQpZBo4uA= X-Google-Smtp-Source: AKy350ZdXxpY1FsQYWCci5zoHDWqXfr/R4sK4OR7K8k8+oSMxZVw0C1MOwCqOi9gSXpDKZHblUbJiQ== X-Received: by 2002:ad4:4ea4:0:b0:5ea:6a2a:140a with SMTP id ed4-20020ad44ea4000000b005ea6a2a140amr26931362qvb.16.1681845210696; Tue, 18 Apr 2023 12:13:30 -0700 (PDT) Received: from localhost ([2620:10d:c091:400::5:e646]) by smtp.gmail.com with ESMTPSA id r1-20020ac87941000000b003df7d7bbc8csm4249673qtt.75.2023.04.18.12.13.30 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Apr 2023 12:13:30 -0700 (PDT) From: Johannes Weiner To: linux-mm@kvack.org Cc: Kaiyang Zhao , Mel Gorman , Vlastimil Babka , David Rientjes , linux-kernel@vger.kernel.org, kernel-team@fb.com Subject: [RFC PATCH 10/26] mm: page_alloc: allow compaction capturing from larger blocks Date: Tue, 18 Apr 2023 15:12:57 -0400 Message-Id: <20230418191313.268131-11-hannes@cmpxchg.org> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230418191313.268131-1-hannes@cmpxchg.org> References: <20230418191313.268131-1-hannes@cmpxchg.org> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 9F5738000E X-Rspamd-Server: rspam01 X-Stat-Signature: xzez1169ryk1cnh6rdqftecz14zxockq X-HE-Tag: 1681845211-684320 X-HE-Meta: U2FsdGVkX18p+t68xTs64ZL+rs2sj3uKsKCj7Uw8zJzxorDBkJ4VmRVjxVUMpQ810Rft2SwKH3138J140B0HcpQ3fBjWISZrmdDFDZl+7qLPMawuZyPIaKvdVo9df8U9IESXg3hm3qLiQJKNZF8J5PLXeqY3vokicI9vht2QebCKfFuDyRBgakGCLr42b3CFDzywg7w0Fee1l1gmKz/MYblR4WLiBS4sOJNuP5xQJ3Htk9u2nro4TlE149fg3de2nX7fh2uSRE3/bicbgKWQAlvTHyus5PMzC+/xfWClpXHVDdettpvulybSwAms+svgTmN9Tl0DXJSSJG/w1w9Qfyain4s/hILlLCjSu5Wh9wputAsPMJZ4qCGxpUYRbZ746ftG8k2NGYKmHJuLgE06nSX+CaXXilF7xgAysvgbJSbdvMZUybEosTcRuumSyreQYkNLauKdbR2EJYq1O2DLtKL8r/NhEqdBj3noZmmW32wrHtiEEAzgRoq+eZcN5VTpp9tju/aFq7gtJUmfBODbaCK9fgYTu22SxywHg5FLAYqgQSCDPh5WVcV2Kk+QZXa+rUzwI/DgcPI1p5vR7jVngzh3HCZfggRenhCkegoGSB5WZnfu3f1i5Gi6FCiQCDUKonAr8JJy2/61Fiiu2OO0PUq/UPVkFEotgmqWAgJJaYdoPk3nYB8cAbKNbhp6sICJGvwHcciARSsCKrVPr09Q4IfUN616FS0dT8hrNNd2ZYh25eq1dBxVYvUbnIdz0U3xapdVkAkFdYopNU6dS/7pXGEDGcFA3kCxm9U5ZB7Hf2MhkJgWOuzU9oDluhEX5sCG6mE982L8gsIrMPblQ4BvicEjnAxE+9J6EP5bQzTpWDhaf/MfR3MVTC2s/OlO7WLK4scpWHZHuq+EZUR53WrX8in1rVPfnudrd6Ie5cqnGRdH3qvOq60bBVhtPJ75MfJoDMjDfBiYN8iFG3Th+1B SkCAegTH CEQXHIVSKGqD+jommSxrz1190Dt2slSg/TxqA9DE24h5nnkOD0WgabqxbzPfGeV9PRsfqCcD8c3MF40hKpXiqKSbPLqLgTEZRko6cERpaCvovQ72khlSjKpxye2X5tzUI6ORfCDOzGoymWv0aUAWYc0qN4ByyGCnRzWgXkzCGy/uL+xz7INlj2Wi6+NGdUetpEJAQljwgxrVd8qePbW//jUGoeH8bmWg0JKJjpjQvy7ug4koBYtdQB6otCmNbCxK0CfhNEOnpIuFSRh84yd10fgRLnPRcDLOWKcPmsCBqBVQIsOk6ArLmiFF3jpQFedTnad9jRy9gmAHCjCVrNRYRCOLbDIpk/HaoHpflzGrevmDNO0E9ZbBAYjgSrs4SFes/hFtfyfv6CZiDS91v1WkZKMwR12aT1haI+XcHR9kDESxw9h6C32UDc/61pbRER7MJl5Dfs/2AUd54W8oneqirFx2IXQaSCQt+GQaZlL1g7TW8MWDnU6m9mBWZiGbNsvFObZKpLGuN+dtkKcR18D0RnSU06A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, capturing only works on matching orders and matching migratetypes. However, if capturing is initially skipped on the migratetype, it's possible that merging continues up to a full pageblock, in which case the migratetype is up for grabs again. Allow capturing to grab smaller chunks from claimed pageblocks, and expand the remainder of the block back onto the freelists. Signed-off-by: Johannes Weiner --- mm/page_alloc.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index cd86f80d7bbe..5ebfcf18537b 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -1067,10 +1067,10 @@ static inline struct capture_control *task_capc(struct zone *zone) } static inline bool -compaction_capture(struct capture_control *capc, struct page *page, - int order, int migratetype) +compaction_capture(struct zone *zone, struct page *page, int order, + int migratetype, struct capture_control *capc) { - if (!capc || order != capc->cc->order) + if (!capc || order < capc->cc->order) return false; /* Do not accidentally pollute CMA or isolated regions*/ @@ -1092,6 +1092,9 @@ compaction_capture(struct capture_control *capc, struct page *page, return false; } + if (order > capc->cc->order) + expand(zone, page, capc->cc->order, order, migratetype); + capc->page = page; return true; } @@ -1103,8 +1106,8 @@ static inline struct capture_control *task_capc(struct zone *zone) } static inline bool -compaction_capture(struct capture_control *capc, struct page *page, - int order, int migratetype) +compaction_capture(struct zone *zone, struct page *page, int order, + int migratetype, struct capture_control *capc) { return false; } @@ -1180,7 +1183,7 @@ static inline void __free_one_page(struct page *page, while (order < MAX_ORDER - 1) { int buddy_mt; - if (compaction_capture(capc, page, order, migratetype)) + if (compaction_capture(zone, page, order, migratetype, capc)) return; buddy = find_buddy_page_pfn(page, pfn, order, &buddy_pfn);