From patchwork Sun Jul 7 09:49:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yafang Shao X-Patchwork-Id: 13725991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 01FF0C30653 for ; Sun, 7 Jul 2024 09:50:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 85D7B6B008C; Sun, 7 Jul 2024 05:50:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 80CC26B0092; Sun, 7 Jul 2024 05:50:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6AD986B0093; Sun, 7 Jul 2024 05:50:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 4AB216B008C for ; Sun, 7 Jul 2024 05:50:45 -0400 (EDT) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id B812AA3791 for ; Sun, 7 Jul 2024 09:50:44 +0000 (UTC) X-FDA: 82312487208.25.978ABF6 Received: from mail-oi1-f177.google.com (mail-oi1-f177.google.com [209.85.167.177]) by imf16.hostedemail.com (Postfix) with ESMTP id EF2F018000D for ; Sun, 7 Jul 2024 09:50:42 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nAbjWl9a; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1720345811; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8ypI2csRyBiyBJMCDlnFfjGoyjtJt9UTQgB5QzLtvc8=; b=pAMxky4zhoxoA1cebEkz/4rbH/7H0jpFTiijeQT8AQZ/H2thP6hwpjlnEP3ZRIZgpzoPsj NcUDmoGoxYCFj7v7P+PVVgvWjjAJ+pKwROS1yga+exdoqkjhbtb8QWu6Bm0s3bTHAbqfDt OgtV7Ks794A+W1pIL5NNSr97sv3H3sM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1720345811; a=rsa-sha256; cv=none; b=QyTEk7gJOAf7XiA2x57YboI80+BOuEpkIqLsnj9I3QfsKTxAMLv19GVnclst6CvcQkN7Wv J0CLivifhqswe3FDmiPfRDeW0Wgu+El6liTaqCHFmpeOmo1AsRq8rTrdfS7RiTE7i+uTjn /pQSTS3Xeif1SSPlsnjpgdNFXLPqTZU= ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=nAbjWl9a; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf16.hostedemail.com: domain of laoar.shao@gmail.com designates 209.85.167.177 as permitted sender) smtp.mailfrom=laoar.shao@gmail.com Received: by mail-oi1-f177.google.com with SMTP id 5614622812f47-3d91a2c95b2so1267872b6e.3 for ; Sun, 07 Jul 2024 02:50:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1720345842; x=1720950642; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=8ypI2csRyBiyBJMCDlnFfjGoyjtJt9UTQgB5QzLtvc8=; b=nAbjWl9aUb17UVohpbkMIpWzp1KxuJTIJKZsU9VDlis9JwdzsZLozFjMuCLXF71oX3 1ummTq61pNNzaGksjaxy5Y8sPo/zZ68IlBbhfobxEOgYIe/60OHRyIqDbtHwHJJcnieR CRmNxZkdv86HQrNK6APeY1oeqaGcN6H+0/p8mbPGMBsED+hDkOyYZhrA9Jzpq3pNzf2z 7B4W6bYjEPCMUMoXN8dHpui6lJdEDAjqGYjuyz22jWOrWMlVj5efYSrS+ad9viD1l+gi FuH/KQPIgRxO6OY+GA7WxEXWjhg9Kev7yR8rxPFD7b/rl0vLWKcotekUZHCqSeLE7pg5 w2EA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1720345842; x=1720950642; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=8ypI2csRyBiyBJMCDlnFfjGoyjtJt9UTQgB5QzLtvc8=; b=hFyTD29eGDblV1eM6oMdV9L9cCX4zvmz3Ps3Vd9RPyHO2Xg+WhxIPT5wNoadkkS9Nk L4fydGdwXWLfPwWgGHAgh5sSvNy1fogs6d5Rn3poY7/qNJ+HtcEPhImu9xjEVwLHdJOE lKLOsSI1W64MoaDbMyt7Y91ZKMacgrr28nweK/Zc1FUlJhB5q3Hp/LN/Ae0dH5yuK3Qv 7zgLoqPxx53QcXdaOkno9jWIVm8CO2otRqMUanjuWZL3bcbDYw/gpw9qhW432Rnhf6GH uburk+bAj/sM2wEVhz1PhgZcCoZHaTe19vNFcgwM9aEbqfXTjzzOYWYXVez35yb5FnDB Sq3g== X-Forwarded-Encrypted: i=1; AJvYcCXEd/4Fa41+t95r64P3UkkGpjVdAMZ11hIOEFZtxLDrjtm2cfjwoapFH2wAWnkqFuzuidr6V0rjpMi/BUVptZbixr8= X-Gm-Message-State: AOJu0Yw9Kxttzpx7Spgg9AC7rCF6fgyc6rqLyHyBLQPPx1u30rH0ApdJ zqSZvYkRPSIdndpMei+UQloew942qkohbZ9SuBBoR5IlTg+AiPHm X-Google-Smtp-Source: AGHT+IFscFmOzTVxQJ1Mzado8UnTN7rHKEWouraXKJGeb8dCg90qotgQ/t1CjGuIii0r2G41HYilZw== X-Received: by 2002:a05:6808:1646:b0:3d6:32a5:66f1 with SMTP id 5614622812f47-3d914eaa843mr11195044b6e.52.1720345840459; Sun, 07 Jul 2024 02:50:40 -0700 (PDT) Received: from localhost.localdomain ([39.144.43.178]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-70b1fdc335csm2184601b3a.22.2024.07.07.02.50.35 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Sun, 07 Jul 2024 02:50:39 -0700 (PDT) From: Yafang Shao To: akpm@linux-foundation.org Cc: ying.huang@intel.com, mgorman@techsingularity.net, linux-mm@kvack.org, Yafang Shao Subject: [PATCH 2/3] mm/page_alloc: Avoid changing pcp->high decaying when adjusting CONFIG_PCP_BATCH_SCALE_MAX Date: Sun, 7 Jul 2024 17:49:55 +0800 Message-Id: <20240707094956.94654-3-laoar.shao@gmail.com> X-Mailer: git-send-email 2.30.1 (Apple Git-130) In-Reply-To: <20240707094956.94654-1-laoar.shao@gmail.com> References: <20240707094956.94654-1-laoar.shao@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: EF2F018000D X-Stat-Signature: ky11osmhc586ai6qh6u3x38qrowfters X-Rspam-User: X-HE-Tag: 1720345842-22256 X-HE-Meta: U2FsdGVkX1/pqbSIum9+Vy6eAtIOcDnxFMbw5sUNDkGdyilrDOq1ui2Wxk9VsEqA0dHvktxnCfvrIpogFh6I2WA3hQ5xeMm4OHVk8IEYzoraCbPtQUhQ9Mb342++/wTQjLir1NuZYftWCW1wVgVLRwj0LvhxhaI3H7Dc2ycn+k14vlprB0GKYpemhI0lfmkYfwtRVGVl7NDFjkA4FlotyoL45UjSpSBYnKQMCnmaMERP5J0bdqJ1Hda4iU07sMqCkHhlXOfYIjsZfaplXAd/UEzaH9DdPp9LBYVpEE9puEZpsOCoAxbdzWEdiI9uedaQqOW5QhGYThGehSWMkCujynPqVJKeWhXz9VnUm7Zjo0+QeA2FE+lxYJ+32w+EGm7doi+rrcWO70nZjIfFjD596jO134Iahfyp9QV0Jhaknm2HnSDXSi058t9FQQ/HcKgeHvEUnSkLy9IdFvK7nFtJDf/FtQe48DTm90pCdKmVfizyAoHA03ZwEhv+9+A+X2JinXp94w3UAyQ8QMNBob1iYmsSCkylDe9+rrezsrNMsOJ0wS5Idlvh/yhIYjMI0nqmMR69KBwK9seHbSXxH78Qqf/TCZ7i81kzc4yF7QQAqbA+ksKDpQwNJkAVNJjr8RKK3mD9e00Z9A4gK6rnMInq7RFNpZX93xEAa0N5I5JzSmtcQo0dt7m/5e3o1wN8D78+lAXeKHt/B5nA/NO7XqNGkv/PPN06ja/gnRK9qZp7/bCGHFp9u1XGUvjq1EY36ztZc7FoU+/0Z2ZNmf7rEmtI59WkMLdEiIxipJZs/vVuRri5GhjpEVaa+g0wOC2K0Qkf0+KInyO0ASJA3sbv6oQhu7LlTw97OqC4508H9vUbzCgJQ0x7Edxgx5jmptubMR4KhR3H0plUPwwFMDZnIOzJG6zQeJIzDDN8uh98CZvqSshvlf5fVb4rx3NOd2neWInszOiItKReA+5XJVN6bT9 fpTc7MRV QPbRvn5DaNyQtGd//+bdn3uj/DbbkntvrjSBq4yorNpQ0QEMa9D05vSmJz0ugOdIuIng+Qa3w0r8IDg8T8IWB4b4Y0cTYHzJ7XJXS31YZDtloMlVkPmiluvOpgs6qSgc0LxA75uUg20Uh7NAKLQx58diNHQNXbPLxjR39ulAdIf6FwEUV2eSylbAMikKjjmlP9vZ6XweIca43FiOUV2UYDJimBrxZTIJdI0oCZmQ5IEMhKpAcI+zEFA8keHN/5T9Ol6ywDZhztEM4Il6nLrcftROyQeOivptW7zVah4iR4dq7hOcSQtoFolMG7S4DNlxt5/nJs/3j/PFAjDY93oWAwXk+ET3DUgFbVpxwrb5NmFRhmyMUf7L2ERltTl/82Ib367jO3S5ehl/jz5d5MXd7xLQfj7Retv4B4EiDyaaiQKCC3R6I2lqlsx/1BH4vLS1k7CGUyt+jGtlIqN0k8bL9oYJVX7RTGs8Vgf7mPxalT5jhgYqcgvlImROtjssVij3dWSYlDa42Ymzh1Ws= X-Bogosity: Ham, tests=bogofilter, spamicity=0.028773, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When adjusting the CONFIG_PCP_BATCH_SCALE_MAX configuration from its default value of 5 to a lower value, such as 0, it's important to ensure that the pcp->high decaying is not inadvertently slowed down. Similarly, when increasing CONFIG_PCP_BATCH_SCALE_MAX to a larger value, like 6, we must avoid inadvertently increasing the number of pages freed in free_pcppages_bulk() as a result of this change. So below improvements are made: - hardcode the default value of 5 to avoiding modifying the pcp->high - refactore free_pcppages_bulk() into multiple steps, with each step processing a fixed batch size of pages Suggested-by: "Huang, Ying" Signed-off-by: Yafang Shao --- mm/page_alloc.c | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 8e2f4e1ab4f2..2b76754a48e0 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -2247,7 +2247,7 @@ static int rmqueue_bulk(struct zone *zone, unsigned int order, int decay_pcp_high(struct zone *zone, struct per_cpu_pages *pcp) { int high_min, to_drain, batch; - int todo = 0; + int todo = 0, count = 0; high_min = READ_ONCE(pcp->high_min); batch = READ_ONCE(pcp->batch); @@ -2257,18 +2257,25 @@ int decay_pcp_high(struct zone *zone, struct per_cpu_pages *pcp) * control latency. This caps pcp->high decrement too. */ if (pcp->high > high_min) { - pcp->high = max3(pcp->count - (batch << CONFIG_PCP_BATCH_SCALE_MAX), + /* When tuning the pcp batch scale value, we want to ensure that + * the pcp->high decay rate is not slowed down. Therefore, we + * hard-code the historical default scale value of 5 here to + * prevent any unintended effects. + */ + pcp->high = max3(pcp->count - (batch << 5), pcp->high - (pcp->high >> 3), high_min); if (pcp->high > high_min) todo++; } to_drain = pcp->count - pcp->high; - if (to_drain > 0) { + while (count < to_drain) { spin_lock(&pcp->lock); - free_pcppages_bulk(zone, to_drain, pcp, 0); + free_pcppages_bulk(zone, batch, pcp, 0); spin_unlock(&pcp->lock); + count += batch; todo++; + cond_resched(); } return todo;