From patchwork Mon Sep 5 03:28:57 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mawupeng X-Patchwork-Id: 12965440 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C2088ECAAA1 for ; Mon, 5 Sep 2022 03:29:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 083E580199; Sun, 4 Sep 2022 23:29:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 032A58019D; Sun, 4 Sep 2022 23:29:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E153180199; Sun, 4 Sep 2022 23:29:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id CC9BD8D0031 for ; Sun, 4 Sep 2022 23:29:40 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 9960D120222 for ; Mon, 5 Sep 2022 03:29:40 +0000 (UTC) X-FDA: 79876602120.20.9F572F5 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf26.hostedemail.com (Postfix) with ESMTP id DB481140084 for ; Mon, 5 Sep 2022 03:29:39 +0000 (UTC) Received: from dggpemm500024.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4MLYnV6M6vz1N7v4; Mon, 5 Sep 2022 11:25:50 +0800 (CST) Received: from dggpemm500014.china.huawei.com (7.185.36.153) by dggpemm500024.china.huawei.com (7.185.36.203) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 5 Sep 2022 11:29:36 +0800 Received: from localhost.localdomain (10.175.112.125) by dggpemm500014.china.huawei.com (7.185.36.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 5 Sep 2022 11:29:35 +0800 From: Wupeng Ma To: , , , , CC: , , , , , , , , , , , , , , Subject: [PATCH -next v3 1/2] mm: Cap zone movable's min wmark to small value Date: Mon, 5 Sep 2022 11:28:57 +0800 Message-ID: <20220905032858.1462927-2-mawupeng1@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220905032858.1462927-1-mawupeng1@huawei.com> References: <20220905032858.1462927-1-mawupeng1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500014.china.huawei.com (7.185.36.153) X-CFilter-Loop: Reflected ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662348580; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=hNCwcaQOwL5QZ+wqWtn7myGnm/r9RkPUDiOcSigpq8o=; b=uByKukpU2pdmzNKtrkzaosA1JIWF+E44IRy0P9spPyy1MPYGgPvQjR1WU26zqDZvwQcZGc jolUFbU9CQN/M7Hi5nZIdtFGQ84wlx5gvBx8B3njAzeb43CUglYgmiDPPnK+jlrAEG4xef TG1pOmkvVpAmQ9Z6zlEakB+x9KT9u1I= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of mawupeng1@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=mawupeng1@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662348580; a=rsa-sha256; cv=none; b=SeQZu0nA4Ge6IeN6GtAPonaQxFzXTHefH6PVbXM0NsTc+zfdmY82Wqnn9QoQ3Xhz/Wdm1o wqSATlZkLiXQtg/0T+Gt9DJpuJVLwg+cYh9iVHkkP/07ZWerrj72YxUaV/g7GaDHbZ4qDJ LvQAMtiA4Rx4SvoLbSN37gvFEV99624= X-Rspam-User: X-Stat-Signature: 73mp6i1bj6zjbcbwoutkp5nxgobaai59 X-Rspamd-Queue-Id: DB481140084 Authentication-Results: imf26.hostedemail.com; dkim=none; spf=pass (imf26.hostedemail.com: domain of mawupeng1@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=mawupeng1@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspamd-Server: rspam04 X-HE-Tag: 1662348579-897412 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ma Wupeng Since min_free_kbytes is based on gfp_zone(GFP_USER) which does not include zone movable. However zone movable will get its min share in __setup_per_zone_wmarks() which does not make any sense. And like highmem pages, __GFP_HIGH and PF_MEMALLOC allocations usually don't need movable pages, so there is no need to assign min pages for zone movable. Let's cap pages_min for zone movable to a small value here just link highmem pages. Signed-off-by: Ma Wupeng --- mm/page_alloc.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index e5486d47406e..f1e4474879f1 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -8636,9 +8636,9 @@ static void __setup_per_zone_wmarks(void) struct zone *zone; unsigned long flags; - /* Calculate total number of !ZONE_HIGHMEM pages */ + /* Calculate total number of none highmem/movable pages */ for_each_zone(zone) { - if (!is_highmem(zone)) + if (!is_highmem(zone) && zone_idx(zone) != ZONE_MOVABLE) lowmem_pages += zone_managed_pages(zone); } @@ -8648,15 +8648,15 @@ static void __setup_per_zone_wmarks(void) spin_lock_irqsave(&zone->lock, flags); tmp = (u64)pages_min * zone_managed_pages(zone); do_div(tmp, lowmem_pages); - if (is_highmem(zone)) { + if (is_highmem(zone) || zone_idx(zone) == ZONE_MOVABLE) { /* * __GFP_HIGH and PF_MEMALLOC allocations usually don't - * need highmem pages, so cap pages_min to a small - * value here. + * need highmem/movable pages, so cap pages_min to a + * small value here. * * The WMARK_HIGH-WMARK_LOW and (WMARK_LOW-WMARK_MIN) * deltas control async page reclaim, and so should - * not be capped for highmem. + * not be capped for highmem/movable zone. */ unsigned long min_pages; From patchwork Mon Sep 5 03:28:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: mawupeng X-Patchwork-Id: 12965442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 04D0AECAAA1 for ; Mon, 5 Sep 2022 03:29:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 983AE8019D; Sun, 4 Sep 2022 23:29:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 933608D0050; Sun, 4 Sep 2022 23:29:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7FBC68019D; Sun, 4 Sep 2022 23:29:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 716EA8D0050 for ; Sun, 4 Sep 2022 23:29:50 -0400 (EDT) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 47C67A02D0 for ; Mon, 5 Sep 2022 03:29:50 +0000 (UTC) X-FDA: 79876602540.28.925A7C8 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf18.hostedemail.com (Postfix) with ESMTP id 47F211C005B for ; Mon, 5 Sep 2022 03:29:49 +0000 (UTC) Received: from dggpemm500021.china.huawei.com (unknown [172.30.72.57]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4MLYnh5lvFz1N7rn; Mon, 5 Sep 2022 11:26:00 +0800 (CST) Received: from dggpemm500014.china.huawei.com (7.185.36.153) by dggpemm500021.china.huawei.com (7.185.36.109) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 5 Sep 2022 11:29:37 +0800 Received: from localhost.localdomain (10.175.112.125) by dggpemm500014.china.huawei.com (7.185.36.153) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Mon, 5 Sep 2022 11:29:36 +0800 From: Wupeng Ma To: , , , , CC: , , , , , , , , , , , , , , , kernel test robot Subject: [PATCH -next v3 2/2] mm: sysctl: Introduce per zone watermark_scale_factor Date: Mon, 5 Sep 2022 11:28:58 +0800 Message-ID: <20220905032858.1462927-3-mawupeng1@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220905032858.1462927-1-mawupeng1@huawei.com> References: <20220905032858.1462927-1-mawupeng1@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggpemm500014.china.huawei.com (7.185.36.153) X-CFilter-Loop: Reflected ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of mawupeng1@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=mawupeng1@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1662348589; a=rsa-sha256; cv=none; b=XrefiszUZhEW7dTJEfEL1SuaQBSC2QstGnnExGzxk89SBq7QHwRQVEfWbC9nwXhB9npgvP M3ctNuYRcdge0g3lwg+nA3RaRSUIYEAoWtkZZfAg9TxiP41jfbaSzBJecDh2N5B9k0Tv5Z y+NJ0Ldi/Fl4u2RYd2KgahMOdFPmbq4= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1662348589; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=K70RHsYPk2qmSnjA8kuggAwzLpOH9/Mw8qmJ01/mmLI=; b=J+1gErQO7iMHgdqHm5pSL19c7urt3XCVexTtQC+5Pk7CAys8b9LAZR2j2eBiU6fMqJzt0k EEQaW5yJZZjoyK+UTDtshNaYt1XL/XDeePd9WBVwAFaIkQjxCEJgZyaNvpocHomtHbW+W+ Uiq425PzlqX8AuZXsIM7w9wUCgAXxwo= X-Rspamd-Server: rspam02 X-Rspam-User: Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of mawupeng1@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=mawupeng1@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Stat-Signature: tf983zbtqod48i7trk5x5ym1eskngr47 X-Rspamd-Queue-Id: 47F211C005B X-HE-Tag: 1662348589-555451 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Ma Wupeng System may have little normal zone memory and huge movable memory in the following situations: - for system with kernelcore=nn% or kernelcore=mirror, movable zone will be added and movable zone is bigger than normal zone in most cases. - system with movable nodes, they will have multiple numa nodes with only movable zone and movable zone will have plenty of memory. Since kernel/driver can only use memory from non-movable zone in most cases, normal zone need to increase its watermark to reserve more memory. However, current watermark_scale_factor is used to control all zones at once and can't be set separately. To reserve memory in non-movable zones, the watermark is increased in movable zones as well. Which will lead to inefficient kswapd. To solve this problem, per zone watermark is introduced to tune each zone's watermark separately. This can bring the following advantages: - each zone can set its own watermark which bring flexibility - lead to more efficient kswapd if this watermark is set fine Here is real watermark data in my qemu machine(with THP disabled). With watermark_scale_factor = 10, there is only 1440(772-68+807-71) pages(5.76M) reserved for a system with 96G of memory. However if the watermark is set to 100, the movable zone's watermark increased to 231908(93M), which is too much. This situation is even worse with 32G of normal zone memory and 1T of movable zone memory. Modified | Vanilla wm_factor = 10 | Vanilla wm_factor = 30 Node 0, zone DMA | Node 0, zone DMA | Node 0, zone DMA min 68 | min 68 | min 68 low 7113 | low 772 | low 7113 high **14158** | high **1476** | high **14158** Node 0, zone Normal | Node 0, zone Normal | Node 0, zone Normal min 71 | min 71 | min 71 low 7438 | low 807 | low 7438 high 14805 | high 1543 | high 14805 Node 0, zone Movable | Node 0, zone Movable | Node 0, zone Movable min 1455 | min 1455 | min 1455 low 16388 | low 16386 | low 150787 high **31321** | high **31317** | high **300119** Node 1, zone Movable | Node 1, zone Movable | Node 1, zone Movable min 804 | min 804 | min 804 low 9061 | low 9061 | low 83379 high **17318** | high **17318** | high **165954** With the modified per zone watermark_scale_factor, only dma/normal zone will increase its watermark via the following command which the huge movable zone stay the same. % echo 100 100 100 10 > /proc/sys/vm/watermark_scale_factor The reason to disable THP is khugepaged_min_free_kbytes_update() will update min watermark. Signed-off-by: Ma Wupeng Reported-by: kernel test robot --- Documentation/admin-guide/sysctl/vm.rst | 6 ++++ include/linux/mm.h | 2 +- kernel/sysctl.c | 2 -- mm/page_alloc.c | 37 ++++++++++++++++++++----- 4 files changed, 37 insertions(+), 10 deletions(-) diff --git a/Documentation/admin-guide/sysctl/vm.rst b/Documentation/admin-guide/sysctl/vm.rst index 9b833e439f09..ec240aa45322 100644 --- a/Documentation/admin-guide/sysctl/vm.rst +++ b/Documentation/admin-guide/sysctl/vm.rst @@ -1002,6 +1002,12 @@ that the number of free pages kswapd maintains for latency reasons is too small for the allocation bursts occurring in the system. This knob can then be used to tune kswapd aggressiveness accordingly. +The watermark_scale_factor is an array. You can set each zone's watermark +separately and can be seen by reading this file:: + + % cat /proc/sys/vm/watermark_scale_factor + 10 10 10 10 + zone_reclaim_mode ================= diff --git a/include/linux/mm.h b/include/linux/mm.h index 21f8b27bd9fd..b291c795f9db 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2532,7 +2532,7 @@ extern void setup_per_cpu_pageset(void); /* page_alloc.c */ extern int min_free_kbytes; extern int watermark_boost_factor; -extern int watermark_scale_factor; +extern int watermark_scale_factor[MAX_NR_ZONES]; extern bool arch_has_descending_max_zone_pfns(void); /* nommu.c */ diff --git a/kernel/sysctl.c b/kernel/sysctl.c index 205d605cacc5..d16d06c71e5a 100644 --- a/kernel/sysctl.c +++ b/kernel/sysctl.c @@ -2251,8 +2251,6 @@ static struct ctl_table vm_table[] = { .maxlen = sizeof(watermark_scale_factor), .mode = 0644, .proc_handler = watermark_scale_factor_sysctl_handler, - .extra1 = SYSCTL_ONE, - .extra2 = SYSCTL_THREE_THOUSAND, }, { .procname = "percpu_pagelist_high_fraction", diff --git a/mm/page_alloc.c b/mm/page_alloc.c index f1e4474879f1..7a6ac3b4ebb6 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -421,7 +421,6 @@ compound_page_dtor * const compound_page_dtors[NR_COMPOUND_DTORS] = { int min_free_kbytes = 1024; int user_min_free_kbytes = -1; int watermark_boost_factor __read_mostly = 15000; -int watermark_scale_factor = 10; static unsigned long nr_kernel_pages __initdata; static unsigned long nr_all_pages __initdata; @@ -449,6 +448,20 @@ EXPORT_SYMBOL(nr_online_nodes); int page_group_by_mobility_disabled __read_mostly; +int watermark_scale_factor[MAX_NR_ZONES] = { +#ifdef CONFIG_ZONE_DMA + [ZONE_DMA] = 10, +#endif +#ifdef CONFIG_ZONE_DMA32 + [ZONE_DMA32] = 10, +#endif + [ZONE_NORMAL] = 10, +#ifdef CONFIG_HIGHMEM + [ZONE_HIGHMEM] = 10, +#endif + [ZONE_MOVABLE] = 10, +}; + #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT /* * During boot we initialize deferred pages on-demand, as needed, but once @@ -8643,6 +8656,7 @@ static void __setup_per_zone_wmarks(void) } for_each_zone(zone) { + int zone_wm_factor; u64 tmp; spin_lock_irqsave(&zone->lock, flags); @@ -8676,9 +8690,10 @@ static void __setup_per_zone_wmarks(void) * scale factor in proportion to available memory, but * ensure a minimum size on small systems. */ + zone_wm_factor = watermark_scale_factor[zone_idx(zone)]; tmp = max_t(u64, tmp >> 2, - mult_frac(zone_managed_pages(zone), - watermark_scale_factor, 10000)); + mult_frac(zone_managed_pages(zone), zone_wm_factor, + 10000)); zone->watermark_boost = 0; zone->_watermark[WMARK_LOW] = min_wmark_pages(zone) + tmp; @@ -8798,11 +8813,19 @@ int min_free_kbytes_sysctl_handler(struct ctl_table *table, int write, int watermark_scale_factor_sysctl_handler(struct ctl_table *table, int write, void *buffer, size_t *length, loff_t *ppos) { - int rc; + int i; - rc = proc_dointvec_minmax(table, write, buffer, length, ppos); - if (rc) - return rc; + proc_dointvec_minmax(table, write, buffer, length, ppos); + + /* + * The unit is in fractions of 10,000. The default value of 10 + * means the distances between watermarks are 0.1% of the available + * memory in the node/system. The maximum value is 3000, or 30% of + * memory. + */ + for (i = 0; i < MAX_NR_ZONES; i++) + watermark_scale_factor[i] = + clamp(watermark_scale_factor[i], 1, 3000); if (write) setup_per_zone_wmarks();