From patchwork Wed Dec 6 16:28:56 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Schatzberg X-Patchwork-Id: 13482012 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E327AC10DC1 for ; Wed, 6 Dec 2023 16:29:36 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 65DDB6B0095; Wed, 6 Dec 2023 11:29:36 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 634716B0096; Wed, 6 Dec 2023 11:29:36 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4ADF56B0098; Wed, 6 Dec 2023 11:29:36 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 3B94E6B0095 for ; Wed, 6 Dec 2023 11:29:36 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0CF9E120167 for ; Wed, 6 Dec 2023 16:29:36 +0000 (UTC) X-FDA: 81536929152.03.1023D77 Received: from mail-qk1-f182.google.com (mail-qk1-f182.google.com [209.85.222.182]) by imf10.hostedemail.com (Postfix) with ESMTP id 2B068C002C for ; Wed, 6 Dec 2023 16:29:33 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SpEgd0tS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of schatzberg.dan@gmail.com designates 209.85.222.182 as permitted sender) smtp.mailfrom=schatzberg.dan@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701880174; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=xwZWXH9p+0yJMhMBcVvnwovBBLZBIMZ/gK454l8lM54=; b=RmrUL0OHcMHu+eJ/j/Wb89II2ZYlagegD8hzYhPFooNzUYAJykdl7PL+4HPeiNQ2ZaTVCy o6IFY4dQDM7QqBsZeDbpgF5AZu9RUmauonewvacenQ0cCGySu3ef2xf8GRkLBol1uC+BYn dOEElf7AKew0H0tWuUqAPPy3OGaWCCo= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=SpEgd0tS; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf10.hostedemail.com: domain of schatzberg.dan@gmail.com designates 209.85.222.182 as permitted sender) smtp.mailfrom=schatzberg.dan@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701880174; a=rsa-sha256; cv=none; b=AQTOBUGByUzzwW70Xf+zrAq0hJ13suaaJ7FG2upWnMi/y7QLNoq45MdNuJ/xEWyhvC3JiH rSdzULzzEoGdK3QeeJKrQrvSskTCwkxehC7wmW/4tIFW3RQoLsLjwzMY2G+tLt/IGi3GYV M5BwcBAx0EZ6P1GPqpZ6dArerl6wiB8= Received: by mail-qk1-f182.google.com with SMTP id af79cd13be357-77bcbc14899so465288085a.1 for ; Wed, 06 Dec 2023 08:29:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701880173; x=1702484973; darn=kvack.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=xwZWXH9p+0yJMhMBcVvnwovBBLZBIMZ/gK454l8lM54=; b=SpEgd0tSkScAHaQg4UY1ZBr8+XRwnXCxK+D/GUviamZqgXXymyy53+CbDM3BCa489c zZjS03AQLAbhpg8wbHco4yhgKuQMmL6wF9tZDJJUmhXPgv8u/d/cjWjJHc4QJx1GCoVB 1abTPckvOEl7IbY5otEYMWZTPgDtKHDx52iFuwEimaPE24KACkZob4jHtPnszDL854c+ GiSK7AH/EC9NTxaJU/pfuK4UMTrlIAjKUF4loJDuPpFsEA4UpNFP25HoMbA13cAxTtvu bt1Am6e/usJxHSGDPZYyaK5hyPCFbzxTz6uYTSa1h9o6q/iD9UpxQIHfeVjm5f84KMd3 c0wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701880173; x=1702484973; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=xwZWXH9p+0yJMhMBcVvnwovBBLZBIMZ/gK454l8lM54=; b=vzONTseSVjbf3KsBK/4FyD26tsOn2Q+UL9oFkSGHOuQEAZa7josYOBbFeUrTuTSWSd wz3FFj3jktgJDYOraM1cFVBpGSJNd751c2RNjd5ujA18CJsYFPYOrCKYijt5JZQurmen +XAX3o5Ld/7j2W4rLg9qXz5u8xvcL+pKQmXiZ9OMhGGX6gKX7gZaRAbFE92wS2mrxsmh XfROkzpXD1PPmRVD5gxdvb97IwaogjdPYYcUSaOHiA0yDBf8zQMrf9wMTBPAbKs04bfU Z3+XyQk2w/9gT/PAZv6JfmW4SpvqCCrkpJehOwuNKS7kp6m0WVaUGHSaSeZqOYPwz4Uh Z5LQ== X-Gm-Message-State: AOJu0YyzKN8riNdB3sAuw0NkkcyOHPva8/LwkdfFE9w4J3bF9QZo/GVv 6DFgRK7RX2vA+SVzvpZqyAc= X-Google-Smtp-Source: AGHT+IGx8a7EzNCQyfTSNswNgSliYrYf9H/wfyFTvaT8dR4sDl26fZNTIsdIibcVYXFPjl6D4wV8QA== X-Received: by 2002:a05:620a:66c:b0:77e:fba3:81e5 with SMTP id a12-20020a05620a066c00b0077efba381e5mr1148000qkh.123.1701880173098; Wed, 06 Dec 2023 08:29:33 -0800 (PST) Received: from localhost ([2620:10d:c091:500::6:c65f]) by smtp.gmail.com with ESMTPSA id br32-20020a05620a462000b0077dd520f844sm65807qkb.84.2023.12.06.08.29.32 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 06 Dec 2023 08:29:32 -0800 (PST) From: Dan Schatzberg To: Johannes Weiner , Roman Gushchin , Yosry Ahmed , Huan Yang Cc: linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Tejun Heo , Zefan Li , Jonathan Corbet , Michal Hocko , Shakeel Butt , Muchun Song , Andrew Morton , David Hildenbrand , Matthew Wilcox , Kefeng Wang , "Vishal Moola (Oracle)" , Yue Zhao , Hugh Dickins Subject: [PATCH 1/1] mm: add swapiness= arg to memory.reclaim Date: Wed, 6 Dec 2023 08:28:56 -0800 Message-Id: <20231206162900.1571025-2-schatzberg.dan@gmail.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231206162900.1571025-1-schatzberg.dan@gmail.com> References: <20231206162900.1571025-1-schatzberg.dan@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: 6u7t6n1k4aygx9yuuzbc9byakmy6uriy X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 2B068C002C X-HE-Tag: 1701880173-350308 X-HE-Meta: U2FsdGVkX1+rExvm+JoUNDSlSdDeituP31r8Gm27EUBBRyHXaln140aekMwvTbzeDT4X70Oyo3e26wpYYjz1fn6HCekTm8demawyF09Hf9CO55QCu9sB3dY+n3FTLeieS6tnqSAy9H0tbFwnQaaINFsqrAMD09bQq8NaXEGP+ms4io2WiPUNII3Nr613zlrCnCO6jDkHJFvVBwv2ax7Z4iUEe7I8vsWVVMaK01tQDLPZWFk495OMEsCEfg+nNUzFmSAORDR7+rCQtz42+KByna+gtUk0LENtVvtsx928axgolyJU77HjZMTvbMkKgWwIGf1C+ce7pnQgJ9CWtekWAeIaraHh5E1yU62Mja60z0gc4MA2Um94Lubcg6+nTq7GwbikvoYCMI17VM+1km0qTKw3X6RvJSL8cXAlTREyrL9BglafU3lidwe66AfGQIlyXBJVQYxJ2MHQKWVnCx/9DkccWeMk74ftpj/0QpLXc2Q+TLLe8bq9sRlgSGBDJkKegd8a3DsLJZenjXw5ykJKWLYOoKGGSxJ0rFzwgtQ5Y3KLox8cByif9G0EV17SmUDHfnd5qnSST9SYRvrIexWMqNR92MzjJz1ra35eXUk9rq60NGCSJf2EL9a3af/SCBdaffAIN1WVb2bZRLzg/yvQMnUqfevYZBmDwyBDlQdSxaVebFXuW826zC6kPFmMPFzXXB88PWdc803x/O9a88NrQEVlciXa7OwWigvj4pv38BkIRpg7PFpBAM4nqc52QGiODWApJzRbWDU403Z1iemjG13jHW7wXBJKRRcm5iTBdv7gd+xMXmMFlggZXOWhy69MvVVgpi/p8aCq4EE2mwddbt51xehlRkJoKc7VZWs5N7hStV04uyVgM7V4wV8hQHCLCecjFup+NpDYGYF4AK72oKL6lWHFxOliXscFpBly7EPoCbn9ci/dYU0A8P3bBBEPTtEeXzrkLHzTfBxApGC Em4w7m9Y 4VGa33zPUTw/dkErTQKQQNKeTYl4VzMOPAMzuVcQfIRoIwD6c/ryz6t61nc3jnuTstgfbEF1q7ZAq0Uqmbw0J+SujmHGjhNxVRaJqHsOaWLa1h8OljPmLpkeocNoots/1lTGBbV8RL8kjizUv3jgp1fJTnuqwblQHwY0TWGKtfbxTBJlQLVjGfKuUYjVkfhgiZ3eClfGEbMt+0titPFSe0net0XnfThlRbzo7p5nC7LPuX2TMIWrEJJlFk/GUn+lF1lyOfqGdDrEwuAJ591gkLbou+DrX4ZgEzVpzlXuKhd2Tng/yFtNj/TNsMGqo/fmWjDFoNftwm6S3YkNNcg0V8n9O82/TydA7YPgqd8qsoqo5iHVimYSvxCQGQhD54+JOBfaBDagO76GJMEfBC4kHJsXm8aLDig8c5OUF0MAxW2/uEACstIWyANogHqSzCBeIAh++ba1MfCJMFwXsk7JicE9zi8f42pnZ5ctrr7yyb7a0UKDeuX7K5b04+dVMYSOHYl3KIE/RcyIIVsSsVFeIlnKzIGUQXUPBs42yhsu8+dE64yh7S4l0xpVtgw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Allow proactive reclaimers to submit an additional swappiness= argument to memory.reclaim. This overrides the global or per-memcg swappiness setting for that reclaim attempt. For example: echo "2M swappiness=0" > /sys/fs/cgroup/memory.reclaim will perform reclaim on the rootcg with a swappiness setting of 0 (no swap) regardless of the vm.swappiness sysctl setting. Signed-off-by: Dan Schatzberg --- Documentation/admin-guide/cgroup-v2.rst | 15 ++++++- include/linux/swap.h | 3 +- mm/memcontrol.c | 55 ++++++++++++++++++++----- mm/vmscan.c | 13 +++++- 4 files changed, 70 insertions(+), 16 deletions(-) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 3f85254f3cef..fc2b379dbd0f 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1282,8 +1282,8 @@ PAGE_SIZE multiple when read back. This is a simple interface to trigger memory reclaim in the target cgroup. - This file accepts a single key, the number of bytes to reclaim. - No nested keys are currently supported. + This file accepts a string which containers thhe number of bytes + to reclaim. Example:: @@ -1304,6 +1304,17 @@ PAGE_SIZE multiple when read back. This means that the networking layer will not adapt based on reclaim induced by memory.reclaim. + This file also allows the user to specify the swappiness value + to be used for the reclaim. For example: + + echo "1G swappiness=60" > memory.reclaim + + The above instructs the kernel to perform the reclaim with + a swappiness value of 60. Note that this has the same semantics + as the vm.swappiness sysctl - it sets the relative IO cost of + reclaiming anon vs file memory but does not allow for reclaiming + specific amounts of anon or file memory. + memory.peak A read-only single value file which exists on non-root cgroups. diff --git a/include/linux/swap.h b/include/linux/swap.h index f6dd6575b905..c6e309199f10 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -410,7 +410,8 @@ extern unsigned long try_to_free_pages(struct zonelist *zonelist, int order, extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - unsigned int reclaim_options); + unsigned int reclaim_options, + int *swappiness); extern unsigned long mem_cgroup_shrink_node(struct mem_cgroup *mem, gfp_t gfp_mask, bool noswap, pg_data_t *pgdat, diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 1c1061df9cd1..ba1c89455ab0 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -63,6 +63,7 @@ #include #include #include +#include #include #include "internal.h" #include @@ -2449,7 +2450,7 @@ static unsigned long reclaim_high(struct mem_cgroup *memcg, psi_memstall_enter(&pflags); nr_reclaimed += try_to_free_mem_cgroup_pages(memcg, nr_pages, gfp_mask, - MEMCG_RECLAIM_MAY_SWAP); + MEMCG_RECLAIM_MAY_SWAP, NULL); psi_memstall_leave(&pflags); } while ((memcg = parent_mem_cgroup(memcg)) && !mem_cgroup_is_root(memcg)); @@ -2740,7 +2741,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, psi_memstall_enter(&pflags); nr_reclaimed = try_to_free_mem_cgroup_pages(mem_over_limit, nr_pages, - gfp_mask, reclaim_options); + gfp_mask, reclaim_options, NULL); psi_memstall_leave(&pflags); if (mem_cgroup_margin(mem_over_limit) >= nr_pages) @@ -3660,7 +3661,7 @@ static int mem_cgroup_resize_max(struct mem_cgroup *memcg, } if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, - memsw ? 0 : MEMCG_RECLAIM_MAY_SWAP)) { + memsw ? 0 : MEMCG_RECLAIM_MAY_SWAP, NULL)) { ret = -EBUSY; break; } @@ -3774,7 +3775,7 @@ static int mem_cgroup_force_empty(struct mem_cgroup *memcg) return -EINTR; if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, - MEMCG_RECLAIM_MAY_SWAP)) + MEMCG_RECLAIM_MAY_SWAP, NULL)) nr_retries--; } @@ -6720,7 +6721,7 @@ static ssize_t memory_high_write(struct kernfs_open_file *of, } reclaimed = try_to_free_mem_cgroup_pages(memcg, nr_pages - high, - GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP); + GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP, NULL); if (!reclaimed && !nr_retries--) break; @@ -6769,7 +6770,7 @@ static ssize_t memory_max_write(struct kernfs_open_file *of, if (nr_reclaims) { if (!try_to_free_mem_cgroup_pages(memcg, nr_pages - max, - GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP)) + GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP, NULL)) nr_reclaims--; continue; } @@ -6895,6 +6896,16 @@ static ssize_t memory_oom_group_write(struct kernfs_open_file *of, return nbytes; } +enum { + MEMORY_RECLAIM_SWAPPINESS = 0, + MEMORY_RECLAIM_NULL, +}; + +static const match_table_t if_tokens = { + { MEMORY_RECLAIM_SWAPPINESS, "swappiness=%d"}, + { MEMORY_RECLAIM_NULL, NULL }, +}; + static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, size_t nbytes, loff_t off) { @@ -6902,12 +6913,33 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, unsigned int nr_retries = MAX_RECLAIM_RETRIES; unsigned long nr_to_reclaim, nr_reclaimed = 0; unsigned int reclaim_options; - int err; + char *old_buf, *start; + substring_t args[MAX_OPT_ARGS]; + int swappiness = -1; buf = strstrip(buf); - err = page_counter_memparse(buf, "", &nr_to_reclaim); - if (err) - return err; + + old_buf = buf; + nr_to_reclaim = memparse(buf, &buf) / PAGE_SIZE; + if (buf == old_buf) + return -EINVAL; + + buf = strstrip(buf); + + while ((start = strsep(&buf, " ")) != NULL) { + if (!strlen(start)) + continue; + switch (match_token(start, if_tokens, args)) { + case MEMORY_RECLAIM_SWAPPINESS: + if (match_int(&args[0], &swappiness)) + return -EINVAL; + if (swappiness < 0 || swappiness > 200) + return -EINVAL; + break; + default: + return -EINVAL; + } + } reclaim_options = MEMCG_RECLAIM_MAY_SWAP | MEMCG_RECLAIM_PROACTIVE; while (nr_reclaimed < nr_to_reclaim) { @@ -6926,7 +6958,8 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, reclaimed = try_to_free_mem_cgroup_pages(memcg, min(nr_to_reclaim - nr_reclaimed, SWAP_CLUSTER_MAX), - GFP_KERNEL, reclaim_options); + GFP_KERNEL, reclaim_options, + swappiness == -1 ? NULL : &swappiness); if (!reclaimed && !nr_retries--) return -EAGAIN; diff --git a/mm/vmscan.c b/mm/vmscan.c index 506f8220c5fe..546704ea01e1 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -136,6 +136,9 @@ struct scan_control { /* Always discard instead of demoting to lower tier memory */ unsigned int no_demotion:1; + /* Swappiness value for reclaim, if NULL use memcg/global value */ + int *swappiness; + /* Allocation order */ s8 order; @@ -2327,7 +2330,8 @@ static void get_scan_count(struct lruvec *lruvec, struct scan_control *sc, struct pglist_data *pgdat = lruvec_pgdat(lruvec); struct mem_cgroup *memcg = lruvec_memcg(lruvec); unsigned long anon_cost, file_cost, total_cost; - int swappiness = mem_cgroup_swappiness(memcg); + int swappiness = sc->swappiness ? + *sc->swappiness : mem_cgroup_swappiness(memcg); u64 fraction[ANON_AND_FILE]; u64 denominator = 0; /* gcc */ enum scan_balance scan_balance; @@ -2608,6 +2612,9 @@ static int get_swappiness(struct lruvec *lruvec, struct scan_control *sc) mem_cgroup_get_nr_swap_pages(memcg) < MIN_LRU_BATCH) return 0; + if (sc->swappiness) + return *sc->swappiness; + return mem_cgroup_swappiness(memcg); } @@ -6433,7 +6440,8 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgroup *memcg, unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - unsigned int reclaim_options) + unsigned int reclaim_options, + int *swappiness) { unsigned long nr_reclaimed; unsigned int noreclaim_flag; @@ -6448,6 +6456,7 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, .may_unmap = 1, .may_swap = !!(reclaim_options & MEMCG_RECLAIM_MAY_SWAP), .proactive = !!(reclaim_options & MEMCG_RECLAIM_PROACTIVE), + .swappiness = swappiness, }; /* * Traverse the ZONELIST_FALLBACK zonelist of the current node to put