From patchwork Thu Apr 21 23:44:23 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12822547 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57CAAC433EF for ; Thu, 21 Apr 2022 23:44:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D08F96B0073; Thu, 21 Apr 2022 19:44:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CB87E6B0074; Thu, 21 Apr 2022 19:44:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AE4596B0075; Thu, 21 Apr 2022 19:44:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id 9D0446B0073 for ; Thu, 21 Apr 2022 19:44:45 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id 637118015B for ; Thu, 21 Apr 2022 23:44:45 +0000 (UTC) X-FDA: 79382518530.17.2EE06F0 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf10.hostedemail.com (Postfix) with ESMTP id B4FF5C0006 for ; Thu, 21 Apr 2022 23:44:41 +0000 (UTC) Received: by mail-pl1-f201.google.com with SMTP id e18-20020a17090301d200b00158faee4449so3273301plh.21 for ; Thu, 21 Apr 2022 16:44:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=y22yfO1Fv+K5GPJ449fnrV9G0OFkzja1qqNu/qrx2Gg=; b=hjarVUfdPQmt3pnzo7zRJoLgybwYHn7Y9qTU6JqtC269NCgTgRHX285btpNiTWP9XY g9A2o2ubpn3xZUdg6cQWwKkCTNuQ6RpTBYPaWEdU4pK2X4eBhBEQjwMWoev7/pXofYrO uTXoGA5KlhqGJ3sfVdbuzyy2wInrA7rj+5tWazekKU10XbHYVv5yNXeYfIrlb1rooGGl nQTZyTh91gn+kqYkq9HERsd37T0QX3hvXDnq7Y6tp1DZtr5YeFJfxc7eBsrEbvbnmmkX MVbx+/xOLnnAi/nvs4VjwBKUBpCJzzWGdLqLwZG1WiJc4W4hPER7dmK4vmVe9EwTskCc QqGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=y22yfO1Fv+K5GPJ449fnrV9G0OFkzja1qqNu/qrx2Gg=; b=Kg9gzchu0QN3mokIYJpxYPZY91UnBFGsuAt5GMTrBgldE11hHlXRtjOZFBe9VyIN5w i8w7U/WBRDi4xr0IhUVEJ+A3tUhe8wiy46Afqeao6TS4N/RJgTttKQQtr/V6pb9/dRZX W6RuIMI7z5tq1YjAPD1xP36jPjQyyyYBXgqHEspEfaKAmbX3WgAKvaC/Ch8yMQwbVtm+ 9XpCrAJfQ1nmcYo0TXZNZjo/nlZCYEAi8iCedeTj89ce5u2uw6Th1sAwxnbUzdLt27WX lQH6b8I8ABN+RgW0sJ3O/xsTz47bfiQnKqwd/kR5g7A+06RjbNBdJYyy53pThiOK+gGI yY4w== X-Gm-Message-State: AOAM531YKokm01I1dwqCXlml/b1NwzWkz/Ve/U6BPqhMwsQPgo981jQc f2lX7YmVoa/WOw2+JYZehPeQWdckyhL20hi1 X-Google-Smtp-Source: ABdhPJxAlGBn6QMvn0O+++c8MJABODm4P9rbS0m7Er4vqz7yH9v9EcG6+AJKyVAYOSa0r2P19Af1jd0r4ddxLjQS X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:902:bc8c:b0:156:bc64:fa47 with SMTP id bb12-20020a170902bc8c00b00156bc64fa47mr1726435plb.135.1650584684042; Thu, 21 Apr 2022 16:44:44 -0700 (PDT) Date: Thu, 21 Apr 2022 23:44:23 +0000 In-Reply-To: <20220421234426.3494842-1-yosryahmed@google.com> Message-Id: <20220421234426.3494842-2-yosryahmed@google.com> Mime-Version: 1.0 References: <20220421234426.3494842-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v4 1/4] memcg: introduce per-memcg reclaim interface From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Shakeel Butt , Andrew Morton , Roman Gushchin Cc: David Rientjes , Tejun Heo , Zefan Li , Jonathan Corbet , Shuah Khan , Yu Zhao , Dave Hansen , Wei Xu , Greg Thelen , Chen Wandun , Vaibhav Jain , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Tim Chen , Dan Schatzberg , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Yosry Ahmed , Michal Hocko X-Rspam-User: X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: B4FF5C0006 X-Stat-Signature: e5r7hmt7t8irthxgamfc3j64qfpdhncq Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=hjarVUfd; spf=pass (imf10.hostedemail.com: domain of 3bOxhYgoKCOUfVZYfHOTLKNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3bOxhYgoKCOUfVZYfHOTLKNVVNSL.JVTSPUbe-TTRcHJR.VYN@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-HE-Tag: 1650584681-619977 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Shakeel Butt Introduce a memcg interface to trigger memory reclaim on a memory cgroup. Use case: Proactive Reclaim --------------------------- A userspace proactive reclaimer can continuously probe the memcg to reclaim a small amount of memory. This gives more accurate and up-to-date workingset estimation as the LRUs are continuously sorted and can potentially provide more deterministic memory overcommit behavior. The memory overcommit controller can provide more proactive response to the changing behavior of the running applications instead of being reactive. A userspace reclaimer's purpose in this case is not a complete replacement for kswapd or direct reclaim, it is to proactively identify memory savings opportunities and reclaim some amount of cold pages set by the policy to free up the memory for more demanding jobs or scheduling new jobs. A user space proactive reclaimer is used in Google data centers. Additionally, Meta's TMO paper recently referenced a very similar interface used for user space proactive reclaim: https://dl.acm.org/doi/pdf/10.1145/3503222.3507731 Benefits of a user space reclaimer: ----------------------------------- 1) More flexible on who should be charged for the cpu of the memory reclaim. For proactive reclaim, it makes more sense to be centralized. 2) More flexible on dedicating the resources (like cpu). The memory overcommit controller can balance the cost between the cpu usage and the memory reclaimed. 3) Provides a way to the applications to keep their LRUs sorted, so, under memory pressure better reclaim candidates are selected. This also gives more accurate and uptodate notion of working set for an application. Why memory.high is not enough? ------------------------------ - memory.high can be used to trigger reclaim in a memcg and can potentially be used for proactive reclaim. However there is a big downside in using memory.high. It can potentially introduce high reclaim stalls in the target application as the allocations from the processes or the threads of the application can hit the temporary memory.high limit. - Userspace proactive reclaimers usually use feedback loops to decide how much memory to proactively reclaim from a workload. The metrics used for this are usually either refaults or PSI, and these metrics will become messy if the application gets throttled by hitting the high limit. - memory.high is a stateful interface, if the userspace proactive reclaimer crashes for any reason while triggering reclaim it can leave the application in a bad state. - If a workload is rapidly expanding, setting memory.high to proactively reclaim memory can result in actually reclaiming more memory than intended. The benefits of such interface and shortcomings of existing interface were further discussed in this RFC thread: https://lore.kernel.org/linux-mm/5df21376-7dd1-bf81-8414-32a73cea45dd@google.com/ Interface: ---------- Introducing a very simple memcg interface 'echo 10M > memory.reclaim' to trigger reclaim in the target memory cgroup. The interface is introduced as a nested-keyed file to allow for future optional arguments to be easily added to configure the behavior of reclaim. Possible Extensions: -------------------- - This interface can be extended with an additional parameter or flags to allow specifying one or more types of memory to reclaim from (e.g. file, anon, ..). - The interface can also be extended with a node mask to reclaim from specific nodes. This has use cases for reclaim-based demotion in memory tiering systens. - A similar per-node interface can also be added to support proactive reclaim and reclaim-based demotion in systems without memcg. - Add a timeout parameter to make it easier for user space to call the interface without worrying about being blocked for an undefined amount of time. For now, let's keep things simple by adding the basic functionality. [yosryahmed@google.com: refreshed to current master, updated commit message based on recent discussions and use cases] Signed-off-by: Shakeel Butt Signed-off-by: Yosry Ahmed Acked-by: Johannes Weiner Acked-by: Michal Hocko Acked-by: Wei Xu Acked-by: Roman Gushchin --- Documentation/admin-guide/cgroup-v2.rst | 21 ++++++++++++ mm/memcontrol.c | 44 +++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 69d7a6983f78..19bcd73cad03 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -1208,6 +1208,27 @@ PAGE_SIZE multiple when read back. high limit is used and monitored properly, this limit's utility is limited to providing the final safety net. + memory.reclaim + A write-only nested-keyed file which exists for all cgroups. + + This is a simple interface to trigger memory reclaim in the + target cgroup. + + This file accepts a single key, the number of bytes to reclaim. + No nested keys are currently supported. + + Example:: + + echo "1G" > memory.reclaim + + The interface can be later extended with nested keys to + configure the reclaim behavior. For example, specify the + type of memory to reclaim from (anon, file, ..). + + Please note that the kernel can over or under reclaim from + the target cgroup. If less bytes are reclaimed than the + specified amount, -EAGAIN is returned. + memory.oom.group A read-write single value file which exists on non-root cgroups. The default value is "0". diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 725f76723220..041c17847769 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -6355,6 +6355,45 @@ static ssize_t memory_oom_group_write(struct kernfs_open_file *of, return nbytes; } +static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, + size_t nbytes, loff_t off) +{ + struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); + unsigned int nr_retries = MAX_RECLAIM_RETRIES; + unsigned long nr_to_reclaim, nr_reclaimed = 0; + int err; + + buf = strstrip(buf); + err = page_counter_memparse(buf, "", &nr_to_reclaim); + if (err) + return err; + + while (nr_reclaimed < nr_to_reclaim) { + unsigned long reclaimed; + + if (signal_pending(current)) + return -EINTR; + + /* This is the final attempt, drain percpu lru caches in the + * hope of introducing more evictable pages for + * try_to_free_mem_cgroup_pages(). + */ + if (!nr_retries) + lru_add_drain_all(); + + reclaimed = try_to_free_mem_cgroup_pages(memcg, + nr_to_reclaim - nr_reclaimed, + GFP_KERNEL, true); + + if (!reclaimed && !nr_retries--) + return -EAGAIN; + + nr_reclaimed += reclaimed; + } + + return nbytes; +} + static struct cftype memory_files[] = { { .name = "current", @@ -6413,6 +6452,11 @@ static struct cftype memory_files[] = { .seq_show = memory_oom_group_show, .write = memory_oom_group_write, }, + { + .name = "reclaim", + .flags = CFTYPE_NS_DELEGATABLE, + .write = memory_reclaim, + }, { } /* terminate */ }; From patchwork Thu Apr 21 23:44:24 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12822548 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BBF22C433FE for ; Thu, 21 Apr 2022 23:44:47 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 540C26B0074; Thu, 21 Apr 2022 19:44:47 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4F0326B0075; Thu, 21 Apr 2022 19:44:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 368E26B0078; Thu, 21 Apr 2022 19:44:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 261506B0074 for ; Thu, 21 Apr 2022 19:44:47 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay11.hostedemail.com (Postfix) with ESMTP id F33B380152 for ; Thu, 21 Apr 2022 23:44:46 +0000 (UTC) X-FDA: 79382518572.30.9F12243 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf27.hostedemail.com (Postfix) with ESMTP id 932804000C for ; Thu, 21 Apr 2022 23:44:45 +0000 (UTC) Received: by mail-pl1-f201.google.com with SMTP id w14-20020a1709027b8e00b0015386056d2bso3289410pll.5 for ; Thu, 21 Apr 2022 16:44:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=o1jm2ZxrobVfc+EDXZbKA8synQY3urRJFo8ts6M9Cos=; b=XNmIj+vh/zg85pOZ7hddsPc4rZoBWgPyUKppuibPQ8HpozXmQF4+KbncIhebNDLW+A OaXTK2I3Y4FSfd6Gk2xjKvidVm8LAaC0kUVGEo9TUpGjqdno7WSghhitFdSbSpl49CsV tN/y2CDjCmmhUJ5bWM3jDcayzcK/VRUFAN/fq1h0P/eGY5Hy+hXEJyzvjDolAnIyIh8w 7uodhfP1m0/4SgHnVku88XUlvSW+55VzyEWrQhAUPNeD6F+CC6uOkZE3RyW5PPK9/T9U kH6ybVBPt6zTUU/GcTYAtffLbYejeX7CoHq85fV8abFrT+jgJziiU3RQIdVNZSFNsFws rRVA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=o1jm2ZxrobVfc+EDXZbKA8synQY3urRJFo8ts6M9Cos=; b=VEssaRH/T/UUWlb+NiLEyUl+2jHWPPPnRDUrI+WZjwDXrK21lO/8uPkGgimwArmI+i uIUnkMCckgsrrI7N5t7S+23mqMoz3c3NFgI9DrsFZ+ic8jf2K36TKQvrYuZbszk4y89T jiw3OVEyppinM0bmQmRDdrv9j5NgjoAF0/YyobcUaXaRyWTj6dNBrxM856Iw2H7wZ2/O +aHLcnv8v4EKn4cdA4qYjvqvJBPh3bRGeNnNtb8r0aaAcKQGvCgXSSLOmeBGbt/BpL7n 2W7JCcwddFK31+QXAEXCNnmGtVuoqX2AxoYULJqJG7QEtGlCpFgmFFiP3xeEclFOprJj tGeg== X-Gm-Message-State: AOAM530EK6VhhUQdGviBhERFBnf7V/+mj52r9BVgl9J81LKhkbZusAoH YSy3SsGbGs14DJL9Mk6QFdfQBoXW98W8+vLc X-Google-Smtp-Source: ABdhPJxtoageBQNYvSk1VtKpcFUVPUrVFQmq32PJxpSAW3BoTHti1Uz9DhnW0LOQD3FUjL5L64RLR2rvuttIZXXf X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:90b:4c85:b0:1d4:bc0b:90eb with SMTP id my5-20020a17090b4c8500b001d4bc0b90ebmr2179433pjb.171.1650584685622; Thu, 21 Apr 2022 16:44:45 -0700 (PDT) Date: Thu, 21 Apr 2022 23:44:24 +0000 In-Reply-To: <20220421234426.3494842-1-yosryahmed@google.com> Message-Id: <20220421234426.3494842-3-yosryahmed@google.com> Mime-Version: 1.0 References: <20220421234426.3494842-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v4 2/4] selftests: cgroup: return -errno from cg_read()/cg_write() on failure From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Shakeel Butt , Andrew Morton , Roman Gushchin Cc: David Rientjes , Tejun Heo , Zefan Li , Jonathan Corbet , Shuah Khan , Yu Zhao , Dave Hansen , Wei Xu , Greg Thelen , Chen Wandun , Vaibhav Jain , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Tim Chen , Dan Schatzberg , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Yosry Ahmed X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: 932804000C X-Rspam-User: Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=XNmIj+vh; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf27.hostedemail.com: domain of 3bexhYgoKCOYgWaZgIPUMLOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3bexhYgoKCOYgWaZgIPUMLOWWOTM.KWUTQVcf-UUSdIKS.WZO@flex--yosryahmed.bounces.google.com X-Stat-Signature: pazqffcjyq8wymf19i1ukjc7i1hoyrfi X-HE-Tag: 1650584685-544955 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, cg_read()/cg_write() returns 0 on success and -1 on failure. Modify them to return the -errno on failure. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt --- tools/testing/selftests/cgroup/cgroup_util.c | 44 +++++++++----------- 1 file changed, 19 insertions(+), 25 deletions(-) diff --git a/tools/testing/selftests/cgroup/cgroup_util.c b/tools/testing/selftests/cgroup/cgroup_util.c index dbaa7aabbb4a..e6f3679cdcc0 100644 --- a/tools/testing/selftests/cgroup/cgroup_util.c +++ b/tools/testing/selftests/cgroup/cgroup_util.c @@ -19,6 +19,7 @@ #include "cgroup_util.h" #include "../clone3/clone3_selftests.h" +/* Returns read len on success, or -errno on failure. */ static ssize_t read_text(const char *path, char *buf, size_t max_len) { ssize_t len; @@ -26,35 +27,29 @@ static ssize_t read_text(const char *path, char *buf, size_t max_len) fd = open(path, O_RDONLY); if (fd < 0) - return fd; + return -errno; len = read(fd, buf, max_len - 1); - if (len < 0) - goto out; - buf[len] = 0; -out: + if (len >= 0) + buf[len] = 0; + close(fd); - return len; + return len < 0 ? -errno : len; } +/* Returns written len on success, or -errno on failure. */ static ssize_t write_text(const char *path, char *buf, ssize_t len) { int fd; fd = open(path, O_WRONLY | O_APPEND); if (fd < 0) - return fd; + return -errno; len = write(fd, buf, len); - if (len < 0) { - close(fd); - return len; - } - close(fd); - - return len; + return len < 0 ? -errno : len; } char *cg_name(const char *root, const char *name) @@ -87,16 +82,16 @@ char *cg_control(const char *cgroup, const char *control) return ret; } +/* Returns 0 on success, or -errno on failure. */ int cg_read(const char *cgroup, const char *control, char *buf, size_t len) { char path[PATH_MAX]; + ssize_t ret; snprintf(path, sizeof(path), "%s/%s", cgroup, control); - if (read_text(path, buf, len) >= 0) - return 0; - - return -1; + ret = read_text(path, buf, len); + return ret >= 0 ? 0 : ret; } int cg_read_strcmp(const char *cgroup, const char *control, @@ -177,17 +172,15 @@ long cg_read_lc(const char *cgroup, const char *control) return cnt; } +/* Returns 0 on success, or -errno on failure. */ int cg_write(const char *cgroup, const char *control, char *buf) { char path[PATH_MAX]; - ssize_t len = strlen(buf); + ssize_t len = strlen(buf), ret; snprintf(path, sizeof(path), "%s/%s", cgroup, control); - - if (write_text(path, buf, len) == len) - return 0; - - return -1; + ret = write_text(path, buf, len); + return ret == len ? 0 : ret; } int cg_find_unified_root(char *root, size_t len) @@ -545,7 +538,8 @@ ssize_t proc_read_text(int pid, bool thread, const char *item, char *buf, size_t else snprintf(path, sizeof(path), "/proc/%d/%s", pid, item); - return read_text(path, buf, size); + size = read_text(path, buf, size); + return size < 0 ? -1 : size; } int proc_read_strstr(int pid, bool thread, const char *item, const char *needle) From patchwork Thu Apr 21 23:44:25 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12822549 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 463AFC433F5 for ; Thu, 21 Apr 2022 23:44:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CE5966B0075; Thu, 21 Apr 2022 19:44:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C94646B0078; Thu, 21 Apr 2022 19:44:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B31B16B007B; Thu, 21 Apr 2022 19:44:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id A56376B0075 for ; Thu, 21 Apr 2022 19:44:48 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 763FE226CE for ; Thu, 21 Apr 2022 23:44:48 +0000 (UTC) X-FDA: 79382518656.06.994C604 Received: from mail-pg1-f202.google.com (mail-pg1-f202.google.com [209.85.215.202]) by imf22.hostedemail.com (Postfix) with ESMTP id 96298C0002 for ; Thu, 21 Apr 2022 23:44:47 +0000 (UTC) Received: by mail-pg1-f202.google.com with SMTP id h14-20020a63530e000000b0039daafb0a84so3605530pgb.7 for ; Thu, 21 Apr 2022 16:44:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=zrjgrKYnU6OpMbvPqxH4AMaOafcQPI/dEJCu4TPzq/A=; b=R5EgEEFiJLJPylJr1GpRvnGVqN1ompyECMnvFf7eJcNcIBFBfprXXAft0A/RmwaXeD sbaE02S2hM0MhJlnoYl4BZ6GwzLLyFeyoaeyfYlebGdZNk2FJqWy0CBTgUDnhS3IZByt vQWqicHWsomUI7gHz2Tvt1gFymu5ka/dsdDO3KlmWLi1mH5fKV33sBgHm0V9J9vTRadg 6MpPFs5NAnuJf5761TBNcYgccwJrkDqvgaOAsSc4lC0uYWWC70kYneQqFgmKZ/IoeuvU TVc1vZldKVcLP7xe8H7/pcshpA4uONKuKc7K/n5XrtPuudMkt4mzmX1nR+nSRuafWS5v ptTQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=zrjgrKYnU6OpMbvPqxH4AMaOafcQPI/dEJCu4TPzq/A=; b=T/SppHjT3SdOrBmyDC0Sc2t/itXuyPDC7yfg3QlFBPC6lJzFKu5jkoVMnmlT7HCCW1 072q6dqMDjFJ/ANvhK1dtV5MXFGwPWwxM+eLtLvyerxUE93XtzkCfpBEpzEWXGaF6fZN EIcqiEVhdtdbZhZaKFXHmohjwwcWduUmShA5GidLJJ4/c+IlswhBi62YaFz/pLa/R8jI 4vgOCq0rSmJ/TUinmTVFlSEmS73eKuxbcdBgSk6/nHF0rCGdAG9yf45X9+a+2bKk8TS8 C7egca3LBhmMQ7PX6+1Ffsj5iEkm1e31j0bcPNMxIb5eK89NjaXDnbbGvX72AQ9JkY4H /J9g== X-Gm-Message-State: AOAM530ULsjatawqSvYKCPug1uaGdXzsFdKpYQ88kBLXyyIuPQfiYKK+ 88mdhTwNCH6PYn08kjzkFO8hhNM3rl7OgmPA X-Google-Smtp-Source: ABdhPJxkyS9552eyRbp5WO6d5madi5NDuaVaOduz7euoUqf8KeXnSqt/NpxhsWB/Uh+lm1g3ddmKZu3hnmlshvfN X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:902:f391:b0:158:f1e6:7233 with SMTP id f17-20020a170902f39100b00158f1e67233mr1863993ple.133.1650584687052; Thu, 21 Apr 2022 16:44:47 -0700 (PDT) Date: Thu, 21 Apr 2022 23:44:25 +0000 In-Reply-To: <20220421234426.3494842-1-yosryahmed@google.com> Message-Id: <20220421234426.3494842-4-yosryahmed@google.com> Mime-Version: 1.0 References: <20220421234426.3494842-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v4 3/4] selftests: cgroup: fix alloc_anon_noexit() instantly freeing memory From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Shakeel Butt , Andrew Morton , Roman Gushchin Cc: David Rientjes , Tejun Heo , Zefan Li , Jonathan Corbet , Shuah Khan , Yu Zhao , Dave Hansen , Wei Xu , Greg Thelen , Chen Wandun , Vaibhav Jain , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Tim Chen , Dan Schatzberg , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Yosry Ahmed X-Rspam-User: X-Rspamd-Server: rspam11 X-Rspamd-Queue-Id: 96298C0002 X-Stat-Signature: rn3j5tdn4d3jdxw4p16s79m1xau756iz Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=R5EgEEFi; spf=pass (imf22.hostedemail.com: domain of 3b-xhYgoKCOgiYcbiKRWONQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yosryahmed.bounces.google.com designates 209.85.215.202 as permitted sender) smtp.mailfrom=3b-xhYgoKCOgiYcbiKRWONQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-HE-Tag: 1650584687-119418 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, alloc_anon_noexit() calls alloc_anon() which instantly frees the allocated memory. alloc_anon_noexit() is usually used with cg_run_nowait() to run a process in the background that allocates memory. It makes sense for the background process to keep the memory allocated and not instantly free it (otherwise there is no point of running it in the background). Signed-off-by: Yosry Ahmed Acked-by: Roman Gushchin Acked-by: Shakeel Butt --- tools/testing/selftests/cgroup/test_memcontrol.c | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c index 36ccf2322e21..f2ffb3a30194 100644 --- a/tools/testing/selftests/cgroup/test_memcontrol.c +++ b/tools/testing/selftests/cgroup/test_memcontrol.c @@ -211,13 +211,17 @@ static int alloc_pagecache_50M_noexit(const char *cgroup, void *arg) static int alloc_anon_noexit(const char *cgroup, void *arg) { int ppid = getppid(); + size_t size = (unsigned long)arg; + char *buf, *ptr; - if (alloc_anon(cgroup, arg)) - return -1; + buf = malloc(size); + for (ptr = buf; ptr < buf + size; ptr += PAGE_SIZE) + *ptr = 0; while (getppid() == ppid) sleep(1); + free(buf); return 0; } From patchwork Thu Apr 21 23:44:26 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12822550 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 45AADC433FE for ; Thu, 21 Apr 2022 23:44:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D2E906B0078; Thu, 21 Apr 2022 19:44:50 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CDDD66B007B; Thu, 21 Apr 2022 19:44:50 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B81D96B007D; Thu, 21 Apr 2022 19:44:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.25]) by kanga.kvack.org (Postfix) with ESMTP id 9D36A6B0078 for ; Thu, 21 Apr 2022 19:44:50 -0400 (EDT) Received: from smtpin27.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 75BEA601E5 for ; Thu, 21 Apr 2022 23:44:50 +0000 (UTC) X-FDA: 79382518740.27.7473259 Received: from mail-pl1-f202.google.com (mail-pl1-f202.google.com [209.85.214.202]) by imf13.hostedemail.com (Postfix) with ESMTP id 643AE20028 for ; Thu, 21 Apr 2022 23:44:47 +0000 (UTC) Received: by mail-pl1-f202.google.com with SMTP id f11-20020a170902684b00b00158c67ef30cso3291168pln.7 for ; Thu, 21 Apr 2022 16:44:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=soZcA9y3PeE+zbNMS5WF05IaFn1Ub9nCQnWDGl/9EM4=; b=AMOxrzCggH4xAXjA65k6jRVjeYiHvCQ4JBuinSbQFuHWjpOZVImwxNzZ4vHPZohQTB aXciCJTbhyQ9+Fe7fHynuCvh/GMWKc3btZ9Y358IdMT3cFDnaS8nhl8U83WMeiZ8Ithp MPQW59CsoBRg/kR2IuaK+bMzzTxWWtYyKW/Dy7he+lQj4bZ/W17RnhFraPRXsfm66/d7 5UbhQH01hqjEcs/bA2iTiTpfx8hnyPldqhb6oZiRPAgE1LWc9rycFDzbTK1wLpo4gCLg /eJv/edTk29F2NU8C8gWTD76IZam53QbHxr/hDRN522g8VA/T1AOWy7SPpB7vGrVABsz FexA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=soZcA9y3PeE+zbNMS5WF05IaFn1Ub9nCQnWDGl/9EM4=; b=LzPR7uJbyBmPdXsHKQHTqcm6LxPIUwBiPOziChh3gVPau/xezBfATYBRQPGGEh48PM UQztfNkH3grIec4ZL2bH1Pv2pAhqH5MckaIRtR7ZtIQNAqBokUklKKWFmXWrAE7y1ts+ 3Rqq9AfQvBwQuZdRV3n/FDXfUHVMgsObTmeSjAYayZwxmtrcarOGERl4P2shaTtApHq7 ZyirUMS0QCMueYMc/JdqSUZFp5HkTpmAVSPdPmqNcjTcBdl5c3TJcqZ436IwN6vl3inw sfYivvMW0rhhdKIvB9BjEdt3Pa+A3Xl7NetrsiBPt1rsTpj8lFb1y7hNTz40U4TR1+Da X3Gg== X-Gm-Message-State: AOAM530FZg1nD7UF0g7Q1NemeVRce69YcmB6IsZifTn4VKvJ4ReTdNug LHMWEkfIomw8KtrTJFWF9FFSrWl7JboMeEw1 X-Google-Smtp-Source: ABdhPJz2CcjMs+n1pq6zI6202/dU8xhtLt6Efq3KS5h9gllRzFGCi7i+ft39wZ54kqGMAmjNN828QLIvEM5o94Aq X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:90b:2384:b0:1cb:5223:9dc4 with SMTP id mr4-20020a17090b238400b001cb52239dc4mr191635pjb.1.1650584688532; Thu, 21 Apr 2022 16:44:48 -0700 (PDT) Date: Thu, 21 Apr 2022 23:44:26 +0000 In-Reply-To: <20220421234426.3494842-1-yosryahmed@google.com> Message-Id: <20220421234426.3494842-5-yosryahmed@google.com> Mime-Version: 1.0 References: <20220421234426.3494842-1-yosryahmed@google.com> X-Mailer: git-send-email 2.36.0.rc2.479.g8af0fa9b8e-goog Subject: [PATCH v4 4/4] selftests: cgroup: add a selftest for memory.reclaim From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Shakeel Butt , Andrew Morton , Roman Gushchin Cc: David Rientjes , Tejun Heo , Zefan Li , Jonathan Corbet , Shuah Khan , Yu Zhao , Dave Hansen , Wei Xu , Greg Thelen , Chen Wandun , Vaibhav Jain , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Tim Chen , Dan Schatzberg , cgroups@vger.kernel.org, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Yosry Ahmed Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=AMOxrzCg; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf13.hostedemail.com: domain of 3cOxhYgoKCOkjZdcjLSXPORZZRWP.NZXWTYfi-XXVgLNV.ZcR@flex--yosryahmed.bounces.google.com designates 209.85.214.202 as permitted sender) smtp.mailfrom=3cOxhYgoKCOkjZdcjLSXPORZZRWP.NZXWTYfi-XXVgLNV.ZcR@flex--yosryahmed.bounces.google.com X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 643AE20028 X-Stat-Signature: f831pw49drbc43smjboe1suoirz5mij4 X-HE-Tag: 1650584687-604202 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000012, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add a new test for memory.reclaim that verifies that the interface correctly reclaims memory as intended, from both anon and file pages. Signed-off-by: Yosry Ahmed Acked-by: Roman Gushchin --- .../selftests/cgroup/test_memcontrol.c | 86 +++++++++++++++++++ 1 file changed, 86 insertions(+) diff --git a/tools/testing/selftests/cgroup/test_memcontrol.c b/tools/testing/selftests/cgroup/test_memcontrol.c index f2ffb3a30194..5f7c20de2426 100644 --- a/tools/testing/selftests/cgroup/test_memcontrol.c +++ b/tools/testing/selftests/cgroup/test_memcontrol.c @@ -760,6 +760,91 @@ static int test_memcg_max(const char *root) return ret; } +/* + * This test checks that memory.reclaim reclaims the given + * amount of memory (from both anon and file). + */ +static int test_memcg_reclaim(const char *root) +{ + int ret = KSFT_FAIL, fd, retries; + char *memcg; + long current, to_reclaim; + char buf[64]; + + memcg = cg_name(root, "memcg_test"); + if (!memcg) + goto cleanup; + + if (cg_create(memcg)) + goto cleanup; + + current = cg_read_long(memcg, "memory.current"); + if (current != 0) + goto cleanup; + + cg_run_nowait(memcg, alloc_anon_noexit, (void *) MB(50)); + sleep(1); + + fd = get_temp_fd(); + if (fd < 0) + goto cleanup; + + cg_run_nowait(memcg, alloc_pagecache_50M_noexit, (void *)(long)fd); + sleep(1); + + current = cg_read_long(memcg, "memory.current"); + if (!values_close(current, MB(100), 10)) + goto cleanup; + + /* + * Reclaim until current reaches 30M, make sure to reclaim over 50M to + * hit both anon and file. + */ + retries = 5; + while (true) { + int err; + + current = cg_read_long(memcg, "memory.current"); + to_reclaim = current - MB(30); + + /* + * We only keep looping if we get EAGAIN, which means we could + * not reclaim the full amount. + */ + if (to_reclaim <= 0) + goto cleanup; + + + snprintf(buf, sizeof(buf), "%ld", to_reclaim); + err = cg_write(memcg, "memory.reclaim", buf); + if (!err) { + /* + * If writing succeeds, then the written amount should have been + * fully reclaimed (and maybe more). + */ + current = cg_read_long(memcg, "memory.current"); + if (!values_close(current, MB(30), 3) && current > MB(30)) + goto cleanup; + break; + } + + /* The kernel could not reclaim the full amount, try again. */ + if (err == -EAGAIN && retries--) + continue; + + /* We got an unexpected error or ran out of retries. */ + goto cleanup; + } + + ret = KSFT_PASS; +cleanup: + cg_destroy(memcg); + free(memcg); + close(fd); + + return ret; +} + static int alloc_anon_50M_check_swap(const char *cgroup, void *arg) { long mem_max = (long)arg; @@ -1263,6 +1348,7 @@ struct memcg_test { T(test_memcg_high), T(test_memcg_high_sync), T(test_memcg_max), + T(test_memcg_reclaim), T(test_memcg_oom_events), T(test_memcg_swap_max), T(test_memcg_sock),