From patchwork Thu Jul 14 06:49:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 12917349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C31DFC43334 for ; Thu, 14 Jul 2022 06:49:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F1715940191; Thu, 14 Jul 2022 02:49:23 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC53E940134; Thu, 14 Jul 2022 02:49:23 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8DB3940191; Thu, 14 Jul 2022 02:49:23 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CA558940134 for ; Thu, 14 Jul 2022 02:49:23 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 8589834F38 for ; Thu, 14 Jul 2022 06:49:23 +0000 (UTC) X-FDA: 79684779006.17.504BE38 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf03.hostedemail.com (Postfix) with ESMTP id 2194620032 for ; Thu, 14 Jul 2022 06:49:22 +0000 (UTC) Received: by mail-yb1-f202.google.com with SMTP id t10-20020a5b07ca000000b0066ec1bb6e2cso840682ybq.14 for ; Wed, 13 Jul 2022 23:49:22 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:message-id:mime-version:subject:from:to:cc; bh=OWO7azjUaMs5aX9KKa+l/Q7z/45AbzWJxqiXiv/5VIU=; b=haGNeiKz2kRNng6AUosqHR6JF78x1FLSdCVHlrlH10qcMkjhiIPZF9S02ryBUY7FXP OTVSF3ccuWZJQS8Rxj/TdSCex2uRmB7BwCCZjAz39OAJ9VW4YkMjnPhyubNmaKqh2HbN pgp65382e8ExXlsqcajmFZBt8S7u5uF4QgFIA97FHGu4g9jVxZx0RavJCpARNopbcKHM sOrIGrmktyTV2eCBOyIhOcYreiwz4dUW22a1q/8kBSNMbNF4AyiNqXQYLcfh6Ujt1pvt CjB5UNYRhx/eG96BkxJ6MsUCrLcD1UHunMNawVGeShicrq9n5m/KPg95Z3yXvXcn+yGQ Nsdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=OWO7azjUaMs5aX9KKa+l/Q7z/45AbzWJxqiXiv/5VIU=; b=PXNrhAz0MxiEYV5bvBU3+g4oIq8ikMtR3uKBY0DAYokWkfmWiyErtNUsfGZOeiHnxW gtfMboyP4ykXzgftgZxXMOJFslHVSYkzx+IK3CSt30HLhyN/psMFN7u0p4O4G8LZX99l iFtcy02kp21OFxqzws11fo+bcL8+m2Hpm0jQxOJCHtBvv02NsaqqXdn40XWX9RG44n8i CGHqw5IL1clpyY9a5PRSW4+hPhEzXIhwNwl3PHKHPtia39UVWQPe50cXe+6zvIXGO7cw E7Q9xL6AknGpnrbyLYQMvR2ARIFlvHWWKVpXBCYNAm6vQNthyI8MkWe3pDNTZEOSpwPA +WqQ== X-Gm-Message-State: AJIora+k4+wfDk0SD5L7zM37crjIRAw3nvzL3OchlREVoReCWk9pREhJ mlkG4lHY0yO2Hvp9xOgqJK8rfr7UdpjjFEDN X-Google-Smtp-Source: AGRyM1u6B/wHfl+IQRGW5yeE9E+fRV09NHiNSnERTgi/Lyj3e6bPqXEu9iK84kwS0wwLfm65Xnvc0vlUeR+yFgUB X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a0d:e882:0:b0:31d:5aa3:2cbd with SMTP id r124-20020a0de882000000b0031d5aa32cbdmr8449855ywe.52.1657781362316; Wed, 13 Jul 2022 23:49:22 -0700 (PDT) Date: Thu, 14 Jul 2022 06:49:18 +0000 Message-Id: <20220714064918.2576464-1-yosryahmed@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.37.0.144.g8ac04bfd2-goog Subject: [PATCH v4] mm: vmpressure: don't count proactive reclaim in vmpressure From: Yosry Ahmed To: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: Matthew Wilcox , Vlastimil Babka , David Hildenbrand , Miaohe Lin , NeilBrown , Alistair Popple , Suren Baghdasaryan , Peter Xu , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1657781363; a=rsa-sha256; cv=none; b=sUyumsVPQqOzPHV2XMh7FIyoX7c1XHj8EN4GRiHIwjatQWLWzFef04pGrbxXJqGS1TuVdp oh2Wbk+rIpyZ/7BcVUpRvzteksuvpg5Z92xSQMrTlbsYYkIeBbrEkDfpJCz87e5wUAOUqw KUwnETyKTwC6PZ2J6mgngM0wtjtILnU= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=haGNeiKz; spf=pass (imf03.hostedemail.com: domain of 3crzPYgoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3crzPYgoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1657781363; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=OWO7azjUaMs5aX9KKa+l/Q7z/45AbzWJxqiXiv/5VIU=; b=3XZ6HW/xDUcNwADkmEPKixrKxU+jXusIMfKXQHg/qZOPoll0YENQhT7oYopBPTlGpuKsrV qS+6KKA6bXasok+pJS5xax+YtRlYERXHkjFq4ePda4dFoxm2nbmJDwJ3Own1UDndsk7VPK 3pPw68GFXjGIDDoknxM2UnEr19c90FY= X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 2194620032 X-Rspam-User: Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=haGNeiKz; spf=pass (imf03.hostedemail.com: domain of 3crzPYgoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3crzPYgoKCEc7x107jqvnmpxxpun.lxvurw36-vvt4jlt.x0p@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com X-Stat-Signature: 4rd4sxe7msbrap651b4ub18jx5k9daik X-HE-Tag: 1657781362-668853 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: vmpressure is used in cgroup v1 to notify userspace of reclaim efficiency events, and is also used in both cgroup v1 and v2 as a signal for memory pressure for networking, see mem_cgroup_under_socket_pressure(). Proactive reclaim intends to probe memcgs for cold memory, without affecting their performance. Hence, reclaim caused by writing to memory.reclaim should not trigger vmpressure. Signed-off-by: Yosry Ahmed Acked-by: Shakeel Butt --- Changes in v4: - Removed unneeded reclaim_options local variables (Andrew). Changes in v3: - Limited the vmpressure change to memory.reclaim, dropped psi changes, updated changelog to reflect new behavior (Michal, Shakeel) Changes in v2: - Removed unnecessary initializations to zero (Andrew). - Separate declarations and initializations when it causes line wrapping (Andrew). --- include/linux/swap.h | 5 ++++- mm/memcontrol.c | 24 ++++++++++++++---------- mm/vmscan.c | 27 +++++++++++++++++---------- 3 files changed, 35 insertions(+), 21 deletions(-) diff --git a/include/linux/swap.h b/include/linux/swap.h index 0c0fed1b348f..f6e9eaa2339f 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -411,10 +411,13 @@ extern void lru_cache_add_inactive_or_unevictable(struct page *page, extern unsigned long zone_reclaimable_pages(struct zone *zone); extern unsigned long try_to_free_pages(struct zonelist *zonelist, int order, gfp_t gfp_mask, nodemask_t *mask); + +#define MEMCG_RECLAIM_MAY_SWAP (1 << 1) +#define MEMCG_RECLAIM_PROACTIVE (1 << 2) extern unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - bool may_swap); + unsigned int reclaim_options); extern unsigned long mem_cgroup_shrink_node(struct mem_cgroup *mem, gfp_t gfp_mask, bool noswap, pg_data_t *pgdat, diff --git a/mm/memcontrol.c b/mm/memcontrol.c index a550042d88c3..b668224142c7 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -2331,7 +2331,8 @@ static unsigned long reclaim_high(struct mem_cgroup *memcg, psi_memstall_enter(&pflags); nr_reclaimed += try_to_free_mem_cgroup_pages(memcg, nr_pages, - gfp_mask, true); + gfp_mask, + MEMCG_RECLAIM_MAY_SWAP); psi_memstall_leave(&pflags); } while ((memcg = parent_mem_cgroup(memcg)) && !mem_cgroup_is_root(memcg)); @@ -2576,7 +2577,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, struct page_counter *counter; unsigned long nr_reclaimed; bool passed_oom = false; - bool may_swap = true; + unsigned int reclaim_options = MEMCG_RECLAIM_MAY_SWAP; bool drained = false; unsigned long pflags; @@ -2593,7 +2594,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, mem_over_limit = mem_cgroup_from_counter(counter, memory); } else { mem_over_limit = mem_cgroup_from_counter(counter, memsw); - may_swap = false; + reclaim_options &= ~MEMCG_RECLAIM_MAY_SWAP; } if (batch > nr_pages) { @@ -2620,7 +2621,7 @@ static int try_charge_memcg(struct mem_cgroup *memcg, gfp_t gfp_mask, psi_memstall_enter(&pflags); nr_reclaimed = try_to_free_mem_cgroup_pages(mem_over_limit, nr_pages, - gfp_mask, may_swap); + gfp_mask, reclaim_options); psi_memstall_leave(&pflags); if (mem_cgroup_margin(mem_over_limit) >= nr_pages) @@ -3402,8 +3403,8 @@ static int mem_cgroup_resize_max(struct mem_cgroup *memcg, continue; } - if (!try_to_free_mem_cgroup_pages(memcg, 1, - GFP_KERNEL, !memsw)) { + if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, + memsw ? 0 : MEMCG_RECLAIM_MAY_SWAP)) { ret = -EBUSY; break; } @@ -3513,7 +3514,8 @@ static int mem_cgroup_force_empty(struct mem_cgroup *memcg) if (signal_pending(current)) return -EINTR; - if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, true)) + if (!try_to_free_mem_cgroup_pages(memcg, 1, GFP_KERNEL, + MEMCG_RECLAIM_MAY_SWAP)) nr_retries--; } @@ -6248,7 +6250,7 @@ static ssize_t memory_high_write(struct kernfs_open_file *of, } reclaimed = try_to_free_mem_cgroup_pages(memcg, nr_pages - high, - GFP_KERNEL, true); + GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP); if (!reclaimed && !nr_retries--) break; @@ -6297,7 +6299,7 @@ static ssize_t memory_max_write(struct kernfs_open_file *of, if (nr_reclaims) { if (!try_to_free_mem_cgroup_pages(memcg, nr_pages - max, - GFP_KERNEL, true)) + GFP_KERNEL, MEMCG_RECLAIM_MAY_SWAP)) nr_reclaims--; continue; } @@ -6426,6 +6428,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, struct mem_cgroup *memcg = mem_cgroup_from_css(of_css(of)); unsigned int nr_retries = MAX_RECLAIM_RETRIES; unsigned long nr_to_reclaim, nr_reclaimed = 0; + unsigned int reclaim_options; int err; buf = strstrip(buf); @@ -6433,6 +6436,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, if (err) return err; + reclaim_options = MEMCG_RECLAIM_MAY_SWAP | MEMCG_RECLAIM_PROACTIVE; while (nr_reclaimed < nr_to_reclaim) { unsigned long reclaimed; @@ -6449,7 +6453,7 @@ static ssize_t memory_reclaim(struct kernfs_open_file *of, char *buf, reclaimed = try_to_free_mem_cgroup_pages(memcg, nr_to_reclaim - nr_reclaimed, - GFP_KERNEL, true); + GFP_KERNEL, reclaim_options); if (!reclaimed && !nr_retries--) return -EAGAIN; diff --git a/mm/vmscan.c b/mm/vmscan.c index f7d9a683e3a7..0969e6408a53 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -102,6 +102,9 @@ struct scan_control { /* Can pages be swapped as part of reclaim? */ unsigned int may_swap:1; + /* Proactive reclaim invoked by userspace through memory.reclaim */ + unsigned int proactive:1; + /* * Cgroup memory below memory.low is protected as long as we * don't threaten to OOM. If any cgroup is reclaimed at @@ -3125,9 +3128,10 @@ static void shrink_node_memcgs(pg_data_t *pgdat, struct scan_control *sc) sc->priority); /* Record the group's reclaim efficiency */ - vmpressure(sc->gfp_mask, memcg, false, - sc->nr_scanned - scanned, - sc->nr_reclaimed - reclaimed); + if (!sc->proactive) + vmpressure(sc->gfp_mask, memcg, false, + sc->nr_scanned - scanned, + sc->nr_reclaimed - reclaimed); } while ((memcg = mem_cgroup_iter(target_memcg, memcg, NULL))); } @@ -3250,9 +3254,10 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) } /* Record the subtree's reclaim efficiency */ - vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, - sc->nr_scanned - nr_scanned, - sc->nr_reclaimed - nr_reclaimed); + if (!sc->proactive) + vmpressure(sc->gfp_mask, sc->target_mem_cgroup, true, + sc->nr_scanned - nr_scanned, + sc->nr_reclaimed - nr_reclaimed); if (sc->nr_reclaimed - nr_reclaimed) reclaimable = true; @@ -3534,8 +3539,9 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist, __count_zid_vm_events(ALLOCSTALL, sc->reclaim_idx, 1); do { - vmpressure_prio(sc->gfp_mask, sc->target_mem_cgroup, - sc->priority); + if (!sc->proactive) + vmpressure_prio(sc->gfp_mask, sc->target_mem_cgroup, + sc->priority); sc->nr_scanned = 0; shrink_zones(zonelist, sc); @@ -3825,7 +3831,7 @@ unsigned long mem_cgroup_shrink_node(struct mem_cgroup *memcg, unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, unsigned long nr_pages, gfp_t gfp_mask, - bool may_swap) + unsigned int reclaim_options) { unsigned long nr_reclaimed; unsigned int noreclaim_flag; @@ -3838,7 +3844,8 @@ unsigned long try_to_free_mem_cgroup_pages(struct mem_cgroup *memcg, .priority = DEF_PRIORITY, .may_writepage = !laptop_mode, .may_unmap = 1, - .may_swap = may_swap, + .may_swap = !!(reclaim_options & MEMCG_RECLAIM_MAY_SWAP), + .proactive = !!(reclaim_options & MEMCG_RECLAIM_PROACTIVE), }; /* * Traverse the ZONELIST_FALLBACK zonelist of the current node to put