From patchwork Tue Oct 10 03:21:13 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13414778 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66234CD68EC for ; Tue, 10 Oct 2023 03:21:28 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 28F366B01FF; Mon, 9 Oct 2023 23:21:27 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 229146B0204; Mon, 9 Oct 2023 23:21:27 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED0516B0200; Mon, 9 Oct 2023 23:21:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id D2C826B00E5 for ; Mon, 9 Oct 2023 23:21:26 -0400 (EDT) Received: from smtpin30.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 8DF6BC0559 for ; Tue, 10 Oct 2023 03:21:26 +0000 (UTC) X-FDA: 81328101372.30.94E286D Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf28.hostedemail.com (Postfix) with ESMTP id D2195C000F for ; Tue, 10 Oct 2023 03:21:24 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=09diwHIJ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of 3M8MkZQoKCHw3txw3fmrjilttlqj.htrqnsz2-rrp0fhp.twl@flex--yosryahmed.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3M8MkZQoKCHw3txw3fmrjilttlqj.htrqnsz2-rrp0fhp.twl@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1696908084; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=hC1pWunCMBCtxVf3YGG9j/q6Xl3rVbI2nQig93EJD1c=; b=Tm8879KvWIiWw6zc0M3MwIqkF8mtzoGoP+cE3S89qIgBhueG08+//VSNxYLTiSCt6BHMT9 uRh8eNNFtvinjHEOZTAs6yHJ22hwHhp2p5HTBaecNWQwj2RwdcL8cYZ/s4OeubD2l/i4Ir nI7er+W6iuWXCGYi6b14Y5TcrSyipY4= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=09diwHIJ; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf28.hostedemail.com: domain of 3M8MkZQoKCHw3txw3fmrjilttlqj.htrqnsz2-rrp0fhp.twl@flex--yosryahmed.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3M8MkZQoKCHw3txw3fmrjilttlqj.htrqnsz2-rrp0fhp.twl@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1696908084; a=rsa-sha256; cv=none; b=p1t9LFk+enDaG+K3/1iNNe+hYaeSru1IlsHPq98194YXXKn21H1DxAw+UROCwMHr9Qg3IX xe+FSeEMsoLgmsQRhTfkmbTIQBSpxCe6Tq0yU3/8g7TcWkKc+lhTt5jCiMe6OoEzPfEx+S 7qmu4FC6nNI5Ph0VImO6DgGUjggT69c= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5a7be940fe1so4404517b3.2 for ; Mon, 09 Oct 2023 20:21:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1696908084; x=1697512884; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=hC1pWunCMBCtxVf3YGG9j/q6Xl3rVbI2nQig93EJD1c=; b=09diwHIJ5/Mh66XOvAYpG2sOJBMr22FYzXEaYVYljupuXh5Knl8bbQHWrsg+zkz0Al B0a86f75fPJHJ+FQkirBzyPiCaq7SI2cV93OHSUumFRQw4RTpjvgwYzDUNPSdJD209ZM 3ymbnVH4MwhQyO0ZJMyeSGxonDMM5NK3CJVNTQ+ALK/vxrzrTWpPgxBLSiYJdpa5kSx6 YyboXjc2QpGe09CSIrX+aeRBTuwIKvzfZCTCaW2IhIU5hwJ+EUAiUcqNiSz8J8LMasW/ +ga7JPQKsRspXCltu8fdrtE/TUYHtE1JdeB/JTLPLtJXQzRUkOxoxJTkXar+WbE9cSmy qjsQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696908084; x=1697512884; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=hC1pWunCMBCtxVf3YGG9j/q6Xl3rVbI2nQig93EJD1c=; b=bKWwXZNPqv7aFxvg1xLaQFDRngTavy2viJ0LfwShLWx8SrXnhwAGWkZUAh7Xt+HFR6 Gw1x61r3Cnus27Lz5B78c/XwdqRxSbu/cndCO+h3ilHFtBqKbbJlEmcu3JGD4yvQ91oi GRGPIX+tZFo80aVcsax9Gv30V3M62JuNcD6K84z9NrP8IzxyGpThmIUHLa+jD7XPSx8Q uQAmmn84QdY8vKrb+jN982yZPm1X/Kj9X5z3T5Jm66lurLDpKFvdnWlVgc2guU8EvDEI ELB73nhO7jsxV6ezQuaVvL/qwskLFMx1iqJaN5Eb1M6hY8K4B21mGqGUSpn/FZogFM9R Gd1w== X-Gm-Message-State: AOJu0YyGyDegWUrgbF/nmX7yjEMPaxpObUgPCyBBrtflUybga9VmAB3u oK+NpHy5NiF+yG1K1n3H2Rkun66ZkbtckQT3 X-Google-Smtp-Source: AGHT+IFIrbr9jnt8AKgjFXbiV/tkMci7vpdggI/qLBKz9UQMIl0xyavjRIGPF8xVVyjEs532DFKybC2OPjpal40i X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a5b:b50:0:b0:d80:904d:c211 with SMTP id b16-20020a5b0b50000000b00d80904dc211mr250724ybr.7.1696908083837; Mon, 09 Oct 2023 20:21:23 -0700 (PDT) Date: Tue, 10 Oct 2023 03:21:13 +0000 In-Reply-To: <20231010032117.1577496-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20231010032117.1577496-1-yosryahmed@google.com> X-Mailer: git-send-email 2.42.0.609.gbb76f46606-goog Message-ID: <20231010032117.1577496-3-yosryahmed@google.com> Subject: [PATCH v2 2/5] mm: memcg: move vmstats structs definition above flushing code From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Ivan Babrou , Tejun Heo , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Waiman Long , kernel-team@cloudflare.com, Wei Xu , Greg Thelen , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D2195C000F X-Stat-Signature: 7bywebwm67of6tepb18koo6fy3gnujft X-Rspam-User: X-HE-Tag: 1696908084-662514 X-HE-Meta: U2FsdGVkX18FO0qmIVJhpvRm5Kx0wG9bFcfRhRNBZbCXMjpNSShGJW81DRf8JqWukHiwr85C5QXtUZs7yV8DRf71LSs56OSMCUGrV6zoR9iZqsAbeNk1kq9ZgjJW9kXlZovY06LY3dcturC14PbDPP8hoacowjYmjiHBHlFqYYYjifsa46pqUmeLhw0+iuPuAoM7U1uJg65mMIn4HxGqPs9ixkxFiYtJEbyjLGQuPz72wrV9kG1u6/Z4i68Zsln7YGpCIlmumgloiOHV5yyqT2kIupqLg3yFTSK6F++92TexFyGlSiLhPHgzSR3r7vrCndC/JWh9WpUIR2XTizgJFvv3uMpHJxOSFR50GXPXPZGEgCosQoJvacYQMbWzgmTVo8pCUE6mFPgMUzeJsvzAIfHCRD9Ml+kZEZi/DYiaDtLxfyAGlf4oE3q7yIUWi8qyDFFW6n8sfHm/UI8HM/HAsfb3e9EHgMnxgo4bHADGSlR+YjAQ4LYhb6DYY4kUnbSx5Cj+Z7tH8FZzF/+9vW3zYDsBLSTYALm7sXnNdBXJVSPxNlEmBEbXykAw6EK2IBOoWje00UQl7hZLY5aFgEprzyfd5MypJiO0DOH5qss5EDpTcLdW+JmmK/PtT1XJVvVflBDgtOhVA/zZmOx64X4aC2wG7Btgx4Srft5/IWwTkNTneu7G5S4iBknwN+byB2viowjq0xLIW6vI1BopdoWT8UX7+B3fGQbIdNB2GjvxDpr9TobyoI4fvVUkh42Pn6U41qBFmSIRkuRGNEH4o3AxRVbFzncAKrWM4mJRTVTKaGmL7ViG7427U5Wkuf8o2u+bkppeECVMBSgKL+9fok9+h16c0THhTYffwPcvDSxsmMmuRxE4eh3SnL+f18lkvaDdmFYPN0WAlkhwJtDreM9Msb20Qd5SpCB8QJBmJpCELReMdRyTyBkTbRC1A27/sf1peO1GPY2b4gkvqlY7EOk zis7hAoS f/9JKPyQ2pI8BQ+c6GtSRmsE1yU29Z0iHmnycTjHmWiiz+a/KFFX8JrKy7/tpruBG+FzJrpXNk1lV6Ur2Eyk1b5idlUGxPYTRM1xLWxCPuZ46mDh9ae3M50jp/RpVAQVAPnq1pdt3HzO4B47s4AMhhOJJmAgbseMGJfJRz9bCeknG21V0eeOO7qNHvgTI0Hi6EyuSmhY2DY+/bJpuUI5fBrDKoch7fj7NJzqDsdMuNAFya2P08HelXcAFd1dTZqIjfZ0pqZ4AtiHQ8dCdUwpz4nxpxZAeFgWZ/YpgPstjCND3HlFCfrDNDIOl7YxDNLmmPifp3m3YqKCJGC17pHb5/VbckLVERZUyJtIqK0mGq7CUPBPWqUNx4++hx6vkavIUO1IhWjDYSURuFAQJmYAEhV2rvCU9hRfbgvAiWtC+t8b4kQq5kWxOX0/z5JbTcpRHd8HsEcg+XqidNKR0BGlQKE/Uq1lseVtHsPLdJ4SBgOyIJDf41dxoACGVdNEcOpktmzh10x5PvK6fuQz1SpmM3yGNuXhsB7/ZI33phmHn3Hl0PdsuqKlrhWiHSHBNgrtDtHLTStoZD1DjsQTQ59huGg+xNf5PAkohFP8juptr+c4+De9SEP5GsTIditY0Cbodb9bM X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The following patch will make use of those structs in the flushing code, so move their definitions (and a few other dependencies) a little bit up to reduce the diff noise in the following patch. No functional change intended. Signed-off-by: Yosry Ahmed --- mm/memcontrol.c | 146 ++++++++++++++++++++++++------------------------ 1 file changed, 73 insertions(+), 73 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 4a194fcc9533..a393f1399a2b 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -570,6 +570,79 @@ mem_cgroup_largest_soft_limit_node(struct mem_cgroup_tree_per_node *mctz) return mz; } +/* Subset of vm_event_item to report for memcg event stats */ +static const unsigned int memcg_vm_event_stat[] = { + PGPGIN, + PGPGOUT, + PGSCAN_KSWAPD, + PGSCAN_DIRECT, + PGSCAN_KHUGEPAGED, + PGSTEAL_KSWAPD, + PGSTEAL_DIRECT, + PGSTEAL_KHUGEPAGED, + PGFAULT, + PGMAJFAULT, + PGREFILL, + PGACTIVATE, + PGDEACTIVATE, + PGLAZYFREE, + PGLAZYFREED, +#if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_ZSWAP) + ZSWPIN, + ZSWPOUT, +#endif +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + THP_FAULT_ALLOC, + THP_COLLAPSE_ALLOC, + THP_SWPOUT, + THP_SWPOUT_FALLBACK, +#endif +}; + +#define NR_MEMCG_EVENTS ARRAY_SIZE(memcg_vm_event_stat) +static int mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; + +static void init_memcg_events(void) +{ + int i; + + for (i = 0; i < NR_MEMCG_EVENTS; ++i) + mem_cgroup_events_index[memcg_vm_event_stat[i]] = i + 1; +} + +static inline int memcg_events_index(enum vm_event_item idx) +{ + return mem_cgroup_events_index[idx] - 1; +} + +struct memcg_vmstats_percpu { + /* Local (CPU and cgroup) page state & events */ + long state[MEMCG_NR_STAT]; + unsigned long events[NR_MEMCG_EVENTS]; + + /* Delta calculation for lockless upward propagation */ + long state_prev[MEMCG_NR_STAT]; + unsigned long events_prev[NR_MEMCG_EVENTS]; + + /* Cgroup1: threshold notifications & softlimit tree updates */ + unsigned long nr_page_events; + unsigned long targets[MEM_CGROUP_NTARGETS]; +}; + +struct memcg_vmstats { + /* Aggregated (CPU and subtree) page state & events */ + long state[MEMCG_NR_STAT]; + unsigned long events[NR_MEMCG_EVENTS]; + + /* Non-hierarchical (CPU aggregated) page state & events */ + long state_local[MEMCG_NR_STAT]; + unsigned long events_local[NR_MEMCG_EVENTS]; + + /* Pending child counts during tree propagation */ + long state_pending[MEMCG_NR_STAT]; + unsigned long events_pending[NR_MEMCG_EVENTS]; +}; + /* * memcg and lruvec stats flushing * @@ -681,79 +754,6 @@ static void flush_memcg_stats_dwork(struct work_struct *w) queue_delayed_work(system_unbound_wq, &stats_flush_dwork, FLUSH_TIME); } -/* Subset of vm_event_item to report for memcg event stats */ -static const unsigned int memcg_vm_event_stat[] = { - PGPGIN, - PGPGOUT, - PGSCAN_KSWAPD, - PGSCAN_DIRECT, - PGSCAN_KHUGEPAGED, - PGSTEAL_KSWAPD, - PGSTEAL_DIRECT, - PGSTEAL_KHUGEPAGED, - PGFAULT, - PGMAJFAULT, - PGREFILL, - PGACTIVATE, - PGDEACTIVATE, - PGLAZYFREE, - PGLAZYFREED, -#if defined(CONFIG_MEMCG_KMEM) && defined(CONFIG_ZSWAP) - ZSWPIN, - ZSWPOUT, -#endif -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - THP_FAULT_ALLOC, - THP_COLLAPSE_ALLOC, - THP_SWPOUT, - THP_SWPOUT_FALLBACK, -#endif -}; - -#define NR_MEMCG_EVENTS ARRAY_SIZE(memcg_vm_event_stat) -static int mem_cgroup_events_index[NR_VM_EVENT_ITEMS] __read_mostly; - -static void init_memcg_events(void) -{ - int i; - - for (i = 0; i < NR_MEMCG_EVENTS; ++i) - mem_cgroup_events_index[memcg_vm_event_stat[i]] = i + 1; -} - -static inline int memcg_events_index(enum vm_event_item idx) -{ - return mem_cgroup_events_index[idx] - 1; -} - -struct memcg_vmstats_percpu { - /* Local (CPU and cgroup) page state & events */ - long state[MEMCG_NR_STAT]; - unsigned long events[NR_MEMCG_EVENTS]; - - /* Delta calculation for lockless upward propagation */ - long state_prev[MEMCG_NR_STAT]; - unsigned long events_prev[NR_MEMCG_EVENTS]; - - /* Cgroup1: threshold notifications & softlimit tree updates */ - unsigned long nr_page_events; - unsigned long targets[MEM_CGROUP_NTARGETS]; -}; - -struct memcg_vmstats { - /* Aggregated (CPU and subtree) page state & events */ - long state[MEMCG_NR_STAT]; - unsigned long events[NR_MEMCG_EVENTS]; - - /* Non-hierarchical (CPU aggregated) page state & events */ - long state_local[MEMCG_NR_STAT]; - unsigned long events_local[NR_MEMCG_EVENTS]; - - /* Pending child counts during tree propagation */ - long state_pending[MEMCG_NR_STAT]; - unsigned long events_pending[NR_MEMCG_EVENTS]; -}; - unsigned long memcg_page_state(struct mem_cgroup *memcg, int idx) { long x = READ_ONCE(memcg->vmstats->state[idx]);