From patchwork Mon Apr 3 22:03:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13198874 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C1D0C77B70 for ; Mon, 3 Apr 2023 22:03:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 07D0A900004; Mon, 3 Apr 2023 18:03:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 005D1900002; Mon, 3 Apr 2023 18:03:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D9AEF900004; Mon, 3 Apr 2023 18:03:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id C8981900002 for ; Mon, 3 Apr 2023 18:03:47 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 9BAD41A06F8 for ; Mon, 3 Apr 2023 22:03:47 +0000 (UTC) X-FDA: 80641457694.19.D5E5127 Received: from mail-pf1-f201.google.com (mail-pf1-f201.google.com [209.85.210.201]) by imf18.hostedemail.com (Postfix) with ESMTP id C959F1C001A for ; Mon, 3 Apr 2023 22:03:45 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=QZZrc3A9; spf=pass (imf18.hostedemail.com: domain of 3QE0rZAoKCK8ndhgnPWbTSVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--yosryahmed.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3QE0rZAoKCK8ndhgnPWbTSVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680559425; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kF9j9xhVt+HAtgUv/sW6MmUPAVRA6NHiHn3mXAJRGAA=; b=GFnwHpeRhyGwOEEgnsz1VcaWfqY+JJc77hH0WVneuEyNpIBFylwbJQ3hWkmIfUSxki7M43 DD6A8e9WrSplsAAKiF8gIy64IjTSs8mia8ab2m3vJzWqA3SW11ILNcgnCYOFk3rlEFECnj 36uJBUbFRRljA6PpzBn+h2wRcCnY/Ng= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=QZZrc3A9; spf=pass (imf18.hostedemail.com: domain of 3QE0rZAoKCK8ndhgnPWbTSVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--yosryahmed.bounces.google.com designates 209.85.210.201 as permitted sender) smtp.mailfrom=3QE0rZAoKCK8ndhgnPWbTSVddVaT.RdbaXcjm-bbZkPRZ.dgV@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680559425; a=rsa-sha256; cv=none; b=tXUbO2flnBWWWF5FMlnpaEtss/gnBLmIX7mFL1by2Ir0gPbIwuYMAuRv5725a5aDkgLcMu N/Sq+xoWYTgpjH1bl4Fr2uz2iB+ohmDtpRXoHJj36rUO0VNrDEL5BEFGSIZLSQVE46gM7z /3CI6k03+hTKeHe+ddcWpEUnxri3GnA= Received: by mail-pf1-f201.google.com with SMTP id y186-20020a62cec3000000b00627df3d6ec4so13957974pfg.12 for ; Mon, 03 Apr 2023 15:03:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680559424; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=kF9j9xhVt+HAtgUv/sW6MmUPAVRA6NHiHn3mXAJRGAA=; b=QZZrc3A9Ala3Y1fUrUeH9aHQEpEHIm56zSkCNE0A0d2CfnkVvyEn83miymwgAbVi5v 9d0zVCVjgbHefP1pXO+Nwo1Ul1PtmNl2tpG2LpPChio/My4WZElf1o1jchSNOgvJZUsW WbV5lrmHTcH3fAqoJ7gZenJSRA3/dWbn3kNYUTeryGOWXdDf83/N+i1IFVKcpHglHDCj nH4nWeMmhgItD/X0NlX5brqxxgrkgNj//KowEl9SOf+dOa58iks3E9glke7V87blHVFo T2x7kiC5rzRzeUUzMZ/tXlij1MqA5Wi+XXZnHplS+57XrL+yaVLwig14jPX/om3gIc+M IAag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680559424; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=kF9j9xhVt+HAtgUv/sW6MmUPAVRA6NHiHn3mXAJRGAA=; b=fKYs7SFPHUnNi0iBPyOQ89yFR5odANzUvIQ4QhwjYhHtF4fW8VnDVhd4IzEFj7MBFz COfW7vRbZKPZMFlb6L858VmY6w3t17rU4RFGTeZmvtmkBkLKNVlbwnE2RWae5Grp3IGv MCdTroLGSeotdrgXlHdcTkm3MOGGoW4uwLvBlGUn/8NBHewHW7wgB542ElBteYeUb8tf 4FkV2LYQeKAsK3Zgin91QNruxKblyWLaqhXC7mqdS1BJ5lNkZ7znh8YDBnpQJor4pLVi UxVAdYtmXuGU44uiYsjfP02PTiy6W9kTHYZrZaZBq/5DV1MOG5j8hsl670mkJSweE0lo ArpA== X-Gm-Message-State: AAQBX9eUrHrVWked9itI4gnNRsGMJPhirTcUPHKzhQWYpap6Jy9L5LhM VEs1J1EEf55SyJYuKtd7ZMIwXghNnPt1KZzi X-Google-Smtp-Source: AKy350Y61C08ugQDQ0T+NdoZquMxt4ZL2sOiJByhoRsOjzxwb2aWtKxGtikw0lJtgE93Bg+tt+KjbLE8KoMgZHnh X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a63:2301:0:b0:503:91ff:8dd8 with SMTP id j1-20020a632301000000b0050391ff8dd8mr27700pgj.4.1680559424700; Mon, 03 Apr 2023 15:03:44 -0700 (PDT) Date: Mon, 3 Apr 2023 22:03:35 +0000 In-Reply-To: <20230403220337.443510-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230403220337.443510-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230403220337.443510-4-yosryahmed@google.com> Subject: [PATCH mm-unstable RFC 3/5] memcg: calculate root usage from global state From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: 7s5f1t6erwd11ji5shybkui8gay9hjud X-Rspamd-Queue-Id: C959F1C001A X-HE-Tag: 1680559425-117063 X-HE-Meta: U2FsdGVkX19M2qf3U+Gb7nYEU/HMat9u2LL1LXzrzJogIXHhvi8yMtXN3pPSv7ovIcdG77lw4QWkjhCWJwWSLPEp+haVmMRYZdBEVe9CVcDLJ6zgzSgjdtciTns8zAvnl9FawN2GDu1pNjhgthykd0YBz6KyzbYssSJX0F/VN5vlhyEZ+bxVhlL+2OV5tfHlWGWqGzpy9uk5N7T7waE0SgIpotp2qXpsLh5BYv8FVlAWgvHtuiZwAdGWpCuvMZiuoQoei0gEMagvKJwwStYGJuhdjIaZ0vNVOmkHsu7WRL8Zyw182H116ZTZdFGrjeynqOcMIA+7s9dvMl5DIY124HcMX9e2qkLSIPu3rOMcGdUTiTQsJ3FwULz+huo+Zzysv9LATtVDngsHf1l65IVIDTHcmdxXiCKWyi03juyrwcYpuj8jsrvV6S4EfWMA06E701NW4Imr2PYfT5ry/FbkyJrIC+VVmJelqh7hfc4+4AzqjFew5PjiHajfZ3Dp9UJ2Tix1tBbub5Kc1JNIQ7mmiW822Jc96KmDg4jjD6/IDk1XjYlvUj0Aq4IMW+sixTuWneUXobemI1foAIq1LDxLYD3OBEiF4R23nYHS8ztkw8jpKGHsRV1DDAHDvQN9mhZmGg8+ZafLzIb6G7LXl0rNLP67ESS2yOzBViKXKZsuX/3s0DmaFluDpEcQZalcDbX75+v66gLXnr7mznOqWOGPc6qKTcBdz0wHwM30SQeYOecKUmU2tsc1ICfwr0sgYsDGuccpDqWcMLCkhDo42pb+/e0i86XszBbwXcTQR3Jyg+K2Bndp9ev3ZI4hRzg8s5kQ8v23oa3n0jD84xCwbzs0cmqMQ6cBbNSodO5moEeA0HC4LMvshhPgTAbLMqIqsyWDRaRJnvCDYanwL242C8VHEjDAp9Dv4wsy3q2Eq2cG4oibtuDPx2D0PZH0sGCm0Km/MPvM6kLlwvNgEVe6Q3C bLFRwH9s nWfbBH44ova8rF61tpL5vHiFFAjS6g2H+XJuiib2uPt4VUpGxRH610vcP4BCDMJ/4uRPQNJsKosSRq3vtRmD0hdfR5W0sxwfC36l3ENPpropve7TSUWlOPU7iQeT1S9NGVn/TxvbYP+DFkZ9ASIhK8Iapz1Co8q0ygE45J9P//TpMy9dJx4EFRUW2ryZ2Qzlx0E5J2L4qT6FJmQd0hyykt6KDWRpLceYvhEXVRQ+9aFzM7scKqGPTiPAr3oaeqhm+ULGfFhtUqXponbP26gJ/OrW+COdWZp5QbIczyCtqoYCVOgvhCkJoMImQxJjqonLsPmesHBcmQwsZkicKmkqU9+kC94/ajYLAUXXp+xiHLcjhZFquLdhKk5UzTwvoBGy3xL9IZZesoExRhND0JR/ZJlMrreReAZpzF2u2UiBiW53BVU9HH1YjC241omLhS9EI+7y1vLC3HV85bSrejiYwj3t40fEDediincFxHK5Y1ZZQXY8766GXlllUWoNQfgAlJjzdAuylZ+XW6fHoWr1K5zpWlLU4itw9XHw51sv0TiW3oP08mN8A99X3d4rsJ8Y+14uiH4WvVzGnDZ61r6/dTJS9zAi0h2Q5EtK/Z3ofjzu/rXXa7+bDjrNvrfHJD6kyUkLIcR+PFYFy16WqFy2wsE5G4A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Currently, we approximate the root usage by adding the memcg stats for anon, file, and conditionally swap (for memsw). To read the memcg stats we need to invoke an rstat flush. rstat flushes can be expensive, they scale with the number of cpus and cgroups on the system. mem_cgroup_usage() is called by memcg_events()->mem_cgroup_threshold() with irqs disabled, so such an expensive operation with irqs disabled can cause problems. Instead, approximate the root usage from global state. This is not 100% accurate, but the root usage has always been ill-defined anyway. Signed-off-by: Yosry Ahmed Reviewed-by: Michal Koutný Acked-by: Shakeel Butt --- mm/memcontrol.c | 24 +++++------------------- 1 file changed, 5 insertions(+), 19 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index bdd52fe9e7e4b..e7fe18c0c0ef2 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -3698,27 +3698,13 @@ static unsigned long mem_cgroup_usage(struct mem_cgroup *memcg, bool swap) if (mem_cgroup_is_root(memcg)) { /* - * We can reach here from irq context through: - * uncharge_batch() - * |--memcg_check_events() - * |--mem_cgroup_threshold() - * |--__mem_cgroup_threshold() - * |--mem_cgroup_usage - * - * rstat flushing is an expensive operation that should not be - * done from irq context; use stale stats in this case. - * Arguably, usage threshold events are not reliable on the root - * memcg anyway since its usage is ill-defined. - * - * Additionally, other call paths through memcg_check_events() - * disable irqs, so make sure we are flushing stats atomically. + * Approximate root's usage from global state. This isn't + * perfect, but the root usage was always an approximation. */ - if (in_task()) - mem_cgroup_flush_stats_atomic(); - val = memcg_page_state(memcg, NR_FILE_PAGES) + - memcg_page_state(memcg, NR_ANON_MAPPED); + val = global_node_page_state(NR_FILE_PAGES) + + global_node_page_state(NR_ANON_MAPPED); if (swap) - val += memcg_page_state(memcg, MEMCG_SWAP); + val += total_swap_pages - get_nr_swap_pages(); } else { if (!swap) val = page_counter_read(&memcg->memory);