From patchwork Thu Aug 31 16:56:10 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13371728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E1CE6C83F01 for ; Thu, 31 Aug 2023 16:56:24 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 785718D001B; Thu, 31 Aug 2023 12:56:24 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 70F8C8D0001; Thu, 31 Aug 2023 12:56:24 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 587C28D001B; Thu, 31 Aug 2023 12:56:24 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 389958D0001 for ; Thu, 31 Aug 2023 12:56:24 -0400 (EDT) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 0EF521C98C4 for ; Thu, 31 Aug 2023 16:56:24 +0000 (UTC) X-FDA: 81185003088.09.7C6526E Received: from mail-pj1-f74.google.com (mail-pj1-f74.google.com [209.85.216.74]) by imf08.hostedemail.com (Postfix) with ESMTP id 2F73B160028 for ; Thu, 31 Aug 2023 16:56:21 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=zRkLlJYF; spf=pass (imf08.hostedemail.com: domain of 3NMbwZAoKCLEpfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3NMbwZAoKCLEpfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693500982; a=rsa-sha256; cv=none; b=mPDAn+JHBlH/bXYNHBkU9RB3eNHuNWXvxYAWwN2apaF2GMtqbhXX0dfevuWM24tL9kyfab yn5TlihvN07W6SjlhYoUA3j8W/Btp0wTqXS5WPbxRACQp2f2pSaKSLs1R5nyTKCtAXm/J4 cWYbZ70y0txvTfAQRUwd2XIq1W+iEH4= ARC-Authentication-Results: i=1; imf08.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=zRkLlJYF; spf=pass (imf08.hostedemail.com: domain of 3NMbwZAoKCLEpfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com designates 209.85.216.74 as permitted sender) smtp.mailfrom=3NMbwZAoKCLEpfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693500982; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=dJkwDc7Wxmywu9Yo0ZIK2ppmyNcYkDsKDS48R26U4fo=; b=1n+3i8HaAznoaGTDM8YOTx4Gt70dVNUzHV90L1Nj/hCi55mERJROdvIzNXZfMpkcy9LRmS Yjc11VA2DvaOJmtlYkCYxvFAcqOBL90k8bMK8fXnXDARDK/xVCn5yJuYleQ9F9C1xsdKK3 wIiN9aA5W+1F3SwiznbPi6kPNCeLQ/M= Received: by mail-pj1-f74.google.com with SMTP id 98e67ed59e1d1-26d49dd574aso1090856a91.1 for ; Thu, 31 Aug 2023 09:56:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1693500981; x=1694105781; darn=kvack.org; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=dJkwDc7Wxmywu9Yo0ZIK2ppmyNcYkDsKDS48R26U4fo=; b=zRkLlJYFYG57ipyrhkSIPSaqiou5gZt/+2eOf8u/tjgAbYjEVdK3dfkxbi1bUw6QOz LfZXorKuox/MR+BGtcA8EgcVoFBDTHxcwudhpw5+JGVQV4+0rh45dtBkw891PVIVkVnb hE9eoLlIvOB3mQIETYYTirUoB1nN8cSQMx1wFWLLV4MiPmmeHow95God3QmUlfBKNs7Y PF0ZZo8G/itUtHhHOn/RJR7K2VDfVMb7+kvjRCvFB57LNbzUCrDZ159yQPqiYnTX/Pwj 5+SRBy55nrJTT/RoqOVx14MPrsxsmWoG/o95A96KnNRKSdoMBXwwJhdYlwGUgVZGStk9 xz1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693500981; x=1694105781; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=dJkwDc7Wxmywu9Yo0ZIK2ppmyNcYkDsKDS48R26U4fo=; b=d7ayFaHyZ8g7AjdDkiSMYUPsQ/1Za/NpnE8+L75QAU10XqFAp1ifLHF3STilmz8nLL NZwQmuE4fHQAE23D379r8r4u1A0VN5CakN6JPdrkcIX3SJFd0M1naS8Puh42d3xz3WJT iQQjTmqyF/3thsIkbtoIKfq8zlpCp+rFo7F+kWkF87G7MAXSIMdbxNpooOQLcSBpNbOD 5UODAOlpUf2nI7ypDKmyVAGRPrlIO7k+8AFw6ExAWMyr6KHYWLsCDI4oj646xras8QoN i8qebV5J6EuMl09Vt0CjB9yUjJ9G42Se+bPaEuwhJbkLo0Iq0amTBT6PD+lilFiAmB4g k+1A== X-Gm-Message-State: AOJu0Yx1os5FD04lGmathttn37JQJeZhKJhlrAvXDW2JM9ReKeLac/1t GuvH1ifhHOUgwnmViOCDGTBNUEi0c36p5H6a X-Google-Smtp-Source: AGHT+IGBXws1kwasa2qMZ2oV47hWaKQbyN2jJKglhtHqwCQYSWOwpg/OejdryTSoKa2sWR5HEgqm/Qoc3DqRPg8i X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:90a:c68c:b0:268:9cfa:171c with SMTP id n12-20020a17090ac68c00b002689cfa171cmr61794pjt.4.1693500980771; Thu, 31 Aug 2023 09:56:20 -0700 (PDT) Date: Thu, 31 Aug 2023 16:56:10 +0000 In-Reply-To: <20230831165611.2610118-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230831165611.2610118-1-yosryahmed@google.com> X-Mailer: git-send-email 2.42.0.rc2.253.gd59a3bf2b4-goog Message-ID: <20230831165611.2610118-4-yosryahmed@google.com> Subject: [PATCH v4 3/4] mm: memcg: let non-unified root stats flushes help unified flushes From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Ivan Babrou , Tejun Heo , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Waiman Long , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 2F73B160028 X-Stat-Signature: ycekp87qfoeib9bt54urcypgsdr1f57z X-Rspam-User: X-HE-Tag: 1693500981-967942 X-HE-Meta: U2FsdGVkX1/CvCA0OXo0/GskqkQGE0D535QgdJ36vr81qM0vP3W8pVG8UihMr6UcHh4A93Qu/QUK6wp5uADaZ1Uce/rY+7VzOsKtMI3bgbkIuWAUsCzvMJIQ6/KaWqDZVgSXeqKIODBkW+JqrdLBUdjOWxfCXpm96oUR0rbEswwiAFa5S6jSH2TH7LoCq5NwnV1tJVP8pyxQtXPxP+lUcIGL+w2ZW2Tnuisb8WXryCcMptINwR5qLJKknFTyVAK2FCwJqI3pcBUYpKrPKdLXa/38vH5I05O3jU4udNCF4SAgKszNuKOZcIi2FRCylEacvrI1NezC+4Iq4QVudUfD/P1RwRAEmaGlHA7blKBGuA2xy4Cj/vrU2zoO2AlJUnBBzuQ9DUQSWw5h4j1OLGzIO3I662PdxJGwGc/F/8UEj0qnwlIjRWCRtd4gtC5cc5mTrRNrjIaQyE/ahXS5FJ3nSMNFAzceP8nJmYPobFuj616TOTMskxvMh4vlJ72ObRBR0sUujMrTOqvpUwcehtsQJgWvAMpmSmuQ8w01c8rUVQM0kCsSV2rOOflHi4w7PNVp27K22ljcQ97FkTacVyy4NLYWpoQ7FiI5PSmBE2nEBOIrWbb7P/5uZFY6KEUdUFo9jN4DzPNlPU9nO8bpYxPuGe98FcWoFcBjem8VZw/oesCXebdlxvF94SfBmQQ02h6SLoBbVdroWeUqnRidetw43hYxGgRGJ/Wd+lVEK+UOpjb4F7NdJKwBpZEKMrHHftQCvkgV/dCdQOcmV5CINcwseRkkOEXTzi/DZfhZT2quI6gv4J8GxuJ2Z5/pJhDkayGmRuLwytY3eJX+j0yk7JwjhkMSfihl4XoGspFiC+K/2s5/qmlEBu7HxuRBluqg8rfa1tqKslDSsmEC5xkHWOMRIZbi6Suz805Q6RFYe9KFp/3z1RUAwfpfF4V8Z90IcfO5LPCho8FxesUMgcOW0yM yNa6Ptid S/hMutzfPVqwfEXwLlhUewoygPkXJGXW0qyfHJVYLaZD0ln49gLsokTxNwPF2W/UWDYer0egTFUJBMUr4TlAFk89RE2KH+M3xsXQ9/d6NTIyhjUvX8Uh3ull2AmFew5PsJjJhYa/76+uVg5ES639MafJEtD/pbZxtm4lcWod9BzKAwljlBWnDrv3Fj0fjQTyFaSGzz/Bbe9k9MXT2Ui8+1mlz7OiaJjRESYQ/Jmm34oeSaXQDWGIgHK8O72rZ4jndIuzAx2syZM2BOyKcW/oYiBvAy/Eg7zSO84VsDwkJ4zBg907TBGsHqNYXhepxExVkxXTevOhi6eGF62cAmNdYgDuF+84oDO4nvKRy2sjP72kwyF0wp0iIkiKDy2P7vGDujZA7yBBl9VR0gzwZkmb81dDI6TLJhi13cx3llDgXghK8pHZxTts+Rd9ncCzWx7hXTGlCJvYvS2UvqFqOl29YYx1WMHIMcPWMlGKFqH0e0zhree8lQLkw2StcaplkK2VotmRTc3KvTOfiSzClyIRVANYO+I0uhXjWGMpvXE/H+U8sMGWak3AZ4SnqKZZYD6CzIncLZ4MpMUt0jwa1jp5Cp4ulBtUvewA+PCUos77AvLm366p7vkcCzPGITxcpFoxa606jiAtURgGxeFaNwTpPmj/PYySySEZe49lCCBh8stb/ABZmdIIiCUC8R0lcKxoNbVHoyh4ug+4FSJk= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Unified flushing of memcg stats keeps track of the magnitude of pending updates, and only allows a flush if that magnitude exceeds a threshold. It also keeps track of the time at which ratelimited flushing should be allowed as flush_next_time. A non-unified flush on the root memcg has the same effect as a unified flush, so let it help unified flushing by resetting pending updates and kicking flush_next_time forward. Move the logic into the common do_stats_flush() helper, and do it for all root flushes, unified or not. There is a subtle change here, we reset stats_flush_threshold before a flush rather than after a flush. This probably okay because: (a) For flushers: only unified flushers check stats_flush_threshold, and those flushers skip anyway if there is another unified flush ongoing. Having them also skip if there is an ongoing non-unified root flush is actually more consistent. (b) For updaters: Resetting stats_flush_threshold early may lead to more atomic updates of stats_flush_threshold, as we start updating it earlier. This should not be significant in practice because we stop updating stats_flush_threshold when it reaches the threshold anyway. If we start early and stop early, the number of atomic updates remain the same. The only difference is the scenario where we reset stats_flush_threshold early, start doing atomic updates early, and then the periodic flusher kicks in before we reach the threshold. In this case, we will have done more atomic updates. However, since the threshold wasn't reached, then we did not do a lot of updates anyway. Suggested-by: Michal Koutný Signed-off-by: Yosry Ahmed --- mm/memcontrol.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 8c046feeaae7..94d5a6751a9e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -647,6 +647,11 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) */ static void do_stats_flush(struct mem_cgroup *memcg) { + /* for unified flushing, root non-unified flushing can help as well */ + if (mem_cgroup_is_root(memcg)) { + WRITE_ONCE(flush_next_time, jiffies_64 + 2*FLUSH_TIME); + atomic_set(&stats_flush_threshold, 0); + } cgroup_rstat_flush(memcg->css.cgroup); } @@ -665,11 +670,8 @@ static void do_unified_stats_flush(void) atomic_xchg(&stats_unified_flush_ongoing, 1)) return; - WRITE_ONCE(flush_next_time, jiffies_64 + 2*FLUSH_TIME); - do_stats_flush(root_mem_cgroup); - atomic_set(&stats_flush_threshold, 0); atomic_set(&stats_unified_flush_ongoing, 0); }