From patchwork Mon Aug 28 23:33:17 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13368410 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72062C83F14 for ; Mon, 28 Aug 2023 23:33:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1644328002C; Mon, 28 Aug 2023 19:33:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 117298E001E; Mon, 28 Aug 2023 19:33:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED53428002C; Mon, 28 Aug 2023 19:33:30 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id D46C98E001E for ; Mon, 28 Aug 2023 19:33:30 -0400 (EDT) Received: from smtpin06.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id A416C1606A0 for ; Mon, 28 Aug 2023 23:33:30 +0000 (UTC) X-FDA: 81175117380.06.70B8A59 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) by imf13.hostedemail.com (Postfix) with ESMTP id DFE1520008 for ; Mon, 28 Aug 2023 23:33:28 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=BaJDLeW2; spf=pass (imf13.hostedemail.com: domain of 3xy7tZAoKCAg6w0z6ipumlowwotm.kwutqv25-uus3iks.wzo@flex--yosryahmed.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=3xy7tZAoKCAg6w0z6ipumlowwotm.kwutqv25-uus3iks.wzo@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1693265608; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=cdnModp3UHPesHrMiYbasA77T6X4L8s34myJUUsNDqw=; b=Q8+p+q5GppDzkTjavQVaUqpm2BiAURfbVV2WVf1Az4I6kw82aiVIFCnKXdjTcpp3rgKHtI bnj1P5CN7ncwX9HyO1E5zoZJ3kOmjZKpnyBw1k9RbZDCRJX7y7b2fTxbqY4JtXNvfk18zT L6OCVSn9ywjy/iXxI3AZB0qXdpAwWic= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1693265608; a=rsa-sha256; cv=none; b=YYNjEhGAZjBNJzZb3E3WenaJyW5qFR9Cte1eXLewJqn5VAlX+0JQuOtHmk/MQz01kc/y+i Bv0pXmfLr9gtR3JG79gXBRNn4+2I/CDlU9K+Ne8gioGi9nhJZn86eN8P6uD6GPj3WQ2KVk /RQVL0c+ZYL3FX5J7XzcmO+M3Y77/JY= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=BaJDLeW2; spf=pass (imf13.hostedemail.com: domain of 3xy7tZAoKCAg6w0z6ipumlowwotm.kwutqv25-uus3iks.wzo@flex--yosryahmed.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=3xy7tZAoKCAg6w0z6ipumlowwotm.kwutqv25-uus3iks.wzo@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pf1-f202.google.com with SMTP id d2e1a72fcca58-68bf02547dbso4650472b3a.1 for ; Mon, 28 Aug 2023 16:33:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1693265608; x=1693870408; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:from:to:cc:subject:date:message-id :reply-to; bh=cdnModp3UHPesHrMiYbasA77T6X4L8s34myJUUsNDqw=; b=BaJDLeW20NuEFec2oAO68NEPY8XNBrqkpeisxk5dZbxgnWHQ0r2PhoENHFifb29a85 wkcQ36f5pboCfp5upZdXeouVQ/0c7EuGiOlftOTQ7QED4HRVXJaE8f3Wet7A9n2Rj856 vSFPpwUufWhdTYEK9e0YxtG9T7V5niVLg1OGtXwve+hU15y1awhfiS+ph4sxssMScLfY 3FyWbOnA+EFg7ZsT4U1Ke97tXPWCjqO0X2CsZjdqw14iDY0ncsKKN+1F4C/05uDjI0Yu Vcck/XfFVgXP+e2uByfABv9Rt2GwqKStdZUxd2y8AJciGF9zoxxWBpNJ74POirqQMS0t +/dQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693265608; x=1693870408; h=content-transfer-encoding:cc:to:from:subject:message-id:references :mime-version:in-reply-to:date:x-gm-message-state:from:to:cc:subject :date:message-id:reply-to; bh=cdnModp3UHPesHrMiYbasA77T6X4L8s34myJUUsNDqw=; b=cbpp+XToWyT/6bQC2gvfJCyQa6y3arJHHGi+nHfErHHfmaH442a9dki97rgXgLnMCc SFVLxH190Zmq3yo9G5C2TKrsnm9/UIuUmWj2TwPuW6ckvOgW6a5RXB+Mi0eekh1ydFDt j9UvKCxZhY0d4CClleQ0GHgbB5QZg8Lnx0fsTbMyjrh10PEg2EG3+5xl79Szofji8SiK hu0tZEs7cWarO1lcKgJw68+jNXpp1nU8amV3RXZmgmoOVYYruEVD+xheRno8bVoYTm4x YcD0pAVM8fmhzcyUlFjgzXhr49b+8uK4HumrCLI8wRfN/gBMW67OUlu205Da4OSVbGzu umDw== X-Gm-Message-State: AOJu0Yz6zssv4GTHHR31S3o+YloXQ6LVS8UPhEdoPSpu9Ai4VJbxOWpP YWQ4wFmQwT6/wYNaL1yS1Vsgh3xY/RsZHZUE X-Google-Smtp-Source: AGHT+IHAq4VgXKo8P6aYPET0HHb7Rp2LuZeLVocuW54MrcmyDai0zFV6jlRJW28aJMih5PpmlindTt5puuPdOpLx X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:13a7:b0:68a:5467:9974 with SMTP id t39-20020a056a0013a700b0068a54679974mr8979137pfg.0.1693265607909; Mon, 28 Aug 2023 16:33:27 -0700 (PDT) Date: Mon, 28 Aug 2023 23:33:17 +0000 In-Reply-To: <20230828233319.340712-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230828233319.340712-1-yosryahmed@google.com> X-Mailer: git-send-email 2.42.0.rc2.253.gd59a3bf2b4-goog Message-ID: <20230828233319.340712-4-yosryahmed@google.com> Subject: [PATCH v2 3/4] mm: memcg: let non-unified root stats flushes help unified flushes From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Ivan Babrou , Tejun Heo , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Waiman Long , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: DFE1520008 X-Rspam-User: X-Stat-Signature: hbks9pgu37dxdfxg9ftpbztmnuh5pxau X-Rspamd-Server: rspam03 X-HE-Tag: 1693265608-391743 X-HE-Meta: U2FsdGVkX19eL/TOR560ENK8pzXBiojRF73zZjvmtY7XoOA4uQk20Ox3bxCPy9WaskxweHzhkKwazS03NYRYYFy3xIrhDp6gh4sTmsCV4ZafFoyCq0x83EJp3aPHn9CLU5kIq6UPiW2pAnUmD3CEi4tG1Fph/Cq0HtVymAUDg/671Qoo20isD9Uspul4fgSiXT+2hiWLgdKuapP1ylY3fyzoPrr1AzakDzzGu2BO/Uu391i+Z1gHA/tQIxSD4kTr3ir7YmgQfjfPChQ6hHOiIzeVf/dgS5jClTjrMpBvY39Wl/7KyFI291MgCeKNuPii12rr30I1PiE1JKTfK8OVtyBHahWAe6kKsfN58q3mWajusCtBFOSVCmWIU5xSnXxTcon6Tsr0tj5zi9hvVdRfQaSq6Y/QhQBqpL28QLpus4sPVQufFps5ISQXXVVaQIooXOCb2pqX//QwwoLTRwbEIgmDutAgcLsTNvK6UdqmLO25ZRuBdAju5Xp7x5PNiBFgvTHCwsWnzGJDrKz3DCqxWxoO2t5AN7AlRSyt/W4OCKkiPi1DHuK43RBUsK/0A2+gcCGhoTm9US1HcLt55BOfjjtiWnG28tmZmhZj5H93wnTfJppbVM66LUFinkkXUDFPMKjT33ewZyvdozWjGFMxa9vw5wKCzccFLfWiyuIb2tT+qh4gnffItrq9E4+R/Y9M/8Mvq9ysh0NLkkWuO/iFrLuNquI1CvRHGvbUCXFM83wMm6l80GnZyeiSqnYaP0eWqzKysLgbotrkHYOTJvE50BSLu+wW3F9V+RV/RF8QGViOrJqx5XmpY7rKC1tt7cxHCwapoQBvfzJgzFBemU26X4+vRkWVOfiMfsWnjsFaRMvSnLwtplVO0w5UYPSjEI6iTSKWInV0kimwGtdjGYCrL27+05a1rirAnRKBtUzBAqYtK4K3TejoIjPar6hrjZ7rsD0Apql1cDr3XfxFkRB EWv1coFy OsjEP/+jMjtfrRHBDfcDgNc/XEjy+0sGyTNsWGKj2YMeJCN7SV7gDzBBXDWHh4kBiPwjMVlbzb0e5pwAecP0+h7MHQxYALorMVFyokyWLC7vpHihbGei0/TvILowwWvOx7goyFIkldYFhD665rTEZdhftYe5SMRagNRzi0sb2mIexUGoj5rP5lMVUTdtwWjCwtqzmth6kB9enWkM1LrZf4OOYOUokeiEmqb0MSeCdBGA4WzOJZFqKY4fpKN3rFpt0J7kZkiONsjI77cWFaj8IgY2+IKP0Q5GpvhJoLJ7q+tN2pBCsezkaOsNXbEQpBxhlF0Xhcsc2vt5j3NZy8If7qwO5j673m0VeaTGyi5njhoz6HokAmg/a7eKCborBRUPtiEKPWi4seWKpJrEaYWMQG14O3JP2cDO9iIIBuq74QfPf+a/SxPAU4xdvknsG53ZzD4q9VAfN6uiy4dJ23cJZWku9/ziX93AVi9lCF2kUnqenRwjtnKIzxj2d3bM5/gg81cHH0xun+ZpvCyz9bvaN/7gtPyWuB6sVX7JVkDycVfEbiWBsm0RAP6iNwtC4iy4vWW1+GMZ9tdqE/sn85anCoxq/70S9HLH2XLN8bhTxNatO58oXwlNWzU3SwnEIv2coSgZ5B6LrYBh/VnT1NZ03ur0afRd4MAYzwXJj4V8S0Q3pzoI9fld0RguQDIzW0yFlA9i6 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Unified flushing of memcg stats keeps track of the magnitude of pending updates, and only allows a flush if that magnitude exceeds a threshold. It also keeps track of the time at which ratelimited flushing should be allowed as flush_next_time. A non-unified flush on the root memcg has the same effect as a unified flush, so let it help unified flushing by resetting pending updates and kicking flush_next_time forward. Move the logic into the common do_stats_flush() helper, and do it for all root flushes, unified or not. There is a subtle change here, we reset stats_flush_threshold before a flush rather than after a flush. This probably okay because: (a) For flushers: only unified flushers check stats_flush_threshold, and those flushers skip anyway if there is another unified flush ongoing. Having them also skip if there is an ongoing non-unified root flush is actually more consistent. (b) For updaters: Resetting stats_flush_threshold early may lead to more atomic updates of stats_flush_threshold, as we start updating it earlier. This should not be significant in practice because we stop updating stats_flush_threshold when it reaches the threshold anyway. If we start early and stop early, the number of atomic updates remain the same. The only difference is the scenario where we reset stats_flush_threshold early, start doing atomic updates early, and then the periodic flusher kicks in before we reach the threshold. In this case, we will have done more atomic updates. However, since the threshold wasn't reached, then we did not do a lot of updates anyway. Suggested-by: Michal Koutný Signed-off-by: Yosry Ahmed --- mm/memcontrol.c | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/mm/memcontrol.c b/mm/memcontrol.c index 90f08b35fa77..f3716478bf4e 100644 --- a/mm/memcontrol.c +++ b/mm/memcontrol.c @@ -647,6 +647,11 @@ static inline void memcg_rstat_updated(struct mem_cgroup *memcg, int val) */ static void do_stats_flush(struct mem_cgroup *memcg) { + /* for unified flushing, root non-unified flushing can help as well */ + if (mem_cgroup_is_root(memcg)) { + WRITE_ONCE(flush_next_time, jiffies_64 + 2*FLUSH_TIME); + atomic_set(&stats_flush_threshold, 0); + } cgroup_rstat_flush(memcg->css.cgroup); } @@ -665,11 +670,7 @@ static void do_unified_stats_flush(void) atomic_xchg(&stats_flush_ongoing, 1)) return; - WRITE_ONCE(flush_next_time, jiffies_64 + 2*FLUSH_TIME); - do_stats_flush(root_mem_cgroup); - - atomic_set(&stats_flush_threshold, 0); atomic_set(&stats_flush_ongoing, 0); }