From patchwork Wed Nov 29 03:21:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13472157 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 679CAC4167B for ; Wed, 29 Nov 2023 03:22:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EF6226B037A; Tue, 28 Nov 2023 22:22:03 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id EA6106B0397; Tue, 28 Nov 2023 22:22:03 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D6E346B0398; Tue, 28 Nov 2023 22:22:03 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id C80B56B037A for ; Tue, 28 Nov 2023 22:22:03 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A0B128028A for ; Wed, 29 Nov 2023 03:22:03 +0000 (UTC) X-FDA: 81509542926.25.621A26D Received: from mail-yw1-f202.google.com (mail-yw1-f202.google.com [209.85.128.202]) by imf19.hostedemail.com (Postfix) with ESMTP id EF3B21A000C for ; Wed, 29 Nov 2023 03:22:01 +0000 (UTC) Authentication-Results: imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=0Nk2UJdn; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf19.hostedemail.com: domain of 3Wa5mZQoKCII4uyx4gnskjmuumrk.iusrot03-ssq1giq.uxm@flex--yosryahmed.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3Wa5mZQoKCII4uyx4gnskjmuumrk.iusrot03-ssq1giq.uxm@flex--yosryahmed.bounces.google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1701228122; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Ky2REY4MxzMO+RxmPId6O0x4sdb+oIV13EhP53U4pHw=; b=0o+f4knwV2LMliCo8dvieJzaOfsDzXTrgPkX3R4q6gybHEa3is56Ou1n7z9wZt4CHgBZ4g ZsQSIT3ch/46odSI55BOIf58G41ttHpjx/jL2UIoXpvY85FLrHWL+EAbIo0VSOdJ0OH7AY AtEgzt0Bwupu6/JV1i7SvbvY0A+asLE= ARC-Authentication-Results: i=1; imf19.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b=0Nk2UJdn; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf19.hostedemail.com: domain of 3Wa5mZQoKCII4uyx4gnskjmuumrk.iusrot03-ssq1giq.uxm@flex--yosryahmed.bounces.google.com designates 209.85.128.202 as permitted sender) smtp.mailfrom=3Wa5mZQoKCII4uyx4gnskjmuumrk.iusrot03-ssq1giq.uxm@flex--yosryahmed.bounces.google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1701228122; a=rsa-sha256; cv=none; b=B5vdlE9cZyKfq1obYIzS6poApVYFxBNKWcUkIu1Bikmq7dAukrEfJhrKXE02pU5jEiuMLZ VW/HHzlr/W6BvsKCHohhLAC4NrQESXhyXoeUizNMptotYaQPYFhehNSD2V4tcmZvNtmL9B rlvoli54JMeUiFCR/fib4joOPcGTJXg= Received: by mail-yw1-f202.google.com with SMTP id 00721157ae682-5d04540d5aaso46049127b3.1 for ; Tue, 28 Nov 2023 19:22:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1701228121; x=1701832921; darn=kvack.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=Ky2REY4MxzMO+RxmPId6O0x4sdb+oIV13EhP53U4pHw=; b=0Nk2UJdnT46yVGrIwYIyau9QZuHzMDmK7llMKQY2234F1xetVmKRgMYD3wmlva8IQt dUyiSKGiP59IsM23AppA7To0x9EcbYwL5OnFLRTFvDQQ2rpkbw0aenLTIWvpbZ8/4kH4 o/KQwTAKRql33llC4iz7m0GX/nhcnAhIqeMaDfT0DXQd+8jcfV2TCPRBmE0jrwsQusck hFnsJCOTudpwHzTsZ653JJoPVzAN15ctqKW6ZNZx1YbFTqzMF5Qlc0JeckrTViXAXgu/ WZXhEkr1JVIyh3m/56gxTBduVK6H7hfsHuf4tIVB1abvFPSHfCYnByV/dbOf3nr6K/C0 tJVw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701228121; x=1701832921; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Ky2REY4MxzMO+RxmPId6O0x4sdb+oIV13EhP53U4pHw=; b=RYFgN537r19PT652zlUmpCStSG94lA06f934/lS3QYL332G8QPIUpD0aJDxwReV6pP s6dHBuk30Wher8Mf8se+2gEQrhDo9wtHxKDXB9luED6RuODKrpGPKMYO3zxiXh1X0xEy FVFJATtpHtNAJb0Gvx2lme0gCI/5pQPbB3Sb6T7YbjvfTsd0JZFD/A7EXRzudDOXk457 o9T/cmX30hzMGQker49/XIVizxRVteGvRqRsVISbEPL4Xt+MVqmUBPftUPEZ5anP4/1y MbsnQHqsLB/s3Tm05tsVdKRrfYGJkOI/VakAe0JzI13yk2zJMbMjo7UR0QRel24/3sFk pqtA== X-Gm-Message-State: AOJu0YyePgui1qGeYOnXuh9ZwxEftZgd2wjJJDTYUuYFXvyq178Qe1+t RrrSWsbFTXKhI4BwGzY3Z6oPeN+frzknJneB X-Google-Smtp-Source: AGHT+IGouOmFvlCqrx0oX4pFpmBSxSf6/wqNiY3Cuy+K3AoX5MxSxr91W70yChZlk6piYooZGVVAx0PEPE6lQeE2 X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:20:ed76:c0a8:29b4]) (user=yosryahmed job=sendgmr) by 2002:a05:690c:2e10:b0:5cd:c47d:d89f with SMTP id et16-20020a05690c2e1000b005cdc47dd89fmr499535ywb.5.1701228121058; Tue, 28 Nov 2023 19:22:01 -0800 (PST) Date: Wed, 29 Nov 2023 03:21:48 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.43.0.rc1.413.gea7ed67945-goog Message-ID: <20231129032154.3710765-1-yosryahmed@google.com> Subject: [mm-unstable v4 0/5] mm: memcg: subtree stats flushing and thresholds From: Yosry Ahmed To: Andrew Morton Cc: Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Ivan Babrou , Tejun Heo , " =?utf-8?q?Michal_Koutn=C3=BD?= " , Waiman Long , kernel-team@cloudflare.com, Wei Xu , Greg Thelen , Domenico Cerasuolo , linux-mm@kvack.org, cgroups@vger.kernel.org, linux-kernel@vger.kernel.org, Yosry Ahmed X-Rspamd-Queue-Id: EF3B21A000C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: 7658opmgw656zu7hs6r8nw9km7fyp4sy X-HE-Tag: 1701228121-756767 X-HE-Meta: U2FsdGVkX1/EJfdg4b/fJAAJ9D+uLK9Hgb1S3rLUqms7OxYjoKSsOtJESVgshhYeGDcPJE5ANSMTb6Fqu8DXWNbfEYdV81yMDZdR8pq5AB41BepV1InqR9Rz9MrwT2ixHi+zBX4/WMTpWri+n0LOTB2HDBm9mCKV0toF42xmN8RtfSEoc+vq2NFfO4SRV8MscZvqBSTz19KKhOtYYahuMWUcyq77qu4Cs552VWOMFtl7t1ST6rstw/okD9XqourNvBDWJLcB8qGFxnsgd7axHhD4SSxoqQuIZj3bNy/h0JMW4XEyYtFhsZ/v3rd7EZBb/Zw4ZchHBJHp+l4xoiPFJ2pf8oujrF2t0LwWBI5aQ8R+uNmSm5dsTdMl4xYPptIp3VPsNrUppu1+Iw99E1iBrkZULxv2Sx31CgDvfjEcYbkoFJlzMtQAdrfBkWkA0k+I088zhglrkBXufCJesHyIoYTwez5Jig6nkWWxRkGYLf5TRmHvo8E88Ye5LAvk0/1wCGz/BieBG3Kso9lcWVoBAcGq7FdjvQBNxoSUP/XxWja3VUQCnRznPI32k0xSs3GmEGnUeo0dU/sp/O0MBcDLl/Ck7HkOxaB5NYloxw41hY3xFvL9PR/armwvxOPnyOZM+YCaHwWC6ltPeNeMo3TeaFPJPchqrXLg427SmH31WlahXKkFFI+PLusHl1lUKF/CKF9A1v9h4F/e6kS2GIlxiZXdbmOWizmochogpAWg1WQr2scgYE76NqEf6STQTwmrYifKuN1ncfHObVoijsa5SA53re5DXscnbTWFK4C8TnvszdcYN7xzuHmz60zfHj2GfgLXBkdNdC3bRrOFMJNDMWNUFacsBKqMZwXkEQytvFR8ifGN7ucHrhKqltSwdXUV2uE3vcG8gbvAWUTntvfO+y9Em2zvmn69lzW6DVBF9Z3SxWZpc7uCq4sVCdSNR4mpf5qGM2kEckS/R+mxs+Z zGm3GkcO 5X9ePey8B/lCKg6B+q4pMskn54F2bThKs5E+kXz4qF/isWa3YuYYLYBnJNOxvDGLoJCBci6B400uQptJ7jBOlPwWQzrxs4V4aeVDwbkYtiVcjVnDc3mHyUszJpRG1YeutHflN6MYv8FVzcE2l5Cm/hzVcBr6otND4BvXqLuFksFW+fqB89+WbZSfPeTt8Smfy5rNcAWcf3gOv49TcqqSYBSPj028yFoYmn6LN5vAXbFsZZj4fAFmQl0l96B7pf6I8z7BbKF06EyfRG76dlXbIIAGWWRHSiVxLEwxGw1Zso2Rg1HMbJ90AW4d0HAMOsG5bVt9iSPhYikmQDTdi4YHCYY+NGRIaYV2s11WMTbJ4Vnp49XsoJ8LKyyOi5XJAC63LyQT6UzbAdd+f6Jfe7S3bl1nhG9WBVyxXGQ+Fm6sbSNKWQFJuowqejA06t2m0HVDNNuQusW430l70Z6FNR2nB+46Oa6r3XeZbNyzm5oLwNdx4RvM+OQJxcV35UUddFFhWZRos/Cn9p+b9TmebD32vq15gFITE32gqWGXxBjnbne2GNJkwgftLcM9gWChTnpi1uy4gJNdQ86GTeQgoduO4lCswX646DtA1Ao4OkrI7aajikl3pPrMZpq4iSvDOhK9FrdOrqDvM+VhHcJouZmxLiSxx+TNKVBg4azhyFdKB1I1m8/57+LTUCEK0ptE3Ryq6kmO3+DMp8uixU82cCxqufKoTZwMvsIDm8hGMcGVg+EzwnUAkXbCZuT/wUA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: This series attempts to address shortages in today's approach for memcg stats flushing, namely occasionally stale or expensive stat reads. The series does so by changing the threshold that we use to decide whether to trigger a flush to be per memcg instead of global (patch 3), and then changing flushing to be per memcg (i.e. subtree flushes) instead of global (patch 5). Patch 3 & 5 are the core of the series, and they include more details and testing results. The rest are either cleanups or prep work. This series replaces the "memcg: more sophisticated stats flushing" series [1], which also replaces another series, in a long list of attempts to improve memcg stats flushing. It is not a new version of the same patchset as it is a completely different approach. This is based on collected feedback from discussions on lkml in all previous attempts. Hopefully, this is the final attempt. There was a reported regression in v2 [2] for will-it-scale::fallocate benchmark. I believe this regression should not affect production workloads. This specific benchmark is allocating and freeing memory (using fallocate/ftruncate) at a rate that is much faster to make actual use of the memory. Testing this series on 100+ machines running production workloads did not show any practical regressions in page fault latency or allocation latency, but it showed great improvements in stats read time. I do not have numbers about the exact improvements for this series, but combined with another optimization for cgroup v1 [3] we see 5-10x improvements. A significant chunk of that is coming from the cgroup v1 optimization, but this series also made an improvement as reported by Domenico [4]. v3 -> v4: - Rebased on top of mm-unstable + "workload-specific and memory pressure-driven zswap writeback" series to fix conflicts [5]. v3: https://lore.kernel.org/all/20231116022411.2250072-1-yosryahmed@google.com/ [1]https://lore.kernel.org/lkml/20230913073846.1528938-1-yosryahmed@google.com/ [2]https://lore.kernel.org/lkml/202310202303.c68e7639-oliver.sang@intel.com/ [3]https://lore.kernel.org/lkml/20230803185046.1385770-1-yosryahmed@google.com/ [4]https://lore.kernel.org/lkml/CAFYChMv_kv_KXOMRkrmTN-7MrfgBHMcK3YXv0dPYEL7nK77e2A@mail.gmail.com/ [5]https://lore.kernel.org/all/20231127234600.2971029-1-nphamcs@gmail.com/ Yosry Ahmed (5): mm: memcg: change flush_next_time to flush_last_time mm: memcg: move vmstats structs definition above flushing code mm: memcg: make stats flushing threshold per-memcg mm: workingset: move the stats flush into workingset_test_recent() mm: memcg: restore subtree stats flushing include/linux/memcontrol.h | 8 +- mm/memcontrol.c | 272 +++++++++++++++++++++---------------- mm/vmscan.c | 2 +- mm/workingset.c | 42 ++++-- 4 files changed, 188 insertions(+), 136 deletions(-) Tested-by: Bagas Sanjaya