From patchwork Mon Apr 3 22:03:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13198871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9FEFC76188 for ; Mon, 3 Apr 2023 22:03:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D9DAF6B0071; Mon, 3 Apr 2023 18:03:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D4C076B0074; Mon, 3 Apr 2023 18:03:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C13DC6B0075; Mon, 3 Apr 2023 18:03:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id ADF306B0071 for ; Mon, 3 Apr 2023 18:03:42 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 741E4C0BE6 for ; Mon, 3 Apr 2023 22:03:42 +0000 (UTC) X-FDA: 80641457484.01.ADC8B47 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf22.hostedemail.com (Postfix) with ESMTP id ACF40C0005 for ; Mon, 3 Apr 2023 22:03:40 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=gmD3EzUX; spf=pass (imf22.hostedemail.com: domain of 3O00rZAoKCKoiYcbiKRWONQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3O00rZAoKCKoiYcbiKRWONQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680559420; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=Oudpa17NQV9icHAF4JVW0zNdSUdsNavz+h9FQ4FdLBA=; b=giUjqmgmosM0mZy9Unlq5696xAU6bmCiQonT/NQ3IoP6bPwZBstVEzGRnOaaTSTh3IUPtJ WWXOUozlc8ynaMlNHeNn7aaaxDLLYPcSHJ2d+0IBLQLDdik4hdJUDwGmHDIQr2o3R9m0mF jNAEsi5O1O4cGUGcSAZb6CFbvs1GOfY= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=gmD3EzUX; spf=pass (imf22.hostedemail.com: domain of 3O00rZAoKCKoiYcbiKRWONQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3O00rZAoKCKoiYcbiKRWONQYYQVO.MYWVSXeh-WWUfKMU.YbQ@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680559420; a=rsa-sha256; cv=none; b=hldb8ZXqd67LEjM5bXpCDai/8746hb/+DcvCuv0PHeWD/ce9albgbgXdvKU7Tk4R+EmYqp GUBFTyzfH2iqFYFgHjKrA4f3k+yjBMiTzHXB+1ePFugoSdm8X3ByqGiXRwhbeVWNP/0NBi p7uOHaz+yv9lHk8gRA/ZjF8d9Pqx7cQ= Received: by mail-pl1-f201.google.com with SMTP id s4-20020a170902ea0400b001a1f4137086so18236948plg.14 for ; Mon, 03 Apr 2023 15:03:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680559419; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=Oudpa17NQV9icHAF4JVW0zNdSUdsNavz+h9FQ4FdLBA=; b=gmD3EzUX38OVjaGFgrpka4qXeIu+V1DoGMvDe0eWXeLuVA1CJdcNcSnyr+n+EpIa7P 3MzjOnSmoFXG3JxX7Cs0Wn8GkvgBkLa16nyiWlhViz9Kto3HXhbMSTrrqLF92l4pjOwM Bl/A9PadEA4W2+qgRGAzaoQf4/LsxKQq3zYjYpbh+vXPlDzaJ4S4K1nV++HobeWezLIl 5VMhWDSPBl0YL4XGy/N1tyENDpHWNaUPOK4Xcs933WjQ4R2VnAzEP/q0VSAq3rp+G1sc XYLTFN0b46pCsuwvdg0Q0NbrDy9LoUkmRlggmmc6W32L6LZuB1HPkybaEDU5BBFcgTTW Tt7w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680559419; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=Oudpa17NQV9icHAF4JVW0zNdSUdsNavz+h9FQ4FdLBA=; b=FeLK4PzPJRlZdhTFuEJoDlEx78BeWZUILaheKR5nCizinkGB5mwpRlV7xrZLlDjMns cmorH81eorqHKR01X8qtPyoZObYniyIp7P5SvSBGUN/ZG6Z89iccf+wTAaRTbvHbyYbL L/53utHntx7/LNqIOR2jn4euuG5O7NeaigOHCLu8ahlxFD6K3w+jnQUa7loaXYD+oq7J n4fl+9WthlmXQiblShYlb9uJPL1VE0VDrspd1vl2HPIByvSWKquvYIvpqkW6l4Mo6LQX MzEWVPFahHa8nYcqha67+GD6bNS/fmlphg7X+iigu1QrW8n7HT32dpCKg/sEiY73HAWY tR8w== X-Gm-Message-State: AAQBX9dpAcZA5KOKT3iL0gw+1mnsIWNPybKOUUtYmLId9j/ceRyOQfo6 erVFt3T4fjJWYYiyzaCnRAcA1tJyfifz5mpp X-Google-Smtp-Source: AKy350bKjU+buVnqVou7mkcDlyP7x8rTHIyL9RVPi5Km1Z/toXrwbT5kmEIGX491iMh7wgy+1apIE3kjSSGe4OaY X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:902:9b91:b0:1a1:9015:4d5c with SMTP id y17-20020a1709029b9100b001a190154d5cmr211513plp.3.1680559419448; Mon, 03 Apr 2023 15:03:39 -0700 (PDT) Date: Mon, 3 Apr 2023 22:03:32 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230403220337.443510-1-yosryahmed@google.com> Subject: [PATCH mm-unstable RFC 0/5] cgroup: eliminate atomic rstat From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: ACF40C0005 X-Stat-Signature: 86zod9we4i3cg8qiychofo9j8aedfae6 X-Rspam-User: X-HE-Tag: 1680559420-186869 X-HE-Meta: U2FsdGVkX185FN8xOkwKBKmkzBCArW5gxIoVy82Vz2fDFpUex9NPLoR6QGlfOQMOeH5liMztsGzLNd15fRUy9GfgkH66AGvHueImLqtZMFvPNSeUfwISR1rkdjAtmockdEPv/Xb9WD/DNg4cnPd8svAxojEAfeyfqV87GanNrYpODi81O4yq3g0Vd18kbS5WAAg9LuYBP1iPWsNy8A8iXZmHxViapkQZoZiImn9gc0cXvWsOTeKWdWF1ay1N7NSklSlztyuima1sn8ckVuT2gcAkgivty/nFmcI4yyqyZmUABe9HTg923/+SIkA40W4HReyGbV23SKhsZzjPBG1NR8RFoy3Exz94EQ+Rn6RFbg9F3eu3MABi2WKz4uJXjauBQB643WFv4l/62Egq+zrjGGg7t5ee4bdGKKYuKxlI3c1vPb4JrVMsn29ratshYjsQ5OzEHNXnaN3ZYHXZgnzQqkFqxhYJYsK+vZCUL6uzQ2TxBn9/PP8q6lsHkkf55XSZ8SsG/VB2Ffi+L+IOymbaNBcYTF44iOmSWsZ2FgxbBBM9OByP1NbonLO+/HolWEBTrv5eMCux/gLb3Y/juy3APphs34trU+4IbsTmuQoyEZ0N86Mv/ovVzDI6BfF4+6EAm1lVXY9IJw7mHccPTjk4php+cyY4vyVg8p8+THH9zmd5h182CbqoigzzYlMUY9QY/6EDPoDzCsWe/aFSqVgkxmOiYHQQfoUsKO45lpvsUaJ90gveLrh8IGsf+0XlckfYoGhc8CISD24JKySa1JjeRh2DkY3ypd9dE04svddECg/OdaqryX1GMISEtfv6JcX7SCuG9ja5fVDeoslK+GB2vYmr51NTImHt5oNxMRS/xTOeTgYwqZj9n+fxHUZZM8eQfrqWK+X8Q/48ZnLa4VPj5kwKKkmmZ5p4zVny2i/ufZvlJHpw99HVrQydOJHEpvLVHtMptyAcV7gTMGXlOT5 Qr9YvNk+ 9HKoXw6gWzNg1G1bPOiyzuquJoFxlU97avty1Pfg/WLwr9tjOQYNq7uth17ja2IkBp7V1R9+qxN7y3kQ6IFUJgdWvjhpxaO/OiPUpbZVOmAHZlSXmcfdB+YiKB+pYjmIVM8GQBDORAsSzXL9gEvIvNkvdkicpwAOu/Zi9He2f181a9TJLDOI/35JxZ/IuVHYKXd+8DtdfOfsklsulyudzDMwmBcajP3UiG0wSMhF1tD8E0QJW4axKvZ+vNumI4m1Xq9bhpD6skYJHTyJqZb5SHxsKQaHaiDdYXlMQ3j6XAWCh65qh/icrI5TYLkygfPoWV/iHY6Rpyva0hJ7zDvvlGFn0Ea1tLJv2Zs2MZzWEWAw0sJz5qLY1FgMogJleGvKyTBukWWzmK0kIQgyy0rHcd6S20dSiuN4C3rOuN+12UBR/6jlot9o0PhsCumqA+O99eYgkj4796samyWyAp7OL8fT01iINSkWbWfFexfH36gE2jb4CRo6PFoCHYGUrfe/UfDFtfNb3LFbfJhL58S/AAL/HXml/lH348pfI8VhRlPyE4eU63xSI0mpADAQoWsdYlJE9YVN5uSwUiNusVXOSAB6gRIvWZOS3iaVtmGolXkfvXwaH8u9J49xgaG3z/tROA4HW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000054, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A previous patch series ([1] currently in mm-unstable) changed most atomic rstat flushing contexts to become non-atomic. This was done to avoid an expensive operation that scales with # cgroups and # cpus to happen with irqs disabled and scheduling not permitted. There were two remaining atomic flushing contexts after that series. This series tries to eliminate them as well, eliminating atomic rstat flushing completely. The two remaining atomic flushing contexts are: (a) wb_over_bg_thresh()->mem_cgroup_wb_stats() (b) mem_cgroup_threshold()->mem_cgroup_usage() For (a), flushing needs to be atomic as wb_writeback() calls wb_over_bg_thresh() with a spinlock held. However, it seems like the call to wb_over_bg_thresh() doesn't need to be protected by that spinlock, so this series proposes a refactoring that moves the call outside the lock criticial section and makes the stats flushing in mem_cgroup_wb_stats() non-atomic. For (b), flushing needs to be atomic as mem_cgroup_threshold() is called with irqs disabled. We only flush the stats when calculating the root usage, as it is approximated as the sum of some memcg stats (file, anon, and optionally swap) instead of the conventional page counter. This series proposes changing this calculation to use the global stats instead, eliminating the need for a memcg stat flush. After these 2 contexts are eliminated, we no longer need mem_cgroup_flush_stats_atomic() or cgroup_rstat_flush_atomic(). We can remove them and simplify the code. Yosry Ahmed (5): writeback: move wb_over_bg_thresh() call outside lock section memcg: flush stats non-atomically in mem_cgroup_wb_stats() memcg: calculate root usage from global state memcg: remove mem_cgroup_flush_stats_atomic() cgroup: remove cgroup_rstat_flush_atomic() fs/fs-writeback.c | 16 +++++++---- include/linux/cgroup.h | 1 - include/linux/memcontrol.h | 5 ---- kernel/cgroup/rstat.c | 26 ++++-------------- mm/memcontrol.c | 54 ++++++++------------------------------ 5 files changed, 27 insertions(+), 75 deletions(-)