From patchwork Fri Apr 21 17:40:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13220539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E85BDC7618E for ; Fri, 21 Apr 2023 17:40:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6113C6B0071; Fri, 21 Apr 2023 13:40:26 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 5C2206B0072; Fri, 21 Apr 2023 13:40:26 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 489F36B0074; Fri, 21 Apr 2023 13:40:26 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 3862D6B0071 for ; Fri, 21 Apr 2023 13:40:26 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 002BE1407FC for ; Fri, 21 Apr 2023 17:40:25 +0000 (UTC) X-FDA: 80706112410.20.5EE7477 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf01.hostedemail.com (Postfix) with ESMTP id 4CE8140010 for ; Fri, 21 Apr 2023 17:40:24 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=RbrkYV0C; spf=pass (imf01.hostedemail.com: domain of 3h8pCZAoKCFAG6A9Gsz4wvy66y3w.u64305CF-442Dsu2.69y@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3h8pCZAoKCFAG6A9Gsz4wvy66y3w.u64305CF-442Dsu2.69y@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1682098824; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=PreKFJTa4Nhy/d8G1S1pWlqvl89TyRSt3luKcVUPmzw=; b=nO5Rk1FKzSzOD4kXTsXuzXMkPNvxJmESfY9oECIk2WTdgOJ3SF1hKmSE31369CaBjVVh/b AuJbYcDKtMjUGfziYXWv1EXE6CHp3IdFV2F1Gd6boc7IpnReCURlpOgXPdD5rWFuVZ9pYN iczfCJeJbGOLDgaEZfQqV/4/xIOVYQY= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b=RbrkYV0C; spf=pass (imf01.hostedemail.com: domain of 3h8pCZAoKCFAG6A9Gsz4wvy66y3w.u64305CF-442Dsu2.69y@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3h8pCZAoKCFAG6A9Gsz4wvy66y3w.u64305CF-442Dsu2.69y@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1682098824; a=rsa-sha256; cv=none; b=HtdtoyaJ6PNqA97YIUclzfmWWuMIz+CWIvNdRkhp5T62MSfvQ6PiZXjckfzVCWXJ69CHq1 pQ7FpppFWwXigCPdh3EBmU3dABw8Vwn7NWjzaMDfhu6YPd7sdARRvKJ1AiSXIzqdFM8Smt k8r+Ar7fRvAgPn1MMTo4PWVeNlLWOwA= Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-1a66bd44f6dso16018085ad.0 for ; Fri, 21 Apr 2023 10:40:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1682098823; x=1684690823; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=PreKFJTa4Nhy/d8G1S1pWlqvl89TyRSt3luKcVUPmzw=; b=RbrkYV0CJdmfBIbmUIvZnVKhJQyLH4CYaft3mk2yfKLvyJZF8dDaaqVVnx3eiK4Ab7 lZDDw30q1ppsBNzI772XXB/FRji5/R8TeuWfOOnXpJHy3+Y7JyutQQ2Gj19sY9tbj8iD l3vp+8XcixoSCatriNQ/fKYwuCkpY6rEOUk7vfFOeJNxNuSbxeoNvs3hSBNc92fiAIj4 a4voAS83/3J7lvlMPpBscMu6mKr9RPsusaetGLt5BbQVCI9eFbxjUyZ5WoZ+y39zt/1d 13legztw2c8S9smKxFGhZzKMk7rQhrAr+Y1NpiTukgQZJEDzm31H36UsudJPK7jznpxJ jqQw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1682098823; x=1684690823; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=PreKFJTa4Nhy/d8G1S1pWlqvl89TyRSt3luKcVUPmzw=; b=NYPMllZ+CsnJF8kzDwejbUcAf36v/2uwygLxmVhlfnV5sSLAb9DAK8ufzS1QACfVFi ka5fQQWr+vsAVgl4sw0Kp4v69s0pFYOLIocE8rvujjPElzpkESnhzzMT089hf20669Jj N2aNVtEuQ7dsM/asvrxXzL8oj1guQEcQBLTm4dNQuv8Oey1T7rb1SApsA6X3OxCy+0EY rkXvd/uTmv6stOhYB5zS1ZS9wTMv6ao6pMogPMv5jaX64DzOZIryVJTVBXZnoh3YD+os 6qa5MswK8x/cIM62+ciX4XIpDySGWV+5uRvdgEhJ0VVx3EBzZVodbJeUYcnqIZ3DbYnS +ZtA== X-Gm-Message-State: AAQBX9eRrJEWBofiSf/jQUHhWBkD1irwFZnVp5pMD5XaHtjUa8k54gxm m5pt5q6liLX5Bk1/RGKiFrlN3z9vU8+V36nA X-Google-Smtp-Source: AKy350ZlnZj0cxHrv/Wjk1pQ1CQV0zgSpBwIct8jNYsZLeVb1GgBjLCU23xFOdjM3zTGe34xq+HU5r4O5Po/289Y X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a17:902:f302:b0:1a2:6e4d:782c with SMTP id c2-20020a170902f30200b001a26e4d782cmr1851742ple.13.1682098823035; Fri, 21 Apr 2023 10:40:23 -0700 (PDT) Date: Fri, 21 Apr 2023 17:40:15 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.0.634.g4ca3ef3211-goog Message-ID: <20230421174020.2994750-1-yosryahmed@google.com> Subject: [PATCH v5 0/5] cgroup: eliminate atomic rstat flushing From: Yosry Ahmed To: Alexander Viro , Christian Brauner , Johannes Weiner , Michal Hocko , Roman Gushchin , Shakeel Butt , Muchun Song , Andrew Morton , Tejun Heo Cc: Jan Kara , Jens Axboe , " =?utf-8?q?Mich?= =?utf-8?q?al_Koutn=C3=BD?= " , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 4CE8140010 X-Stat-Signature: z8yymtskwdaj7kk8z4nfniqztk8kor9e X-Rspam-User: X-HE-Tag: 1682098824-667790 X-HE-Meta: U2FsdGVkX1/tMgciSEo1IE1T+sesGdFaLMIEMX8rU+oKGC9HlybwWihuPfjWt6qQHJql4Vsww0s1fXL3eJWtvzivvV2GKdSjfwmoQkIC9UCKl3cdrGfj2VLjoeatF8Gi8tOPjycbcAhlCCYbwDQ+cC9sCNz4X4iTNBSzE1TiUUxIGo6FYG/P2eBd9nyJL2b+qK/0NsokQ3zfyPQFWQGUCRvLF55eizjzgir+F027Vl3uu+gudEo/rCCXaXjQZp/nZkoy9d0Z7kpGN1nBOeWnHTAVP7QfUclXOPYFp3X1UIpibufOlqckS4ZmYE9A4L/1MVfphKAhhIB9TwvvSddEtJksilV0idPeALhZC/YgUd9PCAs9sdssqGme+EfaAJJeaAvFVKICdr53anesYP7nFoABCj5885oYmE9/RclyYHdxKZb1LAQ56XXrsOpvo4uQc606wsAf+z5dk9aOYKgm42W5vsXL6gKYqXBXF5YEoR0nu2bUY4nESOKw2QVxCTHP/E1KnByKoQY5i1DvJfJrpxSbnYntfHdGbMvcTIIrkc0M7bHeDYlHP+i7FvJji+qCcBN3z9NgN3GOY/t+9gmCDL7ihSBdpMNd+SxMwcEkeOPxuQv0AYt0kQFbo/S4ZOQGMKGAkP/8HEan6W+ipDse/Zjnep0wSeJx99Iv1YdYsrx/c7s7aMp8K5iSHeWDmj4rRq5TggRa1vJ77sPlArvAy5q/78UzFkiCQD9GT4fc9XweLyHOE4/96LEptyLt8LeF3GGcWu6FqOwKvteK8oQLM63QWRhTFzwSRsg4e5gw5koOA4zkNWgZSarTac8rF04eDGQwtvMgX8sBE1Sd9JCoOjjLZUnTBxgwIMTLEqa41mdREwBGfGZ9J5Qb4On8QBKniTAQrObg3RkcUxyV8RECMo549A5JYIvL3IyFJZnO+Kxj2W5b7AfwfM26ILrCYCUq6V2VjOHRbNK4X2J0YFs YFVY8KEw Flt9Lhn3TvyjaUJypSY07ye3b5kxbCVsMWViiK8hRyxora7mkW3D7A5BNW00/5ZaNjAA6x5xD/cYP0y4OC524zZyNmGi1hB0mwXbzqZwsMyrLpatAGJkFQAYanF72ol67+z1Wr9yzj/5cbF2H9CXN0fECl35Eb9KGNjWrRggmMPVUu5QLEP4H2CmypQ0Q6JycpVoUy5FAiXiAWjVDsaRjDxOMHiWRrWsjOnozaIwZMEZPZvjlCkhLVcV1MisyPzpkCXIjdboaxNYljusUIGVZdoDS324/t0m5xuKwA996zJh2vFndGoBS1ccfiTH+2ib7dX1qCGXPmxYc07bjma7VEQeEhyYF425ZTN9AHV7Dt0Mscpuh2XHTl3HKv4rhiHIqGE/blhISqLuUUzualaO7S7Nsgx5JK9lrF7+dry/vmp8PE7GRx4zq0SrQQX/Cmml614cj0ZAZEy/Pq/gQV0+fTRnhXQbeUnPgITfMyW9nm0xSjMfnYLWf7rgJp6JJOUiajs5efvOVZ/4rH+iPDIOFzAjr1oxTT834EzepKkPS80b+uqsyhZBfJVng7s072Ii+olDoMeT1vEhrbXfQnWc55qoFml51ECcr36ajs3xVU3eLG41bIwFbG6xkiqZ4jLylv501ptwC+Fxpt20ulXKrbsH299V8ff4WZsnIfBNOeuOi3g/1EgKlyPi+8xwG0dfgBXcP X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A previous patch series ([1] currently in mm-stable) changed most atomic rstat flushing contexts to become non-atomic. This was done to avoid an expensive operation that scales with # cgroups and # cpus to happen with irqs disabled and scheduling not permitted. There were two remaining atomic flushing contexts after that series. This series tries to eliminate them as well, eliminating atomic rstat flushing completely. The two remaining atomic flushing contexts are: (a) wb_over_bg_thresh()->mem_cgroup_wb_stats() (b) mem_cgroup_threshold()->mem_cgroup_usage() For (a), flushing needs to be atomic as wb_writeback() calls wb_over_bg_thresh() with a spinlock held. However, it seems like the call to wb_over_bg_thresh() doesn't need to be protected by that spinlock, so this series proposes a refactoring that moves the call outside the lock criticial section and makes the stats flushing in mem_cgroup_wb_stats() non-atomic. For (b), flushing needs to be atomic as mem_cgroup_threshold() is called with irqs disabled. We only flush the stats when calculating the root usage, as it is approximated as the sum of some memcg stats (file, anon, and optionally swap) instead of the conventional page counter. This series proposes changing this calculation to use the global stats instead, eliminating the need for a memcg stat flush. After these 2 contexts are eliminated, we no longer need mem_cgroup_flush_stats_atomic() or cgroup_rstat_flush_atomic(). We can remove them and simplify the code. [1] https://lore.kernel.org/linux-mm/20230330191801.1967435-1-yosryahmed@google.com/ RFC -> v1: - Collected R-b's and A-b's (Thanks everyone!). - Rebased onto mm-stable. - Cosmetic changes to commit logs. RFC: https://lore.kernel.org/linux-mm/20230403220337.443510-1-yosryahmed@google.com/ Yosry Ahmed (5): writeback: move wb_over_bg_thresh() call outside lock section memcg: flush stats non-atomically in mem_cgroup_wb_stats() memcg: calculate root usage from global state memcg: remove mem_cgroup_flush_stats_atomic() cgroup: remove cgroup_rstat_flush_atomic() fs/fs-writeback.c | 16 +++++++---- include/linux/cgroup.h | 1 - include/linux/memcontrol.h | 5 ---- kernel/cgroup/rstat.c | 26 ++++-------------- mm/memcontrol.c | 54 ++++++++------------------------------ 5 files changed, 27 insertions(+), 75 deletions(-)