From patchwork Tue Apr 4 00:13:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13198931 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 396F5C761AF for ; Tue, 4 Apr 2023 00:14:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AF0FC6B0074; Mon, 3 Apr 2023 20:14:01 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A7B6F900002; Mon, 3 Apr 2023 20:14:01 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A5F56B0078; Mon, 3 Apr 2023 20:14:01 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 7AF4C6B0074 for ; Mon, 3 Apr 2023 20:14:01 -0400 (EDT) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 24F5F160D82 for ; Tue, 4 Apr 2023 00:14:01 +0000 (UTC) X-FDA: 80641785882.12.03EBE15 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) by imf29.hostedemail.com (Postfix) with ESMTP id 5680C120012 for ; Tue, 4 Apr 2023 00:13:59 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ERBMFlpw; spf=pass (imf29.hostedemail.com: domain of 3xmsrZAoKCHMpfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3xmsrZAoKCHMpfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680567239; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/hxuqZFO5U+CmKAu0Izd1qiRBhW4afUnD0WyfUo4Gcw=; b=tlux+cr7Cbb7VhPScC6rchk4oZLBGAfB86jEh1dlMoDJSS5Hy2XcTW3habtlOMY5bf79Wn E7m8t7++fzlXB8dI56MyRoT7D1YQhg1ESYAyRK7fndJBvMnSjA8YMNmj5/AMCgFRbj3gcj +3pW7x3/WGCBgE2gTzJHDdcEUFmRxKw= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=ERBMFlpw; spf=pass (imf29.hostedemail.com: domain of 3xmsrZAoKCHMpfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com designates 209.85.214.201 as permitted sender) smtp.mailfrom=3xmsrZAoKCHMpfjipRYdVUXffXcV.TfdcZelo-ddbmRTb.fiX@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680567239; a=rsa-sha256; cv=none; b=Q7A9gS3fqSLSxKcTbQrxQxPsqy0VPo/klXes6OnxS1LeQ9NMcXqS4id85PaFYm6q8fz/WI 6GmvQvovZiJhlhJLLGCjEiLmbLIRl3yo8LuDzkQeLV4ki1Fn6b9zkTCUYgpMViJVjjZUAK DaJVtS0FMFMKVtxTD7WTuzfRILrqm9A= Received: by mail-pl1-f201.google.com with SMTP id w14-20020a170902e88e00b001a238a5946cso15856159plg.11 for ; Mon, 03 Apr 2023 17:13:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680567238; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/hxuqZFO5U+CmKAu0Izd1qiRBhW4afUnD0WyfUo4Gcw=; b=ERBMFlpwQ+/e/a/Z3AL8E+p3U9MNbkZrsz1HL/MZqvfsps00VtVZYbkySPDVq2/6aR LHVdBQe3ElTtHckYtN+0h+FDNuiI47HZzvKorWDJwFV06A2ZOZ9rTLgMNb03YVp2wZvD PgyPdh7s9QF5a+LrPpjkbUaqX5VHdnehZzbSO8EQsnkGFGX9ApbhWI2WH3GTZIdOCnMl aWLwqKHsahq90y7bzc7mLyA1MCyJjLXiPgYUQBEdJC8xf/8EamW1fLH2Ww2pGZUCTg1p jUEPpHXUbx3wFMtNPW+KmYfxXVLvMYfkOtlZrUyDZokuwj17Ht/6/tV/rlF0tPwOfpWA Gsag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680567238; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/hxuqZFO5U+CmKAu0Izd1qiRBhW4afUnD0WyfUo4Gcw=; b=dKRN2tCsq9VZrEsOE+fYx5XHT7zTG/xHHIHeaoI2YuNBZ9wEvylxqDwhn9HAJqnsee DDhKabpd2hb7ffOqWEZO4Tbu/BciUh03gOC+KBzoYDrm0tDd7C3J8J+kx5h5ut/ENAwI eNuE7nlIna9PQw9eTiZF8jhQcVMAZGtMjgzrze841J0h0p9GFmCfiAAjBkSchGG6rnJU /DOztN1+rMcdFbPyNYY5RsMdi4TN4RTaow0mOc8adMO0vvrRTMuvtXpSeGwl9yLiU9w/ p18lrWBPpVlxMlWj+qYUx2qgIw947LBqOjdDBs666S5KbozT73MohXUFKz/9HyCtvUw+ OPSQ== X-Gm-Message-State: AAQBX9e2QGe9MKbjd0XQUfWSCkZbuaYA4i8uEfRk/ihpmCgbBsNVnYFe kEDBcm/QutbXafZ0+gbFg7ymoousAqQYEBFb X-Google-Smtp-Source: AKy350Z+Sm8hAQJcQhRAP+xzVs9NeoT30dH7npmf11D1Z/F0xl2wPw5f56Q84WDvH4pUeUa2sdKUL4OGwZiD1m0K X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:23cd:b0:625:5949:6dc0 with SMTP id g13-20020a056a0023cd00b0062559496dc0mr232232pfc.4.1680567238017; Mon, 03 Apr 2023 17:13:58 -0700 (PDT) Date: Tue, 4 Apr 2023 00:13:51 +0000 In-Reply-To: <20230404001353.468224-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230404001353.468224-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230404001353.468224-2-yosryahmed@google.com> Subject: [PATCH v4 1/3] mm: vmscan: move set_task_reclaim_state() after global_reclaim() From: Yosry Ahmed To: Andrew Morton , Alexander Viro , "Darrick J. Wong" , Christoph Lameter , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Matthew Wilcox (Oracle)" , Miaohe Lin , David Hildenbrand , Johannes Weiner , Peter Xu , NeilBrown , Shakeel Butt , Michal Hocko , Yu Zhao , Dave Chinner Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Stat-Signature: rhsdgg1twoagrspzu35sncnmj9jtx1ac X-Rspam-User: X-Rspamd-Queue-Id: 5680C120012 X-Rspamd-Server: rspam06 X-HE-Tag: 1680567239-435398 X-HE-Meta: U2FsdGVkX1+OysQHnGfsXJv3smkS+dYsMM+W17AJW8vFUlhXUuxt+EMP4iqzBjatCEW2INGNYH+tnoVQemjDsTGPQnxXWUc0gGfezWA0Ev7n9uSjNxW1+L1eQH41motW+cY3iX63OLclaKT7lLq1Zc2tZXbF9LrR3f3Fpo78Gp/dM+WxOrD9RLpyt0objjw9sV02EozPNjXrSrzRssvveXJF1t6lIXbmP29kizGdNCYYsDB5tPUMjyXJhGRxHikDSHN9Szy2TLY+O++MogmEPl9NCft5suz2jGbvoQTwrSuME0O2/VrIaBUbamprOgE137OBHZM5WRQoCWwrWcGbjtMTGOvO7659rMhPYqxivUV4Nuw2ifpjDqL1xMuFG6J9j0gQHcDXUO3xxGceClT7ngWQccHCDmbqlLnaoyn72YFcILuPICVrzfKC48cZGZ4W3EPb9UiH912GoftVqCQ+L4W5xlp4PJxp9w8rd5GJFJ/QLB02eKcMWX0BafxeKCbEwhI7I58pJT0ihIMpYhcdJQv+odI06j3hCbGI+DO6ezhar++6yhrT7OV7Aj8BtuGOqhuVr2inslMnTAXb7OjtpkDhMbWhQY7d3VGTe8EXd5r3nk6kyl+VeFY4JFhxCZGKWFwIWWF3+7paMZH3hF78GkgvEEAyoE/47dWudyeQ8/qroaMHUyHyN2BEvjhEnUracRf0gaq4OSgcKWSntVBSxNkhPqK+m+Sxak21nGspZFOHaPLWkvNFQvjzEQSx9Ee6obtCAnou2DmgcOY5TZ/e0cq3xWMKxUlyIqbsQ6magZLKX0CJhOITqX0LSNBwshj9ROxrOfwZh3lXoT0x/2VDhGjdXhLJcybfiUG2wIwmHADHHyXQD5ObE9ZfDdvxl5bnhXI7XzS+WCadDPQvazcFJnnN7UmW9E1yb8pNRB4PSSLpQ23kMYszaBTXN4PNdLKKrESqSOqSvzm0BgMGgtj LkNFpS6D ojJxOGJLwcJjsR0mUVfhqjb7Jq841SGQysxrFjgjKJvjJS/wxlpUPPDsJ6M7GVIEt0gHp6+gFnTMG3O+EC6oj3KxXF27OJcr7rRQCA/lbw7i0E1skK1UtGZnEXs4f6UUs5UrwBYt9JS+wAosUs03XbwE4Epfzt1hiSZBcze0hiW78/WEEkROW1iOil8MSUGDcX4wS0Ur6E0xmDJ8QsTHHfVYx9A00H2s0y9LolRGi8ycI1/CCGJYWSyUrC+TWxptyM1aVoYYOSgfU9T6KD1162pSIfQ8MLjeaoNqZLRN2rZVQsWBu/woxKo3ljA4kqkCvq2jZZb0kejgPGF2b+ZsFBlwWbil32p2zXkl6wYIg5my2S3qy3RsnA+7lLqswXlsRmdl05EQdgIhDBaUkCGnQc9FWNbnfxyMgLkjZP2FM4uzrTLxJK2BhgYxjKmX8UvEi1luflgqYESjqah1wIxj0A53zHBSEug0NfPS6C7kGTZ9r6RKrJxa7Ibzqr6Oe6A01GAq/IbY/6osyjkJAWi9lnd/AqaN+7H6uYldDCoRsYoIBckAEcsmQuANWYrbMmINHy2bYmZpLR2vLyhXY4e0tdYmQ6fMQ9SR4f4zEJ6AomBxJxPDMUmDKrwRAipWmB7XXio22au6ehB0fKQ9Ae9ha5X83j5qe4Pf0oZ4iFKaH60hw63a45k4pOgjAVpSKJlM6qbjgfXngGn8ELn3WRa4A0uewXd39ZS7fIEeLYzGcRvmXchlFo2Gtr8j0Q/MOx/8fyRQVpwEZR4uyhNQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: set_task_reclaim_state() is currently defined in mm/vmscan.c above an #ifdef CONFIG_MEMCG block where global_reclaim() is defined. We are about to add some more helpers that operate on reclaim_state, and will need to use global_reclaim(). Move set_task_reclaim_state() after the #ifdef CONFIG_MEMCG block containing the definition of global_reclaim() to keep helpers operating on reclaim_state together. Signed-off-by: Yosry Ahmed --- mm/vmscan.c | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 9c1c5e8b24b8f..fef7d1c0f82b2 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -188,18 +188,6 @@ struct scan_control { */ int vm_swappiness = 60; -static void set_task_reclaim_state(struct task_struct *task, - struct reclaim_state *rs) -{ - /* Check for an overwrite */ - WARN_ON_ONCE(rs && task->reclaim_state); - - /* Check for the nulling of an already-nulled member */ - WARN_ON_ONCE(!rs && !task->reclaim_state); - - task->reclaim_state = rs; -} - LIST_HEAD(shrinker_list); DECLARE_RWSEM(shrinker_rwsem); @@ -511,6 +499,18 @@ static bool writeback_throttling_sane(struct scan_control *sc) } #endif +static void set_task_reclaim_state(struct task_struct *task, + struct reclaim_state *rs) +{ + /* Check for an overwrite */ + WARN_ON_ONCE(rs && task->reclaim_state); + + /* Check for the nulling of an already-nulled member */ + WARN_ON_ONCE(!rs && !task->reclaim_state); + + task->reclaim_state = rs; +} + static long xchg_nr_deferred(struct shrinker *shrinker, struct shrink_control *sc) { From patchwork Tue Apr 4 00:13:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13198932 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 79B35C77B6C for ; Tue, 4 Apr 2023 00:14:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D6237900003; Mon, 3 Apr 2023 20:14:02 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CEB6A900002; Mon, 3 Apr 2023 20:14:02 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AF03E900003; Mon, 3 Apr 2023 20:14:02 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9347E900002 for ; Mon, 3 Apr 2023 20:14:02 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 65177160D95 for ; Tue, 4 Apr 2023 00:14:02 +0000 (UTC) X-FDA: 80641785924.01.F7F1954 Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf12.hostedemail.com (Postfix) with ESMTP id 944B340011 for ; Tue, 4 Apr 2023 00:14:00 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=LtYxh8Yf; spf=pass (imf12.hostedemail.com: domain of 3x2srZAoKCHQqgkjqSZeWVYggYdW.Ugedafmp-eecnSUc.gjY@flex--yosryahmed.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3x2srZAoKCHQqgkjqSZeWVYggYdW.Ugedafmp-eecnSUc.gjY@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680567240; a=rsa-sha256; cv=none; b=3AzKrs/8qMQ+ZNLnIRP2xM++Z0i9YM5BGcFpUufRG3ghjzDalcP8JQLN/AzbivNezZ3M/T 5NKKry8/I+52wJKLsT59P5nLP938uzNVDhksaFqt0aBobvOrxo5zzL3bNrHWG1Elf/EJrH f5Wci+xuefW9lXmOzAuufuvDazzVqN0= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=LtYxh8Yf; spf=pass (imf12.hostedemail.com: domain of 3x2srZAoKCHQqgkjqSZeWVYggYdW.Ugedafmp-eecnSUc.gjY@flex--yosryahmed.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3x2srZAoKCHQqgkjqSZeWVYggYdW.Ugedafmp-eecnSUc.gjY@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680567240; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=KttjkMRwWn+raVJD4pdmRR27zCHPUdxyxHHqMjZ4zso=; b=nKgd4KxcDlaRf4U3Z4SWHrh4ejyF9/J/jMuAfOdrJ+IYggMBSnCichzyF49NMTpRjq7kAb Bc1AiRInggb9yUNoh1AfzMIEzPsGXKQ1/JU2RFUWqDYB2sfG9UhLUgVXz81SuV4GyMSoyZ AZlAVFDcuRlSfwhCsgtkNljmaYu4A7U= Received: by mail-yb1-f202.google.com with SMTP id x15-20020a25accf000000b00b3b4535c48dso30226966ybd.7 for ; Mon, 03 Apr 2023 17:14:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680567240; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=KttjkMRwWn+raVJD4pdmRR27zCHPUdxyxHHqMjZ4zso=; b=LtYxh8YfdBOctNDOeOLcBvNxi7r6sZaIjGKGM0dSm4YAneG6TwPyrn6ciNpFpw7o05 agIDp8r5g1zsrBgbrkIAj0istxDrIlFVjEGnPtfdwxO+fsBcrB9rTcH6hnSDhjeSGcI9 DNWZ+jLmPaIMNrbRgwWvkKyl3KncBlpUe/ySEPbtedlDu2Al/e4kMdIq0CbOxcu1yZTM p4hiQCl8bRdPl3Qg3dXxTrvvyXCgzXh5Si+jrlA5hVfiT0NfHjRVSIoyov4AHA/OntzE adXxHHHwpx/tQEJ5noRB+DDraQeehxLQoOxrCnjR60/cYEdTl8ijh1e5DVzefIts+Ack bxFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680567240; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=KttjkMRwWn+raVJD4pdmRR27zCHPUdxyxHHqMjZ4zso=; b=Bvxh9WVuz8f4rnywx//bYZ67fq8vP3oJcYdzQI8q113ZKqfwbjkoclbE728jagBpuo pgPEVRXOgE8mDs5rNJDBqsFFb+pupRIlGMlcHBcQyYDdjjQBQ/+GG5UtTzYgT22ZvxDf MABmfi6K/76M01dYpGcE6SZ5vZU+Ok50EyKlb1eKTaNuD0z3racPW27CNM0nsL7CtYMO bo0iq2x1EHp1m8B9kTAF4pl0BQCr0aGJa9O2N35WeEROqj+icZWqRn6j9pooI07XsWDR 6XGnC1iW4ge56yAcsFEtHTrnt30oRHBQjEwqoIMHWhu6iCbpymB7fFkHmlO5OvAYFdXC ahCQ== X-Gm-Message-State: AAQBX9fVTAi2fUZN/Uqf4cbbIlReT3oUTnA9eiYYMRyeun358x2ktWC9 jRMrcDWqWNgpaGhDMttCWslkiKT5ZyjKVEpi X-Google-Smtp-Source: AKy350aw2fB9OoYuquCgQGUwaaV+x9FcDRGjxrCFXXINr0QvKI17cucoPmiPUl1lnvL0dNkgmm2U7bqUVOt3NxkG X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a25:d950:0:b0:b7d:87f:ce3e with SMTP id q77-20020a25d950000000b00b7d087fce3emr663961ybg.6.1680567239851; Mon, 03 Apr 2023 17:13:59 -0700 (PDT) Date: Tue, 4 Apr 2023 00:13:52 +0000 In-Reply-To: <20230404001353.468224-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230404001353.468224-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230404001353.468224-3-yosryahmed@google.com> Subject: [PATCH v4 2/3] mm: vmscan: refactor updating reclaimed pages in reclaim_state From: Yosry Ahmed To: Andrew Morton , Alexander Viro , "Darrick J. Wong" , Christoph Lameter , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Matthew Wilcox (Oracle)" , Miaohe Lin , David Hildenbrand , Johannes Weiner , Peter Xu , NeilBrown , Shakeel Butt , Michal Hocko , Yu Zhao , Dave Chinner Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Rspam-User: X-Rspamd-Queue-Id: 944B340011 X-Rspamd-Server: rspam01 X-Stat-Signature: coe7jzc9t7de9ifdsh5afuah4b5ip7hj X-HE-Tag: 1680567240-390718 X-HE-Meta: U2FsdGVkX18jJvk4Blarn4ayohgWfHHRHJX+etVwazMLRFDmoFG+6G7lfV53WcDEOPzPjU2ZGMSGeq/jI4ruIQC76RT5FijQno76wgY6Lvq3/jvY6khW8zTkmzFyYXlPb7Jk0fyklXSvaT1DgbTco22RlbQiGxsynJhUm37ONTDem6dlcdz91F5iDrwmYzZQE4fT7vDuopu5T3efl5b/UL2NLsrpoj259bDMWbBatvxd8U0LrFdJ682c4b1zhI16N/GUt0GJynitsmTyfK+oeyVesgJr7O5WT9EAPooCU9Eyld7femA4YMedxKJRTjQQ6PtTFihIP7yOwkf4pF2DNoFWHYMxc5H0BQLq0xT8W4jlbhzuIhPncQQAlCMWaeRtciVIIvC0ALGiU+jycKGl9e0hLhimZPQyut7JjS0kA0xu1mtvxiSd3B7xmUbptlV+PbDiywG7IiHApAg+16QeMhIN9Nx7SmtFlDOd1qIR2J4uJTGkODRMscF53cbdVcJrGZC9bgaVxQSObSCDCD49UC3iJL7V+uOxGo+e7xqS85TRR6b+tZbgWHLJh+eELvQbv3DFuQ2mAvgjw7qvVcPE/CWyv5UN9Rr+uDext1OC0+g5b8mfc7WTClQfaQw7AGF5yGq9P0+BB4aK5hUb05Ej7Ag7uOP/2F3nBgYwPvhS7Pn9qYeVO52CwBjbnM6cfODDaQOaD+kELjXcdjOvQmnTaG0LK5kZjJNp308Q99WAPgKdojD5WE4U8hR8hp2gdzGA+lQrqwNGVM5QV5omqII6qkU62b7V5Ka+PDbL5j12Nwu2bpDc3GU0HoAdYgAYY+DEEQOEsiCYaaqapHN31G6/cvYp6jb6YgOU/pFS7uhi/nc8feRtk5+a4bJqniZM1L8njjqhULRw9zzKDD7CI5QjBKhUmEt+aKVFiptEYqz/oPEiP5x10Aagu3yHu7/RfcVOQ71TN9WabRYkLEixDlF xnXF/xTb uM48/76ABGqkbLAvHPy7UF2pU4t78tmdX8BpKfd4tRjc4Y+eaNC+bGmOtev0M9pZGWD291jwJXK8Zrnuue18GR2mngQkR/KO5S5DEFSgVTD+EarSX6IrVfSIg2+cHFdtS7uf36KlWWUT0ezl6DrxlxeIvtzjBQoue6mQ713QOyeduM/DDqpimoqMYJfwMcQZcDXIh5OCCLKaZV/wd64fpCHcSul+rZrRGRqyVKFFqnh2JHQgn5RSY4UM5SoimwmRllRD5CqM/EpTTZPdpXKCHgvdfN12oohTzEJQROgb3Sbc5GcaPUX5Ki1a9MC4d56V8JNXrHOUiiUGX2UPgIGImf6fH6ns6ab9E8vTXL0ylA8dmLLwjpUBrvO9VjwGTQJZaDkPwhzLuSt2cRMMjaAkimu837BnH/GG1jurSeuhsiA+0qegdcuOdCGwlVHR3isEvLwV3htO7BM6UaKw4ADmb6SETJJIiRqX6cyMU8E9DrBlQZv7fuHhn+ZJ7KE+1RB52jgzltXtfJsxyVbCZQ6UHINXxt1o+a/tTIBEgTVUuf+5ZVKfNjupyvfkXuDs1TtN8M7qTuB4WcK7unjx7RFwGyV9fSebi8Pamy4sZG8Bw6+voq/A+wSDWoQjp5yKa6USEA+AzqwwDHSNItoczH56jbjsI3idsM2eRrRindzh1rCxX01l/79ZLob9dLNAa0gxaFAHOE4BvjVXutGKZmjf0r1rUcm6mm5szJLhHzSDmh5ckGKUXrdKc3I0MEC9D2YyWLi2pT+/bGaphCtwbsSzNSp6O6w== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: During reclaim, we keep track of pages reclaimed from other means than LRU-based reclaim through scan_control->reclaim_state->reclaimed_slab, which we stash a pointer to in current task_struct. However, we keep track of more than just reclaimed slab pages through this. We also use it for clean file pages dropped through pruned inodes, and xfs buffer pages freed. Rename reclaimed_slab to reclaimed, and add a helper function that wraps updating it through current, so that future changes to this logic are contained within mm/vmscan.c. Signed-off-by: Yosry Ahmed --- fs/inode.c | 3 +-- fs/xfs/xfs_buf.c | 3 +-- include/linux/swap.h | 17 ++++++++++++++++- mm/slab.c | 3 +-- mm/slob.c | 6 ++---- mm/slub.c | 5 ++--- mm/vmscan.c | 21 +++++++++++++++------ 7 files changed, 38 insertions(+), 20 deletions(-) diff --git a/fs/inode.c b/fs/inode.c index 4558dc2f13557..e60fcc41faf17 100644 --- a/fs/inode.c +++ b/fs/inode.c @@ -864,8 +864,7 @@ static enum lru_status inode_lru_isolate(struct list_head *item, __count_vm_events(KSWAPD_INODESTEAL, reap); else __count_vm_events(PGINODESTEAL, reap); - if (current->reclaim_state) - current->reclaim_state->reclaimed_slab += reap; + mm_account_reclaimed_pages(reap); } iput(inode); spin_lock(lru_lock); diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c index 54c774af6e1c6..15d1e5a7c2d34 100644 --- a/fs/xfs/xfs_buf.c +++ b/fs/xfs/xfs_buf.c @@ -286,8 +286,7 @@ xfs_buf_free_pages( if (bp->b_pages[i]) __free_page(bp->b_pages[i]); } - if (current->reclaim_state) - current->reclaim_state->reclaimed_slab += bp->b_page_count; + mm_account_reclaimed_pages(bp->b_page_count); if (bp->b_pages != bp->b_page_array) kmem_free(bp->b_pages); diff --git a/include/linux/swap.h b/include/linux/swap.h index 209a425739a9f..e131ac155fb95 100644 --- a/include/linux/swap.h +++ b/include/linux/swap.h @@ -153,13 +153,28 @@ union swap_header { * memory reclaim */ struct reclaim_state { - unsigned long reclaimed_slab; + /* pages reclaimed outside of LRU-based reclaim */ + unsigned long reclaimed; #ifdef CONFIG_LRU_GEN /* per-thread mm walk data */ struct lru_gen_mm_walk *mm_walk; #endif }; +/* + * mm_account_reclaimed_pages(): account reclaimed pages outside of LRU-based + * reclaim + * @pages: number of pages reclaimed + * + * If the current process is undergoing a reclaim operation, increment the + * number of reclaimed pages by @pages. + */ +static inline void mm_account_reclaimed_pages(unsigned long pages) +{ + if (current->reclaim_state) + current->reclaim_state->reclaimed += pages; +} + #ifdef __KERNEL__ struct address_space; diff --git a/mm/slab.c b/mm/slab.c index dabc2a671fc6f..64bf1de817b24 100644 --- a/mm/slab.c +++ b/mm/slab.c @@ -1392,8 +1392,7 @@ static void kmem_freepages(struct kmem_cache *cachep, struct slab *slab) smp_wmb(); __folio_clear_slab(folio); - if (current->reclaim_state) - current->reclaim_state->reclaimed_slab += 1 << order; + mm_account_reclaimed_pages(1 << order); unaccount_slab(slab, order, cachep); __free_pages(&folio->page, order); } diff --git a/mm/slob.c b/mm/slob.c index fe567fcfa3a39..79cc8680c973c 100644 --- a/mm/slob.c +++ b/mm/slob.c @@ -61,7 +61,7 @@ #include #include -#include /* struct reclaim_state */ +#include /* mm_account_reclaimed_pages() */ #include #include #include @@ -211,9 +211,7 @@ static void slob_free_pages(void *b, int order) { struct page *sp = virt_to_page(b); - if (current->reclaim_state) - current->reclaim_state->reclaimed_slab += 1 << order; - + mm_account_reclaimed_pages(1 << order); mod_node_page_state(page_pgdat(sp), NR_SLAB_UNRECLAIMABLE_B, -(PAGE_SIZE << order)); __free_pages(sp, order); diff --git a/mm/slub.c b/mm/slub.c index 39327e98fce34..7aa30eef82350 100644 --- a/mm/slub.c +++ b/mm/slub.c @@ -11,7 +11,7 @@ */ #include -#include /* struct reclaim_state */ +#include /* mm_account_reclaimed_pages() */ #include #include #include @@ -2063,8 +2063,7 @@ static void __free_slab(struct kmem_cache *s, struct slab *slab) /* Make the mapping reset visible before clearing the flag */ smp_wmb(); __folio_clear_slab(folio); - if (current->reclaim_state) - current->reclaim_state->reclaimed_slab += pages; + mm_account_reclaimed_pages(pages); unaccount_slab(slab, order, s); __free_pages(&folio->page, order); } diff --git a/mm/vmscan.c b/mm/vmscan.c index fef7d1c0f82b2..8f0e7c4e91ae3 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -511,6 +511,19 @@ static void set_task_reclaim_state(struct task_struct *task, task->reclaim_state = rs; } +/* + * flush_reclaim_state(): add pages reclaimed outside of LRU-based reclaim to + * scan_control->nr_reclaimed. + */ +static void flush_reclaim_state(struct scan_control *sc, + struct reclaim_state *rs) +{ + if (rs) { + sc->nr_reclaimed += rs->reclaimed; + rs->reclaimed = 0; + } +} + static long xchg_nr_deferred(struct shrinker *shrinker, struct shrink_control *sc) { @@ -5346,8 +5359,7 @@ static int shrink_one(struct lruvec *lruvec, struct scan_control *sc) vmpressure(sc->gfp_mask, memcg, false, sc->nr_scanned - scanned, sc->nr_reclaimed - reclaimed); - sc->nr_reclaimed += current->reclaim_state->reclaimed_slab; - current->reclaim_state->reclaimed_slab = 0; + flush_reclaim_state(sc, current->reclaim_state); return success ? MEMCG_LRU_YOUNG : 0; } @@ -6472,10 +6484,7 @@ static void shrink_node(pg_data_t *pgdat, struct scan_control *sc) shrink_node_memcgs(pgdat, sc); - if (reclaim_state) { - sc->nr_reclaimed += reclaim_state->reclaimed_slab; - reclaim_state->reclaimed_slab = 0; - } + flush_reclaim_state(sc, reclaim_state); /* Record the subtree's reclaim efficiency */ if (!sc->proactive) From patchwork Tue Apr 4 00:13:53 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13198933 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8D5AC77B62 for ; Tue, 4 Apr 2023 00:14:05 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4627E900004; Mon, 3 Apr 2023 20:14:05 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3EC29900002; Mon, 3 Apr 2023 20:14:05 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2654C900004; Mon, 3 Apr 2023 20:14:05 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 0ABF3900002 for ; Mon, 3 Apr 2023 20:14:05 -0400 (EDT) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id D61161C61A9 for ; Tue, 4 Apr 2023 00:14:04 +0000 (UTC) X-FDA: 80641786008.17.5D98DF1 Received: from mail-pf1-f202.google.com (mail-pf1-f202.google.com [209.85.210.202]) by imf22.hostedemail.com (Postfix) with ESMTP id ED65DC000F for ; Tue, 4 Apr 2023 00:14:02 +0000 (UTC) Authentication-Results: imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=W5WukSX7; spf=pass (imf22.hostedemail.com: domain of 3yWsrZAoKCHYsimlsUbgYXaiiafY.Wigfchor-ggepUWe.ila@flex--yosryahmed.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=3yWsrZAoKCHYsimlsUbgYXaiiafY.Wigfchor-ggepUWe.ila@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680567243; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=QYyKpmgAe3S55Nk5FWRs2dH0UIocesat7xjNxdFTU4k=; b=MJGhdWYfUfpnAHooGMQvzVuGqWW3SYC3YluA2tTHHHunuGQbDCLQtKERs8mfyW6FdCpcIc jc32MkFS2nIeFS5pH9lVnzeJaf+Jp+cEjTLxdHh7cPpBuCUXz/M9wSvwklVoL72Ht6kxhZ 4WbBJKviyn/kl58TSxXV+uEBZOC0iC0= ARC-Authentication-Results: i=1; imf22.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=W5WukSX7; spf=pass (imf22.hostedemail.com: domain of 3yWsrZAoKCHYsimlsUbgYXaiiafY.Wigfchor-ggepUWe.ila@flex--yosryahmed.bounces.google.com designates 209.85.210.202 as permitted sender) smtp.mailfrom=3yWsrZAoKCHYsimlsUbgYXaiiafY.Wigfchor-ggepUWe.ila@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680567243; a=rsa-sha256; cv=none; b=PdCRavqt8om4BBYJm/4QEIRKvTptMlIYchT51TF5U0sFjw2FPY+eyrD54D0m+fpc6Aot7X 0B32YCVSGGhJGL72NSg/Dv8g1v8RaPdXqoiVDnTFRWXxRhxY+PrTNabic+2Ktmx79/srlo 5uWSWcqNgMyaXA14c6FCN5hseEvKyMo= Received: by mail-pf1-f202.google.com with SMTP id y15-20020a62f24f000000b00627dd180a30so13896551pfl.6 for ; Mon, 03 Apr 2023 17:14:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1680567242; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=QYyKpmgAe3S55Nk5FWRs2dH0UIocesat7xjNxdFTU4k=; b=W5WukSX7xVU0/vv5ALD+tup9Pddeb9Iipi9kb8kJt/jBn+TQTiZoJyEWgKO/OQ61Jh 2tDeM9kRzvnDYGPtmMlhz5iVehjZJg/jQ63YFUQ7WbbPKq0AY2cM2cHrwogahpd+PgEG XwtqC+dGnvs0z0/3t8FCcMDB2O7fMVf3H3E2qFVcjwb9hPLZO3zgEa2XrTMXRi9MFCHf WEtCE6c9sD8f7AqxvmgV8c+yt8qmrF2I9lAqEvjVuhsI1HciUBrepOGZet31fNCDDW8P oX+wUwY7aO9vBuigwgimgUP/rKThFLYxYAVKIiGShyp9evKm8iqS6ChI4jTiXqDaT7MQ ct3Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1680567242; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=QYyKpmgAe3S55Nk5FWRs2dH0UIocesat7xjNxdFTU4k=; b=S8FrFVy11jgkXcQ5QsS+mWquzaboQalWpH1bjnYDY2M24lyiBIQHokdoc91SbOOYSz EqkPr6f2zZ50WxDXzvysClTAK8mK5boXFYRvTXrFbIB7nWTKuBH9EPJU2LcbMWwBJhPg IGMHJeXc+9kPsIaBBz9qPF+Vtyra9p95b+nkHx/v/uuvn9U+hJ0PDUqfZKPCqUj8ZPLe jgSJ9QI9asCyhc0qsHc518+elCJm6XMDDad260YG4L9JoeDTXIy9CsWgwlsOZijIgpfA E/dyGYOeHNeyfIzjCYA/Ogu3CSxlF77fEjYmuPQNPtVqgFm8vZ4AAVsWpimZfZs0T+k4 sI0Q== X-Gm-Message-State: AAQBX9cP/xyR7iZW+b8z8PHSFNLrdmKJOQgeQr5J+YnqA1/WK/6vbuOt R7d26KA1sXnIUyn20aVcRRwMgrSxAY5tEB4S X-Google-Smtp-Source: AKy350a8jpmup3Y0AoRXTeOyKKcgC2LVL+csu37K+C5/P3dMmovoTexr82+Hd4O52pTXV8PmQrABJAtIr5EvWJJc X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a05:6a00:2e14:b0:627:e6d5:ba2d with SMTP id fc20-20020a056a002e1400b00627e6d5ba2dmr223605pfb.6.1680567241767; Mon, 03 Apr 2023 17:14:01 -0700 (PDT) Date: Tue, 4 Apr 2023 00:13:53 +0000 In-Reply-To: <20230404001353.468224-1-yosryahmed@google.com> Mime-Version: 1.0 References: <20230404001353.468224-1-yosryahmed@google.com> X-Mailer: git-send-email 2.40.0.348.gf938b09366-goog Message-ID: <20230404001353.468224-4-yosryahmed@google.com> Subject: [PATCH v4 3/3] mm: vmscan: ignore non-LRU-based reclaim in memcg reclaim From: Yosry Ahmed To: Andrew Morton , Alexander Viro , "Darrick J. Wong" , Christoph Lameter , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Matthew Wilcox (Oracle)" , Miaohe Lin , David Hildenbrand , Johannes Weiner , Peter Xu , NeilBrown , Shakeel Butt , Michal Hocko , Yu Zhao , Dave Chinner Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: ED65DC000F X-Stat-Signature: q7y8jcw7kn1eocnorcpnjy788byu5k7y X-HE-Tag: 1680567242-359231 X-HE-Meta: U2FsdGVkX1/bdgVrY9WanEp9Hm4Z0nNHwDTOh866htY7Bxz8q1BB492pfoLa/kL1qpAlsU25CJAkBElJ6xk5U7Wg2EGPKRMYxLeZZWL53ZGdvI3aDQN4RI0HjIYQrk7lxQF/bqdblKeTNRl5TbIlIU4k+QqWMc/q/SfPUXTYpGsH2iPbLoKxA8vRVTSIEY5ELPa+ln0ARxOWcXSwVOj8WBjK9GddFFi2Sg6h1VbZ0ANyESF5Gs4b9SiTPr5GNea8+Is5CKAtA9HTiPe2SjEMTTM1+/sdm3Ds4dfddyu1ne3eDbudd3ENL8FJL1/Kw8+CNgnNY/eaKwPgmdBwt3wbgodYcRHTkjpwNk/PsMqqC77phf3Cf39iADi8UIp5Jgbj9p8GnKV/gwR2mniUXSGqsmbkRe1V85jut5S2MpmN3qm9mWXFUa/vyowPCE/wooRakPycpkqxL526+3C/V7EvO95OBkDSep8Sqf/9WLQqvQ0/hGE/uXRPfETTVV9Afy3VNdE+XS2e0TB3pxj/4TB0jw457YScNkcZpKRrtc8extr0poGiY+y1/Ni2sjjcQEfbcpw07Avbx5t7eCUa0oOc9zWZVlPCZW2UWyyu9cwD6ni6cRvyBR1UDueotFVDoa1EF9TfC7A2Ruhqf4knrtWuFgK1Mnh+hYWMjcOH/T1thmgNX449c+xtAyPs7OOx3YxzmFFj8MtaPyGBSkng/g2fKC458x9SgRcnQiw+4aK121r3lgNJO1UyPc05XWWBpV4OXqX3R4I0PtVd4rzvLwLz1ts7Uy+nZ7Si/r7wmfYKVv/EiyicJVz8wAX411pibQ1ruva4+xtoCm9Etv+AJ4VCSGd9LjRu2ejgJjTI82vfAZi/WBSqV3O/QrGVssCQ5tAg36+gBiCnr3h92+XKMrLUqmK3N3V8luDdouAO4OqWfvZFB+X6q1amnPaC+BzItLarvtIn/9CDIejPftKDbiY YoijCm1u 0nYBQ8MBSp+HoMRTpimz3lioSdZY6/MJAmLgHGEQGGUaZHxBDdRK2cprh/Luqtl2ay5fA/AAoSTnLAoXgxkUv06BKjloIfcmpDaX7bNIS996t6yCbKcx8vMQzgCycJ+fJ7afppctqTpUHCB+1t/MK+RRBitwWu8KCPA8CK9+Ntv83KdQn7Znhqd0k+DI24U3c9Pziof8UJHI9TulN5apINFusy2dBTRM+SkJMuU3IMwlIj8HqV+dNNFaf6PHsq59aimItLUVQ1ipWmwNon9HYobdWVA/4C+/vl5pifVrROUXMRTIj5XH5O8F/Min2iaqBAQVOTEMHsjAZrkP8iHuSeBx+xcgZt2JQOBoZS8SQP9jQYgCpTQscuHeIQoL2IoSqYB9Ix7RQKNPhlc8pzNpinFbUsfqvM7EGJgKCiSSrqhOQlU4p7/1ltE1++iDqBGGwgLd/Ght9CTamSq0sGO8R6mZulGxX5eGgraPSRio69DStGVMYcgqnRWWUR+BEryPsuF0oK/7kD5KqPXAhQyDS4+HtCXJMBQop0WpXvd7dNQ/UJCEaDymyA+Lg1cPmkAACiB6u18YGRUTW/nKzWXZ4c/wmCmKmwdzf/l2TyQ+wQzBWnlI6x/riUa4oBdoHfoKK+vo7N4WHx1Jp/7E1uLv9BW7YyXf8dknvBTI49mymt4O8O+UGrVHpgKQnwNRct/yz0sCzepPyybxS66z/Ek+n0MEKquaOVZ+JfeRl X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We keep track of different types of reclaimed pages through reclaim_state->reclaimed, and we add them to the reported number of reclaimed pages. For non-memcg reclaim, this makes sense. For memcg reclaim, we have no clue if those pages are charged to the memcg under reclaim. Slab pages are shared by different memcgs, so a freed slab page may have only been partially charged to the memcg under reclaim. The same goes for clean file pages from pruned inodes (on highmem systems) or xfs buffer pages, there is no simple way to currently link them to the memcg under reclaim. Stop reporting those freed pages as reclaimed pages during memcg reclaim. This should make the return value of writing to memory.reclaim, and may help reduce unnecessary reclaim retries during memcg charging. Writing to memory.reclaim on the root memcg is considered as cgroup_reclaim(), but for this case we want to include any freed pages, so use the global_reclaim() check instead. Generally, this should make the return value of try_to_free_mem_cgroup_pages() more accurate. In some limited cases (e.g. freed a slab page that was mostly charged to the memcg under reclaim), the return value of try_to_free_mem_cgroup_pages() can be underestimated, but this should be fine. The freed pages will be uncharged anyway, and we can charge the memcg the next time around as we usually do memcg reclaim in a retry loop. Signed-off-by: Yosry Ahmed --- mm/vmscan.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/mm/vmscan.c b/mm/vmscan.c index 8f0e7c4e91ae3..049e39202e6ce 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -518,7 +518,35 @@ static void set_task_reclaim_state(struct task_struct *task, static void flush_reclaim_state(struct scan_control *sc, struct reclaim_state *rs) { - if (rs) { + /* + * Currently, reclaim_state->reclaimed includes three types of pages + * freed outside of vmscan: + * (1) Slab pages. + * (2) Clean file pages from pruned inodes. + * (3) XFS freed buffer pages. + * + * For all of these cases, we have no way of finding out whether these + * pages were related to the memcg under reclaim. For example, a freed + * slab page could have had only a single object charged to the memcg + * under reclaim. Also, populated inodes are not on shrinker LRUs + * anymore except on highmem systems. + * + * Instead of over-reporting the reclaimed pages in a memcg reclaim, + * only count such pages in global reclaim. This prevents unnecessary + * retries during memcg charging and false positive from proactive + * reclaim (memory.reclaim). + * + * For uncommon cases were the freed pages were actually significantly + * charged to the memcg under reclaim, and we end up under-reporting, it + * should be fine. The freed pages will be uncharged anyway, even if + * they are not reported properly, and we will be able to make forward + * progress in charging (which is usually in a retry loop). + * + * We can go one step further, and report the uncharged objcg pages in + * memcg reclaim, to make reporting more accurate and reduce + * under-reporting, but it's probably not worth the complexity for now. + */ + if (rs && global_reclaim(sc)) { sc->nr_reclaimed += rs->reclaimed; rs->reclaimed = 0; }