From patchwork Thu Mar 9 09:31:06 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yosry Ahmed X-Patchwork-Id: 13167117 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 848F9C61DA4 for ; Thu, 9 Mar 2023 09:31:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1F402280001; Thu, 9 Mar 2023 04:31:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1A37F6B0078; Thu, 9 Mar 2023 04:31:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 06BE8280001; Thu, 9 Mar 2023 04:31:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E805D6B0075 for ; Thu, 9 Mar 2023 04:31:25 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id C5A9D12098C for ; Thu, 9 Mar 2023 09:31:25 +0000 (UTC) X-FDA: 80548841730.10.2A82B57 Received: from mail-yb1-f201.google.com (mail-yb1-f201.google.com [209.85.219.201]) by imf21.hostedemail.com (Postfix) with ESMTP id 1F4C21C000F for ; Thu, 9 Mar 2023 09:31:23 +0000 (UTC) Authentication-Results: imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TFj7MTDs; spf=pass (imf21.hostedemail.com: domain of 3a6cJZAoKCAg6w0z6ipumlowwotm.kwutqv25-uus3iks.wzo@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3a6cJZAoKCAg6w0z6ipumlowwotm.kwutqv25-uus3iks.wzo@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678354284; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=XmqAffVC+m+oVrrIAcB4LBUAdhmCJbGaOFJSOfA6Apk=; b=63JAPDT1oR3DYvNIaJIj2+/qu6AEJiS6W8wC9av7i3M5wiLElimzW3gT67JgijSH4RJF45 GsKbWjwYFcYlnI/v5T6X5OTuhC0OONvBwR5ONFGShtia1mfjzayD2O7m5iE2PnQc2nz9j4 7A/N9rCvBibLg07Et82hQvOilX20FbA= ARC-Authentication-Results: i=1; imf21.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=TFj7MTDs; spf=pass (imf21.hostedemail.com: domain of 3a6cJZAoKCAg6w0z6ipumlowwotm.kwutqv25-uus3iks.wzo@flex--yosryahmed.bounces.google.com designates 209.85.219.201 as permitted sender) smtp.mailfrom=3a6cJZAoKCAg6w0z6ipumlowwotm.kwutqv25-uus3iks.wzo@flex--yosryahmed.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678354284; a=rsa-sha256; cv=none; b=17I1tMYJcHh+MMGu+9YxJgp7AnQ3pUZvGGlAkYZdzHzVXMq5on8zpoemcA24SlTBASkjut pKRzaTGsQ5OZpMhswbbqhB3bM9MIKD+XKYmtsOuI0V+Szr8uG4OAbX3P2oPJpT8RGuwel6 8Y1KHbTtext06h7OzigtJdim0N0azU4= Received: by mail-yb1-f201.google.com with SMTP id l24-20020a25b318000000b007eba3f8e3baso1720673ybj.4 for ; Thu, 09 Mar 2023 01:31:23 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1678354283; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=XmqAffVC+m+oVrrIAcB4LBUAdhmCJbGaOFJSOfA6Apk=; b=TFj7MTDsZpBNZ1Pa0nPpBjAVMapPhqIjEuIqItxSZfA4ak3MR9Zb48iIb7foczQpbX WbksQYQHrdZrln4nHdw6+5sdurXfjnbNe/kDD/0fdzYnrK4FPkh80+fXTAFYX+K7ErEh 1rcQSsoydEEK0CwMuckiZLK2zbAhv/zNi+QuBYMCZBwEZzkU8Itoiq8WU1TgB18gc2Bw EDGa5MXDFojPNO9J/WmuxVzK2IXT3yxoZSKw2S8SO1+F98jJAYgqAMtZYNYjCgHAsfK6 fm7hVlPreLuzvKK1gsnbhaUhDW/3kun5nSsDzOEnqh2wUc39UfeqzmXDz1CyRuaUl0uD IRvg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678354283; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=XmqAffVC+m+oVrrIAcB4LBUAdhmCJbGaOFJSOfA6Apk=; b=VqbyL4xRIT2+eiJ3LlJ9skI21x1euCbaKXRDxrtv72Fj2ku0xZt5OqpSCMhCp2KbmT 5XLApnVtLr3JkVYEJwZNePHr+NrKMoH64brMTYIxRk/7zcfqM2DJk9FWv1ZsoY6628F6 61q84oHOW0U/77RxehAcPK2EXQJ51jkZPrHsaqpNAXqAZDyQlBiqUxaE0dfU2GPqkJvQ v/m2A7v5zZ/Ulg1fOHzQmk6bV/FGT32dqxOXwe8lBh2otwsHgUKutgc5StBLvF91Bc8U ixrDS9LL88Sv0H3yK9msE+9xmFzloMfsfUHmpxchIasK6PfFd4HBEIIZojwErxFvaR7S LMVQ== X-Gm-Message-State: AO0yUKVX+cOZ93aIzpsFC7f+sKEAk8c8Q8SIfPJmgY2ImCvFJQ3xJ2fo trNS6gKzgYX50pgZdXHQjJREMaVAZt+D+nm1 X-Google-Smtp-Source: AK7set8lfhDA1hmab01GBmCcbW1gNF0jJVZZ3P6U2WtB9nZ0L2Sl5efbx+XPbYy5GSpQU4+BOx2RAfHOQQ1GpTqg X-Received: from yosry.c.googlers.com ([fda3:e722:ac3:cc00:7f:e700:c0a8:2327]) (user=yosryahmed job=sendgmr) by 2002:a25:fe04:0:b0:b1a:64ba:9c9b with SMTP id k4-20020a25fe04000000b00b1a64ba9c9bmr4212060ybe.1.1678354283152; Thu, 09 Mar 2023 01:31:23 -0800 (PST) Date: Thu, 9 Mar 2023 09:31:06 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.40.0.rc0.216.gc4246ad0f0-goog Message-ID: <20230309093109.3039327-1-yosryahmed@google.com> Subject: [PATCH v2 0/3] Ignore non-LRU-based reclaim in memcg reclaim From: Yosry Ahmed To: Alexander Viro , "Darrick J. Wong" , Christoph Lameter , David Rientjes , Joonsoo Kim , Vlastimil Babka , Roman Gushchin , Hyeonggon Yoo <42.hyeyoo@gmail.com>, "Matthew Wilcox (Oracle)" , Miaohe Lin , David Hildenbrand , Johannes Weiner , Peter Xu , NeilBrown , Shakeel Butt , Michal Hocko , Yu Zhao Cc: linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, Yosry Ahmed X-Stat-Signature: ay7s55jr5hxqide68gkkys49e8jck9po X-Rspam-User: X-Rspamd-Queue-Id: 1F4C21C000F X-Rspamd-Server: rspam06 X-HE-Tag: 1678354283-976651 X-HE-Meta: U2FsdGVkX18FI4Wp8f+mulmY1eApSZgTUn2mhf/CY89F5XbG7T03HndlhLyv+lJyjnD956upGwHegGjUoRd3mpZ/FuzFRNifAHpaEAd76dLcfSR3K8CyCn8wVa2QtFj8YFtZpLzqWGUiScaJqxTMh+PetKwexafCdBKHu7wZXzr1ruWcpIA0OUbDQ+FCo/KPlyxPq1G8MjLevX7l5kVMVx5bUKBLf37QQ7XMaS4b6/76w4OUm7rtS93Kbzow+gawpyp6Y7rZEXy9K61I+paVFhHN0QtMAlfPuWSMAf3aonj8V5J5mDZ/dyxg200POfPdj+fyiTv8yJVDJSUsn/7h+y8yxY365EfUFYg3o+yt8DZGPfURWbcmcZd1icooLB7941SleVy11kM3rt+ge+cDCxL7eT+pUrjRTtQWC6hSfUUl24MgqD6AAOkdtKvw5JKaeZE7Jf4MR6aD71rwo5lndyya8TMRPgDOegPVPfiZQjP7BA31x79mk0e6Fp2WTSLQgUcekjgPJpKQF9mAFdyVaI7vzBqGNAuh22VtvCua5QJyq3rpiMwNblrfVSPu/H3asVAoqmEXvCdcZuCqGfT+X4qWqukPO2hBRf50F/Y2cO0JdEe0w6YAhZBM2CuYZJEvQEqf5Qp9ZQmiabAhkzefFoPGhu1jHrNigwpb4IOIc88DXYyL2HK4BLYDN/Ct5ncFI4k1CtODA69VIDERbRnJ/BWY5v7rCh1I5094i2/XKKEdIohaCrvQVTCIhSOC5XlO32IRM0wIjz5OvfxWTvMoMT0Zv+IKK/G3ne2QkK0+zBGbVZg4sT8hWAY1GmrTVkLQ65n7OkFU6Y+5cA+/UyEtGhgKJI+td239TVXtkRw5/PeQQu+rS2NDnQJAojRUJEUgwekHN9U3YIafGjnZ430YHSw9nbDlfK5J3dOK00QJA0HGAvX9tccpc/MvfAUZb7GM8l37IJRWqHJchFjYIKm jX7Tt/cx rSXL20JJdPp+cMTs86SMusud3cInfWM2T9Jhr/kUJxZcTWa2fAYIX8xI2VsCU430oI26QzefxsHU1tqBxR/fUT1xBaY/pk226rHIKzZYx0Tarju7zvC9IaHVGUKsj7tTEWKmyBI8RM5c7NiMTVvNNg0j2gQ6PPp/JFkTdhtGk+SpuwJx2cl44HVBtduUarfcE+bEqRw+3TeSfaJr9alpRNQBTAUQVSfCeiVzwtAOpRKXRNVNsDXjgjjr4jW5chVLYw4SFNrnskY/88LnUv0CTiRbcjUbHfnz2YKOGbK/StpqcjYhPWPpLWIUo2OYuG1kLQ+RJhzTCapFERP41GKPGpwjgxPRWw3KGaDeKyxxNiEGpzZJhqjQDR+MjBxT11gl+jvBi7pAf0rCsSwEl/Uvj8IH3gkugdCZ+V+tFMWg6g/fMbNjokNw4NW6ZgVafnkOENWZlwSh5C3f+VE1nIqHEfnJ7IfQ+5+4Ki/z8LAaffcN8+wlE/lPcSYFHhrh6irDxYlAhl02P5LRPCJSwHi6AE4sDJkBr7gS0wykxKtRrI4S2Pv9vCC2vHkfDeWSo25vIZQYQinpqzJlSjNB5r+2MBWMLjtL4Qa3O0ISzY2DUI3TOsuSQ6Q3ZXkx5Fh7r4d+vMP4cae6qWD7Ss81JScV+TdqkoZcoO1xzP5GXUZ6+oG6UF4ubolVx47RRuLMRL9Q0aXXru7kL1BjK8B8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Upon running some proactive reclaim tests using memory.reclaim, we noticed some tests flaking where writing to memory.reclaim would be successful even though we did not reclaim the requested amount fully. Looking further into it, I discovered that *sometimes* we over-report the number of reclaimed pages in memcg reclaim. Reclaimed pages through other means than LRU-based reclaim are tracked through reclaim_state in struct scan_control, which is stashed in current task_struct. These pages are added to the number of reclaimed pages through LRUs. For memcg reclaim, these pages generally cannot be linked to the memcg under reclaim and can cause an overestimated count of reclaimed pages. This short series tries to address that. Patches 1-2 are just refactoring, they add helpers that wrap some operations on current->reclaim_state, and rename reclaim_state->reclaimed_slab to reclaim_state->reclaimed. Patch 3 ignores pages reclaimed outside of LRU reclaim in memcg reclaim. The pages are uncharged anyway, so even if we end up under-reporting reclaimed pages we will still succeed in making progress during charging. Do not let the diff stat deceive you, the core of this series is patch 3, which has one line of code change. All the rest is refactoring and one huge comment. v1 -> v2: - Renamed report_freed_pages() to mm_account_reclaimed_pages(), as suggested by Dave Chinner. There were discussions about leaving updating current->reclaim_state open-coded as it's not worth hiding the current dereferencing to remove one line, but I'd rather have the logic contained with mm/vmscan.c so that the next person that changes this logic doesn't have to change 7 different files. - Renamed add_non_vmscan_reclaimed() to flush_reclaim_state() (Johannes Weiner). - Added more context about how this problem was found in the cover letter (Johannes Weiner). - Added a patch to move set_task_reclaim_state() below the definition of cgroup_reclaim(), and added additional helpers in the same position. This way all the helpers for reclaim_state live together, and there is no need to declare cgroup_reclaim() early or move its definition around to call it from flush_reclaim_state(). This should also fix the build error reported by the bot in !CONFIG_MEMCG. RFC -> v1: - Exported report_freed_pages() in case XFS is built as a module (Matthew Wilcox). - Renamed reclaimed_slab to reclaim in previously missed MGLRU code. - Refactored using reclaim_state to update sc->nr_reclaimed into a helper and added an XL comment explaining why we ignore reclaim_state->reclaimed in memcg reclaim (Johannes Weiner). Yosry Ahmed (3): mm: vmscan: move set_task_reclaim_state() after cgroup_reclaim() mm: vmscan: refactor updating reclaimed pages in reclaim_state mm: vmscan: ignore non-LRU-based reclaim in memcg reclaim fs/inode.c | 3 +- fs/xfs/xfs_buf.c | 3 +- include/linux/swap.h | 5 ++- mm/slab.c | 3 +- mm/slob.c | 6 +-- mm/slub.c | 5 +-- mm/vmscan.c | 88 +++++++++++++++++++++++++++++++++++--------- 7 files changed, 81 insertions(+), 32 deletions(-)