From patchwork Fri Dec 16 19:21:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13075337 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9E67AC4167B for ; Fri, 16 Dec 2022 19:21:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3B90E8E0005; Fri, 16 Dec 2022 14:21:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 341518E0001; Fri, 16 Dec 2022 14:21:55 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E3D58E0005; Fri, 16 Dec 2022 14:21:55 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 0F2938E0001 for ; Fri, 16 Dec 2022 14:21:55 -0500 (EST) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id CF0CB1A1214 for ; Fri, 16 Dec 2022 19:21:54 +0000 (UTC) X-FDA: 80249139348.13.896CF75 Received: from mail-pj1-f54.google.com (mail-pj1-f54.google.com [209.85.216.54]) by imf01.hostedemail.com (Postfix) with ESMTP id 3190940005 for ; Fri, 16 Dec 2022 19:21:52 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=puCmKh+r; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1671218513; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=Pvnk/vLTbFy/R8GqGM3ltG2RQwB1ZvvQeN2YibcBykLiNPOCBToEfVRx0K2MvmXJ4ytJe8 RUwWzfe9ij1nJBB8UVEAgJRfWPKVN3g3vNy6/m3jfgerTJYJP3btr7tRsJvbtV9YkF4rFv PPzUpGGK/+fVJ8CYR0HT9rPvx3mzxOE= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=puCmKh+r; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (imf01.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.54 as permitted sender) smtp.mailfrom=nphamcs@gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1671218513; a=rsa-sha256; cv=none; b=B8Lg+Gyv2BO9+Dcn1+GEzrVEepgN/aoaSGrfYy4rkrE5uUATOaU1YkfRsi4x3088RFb7tu VOImdBfbzBxmWw8KLObCTCwlPuS0P30eaYyMiHmrN8NdLIO73tdqmpg/m8U8u3wtlWT34n iSgss+91iCaTJNUolQXaSODYLmFPmHA= Received: by mail-pj1-f54.google.com with SMTP id b13-20020a17090a5a0d00b0021906102d05so3324461pjd.5 for ; Fri, 16 Dec 2022 11:21:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=puCmKh+rn1JVyyUrtJjttd3FuNpbSs+RvZ8ZxzxDamK/Cuevoa4qtVifp0nUVsS9lh 0su5AgtBOtGAeJcTOGyuWko0BwZQ7Hw6oQcDbbq9Y5NKHzQG9VITawrQHm9EI3qQUtXM AjUeShMh5cwQvSmrSrgxNusU8MJ0Ps5grqGE16Jv+htNHDWoIqXfzOF/h7F71mmC2Q1E NNvHAUFdO6YKiZApC81EFhAdwOFuqt2/XO3bAgLN5JBLh/DKhH01Xf0pI87uxLqwhDrf y630abcVgNDikwUEr0+Rj+UvB/s9o4pMrTtTDexj2Un1ME0s2kgTDdYZM42tUCJ8TC7f LBLg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=tIzNohKd3nz7C3ofhjzw6PCmnRcbGCV8zQqP2colqirXnkq7qv1m39BLYPTHCzDxTg dOZs6rmptCfREoTMg7KUxwJy4au1avWRPzmiN9WB5OrUPtai0aZYPnG+7bdVABY5U5CC 2U2A2dSKW/1xCFrvPL8w9cBWrfFfYYZIz1C6ieVfT+6jA/L5/JqHOBXeIOdiWGPHMV+R yae3gRwzCwYAldlJMmEfuVQa7eoy4E8+c9rBCPo1JkO+wItbDiIKtjmZPq+ly26ya9wJ F79niE9ZJjHQkBkAA8uh2Pjek2NTnh+KVg2PYQUCh5Q2M94EUA/ZqUpX+EKzewZJAeIR G55g== X-Gm-Message-State: ANoB5plxH8AnFVeaK5q0iWgYPbeua3VkVH0ftaQcIrII/CXxNs7N6HVH JNpX/8HXf+maVUnVgbcxOFE= X-Google-Smtp-Source: AA0mqf5iiz7kn+rL8ZNVOrzT6DCoAp/rEnkSuQcxifioDG25jx57bGRL2eOaG2gjKzao6LEbgDEnXw== X-Received: by 2002:a05:6a20:12cc:b0:9d:efbf:8151 with SMTP id v12-20020a056a2012cc00b0009defbf8151mr48168779pzg.26.1671218512074; Fri, 16 Dec 2022 11:21:52 -0800 (PST) Received: from localhost (fwdproxy-prn-011.fbsv.net. [2a03:2880:ff:b::face:b00c]) by smtp.gmail.com with ESMTPSA id q20-20020a631f54000000b00476c2180dbcsm1814843pgm.29.2022.12.16.11.21.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 16 Dec 2022 11:21:51 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bfoster@redhat.com, willy@infradead.org, kernel-team@meta.com Subject: [PATCH v4 1/4] workingset: fix confusion around eviction vs refault container Date: Fri, 16 Dec 2022 11:21:46 -0800 Message-Id: <20221216192149.3902877-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221216192149.3902877-1-nphamcs@gmail.com> References: <20221216192149.3902877-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 3190940005 X-Stat-Signature: utsnwd7p7no5qt9ckdege9it5mmcjipg X-HE-Tag: 1671218512-789701 X-HE-Meta: U2FsdGVkX1+i8mmYM9BN6RHoLIzfpQgyZ8JHmDW/MtTcPKcp56Z25bisaAdXIkHVYqKmV3x0mmuAjJ+8Hfh1TQepUt48lKoEDmAkmDq1FetiaEoZfq88y6v3lk9LNbEGbNjXRL8nD+WM5WgeUCn5+BV09kLKwJQgFax9d6bQvjat1w6du55Lg+h/yQN4hCrmkCMz1ei8BNhdkfDL10/Ay5GqHwWHBO2XuGCDr922ET6SWVc7cO0QaQfQk65R92FE01QF7JbyjB9+vAG9zV9ITNQorotfhunZ5FQ9GYpmkkDh8zSbH6u6rOO3YfXHY/7NrXeSFtOP8n6LfY3VOXMI/lC3O/xdUKdenku3ooe/NHEDIKizhaSYPSSBUvXwsS9oxUwaoFR1cGV93/fnJCK2xWeBb8c8WLx7ZIZpUqYYJP/h7zaerp2EVGdhoKBQq4yfAg8qXVQj59M0bIVxs+k5NP7mY6Bp9QkRo/W2ZxY58Mr90rwXBPBj7nYejzJqTaWazt7AxSK50lOUDbWVyp1c/lMMNWnr6BQrUivXPspb3m0tiDXsaqa0Ri9G1L5tl6F8CvXgqePj9AA143fh0S+lUZAm7taU3FjvUqzto6qYXv/Z0vV6P0WSl8PQXl3I9OF8rzDwDNdD9HInvIts1waU/5EB6pYoLGrQljYwtmCyU4mws295l1RTgm5phcO4mV7dBapgBjRNy8PZ+SVQCziGTAkUdCv0t3xFhIHIqErrNEgQ945umj+PaZibsb63tqp32hUGQ8/+QKRdRbScbedBKn0FrN/T+L3teJTKKJilioEmc5gBleZkB8DfwaYj2CzCxAu/tfC9Bfh1Uj+YlHt0r+n4bNgiaj/mmFVEgCNQGZy9havqhxFSoIS+foxEpp7R62OVOwrIG018F5ktQ5KSNZnZvsALvh2ymTYLdsGhUMMeW1fGSvcodzcRY9OBBDHuXuwE0AepTq/K8tOTYEb rlEWicyV rcUC3G5ILXr905iUIk/crEP14CefEW84JXNkxPhAraCIch0yRzT49JkMsukHVwTxWa43UEusfWrzhdnlIo6gREUnBfXtSySXLS9tvMqZuUXrC9FV/eV3wxISDkgFBlGltsUVk X-Bogosity: Ham, tests=bogofilter, spamicity=0.011115, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Refault decisions are made based on the lruvec where the page was evicted, as that determined its LRU order while it was alive. Stats and workingset aging must then occur on the lruvec of the new page, as that's the node and cgroup that experience the refault and that's the lruvec whose nonresident info ages out by a new resident page. Those lruvecs could be different when a page is shared between cgroups, or the refaulting page is allocated on a different node. There are currently two mix-ups: 1. When swap is available, the resident anon set must be considered when comparing the refault distance. The comparison is made against the right anon set, but the check for swap is not. When pages get evicted from a cgroup with swap, and refault in one without, this can incorrectly consider a hot refault as cold - and vice versa. Fix that by using the eviction cgroup for the swap check. 2. The stats and workingset age are updated against the wrong lruvec altogether: the right cgroup but the wrong NUMA node. When a page refaults on a different NUMA node, this will have confusing stats and distort the workingset age on a different lruvec - again possibly resulting in hot/cold misclassifications down the line. Fix the swap check and the refault pgdat to address both concerns. This was found during code review. It hasn't caused notable issues in production, suggesting that those refault-migrations are relatively rare in practice. Signed-off-by: Johannes Weiner Co-developed-by: Nhat Pham Signed-off-by: Nhat Pham --- mm/workingset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index ae7e984b23c6..79585d55c45d 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -457,6 +457,7 @@ void workingset_refault(struct folio *folio, void *shadow) */ nr = folio_nr_pages(folio); memcg = folio_memcg(folio); + pgdat = folio_pgdat(folio); lruvec = mem_cgroup_lruvec(memcg, pgdat); mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file, nr); @@ -474,7 +475,7 @@ void workingset_refault(struct folio *folio, void *shadow) workingset_size += lruvec_page_state(eviction_lruvec, NR_INACTIVE_FILE); } - if (mem_cgroup_get_nr_swap_pages(memcg) > 0) { + if (mem_cgroup_get_nr_swap_pages(eviction_memcg) > 0) { workingset_size += lruvec_page_state(eviction_lruvec, NR_ACTIVE_ANON); if (file) {