From patchwork Wed Jan 4 22:29:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13089148 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C47CEC46467 for ; Wed, 4 Jan 2023 22:29:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 11FDB8E0002; Wed, 4 Jan 2023 17:29:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0D0078E0001; Wed, 4 Jan 2023 17:29:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id ED95D8E0002; Wed, 4 Jan 2023 17:29:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id DB9868E0001 for ; Wed, 4 Jan 2023 17:29:32 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id A4CB81C654D for ; Wed, 4 Jan 2023 22:29:32 +0000 (UTC) X-FDA: 80318559384.22.D99C9A1 Received: from mail-pj1-f52.google.com (mail-pj1-f52.google.com [209.85.216.52]) by imf29.hostedemail.com (Postfix) with ESMTP id EEE5712000F for ; Wed, 4 Jan 2023 22:29:30 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Hfe+MFjZ; spf=pass (imf29.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1672871371; a=rsa-sha256; cv=none; b=6ZAPoFZ1xG7HFLLFxF1yVcCB8jV4QL//C2qyAkgt0pcquXi5AMC2sPdcJ+warjbFjCvse+ SwNhLaMUVlT37CA9afR2gy0mQPNPhG7P0aVep35Qx5fSJ/1TGMgYz/mzWD46ICAnu7XIBN Wv66e0Q6bzyyzn7AhvEQPdylXHAB8bg= ARC-Authentication-Results: i=1; imf29.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=Hfe+MFjZ; spf=pass (imf29.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.216.52 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1672871371; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=Va9lFegXCnpjRrn7J3MgHZJCIRrNEtKRpxqq34RY1hINyDlCrOoBiYz3J6OFYQaO7B9ZLP FlXTcP4BapKLeelkWoAXBxWjDeCAmxEMM9o5fi9V8hKnbm91uBGBVi0rZKxgqinp8K7Tsw whmkYRFqJLErJ4lfLv655iaOVl+fmVk= Received: by mail-pj1-f52.google.com with SMTP id ge16so34126728pjb.5 for ; Wed, 04 Jan 2023 14:29:30 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=Hfe+MFjZwf4GhbRTb/n0MeBe4tDdfcoDltOF8kUyq95lxJGT1bmwJ3terbJbTvdYgA T3F2vtAqIapoYNnxHgXL0lEy0m6cDr2zwMiUrfMcr5ObyooUGPZ1ilXxaeYA5qBFI88V nUVNDLkDhMco0foAWAF5M5JeYAb4dCcZIGlpyUo6rILHOFH87NsUbwytwTdd5pvcFsqN lFl1ze5EfmtZWQyo53QWgSPHE6F1uUhN9rwM8TJ6WnLU+UDzyEsHEKE9IdB5VeUn4MLl NOme36sCBKuMJieR7fJ4Op2wc1uc1mtAzm8+vVY9BIw7ko2Brfs4RW5pG33DrfBjZdjF eVqA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=JyOuGit3iBdZrJM4o5G45y3tORtpPLyQmedii8GuiL9BsE/Yoruvtbsv3B0U09Oxpo /3jreeDmFdNWZWhzb5JmMmU6km0bBfA+8hn5c99bad8yIlPQ6kGMGgEi7YzO91/vRQYk mCX5RV9Xqo6ONQVimQ8Wk3DRXoedqgrUWucwuND66pCM2epMveO0TWFTA9LDvXLn5UQF lLR+/Wx2Jfn96ilpmqW3q3MXvUKNJXCmbRMHyb5kYi2bQBxahPoxrCoKoFFlOvaFNHhr O82v/e7e2mmervX9hxRS3UgGDMUW3QDQdYck6b3+HRoSQcz+72b3orCVTwKnshx3Ckk4 nZ6Q== X-Gm-Message-State: AFqh2kohCVjIojiziTMAfy1PGLSfxZXFLT13Hg0vELWH7jz0W/Kl5pmo 3y+en4YOxPZLDFbV6QleB9g= X-Google-Smtp-Source: AMrXdXvjHO1/f6Sq+pgRKHHQ4iuZMwTMptCrxrPJngmrmRWnM+LMgeTWuvpeSQVWPX/ja0ASqBHniQ== X-Received: by 2002:a17:902:ab85:b0:192:f469:5283 with SMTP id f5-20020a170902ab8500b00192f4695283mr2563352plr.3.1672871369638; Wed, 04 Jan 2023 14:29:29 -0800 (PST) Received: from localhost (fwdproxy-prn-000.fbsv.net. [2a03:2880:ff::face:b00c]) by smtp.gmail.com with ESMTPSA id t9-20020a170902e1c900b001927ebc40e2sm19138025pla.193.2023.01.04.14.29.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 04 Jan 2023 14:29:28 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bfoster@redhat.com, willy@infradead.org, kernel-team@meta.com Subject: [PATCH] workingset: fix confusion around eviction vs refault container Date: Wed, 4 Jan 2023 14:29:27 -0800 Message-Id: <20230104222927.2378210-1-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: EEE5712000F X-Stat-Signature: 6xtxfcr86uaxs43dmthk8rxx8fd8dncd X-HE-Tag: 1672871370-25171 X-HE-Meta: U2FsdGVkX1/e77zrv/4TzzL4QbQb5vDHrMzpa2jh5fOqxJo2sh0opEYNHaSnAbi908QzAqBAJty9QWLnWMzzQKsUXiZu3fxChy4ToAQ4Pn3iQtPUcf9Im1vAlzAi7zrxeib6F6MllFNgwvTTFyxkyIGHuJ6Om5eLe0INT/BJVFVq+wLsNG+ROo8hSZJIboWUmTe8yvg+UJQxP6aJortB3SM0oT4JhU/LwRk5xvSX/WjOdij2at5S2WkAMf/lqSaroaDXLee3m5TqmqYIubnaR3gBlPYdcCg4RHvX6EQDiL9Le/MrCAtwPc4TM1BNHUlagVKDiyPEWMMmDiF/EXq7ZDdAUs9pcXclPvrGBu/3aNCKwq81Lmh8pxA8lhOXTHJXG6VGLd8p3Ldter3L/6u+yEaRyvRG6DGM2UfF93iAiGtwYtR4D314usJIJ4dGkDXyIoHiwlGGPgHlgt4GY9D8e8o1sGwIGTtm/ZYFeqlNwad/27RDBbBN8eu5E7UtmL0MEIMMHUiXUz6LLO2DcxIS5VYPDA1VMMhHgnTKLYpbz+RQ6WwWeLEf3TggaLJnaVMAB++e1Et33+d+Ey2JBxQlDcwH9WvRPR0dCLR75X+pC5KqGgG8BS86e0JX1AFgq0nBIIhv+LTXFm893WCJ42Vb207U544b8CrmQ8nb1tECzwxk6IB+g/svoNONMwypNfvHlbvrbfzxcNL/B5qn5u7C7Uoic66iOgQBhRIpL6Otw+SX6+omOHrgSY6JYDvoiszPr0VBNdd4iVSlzG1R03c/7d6gu+D0koAHFlXsuJJl/s9S3PHhNJaMHWtpNo/8M+IF77B9WGntzEdxPDAFnklXjdWT7FQT4DVemd6qmebQDohyQoo7x8bPxHwS+RIEo39PJ2fBaev5q21g7k8fjWamaJWymf7kZ8LnXw59IiGlnGeEBeov0KmFQlfOsuvTiUtstmi1lnmbGp2fLBp58Py tqY5gYq9 Ucgl/dzyv/aGc9cUMChbim1A/A11GKLsTfxqds4IXzepfNPFrl8dyj/NC6jPn5w4LYaddnWuY6tSN4HXfhZKhZOaDjqMzEZ9VciYxydiINjfExeVlm5fVLUdy8BGOcxV27Nqi X-Bogosity: Ham, tests=bogofilter, spamicity=0.035094, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Refault decisions are made based on the lruvec where the page was evicted, as that determined its LRU order while it was alive. Stats and workingset aging must then occur on the lruvec of the new page, as that's the node and cgroup that experience the refault and that's the lruvec whose nonresident info ages out by a new resident page. Those lruvecs could be different when a page is shared between cgroups, or the refaulting page is allocated on a different node. There are currently two mix-ups: 1. When swap is available, the resident anon set must be considered when comparing the refault distance. The comparison is made against the right anon set, but the check for swap is not. When pages get evicted from a cgroup with swap, and refault in one without, this can incorrectly consider a hot refault as cold - and vice versa. Fix that by using the eviction cgroup for the swap check. 2. The stats and workingset age are updated against the wrong lruvec altogether: the right cgroup but the wrong NUMA node. When a page refaults on a different NUMA node, this will have confusing stats and distort the workingset age on a different lruvec - again possibly resulting in hot/cold misclassifications down the line. Fix the swap check and the refault pgdat to address both concerns. This was found during code review. It hasn't caused notable issues in production, suggesting that those refault-migrations are relatively rare in practice. Signed-off-by: Johannes Weiner Co-developed-by: Nhat Pham Signed-off-by: Nhat Pham --- mm/workingset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index ae7e984b23c6..79585d55c45d 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -457,6 +457,7 @@ void workingset_refault(struct folio *folio, void *shadow) */ nr = folio_nr_pages(folio); memcg = folio_memcg(folio); + pgdat = folio_pgdat(folio); lruvec = mem_cgroup_lruvec(memcg, pgdat); mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file, nr); @@ -474,7 +475,7 @@ void workingset_refault(struct folio *folio, void *shadow) workingset_size += lruvec_page_state(eviction_lruvec, NR_INACTIVE_FILE); } - if (mem_cgroup_get_nr_swap_pages(memcg) > 0) { + if (mem_cgroup_get_nr_swap_pages(eviction_memcg) > 0) { workingset_size += lruvec_page_state(eviction_lruvec, NR_ACTIVE_ANON); if (file) {