From patchwork Thu Dec 8 20:28:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13068845 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BCCABC001B2 for ; Thu, 8 Dec 2022 20:28:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1DDD38E0006; Thu, 8 Dec 2022 15:28:14 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 13F928E0001; Thu, 8 Dec 2022 15:28:14 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id EFAE68E0006; Thu, 8 Dec 2022 15:28:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id E05D68E0001 for ; Thu, 8 Dec 2022 15:28:13 -0500 (EST) Received: from smtpin14.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id B40BDC0BE8 for ; Thu, 8 Dec 2022 20:28:13 +0000 (UTC) X-FDA: 80220276066.14.28C60A1 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by imf07.hostedemail.com (Postfix) with ESMTP id 032AB4001E for ; Thu, 8 Dec 2022 20:28:11 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PtrML52y; spf=pass (imf07.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.215.172 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1670531292; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=Lb+KwmJkzXx8OFTtW+nTccBj0LGz6VbdaAXwuN9Uf31fkDfpRrzhaYNhvWi7DbLoGkSw+w KqQKoX0SjsjN4fLUdBugXH6tyuJoJKm4F9t8OTMjSS+GgDJI8vsKQ5hKKlSf9167YUIOB7 kNnpEHr21INmTNcirwJWnHHkhIDZ9B0= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=PtrML52y; spf=pass (imf07.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.215.172 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1670531292; a=rsa-sha256; cv=none; b=JIhCD/18AmoPOszJywQEr0JSBd2gKGxcfiVjUCNTOaoINS08ob54vvyPBvcPr4X7c6uKpF MoIGJU/wkrOoYzBoArgVD2cNAF/lmNaA18QDXJ50ojdot/Yd96IQZCNmNSyV5mZFeTodG9 /kmPr4bTm5KB2Mub0pbMZ/pdir/9Z/Y= Received: by mail-pg1-f172.google.com with SMTP id q71so2087046pgq.8 for ; Thu, 08 Dec 2022 12:28:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=PtrML52y4++BjkDz048pa6CUOhJB/mkeWNTiPY7aI2UbgF7O0oTXxRSB+Rgt+XOkh2 IhRH1bB7exc+5heFlRaaYr1yjjAwmfAuerp1aWipjPeaCOdr512Ka1FELYXazYxmhQEi SI7xS7B+WXTt+l2S403kfG2hbvrX5IAdut2oWDHfpaLlRGt4YV6g6R2N/Y+M0OIVauM4 yspqne9tauDtNmi+n/6QI2qbmUkn/MYPIpdZcxb09OW88KyKsov7q5fHtFuGQCLf0/61 TA66+MiESyfsohxgdTOk04PCX9QDVLvzcic6TUKUmrBvMvg3B2oikbP6Edti5vGkX47C FKMQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=rBhftLAHwFPofQQZltmF9p52QRgEhPAwVUCWLT7+aGCR4Cot77d/vDZLutXl3fngP0 T4kxHtgIqJXmeWTX9IQSUI0aL8PmfJuKwF3+fbqQU47zsU8bkvkn+mRdrubNGOeydDTY QbGLwb77UJlVgxJIBzGk3hG1Rr5FzX8kH3d8h3pIf8XoQs8JKCP1n9h/iD60032YYEip 7sKr5vqfjZmWkZE97JPbpmpUKeT6IF6d/D1SmUGB9G4zHC/IajjlM2031KYcARJPSsoa hU6emGDy+fMuqWsouBlNq4gIUgUqynO6OdmtHntAwOoCERW3LWjgVFVYBdXuOGNnH+D3 Czhg== X-Gm-Message-State: ANoB5pm6Dbo9Z/ZTGXOYDeD8G1mwEZrQ0RextD+mH8/s+oit7Os1iytQ 7flxa2Ka3Tpqye4W4kYi+HA= X-Google-Smtp-Source: AA0mqf4PzirRAa+DHb5HDZfpPwkKugYCrcQU4IMR06a44BnH7VGJOBKJqvkTVtEYWESsKuUDvBw8gQ== X-Received: by 2002:a62:3142:0:b0:56d:8d19:f331 with SMTP id x63-20020a623142000000b0056d8d19f331mr99706326pfx.7.1670531290779; Thu, 08 Dec 2022 12:28:10 -0800 (PST) Received: from localhost (fwdproxy-prn-008.fbsv.net. [2a03:2880:ff:8::face:b00c]) by smtp.gmail.com with ESMTPSA id b29-20020aa7951d000000b0057555d35f79sm15702101pfp.101.2022.12.08.12.28.10 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 08 Dec 2022 12:28:10 -0800 (PST) From: Nhat Pham To: akpm@linux-foundation.org Cc: hannes@cmpxchg.org, linux-mm@kvack.org, linux-kernel@vger.kernel.org, bfoster@redhat.com, willy@infradead.org, kernel-team@meta.com Subject: [PATCH v3 1/4] workingset: fix confusion around eviction vs refault container Date: Thu, 8 Dec 2022 12:28:05 -0800 Message-Id: <20221208202808.908690-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221208202808.908690-1-nphamcs@gmail.com> References: <20221208202808.908690-1-nphamcs@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: 032AB4001E X-Stat-Signature: q3cfdn7tghk9d7if1wgc7mfoi9wc3ytp X-Rspam-User: X-HE-Tag: 1670531291-734828 X-HE-Meta: U2FsdGVkX1+xRK/zafcI9b1CGB19W+vQYe6RCTFSNsbZHBgYmkWan7U22VARLf0kqPbjrPgE6PHJ6NKshf+EUyKn9rLAM841cJtd4IA4r5fRtCUIfV95gNVnX1InLsN6pWmYtpT5uTDIsnuaGaHB0+4j48h9o109DTtdB2vveb7jUs3GTOobPlcZ0W5xdgvuR40o6/lWXXXsJiW4fk+EwzcaXOj6t4eHnJt7kZsLZvgkJ1RoKCepz7mGxgj9rh39iuEvGFN984iIJfeGy1mqUOn2Z8UClTLq+Pe1bfqbTLVYpCQ2TmiS0xyXw+lBd7NsWaDBIlIeF6OaUKKWap3AzgpWI//g94bPmxd8uUw17HynCddUlOpfbBN8T7el7zN2fwB6ACnnzx6uNL61s2ylgrQri3D/Kla00mhaAS3F4qQeIQEKxhQKQRZ+ChcExf5P8rcTpj2UZaqzKUeAlHJm76seAqyiOmRqZCDpVxRxkUx1MZ36hEgrycrq5sKAM5ise4jpSBf+eUsApKKnZs7EJ1aQ+uhin9YF1bDPQgr4+FBv5dPBIkrcYivl9ja1lkeAf7EaGUm/1QN/qeZ1No+lZ6dvHWZQSamyxacLzJXWy4M/B8EuusSnfOLxJ5H2Zw6Cl3ZSQy+mNJucKrE5EsDXsUJFXrCCrKajYw8oCjvQXLBdKfetrkfC46ZIBUiAye6nsknDFJ54U8D+exDS3ZKdmWc9ls/p93pVb/fjSCazRFE+woNxYTNRrzDoycBzg7/xMUURcJEjP1ypUHiW0vSyWr8KC8ciCdZ4jOV0LWagbWKTtDl5ZhnVU64mgHPx9vVKj/Zlumzdrk+WhgHShk/qWUz8oP0QiTEUWcENuCiVwFV0yhwxOAQIqLIWN5qBMWhFtdnwARJblmnRrI7y3bkj7Zk7Q0sk9w3SQzpr3XGHPLf0BQhob9WbXzxdUusbFFtmz7m8DgjUu3M142rvRSc l6IrP9ac jU8zBap6ywCPYa+t4u0IlAq/FurMjWbBaVztKNa8PLQstKwCwLfWxRnWvuf4jnyqLwDv/rkIN0mU141ygyHffhLeNEHOkFDv8nD15JhHMEYQ/z5NoFLo01SWOl6qayIen6zGz X-Bogosity: Ham, tests=bogofilter, spamicity=0.000063, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Refault decisions are made based on the lruvec where the page was evicted, as that determined its LRU order while it was alive. Stats and workingset aging must then occur on the lruvec of the new page, as that's the node and cgroup that experience the refault and that's the lruvec whose nonresident info ages out by a new resident page. Those lruvecs could be different when a page is shared between cgroups, or the refaulting page is allocated on a different node. There are currently two mix-ups: 1. When swap is available, the resident anon set must be considered when comparing the refault distance. The comparison is made against the right anon set, but the check for swap is not. When pages get evicted from a cgroup with swap, and refault in one without, this can incorrectly consider a hot refault as cold - and vice versa. Fix that by using the eviction cgroup for the swap check. 2. The stats and workingset age are updated against the wrong lruvec altogether: the right cgroup but the wrong NUMA node. When a page refaults on a different NUMA node, this will have confusing stats and distort the workingset age on a different lruvec - again possibly resulting in hot/cold misclassifications down the line. Fix the swap check and the refault pgdat to address both concerns. This was found during code review. It hasn't caused notable issues in production, suggesting that those refault-migrations are relatively rare in practice. Signed-off-by: Johannes Weiner Co-developed-by: Nhat Pham Signed-off-by: Nhat Pham --- mm/workingset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- 2.30.2 diff --git a/mm/workingset.c b/mm/workingset.c index ae7e984b23c6..79585d55c45d 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -457,6 +457,7 @@ void workingset_refault(struct folio *folio, void *shadow) */ nr = folio_nr_pages(folio); memcg = folio_memcg(folio); + pgdat = folio_pgdat(folio); lruvec = mem_cgroup_lruvec(memcg, pgdat); mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file, nr); @@ -474,7 +475,7 @@ void workingset_refault(struct folio *folio, void *shadow) workingset_size += lruvec_page_state(eviction_lruvec, NR_INACTIVE_FILE); } - if (mem_cgroup_get_nr_swap_pages(memcg) > 0) { + if (mem_cgroup_get_nr_swap_pages(eviction_memcg) > 0) { workingset_size += lruvec_page_state(eviction_lruvec, NR_ACTIVE_ANON); if (file) {