From patchwork Tue Nov 15 18:28:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Nhat Pham X-Patchwork-Id: 13044070 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 32758C433FE for ; Tue, 15 Nov 2022 18:29:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B83726B0072; Tue, 15 Nov 2022 13:29:06 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B33B56B0073; Tue, 15 Nov 2022 13:29:06 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FB586B0074; Tue, 15 Nov 2022 13:29:06 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 90C256B0072 for ; Tue, 15 Nov 2022 13:29:06 -0500 (EST) Received: from smtpin25.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3C58B140A09 for ; Tue, 15 Nov 2022 18:29:06 +0000 (UTC) X-FDA: 80136513492.25.EE555E8 Received: from mail-pf1-f178.google.com (mail-pf1-f178.google.com [209.85.210.178]) by imf10.hostedemail.com (Postfix) with ESMTP id CFC19C000C for ; Tue, 15 Nov 2022 18:29:05 +0000 (UTC) Received: by mail-pf1-f178.google.com with SMTP id 130so14864850pfu.8 for ; Tue, 15 Nov 2022 10:29:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=NPDHr0QlHVIUIhIrDfHxvLDIuBOtKZfo7tv9Z1S0sFSK9TGsnpiguY2DVaj8KQqkUg 2SugxfLILvY2yZb9mSRU2q3++D6K9Vj+iOyidmF/ozO/p/7XOGWmFRfizSWm5iaLk5hj il4Q4TvN2HUyX2sk4tSfr2n2+tzXd77V5UybYcVVdBEnLxiMpx20HM6oe4QsCFyHtEs2 LJ3vSboCcqML9M7WDu6BnBSGDB+A5Hgh9HctMZlRn/EuhBDkfp2Exqkb6OqkY16lFgjL mqYbnVzmCbSOyK+6Jgh/buBRjxAFZ5UldLk6C7sULrRpi2KwbgfgjYgFwkYbz8uiAGV1 sx/w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=T5IzUIXSCsAppRO4YyZ9uyMk1+i4TpEfNbNvlBi4dDUtbVeGNkli0FpaiGZtzOwIEd 489zmA/YUY9BGEcuS/gdj/gawdP+psOitXakcgKPdedRkywHBFpNNdupFNU18g4U7XJ2 jLd92mU+UekJ0cWU1WmJ9mN2BTtk6DZf3rT4LTmEseTpA5YNRJCKNa1dD/U8fOes942a +iad3xpLtMN1GGJDotsn53Q7LUYsbeiH/+4ADSQF05bTSmRFjizDBI8jriZDrY5/ogwk h3YZRx2DqtXsVJlmr4vaU3uSL4GILlgx66do4hkuoaxuhvpP1CvIn1hvxHtUULDsJ7wn m7Gw== X-Gm-Message-State: ANoB5pkQE7MaOq3PmqqqxKtUVOjxA/9XfNDZkBP4EkF2/PDIRX07SGU8 lpB5vWQ6UMuxkRDLsRC4ageWpfcIcbf02Q== X-Google-Smtp-Source: AA0mqf7wqzv071P0x/mRXjpwael9ewocrXAjhS7lO6DJTDtlWHz7zdSkAz307rNNRRW624ZDI3KXTA== X-Received: by 2002:a63:560c:0:b0:476:9983:b4b5 with SMTP id k12-20020a63560c000000b004769983b4b5mr6888328pgb.516.1668536944373; Tue, 15 Nov 2022 10:29:04 -0800 (PST) Received: from localhost (fwdproxy-prn-116.fbsv.net. [2a03:2880:ff:74::face:b00c]) by smtp.gmail.com with ESMTPSA id k15-20020aa7972f000000b0056bbba4302dsm9034046pfg.119.2022.11.15.10.29.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 15 Nov 2022 10:29:03 -0800 (PST) From: Nhat Pham To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: hannes@cmpxchg.org Subject: [PATCH 1/4] workingset: fix confusion around eviction vs refault container Date: Tue, 15 Nov 2022 10:28:58 -0800 Message-Id: <20221115182901.2755368-2-nphamcs@gmail.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20221115182901.2755368-1-nphamcs@gmail.com> References: <20221115182901.2755368-1-nphamcs@gmail.com> MIME-Version: 1.0 ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1668536945; a=rsa-sha256; cv=none; b=J7U8ySL7OkXhO2Pn8Tn3Nopy3LISkKjNyqqo6ald/1QsBI146xy6ZQcpSpT1ydSzQVlf0c wODvb02V5edXj6+bf6DJh6lD3c6YRyoki6bVxCuEvTfMs60skEEFnLfV9/NFKtkPOZv4BM Wc1kbKVsDc/XhucngEBpXTgSKdEFTH0= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NPDHr0Ql; spf=pass (imf10.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1668536945; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=npFjL6Dds3DKnQo07iUTlbqXMaucW9/yvERNl8ZNQRo=; b=wFYfV25OtHF/UXahfCjP5f4b7pO9bRROOvU80PB1/v4iZyd82TckdFOyKLaCECRawCF/gH uKdSzroWtVja/Y6QMiWIw/Ct+sh5Spnr0DLAhE1sKj/Jb10GyUzKye5eJ8mJMVGmYEXqan wbepIya+cDLNIW6Zj+c/aAZtD1usf7Q= X-Stat-Signature: 4croqodadewrydmuu9ofj1edtin5nkpk X-Rspamd-Queue-Id: CFC19C000C Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=gmail.com header.s=20210112 header.b=NPDHr0Ql; spf=pass (imf10.hostedemail.com: domain of nphamcs@gmail.com designates 209.85.210.178 as permitted sender) smtp.mailfrom=nphamcs@gmail.com; dmarc=pass (policy=none) header.from=gmail.com X-Rspam-User: X-Rspamd-Server: rspam12 X-HE-Tag: 1668536945-823009 X-Bogosity: Ham, tests=bogofilter, spamicity=0.006472, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Johannes Weiner Refault decisions are made based on the lruvec where the page was evicted, as that determined its LRU order while it was alive. Stats and workingset aging must then occur on the lruvec of the new page, as that's the node and cgroup that experience the refault and that's the lruvec whose nonresident info ages out by a new resident page. Those lruvecs could be different when a page is shared between cgroups, or the refaulting page is allocated on a different node. There are currently two mix-ups: 1. When swap is available, the resident anon set must be considered when comparing the refault distance. The comparison is made against the right anon set, but the check for swap is not. When pages get evicted from a cgroup with swap, and refault in one without, this can incorrectly consider a hot refault as cold - and vice versa. Fix that by using the eviction cgroup for the swap check. 2. The stats and workingset age are updated against the wrong lruvec altogether: the right cgroup but the wrong NUMA node. When a page refaults on a different NUMA node, this will have confusing stats and distort the workingset age on a different lruvec - again possibly resulting in hot/cold misclassifications down the line. Fix the swap check and the refault pgdat to address both concerns. This was found during code review. It hasn't caused notable issues in production, suggesting that those refault-migrations are relatively rare in practice. Signed-off-by: Johannes Weiner Co-developed-by: Nhat Pham Signed-off-by: Nhat Pham --- mm/workingset.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/mm/workingset.c b/mm/workingset.c index ae7e984b23c6..79585d55c45d 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -457,6 +457,7 @@ void workingset_refault(struct folio *folio, void *shadow) */ nr = folio_nr_pages(folio); memcg = folio_memcg(folio); + pgdat = folio_pgdat(folio); lruvec = mem_cgroup_lruvec(memcg, pgdat); mod_lruvec_state(lruvec, WORKINGSET_REFAULT_BASE + file, nr); @@ -474,7 +475,7 @@ void workingset_refault(struct folio *folio, void *shadow) workingset_size += lruvec_page_state(eviction_lruvec, NR_INACTIVE_FILE); } - if (mem_cgroup_get_nr_swap_pages(memcg) > 0) { + if (mem_cgroup_get_nr_swap_pages(eviction_memcg) > 0) { workingset_size += lruvec_page_state(eviction_lruvec, NR_ACTIVE_ANON); if (file) {