From patchwork Tue Dec 31 04:35:32 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhao X-Patchwork-Id: 13923613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3890BE77188 for ; Tue, 31 Dec 2024 04:35:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD7526B0083; Mon, 30 Dec 2024 23:35:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id B5B486B0085; Mon, 30 Dec 2024 23:35:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9F9FA6B0088; Mon, 30 Dec 2024 23:35:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 779E36B0083 for ; Mon, 30 Dec 2024 23:35:50 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id ED525A0D42 for ; Tue, 31 Dec 2024 04:35:49 +0000 (UTC) X-FDA: 82953988866.01.AAD56FC Received: from mail-pj1-f73.google.com (mail-pj1-f73.google.com [209.85.216.73]) by imf30.hostedemail.com (Postfix) with ESMTP id 2B51980002 for ; Tue, 31 Dec 2024 04:34:16 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="y6v/Ota/"; spf=pass (imf30.hostedemail.com: domain of 3o3RzZwYKCJwUQVD6KCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--yuzhao.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3o3RzZwYKCJwUQVD6KCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1735619726; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Xw2CHOQueA9zkMREwjqXXK1goFWTBSTiadQe8+3ME3M=; b=PHsOCKAO/pQ+2Rjz7tWh7vR+BWROsU1olViFv+ZLZVzBLLaYndFkBZm0XZROebw9SLx2PM RBhkncLXykOgzlXy6TEOkIzNBmSv0e+Or0Znf/VT6cvb6JSy7AtDxuYksdeAI8jqiOxPrf 6BHMpV63fUa0y3DBjZ7UHyc14mnKcew= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1735619726; a=rsa-sha256; cv=none; b=k4ujzX9eXTJBrINcsy5LmcCW/6LTaluGY+63jAGck0UkhmEEkMZsg2/jqG5RoNrM6tRJL5 G4QyGr4Jufha8ARnt+/72urzwyTCLcnaP0PzMOJXp7AgEFOwilH3IDoBoGAc9h/Kb3NkFc u8ebaS0HQODgu+2S+cAlr1omdA57t1w= ARC-Authentication-Results: i=1; imf30.hostedemail.com; dkim=pass header.d=google.com header.s=20230601 header.b="y6v/Ota/"; spf=pass (imf30.hostedemail.com: domain of 3o3RzZwYKCJwUQVD6KCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--yuzhao.bounces.google.com designates 209.85.216.73 as permitted sender) smtp.mailfrom=3o3RzZwYKCJwUQVD6KCKKCHA.8KIHEJQT-IIGR68G.KNC@flex--yuzhao.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com Received: by mail-pj1-f73.google.com with SMTP id 98e67ed59e1d1-2ee5616e986so20974421a91.2 for ; Mon, 30 Dec 2024 20:35:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1735619747; x=1736224547; darn=kvack.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=Xw2CHOQueA9zkMREwjqXXK1goFWTBSTiadQe8+3ME3M=; b=y6v/Ota/PSi0B1865z63EzjBBsMl3mwvADqULr4syp//+aMW6iU6F+GVeLPBaNUCXp 9yyWvJHekOXIymUBxe4WVzfWuuOOIv1h0cAlTQ5hIg4jbNXsI48ntYXu87VmoiPX7VRX XidcEiAvtTMuliMDLp+DF/bFrzUVSZNNNJQ3RCLQgWQ4XzsXhaB018Djco0xL8Tlox00 fVzBIahGyollG4y6y9ASwtrhEeaUx1opppJfjQbePDBIKRaNyBfjMBZ+N6gkILnCedrX YK7ukFQf3+wq1afVoeVJG0SJ4Pj0wMqn4cUoDYWQxssBDFRjECyn6rISymI9Jm6EWZ08 u8kg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1735619747; x=1736224547; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Xw2CHOQueA9zkMREwjqXXK1goFWTBSTiadQe8+3ME3M=; b=awJIzuJSIm02YdEgZbixUseuu/MRykTMRztEwnSZ4v1ebgm4G8IDYwCJ8i5abcgZkT yug5R2OrgJALtQFxxru/iVPqZO2jXWlOM+XKKEd/ks6eIog/YtvTZRCyvvInYFJInKOU YMfyZnPtQ2uGEfnihj6x6McRD/kJW0mylH1EUhZUk2198srDFwzA4B7KP+PVzqhiqg0I 0HZ2tiz18KO04mKtodFGE2tsJAjdOS8fIDNuCoWmIg1onZR5JeKWwz/b0UJ1OaViAqeZ 2WCkPF9lijiTNMTPgeeppSSxM7mlCxzOYWqF8W1hZ25nY5Epub1GQhgSHl4zAznE4gmX Nk1w== X-Gm-Message-State: AOJu0YyApvyfQ7N0SIkvAfBFTsGS6t9cKuLvE1TWK114A/9gZMQgbYro IwVIVKbABgrrxxM3dvZTiWLlDfNclgEfxo81dM+i9G2mmm6CPR3hJDyMHG3L1zXkKF0WZieGnh0 NHw== X-Google-Smtp-Source: AGHT+IFqFvRA79cO3VpI5IoQ9FD/iueUtKICiJreOXklTdeZ0vkOif4CjJ0SLo91oWp/vZ2d3gnF2M2cOO0= X-Received: from pjbpw17.prod.google.com ([2002:a17:90b:2791:b0:2ef:94c6:5048]) (user=yuzhao job=prod-delivery.src-stubby-dispatcher) by 2002:a17:90b:524b:b0:2ee:9d49:3ae6 with SMTP id 98e67ed59e1d1-2f452e1d12fmr56370216a91.10.1735619747003; Mon, 30 Dec 2024 20:35:47 -0800 (PST) Date: Mon, 30 Dec 2024 21:35:32 -0700 In-Reply-To: <20241231043538.4075764-1-yuzhao@google.com> Mime-Version: 1.0 References: <20241231043538.4075764-1-yuzhao@google.com> X-Mailer: git-send-email 2.47.1.613.gc27f4b7a9f-goog Message-ID: <20241231043538.4075764-2-yuzhao@google.com> Subject: [PATCH mm-unstable v4 1/7] mm/mglru: clean up workingset From: Yu Zhao To: Andrew Morton Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Yu Zhao , Kalesh Singh X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 2B51980002 X-Stat-Signature: 99wh5nosomhc3pu1pznosa5rradaiuij X-Rspam-User: X-HE-Tag: 1735619656-202659 X-HE-Meta: U2FsdGVkX18e4W1tBWdjH7X1Fvuzr+De1sohTFDpTW4xoRbczP4hSSDSVYTQ0u4NT1isbv2XgP/KTDG5DT6RRbJyBqlxjvmZXepeSaHF458oPluMSUnYQR0yx37DNYUkC/amg3OClRD8RTOeNoDMCtyZaYqj4UIFPYBEKzJeL446oCl9ENVbkYfP4Xcz+w4R/H1BIWeNQyhDaN8wjwdZWMd/1oStFZwrWkIk1FG3swTeEf6rlFNW68xWHIgL4GeT/sdsaHWdpWEIhfOE4DW6PAhVgBmQ9lNL+XeHlFb38ps4q/Q6vJfWvagCIHdY81gxymZAlMbuBvFzR7SexyJ4jBvzlTsAgS5kMsgpMK2CYIxz26CPk7wv0BbI8HFWgAZ+MZKg44sud1IoWRspRu7mCU9AkSxj6beWjlbjqWjL2wXOjNUL2mHzSvWOGoKWD/3TR2FFiCvIgXsR7xkRFY0C6we3imTytSUiT5BhOm5ULmWEsJ79+MLxFe67fCPqBIUeMbESa3IVmSnDKy7OrU+UidzuvZPmajthNqcOuO3iDK5dN6eZWXd/AqNM0N/qnZwRHIySoQETWjAwoJ/rh5kGz8WmB2nbZtyRQCTjU5Mn1UUMxosvx9ocLHhyIok+vMacFXufyr7ka5bOgV7rIEn4MBdm3lJO1YTFuw9vHY8UG5hm+HhlcrS3whHfQYFNJoYgBR8YI3Bng+DbnAqB9SnRI3fYX6MlyKRxzPHcwzgfz+3DG81tW1+RaA73JYiJMKycXKQ+nyTSLHNQKInOYl+tVc3f5lX31TNsippSi6pq6HQ0zbn3EZ8MJZc2l01ir9PTKxpQobhb3ZxwZ6KT17iEh+WDk3mm6F1XTcTfneEWyoGg9k6Ri4JuU6EvSJhCOMjnEyhS3PLhPifFm4pGdcM/vxliprQzNWzL1H0P86U0r2m9f+2isO5LwXsgLAh2FM6duSAYlgK2Bh9kRgvJScS w4cYmqnG 172MC85EpTPAENtSy0bzMzN5Pv4RzYtN/DJa+eHyHMAW/W/JgPYS8akCFNrnCY5MfY946CKDWr4jMJWTp5flQ1dZF65KvpkqXoAvnyuxLPhw+OjUe6jNwwr3aGKi/+6b4GQlGCEDuiUCp2zaI6OIEYY6DiWWRn0bJ51ase5s5qHAiddTk9Nego73RD9pAm0D74jfTyMc3cZBTKFfsk8JNCzCmXKUyncSq9x413Ntm5xGzkWU0oHZeWcq+gTFU8fxqNTwju1uyC82qtSTitlB1JO/A2nHKR3beOsod95e1zOhtaWKIZHSuTGa3Xnwra1AbKm6HQtalLSBQPcIz+jTaMoopRNfDVVfELS+ROvgTLoZRuZ1WC8wtXT6NSCYMCNXWPmWx0bZ6IR/qaIVeZq8jeAAJN37QeX0AhGcVdYGhlZF++Y8d5L7fIBMp6N+3T3WtLi4px5oSXM7BxjgYkls9l+dqkWBX6OotmzidJa3597BkPSq0o5bIRraFBzKvipMN+v2gTQj0cbHEcnD23d2IfbhUBxqNQJ14JNZi X-Bogosity: Ham, tests=bogofilter, spamicity=0.000769, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Move VM_BUG_ON_FOLIO() to cover both the default and MGLRU paths. Also use a pair of rcu_read_lock() and rcu_read_unlock() within each path, to improve readability. This change should not have any side effects. Signed-off-by: Yu Zhao Tested-by: Kalesh Singh --- mm/workingset.c | 25 ++++++++++++------------- 1 file changed, 12 insertions(+), 13 deletions(-) diff --git a/mm/workingset.c b/mm/workingset.c index a4705e196545..ad181d1b8cf1 100644 --- a/mm/workingset.c +++ b/mm/workingset.c @@ -428,17 +428,17 @@ bool workingset_test_recent(void *shadow, bool file, bool *workingset, struct pglist_data *pgdat; unsigned long eviction; - rcu_read_lock(); - if (lru_gen_enabled()) { - bool recent = lru_gen_test_recent(shadow, file, - &eviction_lruvec, &eviction, workingset); + bool recent; + rcu_read_lock(); + recent = lru_gen_test_recent(shadow, file, &eviction_lruvec, + &eviction, workingset); rcu_read_unlock(); return recent; } - + rcu_read_lock(); unpack_shadow(shadow, &memcgid, &pgdat, &eviction, workingset); eviction <<= bucket_order; @@ -459,14 +459,12 @@ bool workingset_test_recent(void *shadow, bool file, bool *workingset, * configurations instead. */ eviction_memcg = mem_cgroup_from_id(memcgid); - if (!mem_cgroup_disabled() && - (!eviction_memcg || !mem_cgroup_tryget(eviction_memcg))) { - rcu_read_unlock(); + if (!mem_cgroup_tryget(eviction_memcg)) + eviction_memcg = NULL; + rcu_read_unlock(); + + if (!mem_cgroup_disabled() && !eviction_memcg) return false; - } - - rcu_read_unlock(); - /* * Flush stats (and potentially sleep) outside the RCU read section. * @@ -544,6 +542,8 @@ void workingset_refault(struct folio *folio, void *shadow) bool workingset; long nr; + VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); + if (lru_gen_enabled()) { lru_gen_refault(folio, shadow); return; @@ -558,7 +558,6 @@ void workingset_refault(struct folio *folio, void *shadow) * is actually experiencing the refault event. Make sure the folio is * locked to guarantee folio_memcg() stability throughout. */ - VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); nr = folio_nr_pages(folio); memcg = folio_memcg(folio); pgdat = folio_pgdat(folio);