From patchwork Sun Dec 22 12:29:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kairui Song X-Patchwork-Id: 13918037 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5BF4EE77188 for ; Sun, 22 Dec 2024 12:29:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5C9E96B007B; Sun, 22 Dec 2024 07:29:51 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 57A076B0082; Sun, 22 Dec 2024 07:29:51 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 441136B0083; Sun, 22 Dec 2024 07:29:51 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 26CF76B007B for ; Sun, 22 Dec 2024 07:29:51 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 9D51845B61 for ; Sun, 22 Dec 2024 12:29:50 +0000 (UTC) X-FDA: 82922524860.19.2C23083 Received: from mail-pj1-f44.google.com (mail-pj1-f44.google.com [209.85.216.44]) by imf07.hostedemail.com (Postfix) with ESMTP id 1E0F140008 for ; Sun, 22 Dec 2024 12:28:49 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=m2ct3Aq4; spf=pass (imf07.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734870563; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=2QTX0YqqR9AM7fsf9GyW4l3QeD7ywqg3BQ+Z/xOgdSw=; b=5YJ6JdGc5uezWiibCZMGE+ShrNDnoXy+tp5AAP9Un8MXL5+DjYVvhX7EJG29as/uvurQfh gW9KydCro/rtMbT4Yv/U8thb1y2fktz3uqA63XBOMVHHpeDyXmwh6TJD/Mdmjs9yx+gcQk +F+xp32X5GK1PFgSsPO0eUI8cZIWR1g= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=gmail.com header.s=20230601 header.b=m2ct3Aq4; spf=pass (imf07.hostedemail.com: domain of ryncsn@gmail.com designates 209.85.216.44 as permitted sender) smtp.mailfrom=ryncsn@gmail.com; dmarc=pass (policy=none) header.from=gmail.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1734870563; a=rsa-sha256; cv=none; b=s0pt1scyohvG3ehUaSE39tGgcuXXXVGfyiYGu4pF8B/OnfqsPwEhkb3F1oOyIlhHrXIEpp 9X5CvQWL6bgAw6vZfbbAQRRLF5EeWjzShzetaYb7u5MLOX70GOn66ptsJHnBSDlK3/BvOr 2x1zNbyNd3st0Ez8ke/kQnPM656478g= Received: by mail-pj1-f44.google.com with SMTP id 98e67ed59e1d1-2ef87d24c2dso2665739a91.1 for ; Sun, 22 Dec 2024 04:29:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1734870586; x=1735475386; darn=kvack.org; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:from:to:cc:subject:date:message-id:reply-to; bh=2QTX0YqqR9AM7fsf9GyW4l3QeD7ywqg3BQ+Z/xOgdSw=; b=m2ct3Aq4aPqVpLF5vb15U2t9cLWdJ7UI6bHWWBCMhbar3gVrd82pP2G41P95vsC9tU GnrCyOPSdRGvyh3lcuECqtFkytaCdrFWwKMQHI0ogeAWBJ6eyEgClTNCsIgLQnsgFR48 HaWZ0+YKoH0Z+lTbXlSwexTbUGAMY22YHZfsTgc/NJRRRuunM55WP4P0zzmTHVlB7vRT vW41kAT1gr4nae+fiZexR5JvmlFKktu35eQAmg32eVBSBoMVtJlk4koht11dRAsZmFl/ FLBIG6E0KWAjlu0uQ7VlufTpTb2r2gP8WS9S3BHFkogSuNeWcnFPzQo7TSEpK9n8xGNS OK0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1734870586; x=1735475386; h=content-transfer-encoding:mime-version:reply-to:message-id:date :subject:cc:to:from:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=2QTX0YqqR9AM7fsf9GyW4l3QeD7ywqg3BQ+Z/xOgdSw=; b=Avv+UliB/JNxe4figAsFVGHfazPaw1m3rKsI2WMJJ/OGL2OIsuKcBLT+FC2E2iWhkM A7KhaiA6ABrsNXN9K7wGLkTJHSRfjUwIUTf1pIqZ5KciM6J7j51v2JSbVXtCHdB5khMR 6wEVdZWlfZ9wJt3V4EvAPJ7hssD4bUbIF42d3asrnPjeIF4pEhfszamaKDmKosnGZom2 UCakR5wVmII04JPNRM0U/m+vQlouLg4Isebwiy/uWJEpfp7AD1skipTqjDDaWxrjfNHK /2uV86F0iAMqELvP/SUxDLQe51wv+q9weQpyB54/jntIT085wbITchudeG6l4zEy0j0D JCog== X-Gm-Message-State: AOJu0Yyqh271C9Xk1Klunxw6neDcVgnKk9EJ5X/I7GcTf4d7Yxmm9VlM Vj5BnLEyDpknXXEfpNZ0eNFZ1FTplqzCrMe6uvNkZIeewNVm4BHbzJBKERdS X-Gm-Gg: ASbGnctckJkhIMXLi1c6s8/mRu0W3GgB9M3g5BdtA6oSGR4/qERkMayG9vo7+t0Z6X3 00jfYWcb3hO+joCnlDevsipqxroZJ5PLbVmI77i32y9Sh+BuXo9vuqc1zNgRWmBbrwNOJK777cW ViDpXIxOH7ACESLhkcsKKRIeXjlAMSxMVOlUze5k8Z1IUGIMEyXlS+Va7fSNauejGJBQvOZyoYm SPMEuu0+QFTWl5EaFJaE5uguJkO99MiCeszmaBgRwXAxS0PndFxN5Qz+s3umzC4b6GnRSmf6m+T FQ== X-Google-Smtp-Source: AGHT+IGqN1Dkb8gMZ61ekH4lQeiM1STei9P4DnhltR62KGD5Az+TRB3g2gvaTJnTZXRGZQk1m8+N6w== X-Received: by 2002:a17:90a:d00f:b0:2ee:c9b6:4c42 with SMTP id 98e67ed59e1d1-2f452e214b8mr14596073a91.16.1734870586454; Sun, 22 Dec 2024 04:29:46 -0800 (PST) Received: from KASONG-MC4.tencent.com ([101.32.222.185]) by smtp.gmail.com with ESMTPSA id 98e67ed59e1d1-2f447798801sm6552537a91.8.2024.12.22.04.29.42 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Sun, 22 Dec 2024 04:29:45 -0800 (PST) From: Kairui Song To: linux-mm@kvack.org Cc: Andrew Morton , Muchun Song , Matthew Wilcox , Johannes Weiner , Roman Gushchin , Shakeel Butt , Michal Hocko , Chengming Zhou , Qi Zheng , Yu Zhao , Sasha Levin , linux-kernel@vger.kernel.org, Kairui Song , syzbot+38a0cbd267eff2d286ff@syzkaller.appspotmail.com Subject: [PATCH] mm, madvise: fix potential workingset node list_lru leaks Date: Sun, 22 Dec 2024 20:29:36 +0800 Message-ID: <20241222122936.67501-1-ryncsn@gmail.com> X-Mailer: git-send-email 2.47.1 Reply-To: Kairui Song MIME-Version: 1.0 X-Rspamd-Server: rspam05 X-Stat-Signature: gy5o4kq8mouqkpoisaujetdssan5wosf X-Rspamd-Queue-Id: 1E0F140008 X-Rspam-User: X-HE-Tag: 1734870529-476828 X-HE-Meta: U2FsdGVkX1/aN5i5yuguoBG9c9BJylABCi2e457ShaDbtwmo9o58W8fJiiklMXOplFsDZ86XA7DHk7SMfu1XTVXL16eyob/4IFmK/1MNzUrzDf4dqdV277Am8OhvZ3ODCU4YqdHrFo5LkGOK1kIF13WrvBMGbAGrJ4hEWsIneWZ6ra9vW9tpZSD09diDQzwaZBwUQ8k5A8I7y7iluC1z7NsUqhRV2jnKUX3dycNjQwCLv1ZCunbJ5qorNSmEHJZ/bUT+rQrpxzea98rs2FqBzXDCzpP4Oeb3wxNbnIuQuvWIH2t3Gvq2LKXw9Uq0IYD2BMYmm7OOFi8g78ONV/chCW5iuhRCpxQFy8kwBwzMDKA2RAgUnB+nwOViuBqnNdU6sanRSaQygJD73xnJGm416LTBQfynx1rAoLNIl/IVvUAII1DTkt1FPAHsssSHQC9vBO3b3Y7mxaXq2N7P2jyyiW2xjifWU/Mpex9WfdZ2NKmpYU4TmYbFi2ZJjbuoKtohiKtej4EYtyNYyyw19gOjiV5LsZzgou0aNX+Wx0rTW9UvLY/chEyRmp6MAzQ6H5aS3SUB2Iy9cwdDyzFnNL1zC6kyKQ1kBHDkthD28jr6IrUcwn8pLgLt4v8G1NYik0ucJxf6AKy41N1FFcqlKLrpr6L8mVoe6TIayB875/Zg5SZgjVaKEqEWfB23nk0hxD06O9rvbIIIK82DxkPLVtNiUMv5pgXBuGB5RO0ZxuCRw/chvlReGZu1F/VCrEXrr2t7KSEFda1rQAS0hTREeci/AxPZhMMwVfhf7c8akOhmWwkWcXxol3e1PWUF85NlEFZc4ytyoqOJizU1FB4lpXGatmkq0ztuU45JO0HLWkBCO5lW8nEG0dV/WYffXLMd/Zx/q1EM8UjT9B1MYaKxrBy512gHcOBOcI2ZeM2B6f9Rap7nmx+YWIyy06AocBiI6hK7EGMJN0o73PMx+S62hGz YAXgSNsR 12ZcN33uuLZwfYylA0eBhSie2nT7N53VjrD6GzfHD7XDidxWSjqSDHeNCnmxwjNtWxa/O1Q+RU6IdC0D3FdcXFf1aA5Vtm0AFfjQWj2sX5kHR87fjLV/pC9Q5/byUmnSBo1hA3eaLMRjo3Hm6Rw1Ddrq04AGZZSlrpvSlJk+qNDrG3vM8/I5xzIsFLWOREeu6EWlkxuZnQoJOYvCFHE55iv3qU7IiFp/hP4vRhjyYvmQFYtoH0pMEYMDbY5ktW6vBsxoHZ4EgY9k0SsVMDnY2V6bNVIuuXKQjxtdUWVqay0v2xSm/F8qYSI9K3nPHJp8U9wuRWquy50QQj9kfArnH3ks+qquRoZmYEaaK9PIK2kksNUjdae+vuWDgF/hshPFMQaKRCzYvtUGNViEJ25XGtofGVEw0ty1kvbK2/fpgyhb5qyBk33bhND3RUFfQ5oFlfGwaVV7ahIMSywRNx3gUSulyn1uu0cNJBFYP68SxvPltzgji4afIrorK7u7C9ueLW4mNbcYlfXUl3ugQMXBJk3KNsKRCvKUqySwVrHrtW202Siz5ZMxpZCF5tctvIM+tKGdcDg5WY8xXjcjy79OxEvsQGxPWRD+wcOpG8BJN7MsIHiDtZYwGfEfkZLHDNpO0pvaOyZ8Jqyh7CK/xYkQ5Jxy1skmytNJuuAfI/xlUEijawSOhGn4+kiPEopom01xXVxZUIMlmFeQQzTKVsXyVC/BbTrDHsGThNAUEpM3Srban3TYmwMKmM0o5co0WE3Gpakp+M8tOH0O4ryoKNpkPBYlhX+FY2lQ1vziEajY2CCclAfdBEIrjXU8mSaiO3wwLvs2QrkQqN8gBnbxy5oa+x09wLw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Kairui Song Since commit 5abc1e37afa0 ("mm: list_lru: allocate list_lru_one only when needed"), all list_lru users need to allocate the items using the new infrastructure that provides list_lru info for slab allocation, ensuring that the corresponding memcg list_lru is allocated before use. For workingset shadow nodes (which are xa_node), users are converted to use the new infrastructure by commit 9bbdc0f32409 ("xarray: use kmem_cache_alloc_lru to allocate xa_node"). The xas->xa_lru will be set correctly for filemap users. However, there is a missing case: xa_node allocations caused by madvise(..., MADV_COLLAPSE). madvise(..., MADV_COLLAPSE) will also read in the absent parts of file map, and there will be xa_nodes allocated for the caller's memcg (assuming it's not rootcg). However, these allocations won't trigger memcg list_lru allocation because the proper xas info was not set. If nothing else has allocated other xa_nodes for that memcg to trigger list_lru creation, and memory pressure starts to evict file pages, workingset_update_node will try to add these xa_nodes to their corresponding memcg list_lru, and it does not exist (NULL). So they will be added to rootcg's list_lru instead. This shouldn't be a significant issue in practice, but it is indeed unexpected behavior, and these xa_nodes will not be reclaimed effectively. And may lead to incorrect counting of the list_lru->nr_items counter. This problem wasn't exposed until recent commit 28e98022b31ef ("mm/list_lru: simplify reparenting and initial allocation") added a sanity check: only dying memcg could have a NULL list_lru when list_lru_{add,del} is called. This problem triggered this WARNING. So make madvise(..., MADV_COLLAPSE) also call xas_set_lru() to pass the list_lru which we may want to insert xa_node into later. And move mapping_set_update to mm/internal.h, and turn into a macro to avoid including extra headers in mm/internal.h. Fixes: 9bbdc0f32409 ("xarray: use kmem_cache_alloc_lru to allocate xa_node") Reported-by: syzbot+38a0cbd267eff2d286ff@syzkaller.appspotmail.com Closes: https://lore.kernel.org/lkml/675d01e9.050a0220.37aaf.00be.GAE@google.com/ Signed-off-by: Kairui Song --- mm/filemap.c | 9 --------- mm/internal.h | 6 ++++++ mm/khugepaged.c | 3 +++ 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index f61cf51c2238..33b60d448fca 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -124,15 +124,6 @@ * ->private_lock (zap_pte_range->block_dirty_folio) */ -static void mapping_set_update(struct xa_state *xas, - struct address_space *mapping) -{ - if (dax_mapping(mapping) || shmem_mapping(mapping)) - return; - xas_set_update(xas, workingset_update_node); - xas_set_lru(xas, &shadow_nodes); -} - static void page_cache_delete(struct address_space *mapping, struct folio *folio, void *shadow) { diff --git a/mm/internal.h b/mm/internal.h index cb8d8e8e3ffa..4e7a3a93d0a2 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1510,6 +1510,12 @@ static inline void shrinker_debugfs_remove(struct dentry *debugfs_entry, /* Only track the nodes of mappings with shadow entries */ void workingset_update_node(struct xa_node *node); extern struct list_lru shadow_nodes; +#define mapping_set_update(xas, mapping) do { \ + if (!dax_mapping(mapping) && !shmem_mapping(mapping)) { \ + xas_set_update(xas, workingset_update_node); \ + xas_set_lru(xas, &shadow_nodes); \ + } \ +} while (0) /* mremap.c */ unsigned long move_page_tables(struct vm_area_struct *vma, diff --git a/mm/khugepaged.c b/mm/khugepaged.c index 6f8d46d107b4..653dbb1ff05c 100644 --- a/mm/khugepaged.c +++ b/mm/khugepaged.c @@ -19,6 +19,7 @@ #include #include #include +#include #include #include @@ -1837,6 +1838,8 @@ static int collapse_file(struct mm_struct *mm, unsigned long addr, if (result != SCAN_SUCCEED) goto out; + mapping_set_update(&xas, mapping); + __folio_set_locked(new_folio); if (is_shmem) __folio_set_swapbacked(new_folio);