From patchwork Mon Mar 11 19:18:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Matthew Wilcox (Oracle)" X-Patchwork-Id: 13589253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 314F5C5475B for ; Mon, 11 Mar 2024 19:18:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BA6D96B00ED; Mon, 11 Mar 2024 15:18:42 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B570D6B00EE; Mon, 11 Mar 2024 15:18:42 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A1ECE6B00F0; Mon, 11 Mar 2024 15:18:42 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 8E8DC6B00ED for ; Mon, 11 Mar 2024 15:18:42 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 5CAEF1402D9 for ; Mon, 11 Mar 2024 19:18:42 +0000 (UTC) X-FDA: 81885720084.03.0AB259D Received: from casper.infradead.org (casper.infradead.org [90.155.50.34]) by imf13.hostedemail.com (Postfix) with ESMTP id 39BC320008 for ; Mon, 11 Mar 2024 19:18:39 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Od3uRhcx; dmarc=none; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1710184720; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=okrG4mFdWo6R2QSOPxZydVCc9X5+Hk+mNQoCAVCaWHA=; b=zPLsZkEdnslQD6C7P+YdyGsxDa234MUCKh1oXPaQAZAJBXNifdzheVRRtmJFtH8HGq7HvK pFQ9wE9uXH2Dk2zYu1oNArVANmVzzbDkhKmZFwsZ5SGcteU0UlJPnHqze4OAqO8XLs+BD9 H0oe3XcJn6fOLVLmaMMqxUO9xV87Bq8= ARC-Authentication-Results: i=1; imf13.hostedemail.com; dkim=pass header.d=infradead.org header.s=casper.20170209 header.b=Od3uRhcx; dmarc=none; spf=none (imf13.hostedemail.com: domain of willy@infradead.org has no SPF policy when checking 90.155.50.34) smtp.mailfrom=willy@infradead.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1710184720; a=rsa-sha256; cv=none; b=UiXyi+C7Xt+64GYcJ5HLjcG/mNC3++uoHzcMiZ6AKOXWxvxN+iN4sbuu1oly2b/+MhJ1yz 6ir9V7YWDXVNh+LxZZ2kT0hQmR/zp3ue9pkT4YbzvBNZ6+7LvoVhsgn6q3mc7+ywPp7uRv kps72+dhpabnwB3Zt6gcoWuK4TBmdZk= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=casper.20170209; h=Content-Transfer-Encoding:MIME-Version: Message-ID:Date:Subject:Cc:To:From:Sender:Reply-To:Content-Type:Content-ID: Content-Description:In-Reply-To:References; bh=okrG4mFdWo6R2QSOPxZydVCc9X5+Hk+mNQoCAVCaWHA=; b=Od3uRhcxyayBsRbGR7HeKoJaI8 iGr6dZa9Ic1WxBHkajcfVGNfIQWjkSA28nI1nXoVwanXscSXI5bkHfof2gnULpYPyIfkI8lJ3jANK rRQjndiLT/3sWhYeDaeeaGk9bapldF0lT9FiQEZ5XGtXnXZmWx4miZAxoMDQTX099vyC5gIIBUkGp eXfZ9WW589iV5Eoc+cW8PvQmFWRK2amsca/u1gi4TtXBvn+Os41JJGBOtXFVbvjswygcOxRd2vqcU D9P3QHLPIWgSY73Hj0PtjpVkjg3hc5hGcjQd9Dsap5+fXV7ulzMQFuAaeyP/ux1bsaL4GwWLdhhIU TZ8vONLg==; Received: from willy by casper.infradead.org with local (Exim 4.97.1 #2 (Red Hat Linux)) id 1rjlAT-00000001JD3-0IAa; Mon, 11 Mar 2024 19:18:37 +0000 From: "Matthew Wilcox (Oracle)" To: Andrew Morton Cc: "Matthew Wilcox (Oracle)" , linux-mm@kvack.org, Ryan Roberts Subject: [PATCH v2] mm: Remove folio from deferred split list before uncharging it Date: Mon, 11 Mar 2024 19:18:34 +0000 Message-ID: <20240311191835.312162-1-willy@infradead.org> X-Mailer: git-send-email 2.44.0 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 39BC320008 X-Stat-Signature: knxjm8969mb3o1tpe9fwptjm1ttski48 X-HE-Tag: 1710184719-284672 X-HE-Meta: U2FsdGVkX1/YKTyN+y5NkqS6acvXWTAyE95v1A/FGTTBLVMtMnVpgtZGklPNAmBWYAzaNXIQSrhH0VV904Obz0JdhKhDPN9gmpugzp3N2E7A1RT34GJXIiK1NcB4MHdwzRXvempl2AnrrArjzw0+JEO81yIoO2alBA2Lx+M33xpab3TsGkSkLhWg+6Rs20958YQ82fSuORie/iaC3Z5odJw6Ws261i9HkjrSuYsICVr7nkO9K3D2sJKcm+oJNHKQal8wU6Yl8jDtIKDjv5BJfkyqt451HWLE/IdgqdhktEqwbPU+Ek+EFN82H1bZyBueLCPu2B8MY2SzMORm9VU/OXrK11AQs222oGJiWP4fVWZPNUIEJQRCNz5OSUYmHRamc3LEjikgWk+ZddE7YGRPFucTJTRBxahHOSSvsIH0nxlW23XLV71AK3YTjScDoGQ184IjTEt2hmMXLKmDoPaERYqRheXQ5k9cM2x3zDqvqTLXZNWjQAkN9oN1TR1esVpzqvPBhrbHwgappD+LL/DLTwFJRD1MMhwGpZYYoI8r9BN3ezGUVJ5KkASy8pZFXjrnE3kfmlO5FTfcLsYMxGTvlUN0wxnj01A41k619yQZ4olZ5s8ALTzWsX2al3T10cur/UF8YjrBGS9OOqE3YeWQUa3i+1TgQ20M94HjLYykh8kaiGqwPjAEdQN6KZWleR1av4E4cP/grkuQzK5yrr/cZ4qwPUVXkhgE3FQstNgRoejusNZPO5OSZj8LhVXWNG5K7wEOUsVAdyHvJzFzMmQjtqI6S59gR3V8DGh8WvjAue9/xujExOHTUCO2cD/aaqspoViJBLOw25VgeaH/2LTTTH5iPv0avg6Jq/ighNzn2k5wYcWo0RZwI0yW6/02jpUxlGXBVcRlS/wX8jlvlnNPDLPKYubSA410jJv8z4vgRytfPT9z0wLVCNgxrORN1ldNqYT9YwWiW8jckxmMlWh LrU3wqEQ svVswFEES+q6mb7d3p6C21tIpcTOkS+9N93EvWSd4ZOV3oqlfkSXlre9UGfxW1LGC29LSPifZtJOgBqYNhqBkgcVAJDivgAk+uLgOLfcLFBfjAbF1Hu1DlkswaHvH6Muh7x61mMfQDKETpqZ2wMiV1Xz6rJc/CuCQgQWlfGuYQoA4SeFpqoAj9m5whe+P9yBRHZnHP3ccsBEiAlmR0cPnv0YI/1j0lgDOzV+Dv21JyuT0O+pQfJzIedYlmHc85PcrWzfDC5sb9aYYqq26QEIFMqCDPu+cqjr573RWTQCDAvndT50= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: When freeing a large folio, we must remove it from the deferred split list before we uncharge it as each memcg has its own deferred split list (with associated lock) and removing a folio from the deferred split list while holding the wrong lock will corrupt that list and cause various related problems. Link: https://lore.kernel.org/linux-mm/367a14f7-340e-4b29-90ae-bc3fcefdd5f4@arm.com/ Fixes: f77171d241e3 (mm: allow non-hugetlb large folios to be batch processed) Fixes: 29f3843026cf (mm: free folios directly in move_folios_to_lru()) Fixes: bc2ff4cbc329 (mm: free folios in a batch in shrink_folio_list()) Debugged-by: Ryan Roberts Signed-off-by: Matthew Wilcox (Oracle) Tested-by: Ryan Roberts Reviewed-by: David Hildenbrand --- v2: Add two more places Ryan spotted mm/swap.c | 3 +++ mm/vmscan.c | 6 ++++++ 2 files changed, 9 insertions(+) diff --git a/mm/swap.c b/mm/swap.c index eaadbacf7f19..500a09a48dfd 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -1012,6 +1012,9 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs) free_huge_folio(folio); continue; } + if (folio_test_large(folio) && + folio_test_large_rmappable(folio)) + folio_undo_large_rmappable(folio); __page_cache_release(folio, &lruvec, &flags); diff --git a/mm/vmscan.c b/mm/vmscan.c index a0e53999a865..766da7de9e27 100644 --- a/mm/vmscan.c +++ b/mm/vmscan.c @@ -1436,6 +1436,9 @@ static unsigned int shrink_folio_list(struct list_head *folio_list, */ nr_reclaimed += nr_pages; + if (folio_test_large(folio) && + folio_test_large_rmappable(folio)) + folio_undo_large_rmappable(folio); if (folio_batch_add(&free_folios, folio) == 0) { mem_cgroup_uncharge_folios(&free_folios); try_to_unmap_flush(); @@ -1842,6 +1845,9 @@ static unsigned int move_folios_to_lru(struct lruvec *lruvec, if (unlikely(folio_put_testzero(folio))) { __folio_clear_lru_flags(folio); + if (folio_test_large(folio) && + folio_test_large_rmappable(folio)) + folio_undo_large_rmappable(folio); if (folio_batch_add(&free_folios, folio) == 0) { spin_unlock_irq(&lruvec->lru_lock); mem_cgroup_uncharge_folios(&free_folios);