From patchwork Mon Feb 26 20:55:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13572834 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7B738C54E49 for ; Mon, 26 Feb 2024 20:56:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D71B74401BD; Mon, 26 Feb 2024 15:56:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D219444017F; Mon, 26 Feb 2024 15:56:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B71904401BD; Mon, 26 Feb 2024 15:56:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 9E6AA44017F for ; Mon, 26 Feb 2024 15:56:13 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 58EE01409FA for ; Mon, 26 Feb 2024 20:56:13 +0000 (UTC) X-FDA: 81835162626.19.77FA258 Received: from out5-smtp.messagingengine.com (out5-smtp.messagingengine.com [66.111.4.29]) by imf28.hostedemail.com (Postfix) with ESMTP id 78E96C000E for ; Mon, 26 Feb 2024 20:56:11 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=sent.com header.s=fm1 header.b=T5xeqRqY; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=bp4JhDO6; spf=pass (imf28.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.29 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1708980971; h=from:from:sender:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ODE0ROICHsjD0yZP7iLxwoRZvmdJZzbZs1wZ6FX7X9k=; b=RpfN6LDV31cOZcImJNLtUj2PN8SSidd+HmbI+12M8fmqwmOSCBTrXo9RVFARGRG8aRwLTl F0pf7q6SqS+XglA7iufcX/UsjrbPJ2pz/shHe/zZKdsEoaSXeoAcjdfCRo1oeuuE6Bvfwk A0xgipzJ2yoGR6aYwrDDTQXZqsX3R18= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1708980971; a=rsa-sha256; cv=none; b=Gcs9qfIoz8IEyPjGwmvdqvpmWbU2H46us7rKRaQXck7Eymq6TqxvcmKfHZNI6mDPeTDAlE 2rMfDdnNH/0fB3f8aL4gkJ5tYEsMqqZ+I7em3cusrBOD4JhJGG44M0KA1oSNHEqac3knG4 4YQJl4pBRiM6DorVrw9+3fU+8rUs0p0= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=sent.com header.s=fm1 header.b=T5xeqRqY; dkim=pass header.d=messagingengine.com header.s=fm1 header.b=bp4JhDO6; spf=pass (imf28.hostedemail.com: domain of zi.yan@sent.com designates 66.111.4.29 as permitted sender) smtp.mailfrom=zi.yan@sent.com; dmarc=pass (policy=none) header.from=sent.com Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id CE54C5C00A4; Mon, 26 Feb 2024 15:56:10 -0500 (EST) Received: from mailfrontend1 ([10.202.2.162]) by compute5.internal (MEProxy); Mon, 26 Feb 2024 15:56:10 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sent.com; h=cc :cc:content-transfer-encoding:content-type:content-type:date :date:from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:subject:subject:to:to; s=fm1; t= 1708980970; x=1709067370; bh=ODE0ROICHsjD0yZP7iLxwoRZvmdJZzbZs1w Z6FX7X9k=; b=T5xeqRqYOCyBPMLYOa3Br+G8KABMxRYqzZHOqItB1AZrzBWc5++ v+Y7er2Z+fEtOY1M/UnNBzfbkuRk6ewQ6+GDS0M+g8TVhKmRbqYnUvTSSYENgpdl 12S7nVi060wPvZWE00vlxyzNriF+sFFCE/S+KC5feMd22qZNmgdU3fy5tMtsHfvV HT2RBYHADtA7sl/kk2Myd4K/s/xtjY5JP3jJkKGHEW4wIWZnQzJzlBw4hVValEJI pF1ogC001UIVIud1NFAAmMHBXEFLAMxinZXdOZVBdBQDgReZA/38hv+a2tRczoQt LOte/EUa97kieD8yiSWrp9rgQ2oOOQERj3A== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-transfer-encoding :content-type:content-type:date:date:feedback-id:feedback-id :from:from:in-reply-to:in-reply-to:message-id:mime-version :references:reply-to:reply-to:subject:subject:to:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t= 1708980970; x=1709067370; bh=ODE0ROICHsjD0yZP7iLxwoRZvmdJZzbZs1w Z6FX7X9k=; b=bp4JhDO6+sHuaMDjp3R19HRVAcTJ3cT1Z+YMXxBHek2OtVWeSIe bSoGWccfFZxHU/vp1hRV/Wx+/NGe2Nwi5VqPnD17ZmX+OO0h9nYYxtd5cYNsJpSZ k6g2gwk3TgcZZC/YqlkL5NwFr75VwxrFdHv/h4jJiey1C3KamfbEuZhqUTmIB1gT VqM+UEkjuQX2hgvT083qH5JYaQ5FeWxw3SNFXooaEzMoAAkw/w+VQ0Z7HSqLEbl8 wYgwWMCVwHlAZFQ5egIrbRKFj6xvh9Sln31XLTY/XDq2FTFETb5D5nn822mnX2BK oyJKkW5yKnszzEI9XSdKRE0NE9iNtLkdlEA== X-ME-Sender: X-ME-Received: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvledrgedvgddugedtucetufdoteggodetrfdotf fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen uceurghilhhouhhtmecufedttdenucdnrfhurhgthhgrshgvucdluddtmdenucfjughrpe fhvfevufffkffojghfrhggtgfgsehtkeertdertdejnecuhfhrohhmpegkihcujggrnhcu oeiiihdrhigrnhesshgvnhhtrdgtohhmqeenucggtffrrghtthgvrhhnpeettdekjeffke ejudelhfeuteekieejieehjeefvdefiefhteevgeejheegkefggfenucffohhmrghinhep khgvrhhnvghlrdhorhhgnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrg hilhhfrhhomhepiihirdihrghnsehsvghnthdrtghomh X-ME-Proxy: Feedback-ID: iccd040f4:Fastmail Received: by mail.messagingengine.com (Postfix) with ESMTPA; Mon, 26 Feb 2024 15:56:10 -0500 (EST) From: Zi Yan To: "Pankaj Raghav (Samsung)" , linux-mm@kvack.org Cc: Zi Yan , "Matthew Wilcox (Oracle)" , David Hildenbrand , Yang Shi , Yu Zhao , "Kirill A . Shutemov" , Ryan Roberts , =?utf-8?q?Michal_Koutn=C3=BD?= , Roman Gushchin , "Zach O'Keefe" , Hugh Dickins , Luis Chamberlain , Andrew Morton , linux-kernel@vger.kernel.org, cgroups@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-kselftest@vger.kernel.org Subject: [PATCH v5 2/8] mm: Support order-1 folios in the page cache Date: Mon, 26 Feb 2024 15:55:28 -0500 Message-ID: <20240226205534.1603748-3-zi.yan@sent.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240226205534.1603748-1-zi.yan@sent.com> References: <20240226205534.1603748-1-zi.yan@sent.com> Reply-To: Zi Yan MIME-Version: 1.0 X-Rspamd-Queue-Id: 78E96C000E X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: wbuodgd7is5zbgx4pomp9apxdx3t6ix9 X-HE-Tag: 1708980971-77284 X-HE-Meta: U2FsdGVkX18aWbvpGyGP0rQAGZD6jPtJaeMXAkp29EYYvNGFi9Wkn/YccttmC5JcVa33GiXPOKLTaOmgR1/AZQwR0SybdFeTOCUW7pj3hSCyPC4qQvkEeM+aa6Cbax2CeVwEWZYdoXKHHatGHuTErQtfZj3QTQugK9Odr/ArBkKuFj7HGsxhNGraDjag3WqKa+WNlNi0ntz01jXv4kcZubmbZp8cFCzjiNg6WbAg0seKqnTjfxt8JikEDw5SR4jnYxzSjZAA1sdSwvZowK0Cvu3Fi+AfWIJhz+3zsVysSsV9BtOQdfhdG5Ntt1LCHMV51rEwk7gpBwegDQWFddHwWfPs6EFBxiHHm+ifOyzTUZ2K8aveIKpjkjJrmSQ03ZDEeCH6+zWMIfS4uVCAKe91ZeVhwV8AAQ0uohiz2rIkMRFlFkmgxnAfJyiVF+oAd6fI/Y9M7jZqtQnY3ylNqAz124TsjSxFeweTyMSKB6NgrMVqBdd09sd+Ide97MlITHCZtmnOe+5BD5IAAg6WyIR3orPjyZBRr3TCpkTvSRlNS++wV3f9hdKfcQA5IP+CaGvK01DCftsh+H/JDw2KCRgEwJi+vaGkBrB3qWa8jouUly92nO3gPCG7/0NN5XNhOvCB6YyQ6Af1uRHE3fwp6PEncazeysHeVadkHKM3NKxQTkSUSrwf6cbRxRaOtuFzdUGrbeLzYh1RXj60BQ97FpKAazL9oEfO48j278udc4GaEXWA8Aq/cPEV+2bTDFWo8oYi30jYL9dmH07t6NxAezPObVTLsckrdk2DrGiiX5dxmdqUkgit+O6ieCYjZ3zqayKG6kNwU71N4T7+3AJHJ3BS/Q2/YptDWQLWnaN6KICGQVTZG3oHx0S9al5VG3OKZfEiwUMFyL6So7tVyl8nZcsDyy0xRmDhECAbasqduZyDJKvB3AZ7ebUb6pSBv7npyg6V2ppdK5ZhXnCFD2E3pdW ELfAYcwE xu4IRMjoytu6xZfte4HX7qLbeH2N3aXTChIlAOAWiO5ocdZSS+45QhWODktWTZAkHMraYu0wf114UzHGbIjMhdwE3wy2B3dlB0rSjOJ9Yso/3QTM= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Matthew Wilcox (Oracle)" Folios of order 1 have no space to store the deferred list. This is not a problem for the page cache as file-backed folios are never placed on the deferred list. All we need to do is prevent the core MM from touching the deferred list for order 1 folios and remove the code which prevented us from allocating order 1 folios. Link: https://lore.kernel.org/linux-mm/90344ea7-4eec-47ee-5996-0c22f42d6a6a@google.com/ Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Zi Yan --- mm/filemap.c | 2 -- mm/huge_memory.c | 19 +++++++++++++++---- mm/internal.h | 3 +-- mm/readahead.c | 3 --- 4 files changed, 16 insertions(+), 11 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index b7a21551fbc7..b4858d89f1b1 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1912,8 +1912,6 @@ struct folio *__filemap_get_folio(struct address_space *mapping, pgoff_t index, gfp_t alloc_gfp = gfp; err = -ENOMEM; - if (order == 1) - order = 0; if (order > 0) alloc_gfp |= __GFP_NORETRY | __GFP_NOWARN; folio = filemap_alloc_folio(alloc_gfp, order); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index b20e535e874c..9840f312c08f 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -790,8 +790,10 @@ struct deferred_split *get_deferred_split_queue(struct folio *folio) void folio_prep_large_rmappable(struct folio *folio) { - VM_BUG_ON_FOLIO(folio_order(folio) < 2, folio); - INIT_LIST_HEAD(&folio->_deferred_list); + if (!folio || !folio_test_large(folio)) + return; + if (folio_order(folio) > 1) + INIT_LIST_HEAD(&folio->_deferred_list); folio_set_large_rmappable(folio); } @@ -3114,7 +3116,8 @@ int split_huge_page_to_list(struct page *page, struct list_head *list) /* Prevent deferred_split_scan() touching ->_refcount */ spin_lock(&ds_queue->split_queue_lock); if (folio_ref_freeze(folio, 1 + extra_pins)) { - if (!list_empty(&folio->_deferred_list)) { + if (folio_order(folio) > 1 && + !list_empty(&folio->_deferred_list)) { ds_queue->split_queue_len--; list_del(&folio->_deferred_list); } @@ -3165,6 +3168,9 @@ void folio_undo_large_rmappable(struct folio *folio) struct deferred_split *ds_queue; unsigned long flags; + if (folio_order(folio) <= 1) + return; + /* * At this point, there is no one trying to add the folio to * deferred_list. If folio is not in deferred_list, it's safe @@ -3190,7 +3196,12 @@ void deferred_split_folio(struct folio *folio) #endif unsigned long flags; - VM_BUG_ON_FOLIO(folio_order(folio) < 2, folio); + /* + * Order 1 folios have no space for a deferred list, but we also + * won't waste much memory by not adding them to the deferred list. + */ + if (folio_order(folio) <= 1) + return; /* * The try_to_unmap() in page reclaim path might reach here too, diff --git a/mm/internal.h b/mm/internal.h index 2b7efffbe4d7..c4853ebfa030 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -420,8 +420,7 @@ static inline struct folio *page_rmappable_folio(struct page *page) { struct folio *folio = (struct folio *)page; - if (folio && folio_order(folio) > 1) - folio_prep_large_rmappable(folio); + folio_prep_large_rmappable(folio); return folio; } diff --git a/mm/readahead.c b/mm/readahead.c index 1e74455f908e..130c0e7df99f 100644 --- a/mm/readahead.c +++ b/mm/readahead.c @@ -514,9 +514,6 @@ void page_cache_ra_order(struct readahead_control *ractl, /* Don't allocate pages past EOF */ while (index + (1UL << order) - 1 > limit) order--; - /* THP machinery does not support order-1 */ - if (order == 1) - order = 0; err = ra_alloc_folio(ractl, index, mark, order, gfp); if (err) break;