From patchwork Wed Dec 18 02:26:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13912918 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0A6D93597B; Wed, 18 Dec 2024 02:26:30 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488792; cv=none; b=s4eixCHwqUf207X8NjHveEvrb+nDUWB3S3u9k5YjDi5EjBwIhLcoBY8NpYwrNgcguA6hAqIOlVv/dIx4lrJMiWqEOuMueSStBsoeRsNPbAXaMMqruW4X3A3EAgcCWIkHS64NaAsnGFmF5FDDrJws1A1d0QaPFFvLEkg59Xvrxds= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488792; c=relaxed/simple; bh=CDDlCAKlASesiZGLekB4AsPrmJInuCb09o/+qxzGaxw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=sYTfYgFEJofYZnTQhRWE3ZSKRH3lvEzLpBF5Q8Djm9b3UAKoztDcaBxB2ehYgQWk/hyGe5QPElX8ct7iya9xzhE43IprKLeawE+fCyaHMjHQa5XnLMKVmgQxvIiIBY3HB/Uu5ItWOI1Bh3jOGEKANuqILOVD/nQV83M5Ftu2tY0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=BkxYihM/; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="BkxYihM/" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=PAjepITga0a8Mt36xpxUX/NyoUbIA2AsDDdcgVOuJ40=; b=BkxYihM/ixv1x+y1GBn61vuqpI 9uMGmUbRsMUgI7XBDYzqLUWkOar5BL6dzU9F1tUO4CIA4n0+LWX9pCweVDBnSQEWxMvpWm1n6JBKz /Dsn1uG4vzykfhqR7SvsS+CDSJHDeZ+bBrn3s4478DOW2HSvucEKQ5V8SD0zfoIKS0fwbT9gKHRj5 pLcksMtPjjUZFjz+Q9yHaEizn2AXP9Riv8BG6ZdIhs0Y3hhxRkhmGoPiQMmsLe+a7mNR9wk1km02m Itv1KGFNuxgryqijDbmFRyzjE6ia4ohMLRGuTSuLPwnpI0nR6n1/HkGfLKzBMvShiyHd2Jsw3hKxv phXnpcCg==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNjlc-0000000FOFS-0xNt; Wed, 18 Dec 2024 02:26:28 +0000 From: Luis Chamberlain To: hare@suse.de, willy@infradead.org, dave@stgolabs.net, david@fromorbit.com, djwong@kernel.org, kbusch@kernel.org Cc: john.g.garry@oracle.com, hch@lst.de, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, kernel@pankajraghav.com, mcgrof@kernel.org Subject: [PATCH 1/5] fs/buffer: move async batch read code into a helper Date: Tue, 17 Dec 2024 18:26:22 -0800 Message-ID: <20241218022626.3668119-2-mcgrof@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241218022626.3668119-1-mcgrof@kernel.org> References: <20241218022626.3668119-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Luis Chamberlain Move the code from block_read_full_folio() which does a batch of async reads into a helper. No functional changes. Signed-off-by: Luis Chamberlain --- fs/buffer.c | 79 +++++++++++++++++++++++++++++++++-------------------- 1 file changed, 49 insertions(+), 30 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index cc8452f60251..7c6aac0742a6 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2350,6 +2350,53 @@ bool block_is_partially_uptodate(struct folio *folio, size_t from, size_t count) } EXPORT_SYMBOL(block_is_partially_uptodate); +/* + * Stage one is to collect an array of buffer heads which we need a read for, + * you can then use this afterwards. On that effort you should also check + * to see if you really need a read, and if we are already fully mapped. + */ +static void bh_read_batch_async(struct folio *folio, + int nr, struct buffer_head *arr[], + bool fully_mapped, bool no_reads, + bool any_get_block_error) +{ + int i; + struct buffer_head *bh; + + if (fully_mapped) + folio_set_mappedtodisk(folio); + + if (no_reads) { + /* + * All buffers are uptodate or get_block() returned an + * error when trying to map them *all* buffers we can + * finish the read. + */ + folio_end_read(folio, !any_get_block_error); + return; + } + + /* Stage two: lock the buffers */ + for (i = 0; i < nr; i++) { + bh = arr[i]; + lock_buffer(bh); + mark_buffer_async_read(bh); + } + + /* + * Stage three: start the IO. Check for uptodateness + * inside the buffer lock in case another process reading + * the underlying blockdev brought it uptodate (the sct fix). + */ + for (i = 0; i < nr; i++) { + bh = arr[i]; + if (buffer_uptodate(bh)) + end_buffer_async_read(bh, 1); + else + submit_bh(REQ_OP_READ, bh); + } +} + /* * Generic "read_folio" function for block devices that have the normal * get_block functionality. This is most of the block device filesystems. @@ -2383,6 +2430,7 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) nr = 0; i = 0; + /* Stage one - collect buffer heads we need issue a read for */ do { if (buffer_uptodate(bh)) continue; @@ -2414,37 +2462,8 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) arr[nr++] = bh; } while (i++, iblock++, (bh = bh->b_this_page) != head); - if (fully_mapped) - folio_set_mappedtodisk(folio); - - if (!nr) { - /* - * All buffers are uptodate or get_block() returned an - * error when trying to map them - we can finish the read. - */ - folio_end_read(folio, !page_error); - return 0; - } - - /* Stage two: lock the buffers */ - for (i = 0; i < nr; i++) { - bh = arr[i]; - lock_buffer(bh); - mark_buffer_async_read(bh); - } + bh_read_batch_async(folio, nr, arr, fully_mapped, nr == 0, page_error); - /* - * Stage 3: start the IO. Check for uptodateness - * inside the buffer lock in case another process reading - * the underlying blockdev brought it uptodate (the sct fix). - */ - for (i = 0; i < nr; i++) { - bh = arr[i]; - if (buffer_uptodate(bh)) - end_buffer_async_read(bh, 1); - else - submit_bh(REQ_OP_READ, bh); - } return 0; } EXPORT_SYMBOL(block_read_full_folio); From patchwork Wed Dec 18 02:26:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13912919 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 429AA4642D; Wed, 18 Dec 2024 02:26:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488792; cv=none; b=AQ27NSaXKayOeU/JzgSYFj7IqeVbMgLKEWXXgGHLksnt98clY6OvDH0HUakkkEnuMFRZDaFLovY5hiwHtwBGsCIicn4O7xXwdA0khTB7vU2U2sP6McjIdE+FGlKH1rmgJtqdw5FyaVJSYzf1gt+xxKQmvqhwjWUVM2o7g5GKjr4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488792; c=relaxed/simple; bh=22eXPXm/5BoUDqAxOXUyQl9RxqIqzFk5NXYU3k/1j4g=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Wlp+9l10bTQlp4hitMDxXd90qQdkDQCT4BFSB2Av1CrUcf5/sEgYgmQ+hXxcF3LCj2m2imLNVGiOdu7jhpoHTIzYtaCA5zfLaNilU2kBARt61PgUcweDoEH8O3rJEM0WeWBSDd2G9wtGbdnl+eEKgulJyT+TuxsQBe6TnRJ1Ab8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=3qaC05k7; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="3qaC05k7" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=KAcXP1LLN9RIr4LUuyxBmqGvK7FM1npzcMSwryY+7ME=; b=3qaC05k7YgxBzQwXc9ZAOl7DAT k61pID00MJyziEMK2a/rV9rNOMTAiDPENc2QdtSsXU1PRi68fxGTFDy8dAn848eYwuVU4g6SzLUFN vwenXhO4VWLRUjJOHA3Z7Qyf6Yy6knyLtBAB/LupK9s0sdcSjpz8D53qvxjRTzgIcHp/10eV1u5jO lrh0jIW2l6IJq0zzB1AqB3idKyQzCI2cWnmBL4YFePKdAnGbpv/fMBW6odvahgNV7DTzfDcS2NF8b dghX2PPebP4c53s6nR+FcNy2avvEOLCpIjKRipbwyxpSMp2nQsdx43cWKgWU/lxo2IP9GVtm1W1/m BHnGwU/A==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNjlc-0000000FOFU-168x; Wed, 18 Dec 2024 02:26:28 +0000 From: Luis Chamberlain To: hare@suse.de, willy@infradead.org, dave@stgolabs.net, david@fromorbit.com, djwong@kernel.org, kbusch@kernel.org Cc: john.g.garry@oracle.com, hch@lst.de, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, kernel@pankajraghav.com, mcgrof@kernel.org Subject: [PATCH 2/5] fs/buffer: simplify block_read_full_folio() with bh_offset() Date: Tue, 17 Dec 2024 18:26:23 -0800 Message-ID: <20241218022626.3668119-3-mcgrof@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241218022626.3668119-1-mcgrof@kernel.org> References: <20241218022626.3668119-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Luis Chamberlain Remove the temporary variable i on the iteration of all buffer heads on a folio and just use bh_offset(bh) to simplify the loop. Suggested-by: Matthew Wilcox Signed-off-by: Luis Chamberlain --- fs/buffer.c | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 7c6aac0742a6..8baf87db110d 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2410,7 +2410,7 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) sector_t iblock, lblock; struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE]; size_t blocksize; - int nr, i; + int nr; int fully_mapped = 1; bool page_error = false; loff_t limit = i_size_read(inode); @@ -2428,7 +2428,6 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) lblock = div_u64(limit + blocksize - 1, blocksize); bh = head; nr = 0; - i = 0; /* Stage one - collect buffer heads we need issue a read for */ do { @@ -2446,7 +2445,7 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) page_error = true; } if (!buffer_mapped(bh)) { - folio_zero_range(folio, i * blocksize, + folio_zero_range(folio, bh_offset(bh), blocksize); if (!err) set_buffer_uptodate(bh); @@ -2460,7 +2459,7 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) continue; } arr[nr++] = bh; - } while (i++, iblock++, (bh = bh->b_this_page) != head); + } while (iblock++, (bh = bh->b_this_page) != head); bh_read_batch_async(folio, nr, arr, fully_mapped, nr == 0, page_error); From patchwork Wed Dec 18 02:26:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13912920 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5C2C2487A5; Wed, 18 Dec 2024 02:26:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488792; cv=none; b=VTtwon77Ig3wECHNw+aM5hUXreZxe7lJetjJcAS5LmvN80xE8r9EUuq9tNwiK6jE9EaNpHROT8rMctz33HmlyHrxovVcN8WeIPnuFwWoKi46SqhuPHquld885lPmJ6p3+Sy/l61hii1nkKO6FYs0Fgpx8zkfSZpCTJbCBR5r3NY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488792; c=relaxed/simple; bh=NIV9/pYL4zSBZEdju4ZrrpKy6bRxTm4c8YMNGicjqoU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tnZGk3L0Awa1JvxPr+j36cswPVYKbArGl2+F3YGy27GNUci06SZgf9LCB64cylDfjROb2i3ojaypW4JiY5SW9ONE05ZBVdus7cAi58EW88qmTyNQM4VkcZpNdyUnDl2kLu+vumx9uG/qqWBapUID04JlmNu2Lchj7KE0S/mSWNo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=YF8qki9A; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="YF8qki9A" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=/TZr4nFjkp9nWadpjSdHplXgIdajpwwrN5XnPc56j5k=; b=YF8qki9ApWj9O9QPiw4UQveFJ3 NQfKB26z4bEkBtNfWRijXe1dUgTBw9CxUwpzJfYVsCd7Rq3m0PVX5FbT0zu4spnX3P2bTOWTbHIjs BBAKfLiuJkXTrFwI1eODbWULVvv7LXNdVU2+HPaC/SoEa6MMKjk6+1fI6cFnczLZBrKbdhlAH3uDF C9+9eaa4KZovy/GjDgqOnOaKmip06d+jjsyFjML7RHJ3Hdb+H55jjeq0uLDA47ZG/ZvAyfapr3C1p o6Hs0K+5NPo/jujeuy6EEI7JzJXKcEytrh6LDi/AUNYX0/j6taWl5O/Me4vjhI5OGY20dAqBWLowM qmN/oplQ==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNjlc-0000000FOFW-1EJE; Wed, 18 Dec 2024 02:26:28 +0000 From: Luis Chamberlain To: hare@suse.de, willy@infradead.org, dave@stgolabs.net, david@fromorbit.com, djwong@kernel.org, kbusch@kernel.org Cc: john.g.garry@oracle.com, hch@lst.de, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, kernel@pankajraghav.com, mcgrof@kernel.org Subject: [PATCH 3/5] fs/buffer: add a for_each_bh() for block_read_full_folio() Date: Tue, 17 Dec 2024 18:26:24 -0800 Message-ID: <20241218022626.3668119-4-mcgrof@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241218022626.3668119-1-mcgrof@kernel.org> References: <20241218022626.3668119-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Luis Chamberlain We want to be able to work through all buffer heads on a folio for an async read, but in the future we want to support the option to stop before we've processed all linked buffer heads. To make code easier to read and follow adopt a for_each_bh(tmp, head) loop instead of using a do { ... } while () to make the code easier to read and later be expanded in subsequent patches. This introduces no functional changes. Reviewed-by: Hannes Reinecke Signed-off-by: Luis Chamberlain --- fs/buffer.c | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 8baf87db110d..1aeef7dd2281 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2397,6 +2397,17 @@ static void bh_read_batch_async(struct folio *folio, } } +#define bh_is_last(__bh, __head) ((__bh)->b_this_page == (__head)) + +#define bh_next(__bh, __head) \ + (bh_is_last(__bh, __head) ? NULL : (__bh)->b_this_page) + +/* Starts from the provided head */ +#define for_each_bh(__tmp, __head) \ + for ((__tmp) = (__head); \ + (__tmp); \ + (__tmp) = bh_next(__tmp, __head)) + /* * Generic "read_folio" function for block devices that have the normal * get_block functionality. This is most of the block device filesystems. @@ -2426,13 +2437,14 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) iblock = div_u64(folio_pos(folio), blocksize); lblock = div_u64(limit + blocksize - 1, blocksize); - bh = head; nr = 0; /* Stage one - collect buffer heads we need issue a read for */ - do { - if (buffer_uptodate(bh)) + for_each_bh(bh, head) { + if (buffer_uptodate(bh)) { + iblock++; continue; + } if (!buffer_mapped(bh)) { int err = 0; @@ -2449,17 +2461,21 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) blocksize); if (!err) set_buffer_uptodate(bh); + iblock++; continue; } /* * get_block() might have updated the buffer * synchronously */ - if (buffer_uptodate(bh)) + if (buffer_uptodate(bh)) { + iblock++; continue; + } } arr[nr++] = bh; - } while (iblock++, (bh = bh->b_this_page) != head); + iblock++; + } bh_read_batch_async(folio, nr, arr, fully_mapped, nr == 0, page_error); From patchwork Wed Dec 18 02:26:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13912922 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 748335473C; Wed, 18 Dec 2024 02:26:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488793; cv=none; b=Or09oEYkes+cEGV0rfAVNfCsFMP9K8MDiwgrX3YmP/tdKIY9RfK8GZ+73Gl3CbGzbX6sIudnqWDORdp1GykRj6sBKWQVx88rxvVkolZvmZkywOhGcKNWzrF1826fveTQA/7ayqtfNVS950IWT2XwBXZJuOyterrfcy7b1AglHss= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488793; c=relaxed/simple; bh=ER9tSsqhoVNibaFr1MDb1FcOOFy48X7L3Khc3IwaN4w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Bqs6fecTcupFstHMtYhDrkbhfqag8FjJvGZuFEgvwPtS7iqA3LA6tAOUaD58CeUYYc8fVNB7DNQ5YXJzRHbbwB7D0+AGVgQ9/BBxuKaJ0yoweo3ovFFQvi9zv7RGYYbH5skbV8lbhzcCAUSXS2ifhskd63+6CPvaZcBdQojKbiE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=UOyu5RHN; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="UOyu5RHN" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=2tu6vw3HnVklrsZdZBeEEpq3jVFl8xYfgifT0A/bbJA=; b=UOyu5RHNs1b1dIBIPJ3wXeM11K Qem2FStQNp12ErJNPvBxR8pD7ECxcshmwcWWQMoKlNNWXGc2UmQ17TtyzfCe62rNSP1oRjC+o5JJo vs5OeIciaTldFNmK4eHoM/ogSKq/h8jQGFaRs1bjS8wOQHdwl1/lxu28kqdPAcVDjtGDLnFDgmh3U xn5m+02QHyVvqS4WqbA0WIdEyGaai4Y5Io//FfGcKarijsMIoLsfLldTW9A1rttjzbwpg0+tGTjeP Lo2HtB1olSfaksGtbOY3dXGGsH/5C49xgoKFfzDxAdRm4d/8XVovi3sVIlX44mxqR+HpVwLDlmOgY N7Qza3aQ==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNjlc-0000000FOFZ-1MBK; Wed, 18 Dec 2024 02:26:28 +0000 From: Luis Chamberlain To: hare@suse.de, willy@infradead.org, dave@stgolabs.net, david@fromorbit.com, djwong@kernel.org, kbusch@kernel.org Cc: john.g.garry@oracle.com, hch@lst.de, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, kernel@pankajraghav.com, mcgrof@kernel.org Subject: [PATCH 4/5] fs/buffer: add iteration support for block_read_full_folio() Date: Tue, 17 Dec 2024 18:26:25 -0800 Message-ID: <20241218022626.3668119-5-mcgrof@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241218022626.3668119-1-mcgrof@kernel.org> References: <20241218022626.3668119-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Luis Chamberlain Provide a helper to iterate on buffer heads on a folio. We do this as a preliminary step so to make the subsequent changes easier to read. Right now we use an array on stack to loop over all buffer heads in a folio of size MAX_BUF_PER_PAGE, however on CPUs where the system page size is quite larger like Hexagon with 256 KiB page size support this can mean the kernel can end up spewing spews stack growth warnings. To be able to break this down into smaller array chunks add support for processing smaller array chunks of buffer heads at a time. The used array size is not changed yet, that will be done in a subsequent patch, this just adds the iterator support and logic. While at it clarify the booleans used on bh_read_batch_async() and how they are only valid in consideration when we've processed all buffer-heads of a folio, that is when we're on the last buffer head in a folio: * bh_folio_reads * unmapped Reviewed-by: Hannes Reinecke Signed-off-by: Luis Chamberlain --- fs/buffer.c | 134 +++++++++++++++++++++++++++++++++++++--------------- 1 file changed, 97 insertions(+), 37 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index 1aeef7dd2281..b8ba72f2f211 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2402,66 +2402,75 @@ static void bh_read_batch_async(struct folio *folio, #define bh_next(__bh, __head) \ (bh_is_last(__bh, __head) ? NULL : (__bh)->b_this_page) +/* Starts from a pivot which you initialize */ +#define for_each_bh_pivot(__pivot, __last, __head) \ + for ((__pivot) = __last = (__pivot); \ + (__pivot); \ + (__pivot) = bh_next(__pivot, __head), \ + (__last) = (__pivot) ? (__pivot) : (__last)) + /* Starts from the provided head */ #define for_each_bh(__tmp, __head) \ for ((__tmp) = (__head); \ (__tmp); \ (__tmp) = bh_next(__tmp, __head)) +struct bh_iter { + sector_t iblock; + get_block_t *get_block; + bool any_get_block_error; + int unmapped; + int bh_folio_reads; +}; + /* - * Generic "read_folio" function for block devices that have the normal - * get_block functionality. This is most of the block device filesystems. - * Reads the folio asynchronously --- the unlock_buffer() and - * set/clear_buffer_uptodate() functions propagate buffer state into the - * folio once IO has completed. + * Reads up to MAX_BUF_PER_PAGE buffer heads at a time on a folio on the given + * block range iblock to lblock and helps update the number of buffer-heads + * which were not uptodate or unmapped for which we issued an async read for + * on iter->bh_folio_reads for the full folio. Returns the last buffer-head we + * worked on. */ -int block_read_full_folio(struct folio *folio, get_block_t *get_block) -{ - struct inode *inode = folio->mapping->host; - sector_t iblock, lblock; - struct buffer_head *bh, *head, *arr[MAX_BUF_PER_PAGE]; - size_t blocksize; - int nr; - int fully_mapped = 1; - bool page_error = false; - loff_t limit = i_size_read(inode); - - /* This is needed for ext4. */ - if (IS_ENABLED(CONFIG_FS_VERITY) && IS_VERITY(inode)) - limit = inode->i_sb->s_maxbytes; +static struct buffer_head *bh_read_iter(struct folio *folio, + struct buffer_head *pivot, + struct buffer_head *head, + struct inode *inode, + struct bh_iter *iter, sector_t lblock) +{ + struct buffer_head *arr[MAX_BUF_PER_PAGE]; + struct buffer_head *bh = pivot, *last; + int nr = 0, i = 0; + size_t blocksize = head->b_size; + bool no_reads = false; + bool fully_mapped = false; - VM_BUG_ON_FOLIO(folio_test_large(folio), folio); + /* Stage one - collect buffer heads we need issue a read for */ - head = folio_create_buffers(folio, inode, 0); - blocksize = head->b_size; + /* collect buffers not uptodate and not mapped yet */ + for_each_bh_pivot(bh, last, head) { + BUG_ON(nr >= MAX_BUF_PER_PAGE); - iblock = div_u64(folio_pos(folio), blocksize); - lblock = div_u64(limit + blocksize - 1, blocksize); - nr = 0; - - /* Stage one - collect buffer heads we need issue a read for */ - for_each_bh(bh, head) { if (buffer_uptodate(bh)) { - iblock++; + iter->iblock++; continue; } if (!buffer_mapped(bh)) { int err = 0; - fully_mapped = 0; - if (iblock < lblock) { + iter->unmapped++; + if (iter->iblock < lblock) { WARN_ON(bh->b_size != blocksize); - err = get_block(inode, iblock, bh, 0); + err = iter->get_block(inode, iter->iblock, + bh, 0); if (err) - page_error = true; + iter->any_get_block_error = true; } if (!buffer_mapped(bh)) { folio_zero_range(folio, bh_offset(bh), blocksize); if (!err) set_buffer_uptodate(bh); - iblock++; + iter->iblock++; continue; } /* @@ -2469,15 +2478,66 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) * synchronously */ if (buffer_uptodate(bh)) { - iblock++; + iter->iblock++; continue; } } arr[nr++] = bh; - iblock++; + iter->iblock++; + } + + iter->bh_folio_reads += nr; + + WARN_ON_ONCE(!bh_is_last(last, head)); + + if (bh_is_last(last, head)) { + if (!iter->bh_folio_reads) + no_reads = true; + if (!iter->unmapped) + fully_mapped = true; } - bh_read_batch_async(folio, nr, arr, fully_mapped, nr == 0, page_error); + bh_read_batch_async(folio, nr, arr, fully_mapped, no_reads, + iter->any_get_block_error); + + return last; +} + +/* + * Generic "read_folio" function for block devices that have the normal + * get_block functionality. This is most of the block device filesystems. + * Reads the folio asynchronously --- the unlock_buffer() and + * set/clear_buffer_uptodate() functions propagate buffer state into the + * folio once IO has completed. + */ +int block_read_full_folio(struct folio *folio, get_block_t *get_block) +{ + struct inode *inode = folio->mapping->host; + sector_t lblock; + size_t blocksize; + struct buffer_head *bh, *head; + struct bh_iter iter = { + .get_block = get_block, + .unmapped = 0, + .any_get_block_error = false, + .bh_folio_reads = 0, + }; + loff_t limit = i_size_read(inode); + + /* This is needed for ext4. */ + if (IS_ENABLED(CONFIG_FS_VERITY) && IS_VERITY(inode)) + limit = inode->i_sb->s_maxbytes; + + VM_BUG_ON_FOLIO(folio_test_large(folio), folio); + + head = folio_create_buffers(folio, inode, 0); + blocksize = head->b_size; + + iter.iblock = div_u64(folio_pos(folio), blocksize); + lblock = div_u64(limit + blocksize - 1, blocksize); + + for_each_bh(bh, head) + bh = bh_read_iter(folio, bh, head, inode, &iter, lblock); return 0; } From patchwork Wed Dec 18 02:26:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luis Chamberlain X-Patchwork-Id: 13912921 Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 748785FEED; Wed, 18 Dec 2024 02:26:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.137.202.133 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488792; cv=none; b=hXVoV4rPDtmUip9wiTv8X5tjUvBuZQMNYx2Bmf1UT04XDYj4jXMXI247yLRcvRlS0dMRtdnVK2/sMyP7H9lla8S3s0bD5EvlqLR5mexj0GonnpVilpSEDzjxOA3CBdAghmIg98DvXWxXOR6X7i34KhpnG5tF1RuJpHlxdvBMkd8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1734488792; c=relaxed/simple; bh=zoKQiCdk3zMvemNYIWb1hXPGTcHsahTKaEEV/z8Mr5M=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=TjAfTvim4eSDyuTTlhHzPVjfn47Rzn/8OKJR5v4GTHAQON+dQHAQKiXtZhPTtjA2tUfPtLPqVYfsaQA1cICXb/T1MX1DG3ePtZ/U4d3VKvAXIgVgmmkmhekayDK+owHoq5lnDwKZiaAeXBHJO9il5/spHJUthRSAl/IivvhFwMo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org; spf=none smtp.mailfrom=infradead.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b=rOCE0qvP; arc=none smtp.client-ip=198.137.202.133 Authentication-Results: smtp.subspace.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=kernel.org Authentication-Results: smtp.subspace.kernel.org; spf=none smtp.mailfrom=infradead.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=infradead.org header.i=@infradead.org header.b="rOCE0qvP" DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20210309; h=Sender:Content-Transfer-Encoding: MIME-Version:References:In-Reply-To:Message-ID:Date:Subject:Cc:To:From: Reply-To:Content-Type:Content-ID:Content-Description; bh=ZbUYX9Sq5KIho4Fukut+jcN4YUtrDSxc8GT+CZTyNJI=; b=rOCE0qvP1bit6BhC2nOW+ZmD1C MjnSDQBMGg0LebsCOfpJZS5VU+y1MF9ftXASeJHmNQu668uZk62viyqP2nWwMFil83pO0XL6VaLcc 7ebXitKACjEMvt3i+W7gaNBm8ysthbep6l4kO2V96ZIJETU/OEl0ofk9QvN1ogbhPTg9gLYq3LVfy i3b8vLmSXM3vSiaWT0vzDdfIPVuI6xps08Qj8YCSppp+iiqyXLC7IjPxC+NEq3/4A7ICfNRpO6GM4 QYbvzZjEM7IFZaFXNlhwvIVOtZZTL9Kmwf2IQlX1GUowQNUkXADtFpVK9hUCgEOLpdmccNaGqUWra 0NvGIbdA==; Received: from mcgrof by bombadil.infradead.org with local (Exim 4.98 #2 (Red Hat Linux)) id 1tNjlc-0000000FOFb-1UCn; Wed, 18 Dec 2024 02:26:28 +0000 From: Luis Chamberlain To: hare@suse.de, willy@infradead.org, dave@stgolabs.net, david@fromorbit.com, djwong@kernel.org, kbusch@kernel.org Cc: john.g.garry@oracle.com, hch@lst.de, ritesh.list@gmail.com, linux-fsdevel@vger.kernel.org, linux-xfs@vger.kernel.org, linux-mm@kvack.org, linux-block@vger.kernel.org, gost.dev@samsung.com, p.raghav@samsung.com, da.gomez@samsung.com, kernel@pankajraghav.com, mcgrof@kernel.org Subject: [PATCH 5/5] fs/buffer: reduce stack usage on bh_read_iter() Date: Tue, 17 Dec 2024 18:26:26 -0800 Message-ID: <20241218022626.3668119-6-mcgrof@kernel.org> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20241218022626.3668119-1-mcgrof@kernel.org> References: <20241218022626.3668119-1-mcgrof@kernel.org> Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Sender: Luis Chamberlain Now that we can read asynchronously buffer heads from a folio in chunks, we can chop up bh_read_iter() with a smaller array size. Use an array of 8 to avoid stack growth warnings on systems with huge base page sizes. Reviewed-by: Hannes Reinecke Signed-off-by: Luis Chamberlain --- fs/buffer.c | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/fs/buffer.c b/fs/buffer.c index b8ba72f2f211..bfa9c09b8597 100644 --- a/fs/buffer.c +++ b/fs/buffer.c @@ -2415,7 +2415,10 @@ static void bh_read_batch_async(struct folio *folio, (__tmp); \ (__tmp) = bh_next(__tmp, __head)) +#define MAX_BUF_CHUNK 8 + struct bh_iter { + int chunk_number; sector_t iblock; get_block_t *get_block; bool any_get_block_error; @@ -2424,7 +2427,7 @@ struct bh_iter { }; /* - * Reads up to MAX_BUF_PER_PAGE buffer heads at a time on a folio on the given + * Reads up to MAX_BUF_CHUNK buffer heads at a time on a folio on the given * block range iblock to lblock and helps update the number of buffer-heads * which were not uptodate or unmapped for which we issued an async read for * on iter->bh_folio_reads for the full folio. Returns the last buffer-head we @@ -2436,10 +2439,11 @@ static struct buffer_head *bh_read_iter(struct folio *folio, struct inode *inode, struct bh_iter *iter, sector_t lblock) { - struct buffer_head *arr[MAX_BUF_PER_PAGE]; + struct buffer_head *arr[MAX_BUF_CHUNK]; struct buffer_head *bh = pivot, *last; int nr = 0, i = 0; size_t blocksize = head->b_size; + int chunk_idx = MAX_BUF_CHUNK * iter->chunk_number; bool no_reads = false; bool fully_mapped = false; @@ -2447,7 +2451,8 @@ static struct buffer_head *bh_read_iter(struct folio *folio, /* collect buffers not uptodate and not mapped yet */ for_each_bh_pivot(bh, last, head) { - BUG_ON(nr >= MAX_BUF_PER_PAGE); + if (nr >= MAX_BUF_CHUNK) + break; if (buffer_uptodate(bh)) { iter->iblock++; @@ -2487,8 +2492,7 @@ static struct buffer_head *bh_read_iter(struct folio *folio, } iter->bh_folio_reads += nr; - - WARN_ON_ONCE(!bh_is_last(last, head)); + iter->chunk_number++; if (bh_is_last(last, head)) { if (!iter->bh_folio_reads) @@ -2518,6 +2522,7 @@ int block_read_full_folio(struct folio *folio, get_block_t *get_block) struct buffer_head *bh, *head; struct bh_iter iter = { .get_block = get_block, + .chunk_number = 0, .unmapped = 0, .any_get_block_error = false, .bh_folio_reads = 0,