From patchwork Tue Jun 6 11:43:48 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 13268982 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16738C77B7A for ; Tue, 6 Jun 2023 11:44:12 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229827AbjFFLoK (ORCPT ); Tue, 6 Jun 2023 07:44:10 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36788 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237210AbjFFLoH (ORCPT ); Tue, 6 Jun 2023 07:44:07 -0400 Received: from mail-pl1-x62d.google.com (mail-pl1-x62d.google.com [IPv6:2607:f8b0:4864:20::62d]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 09F699E; Tue, 6 Jun 2023 04:44:05 -0700 (PDT) Received: by mail-pl1-x62d.google.com with SMTP id d9443c01a7336-1b075e13a5eso52465765ad.3; Tue, 06 Jun 2023 04:44:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686051844; x=1688643844; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=TAzOgGtsg/+Kih2uty4XyhobUXGR9HLkebgBRhOE9sY=; b=lbZCRp8Tc/XE2OuvD+xEQtEFiUO7e1oohzRNOteCSbJn/iz4E72dUl16QQLm/19EvU mkDgeZUz1dkkX4fS3JQ1jAqJaFAOWEL6PyBHTXSrQpsyH41st3f8wT4/7mC8CiH7Mjhv d1fi6Ya+ypm26zYM/UX+YilTKRc6laUiBVRbtWlnizJgooswCiR398IiSpV87jO2L0Dv oMhhnvJn1V3fEI5fIcqHlgva7s+p2xfqTjf/65m0KG6tZeoFyM0/PWsYguTrvsEsfGIe FS34YKyohM7xjfXiYwkZ8XOpBBAasUBGUZVnS1zAre/WF2nSQLCcJvkrv0IjC8dM60Mn o8zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686051844; x=1688643844; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=TAzOgGtsg/+Kih2uty4XyhobUXGR9HLkebgBRhOE9sY=; b=OGLf7ZGygq+ZGb90u6wyDcNLDf3FK2RpGB8BFx6Y6prkFDVjtFUKr7sK878Q9ZDP3c 87IK5eGmHckbZcZ+x9q7fktXGZn5FHyA+M1mmq8evqg7wyAU5Ym2Cbv+o7jCvOHS0NiQ mUFjy/6VnbzQ9BZ7hUgwMtwFrDbAAqGC6bdQFKWi1MQOPLm0ci2FTLnq8ld2qW7cG5oa I32WNDnOLFi/odqn9REiF0CPtFb7rUPUMpoyFYzwzg3KMPG4EA061ps6zYG0OypOl672 /RFfIGjE0YuTta5CvBhoDB2a3bDzIIIVydJ54n6dAUvCyefjoSCusuEWLWDGfjQRpCVy yg7g== X-Gm-Message-State: AC+VfDxD7oAk3hhlgMIrPd/RNBEAbzYiAVE7Au+gBNmJX3KXhCeSJ3qJ B2Ktm7o6RkoRZpT8HUiq66dfwhLvR4E= X-Google-Smtp-Source: ACHHUZ68P6yhZMiR6wc8mpsq9FRoFW+jW0CbTneg9fNnAUggF1aBd4Nsek86qVA8bNKWmFZQp6Xmbg== X-Received: by 2002:a17:903:1252:b0:1b0:5304:5b4e with SMTP id u18-20020a170903125200b001b053045b4emr2614604plh.43.1686051844064; Tue, 06 Jun 2023 04:44:04 -0700 (PDT) Received: from dw-tp.localdomain ([49.207.220.159]) by smtp.gmail.com with ESMTPSA id y4-20020a170902ed4400b001ab0a30c895sm8325120plb.202.2023.06.06.04.44.00 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Jun 2023 04:44:03 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, "Darrick J. Wong" , Matthew Wilcox , Dave Chinner , Brian Foster , Christoph Hellwig , Andreas Gruenbacher , Ojaswin Mujoo , Disha Goel , "Ritesh Harjani (IBM)" Subject: [PATCHv8 1/5] iomap: Rename iomap_page to iomap_folio and others Date: Tue, 6 Jun 2023 17:13:48 +0530 Message-Id: X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This patch renames - 1. struct iomap_page -> struct iomap_folio 2. iop -> iof 3. iomap_page_create() -> iomap_iof_alloc() 4. iomap_page_release() -> iomap_iof_free() 5. to_iomap_page() -> iomap_get_iof() Since iomap works with folio and all the necessary conversions are already in place. Hence rename the structures like struct iomap_page while we are already working on it (later patches will add per-block dirty status tracking to iomap_folio state bitmap). Later patches also adds more functions for handling iof structure with iomap_iof_** naming conventions. Hence iomap_iof_alloc/free() naming makes more sense to be consistent with all APIs. Signed-off-by: Ritesh Harjani (IBM) --- fs/iomap/buffered-io.c | 138 ++++++++++++++++++++--------------------- 1 file changed, 69 insertions(+), 69 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 063133ec77f4..741baa10c517 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -27,14 +27,14 @@ * Structure allocated for each folio when block size < folio size * to track sub-folio uptodate status and I/O completions. */ -struct iomap_page { +struct iomap_folio { atomic_t read_bytes_pending; atomic_t write_bytes_pending; spinlock_t uptodate_lock; unsigned long uptodate[]; }; -static inline struct iomap_page *to_iomap_page(struct folio *folio) +static inline struct iomap_folio *iomap_get_iof(struct folio *folio) { if (folio_test_private(folio)) return folio_get_private(folio); @@ -43,45 +43,45 @@ static inline struct iomap_page *to_iomap_page(struct folio *folio) static struct bio_set iomap_ioend_bioset; -static struct iomap_page * -iomap_page_create(struct inode *inode, struct folio *folio, unsigned int flags) +static struct iomap_folio *iomap_iof_alloc(struct inode *inode, + struct folio *folio, unsigned int flags) { - struct iomap_page *iop = to_iomap_page(folio); + struct iomap_folio *iof = iomap_get_iof(folio); unsigned int nr_blocks = i_blocks_per_folio(inode, folio); gfp_t gfp; - if (iop || nr_blocks <= 1) - return iop; + if (iof || nr_blocks <= 1) + return iof; if (flags & IOMAP_NOWAIT) gfp = GFP_NOWAIT; else gfp = GFP_NOFS | __GFP_NOFAIL; - iop = kzalloc(struct_size(iop, uptodate, BITS_TO_LONGS(nr_blocks)), + iof = kzalloc(struct_size(iof, uptodate, BITS_TO_LONGS(nr_blocks)), gfp); - if (iop) { - spin_lock_init(&iop->uptodate_lock); + if (iof) { + spin_lock_init(&iof->uptodate_lock); if (folio_test_uptodate(folio)) - bitmap_fill(iop->uptodate, nr_blocks); - folio_attach_private(folio, iop); + bitmap_fill(iof->uptodate, nr_blocks); + folio_attach_private(folio, iof); } - return iop; + return iof; } -static void iomap_page_release(struct folio *folio) +static void iomap_iof_free(struct folio *folio) { - struct iomap_page *iop = folio_detach_private(folio); + struct iomap_folio *iof = folio_detach_private(folio); struct inode *inode = folio->mapping->host; unsigned int nr_blocks = i_blocks_per_folio(inode, folio); - if (!iop) + if (!iof) return; - WARN_ON_ONCE(atomic_read(&iop->read_bytes_pending)); - WARN_ON_ONCE(atomic_read(&iop->write_bytes_pending)); - WARN_ON_ONCE(bitmap_full(iop->uptodate, nr_blocks) != + WARN_ON_ONCE(atomic_read(&iof->read_bytes_pending)); + WARN_ON_ONCE(atomic_read(&iof->write_bytes_pending)); + WARN_ON_ONCE(bitmap_full(iof->uptodate, nr_blocks) != folio_test_uptodate(folio)); - kfree(iop); + kfree(iof); } /* @@ -90,7 +90,7 @@ static void iomap_page_release(struct folio *folio) static void iomap_adjust_read_range(struct inode *inode, struct folio *folio, loff_t *pos, loff_t length, size_t *offp, size_t *lenp) { - struct iomap_page *iop = to_iomap_page(folio); + struct iomap_folio *iof = iomap_get_iof(folio); loff_t orig_pos = *pos; loff_t isize = i_size_read(inode); unsigned block_bits = inode->i_blkbits; @@ -105,12 +105,12 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio, * per-block uptodate status and adjust the offset and length if needed * to avoid reading in already uptodate ranges. */ - if (iop) { + if (iof) { unsigned int i; /* move forward for each leading block marked uptodate */ for (i = first; i <= last; i++) { - if (!test_bit(i, iop->uptodate)) + if (!test_bit(i, iof->uptodate)) break; *pos += block_size; poff += block_size; @@ -120,7 +120,7 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio, /* truncate len if we find any trailing uptodate block(s) */ for ( ; i <= last; i++) { - if (test_bit(i, iop->uptodate)) { + if (test_bit(i, iof->uptodate)) { plen -= (last - i + 1) * block_size; last = i - 1; break; @@ -144,26 +144,26 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio, *lenp = plen; } -static void iomap_iop_set_range_uptodate(struct folio *folio, - struct iomap_page *iop, size_t off, size_t len) +static void iomap_iof_set_range_uptodate(struct folio *folio, + struct iomap_folio *iof, size_t off, size_t len) { struct inode *inode = folio->mapping->host; unsigned first = off >> inode->i_blkbits; unsigned last = (off + len - 1) >> inode->i_blkbits; unsigned long flags; - spin_lock_irqsave(&iop->uptodate_lock, flags); - bitmap_set(iop->uptodate, first, last - first + 1); - if (bitmap_full(iop->uptodate, i_blocks_per_folio(inode, folio))) + spin_lock_irqsave(&iof->uptodate_lock, flags); + bitmap_set(iof->uptodate, first, last - first + 1); + if (bitmap_full(iof->uptodate, i_blocks_per_folio(inode, folio))) folio_mark_uptodate(folio); - spin_unlock_irqrestore(&iop->uptodate_lock, flags); + spin_unlock_irqrestore(&iof->uptodate_lock, flags); } static void iomap_set_range_uptodate(struct folio *folio, - struct iomap_page *iop, size_t off, size_t len) + struct iomap_folio *iof, size_t off, size_t len) { - if (iop) - iomap_iop_set_range_uptodate(folio, iop, off, len); + if (iof) + iomap_iof_set_range_uptodate(folio, iof, off, len); else folio_mark_uptodate(folio); } @@ -171,16 +171,16 @@ static void iomap_set_range_uptodate(struct folio *folio, static void iomap_finish_folio_read(struct folio *folio, size_t offset, size_t len, int error) { - struct iomap_page *iop = to_iomap_page(folio); + struct iomap_folio *iof = iomap_get_iof(folio); if (unlikely(error)) { folio_clear_uptodate(folio); folio_set_error(folio); } else { - iomap_set_range_uptodate(folio, iop, offset, len); + iomap_set_range_uptodate(folio, iof, offset, len); } - if (!iop || atomic_sub_and_test(len, &iop->read_bytes_pending)) + if (!iof || atomic_sub_and_test(len, &iof->read_bytes_pending)) folio_unlock(folio); } @@ -213,7 +213,7 @@ struct iomap_readpage_ctx { static int iomap_read_inline_data(const struct iomap_iter *iter, struct folio *folio) { - struct iomap_page *iop; + struct iomap_folio *iof; const struct iomap *iomap = iomap_iter_srcmap(iter); size_t size = i_size_read(iter->inode) - iomap->offset; size_t poff = offset_in_page(iomap->offset); @@ -231,15 +231,15 @@ static int iomap_read_inline_data(const struct iomap_iter *iter, if (WARN_ON_ONCE(size > iomap->length)) return -EIO; if (offset > 0) - iop = iomap_page_create(iter->inode, folio, iter->flags); + iof = iomap_iof_alloc(iter->inode, folio, iter->flags); else - iop = to_iomap_page(folio); + iof = iomap_get_iof(folio); addr = kmap_local_folio(folio, offset); memcpy(addr, iomap->inline_data, size); memset(addr + size, 0, PAGE_SIZE - poff - size); kunmap_local(addr); - iomap_set_range_uptodate(folio, iop, offset, PAGE_SIZE - poff); + iomap_set_range_uptodate(folio, iof, offset, PAGE_SIZE - poff); return 0; } @@ -260,7 +260,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter, loff_t pos = iter->pos + offset; loff_t length = iomap_length(iter) - offset; struct folio *folio = ctx->cur_folio; - struct iomap_page *iop; + struct iomap_folio *iof; loff_t orig_pos = pos; size_t poff, plen; sector_t sector; @@ -269,20 +269,20 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter, return iomap_read_inline_data(iter, folio); /* zero post-eof blocks as the page may be mapped */ - iop = iomap_page_create(iter->inode, folio, iter->flags); + iof = iomap_iof_alloc(iter->inode, folio, iter->flags); iomap_adjust_read_range(iter->inode, folio, &pos, length, &poff, &plen); if (plen == 0) goto done; if (iomap_block_needs_zeroing(iter, pos)) { folio_zero_range(folio, poff, plen); - iomap_set_range_uptodate(folio, iop, poff, plen); + iomap_set_range_uptodate(folio, iof, poff, plen); goto done; } ctx->cur_folio_in_bio = true; - if (iop) - atomic_add(plen, &iop->read_bytes_pending); + if (iof) + atomic_add(plen, &iof->read_bytes_pending); sector = iomap_sector(iomap, pos); if (!ctx->bio || @@ -436,11 +436,11 @@ EXPORT_SYMBOL_GPL(iomap_readahead); */ bool iomap_is_partially_uptodate(struct folio *folio, size_t from, size_t count) { - struct iomap_page *iop = to_iomap_page(folio); + struct iomap_folio *iof = iomap_get_iof(folio); struct inode *inode = folio->mapping->host; unsigned first, last, i; - if (!iop) + if (!iof) return false; /* Caller's range may extend past the end of this folio */ @@ -451,7 +451,7 @@ bool iomap_is_partially_uptodate(struct folio *folio, size_t from, size_t count) last = (from + count - 1) >> inode->i_blkbits; for (i = first; i <= last; i++) - if (!test_bit(i, iop->uptodate)) + if (!test_bit(i, iof->uptodate)) return false; return true; } @@ -490,7 +490,7 @@ bool iomap_release_folio(struct folio *folio, gfp_t gfp_flags) */ if (folio_test_dirty(folio) || folio_test_writeback(folio)) return false; - iomap_page_release(folio); + iomap_iof_free(folio); return true; } EXPORT_SYMBOL_GPL(iomap_release_folio); @@ -507,12 +507,12 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len) if (offset == 0 && len == folio_size(folio)) { WARN_ON_ONCE(folio_test_writeback(folio)); folio_cancel_dirty(folio); - iomap_page_release(folio); + iomap_iof_free(folio); } else if (folio_test_large(folio)) { - /* Must release the iop so the page can be split */ + /* Must release the iof so the page can be split */ WARN_ON_ONCE(!folio_test_uptodate(folio) && folio_test_dirty(folio)); - iomap_page_release(folio); + iomap_iof_free(folio); } } EXPORT_SYMBOL_GPL(iomap_invalidate_folio); @@ -547,7 +547,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, size_t len, struct folio *folio) { const struct iomap *srcmap = iomap_iter_srcmap(iter); - struct iomap_page *iop; + struct iomap_folio *iof; loff_t block_size = i_blocksize(iter->inode); loff_t block_start = round_down(pos, block_size); loff_t block_end = round_up(pos + len, block_size); @@ -559,8 +559,8 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, return 0; folio_clear_error(folio); - iop = iomap_page_create(iter->inode, folio, iter->flags); - if ((iter->flags & IOMAP_NOWAIT) && !iop && nr_blocks > 1) + iof = iomap_iof_alloc(iter->inode, folio, iter->flags); + if ((iter->flags & IOMAP_NOWAIT) && !iof && nr_blocks > 1) return -EAGAIN; do { @@ -589,7 +589,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, if (status) return status; } - iomap_set_range_uptodate(folio, iop, poff, plen); + iomap_set_range_uptodate(folio, iof, poff, plen); } while ((block_start += plen) < block_end); return 0; @@ -696,7 +696,7 @@ static int iomap_write_begin(struct iomap_iter *iter, loff_t pos, static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len, size_t copied, struct folio *folio) { - struct iomap_page *iop = to_iomap_page(folio); + struct iomap_folio *iof = iomap_get_iof(folio); flush_dcache_folio(folio); /* @@ -712,7 +712,7 @@ static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len, */ if (unlikely(copied < len && !folio_test_uptodate(folio))) return 0; - iomap_set_range_uptodate(folio, iop, offset_in_folio(folio, pos), len); + iomap_set_range_uptodate(folio, iof, offset_in_folio(folio, pos), len); filemap_dirty_folio(inode->i_mapping, folio); return copied; } @@ -1290,17 +1290,17 @@ EXPORT_SYMBOL_GPL(iomap_page_mkwrite); static void iomap_finish_folio_write(struct inode *inode, struct folio *folio, size_t len, int error) { - struct iomap_page *iop = to_iomap_page(folio); + struct iomap_folio *iof = iomap_get_iof(folio); if (error) { folio_set_error(folio); mapping_set_error(inode->i_mapping, error); } - WARN_ON_ONCE(i_blocks_per_folio(inode, folio) > 1 && !iop); - WARN_ON_ONCE(iop && atomic_read(&iop->write_bytes_pending) <= 0); + WARN_ON_ONCE(i_blocks_per_folio(inode, folio) > 1 && !iof); + WARN_ON_ONCE(iof && atomic_read(&iof->write_bytes_pending) <= 0); - if (!iop || atomic_sub_and_test(len, &iop->write_bytes_pending)) + if (!iof || atomic_sub_and_test(len, &iof->write_bytes_pending)) folio_end_writeback(folio); } @@ -1567,7 +1567,7 @@ iomap_can_add_to_ioend(struct iomap_writepage_ctx *wpc, loff_t offset, */ static void iomap_add_to_ioend(struct inode *inode, loff_t pos, struct folio *folio, - struct iomap_page *iop, struct iomap_writepage_ctx *wpc, + struct iomap_folio *iof, struct iomap_writepage_ctx *wpc, struct writeback_control *wbc, struct list_head *iolist) { sector_t sector = iomap_sector(&wpc->iomap, pos); @@ -1585,8 +1585,8 @@ iomap_add_to_ioend(struct inode *inode, loff_t pos, struct folio *folio, bio_add_folio(wpc->ioend->io_bio, folio, len, poff); } - if (iop) - atomic_add(len, &iop->write_bytes_pending); + if (iof) + atomic_add(len, &iof->write_bytes_pending); wpc->ioend->io_size += len; wbc_account_cgroup_owner(wbc, &folio->page, len); } @@ -1612,7 +1612,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, struct writeback_control *wbc, struct inode *inode, struct folio *folio, u64 end_pos) { - struct iomap_page *iop = iomap_page_create(inode, folio, 0); + struct iomap_folio *iof = iomap_iof_alloc(inode, folio, 0); struct iomap_ioend *ioend, *next; unsigned len = i_blocksize(inode); unsigned nblocks = i_blocks_per_folio(inode, folio); @@ -1620,7 +1620,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, int error = 0, count = 0, i; LIST_HEAD(submit_list); - WARN_ON_ONCE(iop && atomic_read(&iop->write_bytes_pending) != 0); + WARN_ON_ONCE(iof && atomic_read(&iof->write_bytes_pending) != 0); /* * Walk through the folio to find areas to write back. If we @@ -1628,7 +1628,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, * invalid, grab a new one. */ for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) { - if (iop && !test_bit(i, iop->uptodate)) + if (iof && !test_bit(i, iof->uptodate)) continue; error = wpc->ops->map_blocks(wpc, inode, pos); @@ -1639,7 +1639,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, continue; if (wpc->iomap.type == IOMAP_HOLE) continue; - iomap_add_to_ioend(inode, pos, folio, iop, wpc, wbc, + iomap_add_to_ioend(inode, pos, folio, iof, wpc, wbc, &submit_list); count++; } From patchwork Tue Jun 6 11:43:49 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 13268983 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5703FC7EE24 for ; Tue, 6 Jun 2023 11:44:13 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237224AbjFFLoM (ORCPT ); Tue, 6 Jun 2023 07:44:12 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36858 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237133AbjFFLoJ (ORCPT ); Tue, 6 Jun 2023 07:44:09 -0400 Received: from mail-pg1-x533.google.com (mail-pg1-x533.google.com [IPv6:2607:f8b0:4864:20::533]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66329A6; Tue, 6 Jun 2023 04:44:08 -0700 (PDT) Received: by mail-pg1-x533.google.com with SMTP id 41be03b00d2f7-517ab9a4a13so5348388a12.1; Tue, 06 Jun 2023 04:44:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686051847; x=1688643847; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=yFQmtCO5X8RL2byGnSDVe/dhJz84S0keklInfDm9x90=; b=rz0d6plF7+jgQGcg2gMjYuusf2MwZwFLhm6ip/gPtoqd+QUi9mN55xveBvaWZqrd9b R4rzdmyvU4Lw6mayNw0znbh/4rnCZwrb8lMAZFz0LdP9kRGtnySXHeGUqx43hAwdl1NT 4SyP+XnkmVLyu/Pb92NLfDLQuodsx+KPiuwno1yhN6b7OGPV4vke8sfuwVPdOG5EYYl7 ZqieNWK9jtAToGm4P/HvTqQZMy3KGYHyoH3R+RUdDiWEnQucT0RxP1GQz/7IrVBaGqng pm68G8lAsU7FQ5Lg0CPWl9W5iSQ923FnC0OvBLlJh1hXxLBEgF5rm3+srz7xoSmvRYQX vrOA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686051847; x=1688643847; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=yFQmtCO5X8RL2byGnSDVe/dhJz84S0keklInfDm9x90=; b=Hsrq/ieEPPQBRceWw6LyD858kD6J4TczErbEG2teoIO4ygakiaxwaQf4zzHwOYM5Br IeckHpKou9jOw+IDSQmHBpDk6XNrZg4DKittNzjRfuOiUjiSTFHXh1X0fhSbEtJhXumb IDgneeSDp9ClkLEw0wCYR9myzcQuiV1V2bdo48x/LIQfxPJhe4iOWBvN0+Mv0XfmBVOz 2TWrsQoMrlMZFYSwLQmKhVXpaWoUIuAzBc/vnndKzeZQPxEcxzn8Vj80UaLF7+1PlI4Q 4d5k19PmUEoYlHSeayfMEcZZfXcHYdW7qhYRZVXjQ9pUIv8eXriqNdM3YIoeAlVnEM8R Hz+g== X-Gm-Message-State: AC+VfDz141Shmzheec5gKSgGLwRsohkqPoyhQo63ov1SGrSsB0G8jz0b kscz/W+R6EM+XILlMu+I17MA5nBGNNw= X-Google-Smtp-Source: ACHHUZ4pf3abHm/IXIAKzaT9X/ZQ/3AJ9nt/1TSG0eXzauzrfr0uS1QLTZXC/RluGrfOzXTslITvcw== X-Received: by 2002:a17:902:da87:b0:1b2:2400:f324 with SMTP id j7-20020a170902da8700b001b22400f324mr2133476plx.64.1686051847487; Tue, 06 Jun 2023 04:44:07 -0700 (PDT) Received: from dw-tp.localdomain ([49.207.220.159]) by smtp.gmail.com with ESMTPSA id y4-20020a170902ed4400b001ab0a30c895sm8325120plb.202.2023.06.06.04.44.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Jun 2023 04:44:07 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, "Darrick J. Wong" , Matthew Wilcox , Dave Chinner , Brian Foster , Christoph Hellwig , Andreas Gruenbacher , Ojaswin Mujoo , Disha Goel , "Ritesh Harjani (IBM)" Subject: [PATCHv8 2/5] iomap: Renames and refactor iomap_folio state bitmap handling Date: Tue, 6 Jun 2023 17:13:49 +0530 Message-Id: <54583c1f42a87f19bda5a015e641c8d08fa72071.1686050333.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This patch renames iomap_folio's uptodate bitmap to state bitmap. Also refactors and adds iof->state handling functions for uptodate state. Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Darrick J. Wong --- fs/iomap/buffered-io.c | 107 +++++++++++++++++++++++------------------ 1 file changed, 59 insertions(+), 48 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 741baa10c517..08f2a1cf0a66 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -24,14 +24,14 @@ #define IOEND_BATCH_SIZE 4096 /* - * Structure allocated for each folio when block size < folio size - * to track sub-folio uptodate status and I/O completions. + * Structure allocated for each folio to track per-block uptodate state + * and I/O completions. */ struct iomap_folio { atomic_t read_bytes_pending; atomic_t write_bytes_pending; - spinlock_t uptodate_lock; - unsigned long uptodate[]; + spinlock_t state_lock; + unsigned long state[]; }; static inline struct iomap_folio *iomap_get_iof(struct folio *folio) @@ -43,6 +43,47 @@ static inline struct iomap_folio *iomap_get_iof(struct folio *folio) static struct bio_set iomap_ioend_bioset; +static inline bool iomap_iof_is_fully_uptodate(struct folio *folio, + struct iomap_folio *iof) +{ + struct inode *inode = folio->mapping->host; + + return bitmap_full(iof->state, i_blocks_per_folio(inode, folio)); +} + +static inline bool iomap_iof_is_block_uptodate(struct iomap_folio *iof, + unsigned int block) +{ + return test_bit(block, iof->state); +} + +static void iomap_iof_set_range_uptodate(struct folio *folio, + struct iomap_folio *iof, size_t off, size_t len) +{ + struct inode *inode = folio->mapping->host; + unsigned int first_blk = off >> inode->i_blkbits; + unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; + unsigned int nr_blks = last_blk - first_blk + 1; + unsigned long flags; + + spin_lock_irqsave(&iof->state_lock, flags); + bitmap_set(iof->state, first_blk, nr_blks); + if (iomap_iof_is_fully_uptodate(folio, iof)) + folio_mark_uptodate(folio); + spin_unlock_irqrestore(&iof->state_lock, flags); +} + +static void iomap_set_range_uptodate(struct folio *folio, size_t off, + size_t len) +{ + struct iomap_folio *iof = iomap_get_iof(folio); + + if (iof) + iomap_iof_set_range_uptodate(folio, iof, off, len); + else + folio_mark_uptodate(folio); +} + static struct iomap_folio *iomap_iof_alloc(struct inode *inode, struct folio *folio, unsigned int flags) { @@ -58,12 +99,12 @@ static struct iomap_folio *iomap_iof_alloc(struct inode *inode, else gfp = GFP_NOFS | __GFP_NOFAIL; - iof = kzalloc(struct_size(iof, uptodate, BITS_TO_LONGS(nr_blocks)), + iof = kzalloc(struct_size(iof, state, BITS_TO_LONGS(nr_blocks)), gfp); if (iof) { - spin_lock_init(&iof->uptodate_lock); + spin_lock_init(&iof->state_lock); if (folio_test_uptodate(folio)) - bitmap_fill(iof->uptodate, nr_blocks); + bitmap_fill(iof->state, nr_blocks); folio_attach_private(folio, iof); } return iof; @@ -72,14 +113,12 @@ static struct iomap_folio *iomap_iof_alloc(struct inode *inode, static void iomap_iof_free(struct folio *folio) { struct iomap_folio *iof = folio_detach_private(folio); - struct inode *inode = folio->mapping->host; - unsigned int nr_blocks = i_blocks_per_folio(inode, folio); if (!iof) return; WARN_ON_ONCE(atomic_read(&iof->read_bytes_pending)); WARN_ON_ONCE(atomic_read(&iof->write_bytes_pending)); - WARN_ON_ONCE(bitmap_full(iof->uptodate, nr_blocks) != + WARN_ON_ONCE(iomap_iof_is_fully_uptodate(folio, iof) != folio_test_uptodate(folio)); kfree(iof); } @@ -110,7 +149,7 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio, /* move forward for each leading block marked uptodate */ for (i = first; i <= last; i++) { - if (!test_bit(i, iof->uptodate)) + if (!iomap_iof_is_block_uptodate(iof, i)) break; *pos += block_size; poff += block_size; @@ -120,7 +159,7 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio, /* truncate len if we find any trailing uptodate block(s) */ for ( ; i <= last; i++) { - if (test_bit(i, iof->uptodate)) { + if (iomap_iof_is_block_uptodate(iof, i)) { plen -= (last - i + 1) * block_size; last = i - 1; break; @@ -144,30 +183,6 @@ static void iomap_adjust_read_range(struct inode *inode, struct folio *folio, *lenp = plen; } -static void iomap_iof_set_range_uptodate(struct folio *folio, - struct iomap_folio *iof, size_t off, size_t len) -{ - struct inode *inode = folio->mapping->host; - unsigned first = off >> inode->i_blkbits; - unsigned last = (off + len - 1) >> inode->i_blkbits; - unsigned long flags; - - spin_lock_irqsave(&iof->uptodate_lock, flags); - bitmap_set(iof->uptodate, first, last - first + 1); - if (bitmap_full(iof->uptodate, i_blocks_per_folio(inode, folio))) - folio_mark_uptodate(folio); - spin_unlock_irqrestore(&iof->uptodate_lock, flags); -} - -static void iomap_set_range_uptodate(struct folio *folio, - struct iomap_folio *iof, size_t off, size_t len) -{ - if (iof) - iomap_iof_set_range_uptodate(folio, iof, off, len); - else - folio_mark_uptodate(folio); -} - static void iomap_finish_folio_read(struct folio *folio, size_t offset, size_t len, int error) { @@ -177,7 +192,7 @@ static void iomap_finish_folio_read(struct folio *folio, size_t offset, folio_clear_uptodate(folio); folio_set_error(folio); } else { - iomap_set_range_uptodate(folio, iof, offset, len); + iomap_set_range_uptodate(folio, offset, len); } if (!iof || atomic_sub_and_test(len, &iof->read_bytes_pending)) @@ -213,7 +228,6 @@ struct iomap_readpage_ctx { static int iomap_read_inline_data(const struct iomap_iter *iter, struct folio *folio) { - struct iomap_folio *iof; const struct iomap *iomap = iomap_iter_srcmap(iter); size_t size = i_size_read(iter->inode) - iomap->offset; size_t poff = offset_in_page(iomap->offset); @@ -231,15 +245,13 @@ static int iomap_read_inline_data(const struct iomap_iter *iter, if (WARN_ON_ONCE(size > iomap->length)) return -EIO; if (offset > 0) - iof = iomap_iof_alloc(iter->inode, folio, iter->flags); - else - iof = iomap_get_iof(folio); + iomap_iof_alloc(iter->inode, folio, iter->flags); addr = kmap_local_folio(folio, offset); memcpy(addr, iomap->inline_data, size); memset(addr + size, 0, PAGE_SIZE - poff - size); kunmap_local(addr); - iomap_set_range_uptodate(folio, iof, offset, PAGE_SIZE - poff); + iomap_set_range_uptodate(folio, offset, PAGE_SIZE - poff); return 0; } @@ -276,7 +288,7 @@ static loff_t iomap_readpage_iter(const struct iomap_iter *iter, if (iomap_block_needs_zeroing(iter, pos)) { folio_zero_range(folio, poff, plen); - iomap_set_range_uptodate(folio, iof, poff, plen); + iomap_set_range_uptodate(folio, poff, plen); goto done; } @@ -451,7 +463,7 @@ bool iomap_is_partially_uptodate(struct folio *folio, size_t from, size_t count) last = (from + count - 1) >> inode->i_blkbits; for (i = first; i <= last; i++) - if (!test_bit(i, iof->uptodate)) + if (!iomap_iof_is_block_uptodate(iof, i)) return false; return true; } @@ -589,7 +601,7 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, if (status) return status; } - iomap_set_range_uptodate(folio, iof, poff, plen); + iomap_set_range_uptodate(folio, poff, plen); } while ((block_start += plen) < block_end); return 0; @@ -696,7 +708,6 @@ static int iomap_write_begin(struct iomap_iter *iter, loff_t pos, static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len, size_t copied, struct folio *folio) { - struct iomap_folio *iof = iomap_get_iof(folio); flush_dcache_folio(folio); /* @@ -712,7 +723,7 @@ static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len, */ if (unlikely(copied < len && !folio_test_uptodate(folio))) return 0; - iomap_set_range_uptodate(folio, iof, offset_in_folio(folio, pos), len); + iomap_set_range_uptodate(folio, offset_in_folio(folio, pos), len); filemap_dirty_folio(inode->i_mapping, folio); return copied; } @@ -1628,7 +1639,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, * invalid, grab a new one. */ for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) { - if (iof && !test_bit(i, iof->uptodate)) + if (iof && !iomap_iof_is_block_uptodate(iof, i)) continue; error = wpc->ops->map_blocks(wpc, inode, pos); From patchwork Tue Jun 6 11:43:50 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 13268984 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id DA55EC77B73 for ; Tue, 6 Jun 2023 11:44:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237227AbjFFLoU (ORCPT ); Tue, 6 Jun 2023 07:44:20 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36872 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237133AbjFFLoN (ORCPT ); Tue, 6 Jun 2023 07:44:13 -0400 Received: from mail-pg1-x52f.google.com (mail-pg1-x52f.google.com [IPv6:2607:f8b0:4864:20::52f]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05B89A6; Tue, 6 Jun 2023 04:44:12 -0700 (PDT) Received: by mail-pg1-x52f.google.com with SMTP id 41be03b00d2f7-542c54455e7so1730462a12.1; Tue, 06 Jun 2023 04:44:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686051851; x=1688643851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=pNbebm9Jn9SSTfoHqa42ibdq4MoOmRy6HfaX/BXkaq4=; b=p+JPDG51wIC2CASS5zqyko+6fomX+2lI6FHRpYzfIiZ9MveF+QGNm0LMFu5kKFiuD5 CoeSDDWGrWwDfY48ZNZfJ/N6vu8mrLdLfgN0LBAGZXRa7tRv6iR0ut0NSfU4HASxcCuo pw2Gk7snKwAAWnPYQTliGYIdnhgj4pNjaSpytMk8IJcAtI4qk9qrIwHozBA0w2ZZl9tU UEhphfYUhNJYwNR4ybfIyGH3BbmLtscVpF34XNPlC26lJ7/K+nRUJDdNrSqUF2N5hFFF dDkUcuFUjTNVe5Kgo+21EniPUxfH+fs3/d83b4lv0lMyAkHzlYGxqSxTlhGuf6FWdNBv J6Dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686051851; x=1688643851; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=pNbebm9Jn9SSTfoHqa42ibdq4MoOmRy6HfaX/BXkaq4=; b=a+dUEmTYxUum3Pow3H/Bj+ButILzbPAiRMYi8H3z+tngRY7mI25KMkcGTQYh9GjBvC aqQ1ZKsYfIpJVtb0DnMA5V7NLU/sFSC8tk7xxKzJc7p65pK3Vkw4NKxOOEVI4NX1mJdm zpBmBge1/O2Ro49nO9AF9fumsYPocviE7h6GFyBGEJpUQTba1E0oMZ7Ox4VkUdoGPwe6 OF/DOy9u2mb+Bpf+i5TQ4RYA4g+9xuufQ8jyD+/huf7ClZVtw/ctFlN9CZzbVlrJe/S9 oW/u9ys3EyMZaHjtJmAxXUsO9YxPQo6mwuQ21coEEti2InLoFl4KvsGPyXRnPjRGhyJv pvzw== X-Gm-Message-State: AC+VfDxoriZZv0yYoQQoTYSZcNWkqdKw/EQi+0n48Ja8RjST2KZy8eMZ Mg5Ecu4MB7X4+CbPh+vjHVZ/dNqwK9M= X-Google-Smtp-Source: ACHHUZ7S87oCH5408p8y/uQg2HAQTMqjRciIbtZjPRxmQKZXRGcrxaOO6e6D6LcK1LONqSiLorLr+g== X-Received: by 2002:a17:903:1250:b0:1b0:4680:3f with SMTP id u16-20020a170903125000b001b04680003fmr681164plh.41.1686051850946; Tue, 06 Jun 2023 04:44:10 -0700 (PDT) Received: from dw-tp.localdomain ([49.207.220.159]) by smtp.gmail.com with ESMTPSA id y4-20020a170902ed4400b001ab0a30c895sm8325120plb.202.2023.06.06.04.44.07 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Jun 2023 04:44:10 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, "Darrick J. Wong" , Matthew Wilcox , Dave Chinner , Brian Foster , Christoph Hellwig , Andreas Gruenbacher , Ojaswin Mujoo , Disha Goel , "Ritesh Harjani (IBM)" Subject: [PATCHv8 3/5] iomap: Refactor iomap_write_delalloc_punch() function out Date: Tue, 6 Jun 2023 17:13:50 +0530 Message-Id: X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org This patch moves iomap_write_delalloc_punch() out of iomap_write_delalloc_scan(). No functionality change in this patch. Reviewed-by: Darrick J. Wong Signed-off-by: Ritesh Harjani (IBM) --- fs/iomap/buffered-io.c | 54 ++++++++++++++++++++++++++---------------- 1 file changed, 34 insertions(+), 20 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 08f2a1cf0a66..89489aed49c0 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -888,6 +888,33 @@ iomap_file_buffered_write(struct kiocb *iocb, struct iov_iter *i, } EXPORT_SYMBOL_GPL(iomap_file_buffered_write); +static int iomap_write_delalloc_punch(struct inode *inode, struct folio *folio, + loff_t *punch_start_byte, loff_t start_byte, loff_t end_byte, + int (*punch)(struct inode *inode, loff_t offset, loff_t length)) +{ + int ret = 0; + + if (!folio_test_dirty(folio)) + return ret; + + /* if dirty, punch up to offset */ + if (start_byte > *punch_start_byte) { + ret = punch(inode, *punch_start_byte, + start_byte - *punch_start_byte); + if (ret) + goto out; + } + /* + * Make sure the next punch start is correctly bound to + * the end of this data range, not the end of the folio. + */ + *punch_start_byte = min_t(loff_t, end_byte, + folio_next_index(folio) << PAGE_SHIFT); + +out: + return ret; +} + /* * Scan the data range passed to us for dirty page cache folios. If we find a * dirty folio, punch out the preceeding range and update the offset from which @@ -911,6 +938,7 @@ static int iomap_write_delalloc_scan(struct inode *inode, { while (start_byte < end_byte) { struct folio *folio; + int ret; /* grab locked page */ folio = filemap_lock_folio(inode->i_mapping, @@ -921,26 +949,12 @@ static int iomap_write_delalloc_scan(struct inode *inode, continue; } - /* if dirty, punch up to offset */ - if (folio_test_dirty(folio)) { - if (start_byte > *punch_start_byte) { - int error; - - error = punch(inode, *punch_start_byte, - start_byte - *punch_start_byte); - if (error) { - folio_unlock(folio); - folio_put(folio); - return error; - } - } - - /* - * Make sure the next punch start is correctly bound to - * the end of this data range, not the end of the folio. - */ - *punch_start_byte = min_t(loff_t, end_byte, - folio_next_index(folio) << PAGE_SHIFT); + ret = iomap_write_delalloc_punch(inode, folio, punch_start_byte, + start_byte, end_byte, punch); + if (ret) { + folio_unlock(folio); + folio_put(folio); + return ret; } /* move offset to start of next folio in range */ From patchwork Tue Jun 6 11:43:51 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 13268985 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id BC4FFC7EE31 for ; Tue, 6 Jun 2023 11:44:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237275AbjFFLoW (ORCPT ); Tue, 6 Jun 2023 07:44:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36884 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237239AbjFFLoQ (ORCPT ); Tue, 6 Jun 2023 07:44:16 -0400 Received: from mail-pl1-x635.google.com (mail-pl1-x635.google.com [IPv6:2607:f8b0:4864:20::635]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66D11E43; Tue, 6 Jun 2023 04:44:15 -0700 (PDT) Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1b04949e5baso52663075ad.0; Tue, 06 Jun 2023 04:44:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686051854; x=1688643854; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Jab6p16he4cD48oIvWzgzNyAp3uY3B9r8JeAYXvFNdU=; b=mRxlRwQNPBbo1voVjBBfQJiVIxXNhvT8Va7UHos2fw8K+YftV+mIx8aLrgkUuclPL0 ZAZuIbfh3vSHBhGdl6k0tjp/6RmpgwbPbuMJ8f5dR3/x6MG7cfWkIWFass7qrmCkdeGJ ojuHjexj/J8XFo4x0yIjRmepHxTbN/+obr8DyI3KXo/OntqKPzsSE4hoN9wYwIoYL2W6 gZeykuz6aULAuSfwSYp/iMULgeB8dt3E9oF+PxBkP64l4qQKiffGok/DlDaivAiAakgQ jKP57I6mXm2gT1LcMuA5fmukt8KKv+MiyP12IPv4CznTLiGccnxieHjpXtAHtQFyP+rX 2XRg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686051854; x=1688643854; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Jab6p16he4cD48oIvWzgzNyAp3uY3B9r8JeAYXvFNdU=; b=kHHNfloFQK1WcbCo/O6bbdC1hlgLUGTbKA0nkix1wIvfOoPJIVqPqQadgGcuFvUvUf cEHBQvY5zEmHva8gHWy1Op24ldcRCx2fU72qp4hVvGos47RCBfWYgRKlN+DmrHm1ejtL gRGbjUF/e28i0dqF7eWCvAjFOKv1m2Jbz4XqzB6zn5M7c9A85gD9bawsOKIPqzXrDkB1 SW8BnliSpn5iFxhCc1QdIF5q8cSc17fQJiJNSFNxCqo+zfUaVvK25Z1jbNqOQ6kE0S+j 7UrSwmAyciMt6TMXg31wi2Tf8pYgfb040uQRxDIyOQm18JoYT12eiYsJPSLGqfQFMlBx z8sg== X-Gm-Message-State: AC+VfDwQX4JtrqJ/zOP0B07rvU7mUPxx95iUjVGxMLJGgVURVc/TGmNe 6vIqC3g5KdhiSJNOX+LYFLiznnfwJN4= X-Google-Smtp-Source: ACHHUZ4o6Op8f8+0vzTG08WPlvELUkJ7ruOMzC2+kkA4hdaAT1QeGay+u/SdK3qcG6qZIW3PGMrOag== X-Received: by 2002:a17:902:ce86:b0:1aa:d545:462e with SMTP id f6-20020a170902ce8600b001aad545462emr2268474plg.13.1686051854535; Tue, 06 Jun 2023 04:44:14 -0700 (PDT) Received: from dw-tp.localdomain ([49.207.220.159]) by smtp.gmail.com with ESMTPSA id y4-20020a170902ed4400b001ab0a30c895sm8325120plb.202.2023.06.06.04.44.11 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Jun 2023 04:44:14 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, "Darrick J. Wong" , Matthew Wilcox , Dave Chinner , Brian Foster , Christoph Hellwig , Andreas Gruenbacher , Ojaswin Mujoo , Disha Goel , "Ritesh Harjani (IBM)" Subject: [PATCHv8 4/5] iomap: Allocate iof in ->write_begin() early Date: Tue, 6 Jun 2023 17:13:51 +0530 Message-Id: <1161fe2bb007361ae47d509e588e7f5b3b819208.1686050333.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org We dont need to allocate an iof in ->write_begin() for writes where the position and length completely overlap with the given folio. Therefore, such cases are skipped. Currently when the folio is uptodate, we only allocate iof at writeback time (in iomap_writepage_map()). This is ok until now, but when we are going to add support for per-block dirty state bitmap in iof, this could cause some performance degradation. The reason is that if we don't allocate iof during ->write_begin(), then we will never mark the necessary dirty bits in ->write_end() call. And we will have to mark all the bits as dirty at the writeback time, that could cause the same write amplification and performance problems as it is now. Reviewed-by: Darrick J. Wong Signed-off-by: Ritesh Harjani (IBM) Reviewed-by: Christoph Hellwig --- fs/iomap/buffered-io.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 89489aed49c0..2b72ca3ba37a 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -567,14 +567,23 @@ static int __iomap_write_begin(const struct iomap_iter *iter, loff_t pos, size_t from = offset_in_folio(folio, pos), to = from + len; size_t poff, plen; - if (folio_test_uptodate(folio)) + /* + * If the write completely overlaps the current folio, then + * entire folio will be dirtied so there is no need for + * per-block state tracking structures to be attached to this folio. + */ + if (pos <= folio_pos(folio) && + pos + len >= folio_pos(folio) + folio_size(folio)) return 0; - folio_clear_error(folio); iof = iomap_iof_alloc(iter->inode, folio, iter->flags); if ((iter->flags & IOMAP_NOWAIT) && !iof && nr_blocks > 1) return -EAGAIN; + if (folio_test_uptodate(folio)) + return 0; + folio_clear_error(folio); + do { iomap_adjust_read_range(iter->inode, folio, &block_start, block_end - block_start, &poff, &plen); From patchwork Tue Jun 6 11:43:52 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Ritesh Harjani (IBM)" X-Patchwork-Id: 13268986 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 603C6C77B73 for ; Tue, 6 Jun 2023 11:44:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236543AbjFFLoX (ORCPT ); Tue, 6 Jun 2023 07:44:23 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36906 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237257AbjFFLoV (ORCPT ); Tue, 6 Jun 2023 07:44:21 -0400 Received: from mail-pg1-x52b.google.com (mail-pg1-x52b.google.com [IPv6:2607:f8b0:4864:20::52b]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 385FCE79; Tue, 6 Jun 2023 04:44:19 -0700 (PDT) Received: by mail-pg1-x52b.google.com with SMTP id 41be03b00d2f7-53404873a19so3085286a12.3; Tue, 06 Jun 2023 04:44:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686051858; x=1688643858; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=DXbyJcIaKkUDRf0vI9YCLPzJeo6JL8VnUjH/ZbPUhfw=; b=dv1EpAlWLib75+8YTlor3+RU43sar+/zaZs2Wq5JGSXAThexLpkkuF7Mc1iCvQo+pr S8yt/qGlKyD0aQt7J1PuYDVsSiJ5cUwK+jYBUVJyAw4aXCoVKVPS2TzhSwFR2HeqVoUz Njz+s+5EeZt+I4pFe3QSmcudFgctRUPdCAvA+jJG3ij+JIFEX+FogsBFfAarrGuD6wet Brsb0GdT0kF9Q3Z9IxRn+QYtrYL8PSrLwb+ZB3fs+LRZ/JjegkheC7R1Hs4DpMM21Ain xO53OObj+CHz7p+Q/wMm0rla0N7lmYxbJidtqG41I0wPJB7OeasFll7sjt5axi0p//3h NP2w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686051858; x=1688643858; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=DXbyJcIaKkUDRf0vI9YCLPzJeo6JL8VnUjH/ZbPUhfw=; b=AWIC338cQAFNRr9bMgEp1fZ6EGLjTKnQMSro2jpyYEnU0DvtJDYpUzbAWal/ecbO5x IcrglMGPNLBYAy+OPZFJVyMT/8LsoB3xV+KBSMsjzQHYEBkwGhbbNmL0rF4E9jG6A7ag WCMenrZcgw3j/sflogGsVIVf0eEmnI0XkUgIm4PHJc7dAx/HepOeocftgTNX7ITZPzG/ PspxoevOttTL+EHZ6KWlUTipbBPWXE0cHzkPbSns9sieiPrBTlaax0Ukl4xoT8zvDdfa rLXEWThRhYiLZP1NatwTE8rjr6N5fcijXX2lcOj7d0n9gcMrdfle6c11vpvhnyGgCZ0X hYiA== X-Gm-Message-State: AC+VfDxtyK2Ix+zHDV1YZuUg7/Z5MFPqvu4Rs26qglus3jA5ArYOycir 3QzshN7pNUplQEcjVuoZLR3uZuT1HeY= X-Google-Smtp-Source: ACHHUZ5KFY0zgAZpZL6DCI4R4EhRWQGLt9O8RqC3pUKm7G8eE8B13tOZ1fEdCXqLJUluL1Nq5YKimA== X-Received: by 2002:a05:6a20:8427:b0:117:1c52:ba7c with SMTP id c39-20020a056a20842700b001171c52ba7cmr1334180pzd.6.1686051858230; Tue, 06 Jun 2023 04:44:18 -0700 (PDT) Received: from dw-tp.localdomain ([49.207.220.159]) by smtp.gmail.com with ESMTPSA id y4-20020a170902ed4400b001ab0a30c895sm8325120plb.202.2023.06.06.04.44.14 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Jun 2023 04:44:17 -0700 (PDT) From: "Ritesh Harjani (IBM)" To: linux-xfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org, "Darrick J. Wong" , Matthew Wilcox , Dave Chinner , Brian Foster , Christoph Hellwig , Andreas Gruenbacher , Ojaswin Mujoo , Disha Goel , "Ritesh Harjani (IBM)" , Aravinda Herle Subject: [PATCHv8 5/5] iomap: Add per-block dirty state tracking to improve performance Date: Tue, 6 Jun 2023 17:13:52 +0530 Message-Id: <0c3108f6ea45e18c7aae87c2fb3d8fa3311c13a4.1686050333.git.ritesh.list@gmail.com> X-Mailer: git-send-email 2.40.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org When filesystem blocksize is less than folio size (either with mapping_large_folio_support() or with blocksize < pagesize) and when the folio is uptodate in pagecache, then even a byte write can cause an entire folio to be written to disk during writeback. This happens because we currently don't have a mechanism to track per-block dirty state within struct iomap_folio. We currently only track uptodate state. This patch implements support for tracking per-block dirty state in iomap_folio->state bitmap. This should help improve the filesystem write performance and help reduce write amplification. Performance testing of below fio workload reveals ~16x performance improvement using nvme with XFS (4k blocksize) on Power (64K pagesize) FIO reported write bw scores improved from around ~28 MBps to ~452 MBps. 1. [global] ioengine=psync rw=randwrite overwrite=1 pre_read=1 direct=0 bs=4k size=1G dir=./ numjobs=8 fdatasync=1 runtime=60 iodepth=64 group_reporting=1 [fio-run] 2. Also our internal performance team reported that this patch improves their database workload performance by around ~83% (with XFS on Power) Reported-by: Aravinda Herle Reported-by: Brian Foster Signed-off-by: Ritesh Harjani (IBM) --- fs/gfs2/aops.c | 2 +- fs/iomap/buffered-io.c | 126 +++++++++++++++++++++++++++++++++++++++-- fs/xfs/xfs_aops.c | 2 +- fs/zonefs/file.c | 2 +- include/linux/iomap.h | 1 + 5 files changed, 126 insertions(+), 7 deletions(-) diff --git a/fs/gfs2/aops.c b/fs/gfs2/aops.c index a5f4be6b9213..75efec3c3b71 100644 --- a/fs/gfs2/aops.c +++ b/fs/gfs2/aops.c @@ -746,7 +746,7 @@ static const struct address_space_operations gfs2_aops = { .writepages = gfs2_writepages, .read_folio = gfs2_read_folio, .readahead = gfs2_readahead, - .dirty_folio = filemap_dirty_folio, + .dirty_folio = iomap_dirty_folio, .release_folio = iomap_release_folio, .invalidate_folio = iomap_invalidate_folio, .bmap = gfs2_bmap, diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 2b72ca3ba37a..b99d611f1cd5 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -84,6 +84,70 @@ static void iomap_set_range_uptodate(struct folio *folio, size_t off, folio_mark_uptodate(folio); } +static inline bool iomap_iof_is_block_dirty(struct folio *folio, + struct iomap_folio *iof, int block) +{ + struct inode *inode = folio->mapping->host; + unsigned int blks_per_folio = i_blocks_per_folio(inode, folio); + + return test_bit(block + blks_per_folio, iof->state); +} + +static void iomap_iof_clear_range_dirty(struct folio *folio, + struct iomap_folio *iof, size_t off, size_t len) +{ + struct inode *inode = folio->mapping->host; + unsigned int blks_per_folio = i_blocks_per_folio(inode, folio); + unsigned int first_blk = off >> inode->i_blkbits; + unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; + unsigned int nr_blks = last_blk - first_blk + 1; + unsigned long flags; + + spin_lock_irqsave(&iof->state_lock, flags); + bitmap_clear(iof->state, first_blk + blks_per_folio, nr_blks); + spin_unlock_irqrestore(&iof->state_lock, flags); +} + +static void iomap_clear_range_dirty(struct folio *folio, size_t off, size_t len) +{ + struct iomap_folio *iof = iomap_get_iof(folio); + + if (!iof) + return; + iomap_iof_clear_range_dirty(folio, iof, off, len); +} + +static void iomap_iof_set_range_dirty(struct inode *inode, struct folio *folio, + struct iomap_folio *iof, size_t off, size_t len) +{ + unsigned int blks_per_folio = i_blocks_per_folio(inode, folio); + unsigned int first_blk = off >> inode->i_blkbits; + unsigned int last_blk = (off + len - 1) >> inode->i_blkbits; + unsigned int nr_blks = last_blk - first_blk + 1; + unsigned long flags; + + spin_lock_irqsave(&iof->state_lock, flags); + bitmap_set(iof->state, first_blk + blks_per_folio, nr_blks); + spin_unlock_irqrestore(&iof->state_lock, flags); +} + +/* + * The reason we pass inode explicitly here is to avoid any race with truncate + * when iomap_set_range_dirty() gets called from iomap_dirty_folio(). + * Check filemap_dirty_folio() & __folio_mark_dirty() for more details. + * Hence we explicitly pass mapping->host to iomap_set_range_dirty() from + * iomap_dirty_folio(). + */ +static void iomap_set_range_dirty(struct inode *inode, struct folio *folio, + size_t off, size_t len) +{ + struct iomap_folio *iof = iomap_get_iof(folio); + + if (!iof) + return; + iomap_iof_set_range_dirty(inode, folio, iof, off, len); +} + static struct iomap_folio *iomap_iof_alloc(struct inode *inode, struct folio *folio, unsigned int flags) { @@ -99,12 +163,20 @@ static struct iomap_folio *iomap_iof_alloc(struct inode *inode, else gfp = GFP_NOFS | __GFP_NOFAIL; - iof = kzalloc(struct_size(iof, state, BITS_TO_LONGS(nr_blocks)), + /* + * iof->state tracks two sets of state flags when the + * filesystem block size is smaller than the folio size. + * The first state tracks per-block uptodate and the + * second tracks per-block dirty state. + */ + iof = kzalloc(struct_size(iof, state, BITS_TO_LONGS(2 * nr_blocks)), gfp); if (iof) { spin_lock_init(&iof->state_lock); if (folio_test_uptodate(folio)) - bitmap_fill(iof->state, nr_blocks); + bitmap_set(iof->state, 0, nr_blocks); + if (folio_test_dirty(folio)) + bitmap_set(iof->state, nr_blocks, nr_blocks); folio_attach_private(folio, iof); } return iof; @@ -529,6 +601,17 @@ void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len) } EXPORT_SYMBOL_GPL(iomap_invalidate_folio); +bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio) +{ + struct inode *inode = mapping->host; + size_t len = folio_size(folio); + + iomap_iof_alloc(inode, folio, 0); + iomap_set_range_dirty(inode, folio, 0, len); + return filemap_dirty_folio(mapping, folio); +} +EXPORT_SYMBOL_GPL(iomap_dirty_folio); + static void iomap_write_failed(struct inode *inode, loff_t pos, unsigned len) { @@ -733,6 +816,8 @@ static size_t __iomap_write_end(struct inode *inode, loff_t pos, size_t len, if (unlikely(copied < len && !folio_test_uptodate(folio))) return 0; iomap_set_range_uptodate(folio, offset_in_folio(folio, pos), len); + iomap_set_range_dirty(inode, folio, offset_in_folio(folio, pos), + copied); filemap_dirty_folio(inode->i_mapping, folio); return copied; } @@ -902,6 +987,10 @@ static int iomap_write_delalloc_punch(struct inode *inode, struct folio *folio, int (*punch)(struct inode *inode, loff_t offset, loff_t length)) { int ret = 0; + struct iomap_folio *iof; + unsigned int first_blk, last_blk, i; + loff_t last_byte; + u8 blkbits = inode->i_blkbits; if (!folio_test_dirty(folio)) return ret; @@ -913,6 +1002,29 @@ static int iomap_write_delalloc_punch(struct inode *inode, struct folio *folio, if (ret) goto out; } + /* + * When we have per-block dirty tracking, there can be + * blocks within a folio which are marked uptodate + * but not dirty. In that case it is necessary to punch + * out such blocks to avoid leaking any delalloc blocks. + */ + iof = iomap_get_iof(folio); + if (!iof) + goto skip_iof_punch; + + last_byte = min_t(loff_t, end_byte - 1, + (folio_next_index(folio) << PAGE_SHIFT) - 1); + first_blk = offset_in_folio(folio, start_byte) >> blkbits; + last_blk = offset_in_folio(folio, last_byte) >> blkbits; + for (i = first_blk; i <= last_blk; i++) { + if (!iomap_iof_is_block_dirty(folio, iof, i)) { + ret = punch(inode, i << blkbits, 1 << blkbits); + if (ret) + goto out; + } + } + +skip_iof_punch: /* * Make sure the next punch start is correctly bound to * the end of this data range, not the end of the folio. @@ -1646,7 +1758,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, struct writeback_control *wbc, struct inode *inode, struct folio *folio, u64 end_pos) { - struct iomap_folio *iof = iomap_iof_alloc(inode, folio, 0); + struct iomap_folio *iof = iomap_get_iof(folio); struct iomap_ioend *ioend, *next; unsigned len = i_blocksize(inode); unsigned nblocks = i_blocks_per_folio(inode, folio); @@ -1654,6 +1766,11 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, int error = 0, count = 0, i; LIST_HEAD(submit_list); + if (!iof && nblocks > 1) { + iof = iomap_iof_alloc(inode, folio, 0); + iomap_set_range_dirty(inode, folio, 0, folio_size(folio)); + } + WARN_ON_ONCE(iof && atomic_read(&iof->write_bytes_pending) != 0); /* @@ -1662,7 +1779,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, * invalid, grab a new one. */ for (i = 0; i < nblocks && pos < end_pos; i++, pos += len) { - if (iof && !iomap_iof_is_block_uptodate(iof, i)) + if (iof && !iomap_iof_is_block_dirty(folio, iof, i)) continue; error = wpc->ops->map_blocks(wpc, inode, pos); @@ -1706,6 +1823,7 @@ iomap_writepage_map(struct iomap_writepage_ctx *wpc, } } + iomap_clear_range_dirty(folio, 0, end_pos - folio_pos(folio)); folio_start_writeback(folio); folio_unlock(folio); diff --git a/fs/xfs/xfs_aops.c b/fs/xfs/xfs_aops.c index 2ef78aa1d3f6..77c7332ae197 100644 --- a/fs/xfs/xfs_aops.c +++ b/fs/xfs/xfs_aops.c @@ -578,7 +578,7 @@ const struct address_space_operations xfs_address_space_operations = { .read_folio = xfs_vm_read_folio, .readahead = xfs_vm_readahead, .writepages = xfs_vm_writepages, - .dirty_folio = filemap_dirty_folio, + .dirty_folio = iomap_dirty_folio, .release_folio = iomap_release_folio, .invalidate_folio = iomap_invalidate_folio, .bmap = xfs_vm_bmap, diff --git a/fs/zonefs/file.c b/fs/zonefs/file.c index 132f01d3461f..e508c8e97372 100644 --- a/fs/zonefs/file.c +++ b/fs/zonefs/file.c @@ -175,7 +175,7 @@ const struct address_space_operations zonefs_file_aops = { .read_folio = zonefs_read_folio, .readahead = zonefs_readahead, .writepages = zonefs_writepages, - .dirty_folio = filemap_dirty_folio, + .dirty_folio = iomap_dirty_folio, .release_folio = iomap_release_folio, .invalidate_folio = iomap_invalidate_folio, .migrate_folio = filemap_migrate_folio, diff --git a/include/linux/iomap.h b/include/linux/iomap.h index e2b836c2e119..eb9335c46bf3 100644 --- a/include/linux/iomap.h +++ b/include/linux/iomap.h @@ -264,6 +264,7 @@ bool iomap_is_partially_uptodate(struct folio *, size_t from, size_t count); struct folio *iomap_get_folio(struct iomap_iter *iter, loff_t pos); bool iomap_release_folio(struct folio *folio, gfp_t gfp_flags); void iomap_invalidate_folio(struct folio *folio, size_t offset, size_t len); +bool iomap_dirty_folio(struct address_space *mapping, struct folio *folio); int iomap_file_unshare(struct inode *inode, loff_t pos, loff_t len, const struct iomap_ops *ops); int iomap_zero_range(struct inode *inode, loff_t pos, loff_t len,