[v3,0/9] btrfs: remove ASSERT()s for folio_order() and folio_test_large()

Message

Qu Wenruo March 17, 2025, 7:10 a.m. UTC

[CHANGELOG]
v3:
- Prepare btrfs_end_repair_bio() to support larger folios
  Unfortunately folio_iter structure is way too large compared to
  bvec_iter, thus it's not a good idea to convert bbio::saved_iter to
  folio_iter.

  Thankfully it's not that complex to grab the folio from a bio_vec.

- Add a new patch to prepare defrag for larger data folios

v2:
- Update the commit message for the first patch

There are quite some ASSERT()s checking folio_order() or
folio_test_large() on data folios.

They are the blockage for the incoming larger data folios.

This series will remove most of them, with the following exceptions
which are not a blockage for larger data folios.

- wait_dev_supers() and write_dev_supers()
  They are folios from block device cache, will not be affected by our
  larger data folios support.

- relocation_one_folio()
  It's about data relocation inode, and we can disable larger data folios
  for such special inode easily.

- btrfs_attach_subpage()
  The folio_test_large() is only for metadata, just change it to be a
  metadata specific check.

The first patch is just a small cleanup for send, exposed during the
removal of ASSERT()s.

The conversion for defrag is only tested to not break the existing
subpage and regular cases.
The full tests can only be done with larger data folios support enabled.

Although there is one real blockage before enabling larger data folios:

- btrfs_buffered_write()
  Currnetly we always reserved space first, then grab the folio.
  This makes it impossible to support variable sized folios.

  We have to align the behavior to iomap, which grab the folio first
  then determine the reserved space according to the folio size.

  This will touch the core function of btrfs and can not be hidden
  behind experimental flags.
  Thus that change will be a dedicated one and needs a lot of
  reviews/tests.

Qu Wenruo (9):
  btrfs: send: remove the again label inside put_file_data()
  btrfs: send: prepare put_file_data() for larger data folios
  btrfs: prepare btrfs_page_mkwrite() for larger data folios
  btrfs: prepare prepare_one_folio() for larger data folios
  btrfs: prepare end_bbio_data_write() for larger data folios
  btrfs: subpage: prepare for larger data folios
  btrfs: zlib: prepare copy_data_into_buffer() for larger data folios
  btrfs: prepare btrfs_end_repair_bio() for larger data folios
  btrfs: enable larger data folios support for defrag

 fs/btrfs/bio.c       | 28 ++++++++++++-----
 fs/btrfs/defrag.c    | 72 ++++++++++++++++++++++++++------------------
 fs/btrfs/extent_io.c |  3 --
 fs/btrfs/file.c      |  6 +---
 fs/btrfs/send.c      | 29 ++++++++----------
 fs/btrfs/subpage.c   |  6 ++--
 fs/btrfs/zlib.c      |  2 --
 7 files changed, 79 insertions(+), 67 deletions(-)

Comments

David Sterba March 17, 2025, 1:55 p.m. UTC | #1

On Mon, Mar 17, 2025 at 05:40:45PM +1030, Qu Wenruo wrote:
> [CHANGELOG]
> v3:
> - Prepare btrfs_end_repair_bio() to support larger folios
>   Unfortunately folio_iter structure is way too large compared to
>   bvec_iter, thus it's not a good idea to convert bbio::saved_iter to
>   folio_iter.
> 
>   Thankfully it's not that complex to grab the folio from a bio_vec.
> 
> - Add a new patch to prepare defrag for larger data folios

I was not expecting v3 as the series was in for-next so I did some edits
to changelogs, namely changing 'larger folios' -> 'large folios' as this
is how it's called in MM. Although technically the folios are larger I'd
like to keep using the same terminology.

There are new patches so feel free to replace the whole series, I'm
going to do a pass over the whole branch anyway so will fix anything
thats left. Right now it's the last chance to get the patches to 6.15 so
I don't want delay it on your side.

David Sterba March 17, 2025, 3:13 p.m. UTC | #2

On Mon, Mar 17, 2025 at 02:55:02PM +0100, David Sterba wrote:
> On Mon, Mar 17, 2025 at 05:40:45PM +1030, Qu Wenruo wrote:
> > [CHANGELOG]
> > v3:
> > - Prepare btrfs_end_repair_bio() to support larger folios
> >   Unfortunately folio_iter structure is way too large compared to
> >   bvec_iter, thus it's not a good idea to convert bbio::saved_iter to
> >   folio_iter.
> > 
> >   Thankfully it's not that complex to grab the folio from a bio_vec.
> > 
> > - Add a new patch to prepare defrag for larger data folios
> 
> I was not expecting v3 as the series was in for-next so I did some edits
[...]

Scratch that, this is not the same series.

Qu Wenruo March 31, 2025, 4:47 a.m. UTC | #3

在 2025/3/18 01:43, David Sterba 写道:
> On Mon, Mar 17, 2025 at 02:55:02PM +0100, David Sterba wrote:
>> On Mon, Mar 17, 2025 at 05:40:45PM +1030, Qu Wenruo wrote:
>>> [CHANGELOG]
>>> v3:
>>> - Prepare btrfs_end_repair_bio() to support larger folios
>>>    Unfortunately folio_iter structure is way too large compared to
>>>    bvec_iter, thus it's not a good idea to convert bbio::saved_iter to
>>>    folio_iter.
>>>
>>>    Thankfully it's not that complex to grab the folio from a bio_vec.
>>>
>>> - Add a new patch to prepare defrag for larger data folios
>>
>> I was not expecting v3 as the series was in for-next so I did some edits
> [...]
>
> Scratch that, this is not the same series.
>

BTW, any extra commends needs to be addressed? (E.g. should I merge them
all into a single patch?)

I notice several small comment conflicts ("larger folio -> large
folio"), but is very easy to resolve (local branch updated).

There is a newer series that is already mostly reviewed:

https://lore.kernel.org/linux-btrfs/[e-mail]/

That relies on this series, and I'm already doing some basic (fsstress
runs) large folio tests now.

So I'm wondering can I push the series now, or should I continue waiting
for extra reviews?

Thanks,
Qu

David Sterba April 3, 2025, 7:54 p.m. UTC | #4

On Mon, Mar 31, 2025 at 03:17:35PM +1030, Qu Wenruo wrote:
> 
> 
> 在 2025/3/18 01:43, David Sterba 写道:
> > On Mon, Mar 17, 2025 at 02:55:02PM +0100, David Sterba wrote:
> >> On Mon, Mar 17, 2025 at 05:40:45PM +1030, Qu Wenruo wrote:
> >>> [CHANGELOG]
> >>> v3:
> >>> - Prepare btrfs_end_repair_bio() to support larger folios
> >>>    Unfortunately folio_iter structure is way too large compared to
> >>>    bvec_iter, thus it's not a good idea to convert bbio::saved_iter to
> >>>    folio_iter.
> >>>
> >>>    Thankfully it's not that complex to grab the folio from a bio_vec.
> >>>
> >>> - Add a new patch to prepare defrag for larger data folios
> >>
> >> I was not expecting v3 as the series was in for-next so I did some edits
> > [...]
> >
> > Scratch that, this is not the same series.
> >
> 
> BTW, any extra commends needs to be addressed? (E.g. should I merge them
> all into a single patch?)

I think the patch separation is good, please leave it as it is.

> I notice several small comment conflicts ("larger folio -> large
> folio"), but is very easy to resolve (local branch updated).
> 
> There is a newer series that is already mostly reviewed:
> 
> https://lore.kernel.org/linux-btrfs/[e-mail]/
> 
> That relies on this series, and I'm already doing some basic (fsstress
> runs) large folio tests now.
> 
> So I'm wondering can I push the series now, or should I continue waiting
> for extra reviews?

Please add it to for-next. I did only a light review, we'll find more
things during testing.

Qu Wenruo April 3, 2025, 9:45 p.m. UTC | #5

在 2025/4/4 06:24, David Sterba 写道:
> On Mon, Mar 31, 2025 at 03:17:35PM +1030, Qu Wenruo wrote:
>>
>>
>> 在 2025/3/18 01:43, David Sterba 写道:
>>> On Mon, Mar 17, 2025 at 02:55:02PM +0100, David Sterba wrote:
>>>> On Mon, Mar 17, 2025 at 05:40:45PM +1030, Qu Wenruo wrote:
>>>>> [CHANGELOG]
>>>>> v3:
>>>>> - Prepare btrfs_end_repair_bio() to support larger folios
>>>>>     Unfortunately folio_iter structure is way too large compared to
>>>>>     bvec_iter, thus it's not a good idea to convert bbio::saved_iter to
>>>>>     folio_iter.
>>>>>
>>>>>     Thankfully it's not that complex to grab the folio from a bio_vec.
>>>>>
>>>>> - Add a new patch to prepare defrag for larger data folios
>>>>
>>>> I was not expecting v3 as the series was in for-next so I did some edits
>>> [...]
>>>
>>> Scratch that, this is not the same series.
>>>
>>
>> BTW, any extra commends needs to be addressed? (E.g. should I merge them
>> all into a single patch?)
> 
> I think the patch separation is good, please leave it as it is.
> 
>> I notice several small comment conflicts ("larger folio -> large
>> folio"), but is very easy to resolve (local branch updated).
>>
>> There is a newer series that is already mostly reviewed:
>>
>> https://lore.kernel.org/linux-btrfs/[e-mail]/
>>
>> That relies on this series, and I'm already doing some basic (fsstress
>> runs) large folio tests now.
>>
>> So I'm wondering can I push the series now, or should I continue waiting
>> for extra reviews?
> 
> Please add it to for-next. I did only a light review, we'll find more
> things during testing.

Thanks, but it looks like the read repair and defrag part (the last two 
patches) should be delayed a little.

I'll push the first 7 safe ASSERT() removals into for-next first.


As I finally fixed the last bug exposed by fsstress runs, I can continue 
with fstests runs.

Instead of the untested defrag and read-repair patches, I can do proper 
test and fix (since the read-repair one seems to cause bugs during my 
local tests) bugs in them.

Thanks,
Qu