mbox series

[v4,0/6] iomap: lift the xfs writepage code into iomap

Message ID 156444945993.2682261.3926017251626679029.stgit@magnolia (mailing list archive)
Headers show
Series iomap: lift the xfs writepage code into iomap | expand

Message

Darrick J. Wong July 30, 2019, 1:17 a.m. UTC
Hi all,

>From Christoph:

This series cleans up the xfs writepage code and then lifts it to
fs/iomap/ so that it could be use by other file systems.  I've been
wanting to [do] this for a while so that I could eventually convert gfs2
over to it, but I never got to it.  Now Damien has a new zonefs file
system for semi-raw access to zoned block devices that would like to use
the iomap code instead of reinventing it, so I finally had to do the
work.

>From Darrick:

For v4, split the series into smaller pieces.  This first part builds
out the new iomap writeback infrastructure needed for gfs2 and zonedfs.
The second part will refactor some of XFS's writeback code to use the
new helpers introduced in the first part; and the third part converts
XFS to use the iomap writeback code.

Changes since v2:
 - rebased to v5.3-rc1
 - folded in a few changes from the gfs2 enablement series

Changes since v1:
 - rebased to the latest xfs for-next tree
 - keep the preallocated transactions for size updates
 - rename list_pop to list_pop_entry and related cleanups
 - better document the nofs context handling
 - document that the iomap tracepoints are not a stable API

If you're going to start using this mess, you probably ought to just
pull from my git trees, which are linked below.

This has been lightly tested with fstests.  Enjoy!
Comments and questions are, as always, welcome.

--D

kernel git tree:
https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git/log/?h=iomap-writeback

Comments

Christoph Hellwig July 30, 2019, 2:48 p.m. UTC | #1
I don't really see the point of the split, but the result looks fine
to me.
Andreas Gruenbacher Aug. 5, 2019, 12:34 p.m. UTC | #2
On Tue, 30 Jul 2019 at 16:48, Christoph Hellwig <hch@infradead.org> wrote:
> I don't really see the point of the split, but the result looks fine
> to me.

The split made it easier for me to debug the gfs2 side by backing out
the xfs changes.

Thanks,
Andreas
Christoph Hellwig Aug. 16, 2019, 6:52 a.m. UTC | #3
Darrick,

are you going to queue this up?
Darrick J. Wong Aug. 17, 2019, 1:46 a.m. UTC | #4
On Thu, Aug 15, 2019 at 11:52:29PM -0700, Christoph Hellwig wrote:
> Darrick,
> 
> are you going to queue this up?

Yes, I'll go promote the iomap writeback branch to iomap-for-next.  I
haven't 100% convinced myself that it's a good idea to hook up xfs to it
yet, if nothing else because of all the other problems I've had getting
5.3 testing to run to completion reliably...

...I assume gfs2 and zonedfs are getting ready to send stuff based on
that branch?

--D
Andreas Gruenbacher Aug. 17, 2019, 8:25 a.m. UTC | #5
On Sat, Aug 17, 2019 at 3:46 AM Darrick J. Wong <darrick.wong@oracle.com> wrote:
>
> On Thu, Aug 15, 2019 at 11:52:29PM -0700, Christoph Hellwig wrote:
> > Darrick,
> >
> > are you going to queue this up?
>
> Yes, I'll go promote the iomap writeback branch to iomap-for-next.  I
> haven't 100% convinced myself that it's a good idea to hook up xfs to it
> yet, if nothing else because of all the other problems I've had getting
> 5.3 testing to run to completion reliably...
>
> ...I assume gfs2 and zonedfs are getting ready to send stuff based on
> that branch?

It doesn't look like the gfs2 bits are going to be ready for 5.4.

Andreas
Damien Le Moal Aug. 17, 2019, 1:15 p.m. UTC | #6
On 2019/08/16 18:47, Darrick J. Wong wrote:
> On Thu, Aug 15, 2019 at 11:52:29PM -0700, Christoph Hellwig wrote:
>> Darrick,
>>
>> are you going to queue this up?
> 
> Yes, I'll go promote the iomap writeback branch to iomap-for-next.  I
> haven't 100% convinced myself that it's a good idea to hook up xfs to it
> yet, if nothing else because of all the other problems I've had getting
> 5.3 testing to run to completion reliably...
> 
> ...I assume gfs2 and zonedfs are getting ready to send stuff based on
> that branch?
> 
> --D
> 

I have zonefs rebased on your branch and good to go. But before posting, I am
finishing a test suite for it as xfstests does not work, for obvious reasons :)
I will post a V2 next week.

As zonefs is not an update but a new fs, it needs to be accepted first. If it
is, it will need your branch.

Best regards.
Christoph Hellwig Aug. 20, 2019, 7:30 a.m. UTC | #7
On Fri, Aug 16, 2019 at 06:46:33PM -0700, Darrick J. Wong wrote:
> On Thu, Aug 15, 2019 at 11:52:29PM -0700, Christoph Hellwig wrote:
> > Darrick,
> > 
> > are you going to queue this up?
> 
> Yes, I'll go promote the iomap writeback branch to iomap-for-next.  I
> haven't 100% convinced myself that it's a good idea to hook up xfs to it
> yet, if nothing else because of all the other problems I've had getting
> 5.3 testing to run to completion reliably...

Oh well.  I had some XFS/iomap patches I want to dust off that depend
on it, but I guess they'll have to wait then.

We just need to be very careful the versions don't get out of sync,
which would be a major pain in the butt.
Christoph Hellwig Sept. 1, 2019, 7:34 a.m. UTC | #8
On Fri, Aug 16, 2019 at 06:46:33PM -0700, Darrick J. Wong wrote:
> On Thu, Aug 15, 2019 at 11:52:29PM -0700, Christoph Hellwig wrote:
> > Darrick,
> > 
> > are you going to queue this up?
> 
> Yes, I'll go promote the iomap writeback branch to iomap-for-next.  I
> haven't 100% convinced myself that it's a good idea to hook up xfs to it
> yet, if nothing else because of all the other problems I've had getting
> 5.3 testing to run to completion reliably...

So what is the current status?  We are going to get an -rc8 to give
you some more time, and I'd really hate to miss the second merge window
for the change, espececially as things tend to get out of sync, and I
have various bits touching the code planned for the 5.5 merge window.
Darrick J. Wong Sept. 1, 2019, 8:44 p.m. UTC | #9
On Sun, Sep 01, 2019 at 12:34:40AM -0700, Christoph Hellwig wrote:
> On Fri, Aug 16, 2019 at 06:46:33PM -0700, Darrick J. Wong wrote:
> > On Thu, Aug 15, 2019 at 11:52:29PM -0700, Christoph Hellwig wrote:
> > > Darrick,
> > > 
> > > are you going to queue this up?
> > 
> > Yes, I'll go promote the iomap writeback branch to iomap-for-next.  I
> > haven't 100% convinced myself that it's a good idea to hook up xfs to it
> > yet, if nothing else because of all the other problems I've had getting
> > 5.3 testing to run to completion reliably...
> 
> So what is the current status?  We are going to get an -rc8 to give
> you some more time, and I'd really hate to miss the second merge window
> for the change, espececially as things tend to get out of sync, and I
> have various bits touching the code planned for the 5.5 merge window.

Heh, I had assumed today was going to be 5.3 final and that would be
that for 5.4.  However, if the 5.4 merge window isn't going to close
until Sept. 29 then there's still time for more soaking.

Would you mind rebasing the remaining patches against iomap-for-next and
sending that out?  I'll try to get to it before I go on vacation 6 - 15
Sept.

Admittedly I think the controversial questions are still "How much
writeback code are we outsourcing to iomap anyway?" and "Do we want to
do the added stress of keeping that going without breaking everyone
else"?  IOWs, more philosophical than just the mechanics of porting code
around.

I still want a CONFIG_IOMAP_DEBUG which turns on stricter checking of
the iomap(s) that ->begin_iomap pass to the actor, if you have the time;
I for one am starting to forget exactly what are the valid combinations
of iomap flag inputs that ->begin_iomap has to handle for a given actor
and what are the valid imaps for each actor that it can pass back.  :)

--D
Christoph Hellwig Sept. 2, 2019, 5:16 p.m. UTC | #10
On Sun, Sep 01, 2019 at 01:44:00PM -0700, Darrick J. Wong wrote:
> Would you mind rebasing the remaining patches against iomap-for-next and
> sending that out?  I'll try to get to it before I go on vacation 6 - 15
> Sept.

Ok.  Testing right now, but the rebase was trivial.

> Admittedly I think the controversial questions are still "How much
> writeback code are we outsourcing to iomap anyway?" and "Do we want to
> do the added stress of keeping that going without breaking everyone
> else"?  IOWs, more philosophical than just the mechanics of porting code
> around.

At least as far as I'm concerned the more code that is common the
better so that I don't have to fix up 4 badly maintained half-assed
forks of the same code (hello mpage, ext4 and f2fs..).

> I still want a CONFIG_IOMAP_DEBUG which turns on stricter checking of
> the iomap(s) that ->begin_iomap pass to the actor, if you have the time;
> I for one am starting to forget exactly what are the valid combinations
> of iomap flag inputs that ->begin_iomap has to handle for a given actor
> and what are the valid imaps for each actor that it can pass back.  :)

I was actually thinking of killing the input IOMAP_ types entirely,
as that "flags" model just doesn't scale, and instead have more
iomap_ops instances in the callers.  But that is just a vague idea
at the moment.  I'll need some more time to think about it.
Christoph Hellwig Sept. 10, 2019, 7:01 a.m. UTC | #11
On Mon, Sep 02, 2019 at 10:16:37AM -0700, Christoph Hellwig wrote:
> On Sun, Sep 01, 2019 at 01:44:00PM -0700, Darrick J. Wong wrote:
> > Would you mind rebasing the remaining patches against iomap-for-next and
> > sending that out?  I'll try to get to it before I go on vacation 6 - 15
> > Sept.
> 
> Ok.  Testing right now, but the rebase was trivial.
> 
> > Admittedly I think the controversial questions are still "How much
> > writeback code are we outsourcing to iomap anyway?" and "Do we want to
> > do the added stress of keeping that going without breaking everyone
> > else"?  IOWs, more philosophical than just the mechanics of porting code
> > around.
> 
> At least as far as I'm concerned the more code that is common the
> better so that I don't have to fix up 4 badly maintained half-assed
> forks of the same code (hello mpage, ext4 and f2fs..).

Any news?
Dave Chinner Sept. 10, 2019, 9:30 p.m. UTC | #12
On Tue, Sep 10, 2019 at 12:01:24AM -0700, Christoph Hellwig wrote:
> On Mon, Sep 02, 2019 at 10:16:37AM -0700, Christoph Hellwig wrote:
> > On Sun, Sep 01, 2019 at 01:44:00PM -0700, Darrick J. Wong wrote:
> > > Would you mind rebasing the remaining patches against iomap-for-next and
> > > sending that out?  I'll try to get to it before I go on vacation 6 - 15
> > > Sept.
> > 
> > Ok.  Testing right now, but the rebase was trivial.
> > 
> > > Admittedly I think the controversial questions are still "How much
> > > writeback code are we outsourcing to iomap anyway?" and "Do we want to
> > > do the added stress of keeping that going without breaking everyone
> > > else"?  IOWs, more philosophical than just the mechanics of porting code
> > > around.
> > 
> > At least as far as I'm concerned the more code that is common the
> > better so that I don't have to fix up 4 badly maintained half-assed
> > forks of the same code (hello mpage, ext4 and f2fs..).
> 
> Any news?

Darrick is still on holidays. The iomap-for-next branch has this
series in it already(*), but I suspect at this point the XFS
conversion is going to have to wait until the next cycle.

Cheers,

Dave.

(*) https://git.kernel.org/pub/scm/fs/xfs/xfs-linux.git/log/?h=iomap-for-next