mbox series

[00/11] rename & split tests

Message ID 20220428151559.947144-1-brauner@kernel.org (mailing list archive)
Headers show
Series rename & split tests | expand

Message

Christian Brauner April 28, 2022, 3:15 p.m. UTC
From: "Christian Brauner (Microsoft)" <brauner@kernel.org>

Hey everyone,

As announced multiple times already we need to rename and split the
idmapped mount testsuite into separate source files and also give it a
better name to reflect the fact that it covers a lot more than just
idmapped mounts.

I have decided against compiling different binaries for now. Instead we
compile a single vfstest binary that can be called with various command
line switches to run the various test suites. This is not different than
what we did for the idmapped-mounts binary. Of course, nothing prevents
us from using multiple binaries in the future.

Note that this will have a conflict with Jan's series at:
https://lore.kernel.org/fstests/20220425131809.qzjrah7cw67mzzcw@zlang-mailbox/T/#m47d8da68ef1aff250918398e8d2228729a6acf97

If this patch series here is acceptable I'd pick up Jan's patch and
apply it on top of mine as rebasing will introduce too many conflicts.

Thanks!
Christian

Christian Brauner (11):
  src: rename idmapped-mounts folder
  src/vfs: rename idmapped-mounts.c file
  vfstest: rename struct t_idmapped_mounts
  utils: add missing global.h include
  utils: add struct vfstest_info
  utils: move helpers into utils
  missing: move sys_execveat() to missing.h
  utils: add struct test_suite
  vfstests: split idmapped mount tests into separate suite
  vfstest: split out btrfs idmapped mounts test
  vfstest: split out remaining idmapped mount tests

 .gitignore                                    |     4 +-
 common/rc                                     |    32 +-
 src/Makefile                                  |     2 +-
 src/detached_mounts_propagation.c             |     2 +-
 src/feature.c                                 |     2 +-
 src/idmapped-mounts/utils.c                   |   425 -
 src/idmapped-mounts/utils.h                   |   130 -
 src/{idmapped-mounts => vfs}/Makefile         |    14 +-
 src/vfs/btrfs-idmapped-mounts.c               |  3854 +++++
 src/vfs/btrfs-idmapped-mounts.h               |    15 +
 .../idmapped-mounts.c                         | 11865 ++++------------
 src/vfs/idmapped-mounts.h                     |    17 +
 src/{idmapped-mounts => vfs}/missing.h        |    11 +
 src/{idmapped-mounts => vfs}/mount-idmapped.c |     0
 src/vfs/utils.c                               |  1003 ++
 src/vfs/utils.h                               |   364 +
 src/vfs/vfstest.c                             |  1952 +++
 tests/btrfs/245                               |     2 +-
 tests/generic/633                             |     2 +-
 tests/generic/644                             |     2 +-
 tests/generic/645                             |     2 +-
 tests/generic/656                             |     2 +-
 tests/xfs/152                                 |     4 +-
 tests/xfs/153                                 |     2 +-
 24 files changed, 9952 insertions(+), 9756 deletions(-)
 delete mode 100644 src/idmapped-mounts/utils.c
 delete mode 100644 src/idmapped-mounts/utils.h
 rename src/{idmapped-mounts => vfs}/Makefile (59%)
 create mode 100644 src/vfs/btrfs-idmapped-mounts.c
 create mode 100644 src/vfs/btrfs-idmapped-mounts.h
 rename src/{idmapped-mounts => vfs}/idmapped-mounts.c (50%)
 create mode 100644 src/vfs/idmapped-mounts.h
 rename src/{idmapped-mounts => vfs}/missing.h (93%)
 rename src/{idmapped-mounts => vfs}/mount-idmapped.c (100%)
 create mode 100644 src/vfs/utils.c
 create mode 100644 src/vfs/utils.h
 create mode 100644 src/vfs/vfstest.c

Comments

Christoph Hellwig April 29, 2022, 3:20 p.m. UTC | #1
From a rather highlevel glance over the patches this looks good to me:

Acked: Christoph Hellwig <hch@lst.de>
Zorro Lang May 1, 2022, 11:46 a.m. UTC | #2
On Thu, Apr 28, 2022 at 05:15:48PM +0200, Christian Brauner wrote:
> From: "Christian Brauner (Microsoft)" <brauner@kernel.org>
> 
> Hey everyone,
> 
> As announced multiple times already we need to rename and split the
> idmapped mount testsuite into separate source files and also give it a
> better name to reflect the fact that it covers a lot more than just
> idmapped mounts.
> 
> I have decided against compiling different binaries for now. Instead we
> compile a single vfstest binary that can be called with various command
> line switches to run the various test suites. This is not different than
> what we did for the idmapped-mounts binary. Of course, nothing prevents
> us from using multiple binaries in the future.
> 
> Note that this will have a conflict with Jan's series at:
> https://lore.kernel.org/fstests/20220425131809.qzjrah7cw67mzzcw@zlang-mailbox/T/#m47d8da68ef1aff250918398e8d2228729a6acf97

Hi Christian,

Thanks for your patches! But even without above conflict patches, I still can't
merge your patchset, failures as below [1].
May you rebase your patches on latest fstests for-next branch, and send out
again? As Christoph Hellwig has Acked your patchset, we'd better to make it
work with current fstests at least, for others might want to give it a try.

Thanks,
Zorro

[1]
$ git reset --hard fbc6486b
HEAD is now at fbc6486b generic: test that renaming into a directory fails with EDQUOT
$ git am -s ./20220428_brauner_rename_split_tests.mbx
Applying: src: rename idmapped-mounts folder
Applying: src/vfs: rename idmapped-mounts.c file
Applying: vfstest: rename struct t_idmapped_mounts
Applying: utils: add missing global.h include
Applying: utils: move helpers into utils
error: patch failed: src/vfs/utils.h:51
error: src/vfs/utils.h: patch does not apply
error: patch failed: src/vfs/vfstest.c:35
error: src/vfs/vfstest.c: patch does not apply
Patch failed at 0005 utils: move helpers into utils
hint: Use 'git am --show-current-patch=diff' to see the failed patch
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

> 
> If this patch series here is acceptable I'd pick up Jan's patch and
> apply it on top of mine as rebasing will introduce too many conflicts.
> 
> Thanks!
> Christian
> 
> Christian Brauner (11):
>   src: rename idmapped-mounts folder
>   src/vfs: rename idmapped-mounts.c file
>   vfstest: rename struct t_idmapped_mounts
>   utils: add missing global.h include
>   utils: add struct vfstest_info
>   utils: move helpers into utils
>   missing: move sys_execveat() to missing.h
>   utils: add struct test_suite
>   vfstests: split idmapped mount tests into separate suite
>   vfstest: split out btrfs idmapped mounts test
>   vfstest: split out remaining idmapped mount tests
> 
>  .gitignore                                    |     4 +-
>  common/rc                                     |    32 +-
>  src/Makefile                                  |     2 +-
>  src/detached_mounts_propagation.c             |     2 +-
>  src/feature.c                                 |     2 +-
>  src/idmapped-mounts/utils.c                   |   425 -
>  src/idmapped-mounts/utils.h                   |   130 -
>  src/{idmapped-mounts => vfs}/Makefile         |    14 +-
>  src/vfs/btrfs-idmapped-mounts.c               |  3854 +++++
>  src/vfs/btrfs-idmapped-mounts.h               |    15 +
>  .../idmapped-mounts.c                         | 11865 ++++------------
>  src/vfs/idmapped-mounts.h                     |    17 +
>  src/{idmapped-mounts => vfs}/missing.h        |    11 +
>  src/{idmapped-mounts => vfs}/mount-idmapped.c |     0
>  src/vfs/utils.c                               |  1003 ++
>  src/vfs/utils.h                               |   364 +
>  src/vfs/vfstest.c                             |  1952 +++
>  tests/btrfs/245                               |     2 +-
>  tests/generic/633                             |     2 +-
>  tests/generic/644                             |     2 +-
>  tests/generic/645                             |     2 +-
>  tests/generic/656                             |     2 +-
>  tests/xfs/152                                 |     4 +-
>  tests/xfs/153                                 |     2 +-
>  24 files changed, 9952 insertions(+), 9756 deletions(-)
>  delete mode 100644 src/idmapped-mounts/utils.c
>  delete mode 100644 src/idmapped-mounts/utils.h
>  rename src/{idmapped-mounts => vfs}/Makefile (59%)
>  create mode 100644 src/vfs/btrfs-idmapped-mounts.c
>  create mode 100644 src/vfs/btrfs-idmapped-mounts.h
>  rename src/{idmapped-mounts => vfs}/idmapped-mounts.c (50%)
>  create mode 100644 src/vfs/idmapped-mounts.h
>  rename src/{idmapped-mounts => vfs}/missing.h (93%)
>  rename src/{idmapped-mounts => vfs}/mount-idmapped.c (100%)
>  create mode 100644 src/vfs/utils.c
>  create mode 100644 src/vfs/utils.h
>  create mode 100644 src/vfs/vfstest.c
> 
> -- 
> 2.32.0
>
Christian Brauner May 7, 2022, 12:01 p.m. UTC | #3
On Sun, May 01, 2022 at 07:46:43PM +0800, Zorro Lang wrote:
> On Thu, Apr 28, 2022 at 05:15:48PM +0200, Christian Brauner wrote:
> > From: "Christian Brauner (Microsoft)" <brauner@kernel.org>
> > 
> > Hey everyone,
> > 
> > As announced multiple times already we need to rename and split the
> > idmapped mount testsuite into separate source files and also give it a
> > better name to reflect the fact that it covers a lot more than just
> > idmapped mounts.
> > 
> > I have decided against compiling different binaries for now. Instead we
> > compile a single vfstest binary that can be called with various command
> > line switches to run the various test suites. This is not different than
> > what we did for the idmapped-mounts binary. Of course, nothing prevents
> > us from using multiple binaries in the future.
> > 
> > Note that this will have a conflict with Jan's series at:
> > https://lore.kernel.org/fstests/20220425131809.qzjrah7cw67mzzcw@zlang-mailbox/T/#m47d8da68ef1aff250918398e8d2228729a6acf97
> 
> Hi Christian,
> 
> Thanks for your patches! But even without above conflict patches, I still can't
> merge your patchset, failures as below [1].

Hi Zorro!

The fstests list has a limit on how large a single patch can be in terms
of number of lines (10.000, I believe). Since this patchset contains
patches that change about 10.000 lines the fstests list drops those
patches. I ran into this issue before when we added the testsuite in the
first place but it has never been fixed.

The consequence of this is that b4 is missing patches since they are on
no list. This becomes more obvious if you use b4's quilt option which
creates a folder with individual patches:

  0001_src_rename_idmapped_mounts_folder.patch
  0002_src_vfs_rename_idmapped_mounts_c_file.patch
  0003_vfstest_rename_struct_t_idmapped_mounts.patch
  0004_utils_add_missing_global_h_include.patch
  0006_utils_move_helpers_into_utils.patch
  0007_missing_move_sys_execveat_to_missing_h.patch
  0008_utils_add_struct_test_suite.patch
  0011_vfstest_split_out_remaining_idmapped_mount_tests.patch
  series

so this is missing patches 9 and 10...

> May you rebase your patches on latest fstests for-next branch, and send out
> again? As Christoph Hellwig has Acked your patchset, we'd better to make it

I think we will have the same problem, i.e., patches will not be on the
list since they are too large and so b4 won't work. To work around this
I'll give you a signed tag to pull from in the cover letter.

Christian
Christian Brauner May 7, 2022, 12:03 p.m. UTC | #4
On Sun, May 01, 2022 at 07:46:43PM +0800, Zorro Lang wrote:
> On Thu, Apr 28, 2022 at 05:15:48PM +0200, Christian Brauner wrote:
> > From: "Christian Brauner (Microsoft)" <brauner@kernel.org>
> > 
> > Hey everyone,
> > 
> > As announced multiple times already we need to rename and split the
> > idmapped mount testsuite into separate source files and also give it a
> > better name to reflect the fact that it covers a lot more than just
> > idmapped mounts.
> > 
> > I have decided against compiling different binaries for now. Instead we
> > compile a single vfstest binary that can be called with various command
> > line switches to run the various test suites. This is not different than
> > what we did for the idmapped-mounts binary. Of course, nothing prevents
> > us from using multiple binaries in the future.
> > 
> > Note that this will have a conflict with Jan's series at:
> > https://lore.kernel.org/fstests/20220425131809.qzjrah7cw67mzzcw@zlang-mailbox/T/#m47d8da68ef1aff250918398e8d2228729a6acf97
> 
> Hi Christian,
> 
> Thanks for your patches! But even without above conflict patches, I still can't
> merge your patchset, failures as below [1].

Hi Zorro!
(Resending, since my last message had a messed-up To: line.)

The fstests list has a limit on how large a single patch can be in terms
of number of lines (10.000, I believe). Since this patchset contains
patches that change about 10.000 lines the fstests list drops those
patches. I ran into this issue before when we added the testsuite in the
first place but it has never been fixed.

The consequence of this is that b4 is missing patches since they are on
no list. This becomes more obvious if you use b4's quilt option which
creates a folder with individual patches:

  0001_src_rename_idmapped_mounts_folder.patch
  0002_src_vfs_rename_idmapped_mounts_c_file.patch
  0003_vfstest_rename_struct_t_idmapped_mounts.patch
  0004_utils_add_missing_global_h_include.patch
  0006_utils_move_helpers_into_utils.patch
  0007_missing_move_sys_execveat_to_missing_h.patch
  0008_utils_add_struct_test_suite.patch
  0011_vfstest_split_out_remaining_idmapped_mount_tests.patch
  series

so this is missing patches 9 and 10...

> May you rebase your patches on latest fstests for-next branch, and send out
> again? As Christoph Hellwig has Acked your patchset, we'd better to make it

I think we will have the same problem, i.e., patches will not be on the
list since they are too large and so b4 won't work. To work around this
I'll give you a signed tag to pull from in the cover letter.

Christian
Zorro Lang May 7, 2022, 12:50 p.m. UTC | #5
On Sat, May 07, 2022 at 02:03:09PM +0200, Christian Brauner wrote:
> On Sun, May 01, 2022 at 07:46:43PM +0800, Zorro Lang wrote:
> > On Thu, Apr 28, 2022 at 05:15:48PM +0200, Christian Brauner wrote:
> > > From: "Christian Brauner (Microsoft)" <brauner@kernel.org>
> > > 
> > > Hey everyone,
> > > 
> > > As announced multiple times already we need to rename and split the
> > > idmapped mount testsuite into separate source files and also give it a
> > > better name to reflect the fact that it covers a lot more than just
> > > idmapped mounts.
> > > 
> > > I have decided against compiling different binaries for now. Instead we
> > > compile a single vfstest binary that can be called with various command
> > > line switches to run the various test suites. This is not different than
> > > what we did for the idmapped-mounts binary. Of course, nothing prevents
> > > us from using multiple binaries in the future.
> > > 
> > > Note that this will have a conflict with Jan's series at:
> > > https://lore.kernel.org/fstests/20220425131809.qzjrah7cw67mzzcw@zlang-mailbox/T/#m47d8da68ef1aff250918398e8d2228729a6acf97
> > 
> > Hi Christian,
> > 
> > Thanks for your patches! But even without above conflict patches, I still can't
> > merge your patchset, failures as below [1].
> 
> Hi Zorro!
> (Resending, since my last message had a messed-up To: line.)
> 
> The fstests list has a limit on how large a single patch can be in terms
> of number of lines (10.000, I believe). Since this patchset contains
> patches that change about 10.000 lines the fstests list drops those
> patches. I ran into this issue before when we added the testsuite in the
> first place but it has never been fixed.
> 
> The consequence of this is that b4 is missing patches since they are on
> no list. This becomes more obvious if you use b4's quilt option which
> creates a folder with individual patches:
> 
>   0001_src_rename_idmapped_mounts_folder.patch
>   0002_src_vfs_rename_idmapped_mounts_c_file.patch
>   0003_vfstest_rename_struct_t_idmapped_mounts.patch
>   0004_utils_add_missing_global_h_include.patch
>   0006_utils_move_helpers_into_utils.patch
>   0007_missing_move_sys_execveat_to_missing_h.patch
>   0008_utils_add_struct_test_suite.patch
>   0011_vfstest_split_out_remaining_idmapped_mount_tests.patch
>   series
> 
> so this is missing patches 9 and 10...

Oh, I didn't notice that. Let me check ...

> 
> > May you rebase your patches on latest fstests for-next branch, and send out
> > again? As Christoph Hellwig has Acked your patchset, we'd better to make it
> 
> I think we will have the same problem, i.e., patches will not be on the
> list since they are too large and so b4 won't work. To work around this
> I'll give you a signed tag to pull from in the cover letter.

Hmm...I just used another method(avoid b4) to merge your 11 patches, I think
it works:
# git log --oneline
14714a1b (HEAD -> for-brauner) vfstest: split out remaining idmapped mount tests
ed027078 vfstest: split out btrfs idmapped mounts test
fd026c1a vfstests: split idmapped mount tests into separate suite
1cec19ec utils: add struct test_suite
3dadfe72 missing: move sys_execveat() to missing.h
ba0370e4 utils: move helpers into utils
781f86e1 utils: add struct vfstest_info
e1ffed6b utils: add missing global.h include
4f68af21 vfstest: rename struct t_idmapped_mounts
8fcd0f72 src/vfs: rename idmapped-mounts.c file
0b03254e src: rename idmapped-mounts folder
fbc6486b (redhat/master, origin/master, origin/HEAD, rh-master, master) generic: test that renaming into a directory fails with EDQUOT
...
...

And yes, patch 9 and 10 are really huge:
$ git show ed027078 | wc -l
8223
$ git show fd026c1a | wc -l
15641

But I still can't merge it into for-next branch, so you have to do once rebase
at least. As we have 3 more idmapped patches to from XuYang to be merged,
you'd better to wait the fstests release this week, then do once rebase
on for-next branch. Hope that can save a little of your time.

Thanks,
Zorro

> 
> Christian
>
Christian Brauner May 7, 2022, 3:43 p.m. UTC | #6
On Sat, May 07, 2022 at 08:50:20PM +0800, Zorro Lang wrote:
> On Sat, May 07, 2022 at 02:03:09PM +0200, Christian Brauner wrote:
> > On Sun, May 01, 2022 at 07:46:43PM +0800, Zorro Lang wrote:
> > > On Thu, Apr 28, 2022 at 05:15:48PM +0200, Christian Brauner wrote:
> > > > From: "Christian Brauner (Microsoft)" <brauner@kernel.org>
> > > > 
> > > > Hey everyone,
> > > > 
> > > > As announced multiple times already we need to rename and split the
> > > > idmapped mount testsuite into separate source files and also give it a
> > > > better name to reflect the fact that it covers a lot more than just
> > > > idmapped mounts.
> > > > 
> > > > I have decided against compiling different binaries for now. Instead we
> > > > compile a single vfstest binary that can be called with various command
> > > > line switches to run the various test suites. This is not different than
> > > > what we did for the idmapped-mounts binary. Of course, nothing prevents
> > > > us from using multiple binaries in the future.
> > > > 
> > > > Note that this will have a conflict with Jan's series at:
> > > > https://lore.kernel.org/fstests/20220425131809.qzjrah7cw67mzzcw@zlang-mailbox/T/#m47d8da68ef1aff250918398e8d2228729a6acf97
> > > 
> > > Hi Christian,
> > > 
> > > Thanks for your patches! But even without above conflict patches, I still can't
> > > merge your patchset, failures as below [1].
> > 
> > Hi Zorro!
> > (Resending, since my last message had a messed-up To: line.)
> > 
> > The fstests list has a limit on how large a single patch can be in terms
> > of number of lines (10.000, I believe). Since this patchset contains
> > patches that change about 10.000 lines the fstests list drops those
> > patches. I ran into this issue before when we added the testsuite in the
> > first place but it has never been fixed.
> > 
> > The consequence of this is that b4 is missing patches since they are on
> > no list. This becomes more obvious if you use b4's quilt option which
> > creates a folder with individual patches:
> > 
> >   0001_src_rename_idmapped_mounts_folder.patch
> >   0002_src_vfs_rename_idmapped_mounts_c_file.patch
> >   0003_vfstest_rename_struct_t_idmapped_mounts.patch
> >   0004_utils_add_missing_global_h_include.patch
> >   0006_utils_move_helpers_into_utils.patch
> >   0007_missing_move_sys_execveat_to_missing_h.patch
> >   0008_utils_add_struct_test_suite.patch
> >   0011_vfstest_split_out_remaining_idmapped_mount_tests.patch
> >   series
> > 
> > so this is missing patches 9 and 10...
> 
> Oh, I didn't notice that. Let me check ...
> 
> > 
> > > May you rebase your patches on latest fstests for-next branch, and send out
> > > again? As Christoph Hellwig has Acked your patchset, we'd better to make it
> > 
> > I think we will have the same problem, i.e., patches will not be on the
> > list since they are too large and so b4 won't work. To work around this
> > I'll give you a signed tag to pull from in the cover letter.
> 
> Hmm...I just used another method(avoid b4) to merge your 11 patches, I think
> it works:
> # git log --oneline
> 14714a1b (HEAD -> for-brauner) vfstest: split out remaining idmapped mount tests
> ed027078 vfstest: split out btrfs idmapped mounts test
> fd026c1a vfstests: split idmapped mount tests into separate suite
> 1cec19ec utils: add struct test_suite
> 3dadfe72 missing: move sys_execveat() to missing.h
> ba0370e4 utils: move helpers into utils
> 781f86e1 utils: add struct vfstest_info
> e1ffed6b utils: add missing global.h include
> 4f68af21 vfstest: rename struct t_idmapped_mounts
> 8fcd0f72 src/vfs: rename idmapped-mounts.c file
> 0b03254e src: rename idmapped-mounts folder
> fbc6486b (redhat/master, origin/master, origin/HEAD, rh-master, master) generic: test that renaming into a directory fails with EDQUOT
> ...
> ...
> 
> And yes, patch 9 and 10 are really huge:
> $ git show ed027078 | wc -l
> 8223
> $ git show fd026c1a | wc -l
> 15641
> 
> But I still can't merge it into for-next branch, so you have to do once rebase
> at least. As we have 3 more idmapped patches to from XuYang to be merged,
> you'd better to wait the fstests release this week, then do once rebase

Yeah, sounds good!

Christian