| Message ID | 162370437774.3800603.15907676407985880109.stgit@locust (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | fstests: move test group lists into test files | expand |
On Mon, Jun 14, 2021 at 01:59:37PM -0700, Darrick J. Wong wrote: > From: Darrick J. Wong <djwong@kernel.org> > > Now that we've moved the group membership details into the test case > files themselves, automatically generate the group files during build. > The autogenerated files are named "group.list" instead of "group" to > avoid conflicts between generated and (stale) SCM files as everyone > rebases. Hmm, so we now need to run make to regenerate the group information? That is for sure going to create various issues with people forgetting to regenerate it. What about regenerating it everytime a group based run is executed?
On Wed, Jun 16, 2021 at 08:32:28AM +0100, Christoph Hellwig wrote: > On Mon, Jun 14, 2021 at 01:59:37PM -0700, Darrick J. Wong wrote: > > From: Darrick J. Wong <djwong@kernel.org> > > > > Now that we've moved the group membership details into the test case > > files themselves, automatically generate the group files during build. > > The autogenerated files are named "group.list" instead of "group" to > > avoid conflicts between generated and (stale) SCM files as everyone > > rebases. > > Hmm, so we now need to run make to regenerate the group information? > That is for sure going to create various issues with people forgetting > to regenerate it. What about regenerating it everytime a group based > run is executed? <shrug> What's wrong with requiring everyone to run 'make' when they change something in fstests? I suspect most people already do that as muscle memory for most every other code project on the planet, or as part of `make install' prior to starting it fstests. --D
On Wed, Jun 16, 2021 at 05:13:20PM -0700, Darrick J. Wong wrote: > <shrug> What's wrong with requiring everyone to run 'make' when they > change something in fstests? I suspect most people already do that as > muscle memory for most every other code project on the planet, or as > part of `make install' prior to starting it fstests. I know I keep forgetting it and I know a few other people that do as well.
On Thu, Jun 17, 2021 at 08:56:01AM +0100, Christoph Hellwig wrote: > On Wed, Jun 16, 2021 at 05:13:20PM -0700, Darrick J. Wong wrote: > > <shrug> What's wrong with requiring everyone to run 'make' when they > > change something in fstests? I suspect most people already do that as > > muscle memory for most every other code project on the planet, or as > > part of `make install' prior to starting it fstests. > > I know I keep forgetting it and I know a few other people that do as > well. I suppose I could make the '-g' switch call 'make group.list', though that's just going to increase the amount of noise for anyone like me who runs fstests with a mostly readonly rootfs. --D
On Thu, Jun 17, 2021 at 10:15:00AM -0700, Darrick J. Wong wrote: > I suppose I could make the '-g' switch call 'make group.list', though > that's just going to increase the amount of noise for anyone like me who > runs fstests with a mostly readonly rootfs. Just stick to the original version and see if anyone screams loud enough. Of course the best long-term plan would be to not even generate group files by just calculate the list in-memory.
On Fri, Jun 18, 2021 at 4:47 PM Christoph Hellwig <hch@infradead.org> wrote: > > On Thu, Jun 17, 2021 at 10:15:00AM -0700, Darrick J. Wong wrote: > > I suppose I could make the '-g' switch call 'make group.list', though > > that's just going to increase the amount of noise for anyone like me who > > runs fstests with a mostly readonly rootfs. > > Just stick to the original version and see if anyone screams loud What is the original version? > enough. Of course the best long-term plan would be to not even generate > group files by just calculate the list in-memory. Why in-memory? check already creates $tmp.list with the list of test files right? Any reason not to the group.list files under /tmp while preparing the test list instead of creating them in the src directory? Thanks, Amir.
On Fri, Jun 18, 2021 at 06:32:18PM +0300, Amir Goldstein wrote: > On Fri, Jun 18, 2021 at 4:47 PM Christoph Hellwig <hch@infradead.org> wrote: > > > > On Thu, Jun 17, 2021 at 10:15:00AM -0700, Darrick J. Wong wrote: > > > I suppose I could make the '-g' switch call 'make group.list', though > > > that's just going to increase the amount of noise for anyone like me who > > > runs fstests with a mostly readonly rootfs. > > > > Just stick to the original version and see if anyone screams loud > > What is the original version? Assuming the developer is smart enough to run 'make all install' before running fstests. --D > > enough. Of course the best long-term plan would be to not even generate > > group files by just calculate the list in-memory. > > Why in-memory? > check already creates $tmp.list with the list of test files right? > Any reason not to the group.list files under /tmp while preparing > the test list instead of creating them in the src directory? > > Thanks, > Amir.
On Fri, Jun 18, 2021 at 06:32:18PM +0300, Amir Goldstein wrote: > > Just stick to the original version and see if anyone screams loud > > What is the original version? The patch that Darrick posted, which requires make to be run to generatethe group files.
On Fri, Jun 18, 2021 at 08:56:30AM -0700, Darrick J. Wong wrote: > On Fri, Jun 18, 2021 at 06:32:18PM +0300, Amir Goldstein wrote: > > On Fri, Jun 18, 2021 at 4:47 PM Christoph Hellwig <hch@infradead.org> wrote: > > > > > > On Thu, Jun 17, 2021 at 10:15:00AM -0700, Darrick J. Wong wrote: > > > > I suppose I could make the '-g' switch call 'make group.list', though > > > > that's just going to increase the amount of noise for anyone like me who > > > > runs fstests with a mostly readonly rootfs. > > > > > > Just stick to the original version and see if anyone screams loud > > > > What is the original version? > > Assuming the developer is smart enough to run 'make all install' before > running fstests. So install is also required now? I have never before installed xfstests I think.
On Mon, Jun 21, 2021 at 06:25:14AM +0100, Christoph Hellwig wrote: > On Fri, Jun 18, 2021 at 08:56:30AM -0700, Darrick J. Wong wrote: > > On Fri, Jun 18, 2021 at 06:32:18PM +0300, Amir Goldstein wrote: > > > On Fri, Jun 18, 2021 at 4:47 PM Christoph Hellwig <hch@infradead.org> wrote: > > > > > > > > On Thu, Jun 17, 2021 at 10:15:00AM -0700, Darrick J. Wong wrote: > > > > > I suppose I could make the '-g' switch call 'make group.list', though > > > > > that's just going to increase the amount of noise for anyone like me who > > > > > runs fstests with a mostly readonly rootfs. > > > > > > > > Just stick to the original version and see if anyone screams loud > > > > > > What is the original version? > > > > Assuming the developer is smart enough to run 'make all install' before > > running fstests. > > So install is also required now? I have never before installed xfstests > I think. No, install is not required, but running make after updating fstests is required, at least some tests may introduce new test binary in src and the binary is missing if you don't run make. So I think make is fine after updating fstests. Thanks, Eryu
On Mon, Jun 21, 2021 at 02:38:45PM +0800, Eryu Guan wrote: > On Mon, Jun 21, 2021 at 06:25:14AM +0100, Christoph Hellwig wrote: > > On Fri, Jun 18, 2021 at 08:56:30AM -0700, Darrick J. Wong wrote: > > > On Fri, Jun 18, 2021 at 06:32:18PM +0300, Amir Goldstein wrote: > > > > On Fri, Jun 18, 2021 at 4:47 PM Christoph Hellwig <hch@infradead.org> wrote: > > > > > > > > > > On Thu, Jun 17, 2021 at 10:15:00AM -0700, Darrick J. Wong wrote: > > > > > > I suppose I could make the '-g' switch call 'make group.list', though > > > > > > that's just going to increase the amount of noise for anyone like me who > > > > > > runs fstests with a mostly readonly rootfs. > > > > > > > > > > Just stick to the original version and see if anyone screams loud > > > > > > > > What is the original version? > > > > > > Assuming the developer is smart enough to run 'make all install' before > > > running fstests. > > > > So install is also required now? I have never before installed xfstests > > I think. Sorry, that should have read “...enough to run 'make all' or 'make install' before running...”. > No, install is not required, but running make after updating fstests is > required, at least some tests may introduce new test binary in src and > the binary is missing if you don't run make. So I think make is fine > after updating fstests. Ok, I'll leave this as it was and resubmit the series with the other tweaks suggested by Christoph. --D > Thanks, > Eryu
diff --git a/.gitignore b/.gitignore index d3194e76..2d72b064 100644 --- a/.gitignore +++ b/.gitignore @@ -11,6 +11,9 @@ tags /local.config /results +# autogenerated group files +/tests/*/group.list + # autoconf generated files /aclocal.m4 /autom4te.cache diff --git a/common/preamble b/common/preamble index eafce487..3a4a6eb0 100644 --- a/common/preamble +++ b/common/preamble @@ -32,6 +32,14 @@ _begin_fstest() fi seq=`basename $0` + + # If we're only running the test to generate a group.list file, + # spit out the group data and exit. + if [ -n "$GENERATE_GROUPS" ]; then + echo "$seq $@" + exit 0 + fi + seqres=$RESULT_DIR/$seq echo "QA output created by $seq" diff --git a/include/buildgrouplist b/include/buildgrouplist new file mode 100644 index 00000000..d898efa3 --- /dev/null +++ b/include/buildgrouplist @@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2021 Oracle, Inc. All Rights Reserved. +# +.PHONY: group.list + +group.list: + @echo " [GROUP] $$PWD/$@" + $(Q)$(TOPDIR)/tools/mkgroupfile $@ diff --git a/tests/Makefile b/tests/Makefile index 8ce8f209..5c8f0b10 100644 --- a/tests/Makefile +++ b/tests/Makefile @@ -7,6 +7,10 @@ include $(TOPDIR)/include/builddefs TESTS_SUBDIRS = $(sort $(dir $(wildcard $(CURDIR)/[[:lower:]]*/))) +SUBDIRS = $(wildcard [[:lower:]]*) + +default: $(SUBDIRS) + include $(BUILDRULES) install: $(addsuffix -install,$(TESTS_SUBDIRS)) diff --git a/tests/btrfs/Makefile b/tests/btrfs/Makefile index 2d936421..1b72a1a1 100644 --- a/tests/btrfs/Makefile +++ b/tests/btrfs/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist BTRFS_DIR = btrfs TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(BTRFS_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/ceph/Makefile b/tests/ceph/Makefile index 55e35d77..2761e1e9 100644 --- a/tests/ceph/Makefile +++ b/tests/ceph/Makefile @@ -2,16 +2,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist CEPH_DIR = ceph TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(CEPH_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/cifs/Makefile b/tests/cifs/Makefile index 0c5cf3be..62c48935 100644 --- a/tests/cifs/Makefile +++ b/tests/cifs/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist CIFS_DIR = cifs TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(CIFS_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/ext4/Makefile b/tests/ext4/Makefile index beb1541f..a2a0d561 100644 --- a/tests/ext4/Makefile +++ b/tests/ext4/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist EXT4_DIR = ext4 TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(EXT4_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/f2fs/Makefile b/tests/f2fs/Makefile index d13bca3f..9d1ed3c6 100644 --- a/tests/f2fs/Makefile +++ b/tests/f2fs/Makefile @@ -5,16 +5,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist F2FS_DIR = f2fs TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(F2FS_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/generic/Makefile b/tests/generic/Makefile index 3878d05c..b464b22b 100644 --- a/tests/generic/Makefile +++ b/tests/generic/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist GENERIC_DIR = generic TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(GENERIC_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/nfs/Makefile b/tests/nfs/Makefile index 754f2b25..128d2a3a 100644 --- a/tests/nfs/Makefile +++ b/tests/nfs/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist NFS_DIR = nfs TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(NFS_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/ocfs2/Makefile b/tests/ocfs2/Makefile index e1337908..260ad31b 100644 --- a/tests/ocfs2/Makefile +++ b/tests/ocfs2/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist OCFS2_DIR = ocfs2 TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(OCFS2_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/overlay/Makefile b/tests/overlay/Makefile index b07f8925..de3203c2 100644 --- a/tests/overlay/Makefile +++ b/tests/overlay/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist TEST_DIR = overlay TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(TEST_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/perf/Makefile b/tests/perf/Makefile index 620f1dbf..0c74ba39 100644 --- a/tests/perf/Makefile +++ b/tests/perf/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist PERF_DIR = perf TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(PERF_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/shared/Makefile b/tests/shared/Makefile index 8a832782..f3128714 100644 --- a/tests/shared/Makefile +++ b/tests/shared/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist SHARED_DIR = shared TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(SHARED_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/udf/Makefile b/tests/udf/Makefile index c9c9f1bd..ed4434f0 100644 --- a/tests/udf/Makefile +++ b/tests/udf/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist UDF_DIR = udf TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(UDF_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tests/xfs/Makefile b/tests/xfs/Makefile index d64800ea..5f413e67 100644 --- a/tests/xfs/Makefile +++ b/tests/xfs/Makefile @@ -4,16 +4,20 @@ TOPDIR = ../.. include $(TOPDIR)/include/builddefs +include $(TOPDIR)/include/buildgrouplist XFS_DIR = xfs TARGET_DIR = $(PKG_LIB_DIR)/$(TESTS_DIR)/$(XFS_DIR) +DIRT = group.list + +default: $(DIRT) include $(BUILDRULES) install: $(INSTALL) -m 755 -d $(TARGET_DIR) $(INSTALL) -m 755 $(TESTS) $(TARGET_DIR) - $(INSTALL) -m 644 group $(TARGET_DIR) + $(INSTALL) -m 644 group.list $(TARGET_DIR) $(INSTALL) -m 644 $(OUTFILES) $(TARGET_DIR) # Nothing. diff --git a/tools/mkgroupfile b/tools/mkgroupfile new file mode 100755 index 00000000..0681e5d2 --- /dev/null +++ b/tools/mkgroupfile @@ -0,0 +1,42 @@ +#!/bin/bash + +# Generate a group file from the _begin_fstest call in each test. + +if [ "$1" = "--help" ]; then + echo "Usage: (cd tests/XXX/ ; ../../tools/mkgroupfile [output])" + exit 1 +fi + +test_dir="$PWD" +groupfile="$1" + +if [ ! -x ../../check ]; then + echo "$0: Run this from tests/XXX/." + exit 1 +fi + +generate_groupfile() { + cat << ENDL +# QA groups control file, automatically generated. +# See _begin_fstest in each test for details. + +ENDL + cd ../../ + export GENERATE_GROUPS=yes + grep -R -l "^_begin_fstest" "$test_dir/" 2>/dev/null | while read testfile; do + test -x "$testfile" && "$testfile" + done | sort -g + cd "$test_dir" +} + +if [ -z "$groupfile" ] || [ "$groupfile" = "-" ]; then + # Dump the group file to stdout and exit + generate_groupfile + exit 0 +fi + +# Otherwise, write the group file to disk somewhere. +ngroupfile="${groupfile}.new" +rm -f "$ngroupfile" +generate_groupfile >> "$ngroupfile" +mv "$ngroupfile" "$groupfile"