| Message ID | 20240306114741.14564-1-anthony.perard@citrix.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [OSSTEST] ap-common: Switch to Linux 6.1 by default on x86 + drop dom0 i386 | expand |
On Wed, Mar 06, 2024 at 11:47:41AM +0000, Anthony PERARD wrote: > linux-4.19 branch in xenbits is outdated, it haven't been updated and > tested since 2020 as it has been disabled in osstest. Also, this 4.19 > branch doesn't build on Bookworm. > > So we will start to use a newer version of Linux. We switch to 6.1 for > the Arm* tests recently, so will use that same version for x86. > > Also, following commit 3a3089c94913 ("mfi-common: Drop Linux dom0 i386 > tests for newer Linux branches"), 32bit dom0 isn't tested on newer > Linux, so we need to drop all dom0 i386 tests wherever the default > linux branch is used. That is, this changes in jobs will apply to > "xen-unstable" branch but also all xen stable branches, seabios, qemu, > osstest, libvirt, so every branch that aren't "linux-*". > [...] > > In any case, the list of test would be the same as for the existing > branch "linux-linus" or "linux-6.1" branches. > > Signed-off-by: Anthony PERARD <anthony.perard@citrix.com> There hasn't been comment on this change, so I guess everyone is happy for this change to go in. :-) I'll push it later this week. Thanks,
On Wed, Mar 06, 2024 at 11:47:41AM +0000, Anthony PERARD wrote: > linux-4.19 branch in xenbits is outdated, it haven't been updated and > tested since 2020 as it has been disabled in osstest. Also, this 4.19 > branch doesn't build on Bookworm. > > So we will start to use a newer version of Linux. We switch to 6.1 for > the Arm* tests recently, so will use that same version for x86. > > Also, following commit 3a3089c94913 ("mfi-common: Drop Linux dom0 i386 > tests for newer Linux branches"), 32bit dom0 isn't tested on newer > Linux, so we need to drop all dom0 i386 tests wherever the default > linux branch is used. That is, this changes in jobs will apply to > "xen-unstable" branch but also all xen stable branches, seabios, qemu, > osstest, libvirt, so every branch that aren't "linux-*". > > Here is the list jobs that changes, and whether they are replace, or > have existing equivalents, on the "xen-unstable" branch. Changes > compared with: > OSSTEST_CONFIG=standalone-config-example nice eatmydata ./standalone-generate-dump-flight-runvars > > Gone, without exiting or new test-amd64-amd64-*: > - test-amd64-i386-freebsd10-amd64 > - test-amd64-i386-freebsd10-i386 I should have removed those long time ago, as FreeBSD has been EOL for ages. > - test-amd64-i386-qemut-rhel6hvm-amd > - test-amd64-i386-qemut-rhel6hvm-intel > - test-amd64-i386-qemuu-rhel6hvm-amd > - test-amd64-i386-qemuu-rhel6hvm-intel > > Gone, but with exiting test-amd64-amd64-* equivalent: > - test-amd64-coresched-i386-xl > - test-amd64-i386-examine > - test-amd64-i386-examine-bios > - test-amd64-i386-examine-uefi > - test-amd64-i386-libvirt > - test-amd64-i386-libvirt-pair > - test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm > - test-amd64-i386-libvirt-xsm > - test-amd64-i386-livepatch > - test-amd64-i386-migrupgrade > - test-amd64-i386-pair > - test-amd64-i386-xl > - test-amd64-i386-xl-pvshim > - test-amd64-i386-xl-qemut-debianhvm-amd64 > - test-amd64-i386-xl-qemut-debianhvm-i386-xsm > - test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm > - test-amd64-i386-xl-qemut-win7-amd64 > - test-amd64-i386-xl-qemut-ws16-amd64 > - test-amd64-i386-xl-qemuu-debianhvm-amd64 > - test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow > - test-amd64-i386-xl-qemuu-debianhvm-i386-xsm > - test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict > - test-amd64-i386-xl-qemuu-ovmf-amd64 > - test-amd64-i386-xl-qemuu-win7-amd64 > - test-amd64-i386-xl-qemuu-ws16-amd64 > - test-amd64-i386-xl-shadow > - test-amd64-i386-xl-simplat-amd64-buster > - test-amd64-i386-xl-xsm > > Gone, but replaced by a new test-amd64-amd64-*: > - test-amd64-i386-libvirt-raw > - test-amd64-i386-xl-vhd > > Some test-amd64-amd64-* are also changed: > - test-amd64-amd64-libvirt-vhd > - test-amd64-amd64-qemuu-freebsd11-amd64 > - test-amd64-amd64-qemuu-freebsd12-amd64 > - test-amd64-amd64-xl-qcow2 > + test-amd64-amd64-freebsd11-amd64 > + test-amd64-amd64-freebsd12-amd64 > + test-amd64-amd64-libvirt-qcow2 > + test-amd64-amd64-libvirt-raw > + test-amd64-amd64-xl-vhd Is this purely a name change, or there's some kind of functional change? I'm mostly worried whether this could lead to regressions getting in as osstest would detect those jobs as new instead of inheriting from the previous jobs. Also the "History for test ..." won't link to jobs prior to the rename? I guess I'm confused about what causes the qemuu to be dropped from the job names above. Thanks, Roger.
On Mon, Mar 11, 2024 at 06:27:52PM +0100, Roger Pau Monné wrote: > On Wed, Mar 06, 2024 at 11:47:41AM +0000, Anthony PERARD wrote: > > Gone, but replaced by a new test-amd64-amd64-*: > > - test-amd64-i386-libvirt-raw > > - test-amd64-i386-xl-vhd > > > > Some test-amd64-amd64-* are also changed: > > - test-amd64-amd64-libvirt-vhd > > - test-amd64-amd64-qemuu-freebsd11-amd64 > > - test-amd64-amd64-qemuu-freebsd12-amd64 > > - test-amd64-amd64-xl-qcow2 > > + test-amd64-amd64-freebsd11-amd64 > > + test-amd64-amd64-freebsd12-amd64 > > + test-amd64-amd64-libvirt-qcow2 > > + test-amd64-amd64-libvirt-raw > > + test-amd64-amd64-xl-vhd > > Is this purely a name change, or there's some kind of functional > change? For test-amd64-amd64-qemuu-freebsd1{1,2}-amd64, it looks like the "-qemuu" is a bug. The freebsd jobs shouldn't have used $qemuu_suffix, as it doesn't use $qemuu_runvar. So I'm guessing $qemuu_suffix was just the value left from a previous call of test_matrix_do_one() with dom0arch==i386. The new name is already used by "linux-linus" branch. As for the few changes in {xl,libvirt}-{raw,vhd,qcow2} tests, this is because of - f536e834f673 ("make-flight: Trim the matrix of disk format flights") - 5c70735f177f ("fmtarches: Use dom0arches, not hardcoded arch list") > It's not ideal because it means tests jumping about across arches and > might let regressions go through but it's better than just dropping > them, and doing a better approach is complex. So, for those, yes, it's different tests. It adds one "-raw" test, and otherwise changes the toolstack used for the two other disk formats. > I'm mostly worried whether this could lead to regressions getting in > as osstest would detect those jobs as new instead of inheriting from > the previous jobs. Also the "History for test ..." won't link to jobs > prior to the rename? Well, those tests are already in the "linux-linus" branch and the new "linux-6.1" branch, so we can already check manually if there's a regression or not. So if they pass in linux-6.1, regression will be detected, at least in the linux branch. The rename of the freebsd jobs is unfortunate, but at least, it's going to be the correct name, instead of having a spurious "-qemuu" suffix. As for "history of tests " pages, no I don't think we can easly link to to the previous job name. But that only apply to both freebsd* jobs. We could keep the -qemuu in freebsd* jobs even if it's misleading to someone who know what it should mean, that would keep the job name to all branches and rename it in "linux-linus" branch. I don't know which one is better. > I guess I'm confused about what causes the qemuu to be dropped from > the job names above. It just shell script having variable been global by default. Cheers,
On Tue, Mar 12, 2024 at 05:19:46PM +0000, Anthony PERARD wrote: > On Mon, Mar 11, 2024 at 06:27:52PM +0100, Roger Pau Monné wrote: > > On Wed, Mar 06, 2024 at 11:47:41AM +0000, Anthony PERARD wrote: > > > Gone, but replaced by a new test-amd64-amd64-*: > > > - test-amd64-i386-libvirt-raw > > > - test-amd64-i386-xl-vhd > > > > > > Some test-amd64-amd64-* are also changed: > > > - test-amd64-amd64-libvirt-vhd > > > - test-amd64-amd64-qemuu-freebsd11-amd64 > > > - test-amd64-amd64-qemuu-freebsd12-amd64 > > > - test-amd64-amd64-xl-qcow2 > > > + test-amd64-amd64-freebsd11-amd64 > > > + test-amd64-amd64-freebsd12-amd64 > > > + test-amd64-amd64-libvirt-qcow2 > > > + test-amd64-amd64-libvirt-raw > > > + test-amd64-amd64-xl-vhd > > > > Is this purely a name change, or there's some kind of functional > > change? > > For test-amd64-amd64-qemuu-freebsd1{1,2}-amd64, it looks like the > "-qemuu" is a bug. The freebsd jobs shouldn't have used $qemuu_suffix, > as it doesn't use $qemuu_runvar. So I'm guessing $qemuu_suffix was just > the value left from a previous call of test_matrix_do_one() with > dom0arch==i386. The new name is already used by "linux-linus" branch. FTAOD, could you mention this in the commit message? FreeBSD doesn't use `$qemuu_runvar` because it was always using QEMU upstream (when the FreeBSD test was added we decided to only test with QEMU upstream). So there's indeed no `-qemut` variant, but by dropping the -qemuu prefix it could be confused with a PV guest based test. > As for the few changes in {xl,libvirt}-{raw,vhd,qcow2} tests, this is > because of > - f536e834f673 ("make-flight: Trim the matrix of disk format flights") > - 5c70735f177f ("fmtarches: Use dom0arches, not hardcoded arch list") Probably a dummy question, but why haven't those commits changed the output of make-flight earlier? I'm fine with the change, but I don't really get why we are seeing it only now. > > It's not ideal because it means tests jumping about across arches and > > might let regressions go through but it's better than just dropping > > them, and doing a better approach is complex. > > So, for those, yes, it's different tests. It adds one "-raw" test, and > otherwise changes the toolstack used for the two other disk formats. > > > I'm mostly worried whether this could lead to regressions getting in > > as osstest would detect those jobs as new instead of inheriting from > > the previous jobs. Also the "History for test ..." won't link to jobs > > prior to the rename? > > Well, those tests are already in the "linux-linus" branch and the new > "linux-6.1" branch, so we can already check manually if there's a > regression or not. So if they pass in linux-6.1, regression will be > detected, at least in the linux branch. > > The rename of the freebsd jobs is unfortunate, but at least, it's going > to be the correct name, instead of having a spurious "-qemuu" suffix. > > As for "history of tests " pages, no I don't think we can easly link to > to the previous job name. But that only apply to both freebsd* jobs. > > We could keep the -qemuu in freebsd* jobs even if it's misleading to > someone who know what it should mean, that would keep the job name to > all branches and rename it in "linux-linus" branch. I don't know which > one is better. > > > I guess I'm confused about what causes the qemuu to be dropped from > > the job names above. > > It just shell script having variable been global by default. Thanks for the clarification. Roger.
On Wed, Mar 13, 2024 at 11:48:11AM +0100, Roger Pau Monné wrote: > On Tue, Mar 12, 2024 at 05:19:46PM +0000, Anthony PERARD wrote: > > On Mon, Mar 11, 2024 at 06:27:52PM +0100, Roger Pau Monné wrote: > > > On Wed, Mar 06, 2024 at 11:47:41AM +0000, Anthony PERARD wrote: > > > > Gone, but replaced by a new test-amd64-amd64-*: > > > > - test-amd64-i386-libvirt-raw > > > > - test-amd64-i386-xl-vhd > > > > > > > > Some test-amd64-amd64-* are also changed: > > > > - test-amd64-amd64-libvirt-vhd > > > > - test-amd64-amd64-qemuu-freebsd11-amd64 > > > > - test-amd64-amd64-qemuu-freebsd12-amd64 > > > > - test-amd64-amd64-xl-qcow2 > > > > + test-amd64-amd64-freebsd11-amd64 > > > > + test-amd64-amd64-freebsd12-amd64 > > > > + test-amd64-amd64-libvirt-qcow2 > > > > + test-amd64-amd64-libvirt-raw > > > > + test-amd64-amd64-xl-vhd > > > > > > Is this purely a name change, or there's some kind of functional > > > change? > > > > For test-amd64-amd64-qemuu-freebsd1{1,2}-amd64, it looks like the > > "-qemuu" is a bug. The freebsd jobs shouldn't have used $qemuu_suffix, > > as it doesn't use $qemuu_runvar. So I'm guessing $qemuu_suffix was just > > the value left from a previous call of test_matrix_do_one() with > > dom0arch==i386. The new name is already used by "linux-linus" branch. > > FTAOD, could you mention this in the commit message? > > FreeBSD doesn't use `$qemuu_runvar` because it was always using QEMU > upstream (when the FreeBSD test was added we decided to only test with > QEMU upstream). So there's indeed no `-qemut` variant, but by > dropping the -qemuu prefix it could be confused with a PV guest > based test. I thought "-qemuu" just meant we were using the qemu-upstream as device-model instead of the default one, but it seems that instead no jobs are generated with "$qemuu_suffix==''", (or only for Xen 4.2 and older). So now I guess that all HVM tests have either -qemut or -qemuu. We that, I guess it makes sense to keep "-qemuu" for freebsd tests. That would just change branch "linux-linus" and "linux-6.1". I'll prepare a new patch with that. > > As for the few changes in {xl,libvirt}-{raw,vhd,qcow2} tests, this is > > because of > > - f536e834f673 ("make-flight: Trim the matrix of disk format flights") > > - 5c70735f177f ("fmtarches: Use dom0arches, not hardcoded arch list") > > Probably a dummy question, but why haven't those commits changed the > output of make-flight earlier? I'm fine with the change, but I don't > really get why we are seeing it only now. Looking at the commit message, they did some changes. But the second commit might have made changes in a way that wasn't intended. For example, "xl-raw" test disappear. First commit intended to have exactly 6 jobs spread across all arches. It used as input "i386 armhf amd64 armhf" and just check that the arch corresponded to the current "$dom0arch". The function is called several time with different $dom0arch. The second mention commit changes to use $dom0arches which takes different values " i386 amd64", then " armhf" and " arm64". The extra space at the beginning of the string is why there's no more "xl-raw" tests, because now '' became one of the possible dom0arch which of course it doesn't exist. Then we have the same exact list of 3 test on armhf and arm64, which defeat the original intention of the first mention commit. So, now I feel like I need to rework do_pv_debian_tests() to at least re-add "xl-raw" test, and maybe keep only 6 toolstack-disk_format tests spread across all arches? That could maybe reduce the number of tests on arm64 which is the current bottleneck. Cheers,
diff --git a/ap-common b/ap-common index 48a95a22..5384891b 100644 --- a/ap-common +++ b/ap-common @@ -59,7 +59,7 @@ : ${PUSH_TREE_LINUX:=$XENBITS:/home/xen/git/linux-pvops.git} : ${BASE_TREE_LINUX:=git://xenbits.xen.org/linux-pvops.git} -: ${BASE_TAG_LINUX:=tested/linux-4.19} +: ${BASE_TAG_LINUX:=tested/linux-6.1} : ${BASE_TAG_LINUX_ARM:=tested/linux-6.1} if [ "x${TREE_LINUX}" = x ]; then diff --git a/mfi-common b/mfi-common index d8d167fd..e3a91e24 100644 --- a/mfi-common +++ b/mfi-common @@ -662,7 +662,7 @@ test_matrix_iterate () { case "${branch}_${dom0arch}" in linux-5.4_i386 | linux-[2-4].*_i386) ;; # keep 32-bit for old linux - linux-*_i386) continue;; + *_i386) continue;; esac dom0arches+=" $dom0arch"
linux-4.19 branch in xenbits is outdated, it haven't been updated and tested since 2020 as it has been disabled in osstest. Also, this 4.19 branch doesn't build on Bookworm. So we will start to use a newer version of Linux. We switch to 6.1 for the Arm* tests recently, so will use that same version for x86. Also, following commit 3a3089c94913 ("mfi-common: Drop Linux dom0 i386 tests for newer Linux branches"), 32bit dom0 isn't tested on newer Linux, so we need to drop all dom0 i386 tests wherever the default linux branch is used. That is, this changes in jobs will apply to "xen-unstable" branch but also all xen stable branches, seabios, qemu, osstest, libvirt, so every branch that aren't "linux-*". Here is the list jobs that changes, and whether they are replace, or have existing equivalents, on the "xen-unstable" branch. Changes compared with: OSSTEST_CONFIG=standalone-config-example nice eatmydata ./standalone-generate-dump-flight-runvars Gone, without exiting or new test-amd64-amd64-*: - test-amd64-i386-freebsd10-amd64 - test-amd64-i386-freebsd10-i386 - test-amd64-i386-qemut-rhel6hvm-amd - test-amd64-i386-qemut-rhel6hvm-intel - test-amd64-i386-qemuu-rhel6hvm-amd - test-amd64-i386-qemuu-rhel6hvm-intel Gone, but with exiting test-amd64-amd64-* equivalent: - test-amd64-coresched-i386-xl - test-amd64-i386-examine - test-amd64-i386-examine-bios - test-amd64-i386-examine-uefi - test-amd64-i386-libvirt - test-amd64-i386-libvirt-pair - test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm - test-amd64-i386-libvirt-xsm - test-amd64-i386-livepatch - test-amd64-i386-migrupgrade - test-amd64-i386-pair - test-amd64-i386-xl - test-amd64-i386-xl-pvshim - test-amd64-i386-xl-qemut-debianhvm-amd64 - test-amd64-i386-xl-qemut-debianhvm-i386-xsm - test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm - test-amd64-i386-xl-qemut-win7-amd64 - test-amd64-i386-xl-qemut-ws16-amd64 - test-amd64-i386-xl-qemuu-debianhvm-amd64 - test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow - test-amd64-i386-xl-qemuu-debianhvm-i386-xsm - test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict - test-amd64-i386-xl-qemuu-ovmf-amd64 - test-amd64-i386-xl-qemuu-win7-amd64 - test-amd64-i386-xl-qemuu-ws16-amd64 - test-amd64-i386-xl-shadow - test-amd64-i386-xl-simplat-amd64-buster - test-amd64-i386-xl-xsm Gone, but replaced by a new test-amd64-amd64-*: - test-amd64-i386-libvirt-raw - test-amd64-i386-xl-vhd Some test-amd64-amd64-* are also changed: - test-amd64-amd64-libvirt-vhd - test-amd64-amd64-qemuu-freebsd11-amd64 - test-amd64-amd64-qemuu-freebsd12-amd64 - test-amd64-amd64-xl-qcow2 + test-amd64-amd64-freebsd11-amd64 + test-amd64-amd64-freebsd12-amd64 + test-amd64-amd64-libvirt-qcow2 + test-amd64-amd64-libvirt-raw + test-amd64-amd64-xl-vhd In any case, the list of test would be the same as for the existing branch "linux-linus" or "linux-6.1" branches. Signed-off-by: Anthony PERARD <anthony.perard@citrix.com> --- ap-common | 2 +- mfi-common | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)