| Message ID | c95959adca794a90465abd10f579dc9159a7697f.1701966261.git.oleksii.kurochko@gmail.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | ARCH_FIXED_CONFIG introduction for randconfig | expand |
On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: > ARCH_FIXED_CONFIG is required in the case of randconfig > and CI for configs that aren't ready or are not > supposed to be implemented for specific architecture. > These configs should always be disabled to prevent randconfig > related tests from failing. > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > --- > xen/Makefile | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) > > diff --git a/xen/Makefile b/xen/Makefile > index ca571103c8..8ae8fe1480 100644 > --- a/xen/Makefile > +++ b/xen/Makefile > @@ -336,11 +336,14 @@ ifeq ($(config-build),y) > # *config targets only - make sure prerequisites are updated, and descend > # in tools/kconfig to make the *config target > > +ARCH_FORCED_CONFIG := $(srctree)/arch/$(SRCARCH)/configs/randomforced.config > + > # Create a file for KCONFIG_ALLCONFIG which depends on the environment. > # This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig > filechk_kconfig_allconfig = \ > $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_POLICY=n';) \ > - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ > + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ > + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat $(ARCH_FORCED_CONFIG);) ) \ > : > > .allconfig.tmp: FORCE We already have infrastructure for this. What's wrong with EXTRA_FIXED_RANDCONFIG? ---8<--- CI: Revert "automation: Drop ppc64le-*randconfig jobs", fix Randconfig with existing infrastructure This reverts commit cbb71b95dd708b1e26899bbe1e7bf9a85081fd60. Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> diff --git a/automation/gitlab-ci/build.yaml b/automation/gitlab-ci/build.yaml index 32af30ccedc9..346d0400ed09 100644 --- a/automation/gitlab-ci/build.yaml +++ b/automation/gitlab-ci/build.yaml @@ -538,6 +538,7 @@ archlinux-current-gcc-riscv64-randconfig: RANDCONFIG: y EXTRA_FIXED_RANDCONFIG: CONFIG_COVERAGE=n + CONFIG_GRANT_TABLE=n archlinux-current-gcc-riscv64-debug-randconfig: extends: .gcc-riscv64-cross-build-debug @@ -547,6 +548,7 @@ archlinux-current-gcc-riscv64-debug-randconfig: RANDCONFIG: y EXTRA_FIXED_RANDCONFIG: CONFIG_COVERAGE=n + CONFIG_GRANT_TABLE=n # Power cross-build debian-bullseye-gcc-ppc64le: @@ -563,6 +565,26 @@ debian-bullseye-gcc-ppc64le-debug: KBUILD_DEFCONFIG: ppc64_defconfig HYPERVISOR_ONLY: y +debian-bullseye-gcc-ppc64le-randconfig: + extends: .gcc-ppc64le-cross-build + variables: + CONTAINER: debian:bullseye-ppc64le + KBUILD_DEFCONFIG: ppc64_defconfig + RANDCONFIG: y + EXTRA_FIXED_RANDCONFIG: + CONFIG_COVERAGE=n + CONFIG_GRANT_TABLE=n + +debian-bullseye-gcc-ppc64le-debug-randconfig: + extends: .gcc-ppc64le-cross-build-debug + variables: + CONTAINER: debian:bullseye-ppc64le + KBUILD_DEFCONFIG: ppc64_defconfig + RANDCONFIG: y + EXTRA_FIXED_RANDCONFIG: + CONFIG_COVERAGE=n + CONFIG_GRANT_TABLE=n + # Yocto test jobs yocto-qemuarm64: extends: .yocto-test-arm64
On Thu, 2023-12-07 at 20:17 +0000, Andrew Cooper wrote: > On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: > > ARCH_FIXED_CONFIG is required in the case of randconfig > > and CI for configs that aren't ready or are not > > supposed to be implemented for specific architecture. > > These configs should always be disabled to prevent randconfig > > related tests from failing. > > > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > > --- > > xen/Makefile | 5 ++++- > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > diff --git a/xen/Makefile b/xen/Makefile > > index ca571103c8..8ae8fe1480 100644 > > --- a/xen/Makefile > > +++ b/xen/Makefile > > @@ -336,11 +336,14 @@ ifeq ($(config-build),y) > > # *config targets only - make sure prerequisites are updated, and > > descend > > # in tools/kconfig to make the *config target > > > > +ARCH_FORCED_CONFIG := > > $(srctree)/arch/$(SRCARCH)/configs/randomforced.config > > + > > # Create a file for KCONFIG_ALLCONFIG which depends on the > > environment. > > # This will be use by kconfig targets > > allyesconfig/allmodconfig/allnoconfig/randconfig > > filechk_kconfig_allconfig = \ > > $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo > > 'CONFIG_XSM_FLASK_POLICY=n';) \ > > - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ > > + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ > > + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat > > $(ARCH_FORCED_CONFIG);) ) \ > > : > > > > .allconfig.tmp: FORCE > > We already have infrastructure for this. What's wrong with > EXTRA_FIXED_RANDCONFIG? Everything is fine; I don't know why there was only an issue with CONFIG_GRANT_TABLE on PPC. On the RISC-V side, there were more configs issues, prompting me to include all the configurations not implemented for RISC-V in EXTRA_FIXED_RANDCONFIG. You can find the added configurations in this commit: https://lore.kernel.org/xen-devel/b4e85f8f58787b4d179022973ce25673d6b56e36.1700761381.git.oleksii.kurochko@gmail.com/#Z31automation:gitlab-ci:build.yaml One challenge is that the same configurations need to be added multiple times for each build test using randconfig. Another reason for this approach is a suggestion from Jan (probably I misunderstood it), who proposed using a template to instruct randconfig not to modify currently unnecessary configurations. You can find the suggestion and discussion here: https://lore.kernel.org/xen-devel/008d0c66-6816-4d12-9e1f-1878e982f9fc@suse.com/ Perhaps we could enhance the build script to fetch "fixed" configs from the architecture-specific fixed-defconfig instead of modifying the Makefile directly. > > ---8<--- > > CI: Revert "automation: Drop ppc64le-*randconfig jobs", fix > Randconfig > with existing infrastructure > > This reverts commit cbb71b95dd708b1e26899bbe1e7bf9a85081fd60. > > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> > > diff --git a/automation/gitlab-ci/build.yaml > b/automation/gitlab-ci/build.yaml > index 32af30ccedc9..346d0400ed09 100644 > --- a/automation/gitlab-ci/build.yaml > +++ b/automation/gitlab-ci/build.yaml > @@ -538,6 +538,7 @@ archlinux-current-gcc-riscv64-randconfig: > RANDCONFIG: y > EXTRA_FIXED_RANDCONFIG: > CONFIG_COVERAGE=n > + CONFIG_GRANT_TABLE=n > > archlinux-current-gcc-riscv64-debug-randconfig: > extends: .gcc-riscv64-cross-build-debug > @@ -547,6 +548,7 @@ archlinux-current-gcc-riscv64-debug-randconfig: > RANDCONFIG: y > EXTRA_FIXED_RANDCONFIG: > CONFIG_COVERAGE=n > + CONFIG_GRANT_TABLE=n > > # Power cross-build > debian-bullseye-gcc-ppc64le: > @@ -563,6 +565,26 @@ debian-bullseye-gcc-ppc64le-debug: > KBUILD_DEFCONFIG: ppc64_defconfig > HYPERVISOR_ONLY: y > > +debian-bullseye-gcc-ppc64le-randconfig: > + extends: .gcc-ppc64le-cross-build > + variables: > + CONTAINER: debian:bullseye-ppc64le > + KBUILD_DEFCONFIG: ppc64_defconfig > + RANDCONFIG: y > + EXTRA_FIXED_RANDCONFIG: > + CONFIG_COVERAGE=n > + CONFIG_GRANT_TABLE=n > + > +debian-bullseye-gcc-ppc64le-debug-randconfig: > + extends: .gcc-ppc64le-cross-build-debug > + variables: > + CONTAINER: debian:bullseye-ppc64le > + KBUILD_DEFCONFIG: ppc64_defconfig > + RANDCONFIG: y > + EXTRA_FIXED_RANDCONFIG: > + CONFIG_COVERAGE=n > + CONFIG_GRANT_TABLE=n > + > # Yocto test jobs > yocto-qemuarm64: > extends: .yocto-test-arm64 > ~ Oleksii
On Thu, 7 Dec 2023, Oleksii wrote: > On Thu, 2023-12-07 at 20:17 +0000, Andrew Cooper wrote: > > On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: > > > ARCH_FIXED_CONFIG is required in the case of randconfig > > > and CI for configs that aren't ready or are not > > > supposed to be implemented for specific architecture. > > > These configs should always be disabled to prevent randconfig > > > related tests from failing. > > > > > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > > > --- > > > xen/Makefile | 5 ++++- > > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > > > diff --git a/xen/Makefile b/xen/Makefile > > > index ca571103c8..8ae8fe1480 100644 > > > --- a/xen/Makefile > > > +++ b/xen/Makefile > > > @@ -336,11 +336,14 @@ ifeq ($(config-build),y) > > > # *config targets only - make sure prerequisites are updated, and > > > descend > > > # in tools/kconfig to make the *config target > > > > > > +ARCH_FORCED_CONFIG := > > > $(srctree)/arch/$(SRCARCH)/configs/randomforced.config > > > + > > > # Create a file for KCONFIG_ALLCONFIG which depends on the > > > environment. > > > # This will be use by kconfig targets > > > allyesconfig/allmodconfig/allnoconfig/randconfig > > > filechk_kconfig_allconfig = \ > > > $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo > > > 'CONFIG_XSM_FLASK_POLICY=n';) \ > > > - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ > > > + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ > > > + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat > > > $(ARCH_FORCED_CONFIG);) ) \ > > > : > > > > > > .allconfig.tmp: FORCE > > > > We already have infrastructure for this. What's wrong with > > EXTRA_FIXED_RANDCONFIG? > Everything is fine; I don't know why there was only an issue with > CONFIG_GRANT_TABLE on PPC. On the RISC-V side, there were more configs > issues, prompting me to include all the configurations not implemented > for RISC-V in EXTRA_FIXED_RANDCONFIG. You can find the added > configurations in this commit: > https://lore.kernel.org/xen-devel/b4e85f8f58787b4d179022973ce25673d6b56e36.1700761381.git.oleksii.kurochko@gmail.com/#Z31automation:gitlab-ci:build.yaml > > One challenge is that the same configurations need to be added multiple > times for each build test using randconfig. That's a lot of extra configs to add. Could you use a yaml anchor or a .something to include? So that you define the full list only once at the top of the file and then reuse it everywhere as needed. > Another reason for this approach is a suggestion from Jan (probably I > misunderstood it), who proposed using a template to instruct randconfig > not to modify currently unnecessary configurations. You can find the > suggestion and discussion here: > https://lore.kernel.org/xen-devel/008d0c66-6816-4d12-9e1f-1878e982f9fc@suse.com/ > > Perhaps we could enhance the build script to fetch "fixed" configs from > the architecture-specific fixed-defconfig instead of modifying the > Makefile directly. Sorry I missed the original thread somehow. Please use "automation" as subject line tag for automation patches.
On Thu, 2023-12-07 at 17:01 -0800, Stefano Stabellini wrote: > On Thu, 7 Dec 2023, Oleksii wrote: > > On Thu, 2023-12-07 at 20:17 +0000, Andrew Cooper wrote: > > > On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: > > > > ARCH_FIXED_CONFIG is required in the case of randconfig > > > > and CI for configs that aren't ready or are not > > > > supposed to be implemented for specific architecture. > > > > These configs should always be disabled to prevent randconfig > > > > related tests from failing. > > > > > > > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > > > > --- > > > > xen/Makefile | 5 ++++- > > > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > > > > > diff --git a/xen/Makefile b/xen/Makefile > > > > index ca571103c8..8ae8fe1480 100644 > > > > --- a/xen/Makefile > > > > +++ b/xen/Makefile > > > > @@ -336,11 +336,14 @@ ifeq ($(config-build),y) > > > > # *config targets only - make sure prerequisites are updated, > > > > and > > > > descend > > > > # in tools/kconfig to make the *config target > > > > > > > > +ARCH_FORCED_CONFIG := > > > > $(srctree)/arch/$(SRCARCH)/configs/randomforced.config > > > > + > > > > # Create a file for KCONFIG_ALLCONFIG which depends on the > > > > environment. > > > > # This will be use by kconfig targets > > > > allyesconfig/allmodconfig/allnoconfig/randconfig > > > > filechk_kconfig_allconfig = \ > > > > $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo > > > > 'CONFIG_XSM_FLASK_POLICY=n';) \ > > > > - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ > > > > + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ > > > > + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat > > > > $(ARCH_FORCED_CONFIG);) ) \ > > > > : > > > > > > > > .allconfig.tmp: FORCE > > > > > > We already have infrastructure for this. What's wrong with > > > EXTRA_FIXED_RANDCONFIG? > > Everything is fine; I don't know why there was only an issue with > > CONFIG_GRANT_TABLE on PPC. On the RISC-V side, there were more > > configs > > issues, prompting me to include all the configurations not > > implemented > > for RISC-V in EXTRA_FIXED_RANDCONFIG. You can find the added > > configurations in this commit: > > https://lore.kernel.org/xen-devel/b4e85f8f58787b4d179022973ce25673d6b56e36.1700761381.git.oleksii.kurochko@gmail.com/#Z31automation:gitlab-ci:build.yaml > > > > One challenge is that the same configurations need to be added > > multiple > > times for each build test using randconfig. > > That's a lot of extra configs to add. Could you use a yaml anchor or > a > .something to include? So that you define the full list only once at > the > top of the file and then reuse it everywhere as needed. I didn't know that it work. If there is such options, I think it can be a solution. Thanks. I'll try. > > > > Another reason for this approach is a suggestion from Jan (probably > > I > > misunderstood it), who proposed using a template to instruct > > randconfig > > not to modify currently unnecessary configurations. You can find > > the > > suggestion and discussion here: > > https://lore.kernel.org/xen-devel/008d0c66-6816-4d12-9e1f-1878e982f9fc@suse.com/ > > > > Perhaps we could enhance the build script to fetch "fixed" configs > > from > > the architecture-specific fixed-defconfig instead of modifying the > > Makefile directly. > > Sorry I missed the original thread somehow. Please use "automation" > as > subject line tag for automation patches. Sure. I'll do next time. ~ Oleksii
On 07.12.2023 21:17, Andrew Cooper wrote: > On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: >> ARCH_FIXED_CONFIG is required in the case of randconfig >> and CI for configs that aren't ready or are not >> supposed to be implemented for specific architecture. >> These configs should always be disabled to prevent randconfig >> related tests from failing. >> >> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> >> --- >> xen/Makefile | 5 ++++- >> 1 file changed, 4 insertions(+), 1 deletion(-) >> >> diff --git a/xen/Makefile b/xen/Makefile >> index ca571103c8..8ae8fe1480 100644 >> --- a/xen/Makefile >> +++ b/xen/Makefile >> @@ -336,11 +336,14 @@ ifeq ($(config-build),y) >> # *config targets only - make sure prerequisites are updated, and descend >> # in tools/kconfig to make the *config target >> >> +ARCH_FORCED_CONFIG := $(srctree)/arch/$(SRCARCH)/configs/randomforced.config >> + >> # Create a file for KCONFIG_ALLCONFIG which depends on the environment. >> # This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig >> filechk_kconfig_allconfig = \ >> $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_POLICY=n';) \ >> - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ >> + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ >> + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat $(ARCH_FORCED_CONFIG);) ) \ >> : >> >> .allconfig.tmp: FORCE > > We already have infrastructure for this. What's wrong with > EXTRA_FIXED_RANDCONFIG? What I don't understand here is why dealing with the issue would want limiting to gitlab-CI. Anyone could run randconfig on their own, and imo it would be helpful if the same issue(s) could be prevented there, too. Hence my earlier suggestion to have a snippet which can be used by "interested" parties. And once dealt with in e.g. the makefile there should not be a need for any overrides in the CI config anymore. Jan > ---8<--- > > CI: Revert "automation: Drop ppc64le-*randconfig jobs", fix Randconfig > with existing infrastructure > > This reverts commit cbb71b95dd708b1e26899bbe1e7bf9a85081fd60. > > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> > > diff --git a/automation/gitlab-ci/build.yaml > b/automation/gitlab-ci/build.yaml > index 32af30ccedc9..346d0400ed09 100644 > --- a/automation/gitlab-ci/build.yaml > +++ b/automation/gitlab-ci/build.yaml > @@ -538,6 +538,7 @@ archlinux-current-gcc-riscv64-randconfig: > RANDCONFIG: y > EXTRA_FIXED_RANDCONFIG: > CONFIG_COVERAGE=n > + CONFIG_GRANT_TABLE=n > > archlinux-current-gcc-riscv64-debug-randconfig: > extends: .gcc-riscv64-cross-build-debug > @@ -547,6 +548,7 @@ archlinux-current-gcc-riscv64-debug-randconfig: > RANDCONFIG: y > EXTRA_FIXED_RANDCONFIG: > CONFIG_COVERAGE=n > + CONFIG_GRANT_TABLE=n > > # Power cross-build > debian-bullseye-gcc-ppc64le: > @@ -563,6 +565,26 @@ debian-bullseye-gcc-ppc64le-debug: > KBUILD_DEFCONFIG: ppc64_defconfig > HYPERVISOR_ONLY: y > > +debian-bullseye-gcc-ppc64le-randconfig: > + extends: .gcc-ppc64le-cross-build > + variables: > + CONTAINER: debian:bullseye-ppc64le > + KBUILD_DEFCONFIG: ppc64_defconfig > + RANDCONFIG: y > + EXTRA_FIXED_RANDCONFIG: > + CONFIG_COVERAGE=n > + CONFIG_GRANT_TABLE=n > + > +debian-bullseye-gcc-ppc64le-debug-randconfig: > + extends: .gcc-ppc64le-cross-build-debug > + variables: > + CONTAINER: debian:bullseye-ppc64le > + KBUILD_DEFCONFIG: ppc64_defconfig > + RANDCONFIG: y > + EXTRA_FIXED_RANDCONFIG: > + CONFIG_COVERAGE=n > + CONFIG_GRANT_TABLE=n > + > # Yocto test jobs > yocto-qemuarm64: > extends: .yocto-test-arm64 >
On Mon, 2023-12-11 at 16:56 +0100, Jan Beulich wrote: > On 07.12.2023 21:17, Andrew Cooper wrote: > > On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: > > > ARCH_FIXED_CONFIG is required in the case of randconfig > > > and CI for configs that aren't ready or are not > > > supposed to be implemented for specific architecture. > > > These configs should always be disabled to prevent randconfig > > > related tests from failing. > > > > > > Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > > > --- > > > xen/Makefile | 5 ++++- > > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > > > diff --git a/xen/Makefile b/xen/Makefile > > > index ca571103c8..8ae8fe1480 100644 > > > --- a/xen/Makefile > > > +++ b/xen/Makefile > > > @@ -336,11 +336,14 @@ ifeq ($(config-build),y) > > > # *config targets only - make sure prerequisites are updated, > > > and descend > > > # in tools/kconfig to make the *config target > > > > > > +ARCH_FORCED_CONFIG := > > > $(srctree)/arch/$(SRCARCH)/configs/randomforced.config > > > + > > > # Create a file for KCONFIG_ALLCONFIG which depends on the > > > environment. > > > # This will be use by kconfig targets > > > allyesconfig/allmodconfig/allnoconfig/randconfig > > > filechk_kconfig_allconfig = \ > > > $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo > > > 'CONFIG_XSM_FLASK_POLICY=n';) \ > > > - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ > > > + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ > > > + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat > > > $(ARCH_FORCED_CONFIG);) ) \ > > > : > > > > > > .allconfig.tmp: FORCE > > > > We already have infrastructure for this. What's wrong with > > EXTRA_FIXED_RANDCONFIG? > > What I don't understand here is why dealing with the issue would want > limiting to gitlab-CI. Anyone could run randconfig on their own, and > imo it would be helpful if the same issue(s) could be prevented > there, > too. Hence my earlier suggestion to have a snippet which can be used > by "interested" parties. And once dealt with in e.g. the makefile > there should not be a need for any overrides in the CI config > anymore. I agree with Jan's point of view that having a more universal solution for cases where it is necessary to disable a config for randconfig would be beneficial. Sometimes, I test randconfig locally, especially during the RISC-V Xen full build patch series where numerous configs are disabled. It would be convenient to have only one place to disable configs instead of duplicating them in two different places. Does anyone have any objections? ~ Oleksii > > > ---8<--- > > > > CI: Revert "automation: Drop ppc64le-*randconfig jobs", fix > > Randconfig > > with existing infrastructure > > > > This reverts commit cbb71b95dd708b1e26899bbe1e7bf9a85081fd60. > > > > Signed-off-by: Andrew Cooper <andrew.cooper3@citrix.com> > > > > diff --git a/automation/gitlab-ci/build.yaml > > b/automation/gitlab-ci/build.yaml > > index 32af30ccedc9..346d0400ed09 100644 > > --- a/automation/gitlab-ci/build.yaml > > +++ b/automation/gitlab-ci/build.yaml > > @@ -538,6 +538,7 @@ archlinux-current-gcc-riscv64-randconfig: > > RANDCONFIG: y > > EXTRA_FIXED_RANDCONFIG: > > CONFIG_COVERAGE=n > > + CONFIG_GRANT_TABLE=n > > > > archlinux-current-gcc-riscv64-debug-randconfig: > > extends: .gcc-riscv64-cross-build-debug > > @@ -547,6 +548,7 @@ archlinux-current-gcc-riscv64-debug-randconfig: > > RANDCONFIG: y > > EXTRA_FIXED_RANDCONFIG: > > CONFIG_COVERAGE=n > > + CONFIG_GRANT_TABLE=n > > > > # Power cross-build > > debian-bullseye-gcc-ppc64le: > > @@ -563,6 +565,26 @@ debian-bullseye-gcc-ppc64le-debug: > > KBUILD_DEFCONFIG: ppc64_defconfig > > HYPERVISOR_ONLY: y > > > > +debian-bullseye-gcc-ppc64le-randconfig: > > + extends: .gcc-ppc64le-cross-build > > + variables: > > + CONTAINER: debian:bullseye-ppc64le > > + KBUILD_DEFCONFIG: ppc64_defconfig > > + RANDCONFIG: y > > + EXTRA_FIXED_RANDCONFIG: > > + CONFIG_COVERAGE=n > > + CONFIG_GRANT_TABLE=n > > + > > +debian-bullseye-gcc-ppc64le-debug-randconfig: > > + extends: .gcc-ppc64le-cross-build-debug > > + variables: > > + CONTAINER: debian:bullseye-ppc64le > > + KBUILD_DEFCONFIG: ppc64_defconfig > > + RANDCONFIG: y > > + EXTRA_FIXED_RANDCONFIG: > > + CONFIG_COVERAGE=n > > + CONFIG_GRANT_TABLE=n > > + > > # Yocto test jobs > > yocto-qemuarm64: > > extends: .yocto-test-arm64 > > >
On 11/12/2023 3:56 pm, Jan Beulich wrote: > On 07.12.2023 21:17, Andrew Cooper wrote: >> On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: >>> ARCH_FIXED_CONFIG is required in the case of randconfig >>> and CI for configs that aren't ready or are not >>> supposed to be implemented for specific architecture. >>> These configs should always be disabled to prevent randconfig >>> related tests from failing. >>> >>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> >>> --- >>> xen/Makefile | 5 ++++- >>> 1 file changed, 4 insertions(+), 1 deletion(-) >>> >>> diff --git a/xen/Makefile b/xen/Makefile >>> index ca571103c8..8ae8fe1480 100644 >>> --- a/xen/Makefile >>> +++ b/xen/Makefile >>> @@ -336,11 +336,14 @@ ifeq ($(config-build),y) >>> # *config targets only - make sure prerequisites are updated, and descend >>> # in tools/kconfig to make the *config target >>> >>> +ARCH_FORCED_CONFIG := $(srctree)/arch/$(SRCARCH)/configs/randomforced.config >>> + >>> # Create a file for KCONFIG_ALLCONFIG which depends on the environment. >>> # This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig >>> filechk_kconfig_allconfig = \ >>> $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_POLICY=n';) \ >>> - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ >>> + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ >>> + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat $(ARCH_FORCED_CONFIG);) ) \ >>> : >>> >>> .allconfig.tmp: FORCE >> We already have infrastructure for this. What's wrong with >> EXTRA_FIXED_RANDCONFIG? > What I don't understand here is why dealing with the issue would want > limiting to gitlab-CI. Anyone could run randconfig on their own, and > imo it would be helpful if the same issue(s) could be prevented there, > too. Hence my earlier suggestion to have a snippet which can be used > by "interested" parties. And once dealt with in e.g. the makefile > there should not be a need for any overrides in the CI config anymore. This is trying to find a solution to a problem which doesn't exist. RISC-V and PPC are experimental in Xen. Noone else is going to come and randconfig them until they're rather more production ready, and a prerequisite of that is removing this list of exclusions. Until you can actually find an interested party to comment, I think this is just churn for no useful improvement. If nothing else, calling it randomforced.config isn't appropriate given the explanation of what this target is used for... ~Andrew
On 18.12.2023 17:07, Andrew Cooper wrote: > On 11/12/2023 3:56 pm, Jan Beulich wrote: >> On 07.12.2023 21:17, Andrew Cooper wrote: >>> On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: >>>> ARCH_FIXED_CONFIG is required in the case of randconfig >>>> and CI for configs that aren't ready or are not >>>> supposed to be implemented for specific architecture. >>>> These configs should always be disabled to prevent randconfig >>>> related tests from failing. >>>> >>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> >>>> --- >>>> xen/Makefile | 5 ++++- >>>> 1 file changed, 4 insertions(+), 1 deletion(-) >>>> >>>> diff --git a/xen/Makefile b/xen/Makefile >>>> index ca571103c8..8ae8fe1480 100644 >>>> --- a/xen/Makefile >>>> +++ b/xen/Makefile >>>> @@ -336,11 +336,14 @@ ifeq ($(config-build),y) >>>> # *config targets only - make sure prerequisites are updated, and descend >>>> # in tools/kconfig to make the *config target >>>> >>>> +ARCH_FORCED_CONFIG := $(srctree)/arch/$(SRCARCH)/configs/randomforced.config >>>> + >>>> # Create a file for KCONFIG_ALLCONFIG which depends on the environment. >>>> # This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig >>>> filechk_kconfig_allconfig = \ >>>> $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_POLICY=n';) \ >>>> - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ >>>> + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ >>>> + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat $(ARCH_FORCED_CONFIG);) ) \ >>>> : >>>> >>>> .allconfig.tmp: FORCE >>> We already have infrastructure for this. What's wrong with >>> EXTRA_FIXED_RANDCONFIG? >> What I don't understand here is why dealing with the issue would want >> limiting to gitlab-CI. Anyone could run randconfig on their own, and >> imo it would be helpful if the same issue(s) could be prevented there, >> too. Hence my earlier suggestion to have a snippet which can be used >> by "interested" parties. And once dealt with in e.g. the makefile >> there should not be a need for any overrides in the CI config anymore. > > This is trying to find a solution to a problem which doesn't exist. > > RISC-V and PPC are experimental in Xen. Noone else is going to come and > randconfig them until they're rather more production ready, and a > prerequisite of that is removing this list of exclusions. > > Until you can actually find an interested party to comment, I think this > is just churn for no useful improvement. If nothing else, calling it > randomforced.config isn't appropriate given the explanation of what this > target is used for... "random" in the name can't possibly be right anyway. Such collection of fixed settings would also be relevant to e.g. all{yes,no}config. Yet that's still not the same as any kind of "default" config, which the two architectures presently kind of abuse for the purpose of defining required-fixed settings. Jan
On Mon, 18 Dec 2023, Jan Beulich wrote: > On 18.12.2023 17:07, Andrew Cooper wrote: > > On 11/12/2023 3:56 pm, Jan Beulich wrote: > >> On 07.12.2023 21:17, Andrew Cooper wrote: > >>> On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: > >>>> ARCH_FIXED_CONFIG is required in the case of randconfig > >>>> and CI for configs that aren't ready or are not > >>>> supposed to be implemented for specific architecture. > >>>> These configs should always be disabled to prevent randconfig > >>>> related tests from failing. > >>>> > >>>> Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> > >>>> --- > >>>> xen/Makefile | 5 ++++- > >>>> 1 file changed, 4 insertions(+), 1 deletion(-) > >>>> > >>>> diff --git a/xen/Makefile b/xen/Makefile > >>>> index ca571103c8..8ae8fe1480 100644 > >>>> --- a/xen/Makefile > >>>> +++ b/xen/Makefile > >>>> @@ -336,11 +336,14 @@ ifeq ($(config-build),y) > >>>> # *config targets only - make sure prerequisites are updated, and descend > >>>> # in tools/kconfig to make the *config target > >>>> > >>>> +ARCH_FORCED_CONFIG := $(srctree)/arch/$(SRCARCH)/configs/randomforced.config > >>>> + > >>>> # Create a file for KCONFIG_ALLCONFIG which depends on the environment. > >>>> # This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig > >>>> filechk_kconfig_allconfig = \ > >>>> $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_POLICY=n';) \ > >>>> - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ > >>>> + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ > >>>> + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat $(ARCH_FORCED_CONFIG);) ) \ > >>>> : > >>>> > >>>> .allconfig.tmp: FORCE > >>> We already have infrastructure for this. What's wrong with > >>> EXTRA_FIXED_RANDCONFIG? > >> What I don't understand here is why dealing with the issue would want > >> limiting to gitlab-CI. Anyone could run randconfig on their own, and > >> imo it would be helpful if the same issue(s) could be prevented there, > >> too. Hence my earlier suggestion to have a snippet which can be used > >> by "interested" parties. And once dealt with in e.g. the makefile > >> there should not be a need for any overrides in the CI config anymore. > > > > This is trying to find a solution to a problem which doesn't exist. > > > > RISC-V and PPC are experimental in Xen. Noone else is going to come and > > randconfig them until they're rather more production ready, and a > > prerequisite of that is removing this list of exclusions. > > > > Until you can actually find an interested party to comment, I think this > > is just churn for no useful improvement. If nothing else, calling it > > randomforced.config isn't appropriate given the explanation of what this > > target is used for... > > "random" in the name can't possibly be right anyway. Such collection of > fixed settings would also be relevant to e.g. all{yes,no}config. Yet > that's still not the same as any kind of "default" config, which the > two architectures presently kind of abuse for the purpose of defining > required-fixed settings. One thing for sure, I don't think it would be a good idea to add extra temporary Kconfig changes like these: [1] https://lore.kernel.org/xen-devel/cdc20255540a66ba0b6946ac6d48c11029cd3385.1701453087.git.oleksii.kurochko@gmail.com/ [2] https://lore.kernel.org/xen-devel/d42a34866edc70a12736b5c6976aa1b44b4ebd8a.1701453087.git.oleksii.kurochko@gmail.com/ I agree with Andrew that RISC-V and PPC are experimental so whatever works to enable them to make progress on this issue with a small effort is sufficient. I would be happy with a quick respin of this series following the gitlab-ci approach. This is good enough. And I think that having some sort of fixed seed (seed.config?) for randconfig would also be fine and potentially more reusable. But I think Oleksii should go forward with whatever approach he prefers and he is more comfortable with, as long as it is not [1] and [2].
On Mon, 2023-12-18 at 18:03 -0800, Stefano Stabellini wrote: > On Mon, 18 Dec 2023, Jan Beulich wrote: > > On 18.12.2023 17:07, Andrew Cooper wrote: > > > On 11/12/2023 3:56 pm, Jan Beulich wrote: > > > > On 07.12.2023 21:17, Andrew Cooper wrote: > > > > > On 07/12/2023 5:03 pm, Oleksii Kurochko wrote: > > > > > > ARCH_FIXED_CONFIG is required in the case of randconfig > > > > > > and CI for configs that aren't ready or are not > > > > > > supposed to be implemented for specific architecture. > > > > > > These configs should always be disabled to prevent > > > > > > randconfig > > > > > > related tests from failing. > > > > > > > > > > > > Signed-off-by: Oleksii Kurochko > > > > > > <oleksii.kurochko@gmail.com> > > > > > > --- > > > > > > xen/Makefile | 5 ++++- > > > > > > 1 file changed, 4 insertions(+), 1 deletion(-) > > > > > > > > > > > > diff --git a/xen/Makefile b/xen/Makefile > > > > > > index ca571103c8..8ae8fe1480 100644 > > > > > > --- a/xen/Makefile > > > > > > +++ b/xen/Makefile > > > > > > @@ -336,11 +336,14 @@ ifeq ($(config-build),y) > > > > > > # *config targets only - make sure prerequisites are > > > > > > updated, and descend > > > > > > # in tools/kconfig to make the *config target > > > > > > > > > > > > +ARCH_FORCED_CONFIG := > > > > > > $(srctree)/arch/$(SRCARCH)/configs/randomforced.config > > > > > > + > > > > > > # Create a file for KCONFIG_ALLCONFIG which depends on the > > > > > > environment. > > > > > > # This will be use by kconfig targets > > > > > > allyesconfig/allmodconfig/allnoconfig/randconfig > > > > > > filechk_kconfig_allconfig = \ > > > > > > $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo > > > > > > 'CONFIG_XSM_FLASK_POLICY=n';) \ > > > > > > - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) > > > > > > \ > > > > > > + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ > > > > > > + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat > > > > > > $(ARCH_FORCED_CONFIG);) ) \ > > > > > > : > > > > > > > > > > > > .allconfig.tmp: FORCE > > > > > We already have infrastructure for this. What's wrong with > > > > > EXTRA_FIXED_RANDCONFIG? > > > > What I don't understand here is why dealing with the issue > > > > would want > > > > limiting to gitlab-CI. Anyone could run randconfig on their > > > > own, and > > > > imo it would be helpful if the same issue(s) could be prevented > > > > there, > > > > too. Hence my earlier suggestion to have a snippet which can be > > > > used > > > > by "interested" parties. And once dealt with in e.g. the > > > > makefile > > > > there should not be a need for any overrides in the CI config > > > > anymore. > > > > > > This is trying to find a solution to a problem which doesn't > > > exist. > > > > > > RISC-V and PPC are experimental in Xen. Noone else is going to > > > come and > > > randconfig them until they're rather more production ready, and a > > > prerequisite of that is removing this list of exclusions. > > > > > > Until you can actually find an interested party to comment, I > > > think this > > > is just churn for no useful improvement. If nothing else, > > > calling it > > > randomforced.config isn't appropriate given the explanation of > > > what this > > > target is used for... > > > > "random" in the name can't possibly be right anyway. Such > > collection of > > fixed settings would also be relevant to e.g. all{yes,no}config. > > Yet > > that's still not the same as any kind of "default" config, which > > the > > two architectures presently kind of abuse for the purpose of > > defining > > required-fixed settings. > > One thing for sure, I don't think it would be a good idea to add > extra > temporary Kconfig changes like these: > > [1] > https://lore.kernel.org/xen-devel/cdc20255540a66ba0b6946ac6d48c11029cd3385.1701453087.git.oleksii.kurochko@gmail.com/ > [2] > https://lore.kernel.org/xen-devel/d42a34866edc70a12736b5c6976aa1b44b4ebd8a.1701453087.git.oleksii.kurochko@gmail.com/ > > > I agree with Andrew that RISC-V and PPC are experimental so whatever > works to enable them to make progress on this issue with a small > effort > is sufficient. I would be happy with a quick respin of this series > following the gitlab-ci approach. This is good enough. > > And I think that having some sort of fixed seed (seed.config?) for > randconfig would also be fine and potentially more reusable. > > But I think Oleksii should go forward with whatever approach he > prefers > and he is more comfortable with, as long as it is not [1] and [2]. Despite the fact that I am still comfortable with both approaches, let's stick to the approach that requires minimal effort. For the time being, let's update patch series [1] with the GitLab CI approach and revert to the current patch if there are more cases where these changes are required. Thank you, everyone, for your feedback. [1] https://lore.kernel.org/xen-devel/cover.1701453087.git.oleksii.kurochko@gmail.com/
diff --git a/xen/Makefile b/xen/Makefile index ca571103c8..8ae8fe1480 100644 --- a/xen/Makefile +++ b/xen/Makefile @@ -336,11 +336,14 @@ ifeq ($(config-build),y) # *config targets only - make sure prerequisites are updated, and descend # in tools/kconfig to make the *config target +ARCH_FORCED_CONFIG := $(srctree)/arch/$(SRCARCH)/configs/randomforced.config + # Create a file for KCONFIG_ALLCONFIG which depends on the environment. # This will be use by kconfig targets allyesconfig/allmodconfig/allnoconfig/randconfig filechk_kconfig_allconfig = \ $(if $(findstring n,$(XEN_HAS_CHECKPOLICY)), echo 'CONFIG_XSM_FLASK_POLICY=n';) \ - $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG);) \ + $(if $(KCONFIG_ALLCONFIG), cat $(KCONFIG_ALLCONFIG); \ + $(if $(wildcard $(ARCH_FORCED_CONFIG)), cat $(ARCH_FORCED_CONFIG);) ) \ : .allconfig.tmp: FORCE
ARCH_FIXED_CONFIG is required in the case of randconfig and CI for configs that aren't ready or are not supposed to be implemented for specific architecture. These configs should always be disabled to prevent randconfig related tests from failing. Signed-off-by: Oleksii Kurochko <oleksii.kurochko@gmail.com> --- xen/Makefile | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)