| Message ID | 20221201002654.2238906-1-vt@altlinux.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [ima-evm-utils] Experimental fsverity.test related GA CI improvements | expand |
Mimi, On Thu, Dec 01, 2022 at 03:26:54AM +0300, Vitaly Chikunov wrote: > From: Mimi Zohar <zohar@linux.ibm.com> Forgot to --reset-author. Thanks, > > This does not make fsverity.test working on GA CI, though. > > - `--device /dev/loop-control' is required for losetup(8) to work. > - `--privileged' is required foo mount(8) to work, and this makes > `--security-opt seccomp=unconfined' redundant. > - GA container does not have `/sys/kernel/security' mounted which is > needed for `/sys/kernel/security/integrity/ima/policy'. > - Enable `set -x` in CI as the logs is everything we have to analyze on > failures. > > Signed-off-by: Vitaly Chikunov <vt@altlinux.org> > --- > .github/workflows/ci.yml | 2 +- > build.sh | 12 +++++++++++- > tests/fsverity.test | 2 +- > 3 files changed, 13 insertions(+), 3 deletions(-) > > diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml > index 8223b87..d2afdfe 100644 > --- a/.github/workflows/ci.yml > +++ b/.github/workflows/ci.yml > @@ -98,7 +98,7 @@ jobs: > container: > image: ${{ matrix.container }} > env: ${{ matrix.env }} > - options: --security-opt seccomp=unconfined > + options: --privileged --device /dev/loop-control > > steps: > - name: Show OS > diff --git a/build.sh b/build.sh > index cc5a258..4e2f1bb 100755 > --- a/build.sh > +++ b/build.sh > @@ -1,6 +1,16 @@ > #!/bin/sh > # Copyright (c) 2020 Petr Vorel <pvorel@suse.cz> > > +if [ -n "$CI" ]; then > + # If we under CI only thing we can analyze is logs so better to enable > + # verbosity to a maximum. > + set -x > + # This is to make stdout and stderr synchronous in the logs. > + exec 2>&1 > + > + mount -t securityfs -o rw securityfs /sys/kernel/security > +fi > + > set -e > > CC="${CC:-gcc}" > @@ -100,7 +110,7 @@ if [ $ret -eq 0 ]; then > tail -20 tests/boot_aggregate.log > > if [ -f tests/fsverity.log ]; then > - tail -4 tests/fsverity.log > + [ -n "$CI" ] && cat tests/fsverity.log || tail tests/fsverity.log > grep "skipped" tests/fsverity.log && \ > grep "skipped" tests/fsverity.log | wc -l > fi > diff --git a/tests/fsverity.test b/tests/fsverity.test > index def06f8..1bb8362 100755 > --- a/tests/fsverity.test > +++ b/tests/fsverity.test > @@ -78,7 +78,7 @@ mount_loopback_file() { > exit "$FAIL" > fi > > - mount -o loop ${TST_IMG} $TST_MNT > + mount -v -o loop ${TST_IMG} $TST_MNT > ret=$? > > if [ "${ret}" -eq 0 ]; then > -- > 2.33.4
Hi Vitaly, On Thu, 2022-12-01 at 03:26 +0300, Vitaly Chikunov wrote: > From: Mimi Zohar <zohar@linux.ibm.com> > > This does not make fsverity.test working on GA CI, though. > > - `--device /dev/loop-control' is required for losetup(8) to work. > - `--privileged' is required foo mount(8) to work, and this makes > `--security-opt seccomp=unconfined' redundant. > - GA container does not have `/sys/kernel/security' mounted which is > needed for `/sys/kernel/security/integrity/ima/policy'. > - Enable `set -x` in CI as the logs is everything we have to analyze on > failures. > Agreed, even with these changes the fsverity test will not be executed, but skipped. However, the reason for them being skipped is totally different than prior to this patch. Once the distros have enabled both fsverity support and are running a recent enough kernel with IMA support for fsverity, the fsverity test should succeed. So the problem isn't the GitHub actions architecture or the fsverity test itself, but the lack of IMA kernel support for it. In addition to the ima-evm-utils distro tests, there needs to be a way for testing new kernel integrity features. Roberto's proposed ima-evm-utils UML patch set downloads and uses a UML kernel for this purpose. Unless someone can recommend a better alternative, a single UML "distro" test could be defined and would be executed if a UML kernel is supplied. Additional UML tests could be specified. thanks, Mimi
Mimi, On Mon, Dec 05, 2022 at 08:39:32AM -0500, Mimi Zohar wrote: > > On Thu, 2022-12-01 at 03:26 +0300, Vitaly Chikunov wrote: > > From: Mimi Zohar <zohar@linux.ibm.com> > > > > This does not make fsverity.test working on GA CI, though. > > > > - `--device /dev/loop-control' is required for losetup(8) to work. > > - `--privileged' is required foo mount(8) to work, and this makes > > `--security-opt seccomp=unconfined' redundant. > > - GA container does not have `/sys/kernel/security' mounted which is > > needed for `/sys/kernel/security/integrity/ima/policy'. > > - Enable `set -x` in CI as the logs is everything we have to analyze on > > failures. > > > > Agreed, even with these changes the fsverity test will not be executed, > but skipped. > > However, the reason for them being skipped is totally different than > prior to this patch. Once the distros have enabled both fsverity > support and are running a recent enough kernel with IMA support for > fsverity, the fsverity test should succeed. > > So the problem isn't the GitHub actions architecture or the fsverity > test itself, but the lack of IMA kernel support for it. In addition to > the ima-evm-utils distro tests, there needs to be a way for testing new > kernel integrity features. Roberto's proposed ima-evm-utils UML patch > set downloads and uses a UML kernel for this purpose. > > Unless someone can recommend a better alternative, a single UML > "distro" test could be defined and would be executed if a UML kernel is > supplied. Additional UML tests could be specified. Just as an idea. I did some CI testing for LKRG on GA, https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.yml https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.sh It's possible to boot in QEMU system created in Docker (alas without KVM as GA does not support it). But this will install distribution's kernel. So it would need to find distribution with the appropriate kernel. Also, GA have cache functionality, so there could be dependent job to build the kernel with required options and then save it into a cache (to save time, bandwidth, and CPU resources). And another possibility is, instead of using Docker it's possible to use cloud images that many distributions have, and then same as with docker (install or build kernel, save into cache and use in next CI runs). Never tried this method myself. AFAIK this will require to use cloud-init to set up system on first boot. Thanks, > > thanks, > > Mimi
On Mon, 2022-12-05 at 17:44 +0300, Vitaly Chikunov wrote: > Mimi, > > On Mon, Dec 05, 2022 at 08:39:32AM -0500, Mimi Zohar wrote: > > On Thu, 2022-12-01 at 03:26 +0300, Vitaly Chikunov wrote: > > > From: Mimi Zohar <zohar@linux.ibm.com> > > > > > > This does not make fsverity.test working on GA CI, though. > > > > > > - `--device /dev/loop-control' is required for losetup(8) to work. > > > - `--privileged' is required foo mount(8) to work, and this makes > > > `--security-opt seccomp=unconfined' redundant. > > > - GA container does not have `/sys/kernel/security' mounted which is > > > needed for `/sys/kernel/security/integrity/ima/policy'. > > > - Enable `set -x` in CI as the logs is everything we have to analyze on > > > failures. > > > > > > > Agreed, even with these changes the fsverity test will not be executed, > > but skipped. > > > > However, the reason for them being skipped is totally different than > > prior to this patch. Once the distros have enabled both fsverity > > support and are running a recent enough kernel with IMA support for > > fsverity, the fsverity test should succeed. > > > > So the problem isn't the GitHub actions architecture or the fsverity > > test itself, but the lack of IMA kernel support for it. In addition to > > the ima-evm-utils distro tests, there needs to be a way for testing new > > kernel integrity features. Roberto's proposed ima-evm-utils UML patch > > set downloads and uses a UML kernel for this purpose. > > > > Unless someone can recommend a better alternative, a single UML > > "distro" test could be defined and would be executed if a UML kernel is > > supplied. Additional UML tests could be specified. > > Just as an idea. I did some CI testing for LKRG on GA, > https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.yml > https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.sh > > It's possible to boot in QEMU system created in Docker (alas without > KVM as GA does not support it). But this will install distribution's kernel. > So it would need to find distribution with the appropriate kernel. Uhm, yes. It is an option, but I guess it requires to build the image you want to boot the virtual machine with. Until now, the UML kernel was more than sufficient to test a new functionality or bug fixes. > Also, GA have cache functionality, so there could be dependent job > to build the kernel with required options and then save it into a cache > (to save time, bandwidth, and CPU resources). Yes, automatizing this step would be better. Currently, I have two separate repos for the kernel and ima-evm-utils. I build the kernel first, to create a release (kernel binary + signing key) and then ima- evm-utils, which fetches the kernel to test. I do these two steps separately, but I guess you can launch a kernel build from ima-evm- utils. > And another possibility is, instead of using Docker it's possible to use > cloud images that many distributions have, and then same as with docker > (install or build kernel, save into cache and use in next CI runs). > Never tried this method myself. AFAIK this will require to use cloud-init > to set up system on first boot. One cool thing about the UML kernel approach is that the script detects if it exists, and if not runs the test in the current environment. The modifications to support both I guess are not too invasive. Another thing that I like about this approach is that I can launch the UML kernel several times in the same script, and each time I execute different tests (useful for example if you need a specific EVM mode, as you cannot go back). That could be done with the virtual machine approach too, passing the necessary environment variables. It could however require some effort (I'm using UML kernels in Github Actions for a while). These are the two tests for the latest patch set I developed: https://github.com/robertosassu/linux/actions/runs/3591117649 https://github.com/robertosassu/ima-evm-utils/actions/runs/3590837109 Roberto
Hi Vitaly, On Mon, 2022-12-05 at 17:44 +0300, Vitaly Chikunov wrote: > On Mon, Dec 05, 2022 at 08:39:32AM -0500, Mimi Zohar wrote: > > > > On Thu, 2022-12-01 at 03:26 +0300, Vitaly Chikunov wrote: > > > From: Mimi Zohar <zohar@linux.ibm.com> > > > > > > This does not make fsverity.test working on GA CI, though. > > > > > > - `--device /dev/loop-control' is required for losetup(8) to work. > > > - `--privileged' is required foo mount(8) to work, and this makes > > > `--security-opt seccomp=unconfined' redundant. > > > - GA container does not have `/sys/kernel/security' mounted which is > > > needed for `/sys/kernel/security/integrity/ima/policy'. > > > - Enable `set -x` in CI as the logs is everything we have to analyze on > > > failures. > > > > > > > Agreed, even with these changes the fsverity test will not be executed, > > but skipped. > > > > However, the reason for them being skipped is totally different than > > prior to this patch. Once the distros have enabled both fsverity > > support and are running a recent enough kernel with IMA support for > > fsverity, the fsverity test should succeed. > > > > So the problem isn't the GitHub actions architecture or the fsverity > > test itself, but the lack of IMA kernel support for it. In addition to > > the ima-evm-utils distro tests, there needs to be a way for testing new > > kernel integrity features. Roberto's proposed ima-evm-utils UML patch > > set downloads and uses a UML kernel for this purpose. > > > > Unless someone can recommend a better alternative, a single UML > > "distro" test could be defined and would be executed if a UML kernel is > > supplied. Additional UML tests could be specified. > > Just as an idea. I did some CI testing for LKRG on GA, > https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.yml > https://github.com/lkrg-org/lkrg/blob/main/.github/workflows/docker-boot.sh > > It's possible to boot in QEMU system created in Docker (alas without > KVM as GA does not support it). But this will install distribution's kernel. > So it would need to find distribution with the appropriate kernel. > > Also, GA have cache functionality, so there could be dependent job > to build the kernel with required options and then save it into a cache > (to save time, bandwidth, and CPU resources). > > And another possibility is, instead of using Docker it's possible to use > cloud images that many distributions have, and then same as with docker > (install or build kernel, save into cache and use in next CI runs). > Never tried this method myself. AFAIK this will require to use cloud-init > to set up system on first boot. Roberto's v3 "Support testing in new enviroments" patch adds the UML support, but leaves open the option for using other environments like virtual machines. With the support for building a UML kernel with the appropriate Kconfig options, the fsverity.test is now working properly. I just posted "ci: cleanup build.sh test log output". With these changes, I'd appreciate your updating this patch accordingly.
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 8223b87..d2afdfe 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -98,7 +98,7 @@ jobs: container: image: ${{ matrix.container }} env: ${{ matrix.env }} - options: --security-opt seccomp=unconfined + options: --privileged --device /dev/loop-control steps: - name: Show OS diff --git a/build.sh b/build.sh index cc5a258..4e2f1bb 100755 --- a/build.sh +++ b/build.sh @@ -1,6 +1,16 @@ #!/bin/sh # Copyright (c) 2020 Petr Vorel <pvorel@suse.cz> +if [ -n "$CI" ]; then + # If we under CI only thing we can analyze is logs so better to enable + # verbosity to a maximum. + set -x + # This is to make stdout and stderr synchronous in the logs. + exec 2>&1 + + mount -t securityfs -o rw securityfs /sys/kernel/security +fi + set -e CC="${CC:-gcc}" @@ -100,7 +110,7 @@ if [ $ret -eq 0 ]; then tail -20 tests/boot_aggregate.log if [ -f tests/fsverity.log ]; then - tail -4 tests/fsverity.log + [ -n "$CI" ] && cat tests/fsverity.log || tail tests/fsverity.log grep "skipped" tests/fsverity.log && \ grep "skipped" tests/fsverity.log | wc -l fi diff --git a/tests/fsverity.test b/tests/fsverity.test index def06f8..1bb8362 100755 --- a/tests/fsverity.test +++ b/tests/fsverity.test @@ -78,7 +78,7 @@ mount_loopback_file() { exit "$FAIL" fi - mount -o loop ${TST_IMG} $TST_MNT + mount -v -o loop ${TST_IMG} $TST_MNT ret=$? if [ "${ret}" -eq 0 ]; then