| Message ID | 20200520163421.27965-2-william.c.roberts@intel.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
| Series | [v2] ci: run SELinux kernel test suite | expand |
On Wed, May 20, 2020 at 6:34 PM <bill.c.roberts@gmail.com> wrote: > From: William Roberts <william.c.roberts@intel.com> > > The current Travis CI runs the userspace tooling and libraries against > policy files, but cannot test against an SELinux enabled kernel. Thus, > some tests are not being done in the CI. Travis, unfortunately only > provides Ubuntu images, so in order to run against a modern distro with > SELinux in enforcing mode, we need to launch a KVM with something like > Fedora. > > This patch enables this support by launching a Fedora32 Cloud Image with > the SELinux userspace library passed on from the Travis clone, it then > builds and replaces the current SELinux bits on the Fedora32 image and > runs the SELinux testsuite. > > Signed-off-by: William Roberts <william.c.roberts@intel.com> From the text above I infer that this patch is intended against the userspace repo, right? If so, I don't quite see the usefulness of running the selinux-testsuite on every userspace change... It is mainly intended for testing the kernel and only a small part of its running time is spent on running (i.e. testing in a sense) the SELinux userspace programs. Not to mention that in your patch it runs with the userspace shipped in Fedora and not the version from the given commit... However, it could be very useful if this was added to the testsuite's CI instead so that it can verify that the testsuite patches don't break something. But note that you'd need to modify the script a bit to copy over the testsuite snapshot being tested to the VM and run that, instead of the current master. Anyway, thank you for working on this! I never realized that it could be so easy to run a Fedora VM from Travis. If I find some time maybe I can find some more ways to use this... For example we could run the Fedora/RHEL SELinux userspace tests from [1] after installing (not yet sure how) the userspace built from the currently tested userspace repo commit. [1] https://src.fedoraproject.org/tests/selinux/tree/master
On Thu, May 21, 2020 at 4:51 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > On Wed, May 20, 2020 at 6:34 PM <bill.c.roberts@gmail.com> wrote: > > From: William Roberts <william.c.roberts@intel.com> > > > > The current Travis CI runs the userspace tooling and libraries against > > policy files, but cannot test against an SELinux enabled kernel. Thus, > > some tests are not being done in the CI. Travis, unfortunately only > > provides Ubuntu images, so in order to run against a modern distro with > > SELinux in enforcing mode, we need to launch a KVM with something like > > Fedora. > > > > This patch enables this support by launching a Fedora32 Cloud Image with > > the SELinux userspace library passed on from the Travis clone, it then > > builds and replaces the current SELinux bits on the Fedora32 image and > > runs the SELinux testsuite. > > > > Signed-off-by: William Roberts <william.c.roberts@intel.com> > > From the text above I infer that this patch is intended against the > userspace repo, right? > > If so, I don't quite see the usefulness of running the > selinux-testsuite on every userspace change... It is mainly intended > for testing the kernel and only a small part of its running time is > spent on running (i.e. testing in a sense) the SELinux userspace > programs. Not to mention that in your patch it runs with the userspace > shipped in Fedora and not the version from the given commit... Last I looked, his script builds and installs the userspace code on top of the Fedora libraries and programs (make LIBDIR=... install...) and then runs the testsuite. That was my suggestion. While it is the kernel testsuite, it exercises a lot of SELinux userspace functionality that isn't tested by the userspace tests. > > However, it could be very useful if this was added to the testsuite's > CI instead so that it can verify that the testsuite patches don't > break something. But note that you'd need to modify the script a bit > to copy over the testsuite snapshot being tested to the VM and run > that, instead of the current master. > > Anyway, thank you for working on this! I never realized that it could > be so easy to run a Fedora VM from Travis. If I find some time maybe I > can find some more ways to use this... For example we could run the > Fedora/RHEL SELinux userspace tests from [1] after installing (not yet > sure how) the userspace built from the currently tested userspace repo > commit. > > [1] https://src.fedoraproject.org/tests/selinux/tree/master
On Thu, May 21, 2020 at 2:52 PM Stephen Smalley <stephen.smalley.work@gmail.com> wrote: > On Thu, May 21, 2020 at 4:51 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > > > On Wed, May 20, 2020 at 6:34 PM <bill.c.roberts@gmail.com> wrote: > > > From: William Roberts <william.c.roberts@intel.com> > > > > > > The current Travis CI runs the userspace tooling and libraries against > > > policy files, but cannot test against an SELinux enabled kernel. Thus, > > > some tests are not being done in the CI. Travis, unfortunately only > > > provides Ubuntu images, so in order to run against a modern distro with > > > SELinux in enforcing mode, we need to launch a KVM with something like > > > Fedora. > > > > > > This patch enables this support by launching a Fedora32 Cloud Image with > > > the SELinux userspace library passed on from the Travis clone, it then > > > builds and replaces the current SELinux bits on the Fedora32 image and > > > runs the SELinux testsuite. > > > > > > Signed-off-by: William Roberts <william.c.roberts@intel.com> > > > > From the text above I infer that this patch is intended against the > > userspace repo, right? > > > > If so, I don't quite see the usefulness of running the > > selinux-testsuite on every userspace change... It is mainly intended > > for testing the kernel and only a small part of its running time is > > spent on running (i.e. testing in a sense) the SELinux userspace > > programs. Not to mention that in your patch it runs with the userspace > > shipped in Fedora and not the version from the given commit... > > Last I looked, his script builds and installs the userspace code on > top of the Fedora libraries and programs (make LIBDIR=... install...) > and then runs the testsuite. That was my suggestion. Ah, yes, I can see that line now. Sorry, somehow I missed it before. > While it is the > kernel testsuite, it exercises a lot of SELinux userspace > functionality that isn't tested by the userspace tests. OK, I suppose it's better than nothing...
On Thu, May 21, 2020 at 7:58 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > On Thu, May 21, 2020 at 2:52 PM Stephen Smalley > <stephen.smalley.work@gmail.com> wrote: > > On Thu, May 21, 2020 at 4:51 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > > > > > On Wed, May 20, 2020 at 6:34 PM <bill.c.roberts@gmail.com> wrote: > > > > From: William Roberts <william.c.roberts@intel.com> > > > > > > > > The current Travis CI runs the userspace tooling and libraries against > > > > policy files, but cannot test against an SELinux enabled kernel. Thus, > > > > some tests are not being done in the CI. Travis, unfortunately only > > > > provides Ubuntu images, so in order to run against a modern distro with > > > > SELinux in enforcing mode, we need to launch a KVM with something like > > > > Fedora. > > > > > > > > This patch enables this support by launching a Fedora32 Cloud Image with > > > > the SELinux userspace library passed on from the Travis clone, it then > > > > builds and replaces the current SELinux bits on the Fedora32 image and > > > > runs the SELinux testsuite. > > > > > > > > Signed-off-by: William Roberts <william.c.roberts@intel.com> > > > > > > From the text above I infer that this patch is intended against the > > > userspace repo, right? > > > > > > If so, I don't quite see the usefulness of running the > > > selinux-testsuite on every userspace change... It is mainly intended > > > for testing the kernel and only a small part of its running time is > > > spent on running (i.e. testing in a sense) the SELinux userspace > > > programs. Not to mention that in your patch it runs with the userspace > > > shipped in Fedora and not the version from the given commit... Totally doable, and you just copy + edit those scripts to achieve that. Currently we inject the travis repo with patches, which is the selinux userspace, into the VM image with virt-sysprep. But if this was selinux-testsuite ci you would just virt-sysprep inject it's repo. > > > > Last I looked, his script builds and installs the userspace code on > > top of the Fedora libraries and programs (make LIBDIR=... install...) > > and then runs the testsuite. That was my suggestion. > > Ah, yes, I can see that line now. Sorry, somehow I missed it before. > > > While it is the > > kernel testsuite, it exercises a lot of SELinux userspace > > functionality that isn't tested by the userspace tests. > > OK, I suppose it's better than nothing... > Stephen pointed out the additional ways userspace gets tested, and perhaps my title and description of the patch could be better. But the main point is to increase the test coverage and perform the testing steps we expect are done before a release in the CI. We should have the testing coverage and the confidence to release userspace from master at any point. We also have forward facing proof that tests are being executed and we can make sure nothing regresses. My ultimate goal here, is to help make sure that if Petr gets hit by a bus, releases will move forward without worry and without any change in quality among the various maintainers. Additionally, we pick up some cross project testing and can find other surprises.
On Wed, May 20, 2020 at 6:34 PM <bill.c.roberts@gmail.com> wrote: > > From: William Roberts <william.c.roberts@intel.com> > > The current Travis CI runs the userspace tooling and libraries against > policy files, but cannot test against an SELinux enabled kernel. Thus, > some tests are not being done in the CI. Travis, unfortunately only > provides Ubuntu images, so in order to run against a modern distro with > SELinux in enforcing mode, we need to launch a KVM with something like > Fedora. > > This patch enables this support by launching a Fedora32 Cloud Image with > the SELinux userspace library passed on from the Travis clone, it then > builds and replaces the current SELinux bits on the Fedora32 image and > runs the SELinux testsuite. > > Signed-off-by: William Roberts <william.c.roberts@intel.com> Hi, Thanks for working on this. The CI scripts are quite easy to follow, thanks to all the comments :) Anyway, here are some suggestions/comments in order to make this patch even better (if you want to do a v3): * ShellCheck detects that some strings are not quoted, that using *-CHECKSUM is dangerous (using ./*-CHECKSUM prevents ill things from happening if a file in the directory happens to match the pattern and start with a dash), etc. Most issues reported by ShellCheck are not relevant here, but one: ipaddy=$(grep fedoravm dhcp-leases.txt | awk {'print $5'} | cut -d'/' -f 1-1) ^-- SC1083: This { is literal. Check expression (missing ;/\n?) or quote it. Using awk '{print $5}' (single quotes outside of the braces) seems to better match the intent. Moreover the whole $(...) expression could be double-quoted. * fedora-test-runner.sh uses ~, which is expanded to /root as it does not make sense to run this script as non-root user. In my humble opinion, using /root instead of ~ makes the script even easier to read. * There is a comment about issues with -jX with "make LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install install-pywrap relabel". I usually split the targets when using "make -jX" (ie. make -j$(nproc) install && make -j$(nproc) install-pywrap && make relabel), so if not using -jX is really an issue, this could be considered. * Does the script runs "make test" in selinux/? I see the "make test" for selinux-testsuite, but if the main point of fedora-test-runner.sh is to run selinux's tests (in a VM), I missed the command that does so. Thanks, Nicolas > --- > .travis.yml | 8 +++ > scripts/ci/README.md | 8 +++ > scripts/ci/fedora-test-runner.sh | 89 ++++++++++++++++++++++++ > scripts/ci/travis-kvm-setup.sh | 113 +++++++++++++++++++++++++++++++ > 4 files changed, 218 insertions(+) > create mode 100644 scripts/ci/README.md > create mode 100755 scripts/ci/fedora-test-runner.sh > create mode 100755 scripts/ci/travis-kvm-setup.sh > > diff --git a/.travis.yml b/.travis.yml > index c36e721a5e1d..63a856672f9b 100644 > --- a/.travis.yml > +++ b/.travis.yml > @@ -34,6 +34,14 @@ matrix: > env: PYVER=python3.8 RUBYLIBVER=2.7 LINKER=gold > - compiler: clang > env: PYVER=python3.8 RUBYLIBVER=2.7 LINKER=bfd > + include: > + - compiler: gcc > + env: TRAVIS_RUN_KVM=true > + install: > + - skip > + before_script: > + - skip > + script: scripts/ci/travis-kvm-setup.sh > > # Use Travis-CI Ubuntu 18.04 Bionic Beaver, "full image" variant > sudo: required > diff --git a/scripts/ci/README.md b/scripts/ci/README.md > new file mode 100644 > index 000000000000..04a134a438c2 > --- /dev/null > +++ b/scripts/ci/README.md > @@ -0,0 +1,8 @@ > +# Continuous Integration Scripts > + > +The scripts under `scripts/ci` are designed specifically > +for the Travis CI system. While nothing prevents you > +from mimicking that environment and using them locally, > +they are not applicable for general consumption. Any > +thing in this directory should never be considered as > +a stable API. > diff --git a/scripts/ci/fedora-test-runner.sh b/scripts/ci/fedora-test-runner.sh > new file mode 100755 > index 000000000000..14bcf5fc469d > --- /dev/null > +++ b/scripts/ci/fedora-test-runner.sh > @@ -0,0 +1,89 @@ > +#!/usr/bin/env bash > + > +set -ev > + > +# CI Debug output if things go squirrely. > +getenforce > +id -Z > +nproc > +pwd > + > +# Turn off enforcing for the setup to prevent any weirdness from breaking > +# the CI. > +setenforce 0 > + > +dnf clean all -y > +dnf install -y \ > + --allowerasing \ > + --skip-broken \ > + git \ > + audit-libs-devel \ > + bison \ > + bzip2-devel \ > + CUnit-devel \ > + diffutils \ > + flex \ > + gcc \ > + gettext \ > + glib2-devel \ > + make \ > + libcap-devel \ > + libcap-ng-devel \ > + pam-devel \ > + pcre-devel \ > + xmlto \ > + python3-devel \ > + ruby-devel \ > + swig \ > + perl-Test \ > + perl-Test-Harness \ > + perl-Test-Simple \ > + selinux-policy-devel \ > + gcc \ > + libselinux-devel \ > + net-tools \ > + netlabel_tools \ > + iptables \ > + lksctp-tools-devel \ > + attr \ > + libbpf-devel \ > + keyutils-libs-devel \ > + kernel-devel \ > + quota \ > + xfsprogs-devel \ > + libuuid-devel \ > + kernel-devel-$(uname -r) \ > + kernel-modules-$(uname -r) > + > +# > +# Move to selinux code and build > +# > +cd ~/selinux > + > +# Show HEAD commit for sanity checking > +git log -1 > + > +# > +# Build and replace userspace components > +# > +# Note: You can't use parallel builds here (make -jX), you'll end up > +# with race conditions that manifest like: > +# semanage_store.lo: file not recognized: file format not recognized > +# > +make LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install install-pywrap relabel > + > +# > +# Get the selinux testsuite, but don't clone it in ~/selinux, move to ~ > +# first. > +# > +cd ~ > +git clone --depth=1 https://github.com/SELinuxProject/selinux-testsuite.git > +cd selinux-testsuite > + > +# The testsuite must be run in enforcing mode > +setenforce 1 > + > +# > +# Run the test suite > +# > +make test > diff --git a/scripts/ci/travis-kvm-setup.sh b/scripts/ci/travis-kvm-setup.sh > new file mode 100755 > index 000000000000..66606e9d4a5b > --- /dev/null > +++ b/scripts/ci/travis-kvm-setup.sh > @@ -0,0 +1,113 @@ > +#!/usr/bin/env bash > + > +set -ev > + > +TEST_RUNNER="scripts/ci/fedora-test-runner.sh" > + > +# > +# Travis gives us 7.5GB of RAM and two cores: > +# https://docs.travis-ci.com/user/reference/overview/ > +# > +MEMORY=4096 > +VCPUS=2 > + > +# Install these here so other builds don't have to wait on these deps to download and install > +sudo apt-get install qemu-kvm libvirt-bin virtinst bridge-utils cpu-checker libguestfs-tools > + > +sudo usermod -a -G kvm,libvirt,libvirt-qemu $USER > + > +# Verify that KVM is working, useful if Travis every changes anything. > +kvm-ok > + > +sudo systemctl enable libvirtd > +sudo systemctl start libvirtd > + > +# Set up a key so we can ssh into the VM > +ssh-keygen -N "" -f "$HOME/.ssh/id_rsa" > + > +# > +# Get the Fedora Cloud Image, It is a base image that small and ready to go, extract it and modify it with virt-sysprep > +# - https://alt.fedoraproject.org/en/verify.html > +cd $HOME > +wget https://download.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.raw.xz > + > +# Verify the image > +curl https://getfedora.org/static/fedora.gpg | gpg --import > +wget https://getfedora.org/static/checksums/Fedora-Cloud-32-1.6-x86_64-CHECKSUM > +gpg --verify-files *-CHECKSUM > +sha256sum --ignore-missing -c *-CHECKSUM > + > +# Extract the image > +unxz -T0 Fedora-Cloud-Base-32-1.6.x86_64.raw.xz > + > +# Search is needed for $HOME so virt service can access the image file. > +chmod a+x $HOME > + > +# > +# Modify the virtual image to: > +# - Enable a login, we just use root > +# - Enable passwordless login > +# - Force a relabel to fix labels on ssh keys > +# > +sudo virt-sysprep -a "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ > + --root-password password:123456 \ > + --hostname fedoravm \ > + --append-line '/etc/ssh/sshd_config:PermitRootLogin yes' \ > + --append-line '/etc/ssh/sshd_config:PubkeyAuthentication yes' \ > + --mkdir /root/.ssh \ > + --upload "$HOME/.ssh/id_rsa.pub:/root/.ssh/authorized_keys" \ > + --chmod '0600:/root/.ssh/authorized_keys' \ > + --run-command 'chown root:root /root/.ssh/authorized_keys' \ > + --copy-in "$TRAVIS_BUILD_DIR:/root" \ > + --network \ > + --selinux-relabel > + > +# > +# Now we create a domain by using virt-install. This not only creates the domain, but runs the VM as well > +# It should be ready to go for ssh, once ssh starts. > +# > +sudo virt-install \ > + --name fedoravm \ > + --memory $MEMORY \ > + --vcpus $VCPUS \ > + --disk "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ > + --import --noautoconsole > + > +# > +# Here comes the tricky part, we have to figure out when the VM comes up AND we need the ip address for ssh. So we > +# can check the net-dhcp leases, for our host. We have to poll, and we will poll for up 3 minutes in 6 second > +# intervals, so 30 poll attempts (0-29 inclusive). I don't know of a better way to do this. > +# > +# We have a full reboot + relabel, so first sleep gets us close > +# > +sleep 30 > +for i in $(seq 0 29); do > + echo "loop $i" > + sleep 6s > + # Get the leases, but tee it so it's easier to debug > + sudo virsh net-dhcp-leases default | tee dhcp-leases.txt > + > + # get our ipaddress > + ipaddy=$(grep fedoravm dhcp-leases.txt | awk {'print $5'} | cut -d'/' -f 1-1) > + if [ -n "$ipaddy" ]; then > + # found it, we're done looking, print it for debug logs > + echo "ipaddy: $ipaddy" > + break > + fi > + # it's empty/not found, loop back and try again. > +done > + > +# Did we find it? If not die. > +if [ -z "$ipaddy" ]; then > + echo "ipaddy zero length, exiting with error 1" > + exit 1 > +fi > + > +# > +# Great we have a host running, ssh into it. We specify -o so > +# we don't get blocked on asking to add the servers key to > +# our known_hosts. > +# > +ssh -tt -o StrictHostKeyChecking=no -o LogLevel=QUIET "root@$ipaddy" "/root/selinux/$TEST_RUNNER" > + > +exit 0 > -- > 2.17.1 >
On Thu, May 21, 2020 at 2:54 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote: > > On Wed, May 20, 2020 at 6:34 PM <bill.c.roberts@gmail.com> wrote: > > > > From: William Roberts <william.c.roberts@intel.com> > > > > The current Travis CI runs the userspace tooling and libraries against > > policy files, but cannot test against an SELinux enabled kernel. Thus, > > some tests are not being done in the CI. Travis, unfortunately only > > provides Ubuntu images, so in order to run against a modern distro with > > SELinux in enforcing mode, we need to launch a KVM with something like > > Fedora. > > > > This patch enables this support by launching a Fedora32 Cloud Image with > > the SELinux userspace library passed on from the Travis clone, it then > > builds and replaces the current SELinux bits on the Fedora32 image and > > runs the SELinux testsuite. > > > > Signed-off-by: William Roberts <william.c.roberts@intel.com> > > Hi, > Thanks for working on this. The CI scripts are quite easy to follow, > thanks to all the comments :) > > Anyway, here are some suggestions/comments in order to make this patch > even better (if you want to do a v3): > > * ShellCheck detects that some strings are not quoted, that using > *-CHECKSUM is dangerous (using ./*-CHECKSUM prevents ill things from > happening if a file in the directory happens to match the pattern and > start with a dash), etc. Most issues reported by ShellCheck are not > relevant here, but one: > > ipaddy=$(grep fedoravm dhcp-leases.txt | awk {'print $5'} | cut > -d'/' -f 1-1) > ^-- SC1083: This { is > literal. Check expression (missing ;/\n?) or quote it. > > Using awk '{print $5}' (single quotes outside of the braces) seems to > better match the intent. Moreover the whole $(...) expression could be > double-quoted. > > * fedora-test-runner.sh uses ~, which is expanded to /root as it does > not make sense to run this script as non-root user. In my humble > opinion, using /root instead of ~ makes the script even easier to > read. Fixing all the shell check issues for v3 seems prudent, so will respin with those changes. > > * There is a comment about issues with -jX with "make > LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install install-pywrap relabel". I > usually split the targets when using "make -jX" (ie. make -j$(nproc) > install && make -j$(nproc) install-pywrap && make relabel), so if not > using -jX is really an issue, this could be considered. Oh thanks for the suggestion, I didn't think of that, I can do that to reduce whatever time we can. > > * Does the script runs "make test" in selinux/? I see the "make test" > for selinux-testsuite, but if the main point of fedora-test-runner.sh > is to run selinux's tests (in a VM), I missed the command that does > so. I thought about that, but libselinux and things that use libselinux: - libselinux - libsemanage - mcstrans - policycoreutils - restorecond - sandbox All have *empty* Makefile test targets *with the exception of sandbox*. We already run make check on the Ubuntu VM on Travis, which tests all the projects that have defined test targets and since they don't use libselinux likely don't need a libselinux enabled host. I'm not opposed to it, just trying to trim down to things that aren't already being done. I guess at a minimum it ensures that tests didn't break between Ubuntu and Fedora, perhaps over things like dependency versions (python version for instance) and perhaps we pick up the sandbox tests. But the Ubuntu VM's already test with multiple (22 different build flavors) of different python, ruby, compilers and linkers and those only take a few minutes to complete. I'm definitely not opposed to adding it, perhaps i'm ignorant to the additional things that might get covered and thus someone can make stronger arguments to adding it than I can. > > Thanks, > Nicolas > > > --- > > .travis.yml | 8 +++ > > scripts/ci/README.md | 8 +++ > > scripts/ci/fedora-test-runner.sh | 89 ++++++++++++++++++++++++ > > scripts/ci/travis-kvm-setup.sh | 113 +++++++++++++++++++++++++++++++ > > 4 files changed, 218 insertions(+) > > create mode 100644 scripts/ci/README.md > > create mode 100755 scripts/ci/fedora-test-runner.sh > > create mode 100755 scripts/ci/travis-kvm-setup.sh > > > > diff --git a/.travis.yml b/.travis.yml > > index c36e721a5e1d..63a856672f9b 100644 > > --- a/.travis.yml > > +++ b/.travis.yml > > @@ -34,6 +34,14 @@ matrix: > > env: PYVER=python3.8 RUBYLIBVER=2.7 LINKER=gold > > - compiler: clang > > env: PYVER=python3.8 RUBYLIBVER=2.7 LINKER=bfd > > + include: > > + - compiler: gcc > > + env: TRAVIS_RUN_KVM=true > > + install: > > + - skip > > + before_script: > > + - skip > > + script: scripts/ci/travis-kvm-setup.sh > > > > # Use Travis-CI Ubuntu 18.04 Bionic Beaver, "full image" variant > > sudo: required > > diff --git a/scripts/ci/README.md b/scripts/ci/README.md > > new file mode 100644 > > index 000000000000..04a134a438c2 > > --- /dev/null > > +++ b/scripts/ci/README.md > > @@ -0,0 +1,8 @@ > > +# Continuous Integration Scripts > > + > > +The scripts under `scripts/ci` are designed specifically > > +for the Travis CI system. While nothing prevents you > > +from mimicking that environment and using them locally, > > +they are not applicable for general consumption. Any > > +thing in this directory should never be considered as > > +a stable API. > > diff --git a/scripts/ci/fedora-test-runner.sh b/scripts/ci/fedora-test-runner.sh > > new file mode 100755 > > index 000000000000..14bcf5fc469d > > --- /dev/null > > +++ b/scripts/ci/fedora-test-runner.sh > > @@ -0,0 +1,89 @@ > > +#!/usr/bin/env bash > > + > > +set -ev > > + > > +# CI Debug output if things go squirrely. > > +getenforce > > +id -Z > > +nproc > > +pwd > > + > > +# Turn off enforcing for the setup to prevent any weirdness from breaking > > +# the CI. > > +setenforce 0 > > + > > +dnf clean all -y > > +dnf install -y \ > > + --allowerasing \ > > + --skip-broken \ > > + git \ > > + audit-libs-devel \ > > + bison \ > > + bzip2-devel \ > > + CUnit-devel \ > > + diffutils \ > > + flex \ > > + gcc \ > > + gettext \ > > + glib2-devel \ > > + make \ > > + libcap-devel \ > > + libcap-ng-devel \ > > + pam-devel \ > > + pcre-devel \ > > + xmlto \ > > + python3-devel \ > > + ruby-devel \ > > + swig \ > > + perl-Test \ > > + perl-Test-Harness \ > > + perl-Test-Simple \ > > + selinux-policy-devel \ > > + gcc \ > > + libselinux-devel \ > > + net-tools \ > > + netlabel_tools \ > > + iptables \ > > + lksctp-tools-devel \ > > + attr \ > > + libbpf-devel \ > > + keyutils-libs-devel \ > > + kernel-devel \ > > + quota \ > > + xfsprogs-devel \ > > + libuuid-devel \ > > + kernel-devel-$(uname -r) \ > > + kernel-modules-$(uname -r) > > + > > +# > > +# Move to selinux code and build > > +# > > +cd ~/selinux > > + > > +# Show HEAD commit for sanity checking > > +git log -1 > > + > > +# > > +# Build and replace userspace components > > +# > > +# Note: You can't use parallel builds here (make -jX), you'll end up > > +# with race conditions that manifest like: > > +# semanage_store.lo: file not recognized: file format not recognized > > +# > > +make LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install install-pywrap relabel > > + > > +# > > +# Get the selinux testsuite, but don't clone it in ~/selinux, move to ~ > > +# first. > > +# > > +cd ~ > > +git clone --depth=1 https://github.com/SELinuxProject/selinux-testsuite.git > > +cd selinux-testsuite > > + > > +# The testsuite must be run in enforcing mode > > +setenforce 1 > > + > > +# > > +# Run the test suite > > +# > > +make test > > diff --git a/scripts/ci/travis-kvm-setup.sh b/scripts/ci/travis-kvm-setup.sh > > new file mode 100755 > > index 000000000000..66606e9d4a5b > > --- /dev/null > > +++ b/scripts/ci/travis-kvm-setup.sh > > @@ -0,0 +1,113 @@ > > +#!/usr/bin/env bash > > + > > +set -ev > > + > > +TEST_RUNNER="scripts/ci/fedora-test-runner.sh" > > + > > +# > > +# Travis gives us 7.5GB of RAM and two cores: > > +# https://docs.travis-ci.com/user/reference/overview/ > > +# > > +MEMORY=4096 > > +VCPUS=2 > > + > > +# Install these here so other builds don't have to wait on these deps to download and install > > +sudo apt-get install qemu-kvm libvirt-bin virtinst bridge-utils cpu-checker libguestfs-tools > > + > > +sudo usermod -a -G kvm,libvirt,libvirt-qemu $USER > > + > > +# Verify that KVM is working, useful if Travis every changes anything. > > +kvm-ok > > + > > +sudo systemctl enable libvirtd > > +sudo systemctl start libvirtd > > + > > +# Set up a key so we can ssh into the VM > > +ssh-keygen -N "" -f "$HOME/.ssh/id_rsa" > > + > > +# > > +# Get the Fedora Cloud Image, It is a base image that small and ready to go, extract it and modify it with virt-sysprep > > +# - https://alt.fedoraproject.org/en/verify.html > > +cd $HOME > > +wget https://download.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.raw.xz > > + > > +# Verify the image > > +curl https://getfedora.org/static/fedora.gpg | gpg --import > > +wget https://getfedora.org/static/checksums/Fedora-Cloud-32-1.6-x86_64-CHECKSUM > > +gpg --verify-files *-CHECKSUM > > +sha256sum --ignore-missing -c *-CHECKSUM > > + > > +# Extract the image > > +unxz -T0 Fedora-Cloud-Base-32-1.6.x86_64.raw.xz > > + > > +# Search is needed for $HOME so virt service can access the image file. > > +chmod a+x $HOME > > + > > +# > > +# Modify the virtual image to: > > +# - Enable a login, we just use root > > +# - Enable passwordless login > > +# - Force a relabel to fix labels on ssh keys > > +# > > +sudo virt-sysprep -a "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ > > + --root-password password:123456 \ > > + --hostname fedoravm \ > > + --append-line '/etc/ssh/sshd_config:PermitRootLogin yes' \ > > + --append-line '/etc/ssh/sshd_config:PubkeyAuthentication yes' \ > > + --mkdir /root/.ssh \ > > + --upload "$HOME/.ssh/id_rsa.pub:/root/.ssh/authorized_keys" \ > > + --chmod '0600:/root/.ssh/authorized_keys' \ > > + --run-command 'chown root:root /root/.ssh/authorized_keys' \ > > + --copy-in "$TRAVIS_BUILD_DIR:/root" \ > > + --network \ > > + --selinux-relabel > > + > > +# > > +# Now we create a domain by using virt-install. This not only creates the domain, but runs the VM as well > > +# It should be ready to go for ssh, once ssh starts. > > +# > > +sudo virt-install \ > > + --name fedoravm \ > > + --memory $MEMORY \ > > + --vcpus $VCPUS \ > > + --disk "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ > > + --import --noautoconsole > > + > > +# > > +# Here comes the tricky part, we have to figure out when the VM comes up AND we need the ip address for ssh. So we > > +# can check the net-dhcp leases, for our host. We have to poll, and we will poll for up 3 minutes in 6 second > > +# intervals, so 30 poll attempts (0-29 inclusive). I don't know of a better way to do this. > > +# > > +# We have a full reboot + relabel, so first sleep gets us close > > +# > > +sleep 30 > > +for i in $(seq 0 29); do > > + echo "loop $i" > > + sleep 6s > > + # Get the leases, but tee it so it's easier to debug > > + sudo virsh net-dhcp-leases default | tee dhcp-leases.txt > > + > > + # get our ipaddress > > + ipaddy=$(grep fedoravm dhcp-leases.txt | awk {'print $5'} | cut -d'/' -f 1-1) > > + if [ -n "$ipaddy" ]; then > > + # found it, we're done looking, print it for debug logs > > + echo "ipaddy: $ipaddy" > > + break > > + fi > > + # it's empty/not found, loop back and try again. > > +done > > + > > +# Did we find it? If not die. > > +if [ -z "$ipaddy" ]; then > > + echo "ipaddy zero length, exiting with error 1" > > + exit 1 > > +fi > > + > > +# > > +# Great we have a host running, ssh into it. We specify -o so > > +# we don't get blocked on asking to add the servers key to > > +# our known_hosts. > > +# > > +ssh -tt -o StrictHostKeyChecking=no -o LogLevel=QUIET "root@$ipaddy" "/root/selinux/$TEST_RUNNER" > > + > > +exit 0 > > -- > > 2.17.1 > > >
<snip> > * fedora-test-runner.sh uses ~, which is expanded to /root as it does > not make sense to run this script as non-root user. In my humble > opinion, using /root instead of ~ makes the script even easier to > read. I forgot to mention this in my previous email, this is the only thing I really don't want to change. You could, in theory, configure any user to run this. <snip>
On Thu, May 21, 2020 at 4:12 PM William Roberts <bill.c.roberts@gmail.com> wrote: > On Thu, May 21, 2020 at 7:58 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > > > On Thu, May 21, 2020 at 2:52 PM Stephen Smalley > > <stephen.smalley.work@gmail.com> wrote: [...] > > > Last I looked, his script builds and installs the userspace code on > > > top of the Fedora libraries and programs (make LIBDIR=... install...) > > > and then runs the testsuite. That was my suggestion. > > > > Ah, yes, I can see that line now. Sorry, somehow I missed it before. > > > > > While it is the > > > kernel testsuite, it exercises a lot of SELinux userspace > > > functionality that isn't tested by the userspace tests. > > > > OK, I suppose it's better than nothing... > > > > Stephen pointed out the additional ways userspace gets tested, and > perhaps my title and description > of the patch could be better. But the main point is to increase the > test coverage > and perform the testing steps we expect are done before a release in > the CI. We should have > the testing coverage and the confidence to release userspace from master at any > point. We also have forward facing proof that tests are being executed > and we can make sure > nothing regresses. > > My ultimate goal here, is to help make sure that if Petr gets hit by a > bus, releases will > move forward without worry and without any change in quality among the various > maintainers. > > Additionally, we pick up some cross project testing and can find other > surprises. Ah, so you want to move an integration test, which we would normally run only before release, down to per-commit testing, which is fine because it is quite fast... OK, it started to make sense to me now :) If I find time I'll have a closer look at the scripts. I already see some tiny possible improvements... -- Ondrej Mosnacek <omosnace at redhat dot com> Software Engineer, Security Technologies Red Hat, Inc.
On Fri, May 22, 2020 at 12:39 AM William Roberts <bill.c.roberts@gmail.com> wrote: > > <snip> > > * fedora-test-runner.sh uses ~, which is expanded to /root as it does > > not make sense to run this script as non-root user. In my humble > > opinion, using /root instead of ~ makes the script even easier to > > read. > > I forgot to mention this in my previous email, this is the only thing > I really don't > want to change. You could, in theory, configure any user to run this. > > <snip> The script executes "setenforce 0", runs dnf to install packages and overwrites binaries and libraries in system directories (/usr/bin, /usr/sbin, /usr/lib64, etc.). How do you allow any user to perform these actions, without being root? Anyway, if you do not want to hardcode /root, to could use "$HOME" instead of ~. It makes things appear less magical, in my humble opinion. Nicolas
On Fri, May 22, 2020 at 2:07 PM Nicolas Iooss <nicolas.iooss@m4x.org> wrote: > > On Fri, May 22, 2020 at 12:39 AM William Roberts > <bill.c.roberts@gmail.com> wrote: > > > > <snip> > > > * fedora-test-runner.sh uses ~, which is expanded to /root as it does > > > not make sense to run this script as non-root user. In my humble > > > opinion, using /root instead of ~ makes the script even easier to > > > read. > > > > I forgot to mention this in my previous email, this is the only thing > > I really don't > > want to change. You could, in theory, configure any user to run this. > > > > <snip> > > The script executes "setenforce 0", runs dnf to install packages and > overwrites binaries and libraries in system directories (/usr/bin, > /usr/sbin, /usr/lib64, etc.). How do you allow any user to perform > these actions, without being root? Its called capabilities. I could build you a box where root is neutered and cannot do anything. > > Anyway, if you do not want to hardcode /root, to could use "$HOME" > instead of ~. It makes things appear less magical, in my humble > opinion. Sure, it actually makes it consistent with the other script. > > Nicolas >
On Fri, May 22, 2020 at 2:40 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > On Thu, May 21, 2020 at 4:12 PM William Roberts > <bill.c.roberts@gmail.com> wrote: > > On Thu, May 21, 2020 at 7:58 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > > > > > On Thu, May 21, 2020 at 2:52 PM Stephen Smalley > > > <stephen.smalley.work@gmail.com> wrote: > [...] > > > > Last I looked, his script builds and installs the userspace code on > > > > top of the Fedora libraries and programs (make LIBDIR=... install...) > > > > and then runs the testsuite. That was my suggestion. > > > > > > Ah, yes, I can see that line now. Sorry, somehow I missed it before. > > > > > > > While it is the > > > > kernel testsuite, it exercises a lot of SELinux userspace > > > > functionality that isn't tested by the userspace tests. > > > > > > OK, I suppose it's better than nothing... > > > > > > > Stephen pointed out the additional ways userspace gets tested, and > > perhaps my title and description > > of the patch could be better. But the main point is to increase the > > test coverage > > and perform the testing steps we expect are done before a release in > > the CI. We should have > > the testing coverage and the confidence to release userspace from master at any > > point. We also have forward facing proof that tests are being executed > > and we can make sure > > nothing regresses. > > > > My ultimate goal here, is to help make sure that if Petr gets hit by a > > bus, releases will > > move forward without worry and without any change in quality among the various > > maintainers. > > > > Additionally, we pick up some cross project testing and can find other > > surprises. > > Ah, so you want to move an integration test, which we would normally > run only before release, down to per-commit testing, which is fine > because it is quite fast... OK, it started to make sense to me now :) Exactly, plus we pick up a little more test coverage on the userspace bits by swapping out what's installed in the VM with the current build and running the tests. The speed is less important, it's just fast enough where our CI isn't going to take years to complete. CI doesn't need to be super snappy per se, but it also cannot take a fortnight. > > If I find time I'll have a closer look at the scripts. I already see > some tiny possible improvements... I have Nicolas's last comments addressed and staged, so ill wait a few days and see what you come back with and re-send a V3.
On Sun, May 24, 2020 at 12:18 PM William Roberts <bill.c.roberts@gmail.com> wrote: > > On Fri, May 22, 2020 at 2:40 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > > > On Thu, May 21, 2020 at 4:12 PM William Roberts > > <bill.c.roberts@gmail.com> wrote: > > > On Thu, May 21, 2020 at 7:58 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > > > > > > > On Thu, May 21, 2020 at 2:52 PM Stephen Smalley > > > > <stephen.smalley.work@gmail.com> wrote: > > [...] > > > > > Last I looked, his script builds and installs the userspace code on > > > > > top of the Fedora libraries and programs (make LIBDIR=... install...) > > > > > and then runs the testsuite. That was my suggestion. > > > > > > > > Ah, yes, I can see that line now. Sorry, somehow I missed it before. > > > > > > > > > While it is the > > > > > kernel testsuite, it exercises a lot of SELinux userspace > > > > > functionality that isn't tested by the userspace tests. > > > > > > > > OK, I suppose it's better than nothing... > > > > > > > > > > Stephen pointed out the additional ways userspace gets tested, and > > > perhaps my title and description > > > of the patch could be better. But the main point is to increase the > > > test coverage > > > and perform the testing steps we expect are done before a release in > > > the CI. We should have > > > the testing coverage and the confidence to release userspace from master at any > > > point. We also have forward facing proof that tests are being executed > > > and we can make sure > > > nothing regresses. > > > > > > My ultimate goal here, is to help make sure that if Petr gets hit by a > > > bus, releases will > > > move forward without worry and without any change in quality among the various > > > maintainers. > > > > > > Additionally, we pick up some cross project testing and can find other > > > surprises. > > > > Ah, so you want to move an integration test, which we would normally > > run only before release, down to per-commit testing, which is fine > > because it is quite fast... OK, it started to make sense to me now :) > > Exactly, plus we pick up a little more test coverage on the userspace bits > by swapping out what's installed in the VM with the current build and running > the tests. The speed is less important, it's just fast enough where our CI isn't > going to take years to complete. CI doesn't need to be super snappy per se, > but it also cannot take a fortnight. > > > > > If I find time I'll have a closer look at the scripts. I already see > > some tiny possible improvements... I have Nicolas's last comments > addressed and staged, so ill wait a few days and see what you come > back with and re-send a V3. This is looking good to me. Only questions I have are: 1) Have you confirmed that a testsuite failure within the VM gets correctly propagated up and treated as a failure by travis-ci itself? 2) Have you seen any problems with instability in running of the tests due to the additional complexity and time? I've certainly already seen instances where travis-ci of selinux or selinux-testsuite fails randomly due to timeouts or something when downloading external components.
On Fri, May 29, 2020 at 8:24 AM Stephen Smalley <stephen.smalley.work@gmail.com> wrote: > > On Sun, May 24, 2020 at 12:18 PM William Roberts > <bill.c.roberts@gmail.com> wrote: > > > > On Fri, May 22, 2020 at 2:40 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > > > > > On Thu, May 21, 2020 at 4:12 PM William Roberts > > > <bill.c.roberts@gmail.com> wrote: > > > > On Thu, May 21, 2020 at 7:58 AM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > > > > > > > > > On Thu, May 21, 2020 at 2:52 PM Stephen Smalley > > > > > <stephen.smalley.work@gmail.com> wrote: > > > [...] > > > > > > Last I looked, his script builds and installs the userspace code on > > > > > > top of the Fedora libraries and programs (make LIBDIR=... install...) > > > > > > and then runs the testsuite. That was my suggestion. > > > > > > > > > > Ah, yes, I can see that line now. Sorry, somehow I missed it before. > > > > > > > > > > > While it is the > > > > > > kernel testsuite, it exercises a lot of SELinux userspace > > > > > > functionality that isn't tested by the userspace tests. > > > > > > > > > > OK, I suppose it's better than nothing... > > > > > > > > > > > > > Stephen pointed out the additional ways userspace gets tested, and > > > > perhaps my title and description > > > > of the patch could be better. But the main point is to increase the > > > > test coverage > > > > and perform the testing steps we expect are done before a release in > > > > the CI. We should have > > > > the testing coverage and the confidence to release userspace from master at any > > > > point. We also have forward facing proof that tests are being executed > > > > and we can make sure > > > > nothing regresses. > > > > > > > > My ultimate goal here, is to help make sure that if Petr gets hit by a > > > > bus, releases will > > > > move forward without worry and without any change in quality among the various > > > > maintainers. > > > > > > > > Additionally, we pick up some cross project testing and can find other > > > > surprises. > > > > > > Ah, so you want to move an integration test, which we would normally > > > run only before release, down to per-commit testing, which is fine > > > because it is quite fast... OK, it started to make sense to me now :) > > > > Exactly, plus we pick up a little more test coverage on the userspace bits > > by swapping out what's installed in the VM with the current build and running > > the tests. The speed is less important, it's just fast enough where our CI isn't > > going to take years to complete. CI doesn't need to be super snappy per se, > > but it also cannot take a fortnight. > > > > > > > > If I find time I'll have a closer look at the scripts. I already see > > > some tiny possible improvements... I have Nicolas's last comments > > addressed and staged, so ill wait a few days and see what you come > > back with and re-send a V3. > > This is looking good to me. Only questions I have are: > 1) Have you confirmed that a testsuite failure within the VM gets > correctly propagated up and treated as a failure by travis-ci itself? Yes, when I was testing and didn't have SCTP support, the test suite would fail and propagate back. > 2) Have you seen any problems with instability in running of the tests > due to the additional complexity and time? I've certainly already > seen instances where travis-ci of selinux or selinux-testsuite fails > randomly due to timeouts or something when downloading external > components. No, and I did a bunch of additional runs, like 10-12 and never saw additional failures. I see some of those intermittent travis CI failures with all my projects on Travis, but I don't see this as adding any more issues. Timeouts are caused by nothing sent to stdout for 10 mins (IIRC), so we should be good there. Im just waiting on Ondrej, he said he had some feedback and I was gonna send round 3. If I don't here anything back on Monday ill send round 3.
Apologies for getting back to this so late... Just some small nitpicks. On Wed, May 20, 2020 at 6:34 PM <bill.c.roberts@gmail.com> wrote: > From: William Roberts <william.c.roberts@intel.com> > > The current Travis CI runs the userspace tooling and libraries against > policy files, but cannot test against an SELinux enabled kernel. Thus, > some tests are not being done in the CI. Travis, unfortunately only > provides Ubuntu images, so in order to run against a modern distro with > SELinux in enforcing mode, we need to launch a KVM with something like > Fedora. > > This patch enables this support by launching a Fedora32 Cloud Image with > the SELinux userspace library passed on from the Travis clone, it then > builds and replaces the current SELinux bits on the Fedora32 image and > runs the SELinux testsuite. > > Signed-off-by: William Roberts <william.c.roberts@intel.com> > --- > .travis.yml | 8 +++ > scripts/ci/README.md | 8 +++ > scripts/ci/fedora-test-runner.sh | 89 ++++++++++++++++++++++++ > scripts/ci/travis-kvm-setup.sh | 113 +++++++++++++++++++++++++++++++ > 4 files changed, 218 insertions(+) > create mode 100644 scripts/ci/README.md > create mode 100755 scripts/ci/fedora-test-runner.sh > create mode 100755 scripts/ci/travis-kvm-setup.sh > [...] > diff --git a/scripts/ci/fedora-test-runner.sh b/scripts/ci/fedora-test-runner.sh > new file mode 100755 > index 000000000000..14bcf5fc469d > --- /dev/null > +++ b/scripts/ci/fedora-test-runner.sh > @@ -0,0 +1,89 @@ > +#!/usr/bin/env bash > + > +set -ev > + > +# CI Debug output if things go squirrely. > +getenforce > +id -Z > +nproc > +pwd I'd add also "uname -r" here to dump the running kernel version (will probably be also printed later somewhere, but better to have it also in one place with the other debug info). > + > +# Turn off enforcing for the setup to prevent any weirdness from breaking > +# the CI. > +setenforce 0 > + > +dnf clean all -y > +dnf install -y \ [...] > diff --git a/scripts/ci/travis-kvm-setup.sh b/scripts/ci/travis-kvm-setup.sh > new file mode 100755 > index 000000000000..66606e9d4a5b > --- /dev/null > +++ b/scripts/ci/travis-kvm-setup.sh > @@ -0,0 +1,113 @@ > +#!/usr/bin/env bash > + > +set -ev > + > +TEST_RUNNER="scripts/ci/fedora-test-runner.sh" > + > +# > +# Travis gives us 7.5GB of RAM and two cores: > +# https://docs.travis-ci.com/user/reference/overview/ > +# > +MEMORY=4096 > +VCPUS=2 Why not "VCPUS=$(nproc)"? > + > +# Install these here so other builds don't have to wait on these deps to download and install > +sudo apt-get install qemu-kvm libvirt-bin virtinst bridge-utils cpu-checker libguestfs-tools > + > +sudo usermod -a -G kvm,libvirt,libvirt-qemu $USER > + > +# Verify that KVM is working, useful if Travis every changes anything. s/every/ever/ > +kvm-ok > + > +sudo systemctl enable libvirtd > +sudo systemctl start libvirtd > + > +# Set up a key so we can ssh into the VM > +ssh-keygen -N "" -f "$HOME/.ssh/id_rsa" > + > +# > +# Get the Fedora Cloud Image, It is a base image that small and ready to go, extract it and modify it with virt-sysprep > +# - https://alt.fedoraproject.org/en/verify.html > +cd $HOME > +wget https://download.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.raw.xz I'd suggest extracting the Fedora release version (32) + the image version (1.6) into variables, so they can be easily bumped later. > + > +# Verify the image > +curl https://getfedora.org/static/fedora.gpg | gpg --import > +wget https://getfedora.org/static/checksums/Fedora-Cloud-32-1.6-x86_64-CHECKSUM > +gpg --verify-files *-CHECKSUM > +sha256sum --ignore-missing -c *-CHECKSUM > + > +# Extract the image > +unxz -T0 Fedora-Cloud-Base-32-1.6.x86_64.raw.xz > + > +# Search is needed for $HOME so virt service can access the image file. > +chmod a+x $HOME > + > +# > +# Modify the virtual image to: > +# - Enable a login, we just use root > +# - Enable passwordless login > +# - Force a relabel to fix labels on ssh keys > +# > +sudo virt-sysprep -a "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ > + --root-password password:123456 \ Do you need to set the password when you use an SSH key to login? > + --hostname fedoravm \ > + --append-line '/etc/ssh/sshd_config:PermitRootLogin yes' \ > + --append-line '/etc/ssh/sshd_config:PubkeyAuthentication yes' \ > + --mkdir /root/.ssh \ > + --upload "$HOME/.ssh/id_rsa.pub:/root/.ssh/authorized_keys" \ > + --chmod '0600:/root/.ssh/authorized_keys' \ > + --run-command 'chown root:root /root/.ssh/authorized_keys' \ Could these be replaced with just "--ssh-inject root"? > + --copy-in "$TRAVIS_BUILD_DIR:/root" \ > + --network \ > + --selinux-relabel > + > +# > +# Now we create a domain by using virt-install. This not only creates the domain, but runs the VM as well > +# It should be ready to go for ssh, once ssh starts. > +# > +sudo virt-install \ > + --name fedoravm \ > + --memory $MEMORY \ > + --vcpus $VCPUS \ > + --disk "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ > + --import --noautoconsole > + > +# > +# Here comes the tricky part, we have to figure out when the VM comes up AND we need the ip address for ssh. So we > +# can check the net-dhcp leases, for our host. We have to poll, and we will poll for up 3 minutes in 6 second > +# intervals, so 30 poll attempts (0-29 inclusive). I don't know of a better way to do this. > +# > +# We have a full reboot + relabel, so first sleep gets us close > +# > +sleep 30 > +for i in $(seq 0 29); do > + echo "loop $i" > + sleep 6s > + # Get the leases, but tee it so it's easier to debug > + sudo virsh net-dhcp-leases default | tee dhcp-leases.txt > + > + # get our ipaddress > + ipaddy=$(grep fedoravm dhcp-leases.txt | awk {'print $5'} | cut -d'/' -f 1-1) Looks cleaner this way: [...] | awk '{ print $5 }' | cut -d / -f 1) > + if [ -n "$ipaddy" ]; then > + # found it, we're done looking, print it for debug logs > + echo "ipaddy: $ipaddy" > + break > + fi > + # it's empty/not found, loop back and try again. > +done > + > +# Did we find it? If not die. > +if [ -z "$ipaddy" ]; then > + echo "ipaddy zero length, exiting with error 1" > + exit 1 > +fi > + > +# > +# Great we have a host running, ssh into it. We specify -o so > +# we don't get blocked on asking to add the servers key to > +# our known_hosts. > +# > +ssh -tt -o StrictHostKeyChecking=no -o LogLevel=QUIET "root@$ipaddy" "/root/selinux/$TEST_RUNNER" > + > +exit 0 > -- > 2.17.1
On Fri, May 29, 2020 at 1:43 PM Ondrej Mosnacek <omosnace@redhat.com> wrote: > > Apologies for getting back to this so late... Just some small nitpicks. > > On Wed, May 20, 2020 at 6:34 PM <bill.c.roberts@gmail.com> wrote: > > From: William Roberts <william.c.roberts@intel.com> > > > > The current Travis CI runs the userspace tooling and libraries against > > policy files, but cannot test against an SELinux enabled kernel. Thus, > > some tests are not being done in the CI. Travis, unfortunately only > > provides Ubuntu images, so in order to run against a modern distro with > > SELinux in enforcing mode, we need to launch a KVM with something like > > Fedora. > > > > This patch enables this support by launching a Fedora32 Cloud Image with > > the SELinux userspace library passed on from the Travis clone, it then > > builds and replaces the current SELinux bits on the Fedora32 image and > > runs the SELinux testsuite. > > > > Signed-off-by: William Roberts <william.c.roberts@intel.com> > > --- > > .travis.yml | 8 +++ > > scripts/ci/README.md | 8 +++ > > scripts/ci/fedora-test-runner.sh | 89 ++++++++++++++++++++++++ > > scripts/ci/travis-kvm-setup.sh | 113 +++++++++++++++++++++++++++++++ > > 4 files changed, 218 insertions(+) > > create mode 100644 scripts/ci/README.md > > create mode 100755 scripts/ci/fedora-test-runner.sh > > create mode 100755 scripts/ci/travis-kvm-setup.sh > > > [...] > > diff --git a/scripts/ci/fedora-test-runner.sh b/scripts/ci/fedora-test-runner.sh > > new file mode 100755 > > index 000000000000..14bcf5fc469d > > --- /dev/null > > +++ b/scripts/ci/fedora-test-runner.sh > > @@ -0,0 +1,89 @@ > > +#!/usr/bin/env bash > > + > > +set -ev > > + > > +# CI Debug output if things go squirrely. > > +getenforce > > +id -Z > > +nproc > > +pwd > > I'd add also "uname -r" here to dump the running kernel version (will > probably be also printed later somewhere, but better to have it also > in one place with the other debug info). > > > + > > +# Turn off enforcing for the setup to prevent any weirdness from breaking > > +# the CI. > > +setenforce 0 > > + > > +dnf clean all -y > > +dnf install -y \ > [...] > > diff --git a/scripts/ci/travis-kvm-setup.sh b/scripts/ci/travis-kvm-setup.sh > > new file mode 100755 > > index 000000000000..66606e9d4a5b > > --- /dev/null > > +++ b/scripts/ci/travis-kvm-setup.sh > > @@ -0,0 +1,113 @@ > > +#!/usr/bin/env bash > > + > > +set -ev > > + > > +TEST_RUNNER="scripts/ci/fedora-test-runner.sh" > > + > > +# > > +# Travis gives us 7.5GB of RAM and two cores: > > +# https://docs.travis-ci.com/user/reference/overview/ > > +# > > +MEMORY=4096 > > +VCPUS=2 > > Why not "VCPUS=$(nproc)"? +1: Initially I just had this set to what travis provides. I don't know why I didn't do that. > > > + > > +# Install these here so other builds don't have to wait on these deps to download and install > > +sudo apt-get install qemu-kvm libvirt-bin virtinst bridge-utils cpu-checker libguestfs-tools > > + > > +sudo usermod -a -G kvm,libvirt,libvirt-qemu $USER > > + > > +# Verify that KVM is working, useful if Travis every changes anything. > > s/every/ever/ +1 > > > +kvm-ok > > + > > +sudo systemctl enable libvirtd > > +sudo systemctl start libvirtd > > + > > +# Set up a key so we can ssh into the VM > > +ssh-keygen -N "" -f "$HOME/.ssh/id_rsa" > > + > > +# > > +# Get the Fedora Cloud Image, It is a base image that small and ready to go, extract it and modify it with virt-sysprep > > +# - https://alt.fedoraproject.org/en/verify.html > > +cd $HOME > > +wget https://download.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.raw.xz > > I'd suggest extracting the Fedora release version (32) + the image > version (1.6) into variables, so they can be easily bumped later. Sure, I forget why I ended up not doing it this way. I remember it being late at night, and cursing for some reason. > > > + > > +# Verify the image > > +curl https://getfedora.org/static/fedora.gpg | gpg --import > > +wget https://getfedora.org/static/checksums/Fedora-Cloud-32-1.6-x86_64-CHECKSUM > > +gpg --verify-files *-CHECKSUM > > +sha256sum --ignore-missing -c *-CHECKSUM > > + > > +# Extract the image > > +unxz -T0 Fedora-Cloud-Base-32-1.6.x86_64.raw.xz > > + > > +# Search is needed for $HOME so virt service can access the image file. > > +chmod a+x $HOME > > + > > +# > > +# Modify the virtual image to: > > +# - Enable a login, we just use root > > +# - Enable passwordless login > > +# - Force a relabel to fix labels on ssh keys > > +# > > +sudo virt-sysprep -a "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ > > + --root-password password:123456 \ > > Do you need to set the password when you use an SSH key to login? Yeah the account is disabled unless you do this. Plus it was helpful when using the scripts locally and using virsh console > > > + --hostname fedoravm \ > > + --append-line '/etc/ssh/sshd_config:PermitRootLogin yes' \ > > + --append-line '/etc/ssh/sshd_config:PubkeyAuthentication yes' \ > > + --mkdir /root/.ssh \ > > + --upload "$HOME/.ssh/id_rsa.pub:/root/.ssh/authorized_keys" \ > > + --chmod '0600:/root/.ssh/authorized_keys' \ > > + --run-command 'chown root:root /root/.ssh/authorized_keys' \ > > Could these be replaced with just "--ssh-inject root"? No, and I went through immense pain trying to get it to work. The reason is that the tool will dump it under /home/root instead of /root. So it won't get picked up without some other magic anyways. > > > + --copy-in "$TRAVIS_BUILD_DIR:/root" \ > > + --network \ > > + --selinux-relabel > > + > > +# > > +# Now we create a domain by using virt-install. This not only creates the domain, but runs the VM as well > > +# It should be ready to go for ssh, once ssh starts. > > +# > > +sudo virt-install \ > > + --name fedoravm \ > > + --memory $MEMORY \ > > + --vcpus $VCPUS \ > > + --disk "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ > > + --import --noautoconsole > > + > > +# > > +# Here comes the tricky part, we have to figure out when the VM comes up AND we need the ip address for ssh. So we > > +# can check the net-dhcp leases, for our host. We have to poll, and we will poll for up 3 minutes in 6 second > > +# intervals, so 30 poll attempts (0-29 inclusive). I don't know of a better way to do this. > > +# > > +# We have a full reboot + relabel, so first sleep gets us close > > +# > > +sleep 30 > > +for i in $(seq 0 29); do > > + echo "loop $i" > > + sleep 6s > > + # Get the leases, but tee it so it's easier to debug > > + sudo virsh net-dhcp-leases default | tee dhcp-leases.txt > > + > > + # get our ipaddress > > + ipaddy=$(grep fedoravm dhcp-leases.txt | awk {'print $5'} | cut -d'/' -f 1-1) > > Looks cleaner this way: > [...] | awk '{ print $5 }' | cut -d / -f 1) > > > + if [ -n "$ipaddy" ]; then > > + # found it, we're done looking, print it for debug logs > > + echo "ipaddy: $ipaddy" > > + break > > + fi > > + # it's empty/not found, loop back and try again. > > +done > > + > > +# Did we find it? If not die. > > +if [ -z "$ipaddy" ]; then > > + echo "ipaddy zero length, exiting with error 1" > > + exit 1 > > +fi > > + > > +# > > +# Great we have a host running, ssh into it. We specify -o so > > +# we don't get blocked on asking to add the servers key to > > +# our known_hosts. > > +# > > +ssh -tt -o StrictHostKeyChecking=no -o LogLevel=QUIET "root@$ipaddy" "/root/selinux/$TEST_RUNNER" > > + > > +exit 0 > > -- > > 2.17.1 > Thanks Ondrej. Bill
diff --git a/.travis.yml b/.travis.yml index c36e721a5e1d..63a856672f9b 100644 --- a/.travis.yml +++ b/.travis.yml @@ -34,6 +34,14 @@ matrix: env: PYVER=python3.8 RUBYLIBVER=2.7 LINKER=gold - compiler: clang env: PYVER=python3.8 RUBYLIBVER=2.7 LINKER=bfd + include: + - compiler: gcc + env: TRAVIS_RUN_KVM=true + install: + - skip + before_script: + - skip + script: scripts/ci/travis-kvm-setup.sh # Use Travis-CI Ubuntu 18.04 Bionic Beaver, "full image" variant sudo: required diff --git a/scripts/ci/README.md b/scripts/ci/README.md new file mode 100644 index 000000000000..04a134a438c2 --- /dev/null +++ b/scripts/ci/README.md @@ -0,0 +1,8 @@ +# Continuous Integration Scripts + +The scripts under `scripts/ci` are designed specifically +for the Travis CI system. While nothing prevents you +from mimicking that environment and using them locally, +they are not applicable for general consumption. Any +thing in this directory should never be considered as +a stable API. diff --git a/scripts/ci/fedora-test-runner.sh b/scripts/ci/fedora-test-runner.sh new file mode 100755 index 000000000000..14bcf5fc469d --- /dev/null +++ b/scripts/ci/fedora-test-runner.sh @@ -0,0 +1,89 @@ +#!/usr/bin/env bash + +set -ev + +# CI Debug output if things go squirrely. +getenforce +id -Z +nproc +pwd + +# Turn off enforcing for the setup to prevent any weirdness from breaking +# the CI. +setenforce 0 + +dnf clean all -y +dnf install -y \ + --allowerasing \ + --skip-broken \ + git \ + audit-libs-devel \ + bison \ + bzip2-devel \ + CUnit-devel \ + diffutils \ + flex \ + gcc \ + gettext \ + glib2-devel \ + make \ + libcap-devel \ + libcap-ng-devel \ + pam-devel \ + pcre-devel \ + xmlto \ + python3-devel \ + ruby-devel \ + swig \ + perl-Test \ + perl-Test-Harness \ + perl-Test-Simple \ + selinux-policy-devel \ + gcc \ + libselinux-devel \ + net-tools \ + netlabel_tools \ + iptables \ + lksctp-tools-devel \ + attr \ + libbpf-devel \ + keyutils-libs-devel \ + kernel-devel \ + quota \ + xfsprogs-devel \ + libuuid-devel \ + kernel-devel-$(uname -r) \ + kernel-modules-$(uname -r) + +# +# Move to selinux code and build +# +cd ~/selinux + +# Show HEAD commit for sanity checking +git log -1 + +# +# Build and replace userspace components +# +# Note: You can't use parallel builds here (make -jX), you'll end up +# with race conditions that manifest like: +# semanage_store.lo: file not recognized: file format not recognized +# +make LIBDIR=/usr/lib64 SHLIBDIR=/lib64 install install-pywrap relabel + +# +# Get the selinux testsuite, but don't clone it in ~/selinux, move to ~ +# first. +# +cd ~ +git clone --depth=1 https://github.com/SELinuxProject/selinux-testsuite.git +cd selinux-testsuite + +# The testsuite must be run in enforcing mode +setenforce 1 + +# +# Run the test suite +# +make test diff --git a/scripts/ci/travis-kvm-setup.sh b/scripts/ci/travis-kvm-setup.sh new file mode 100755 index 000000000000..66606e9d4a5b --- /dev/null +++ b/scripts/ci/travis-kvm-setup.sh @@ -0,0 +1,113 @@ +#!/usr/bin/env bash + +set -ev + +TEST_RUNNER="scripts/ci/fedora-test-runner.sh" + +# +# Travis gives us 7.5GB of RAM and two cores: +# https://docs.travis-ci.com/user/reference/overview/ +# +MEMORY=4096 +VCPUS=2 + +# Install these here so other builds don't have to wait on these deps to download and install +sudo apt-get install qemu-kvm libvirt-bin virtinst bridge-utils cpu-checker libguestfs-tools + +sudo usermod -a -G kvm,libvirt,libvirt-qemu $USER + +# Verify that KVM is working, useful if Travis every changes anything. +kvm-ok + +sudo systemctl enable libvirtd +sudo systemctl start libvirtd + +# Set up a key so we can ssh into the VM +ssh-keygen -N "" -f "$HOME/.ssh/id_rsa" + +# +# Get the Fedora Cloud Image, It is a base image that small and ready to go, extract it and modify it with virt-sysprep +# - https://alt.fedoraproject.org/en/verify.html +cd $HOME +wget https://download.fedoraproject.org/pub/fedora/linux/releases/32/Cloud/x86_64/images/Fedora-Cloud-Base-32-1.6.x86_64.raw.xz + +# Verify the image +curl https://getfedora.org/static/fedora.gpg | gpg --import +wget https://getfedora.org/static/checksums/Fedora-Cloud-32-1.6-x86_64-CHECKSUM +gpg --verify-files *-CHECKSUM +sha256sum --ignore-missing -c *-CHECKSUM + +# Extract the image +unxz -T0 Fedora-Cloud-Base-32-1.6.x86_64.raw.xz + +# Search is needed for $HOME so virt service can access the image file. +chmod a+x $HOME + +# +# Modify the virtual image to: +# - Enable a login, we just use root +# - Enable passwordless login +# - Force a relabel to fix labels on ssh keys +# +sudo virt-sysprep -a "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ + --root-password password:123456 \ + --hostname fedoravm \ + --append-line '/etc/ssh/sshd_config:PermitRootLogin yes' \ + --append-line '/etc/ssh/sshd_config:PubkeyAuthentication yes' \ + --mkdir /root/.ssh \ + --upload "$HOME/.ssh/id_rsa.pub:/root/.ssh/authorized_keys" \ + --chmod '0600:/root/.ssh/authorized_keys' \ + --run-command 'chown root:root /root/.ssh/authorized_keys' \ + --copy-in "$TRAVIS_BUILD_DIR:/root" \ + --network \ + --selinux-relabel + +# +# Now we create a domain by using virt-install. This not only creates the domain, but runs the VM as well +# It should be ready to go for ssh, once ssh starts. +# +sudo virt-install \ + --name fedoravm \ + --memory $MEMORY \ + --vcpus $VCPUS \ + --disk "$HOME/Fedora-Cloud-Base-32-1.6.x86_64.raw" \ + --import --noautoconsole + +# +# Here comes the tricky part, we have to figure out when the VM comes up AND we need the ip address for ssh. So we +# can check the net-dhcp leases, for our host. We have to poll, and we will poll for up 3 minutes in 6 second +# intervals, so 30 poll attempts (0-29 inclusive). I don't know of a better way to do this. +# +# We have a full reboot + relabel, so first sleep gets us close +# +sleep 30 +for i in $(seq 0 29); do + echo "loop $i" + sleep 6s + # Get the leases, but tee it so it's easier to debug + sudo virsh net-dhcp-leases default | tee dhcp-leases.txt + + # get our ipaddress + ipaddy=$(grep fedoravm dhcp-leases.txt | awk {'print $5'} | cut -d'/' -f 1-1) + if [ -n "$ipaddy" ]; then + # found it, we're done looking, print it for debug logs + echo "ipaddy: $ipaddy" + break + fi + # it's empty/not found, loop back and try again. +done + +# Did we find it? If not die. +if [ -z "$ipaddy" ]; then + echo "ipaddy zero length, exiting with error 1" + exit 1 +fi + +# +# Great we have a host running, ssh into it. We specify -o so +# we don't get blocked on asking to add the servers key to +# our known_hosts. +# +ssh -tt -o StrictHostKeyChecking=no -o LogLevel=QUIET "root@$ipaddy" "/root/selinux/$TEST_RUNNER" + +exit 0