| Message ID | 20210118063808.12471-9-jiaxun.yang@flygoat.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Alpine Linux build fix and CI pipeline | expand |
On Mon, Jan 18, 2021 at 02:38:07PM +0800, Jiaxun Yang wrote: > Alpine Linux[1] is a security-oriented, lightweight Linux distribution > based on musl libc and busybox. > > It it popular among Docker guests and embedded applications. > > Adding it to test against different libc. > > [1]: https://alpinelinux.org/ > > Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> > --- > tests/docker/dockerfiles/alpine.docker | 57 ++++++++++++++++++++++++++ > 1 file changed, 57 insertions(+) > create mode 100644 tests/docker/dockerfiles/alpine.docker > > diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker > new file mode 100644 > index 0000000000..5be5198d00 > --- /dev/null > +++ b/tests/docker/dockerfiles/alpine.docker > @@ -0,0 +1,57 @@ > + > +FROM alpine:edge > + > +RUN apk update > +RUN apk upgrade > + > +# Please keep this list sorted alphabetically > +ENV PACKAGES \ > + alsa-lib-dev \ > + bash \ > + bison \ This shouldn't be required. > + build-base \ This seems to be a meta packae that pulls in other misc toolchain packages. Please list the pieces we need explicitly instead. > + coreutils \ > + curl-dev \ > + flex \ This shouldn't be needed. > + git \ > + glib-dev \ > + glib-static \ > + gnutls-dev \ > + gtk+3.0-dev \ > + libaio-dev \ > + libcap-dev \ Should not be required, as we use cap-ng. > + libcap-ng-dev \ > + libjpeg-turbo-dev \ > + libnfs-dev \ > + libpng-dev \ > + libseccomp-dev \ > + libssh-dev \ > + libusb-dev \ > + libxml2-dev \ > + linux-headers \ Is this really needed ? We don't install kernel-headers on other distros AFAICT. > + lzo-dev \ > + mesa-dev \ > + mesa-egl \ > + mesa-gbm \ > + meson \ > + ncurses-dev \ > + ninja \ > + paxmark \ What is this needed for ? > + perl \ > + pulseaudio-dev \ > + python3 \ > + py3-sphinx \ > + shadow \ Is this really needed ? > + snappy-dev \ > + spice-dev \ > + texinfo \ > + usbredir-dev \ > + util-linux-dev \ > + vde2-dev \ > + virglrenderer-dev \ > + vte3-dev \ > + xfsprogs-dev \ > + zlib-dev \ > + zlib-static > + > +RUN apk add $PACKAGES Regards, Daniel
On 18/01/2021 11.33, Daniel P. Berrangé wrote: > On Mon, Jan 18, 2021 at 02:38:07PM +0800, Jiaxun Yang wrote: >> Alpine Linux[1] is a security-oriented, lightweight Linux distribution >> based on musl libc and busybox. >> >> It it popular among Docker guests and embedded applications. >> >> Adding it to test against different libc. >> >> [1]: https://alpinelinux.org/ >> >> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> >> --- >> tests/docker/dockerfiles/alpine.docker | 57 ++++++++++++++++++++++++++ >> 1 file changed, 57 insertions(+) >> create mode 100644 tests/docker/dockerfiles/alpine.docker >> >> diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker >> new file mode 100644 >> index 0000000000..5be5198d00 >> --- /dev/null >> +++ b/tests/docker/dockerfiles/alpine.docker >> @@ -0,0 +1,57 @@ >> + >> +FROM alpine:edge >> + >> +RUN apk update >> +RUN apk upgrade >> + >> +# Please keep this list sorted alphabetically >> +ENV PACKAGES \ >> + alsa-lib-dev \ >> + bash \ >> + bison \ > > This shouldn't be required. bison and flex were required to avoid some warnings in the past while compiling the dtc submodule ... but I thought we got rid of the problem at one point in time, so this can be removed now, indeed. >> + build-base \ > > This seems to be a meta packae that pulls in other > misc toolchain packages. Please list the pieces we > need explicitly instead. Looking at the "Depends" list on https://pkgs.alpinelinux.org/package/v3.3/main/x86/build-base there are only 6 dependencies and we need most of those for QEMU anyway, so I think it is ok to keep build-base here. >> + coreutils \ >> + curl-dev \ >> + flex \ > > This shouldn't be needed. > >> + git \ >> + glib-dev \ >> + glib-static \ >> + gnutls-dev \ >> + gtk+3.0-dev \ >> + libaio-dev \ >> + libcap-dev \ > > Should not be required, as we use cap-ng. Right. >> + libcap-ng-dev \ >> + libjpeg-turbo-dev \ >> + libnfs-dev \ >> + libpng-dev \ >> + libseccomp-dev \ >> + libssh-dev \ >> + libusb-dev \ >> + libxml2-dev \ >> + linux-headers \ > > Is this really needed ? We don't install kernel-headers on other > distros AFAICT. I tried a build without this package, and it works fine indeed. >> + lzo-dev \ >> + mesa-dev \ >> + mesa-egl \ >> + mesa-gbm \ >> + meson \ >> + ncurses-dev \ >> + ninja \ >> + paxmark \ > > What is this needed for ? Seems like it also can be dropped. >> + perl \ >> + pulseaudio-dev \ >> + python3 \ >> + py3-sphinx \ >> + shadow \ > > Is this really needed ? See: https://www.spinics.net/lists/kvm/msg231556.html I can remove the superfluous packages when picking up the patch, no need to respin just because of this. Thomas
On Tue, Jan 19, 2021 at 02:41:47PM +0100, Thomas Huth wrote: > On 18/01/2021 11.33, Daniel P. Berrangé wrote: > > On Mon, Jan 18, 2021 at 02:38:07PM +0800, Jiaxun Yang wrote: > > > Alpine Linux[1] is a security-oriented, lightweight Linux distribution > > > based on musl libc and busybox. > > > > > > It it popular among Docker guests and embedded applications. > > > > > > Adding it to test against different libc. > > > > > > [1]: https://alpinelinux.org/ > > > > > > Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> > > > --- > > > tests/docker/dockerfiles/alpine.docker | 57 ++++++++++++++++++++++++++ > > > 1 file changed, 57 insertions(+) > > > create mode 100644 tests/docker/dockerfiles/alpine.docker > > > > > > diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker > > > new file mode 100644 > > > index 0000000000..5be5198d00 > > > --- /dev/null > > > +++ b/tests/docker/dockerfiles/alpine.docker > > > @@ -0,0 +1,57 @@ > > > + > > > +FROM alpine:edge > > > + > > > +RUN apk update > > > +RUN apk upgrade > > > + > > > +# Please keep this list sorted alphabetically > > > +ENV PACKAGES \ > > > + alsa-lib-dev \ > > > + bash \ > > > + bison \ > > > > This shouldn't be required. > > bison and flex were required to avoid some warnings in the past while > compiling the dtc submodule ... but I thought we got rid of the problem at > one point in time, so this can be removed now, indeed. > > > > + build-base \ > > > > This seems to be a meta packae that pulls in other > > misc toolchain packages. Please list the pieces we > > need explicitly instead. > > Looking at the "Depends" list on > https://pkgs.alpinelinux.org/package/v3.3/main/x86/build-base there are only > 6 dependencies and we need most of those for QEMU anyway, so I think it is > ok to keep build-base here. I would like to keep them all explicit, as it makes it easier for us to ensure that we have provided the same set of dependancies across all our dockerfiles. Ideally we'll add Alpiine to libvirt-ci, so that we can then auto-generate this dockerfile in future. > > > + perl \ > > > + pulseaudio-dev \ > > > + python3 \ > > > + py3-sphinx \ > > > + shadow \ > > > > Is this really needed ? > > See: > https://www.spinics.net/lists/kvm/msg231556.html Ok, so this is required by the docker.py commands running extra tools. The other dockerfiles are working just by luck, and we should make this package expicit on all of them too Regards, Daniel
On 1/19/21 8:41 AM, Thomas Huth wrote: > On 18/01/2021 11.33, Daniel P. Berrangé wrote: >> On Mon, Jan 18, 2021 at 02:38:07PM +0800, Jiaxun Yang wrote: >>> Alpine Linux[1] is a security-oriented, lightweight Linux distribution >>> based on musl libc and busybox. >>> >>> It it popular among Docker guests and embedded applications. >>> >>> Adding it to test against different libc. >>> >>> [1]: https://alpinelinux.org/ >>> >>> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> >>> --- >>> tests/docker/dockerfiles/alpine.docker | 57 ++++++++++++++++++++++++++ >>> 1 file changed, 57 insertions(+) >>> create mode 100644 tests/docker/dockerfiles/alpine.docker >>> >>> diff --git a/tests/docker/dockerfiles/alpine.docker >>> b/tests/docker/dockerfiles/alpine.docker >>> new file mode 100644 >>> index 0000000000..5be5198d00 >>> --- /dev/null >>> +++ b/tests/docker/dockerfiles/alpine.docker >>> @@ -0,0 +1,57 @@ >>> + >>> +FROM alpine:edge >>> + >>> +RUN apk update >>> +RUN apk upgrade >>> + >>> +# Please keep this list sorted alphabetically >>> +ENV PACKAGES \ >>> + alsa-lib-dev \ >>> + bash \ >>> + bison \ >> >> This shouldn't be required. > > bison and flex were required to avoid some warnings in the past while > compiling the dtc submodule ... but I thought we got rid of the problem > at one point in time, so this can be removed now, indeed. > >>> + build-base \ >> >> This seems to be a meta packae that pulls in other >> misc toolchain packages. Please list the pieces we >> need explicitly instead. > > Looking at the "Depends" list on > https://pkgs.alpinelinux.org/package/v3.3/main/x86/build-base there are > only 6 dependencies and we need most of those for QEMU anyway, so I > think it is ok to keep build-base here. > >>> + coreutils \ >>> + curl-dev \ >>> + flex \ >> >> This shouldn't be needed. >> >>> + git \ >>> + glib-dev \ >>> + glib-static \ >>> + gnutls-dev \ >>> + gtk+3.0-dev \ >>> + libaio-dev \ >>> + libcap-dev \ >> >> Should not be required, as we use cap-ng. > > Right. > >>> + libcap-ng-dev \ >>> + libjpeg-turbo-dev \ >>> + libnfs-dev \ >>> + libpng-dev \ >>> + libseccomp-dev \ >>> + libssh-dev \ >>> + libusb-dev \ >>> + libxml2-dev \ >>> + linux-headers \ >> >> Is this really needed ? We don't install kernel-headers on other >> distros AFAICT. > > I tried a build without this package, and it works fine indeed. > >>> + lzo-dev \ >>> + mesa-dev \ >>> + mesa-egl \ >>> + mesa-gbm \ >>> + meson \ >>> + ncurses-dev \ >>> + ninja \ >>> + paxmark \ >> >> What is this needed for ? > > Seems like it also can be dropped. > >>> + perl \ >>> + pulseaudio-dev \ >>> + python3 \ >>> + py3-sphinx \ >>> + shadow \ >> >> Is this really needed ? > > See: > https://www.spinics.net/lists/kvm/msg231556.html > > I can remove the superfluous packages when picking up the patch, no need > to respin just because of this. > > Thomas > > You can refer to my post earlier this January for a "minimal" Alpine Linux build, if you wish. My goal was to find the smallest set of packages possible without passing any explicit configure flags. I wonder if it's worth having layered "core build" and "test build" images so that we can smoke test the minimalistic build from time to time -- I seem to recall Dan posting information about a dependency management tool for Dockerfiles, but I admit I didn't look too closely at what problem that solves, exactly. --js
On Tue, Jan 26, 2021 at 04:38:57PM -0500, John Snow wrote: > On 1/19/21 8:41 AM, Thomas Huth wrote: > > On 18/01/2021 11.33, Daniel P. Berrangé wrote: > > > On Mon, Jan 18, 2021 at 02:38:07PM +0800, Jiaxun Yang wrote: > > > > Alpine Linux[1] is a security-oriented, lightweight Linux distribution > > > > based on musl libc and busybox. > > > > > > > > It it popular among Docker guests and embedded applications. > > > > > > > > Adding it to test against different libc. > > > > > > > > [1]: https://alpinelinux.org/ > > > > > > > > Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> > > > > --- > > > > tests/docker/dockerfiles/alpine.docker | 57 ++++++++++++++++++++++++++ > > > > 1 file changed, 57 insertions(+) > > > > create mode 100644 tests/docker/dockerfiles/alpine.docker > > > > > > > > diff --git a/tests/docker/dockerfiles/alpine.docker > > > > b/tests/docker/dockerfiles/alpine.docker > > > > new file mode 100644 > > > > index 0000000000..5be5198d00 > > > > --- /dev/null > > > > +++ b/tests/docker/dockerfiles/alpine.docker > > > > @@ -0,0 +1,57 @@ > > > > + > > > > +FROM alpine:edge > > > > + > > > > +RUN apk update > > > > +RUN apk upgrade > > > > + > > > > +# Please keep this list sorted alphabetically > > > > +ENV PACKAGES \ > > > > + alsa-lib-dev \ > > > > + bash \ > > > > + bison \ > > > > > > This shouldn't be required. > > > > bison and flex were required to avoid some warnings in the past while > > compiling the dtc submodule ... but I thought we got rid of the problem > > at one point in time, so this can be removed now, indeed. > > > > > > + build-base \ > > > > > > This seems to be a meta packae that pulls in other > > > misc toolchain packages. Please list the pieces we > > > need explicitly instead. > > > > Looking at the "Depends" list on > > https://pkgs.alpinelinux.org/package/v3.3/main/x86/build-base there are > > only 6 dependencies and we need most of those for QEMU anyway, so I > > think it is ok to keep build-base here. > > > > > > + coreutils \ > > > > + curl-dev \ > > > > + flex \ > > > > > > This shouldn't be needed. > > > > > > > + git \ > > > > + glib-dev \ > > > > + glib-static \ > > > > + gnutls-dev \ > > > > + gtk+3.0-dev \ > > > > + libaio-dev \ > > > > + libcap-dev \ > > > > > > Should not be required, as we use cap-ng. > > > > Right. > > > > > > + libcap-ng-dev \ > > > > + libjpeg-turbo-dev \ > > > > + libnfs-dev \ > > > > + libpng-dev \ > > > > + libseccomp-dev \ > > > > + libssh-dev \ > > > > + libusb-dev \ > > > > + libxml2-dev \ > > > > + linux-headers \ > > > > > > Is this really needed ? We don't install kernel-headers on other > > > distros AFAICT. > > > > I tried a build without this package, and it works fine indeed. > > > > > > + lzo-dev \ > > > > + mesa-dev \ > > > > + mesa-egl \ > > > > + mesa-gbm \ > > > > + meson \ > > > > + ncurses-dev \ > > > > + ninja \ > > > > + paxmark \ > > > > > > What is this needed for ? > > > > Seems like it also can be dropped. > > > > > > + perl \ > > > > + pulseaudio-dev \ > > > > + python3 \ > > > > + py3-sphinx \ > > > > + shadow \ > > > > > > Is this really needed ? > > > > See: > > https://www.spinics.net/lists/kvm/msg231556.html > > > > I can remove the superfluous packages when picking up the patch, no need > > to respin just because of this. > > > > Thomas > > > > > > You can refer to my post earlier this January for a "minimal" Alpine Linux > build, if you wish. > > My goal was to find the smallest set of packages possible without passing > any explicit configure flags. > > I wonder if it's worth having layered "core build" and "test build" images > so that we can smoke test the minimalistic build from time to time -- I seem > to recall Dan posting information about a dependency management tool for > Dockerfiles, but I admit I didn't look too closely at what problem that > solves, exactly. I'd rather we avoid layered images entirely as it creates extra stages in the gitlab pipeline, and also makes it more complex to auto-generate. Once this initial alpine image is merged, then I intend to add it to the set which are auto-generated from my other patch series. Regards, Daniel
On 1/27/21 4:58 AM, Daniel P. Berrangé wrote: > On Tue, Jan 26, 2021 at 04:38:57PM -0500, John Snow wrote: >> On 1/19/21 8:41 AM, Thomas Huth wrote: >>> On 18/01/2021 11.33, Daniel P. Berrangé wrote: >>>> On Mon, Jan 18, 2021 at 02:38:07PM +0800, Jiaxun Yang wrote: >>>>> Alpine Linux[1] is a security-oriented, lightweight Linux distribution >>>>> based on musl libc and busybox. >>>>> >>>>> It it popular among Docker guests and embedded applications. >>>>> >>>>> Adding it to test against different libc. >>>>> >>>>> [1]: https://alpinelinux.org/ >>>>> >>>>> Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> >>>>> --- >>>>> tests/docker/dockerfiles/alpine.docker | 57 ++++++++++++++++++++++++++ >>>>> 1 file changed, 57 insertions(+) >>>>> create mode 100644 tests/docker/dockerfiles/alpine.docker >>>>> >>>>> diff --git a/tests/docker/dockerfiles/alpine.docker >>>>> b/tests/docker/dockerfiles/alpine.docker >>>>> new file mode 100644 >>>>> index 0000000000..5be5198d00 >>>>> --- /dev/null >>>>> +++ b/tests/docker/dockerfiles/alpine.docker >>>>> @@ -0,0 +1,57 @@ >>>>> + >>>>> +FROM alpine:edge >>>>> + >>>>> +RUN apk update >>>>> +RUN apk upgrade >>>>> + >>>>> +# Please keep this list sorted alphabetically >>>>> +ENV PACKAGES \ >>>>> + alsa-lib-dev \ >>>>> + bash \ >>>>> + bison \ >>>> >>>> This shouldn't be required. >>> >>> bison and flex were required to avoid some warnings in the past while >>> compiling the dtc submodule ... but I thought we got rid of the problem >>> at one point in time, so this can be removed now, indeed. >>> >>>>> + build-base \ >>>> >>>> This seems to be a meta packae that pulls in other >>>> misc toolchain packages. Please list the pieces we >>>> need explicitly instead. >>> >>> Looking at the "Depends" list on >>> https://pkgs.alpinelinux.org/package/v3.3/main/x86/build-base there are >>> only 6 dependencies and we need most of those for QEMU anyway, so I >>> think it is ok to keep build-base here. >>> >>>>> + coreutils \ >>>>> + curl-dev \ >>>>> + flex \ >>>> >>>> This shouldn't be needed. >>>> >>>>> + git \ >>>>> + glib-dev \ >>>>> + glib-static \ >>>>> + gnutls-dev \ >>>>> + gtk+3.0-dev \ >>>>> + libaio-dev \ >>>>> + libcap-dev \ >>>> >>>> Should not be required, as we use cap-ng. >>> >>> Right. >>> >>>>> + libcap-ng-dev \ >>>>> + libjpeg-turbo-dev \ >>>>> + libnfs-dev \ >>>>> + libpng-dev \ >>>>> + libseccomp-dev \ >>>>> + libssh-dev \ >>>>> + libusb-dev \ >>>>> + libxml2-dev \ >>>>> + linux-headers \ >>>> >>>> Is this really needed ? We don't install kernel-headers on other >>>> distros AFAICT. >>> >>> I tried a build without this package, and it works fine indeed. >>> >>>>> + lzo-dev \ >>>>> + mesa-dev \ >>>>> + mesa-egl \ >>>>> + mesa-gbm \ >>>>> + meson \ >>>>> + ncurses-dev \ >>>>> + ninja \ >>>>> + paxmark \ >>>> >>>> What is this needed for ? >>> >>> Seems like it also can be dropped. >>> >>>>> + perl \ >>>>> + pulseaudio-dev \ >>>>> + python3 \ >>>>> + py3-sphinx \ >>>>> + shadow \ >>>> >>>> Is this really needed ? >>> >>> See: >>> https://www.spinics.net/lists/kvm/msg231556.html >>> >>> I can remove the superfluous packages when picking up the patch, no need >>> to respin just because of this. >>> >>> Thomas >>> >>> >> >> You can refer to my post earlier this January for a "minimal" Alpine Linux >> build, if you wish. >> >> My goal was to find the smallest set of packages possible without passing >> any explicit configure flags. >> >> I wonder if it's worth having layered "core build" and "test build" images >> so that we can smoke test the minimalistic build from time to time -- I seem >> to recall Dan posting information about a dependency management tool for >> Dockerfiles, but I admit I didn't look too closely at what problem that >> solves, exactly. > > I'd rather we avoid layered images entirely as it creates extra stages > in the gitlab pipeline, and also makes it more complex to auto-generate. > > Once this initial alpine image is merged, then I intend to add it to the > set which are auto-generated from my other patch series. > OK, happy to defer to you here for best practices. --js
diff --git a/tests/docker/dockerfiles/alpine.docker b/tests/docker/dockerfiles/alpine.docker new file mode 100644 index 0000000000..5be5198d00 --- /dev/null +++ b/tests/docker/dockerfiles/alpine.docker @@ -0,0 +1,57 @@ + +FROM alpine:edge + +RUN apk update +RUN apk upgrade + +# Please keep this list sorted alphabetically +ENV PACKAGES \ + alsa-lib-dev \ + bash \ + bison \ + build-base \ + coreutils \ + curl-dev \ + flex \ + git \ + glib-dev \ + glib-static \ + gnutls-dev \ + gtk+3.0-dev \ + libaio-dev \ + libcap-dev \ + libcap-ng-dev \ + libjpeg-turbo-dev \ + libnfs-dev \ + libpng-dev \ + libseccomp-dev \ + libssh-dev \ + libusb-dev \ + libxml2-dev \ + linux-headers \ + lzo-dev \ + mesa-dev \ + mesa-egl \ + mesa-gbm \ + meson \ + ncurses-dev \ + ninja \ + paxmark \ + perl \ + pulseaudio-dev \ + python3 \ + py3-sphinx \ + shadow \ + snappy-dev \ + spice-dev \ + texinfo \ + usbredir-dev \ + util-linux-dev \ + vde2-dev \ + virglrenderer-dev \ + vte3-dev \ + xfsprogs-dev \ + zlib-dev \ + zlib-static + +RUN apk add $PACKAGES
Alpine Linux[1] is a security-oriented, lightweight Linux distribution based on musl libc and busybox. It it popular among Docker guests and embedded applications. Adding it to test against different libc. [1]: https://alpinelinux.org/ Signed-off-by: Jiaxun Yang <jiaxun.yang@flygoat.com> --- tests/docker/dockerfiles/alpine.docker | 57 ++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 tests/docker/dockerfiles/alpine.docker