| Message ID | 20241118172357.475281-5-pierrick.bouvier@linaro.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | Enhance documentation for new developers | expand |
On Mon, Nov 18, 2024 at 09:23:54AM -0800, Pierrick Bouvier wrote: > MacOS and Linux are straightforward, but Windows needs a bit more > details. > > Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> > --- > docs/about/build-platforms.rst | 4 +- > docs/devel/build-environment.rst | 114 +++++++++++++++++++++++++++++++ > docs/devel/index-build.rst | 1 + > 3 files changed, 118 insertions(+), 1 deletion(-) > create mode 100644 docs/devel/build-environment.rst > > diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst > index 6102f00aec0..c1ea53db834 100644 > --- a/docs/about/build-platforms.rst > +++ b/docs/about/build-platforms.rst > @@ -29,6 +29,9 @@ The `Repology`_ site is a useful resource to identify > currently shipped versions of software in various operating systems, > though it does not cover all distros listed below. > > +You can find how to install build dependencies for different systems on the > +:ref:`setup-build-env` page. > + > Supported host architectures > ---------------------------- > > @@ -130,7 +133,6 @@ Optional build dependencies > cross compilation using ``docker`` or ``podman``, or to use pre-built > binaries distributed with QEMU. > > - > Windows > ------- > > diff --git a/docs/devel/build-environment.rst b/docs/devel/build-environment.rst > new file mode 100644 > index 00000000000..d9a66f5fcc6 > --- /dev/null > +++ b/docs/devel/build-environment.rst > @@ -0,0 +1,114 @@ > + > +.. _setup-build-env: > + > +Setup build environment > +======================= > + > +QEMU uses a lot of dependencies on the host system. glib2 is used everywhere in > +the code base, and most of the other dependencies are optional. > + > +We present here simple instructions to enable native builds on most popular > +systems. > + > +You can find additional instructions on `QEMU wiki <https://wiki.qemu.org/>`_: > + > +- `Linux <https://wiki.qemu.org/Hosts/Linux>`_ > +- `MacOS <https://wiki.qemu.org/Hosts/Mac>`_ > +- `Windows <https://wiki.qemu.org/Hosts/W32>`_ > +- `BSD <https://wiki.qemu.org/Hosts/BSD>`_ We generally suffer from having information spread over multiple sources, giving us the burden of keeping the different places consistent, which we pretty consistently fail at. It is a good think to add build env docs to qemu.git where we actually have oversight / review to catch mistakes. If we do this though, IMHO, we should be deleting the wiki pages and making them into 302 redirects to our new in-tree docs. > + > +Linux > +----- > + > +Fedora > +++++++ > + > +:: > + > + sudo dnf update && sudo dnf builddep qemu > + > +Debian/Ubuntu > ++++++++++++++ > + > +You first need to enable `Sources List <https://wiki.debian.org/SourcesList>`_. > +Then, use apt to install dependencies: > + > +:: > + > + sudo apt update && sudo apt build-dep qemu > + > +MacOS > +----- > + > +You first need to install `Homebrew <https://brew.sh/>`_. Then, use it to > +install dependencies: > + > +:: > + > + brew update && brew install $(brew deps --include-build qemu) The downside in recommending the 'build dep' approach is that it misses out on deps that have been newly introduced in qemu.git, since whatever old version of QEMU the distros are shipping. It also misses deps for any features the distro vendor has decided to exclude. Can we put a caveat describing this limitation at the top, so that users have pointer if things don't quite go the way we expected. > + > +Windows > +------- > + > +You first need to install `MSYS2 <https://www.msys2.org/>`_. > +MSYS2 offers `different environments <https://www.msys2.org/docs/environments/>`_. > +x86_64 environments are based on GCC, while aarch64 is based on Clang. > + > +We recommend to use MINGW64 for windows-x86_64 and CLANGARM64 for windows-aarch64 > +(only available on windows-aarch64 hosts). Does CLANGARM64 really work with QEMU ? We go out of our way to actively block the use of CLang for Windows because of its lack of support for 'gcc_struct' attributes, so I would have expected it to fail > + > +Then, you can open a windows shell, and enter msys2 env using: > + > +:: > + > + c:/msys64/msys2_shell.cmd -defterm -here -no-start -mingw64 > + # Replace -ucrt64 by -clangarm64 or -ucrt64 for other environments. > + > +MSYS2 package manager does not offer a built-in way to install build > +dependencies. You can start with this list of packages using pacman: > + With regards, Daniel
Daniel P. Berrangé <berrange@redhat.com> writes: > On Mon, Nov 18, 2024 at 09:23:54AM -0800, Pierrick Bouvier wrote: >> MacOS and Linux are straightforward, but Windows needs a bit more >> details. >> >> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> >> --- >> docs/about/build-platforms.rst | 4 +- >> docs/devel/build-environment.rst | 114 +++++++++++++++++++++++++++++++ >> docs/devel/index-build.rst | 1 + >> 3 files changed, 118 insertions(+), 1 deletion(-) >> create mode 100644 docs/devel/build-environment.rst >> <snip> >> +Fedora >> +++++++ >> + >> +:: >> + >> + sudo dnf update && sudo dnf builddep qemu >> + >> +Debian/Ubuntu >> ++++++++++++++ >> + >> +You first need to enable `Sources List <https://wiki.debian.org/SourcesList>`_. >> +Then, use apt to install dependencies: >> + >> +:: >> + >> + sudo apt update && sudo apt build-dep qemu >> + >> +MacOS >> +----- >> + >> +You first need to install `Homebrew <https://brew.sh/>`_. Then, use it to >> +install dependencies: >> + >> +:: >> + >> + brew update && brew install $(brew deps --include-build qemu) > > > The downside in recommending the 'build dep' approach is that it misses > out on deps that have been newly introduced in qemu.git, since whatever > old version of QEMU the distros are shipping. It also misses deps for > any features the distro vendor has decided to exclude. > > Can we put a caveat describing this limitation at the top, so that users > have pointer if things don't quite go the way we expected. At a recent QEMU workshop I presented I pointed to lcitool as a way to list the minimal dependencies QEMU needs: ./tests/lcitool/libvirt-ci/bin/lcitool variables -f yaml debian-12 ./tests/lcitool/projects/qemu-minimal.yml It wouldn't be the hardest thing to extend lcitool to generate a pkg install line instead of a list. Although I notice it fails with the full set of dependancies: ✗ ./tests/lcitool/libvirt-ci/bin/lcitool -d tests/lcitool variables -f shell debian-12 ./tests/lcitool/projects/qemu.yml Package generic name resolution error: Package libcbor not present in mappings I note there is already an "install" action but I'm wary of what it does. > >> + >> +Windows >> +------- >> + >> +You first need to install `MSYS2 <https://www.msys2.org/>`_. >> +MSYS2 offers `different environments <https://www.msys2.org/docs/environments/>`_. >> +x86_64 environments are based on GCC, while aarch64 is based on Clang. >> + >> +We recommend to use MINGW64 for windows-x86_64 and CLANGARM64 for windows-aarch64 >> +(only available on windows-aarch64 hosts). > > Does CLANGARM64 really work with QEMU ? We go out of our way to actively > block the use of CLang for Windows because of its lack of support for > 'gcc_struct' attributes, so I would have expected it to fail > >> + >> +Then, you can open a windows shell, and enter msys2 env using: >> + >> +:: >> + >> + c:/msys64/msys2_shell.cmd -defterm -here -no-start -mingw64 >> + # Replace -ucrt64 by -clangarm64 or -ucrt64 for other environments. >> + >> +MSYS2 package manager does not offer a built-in way to install build >> +dependencies. You can start with this list of packages using pacman: >> + > > With regards, > Daniel
On Tue, Nov 19, 2024 at 11:08:12AM +0000, Alex Bennée wrote: > Daniel P. Berrangé <berrange@redhat.com> writes: > > > On Mon, Nov 18, 2024 at 09:23:54AM -0800, Pierrick Bouvier wrote: > >> MacOS and Linux are straightforward, but Windows needs a bit more > >> details. > >> > >> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> > >> --- > >> docs/about/build-platforms.rst | 4 +- > >> docs/devel/build-environment.rst | 114 +++++++++++++++++++++++++++++++ > >> docs/devel/index-build.rst | 1 + > >> 3 files changed, 118 insertions(+), 1 deletion(-) > >> create mode 100644 docs/devel/build-environment.rst > >> > <snip> > >> +Fedora > >> +++++++ > >> + > >> +:: > >> + > >> + sudo dnf update && sudo dnf builddep qemu > >> + > >> +Debian/Ubuntu > >> ++++++++++++++ > >> + > >> +You first need to enable `Sources List <https://wiki.debian.org/SourcesList>`_. > >> +Then, use apt to install dependencies: > >> + > >> +:: > >> + > >> + sudo apt update && sudo apt build-dep qemu > >> + > >> +MacOS > >> +----- > >> + > >> +You first need to install `Homebrew <https://brew.sh/>`_. Then, use it to > >> +install dependencies: > >> + > >> +:: > >> + > >> + brew update && brew install $(brew deps --include-build qemu) > > > > > > The downside in recommending the 'build dep' approach is that it misses > > out on deps that have been newly introduced in qemu.git, since whatever > > old version of QEMU the distros are shipping. It also misses deps for > > any features the distro vendor has decided to exclude. > > > > Can we put a caveat describing this limitation at the top, so that users > > have pointer if things don't quite go the way we expected. > > At a recent QEMU workshop I presented I pointed to lcitool as a way to > list the minimal dependencies QEMU needs: > > ./tests/lcitool/libvirt-ci/bin/lcitool variables -f yaml debian-12 ./tests/lcitool/projects/qemu-minimal.yml > > It wouldn't be the hardest thing to extend lcitool to generate a pkg > install line instead of a list. Although I notice it fails with the full > set of dependancies: > > ✗ ./tests/lcitool/libvirt-ci/bin/lcitool -d tests/lcitool variables -f shell debian-12 ./tests/lcitool/projects/qemu.yml > Package generic name resolution error: Package libcbor not present in mappings I think your libvirt-ci git submodule might be out of date. The current generated Dockerfiles include libcbor and its in the mappings. > I note there is already an "install" action but I'm wary of what it does. That's related to lcitool functionality for provisioning VMs - kinda like QEMUs tests/vm/ stuff. With regards, Daniel
On 11/19/24 01:24, Daniel P. Berrangé wrote: > On Mon, Nov 18, 2024 at 09:23:54AM -0800, Pierrick Bouvier wrote: >> MacOS and Linux are straightforward, but Windows needs a bit more >> details. >> >> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> >> --- >> docs/about/build-platforms.rst | 4 +- >> docs/devel/build-environment.rst | 114 +++++++++++++++++++++++++++++++ >> docs/devel/index-build.rst | 1 + >> 3 files changed, 118 insertions(+), 1 deletion(-) >> create mode 100644 docs/devel/build-environment.rst >> >> diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst >> index 6102f00aec0..c1ea53db834 100644 >> --- a/docs/about/build-platforms.rst >> +++ b/docs/about/build-platforms.rst >> @@ -29,6 +29,9 @@ The `Repology`_ site is a useful resource to identify >> currently shipped versions of software in various operating systems, >> though it does not cover all distros listed below. >> >> +You can find how to install build dependencies for different systems on the >> +:ref:`setup-build-env` page. >> + >> Supported host architectures >> ---------------------------- >> >> @@ -130,7 +133,6 @@ Optional build dependencies >> cross compilation using ``docker`` or ``podman``, or to use pre-built >> binaries distributed with QEMU. >> >> - >> Windows >> ------- >> >> diff --git a/docs/devel/build-environment.rst b/docs/devel/build-environment.rst >> new file mode 100644 >> index 00000000000..d9a66f5fcc6 >> --- /dev/null >> +++ b/docs/devel/build-environment.rst >> @@ -0,0 +1,114 @@ >> + >> +.. _setup-build-env: >> + >> +Setup build environment >> +======================= >> + >> +QEMU uses a lot of dependencies on the host system. glib2 is used everywhere in >> +the code base, and most of the other dependencies are optional. >> + >> +We present here simple instructions to enable native builds on most popular >> +systems. >> + >> +You can find additional instructions on `QEMU wiki <https://wiki.qemu.org/>`_: >> + >> +- `Linux <https://wiki.qemu.org/Hosts/Linux>`_ >> +- `MacOS <https://wiki.qemu.org/Hosts/Mac>`_ >> +- `Windows <https://wiki.qemu.org/Hosts/W32>`_ >> +- `BSD <https://wiki.qemu.org/Hosts/BSD>`_ > > > We generally suffer from having information spread over multiple sources, > giving us the burden of keeping the different places consistent, which we > pretty consistently fail at. > Agree on that. > It is a good think to add build env docs to qemu.git where we actually > have oversight / review to catch mistakes. If we do this though, IMHO, > we should be deleting the wiki pages and making them into 302 redirects > to our new in-tree docs. > Sure. However, there is still some useful information there, that are not reflected in this patch. We could migrate the information later, but for now, I can't spend many cycles to try to build all configurations described on the wiki to make sure it's up to date. So I just included information I'm sure it works, and that cover the basic needs to build on all major systems. Hope this is a starting point we can maybe accept, and we can enhance the documentation later. But asking to do all this at once is a big task, and I feel it's a lot to ask for it as part of the current series. Would you accept that, or prefer to not include anything before we have purged the wikis completely? >> + >> +Linux >> +----- >> + >> +Fedora >> +++++++ >> + >> +:: >> + >> + sudo dnf update && sudo dnf builddep qemu >> + >> +Debian/Ubuntu >> ++++++++++++++ >> + >> +You first need to enable `Sources List <https://wiki.debian.org/SourcesList>`_. >> +Then, use apt to install dependencies: >> + >> +:: >> + >> + sudo apt update && sudo apt build-dep qemu >> + >> +MacOS >> +----- >> + >> +You first need to install `Homebrew <https://brew.sh/>`_. Then, use it to >> +install dependencies: >> + >> +:: >> + >> + brew update && brew install $(brew deps --include-build qemu) > > > The downside in recommending the 'build dep' approach is that it misses > out on deps that have been newly introduced in qemu.git, since whatever > old version of QEMU the distros are shipping. It also misses deps for > any features the distro vendor has decided to exclude. > > Can we put a caveat describing this limitation at the top, so that users > have pointer if things don't quite go the way we expected. > I can add a note about it. It's especially true for Linux, as on Windows (MSYS2) and MacOS (homebrew), they usually track latest release of QEMU, so the build list should match, given a reduced time frame after the release. >> + >> +Windows >> +------- >> + >> +You first need to install `MSYS2 <https://www.msys2.org/>`_. >> +MSYS2 offers `different environments <https://www.msys2.org/docs/environments/>`_. >> +x86_64 environments are based on GCC, while aarch64 is based on Clang. >> + >> +We recommend to use MINGW64 for windows-x86_64 and CLANGARM64 for windows-aarch64 >> +(only available on windows-aarch64 hosts). > > Does CLANGARM64 really work with QEMU ? We go out of our way to actively > block the use of CLang for Windows because of its lack of support for > 'gcc_struct' attributes, so I would have expected it to fail > I can drop the windows-arm64 part. For the support, it's a discussion we add previously on the concerned series. In current master, it won't even go past the configure step because of intentional restriction we have in meson.build. My initial hope was to remove it, but I understand that we prefer to wait for gcc_struct support in clang. So be it, I can live with a local fix for that meanwhile :). >> + >> +Then, you can open a windows shell, and enter msys2 env using: >> + >> +:: >> + >> + c:/msys64/msys2_shell.cmd -defterm -here -no-start -mingw64 >> + # Replace -ucrt64 by -clangarm64 or -ucrt64 for other environments. >> + >> +MSYS2 package manager does not offer a built-in way to install build >> +dependencies. You can start with this list of packages using pacman: >> + > > With regards, > Daniel
diff --git a/docs/about/build-platforms.rst b/docs/about/build-platforms.rst index 6102f00aec0..c1ea53db834 100644 --- a/docs/about/build-platforms.rst +++ b/docs/about/build-platforms.rst @@ -29,6 +29,9 @@ The `Repology`_ site is a useful resource to identify currently shipped versions of software in various operating systems, though it does not cover all distros listed below. +You can find how to install build dependencies for different systems on the +:ref:`setup-build-env` page. + Supported host architectures ---------------------------- @@ -130,7 +133,6 @@ Optional build dependencies cross compilation using ``docker`` or ``podman``, or to use pre-built binaries distributed with QEMU. - Windows ------- diff --git a/docs/devel/build-environment.rst b/docs/devel/build-environment.rst new file mode 100644 index 00000000000..d9a66f5fcc6 --- /dev/null +++ b/docs/devel/build-environment.rst @@ -0,0 +1,114 @@ + +.. _setup-build-env: + +Setup build environment +======================= + +QEMU uses a lot of dependencies on the host system. glib2 is used everywhere in +the code base, and most of the other dependencies are optional. + +We present here simple instructions to enable native builds on most popular +systems. + +You can find additional instructions on `QEMU wiki <https://wiki.qemu.org/>`_: + +- `Linux <https://wiki.qemu.org/Hosts/Linux>`_ +- `MacOS <https://wiki.qemu.org/Hosts/Mac>`_ +- `Windows <https://wiki.qemu.org/Hosts/W32>`_ +- `BSD <https://wiki.qemu.org/Hosts/BSD>`_ + +Linux +----- + +Fedora +++++++ + +:: + + sudo dnf update && sudo dnf builddep qemu + +Debian/Ubuntu ++++++++++++++ + +You first need to enable `Sources List <https://wiki.debian.org/SourcesList>`_. +Then, use apt to install dependencies: + +:: + + sudo apt update && sudo apt build-dep qemu + +MacOS +----- + +You first need to install `Homebrew <https://brew.sh/>`_. Then, use it to +install dependencies: + +:: + + brew update && brew install $(brew deps --include-build qemu) + +Windows +------- + +You first need to install `MSYS2 <https://www.msys2.org/>`_. +MSYS2 offers `different environments <https://www.msys2.org/docs/environments/>`_. +x86_64 environments are based on GCC, while aarch64 is based on Clang. + +We recommend to use MINGW64 for windows-x86_64 and CLANGARM64 for windows-aarch64 +(only available on windows-aarch64 hosts). + +Then, you can open a windows shell, and enter msys2 env using: + +:: + + c:/msys64/msys2_shell.cmd -defterm -here -no-start -mingw64 + # Replace -ucrt64 by -clangarm64 or -ucrt64 for other environments. + +MSYS2 package manager does not offer a built-in way to install build +dependencies. You can start with this list of packages using pacman: + +Note: Dependencies need to be installed again if you use a different MSYS2 +environment. + +:: + + # update MSYS2 itself, you need to reopen your shell at the end. + pacman -Syu + pacman -S \ + base-devel binutils bison diffutils flex git grep make sed \ + ${MINGW_PACKAGE_PREFIX}-toolchain \ + ${MINGW_PACKAGE_PREFIX}-glib2 \ + ${MINGW_PACKAGE_PREFIX}-gtk3 \ + ${MINGW_PACKAGE_PREFIX}-libnfs \ + ${MINGW_PACKAGE_PREFIX}-libssh \ + ${MINGW_PACKAGE_PREFIX}-ninja \ + ${MINGW_PACKAGE_PREFIX}-pixman \ + ${MINGW_PACKAGE_PREFIX}-pkgconf \ + ${MINGW_PACKAGE_PREFIX}-python \ + ${MINGW_PACKAGE_PREFIX}-SDL2 \ + ${MINGW_PACKAGE_PREFIX}-zstd + +If you want to install all dependencies, it's possible to use recipe used to +build QEMU in MSYS2 itself. + +:: + + pacman -S wget + wget https://raw.githubusercontent.com/msys2/MINGW-packages/refs/heads/master/mingw-w64-qemu/PKGBUILD + # Some packages may be missing for your environment, installation will still + # be done though. + makepkg -s PKGBUILD || true + +Build on windows-aarch64 +++++++++++++++++++++++++ + +When trying to cross compile meson for x86_64 using UCRT64 or MINGW64 env, +configure will run into an error because the cpu detected is not correct. + +Meson detects x86_64 processes emulated, so you need to manually set the cpu, +and force a cross compilation (with empty prefix). + +:: + + ./configure --cpu=x86_64 --cross-prefix= + diff --git a/docs/devel/index-build.rst b/docs/devel/index-build.rst index 0023953be36..0745c81a264 100644 --- a/docs/devel/index-build.rst +++ b/docs/devel/index-build.rst @@ -8,6 +8,7 @@ some of the basics if you are adding new files and targets to the build. :maxdepth: 3 build-system + build-environment kconfig docs qapi-code-gen
MacOS and Linux are straightforward, but Windows needs a bit more details. Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org> --- docs/about/build-platforms.rst | 4 +- docs/devel/build-environment.rst | 114 +++++++++++++++++++++++++++++++ docs/devel/index-build.rst | 1 + 3 files changed, 118 insertions(+), 1 deletion(-) create mode 100644 docs/devel/build-environment.rst