diff mbox series

[v3,02/12] vfio-user: build library

Message ID 48b1ca2b5070f3655075e02966c40786028dbfd9.1633929457.git.jag.raman@oracle.com (mailing list archive)
State New, archived
Headers show
Series vfio-user server in QEMU | expand

Commit Message

Jag Raman Oct. 11, 2021, 5:31 a.m. UTC 
add the libvfio-user library as a submodule. build it as a cmake
subproject.

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
---
 configure                                  | 13 +++++++-
 meson.build                                | 39 ++++++++++++++++++++++
 .gitlab-ci.d/buildtest.yml                 |  2 ++
 .gitmodules                                |  3 ++
 MAINTAINERS                                |  1 +
 hw/remote/Kconfig                          |  5 +++
 hw/remote/meson.build                      |  2 ++
 subprojects/libvfio-user                   |  1 +
 tests/acceptance/multiprocess.py           |  2 ++
 tests/docker/dockerfiles/centos8.docker    |  2 ++
 tests/docker/dockerfiles/ubuntu2004.docker |  2 ++
 11 files changed, 71 insertions(+), 1 deletion(-)
 create mode 160000 subprojects/libvfio-user

Comments

Stefan Hajnoczi Oct. 27, 2021, 3:17 p.m. UTC | #1 
On Mon, Oct 11, 2021 at 01:31:07AM -0400, Jagannathan Raman wrote:
> diff --git a/hw/remote/Kconfig b/hw/remote/Kconfig
> index 08c16e235f..f9e512d44a 100644
> --- a/hw/remote/Kconfig
> +++ b/hw/remote/Kconfig
> @@ -1,4 +1,9 @@
> +config VFIO_USER_SERVER
> +    bool
> +    default n

Does VFIO_USER_SERVER depend on MULTIPROCESS?

> diff --git a/tests/acceptance/multiprocess.py b/tests/acceptance/multiprocess.py
> index 96627f022a..7383c6eb58 100644
> --- a/tests/acceptance/multiprocess.py
> +++ b/tests/acceptance/multiprocess.py
> @@ -67,6 +67,7 @@ def do_test(self, kernel_url, initrd_url, kernel_command_line,
>      def test_multiprocess_x86_64(self):
>          """
>          :avocado: tags=arch:x86_64
> +        :avocado: tags=distro:centos
>          """
>          kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                        '/linux/releases/31/Everything/x86_64/os/images'
> @@ -82,6 +83,7 @@ def test_multiprocess_x86_64(self):
>      def test_multiprocess_aarch64(self):
>          """
>          :avocado: tags=arch:aarch64
> +        :avocado: tags=distro:ubuntu
>          """
>          kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>                        '/linux/releases/31/Everything/aarch64/os/images'

Did you tag them with different distros in order to get coverage on both
CentOS and Ubuntu (even though that's orthogonal to x86_64 vs aarch64)?
Maybe a comment is necessary so it's clear why these tags are in place
because the test isn't actually limited to that distro.
Jag Raman Oct. 29, 2021, 2:17 p.m. UTC | #2 
Hi Stefan,

> On Oct 27, 2021, at 11:17 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> 
> On Mon, Oct 11, 2021 at 01:31:07AM -0400, Jagannathan Raman wrote:
>> diff --git a/hw/remote/Kconfig b/hw/remote/Kconfig
>> index 08c16e235f..f9e512d44a 100644
>> --- a/hw/remote/Kconfig
>> +++ b/hw/remote/Kconfig
>> @@ -1,4 +1,9 @@
>> +config VFIO_USER_SERVER
>> +    bool
>> +    default n
> 
> Does VFIO_USER_SERVER depend on MULTIPROCESS?

Yes, VFIO_USER_SERVER presently depends on MULTIPROCESS.

This is because it needs some object and functions implemented by multiprocess
such as TYPE_REMOTE_MACHINE and TYPE_REMOTE_PCIHOST.

> 
>> diff --git a/tests/acceptance/multiprocess.py b/tests/acceptance/multiprocess.py
>> index 96627f022a..7383c6eb58 100644
>> --- a/tests/acceptance/multiprocess.py
>> +++ b/tests/acceptance/multiprocess.py
>> @@ -67,6 +67,7 @@ def do_test(self, kernel_url, initrd_url, kernel_command_line,
>>     def test_multiprocess_x86_64(self):
>>         """
>>         :avocado: tags=arch:x86_64
>> +        :avocado: tags=distro:centos
>>         """
>>         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>>                       '/linux/releases/31/Everything/x86_64/os/images'
>> @@ -82,6 +83,7 @@ def test_multiprocess_x86_64(self):
>>     def test_multiprocess_aarch64(self):
>>         """
>>         :avocado: tags=arch:aarch64
>> +        :avocado: tags=distro:ubuntu
>>         """
>>         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
>>                       '/linux/releases/31/Everything/aarch64/os/images'
> 
> Did you tag them with different distros in order to get coverage on both
> CentOS and Ubuntu (even though that's orthogonal to x86_64 vs aarch64)?
> Maybe a comment is necessary so it's clear why these tags are in place
> because the test isn't actually limited to that distro.

OK, I’ll add a comment to explain this.

For background, we disabled multiprocess by default in this series. This is
because, not all docker images have the json-c package available to build
libvfio-user library. So we had to enable it on selected docker images where
that package would be available. As such, we also had to limit the multiprocess
acceptance tests to the distros which had multiprocess enabled.

Going by “.gitlab-ci.d/buildtest.yml” file, the distros that support x86_64 target
are centos (build-system-centos) and opensuse (build-system-opensuse). So
we picked centos for x86_64, as running this test on other builds could
cause a failure. Likewise, ubuntu supported aarch64.

--
Jag
Stefan Hajnoczi Nov. 1, 2021, 9:56 a.m. UTC | #3 
On Fri, Oct 29, 2021 at 02:17:43PM +0000, Jag Raman wrote:
> Hi Stefan,
> 
> > On Oct 27, 2021, at 11:17 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> > 
> > On Mon, Oct 11, 2021 at 01:31:07AM -0400, Jagannathan Raman wrote:
> >> diff --git a/hw/remote/Kconfig b/hw/remote/Kconfig
> >> index 08c16e235f..f9e512d44a 100644
> >> --- a/hw/remote/Kconfig
> >> +++ b/hw/remote/Kconfig
> >> @@ -1,4 +1,9 @@
> >> +config VFIO_USER_SERVER
> >> +    bool
> >> +    default n
> > 
> > Does VFIO_USER_SERVER depend on MULTIPROCESS?
> 
> Yes, VFIO_USER_SERVER presently depends on MULTIPROCESS.
> 
> This is because it needs some object and functions implemented by multiprocess
> such as TYPE_REMOTE_MACHINE and TYPE_REMOTE_PCIHOST.

I wonder if it's necessary to specify that dependency in the Kconfig
file? If someone disables MULTIPROCESS but enables VFIO_USER_SERVER then
there should probably be an error (or MULTIPROCESS becomes enabled again
automatically).

Stefan
diff mbox series

Patch

diff --git a/configure b/configure
index e804dfba2f..88fb44f15a 100755
--- a/configure
+++ b/configure
@@ -443,7 +443,7 @@  skip_meson=no
 gettext="auto"
 fuse="auto"
 fuse_lseek="auto"
-multiprocess="auto"
+multiprocess="disabled"
 slirp_smbd="$default_feature"
 
 malloc_trim="auto"
@@ -4284,6 +4284,17 @@  but not implemented on your system"
     fi
 fi
 
+##########################################
+# check for multiprocess
+
+case "$multiprocess" in
+  auto | enabled )
+    if test "$git_submodules_action" != "ignore"; then
+      git_submodules="${git_submodules} subprojects/libvfio-user"
+    fi
+    ;;
+esac
+
 ##########################################
 # End of CC checks
 # After here, no more $cc or $ld runs
diff --git a/meson.build b/meson.build
index 99a0a3e689..5c91305f2d 100644
--- a/meson.build
+++ b/meson.build
@@ -172,6 +172,10 @@  if targetos != 'linux' and get_option('multiprocess').enabled()
 endif
 multiprocess_allowed = targetos == 'linux' and not get_option('multiprocess').disabled()
 
+# libvfiouser is enabled with multiprocess. Presently, libvfiouser depends on
+# multiprocess code, as such it can't be enabled independently
+libvfiouser_allowed = multiprocess_allowed
+
 libm = cc.find_library('m', required: false)
 threads = dependency('threads')
 util = cc.find_library('util', required: false)
@@ -1903,6 +1907,41 @@  if get_option('cfi') and slirp_opt == 'system'
          + ' Please configure with --enable-slirp=git')
 endif
 
+vfiouser = not_found
+if have_system and libvfiouser_allowed
+  have_internal = fs.exists(meson.current_source_dir() / 'subprojects/libvfio-user/Makefile')
+
+  if not have_internal
+    error('libvfio-user source not found - please pull git submodule')
+  endif
+
+  json_c = dependency('json-c', required: false)
+  if not json_c.found()
+    json_c = dependency('libjson-c', required: false)
+  endif
+  if not json_c.found()
+    json_c = dependency('libjson-c-dev', required: false)
+  endif
+
+  if not json_c.found()
+    error('Unable to find json-c package')
+  endif
+
+  cmake = import('cmake')
+
+  vfiouser_subproj = cmake.subproject('libvfio-user')
+
+  vfiouser_sl = vfiouser_subproj.dependency('vfio-user-static')
+
+  # Although cmake links the json-c library with vfio-user-static
+  # target, that info is not available to meson via cmake.subproject.
+  # As such, we have to separately declare the json-c dependency here.
+  # This appears to be a current limitation of using cmake inside meson.
+  # libvfio-user is planning a switch to meson in the future, which
+  # would address this item automatically.
+  vfiouser = declare_dependency(dependencies: [vfiouser_sl, json_c])
+endif
+
 fdt = not_found
 fdt_opt = get_option('fdt')
 if have_system
diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml
index 5c378e35f9..515ae40d1f 100644
--- a/.gitlab-ci.d/buildtest.yml
+++ b/.gitlab-ci.d/buildtest.yml
@@ -42,6 +42,7 @@  build-system-ubuntu:
   variables:
     IMAGE: ubuntu2004
     CONFIGURE_ARGS: --enable-docs --enable-fdt=system --enable-slirp=system
+                    --enable-multiprocess
     TARGETS: aarch64-softmmu alpha-softmmu cris-softmmu hppa-softmmu
       microblazeel-softmmu mips64el-softmmu
     MAKE_CHECK_ARGS: check-build
@@ -142,6 +143,7 @@  build-system-centos:
     IMAGE: centos8
     CONFIGURE_ARGS: --disable-nettle --enable-gcrypt --enable-fdt=system
                     --enable-modules --enable-trace-backends=dtrace
+                    --enable-multiprocess
     TARGETS: ppc64-softmmu or1k-softmmu s390x-softmmu
       x86_64-softmmu rx-softmmu sh4-softmmu nios2-softmmu
     MAKE_CHECK_ARGS: check-build
diff --git a/.gitmodules b/.gitmodules
index 08b1b48a09..cfeea7cf20 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -64,3 +64,6 @@ 
 [submodule "roms/vbootrom"]
 	path = roms/vbootrom
 	url = https://gitlab.com/qemu-project/vbootrom.git
+[submodule "subprojects/libvfio-user"]
+	path = subprojects/libvfio-user
+	url = https://github.com/nutanix/libvfio-user.git
diff --git a/MAINTAINERS b/MAINTAINERS
index 50435b8d2f..661f91a160 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3420,6 +3420,7 @@  F: hw/remote/proxy-memory-listener.c
 F: include/hw/remote/proxy-memory-listener.h
 F: hw/remote/iohub.c
 F: include/hw/remote/iohub.h
+F: subprojects/libvfio-user
 
 EBPF:
 M: Jason Wang <jasowang@redhat.com>
diff --git a/hw/remote/Kconfig b/hw/remote/Kconfig
index 08c16e235f..f9e512d44a 100644
--- a/hw/remote/Kconfig
+++ b/hw/remote/Kconfig
@@ -1,4 +1,9 @@ 
+config VFIO_USER_SERVER
+    bool
+    default n
+
 config MULTIPROCESS
     bool
     depends on PCI && PCI_EXPRESS && KVM
     select REMOTE_PCIHOST
+    select VFIO_USER_SERVER
diff --git a/hw/remote/meson.build b/hw/remote/meson.build
index e6a5574242..dfea6b533b 100644
--- a/hw/remote/meson.build
+++ b/hw/remote/meson.build
@@ -7,6 +7,8 @@  remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('remote-obj.c'))
 remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy.c'))
 remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('iohub.c'))
 
+remote_ss.add(when: 'CONFIG_VFIO_USER_SERVER', if_true: vfiouser)
+
 specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('memory.c'))
 specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy-memory-listener.c'))
 
diff --git a/subprojects/libvfio-user b/subprojects/libvfio-user
new file mode 160000
index 0000000000..647c9341d2
--- /dev/null
+++ b/subprojects/libvfio-user
@@ -0,0 +1 @@ 
+Subproject commit 647c9341d2e06266a710ddd075f69c95dd3b8446
diff --git a/tests/acceptance/multiprocess.py b/tests/acceptance/multiprocess.py
index 96627f022a..7383c6eb58 100644
--- a/tests/acceptance/multiprocess.py
+++ b/tests/acceptance/multiprocess.py
@@ -67,6 +67,7 @@  def do_test(self, kernel_url, initrd_url, kernel_command_line,
     def test_multiprocess_x86_64(self):
         """
         :avocado: tags=arch:x86_64
+        :avocado: tags=distro:centos
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/31/Everything/x86_64/os/images'
@@ -82,6 +83,7 @@  def test_multiprocess_x86_64(self):
     def test_multiprocess_aarch64(self):
         """
         :avocado: tags=arch:aarch64
+        :avocado: tags=distro:ubuntu
         """
         kernel_url = ('https://archives.fedoraproject.org/pub/archive/fedora'
                       '/linux/releases/31/Everything/aarch64/os/images'
diff --git a/tests/docker/dockerfiles/centos8.docker b/tests/docker/dockerfiles/centos8.docker
index 46398c61ee..646abcda1f 100644
--- a/tests/docker/dockerfiles/centos8.docker
+++ b/tests/docker/dockerfiles/centos8.docker
@@ -12,6 +12,7 @@  ENV PACKAGES \
     capstone-devel \
     ccache \
     clang \
+    cmake \
     ctags \
     cyrus-sasl-devel \
     daxctl-devel \
@@ -32,6 +33,7 @@  ENV PACKAGES \
     gtk3-devel \
     hostname \
     jemalloc-devel \
+    json-c-devel \
     libaio-devel \
     libasan \
     libattr-devel \
diff --git a/tests/docker/dockerfiles/ubuntu2004.docker b/tests/docker/dockerfiles/ubuntu2004.docker
index 39de63d012..ca4dff0e6b 100644
--- a/tests/docker/dockerfiles/ubuntu2004.docker
+++ b/tests/docker/dockerfiles/ubuntu2004.docker
@@ -6,6 +6,7 @@  ENV PACKAGES \
     ca-certificates \
     ccache \
     clang \
+    cmake \
     dbus \
     debianutils \
     diffutils \
@@ -44,6 +45,7 @@  ENV PACKAGES \
     libiscsi-dev \
     libjemalloc-dev \
     libjpeg-turbo8-dev \
+    libjson-c-dev \
     liblttng-ust-dev \
     liblzo2-dev \
     libncursesw5-dev \