diff mbox series

[RFC,server,v2,01/11] vfio-user: build library

Message ID b556fb2bb8e364657a867dc5a83df937ffbf5880.1630084211.git.jag.raman@oracle.com (mailing list archive)
State New, archived
Headers show
Series vfio-user server in QEMU | expand

Commit Message

Jag Raman Aug. 27, 2021, 5:53 p.m. UTC 
add the libvfio-user library as a submodule. build it as a cmake
subproject.

Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
---
 configure                | 11 +++++++++++
 meson.build              | 28 ++++++++++++++++++++++++++++
 .gitmodules              |  3 +++
 MAINTAINERS              |  7 +++++++
 hw/remote/meson.build    |  2 ++
 subprojects/libvfio-user |  1 +
 6 files changed, 52 insertions(+)
 create mode 160000 subprojects/libvfio-user

Comments

Jag Raman Aug. 27, 2021, 6:05 p.m. UTC | #1 
> On Aug 27, 2021, at 1:53 PM, Jag Raman <jag.raman@oracle.com> wrote:
> 
> add the libvfio-user library as a submodule. build it as a cmake
> subproject.
> 
> Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
> Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
> ---
> configure                | 11 +++++++++++
> meson.build              | 28 ++++++++++++++++++++++++++++
> .gitmodules              |  3 +++
> MAINTAINERS              |  7 +++++++
> hw/remote/meson.build    |  2 ++
> subprojects/libvfio-user |  1 +
> 6 files changed, 52 insertions(+)
> create mode 160000 subprojects/libvfio-user
> 
> diff --git a/configure b/configure
> index 9a79a00..794e900 100755
> --- a/configure
> +++ b/configure
> @@ -4291,6 +4291,17 @@ but not implemented on your system"
> fi
> 
> ##########################################
> +# check for multiprocess
> +
> +case "$multiprocess" in
> +  auto | enabled )
> +    if test "$git_submodules_action" != "ignore"; then
> +      git_submodules="${git_submodules} libvfio-user"
> +    fi
> +    ;;
> +esac
> +
> +##########################################
> # End of CC checks
> # After here, no more $cc or $ld runs
> 
> diff --git a/meson.build b/meson.build
> index bf63784..2b2d5c2 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1898,6 +1898,34 @@ if get_option('cfi') and slirp_opt == 'system'
>          + ' Please configure with --enable-slirp=git')
> endif
> 
> +vfiouser = not_found
> +if have_system and multiprocess_allowed
> +  have_internal = fs.exists(meson.current_source_dir() / 'subprojects/libvfio-user/Makefile')
> +
> +  if not have_internal
> +    error('libvfio-user source not found - please pull git submodule')
> +  endif
> +
> +  json_c = dependency('json-c', required: false)
> +    if not json_c.found()
> +      json_c = dependency('libjson-c')
> +  endif

One of the things we’re wondering is about this json-c package that we need to build
libvfio-user library.

The gitlab runners typically don’t have this package installed, as such the gitlab builds
fail. Wondering if there's a way to install this package for all QEMU builds?

We checked out the various jobs defined in “.gitlab-ci.d/buildtest.yml” - there is a
“before_script” keyword which we could use to install this package. The “before_script”
keyword appears to be run every time before a job’s script is executed. But this option
appears to be per job/build. Wondering if there's a distro-independent global way to
install a required package for all builds.

Thank you!
--
Jag

> +
> +  cmake = import('cmake')
> +
> +  vfiouser_subproj = cmake.subproject('libvfio-user')
> +
> +  vfiouser_sl = vfiouser_subproj.dependency('vfio-user-static')
> +
> +  # Although cmake links the json-c library with vfio-user-static
> +  # target, that info is not available to meson via cmake.subproject.
> +  # As such, we have to separately declare the json-c dependency here.
> +  # This appears to be a current limitation of using cmake inside meson.
> +  # libvfio-user is planning a switch to meson in the future, which
> +  # would address this item automatically.
> +  vfiouser = declare_dependency(dependencies: [vfiouser_sl, json_c])
> +endif
> +
> fdt = not_found
> fdt_opt = get_option('fdt')
> if have_system
> diff --git a/.gitmodules b/.gitmodules
> index 08b1b48..cfeea7c 100644
> --- a/.gitmodules
> +++ b/.gitmodules
> @@ -64,3 +64,6 @@
> [submodule "roms/vbootrom"]
> 	path = roms/vbootrom
> 	url = https://gitlab.com/qemu-project/vbootrom.git
> +[submodule "subprojects/libvfio-user"]
> +	path = subprojects/libvfio-user
> +	url = https://github.com/nutanix/libvfio-user.git
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 4039d3c..0c5a18e 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3361,6 +3361,13 @@ F: semihosting/
> F: include/semihosting/
> F: tests/tcg/multiarch/arm-compat-semi/
> 
> +libvfio-user Library
> +M: Thanos Makatos <thanos.makatos@nutanix.com>
> +M: John Levon <john.levon@nutanix.com>
> +T: https://github.com/nutanix/libvfio-user.git
> +S: Maintained
> +F: subprojects/libvfio-user/*
> +
> Multi-process QEMU
> M: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> M: Jagannathan Raman <jag.raman@oracle.com>
> diff --git a/hw/remote/meson.build b/hw/remote/meson.build
> index e6a5574..fb35fb8 100644
> --- a/hw/remote/meson.build
> +++ b/hw/remote/meson.build
> @@ -7,6 +7,8 @@ remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('remote-obj.c'))
> remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy.c'))
> remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('iohub.c'))
> 
> +remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: vfiouser)
> +
> specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('memory.c'))
> specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy-memory-listener.c'))
> 
> diff --git a/subprojects/libvfio-user b/subprojects/libvfio-user
> new file mode 160000
> index 0000000..647c934
> --- /dev/null
> +++ b/subprojects/libvfio-user
> @@ -0,0 +1 @@
> +Subproject commit 647c9341d2e06266a710ddd075f69c95dd3b8446
> -- 
> 1.8.3.1
>
Stefan Hajnoczi Sept. 8, 2021, 12:25 p.m. UTC | #2 
On Fri, Aug 27, 2021 at 01:53:20PM -0400, Jagannathan Raman wrote:
> diff --git a/meson.build b/meson.build
> index bf63784..2b2d5c2 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -1898,6 +1898,34 @@ if get_option('cfi') and slirp_opt == 'system'
>           + ' Please configure with --enable-slirp=git')
>  endif
>  
> +vfiouser = not_found
> +if have_system and multiprocess_allowed
> +  have_internal = fs.exists(meson.current_source_dir() / 'subprojects/libvfio-user/Makefile')
> +
> +  if not have_internal
> +    error('libvfio-user source not found - please pull git submodule')
> +  endif
> +
> +  json_c = dependency('json-c', required: false)
> +    if not json_c.found()

Indentation is off.

> +      json_c = dependency('libjson-c')
> +  endif
> +
> +  cmake = import('cmake')
> +
> +  vfiouser_subproj = cmake.subproject('libvfio-user')
> +
> +  vfiouser_sl = vfiouser_subproj.dependency('vfio-user-static')
> +
> +  # Although cmake links the json-c library with vfio-user-static
> +  # target, that info is not available to meson via cmake.subproject.
> +  # As such, we have to separately declare the json-c dependency here.
> +  # This appears to be a current limitation of using cmake inside meson.
> +  # libvfio-user is planning a switch to meson in the future, which
> +  # would address this item automatically.
> +  vfiouser = declare_dependency(dependencies: [vfiouser_sl, json_c])
> +endif
> +
>  fdt = not_found
>  fdt_opt = get_option('fdt')
>  if have_system
> diff --git a/.gitmodules b/.gitmodules
> index 08b1b48..cfeea7c 100644
> --- a/.gitmodules
> +++ b/.gitmodules
> @@ -64,3 +64,6 @@
>  [submodule "roms/vbootrom"]
>  	path = roms/vbootrom
>  	url = https://gitlab.com/qemu-project/vbootrom.git
> +[submodule "subprojects/libvfio-user"]
> +	path = subprojects/libvfio-user
> +	url = https://github.com/nutanix/libvfio-user.git

Once this is merged I'll set up a
gitlab.com/qemu-project/libvfio-user.git mirror. This ensures that no
matter what happens with upstream libvfio-user.git, the source code that
QEMU builds against will remain archived/available.

> diff --git a/MAINTAINERS b/MAINTAINERS
> index 4039d3c..0c5a18e 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -3361,6 +3361,13 @@ F: semihosting/
>  F: include/semihosting/
>  F: tests/tcg/multiarch/arm-compat-semi/
>  
> +libvfio-user Library
> +M: Thanos Makatos <thanos.makatos@nutanix.com>
> +M: John Levon <john.levon@nutanix.com>
> +T: https://github.com/nutanix/libvfio-user.git
> +S: Maintained
> +F: subprojects/libvfio-user/*

A MAINTAINERS entry isn't necessary for git submodules. This could
become outdated. People should look at the upstream project instead for
information on maintainership and how to contribute.
Philippe Mathieu-Daudé Sept. 10, 2021, 3:20 p.m. UTC | #3 
On 8/27/21 7:53 PM, Jagannathan Raman wrote:
> add the libvfio-user library as a submodule. build it as a cmake
> subproject.
> 
> Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
> Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
> ---
>  configure                | 11 +++++++++++
>  meson.build              | 28 ++++++++++++++++++++++++++++
>  .gitmodules              |  3 +++
>  MAINTAINERS              |  7 +++++++
>  hw/remote/meson.build    |  2 ++
>  subprojects/libvfio-user |  1 +
>  6 files changed, 52 insertions(+)
>  create mode 160000 subprojects/libvfio-user

> diff --git a/subprojects/libvfio-user b/subprojects/libvfio-user
> new file mode 160000
> index 0000000..647c934
> --- /dev/null
> +++ b/subprojects/libvfio-user
> @@ -0,0 +1 @@
> +Subproject commit 647c9341d2e06266a710ddd075f69c95dd3b8446
> 

Could we point to a sha1 of a released tag instead?
Philippe Mathieu-Daudé Sept. 10, 2021, 3:21 p.m. UTC | #4 
On 9/8/21 2:25 PM, Stefan Hajnoczi wrote:
> On Fri, Aug 27, 2021 at 01:53:20PM -0400, Jagannathan Raman wrote:

>> diff --git a/.gitmodules b/.gitmodules
>> index 08b1b48..cfeea7c 100644
>> --- a/.gitmodules
>> +++ b/.gitmodules
>> @@ -64,3 +64,6 @@
>>  [submodule "roms/vbootrom"]
>>  	path = roms/vbootrom
>>  	url = https://gitlab.com/qemu-project/vbootrom.git
>> +[submodule "subprojects/libvfio-user"]
>> +	path = subprojects/libvfio-user
>> +	url = https://github.com/nutanix/libvfio-user.git
> 
> Once this is merged I'll set up a
> gitlab.com/qemu-project/libvfio-user.git mirror. This ensures that no
> matter what happens with upstream libvfio-user.git, the source code that
> QEMU builds against will remain archived/available.

Can we do it the other way around? When the series is OK to be merged,
setup the https://gitlab.com/qemu-project/libvfio-user.git mirror and
have the submodule point to it?
Jag Raman Sept. 10, 2021, 5:08 p.m. UTC | #5 
> On Sep 10, 2021, at 11:20 AM, Philippe Mathieu-Daudé <philmd@redhat.com> wrote:
> 
> On 8/27/21 7:53 PM, Jagannathan Raman wrote:
>> add the libvfio-user library as a submodule. build it as a cmake
>> subproject.
>> 
>> Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
>> Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
>> Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
>> ---
>> configure                | 11 +++++++++++
>> meson.build              | 28 ++++++++++++++++++++++++++++
>> .gitmodules              |  3 +++
>> MAINTAINERS              |  7 +++++++
>> hw/remote/meson.build    |  2 ++
>> subprojects/libvfio-user |  1 +
>> 6 files changed, 52 insertions(+)
>> create mode 160000 subprojects/libvfio-user
> 
>> diff --git a/subprojects/libvfio-user b/subprojects/libvfio-user
>> new file mode 160000
>> index 0000000..647c934
>> --- /dev/null
>> +++ b/subprojects/libvfio-user
>> @@ -0,0 +1 @@
>> +Subproject commit 647c9341d2e06266a710ddd075f69c95dd3b8446
>> 
> 
> Could we point to a sha1 of a released tag instead?

OK, will do.

--
Jag

>
John Levon Sept. 11, 2021, 10:29 p.m. UTC | #6 
On Fri, Sep 10, 2021 at 05:20:09PM +0200, Philippe Mathieu-Daudé wrote:

> On 8/27/21 7:53 PM, Jagannathan Raman wrote:
> > add the libvfio-user library as a submodule. build it as a cmake
> > subproject.
> > 
> > Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
> > Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
> > Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
> > ---
> >  configure                | 11 +++++++++++
> >  meson.build              | 28 ++++++++++++++++++++++++++++
> >  .gitmodules              |  3 +++
> >  MAINTAINERS              |  7 +++++++
> >  hw/remote/meson.build    |  2 ++
> >  subprojects/libvfio-user |  1 +
> >  6 files changed, 52 insertions(+)
> >  create mode 160000 subprojects/libvfio-user
> 
> > diff --git a/subprojects/libvfio-user b/subprojects/libvfio-user
> > new file mode 160000
> > index 0000000..647c934
> > --- /dev/null
> > +++ b/subprojects/libvfio-user
> > @@ -0,0 +1 @@
> > +Subproject commit 647c9341d2e06266a710ddd075f69c95dd3b8446
> 
> Could we point to a sha1 of a released tag instead?

We don't have releases (yet) partly because we haven't yet stabilized the API.

regards
john
Philippe Mathieu-Daudé Sept. 13, 2021, 10:19 a.m. UTC | #7 
On 9/12/21 12:29 AM, John Levon wrote:
> On Fri, Sep 10, 2021 at 05:20:09PM +0200, Philippe Mathieu-Daudé wrote:
>> On 8/27/21 7:53 PM, Jagannathan Raman wrote:
>>> add the libvfio-user library as a submodule. build it as a cmake
>>> subproject.
>>>
>>> Signed-off-by: Elena Ufimtseva <elena.ufimtseva@oracle.com>
>>> Signed-off-by: John G Johnson <john.g.johnson@oracle.com>
>>> Signed-off-by: Jagannathan Raman <jag.raman@oracle.com>
>>> ---
>>>  configure                | 11 +++++++++++
>>>  meson.build              | 28 ++++++++++++++++++++++++++++
>>>  .gitmodules              |  3 +++
>>>  MAINTAINERS              |  7 +++++++
>>>  hw/remote/meson.build    |  2 ++
>>>  subprojects/libvfio-user |  1 +
>>>  6 files changed, 52 insertions(+)
>>>  create mode 160000 subprojects/libvfio-user
>>
>>> diff --git a/subprojects/libvfio-user b/subprojects/libvfio-user
>>> new file mode 160000
>>> index 0000000..647c934
>>> --- /dev/null
>>> +++ b/subprojects/libvfio-user
>>> @@ -0,0 +1 @@
>>> +Subproject commit 647c9341d2e06266a710ddd075f69c95dd3b8446
>>
>> Could we point to a sha1 of a released tag instead?
> 
> We don't have releases (yet) partly because we haven't yet stabilized the API.

OK. Maybe acceptable, up to the maintainer then ¯\_(ツ)_/¯
Stefan Hajnoczi Sept. 13, 2021, 12:15 p.m. UTC | #8 
On Fri, Sep 10, 2021 at 05:21:33PM +0200, Philippe Mathieu-Daudé wrote:
> On 9/8/21 2:25 PM, Stefan Hajnoczi wrote:
> > On Fri, Aug 27, 2021 at 01:53:20PM -0400, Jagannathan Raman wrote:
> 
> >> diff --git a/.gitmodules b/.gitmodules
> >> index 08b1b48..cfeea7c 100644
> >> --- a/.gitmodules
> >> +++ b/.gitmodules
> >> @@ -64,3 +64,6 @@
> >>  [submodule "roms/vbootrom"]
> >>  	path = roms/vbootrom
> >>  	url = https://gitlab.com/qemu-project/vbootrom.git
> >> +[submodule "subprojects/libvfio-user"]
> >> +	path = subprojects/libvfio-user
> >> +	url = https://github.com/nutanix/libvfio-user.git
> > 
> > Once this is merged I'll set up a
> > gitlab.com/qemu-project/libvfio-user.git mirror. This ensures that no
> > matter what happens with upstream libvfio-user.git, the source code that
> > QEMU builds against will remain archived/available.
> 
> Can we do it the other way around? When the series is OK to be merged,
> setup the https://gitlab.com/qemu-project/libvfio-user.git mirror and
> have the submodule point to it?

Yes, good idea.

Stefan
diff mbox series

Patch

diff --git a/configure b/configure
index 9a79a00..794e900 100755
--- a/configure
+++ b/configure
@@ -4291,6 +4291,17 @@  but not implemented on your system"
 fi
 
 ##########################################
+# check for multiprocess
+
+case "$multiprocess" in
+  auto | enabled )
+    if test "$git_submodules_action" != "ignore"; then
+      git_submodules="${git_submodules} libvfio-user"
+    fi
+    ;;
+esac
+
+##########################################
 # End of CC checks
 # After here, no more $cc or $ld runs
 
diff --git a/meson.build b/meson.build
index bf63784..2b2d5c2 100644
--- a/meson.build
+++ b/meson.build
@@ -1898,6 +1898,34 @@  if get_option('cfi') and slirp_opt == 'system'
          + ' Please configure with --enable-slirp=git')
 endif
 
+vfiouser = not_found
+if have_system and multiprocess_allowed
+  have_internal = fs.exists(meson.current_source_dir() / 'subprojects/libvfio-user/Makefile')
+
+  if not have_internal
+    error('libvfio-user source not found - please pull git submodule')
+  endif
+
+  json_c = dependency('json-c', required: false)
+    if not json_c.found()
+      json_c = dependency('libjson-c')
+  endif
+
+  cmake = import('cmake')
+
+  vfiouser_subproj = cmake.subproject('libvfio-user')
+
+  vfiouser_sl = vfiouser_subproj.dependency('vfio-user-static')
+
+  # Although cmake links the json-c library with vfio-user-static
+  # target, that info is not available to meson via cmake.subproject.
+  # As such, we have to separately declare the json-c dependency here.
+  # This appears to be a current limitation of using cmake inside meson.
+  # libvfio-user is planning a switch to meson in the future, which
+  # would address this item automatically.
+  vfiouser = declare_dependency(dependencies: [vfiouser_sl, json_c])
+endif
+
 fdt = not_found
 fdt_opt = get_option('fdt')
 if have_system
diff --git a/.gitmodules b/.gitmodules
index 08b1b48..cfeea7c 100644
--- a/.gitmodules
+++ b/.gitmodules
@@ -64,3 +64,6 @@ 
 [submodule "roms/vbootrom"]
 	path = roms/vbootrom
 	url = https://gitlab.com/qemu-project/vbootrom.git
+[submodule "subprojects/libvfio-user"]
+	path = subprojects/libvfio-user
+	url = https://github.com/nutanix/libvfio-user.git
diff --git a/MAINTAINERS b/MAINTAINERS
index 4039d3c..0c5a18e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -3361,6 +3361,13 @@  F: semihosting/
 F: include/semihosting/
 F: tests/tcg/multiarch/arm-compat-semi/
 
+libvfio-user Library
+M: Thanos Makatos <thanos.makatos@nutanix.com>
+M: John Levon <john.levon@nutanix.com>
+T: https://github.com/nutanix/libvfio-user.git
+S: Maintained
+F: subprojects/libvfio-user/*
+
 Multi-process QEMU
 M: Elena Ufimtseva <elena.ufimtseva@oracle.com>
 M: Jagannathan Raman <jag.raman@oracle.com>
diff --git a/hw/remote/meson.build b/hw/remote/meson.build
index e6a5574..fb35fb8 100644
--- a/hw/remote/meson.build
+++ b/hw/remote/meson.build
@@ -7,6 +7,8 @@  remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('remote-obj.c'))
 remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy.c'))
 remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('iohub.c'))
 
+remote_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: vfiouser)
+
 specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('memory.c'))
 specific_ss.add(when: 'CONFIG_MULTIPROCESS', if_true: files('proxy-memory-listener.c'))
 
diff --git a/subprojects/libvfio-user b/subprojects/libvfio-user
new file mode 160000
index 0000000..647c934
--- /dev/null
+++ b/subprojects/libvfio-user
@@ -0,0 +1 @@ 
+Subproject commit 647c9341d2e06266a710ddd075f69c95dd3b8446