diff mbox

[trivial] qapi: Build-depend on all json files

Message ID 20160204154425.GY30301@redhat.com (mailing list archive)
State New, archived
Headers show

Commit Message

Daniel P. Berrangé Feb. 4, 2016, 3:44 p.m. UTC 
On Thu, Feb 04, 2016 at 03:36:56PM +0000, Peter Maydell wrote:
> On 4 February 2016 at 15:32, Eric Blake <eblake@redhat.com> wrote:
> > On 02/04/2016 07:55 AM, Peter Maydell wrote:
> >> On 4 February 2016 at 14:39, Lluís Vilanova <vilanova@ac.upc.edu> wrote:
> >>> Dynamically detects the files used to generate QAPI code, thus ensuring
> >>> it's never out of sync with the sources.
> >>>
> >>> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
> >>> ---
> >>>  Makefile |    6 ++----
> >>>  1 file changed, 2 insertions(+), 4 deletions(-)
> >>>
> >>> diff --git a/Makefile b/Makefile
> >>> index d0de2d4..627f772 100644
> >>> --- a/Makefile
> >>> +++ b/Makefile
> >>> @@ -269,10 +269,8 @@ $(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
> >>>                 $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \
> >>>                 "  GEN   $@")
> >>>
> >>> -qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
> >>> -               $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
> >>> -               $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
> >>> -               $(SRC_PATH)/qapi/crypto.json
> >>> +qapi-modules = $(SRC_PATH)/qapi-schema.json
> >>> +qapi-modules += $(shell find $(SRC_PATH)/qapi -name "*.json")
> >>
> >> All the .json files are in the same directory, so I don't think we should
> >> need to use find here. Does
> >>
> >> qapi-modules += $(wildcard $(SRC_PATH)/qapi/*.json))
> >>
> >> work ?
> >
> > Does this wildcard affect what goes into a tarball? I'm worried that we
> > may run the risk of a stale .json file on one developer's machine
> > causing an unreproducible build on other machines where the file is not
> > found; explicit lists tend to be safer than wildcards.
> >
> > I won't reject the patch if others like it, but I won't approve it myself.
> 
> You need to ask Mike Roth about our tarball generation process, not me.
> 
> I do agree that this patch needs to make the case for why .json source
> files are special and should be wildcarded, when for instance all our
> C source files are explicitly listed in makefiles.

Yes, normal practice is that the files are listed in a Makefile that lives
in the same dir as the file. The qapi json files are not following that
since they live in a dir above. I'd be inclined to say any patch should
take us in line with source files, and thus look more like this:





NB, this is completely untested - i just hacked the makefile for purpose
of illustration.

Regards,
Daniel

Comments

Lluís Vilanova Feb. 4, 2016, 4:34 p.m. UTC | #1 
Daniel P Berrange writes:

> On Thu, Feb 04, 2016 at 03:36:56PM +0000, Peter Maydell wrote:
>> On 4 February 2016 at 15:32, Eric Blake <eblake@redhat.com> wrote:
>> > On 02/04/2016 07:55 AM, Peter Maydell wrote:
>> >> On 4 February 2016 at 14:39, Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>> >>> Dynamically detects the files used to generate QAPI code, thus ensuring
>> >>> it's never out of sync with the sources.
>> >>>
>> >>> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
>> >>> ---
>> >>>  Makefile |    6 ++----
>> >>>  1 file changed, 2 insertions(+), 4 deletions(-)
>> >>>
>> >>> diff --git a/Makefile b/Makefile
>> >>> index d0de2d4..627f772 100644
>> >>> --- a/Makefile
>> >>> +++ b/Makefile
>> >>> @@ -269,10 +269,8 @@ $(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
>> >>>                 $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \
>> >>>                 "  GEN   $@")
>> >>>
>> >>> -qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
>> >>> -               $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
>> >>> -               $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
>> >>> -               $(SRC_PATH)/qapi/crypto.json
>> >>> +qapi-modules = $(SRC_PATH)/qapi-schema.json
>> >>> +qapi-modules += $(shell find $(SRC_PATH)/qapi -name "*.json")
>> >>
>> >> All the .json files are in the same directory, so I don't think we should
>> >> need to use find here. Does
>> >>
>> >> qapi-modules += $(wildcard $(SRC_PATH)/qapi/*.json))
>> >>
>> >> work ?
>> >
>> > Does this wildcard affect what goes into a tarball? I'm worried that we
>> > may run the risk of a stale .json file on one developer's machine
>> > causing an unreproducible build on other machines where the file is not
>> > found; explicit lists tend to be safer than wildcards.
>> >
>> > I won't reject the patch if others like it, but I won't approve it myself.
>> 
>> You need to ask Mike Roth about our tarball generation process, not me.
>> 
>> I do agree that this patch needs to make the case for why .json source
>> files are special and should be wildcarded, when for instance all our
>> C source files are explicitly listed in makefiles.

> Yes, normal practice is that the files are listed in a Makefile that lives
> in the same dir as the file. The qapi json files are not following that
> since they live in a dir above. I'd be inclined to say any patch should
> take us in line with source files, and thus look more like this:


> diff --git a/Makefile b/Makefile
> index b7b0f24..5481b57 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -161,7 +161,8 @@ dummy := $(call unnest-vars,, \
>                  qom-obj-y \
>                  io-obj-y \
>                  common-obj-y \
> -                common-obj-m)
> +                common-obj-m \
> +                qapi-modules)
 
>  ifneq ($(wildcard config-host.mak),)
>  include $(SRC_PATH)/tests/Makefile
> @@ -269,10 +270,7 @@ $(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
>                 $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \
>                 "  GEN   $@")
 
> -qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
> -               $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
> -               $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
> -               $(SRC_PATH)/qapi/crypto.json
> +qapi-modules += $(SRC_PATH)/qapi-schema.json
 
>  qapi-types.c qapi-types.h :\
>  $(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
> diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
> index 2278970..a644a69 100644
> --- a/qapi/Makefile.objs
> +++ b/qapi/Makefile.objs
> @@ -4,3 +4,10 @@ util-obj-y += string-input-visitor.o string-output-visitor.o
>  util-obj-y += opts-visitor.o
>  util-obj-y += qmp-event.o
>  util-obj-y += qapi-util.o
> +
> +qapi-modules += common.json
> +qapi-modules += block.json
> +qapi-modules += block-core.json
> +qapi-modules += event.json
> +qapi-modules += introspect.json
> +qapi-modules += crypto.json

Moving it into the qapi directory is definitely an improvement (regardless of
whether it uses an explicit list or find/wildcard).

I guess you're missing a:

  qapi-modules += qapi/

somewhere on the top-level makefiles.

Cheers,
  Lluis
Lluís Vilanova Feb. 4, 2016, 6:23 p.m. UTC | #2 
Lluís Vilanova writes:

> Daniel P Berrange writes:
>> On Thu, Feb 04, 2016 at 03:36:56PM +0000, Peter Maydell wrote:
>>> On 4 February 2016 at 15:32, Eric Blake <eblake@redhat.com> wrote:
>>> > On 02/04/2016 07:55 AM, Peter Maydell wrote:
>>> >> On 4 February 2016 at 14:39, Lluís Vilanova <vilanova@ac.upc.edu> wrote:
>>> >>> Dynamically detects the files used to generate QAPI code, thus ensuring
>>> >>> it's never out of sync with the sources.
>>> >>>
>>> >>> Signed-off-by: Lluís Vilanova <vilanova@ac.upc.edu>
>>> >>> ---
>>> >>>  Makefile |    6 ++----
>>> >>>  1 file changed, 2 insertions(+), 4 deletions(-)
>>> >>>
>>> >>> diff --git a/Makefile b/Makefile
>>> >>> index d0de2d4..627f772 100644
>>> >>> --- a/Makefile
>>> >>> +++ b/Makefile
>>> >>> @@ -269,10 +269,8 @@ $(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
>>> >>>                 $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \
>>> >>>                 "  GEN   $@")
>>> >>>
>>> >>> -qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
>>> >>> -               $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
>>> >>> -               $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
>>> >>> -               $(SRC_PATH)/qapi/crypto.json
>>> >>> +qapi-modules = $(SRC_PATH)/qapi-schema.json
>>> >>> +qapi-modules += $(shell find $(SRC_PATH)/qapi -name "*.json")
>>> >>
>>> >> All the .json files are in the same directory, so I don't think we should
>>> >> need to use find here. Does
>>> >>
>>> >> qapi-modules += $(wildcard $(SRC_PATH)/qapi/*.json))
>>> >>
>>> >> work ?
>>> >
>>> > Does this wildcard affect what goes into a tarball? I'm worried that we
>>> > may run the risk of a stale .json file on one developer's machine
>>> > causing an unreproducible build on other machines where the file is not
>>> > found; explicit lists tend to be safer than wildcards.
>>> >
>>> > I won't reject the patch if others like it, but I won't approve it myself.
>>> 
>>> You need to ask Mike Roth about our tarball generation process, not me.
>>> 
>>> I do agree that this patch needs to make the case for why .json source
>>> files are special and should be wildcarded, when for instance all our
>>> C source files are explicitly listed in makefiles.

>> Yes, normal practice is that the files are listed in a Makefile that lives
>> in the same dir as the file. The qapi json files are not following that
>> since they live in a dir above. I'd be inclined to say any patch should
>> take us in line with source files, and thus look more like this:


>> diff --git a/Makefile b/Makefile
>> index b7b0f24..5481b57 100644
>> --- a/Makefile
>> +++ b/Makefile
>> @@ -161,7 +161,8 @@ dummy := $(call unnest-vars,, \
>> qom-obj-y \
>> io-obj-y \
>> common-obj-y \
>> -                common-obj-m)
>> +                common-obj-m \
>> +                qapi-modules)
 
>> ifneq ($(wildcard config-host.mak),)
>> include $(SRC_PATH)/tests/Makefile
>> @@ -269,10 +270,7 @@ $(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
>> $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \
>> "  GEN   $@")
 
>> -qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
>> -               $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
>> -               $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
>> -               $(SRC_PATH)/qapi/crypto.json
>> +qapi-modules += $(SRC_PATH)/qapi-schema.json
 
>> qapi-types.c qapi-types.h :\
>> $(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
>> diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
>> index 2278970..a644a69 100644
>> --- a/qapi/Makefile.objs
>> +++ b/qapi/Makefile.objs
>> @@ -4,3 +4,10 @@ util-obj-y += string-input-visitor.o string-output-visitor.o
>> util-obj-y += opts-visitor.o
>> util-obj-y += qmp-event.o
>> util-obj-y += qapi-util.o
>> +
>> +qapi-modules += common.json
>> +qapi-modules += block.json
>> +qapi-modules += block-core.json
>> +qapi-modules += event.json
>> +qapi-modules += introspect.json
>> +qapi-modules += crypto.json

> Moving it into the qapi directory is definitely an improvement (regardless of
> whether it uses an explicit list or find/wildcard).

For the record: I've played with this for a bit and it does not work. In your
example, it cannot find the source files (for some reason cannot find them in
vpath although I've changed VPATH_SUFFIXES in rules.mak to account for %.json),
and using an absolute path seems to break the makefile macro that recurses into
directories (unnest-vars).


Cheers,
  Lluis
diff mbox

Patch

diff --git a/Makefile b/Makefile
index b7b0f24..5481b57 100644
--- a/Makefile
+++ b/Makefile
@@ -161,7 +161,8 @@  dummy := $(call unnest-vars,, \
                 qom-obj-y \
                 io-obj-y \
                 common-obj-y \
-                common-obj-m)
+                common-obj-m \
+                qapi-modules)
 
 ifneq ($(wildcard config-host.mak),)
 include $(SRC_PATH)/tests/Makefile
@@ -269,10 +270,7 @@  $(SRC_PATH)/qga/qapi-schema.json $(SRC_PATH)/scripts/qapi-commands.py $(qapi-py)
                $(gen-out-type) -o qga/qapi-generated -p "qga-" $<, \
                "  GEN   $@")
 
-qapi-modules = $(SRC_PATH)/qapi-schema.json $(SRC_PATH)/qapi/common.json \
-               $(SRC_PATH)/qapi/block.json $(SRC_PATH)/qapi/block-core.json \
-               $(SRC_PATH)/qapi/event.json $(SRC_PATH)/qapi/introspect.json \
-               $(SRC_PATH)/qapi/crypto.json
+qapi-modules += $(SRC_PATH)/qapi-schema.json
 
 qapi-types.c qapi-types.h :\
 $(qapi-modules) $(SRC_PATH)/scripts/qapi-types.py $(qapi-py)
diff --git a/qapi/Makefile.objs b/qapi/Makefile.objs
index 2278970..a644a69 100644
--- a/qapi/Makefile.objs
+++ b/qapi/Makefile.objs
@@ -4,3 +4,10 @@  util-obj-y += string-input-visitor.o string-output-visitor.o
 util-obj-y += opts-visitor.o
 util-obj-y += qmp-event.o
 util-obj-y += qapi-util.o
+
+qapi-modules += common.json
+qapi-modules += block.json
+qapi-modules += block-core.json
+qapi-modules += event.json
+qapi-modules += introspect.json
+qapi-modules += crypto.json