| Message ID | 64fc67bc2227d6cf92e079228c9f8d2d6404b001.1603725003.git.frederic.pierret@qubes-os.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | Improve reproducible builds | expand |
On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote: > --- a/xen/arch/x86/Makefile > +++ b/xen/arch/x86/Makefile > @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION) > EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION) > EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0 > EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0 > +EFI_LDFLAGS += --no-insert-timestamp Generally I prefer binaries to carry timestamps, when they are intended to do so (i.e. when they have a respective field). So I think if no timestamp is wanted, it should be as an option (not sure about the default). This said, I didn't think time stamps got meaningfully in the way of reproducible builds - ignoring the minor differences cause by them, especially when they sit at well known offsets in the binaries, shouldn't be a big deal. Jan
On Fri, Oct 30, 2020 at 01:08:44PM +0100, Jan Beulich wrote: > On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote: > > > --- a/xen/arch/x86/Makefile > > +++ b/xen/arch/x86/Makefile > > @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION) > > EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION) > > EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0 > > EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0 > > +EFI_LDFLAGS += --no-insert-timestamp > > Generally I prefer binaries to carry timestamps, when they are > intended to do so (i.e. when they have a respective field). So > I think if no timestamp is wanted, it should be as an option > (not sure about the default). What about setting it to the SOURCE_DATE_EPOCH[1] variable value, if present? Of course if there is an option to set explicit timestamp value. [1] https://reproducible-builds.org/docs/source-date-epoch/ > This said, I didn't think time stamps got meaningfully in the > way of reproducible builds - ignoring the minor differences > cause by them, especially when they sit at well known offsets > in the binaries, shouldn't be a big deal. It is a big deal. There is a huge difference between running sha256sum (or your other favorite hash) on two build artifacts, and using a specialized tool/script to compare each file separately. Note the xen.efi may be buried very deep in the thing you compare, for example inside deb/rpm and then ISO image (installation image), at which point it's far from "they sit at well known offsets".
On 30.10.2020 13:23, Marek Marczykowski-Górecki wrote: > On Fri, Oct 30, 2020 at 01:08:44PM +0100, Jan Beulich wrote: >> On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote: >> >>> --- a/xen/arch/x86/Makefile >>> +++ b/xen/arch/x86/Makefile >>> @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION) >>> EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION) >>> EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0 >>> EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0 >>> +EFI_LDFLAGS += --no-insert-timestamp >> >> Generally I prefer binaries to carry timestamps, when they are >> intended to do so (i.e. when they have a respective field). So >> I think if no timestamp is wanted, it should be as an option >> (not sure about the default). > > What about setting it to the SOURCE_DATE_EPOCH[1] variable value, if > present? Of course if there is an option to set explicit timestamp > value. > > [1] https://reproducible-builds.org/docs/source-date-epoch/ Why not. >> This said, I didn't think time stamps got meaningfully in the >> way of reproducible builds - ignoring the minor differences >> cause by them, especially when they sit at well known offsets >> in the binaries, shouldn't be a big deal. > > It is a big deal. There is a huge difference between running sha256sum > (or your other favorite hash) on two build artifacts, and using a > specialized tool/script to compare each file separately. Note the > xen.efi may be buried very deep in the thing you compare, for example > inside deb/rpm and then ISO image (installation image), at which point > it's far from "they sit at well known offsets". If you care about checking images / blobs where binaries with time stamps are merely constituent parts, why not strip the time stamps at the time of creation of those images / blobs (or as a minor intermediate step, in case you want to e.g. record the hashes for later reference)? Jan
On 30/10/2020 12:48, Jan Beulich wrote: > On 30.10.2020 13:23, Marek Marczykowski-Górecki wrote: >> On Fri, Oct 30, 2020 at 01:08:44PM +0100, Jan Beulich wrote: >>> On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote: >>> >>>> --- a/xen/arch/x86/Makefile >>>> +++ b/xen/arch/x86/Makefile >>>> @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION) >>>> EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION) >>>> EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0 >>>> EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0 >>>> +EFI_LDFLAGS += --no-insert-timestamp >>> Generally I prefer binaries to carry timestamps, when they are >>> intended to do so (i.e. when they have a respective field). So >>> I think if no timestamp is wanted, it should be as an option >>> (not sure about the default). >> What about setting it to the SOURCE_DATE_EPOCH[1] variable value, if >> present? Of course if there is an option to set explicit timestamp >> value. >> >> [1] https://reproducible-builds.org/docs/source-date-epoch/ > Why not. SOURCE_DATE_EPOCH is the right way to fix this. It probably wants to default to something sane in the root Makefile, so it covers tools as well. >>> This said, I didn't think time stamps got meaningfully in the >>> way of reproducible builds - ignoring the minor differences >>> cause by them, especially when they sit at well known offsets >>> in the binaries, shouldn't be a big deal. >> It is a big deal. There is a huge difference between running sha256sum >> (or your other favorite hash) on two build artifacts, and using a >> specialized tool/script to compare each file separately. Note the >> xen.efi may be buried very deep in the thing you compare, for example >> inside deb/rpm and then ISO image (installation image), at which point >> it's far from "they sit at well known offsets". > If you care about checking images / blobs where binaries with time > stamps are merely constituent parts, why not strip the time stamps > at the time of creation of those images / blobs (or as a minor > intermediate step, in case you want to e.g. record the hashes for > later reference)? Because that is a disaster to maintain. A critical part of reproducible builds is not needing custom comparison logic for every binary artefact. ~Andrew
On Fri, Oct 30, 2020 at 01:30:08PM +0000, Andrew Cooper wrote: > On 30/10/2020 12:48, Jan Beulich wrote: > > On 30.10.2020 13:23, Marek Marczykowski-Górecki wrote: > >> On Fri, Oct 30, 2020 at 01:08:44PM +0100, Jan Beulich wrote: > >>> On 30.10.2020 13:03, Frédéric Pierret (fepitre) wrote: > >>> > >>>> --- a/xen/arch/x86/Makefile > >>>> +++ b/xen/arch/x86/Makefile > >>>> @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION) > >>>> EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION) > >>>> EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0 > >>>> EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0 > >>>> +EFI_LDFLAGS += --no-insert-timestamp > >>> Generally I prefer binaries to carry timestamps, when they are > >>> intended to do so (i.e. when they have a respective field). So > >>> I think if no timestamp is wanted, it should be as an option > >>> (not sure about the default). > >> What about setting it to the SOURCE_DATE_EPOCH[1] variable value, if > >> present? Of course if there is an option to set explicit timestamp > >> value. > >> > >> [1] https://reproducible-builds.org/docs/source-date-epoch/ > > Why not. > > SOURCE_DATE_EPOCH is the right way to fix this. Hmm, reading 'ld' man page, I don't see an option to set explicit value, on --insert-timestamp / --no-insert-timestamp. > It probably wants to default to something sane in the root Makefile, so > it covers tools as well. In practice, the package build system (deb, rpm, etc) do set it based on last package changelog entry, so _for package build case_ it isn't needed. But probably nice addition.
diff --git a/xen/arch/x86/Makefile b/xen/arch/x86/Makefile index b388861679..f5a529afd5 100644 --- a/xen/arch/x86/Makefile +++ b/xen/arch/x86/Makefile @@ -170,6 +170,7 @@ EFI_LDFLAGS += --major-image-version=$(XEN_VERSION) EFI_LDFLAGS += --minor-image-version=$(XEN_SUBVERSION) EFI_LDFLAGS += --major-os-version=2 --minor-os-version=0 EFI_LDFLAGS += --major-subsystem-version=2 --minor-subsystem-version=0 +EFI_LDFLAGS += --no-insert-timestamp # Check if the compiler supports the MS ABI. export XEN_BUILD_EFI := $(shell $(CC) $(XEN_CFLAGS) -c efi/check.c -o efi/check.o 2>/dev/null && echo y)
This is for improving reproducible builds. Signed-off-by: Frédéric Pierret (fepitre) <frederic.pierret@qubes-os.org> --- xen/arch/x86/Makefile | 1 + 1 file changed, 1 insertion(+)