| Message ID | 20160405003237.GK21187@decadent.org.uk (mailing list archive) |
|---|---|
| State | Deferred |
| Headers | show |
Hi, CC'ing Rusty On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings <ben@decadent.org.uk> wrote: > Debian will not sign modules during the kernel package build, as this > conflicts with the goal of reproducible builds. Instead, we will > generate detached signatures offline and include them in a second > package. Is this a decision already? It doesn't look as a good reason - you would already need to provide a signing key (CONFIG_MODULE_SIG_KEY) anyway for this to work. How is leaving the module signature in another package be any better than just signing the module? If you have the signature, the build is just as reproducible as before. > We could attach the signatures when building this second package or at > installation time, but that leads to duplication of all modules, > either in the archive or on users' systems. > > To avoid this, add support to libkmod for concatenating modules with > detached signatures (files with the '.sig' extension) at load time. this has the drawback that finit_module() can't be used. > Signed-off-by: Ben Hutchings <ben@decadent.org.uk> > --- > v2: Fix syntax error in the xz case > > Missed this because I didn't realise the Debian package disables gzip > and xz support. > > Ben. > > libkmod/libkmod-file.c | 110 +++++++++++++++++++++++++++++++++++++++++++++---- > 1 file changed, 103 insertions(+), 7 deletions(-) > > diff --git a/libkmod/libkmod-file.c b/libkmod/libkmod-file.c > index 5eeba6a912a2..cb1da3c9e2ae 100644 > --- a/libkmod/libkmod-file.c > +++ b/libkmod/libkmod-file.c ... > @@ -292,12 +370,25 @@ struct kmod_file *kmod_file_open(const struct kmod_ctx *ctx, > if (file == NULL) > return NULL; > > + file->sig_fd = -1; > + > file->fd = open(filename, O_RDONLY|O_CLOEXEC); > if (file->fd < 0) { > err = -errno; > goto error; > } > > + /* Try to open a detached signature. If it's missing, that's OK. */ > + if (asprintf(&sig_filename, "%s.sig", filename) < 0) { > + err = -errno; > + goto error; > + } > + file->sig_fd = open(sig_filename, O_RDONLY|O_CLOEXEC); > + if (file->sig_fd < 0 && errno != ENOENT) { > + err = -errno; > + goto error; > + } This can't really work if the module is being loaded uncompressed (I think nowadays we can even add support for compressed modules... Rusty, any input here?). When the module is being directly loaded, the direct flag gets set so kmod_module_insert_module() knows it can try to use finit_module(). Since you have an external signature what would happen is that we would load the signature, but try to load the module in the kernel without it. I'm still not convinced the split module + signature is actually a good thing. Lucas De Marchi -- To unsubscribe from this list: send the line "unsubscribe linux-modules" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, 2016-04-13 at 01:05 -0300, Lucas De Marchi wrote: > Hi, > > CC'ing Rusty > > On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings <ben@decadent.org.uk> wrote: > > > > Debian will not sign modules during the kernel package build, as this > > conflicts with the goal of reproducible builds. Instead, we will > > generate detached signatures offline and include them in a second > > package. > Is this a decision already? It doesn't look as a good reason - you > would already need to provide a signing key (CONFIG_MODULE_SIG_KEY) > anyway for this to work. How is leaving the module signature in > another package be any better than just signing the module? If you > have the signature, the build is just as reproducible as before. I think we may have different ideas about what reproducibility means. When I say reproducible I mean *anyone* with the right tools installed can reproduce the binary packages (.deb) from the source package (.dsc and tarballs). The signing key obviously isn't available to everyone, so the source package has to include detached signatures prepared outside of the package build process. But we can't put them in the linux source package, because that results in a dependency loop. > > > > We could attach the signatures when building this second package or at > > installation time, but that leads to duplication of all modules, > > either in the archive or on users' systems. > > > > To avoid this, add support to libkmod for concatenating modules with > > detached signatures (files with the '.sig' extension) at load time. > this has the drawback that finit_module() can't be used. So does module compression, but it's still a supported option. [...] > > + /* Try to open a detached signature. If it's missing, that's OK. */ > > + if (asprintf(&sig_filename, "%s.sig", filename) < 0) { > > + err = -errno; > > + goto error; > > + } > > + file->sig_fd = open(sig_filename, O_RDONLY|O_CLOEXEC); > > + if (file->sig_fd < 0 && errno != ENOENT) { > > + err = -errno; > > + goto error; > > + } > This can't really work if the module is being loaded uncompressed (I > think nowadays we can even add support for compressed modules... > Rusty, any input here?). > > When the module is being directly loaded, the direct flag gets set so > kmod_module_insert_module() knows it can try to use finit_module(). > Since you have an external signature what would happen is that we > would load the signature, but try to load the module in the kernel > without it. It does work. I changed load_reg() to disable direct loading.when there's a detached signature. Ben. > I'm still not convinced the split module + signature is actually a good thing. > > > Lucas De Marchi
On Wed, 2016-04-13 at 11:00 +0100, Ben Hutchings wrote: > On Wed, 2016-04-13 at 01:05 -0300, Lucas De Marchi wrote: > > > > Hi, > > > > CC'ing Rusty > > > > On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings <ben@decadent.org.uk> wrote: > > > > > > > > > Debian will not sign modules during the kernel package build, as this > > > conflicts with the goal of reproducible builds. Instead, we will > > > generate detached signatures offline and include them in a second > > > package. > > Is this a decision already? It doesn't look as a good reason - you > > would already need to provide a signing key (CONFIG_MODULE_SIG_KEY) > > anyway for this to work. How is leaving the module signature in > > another package be any better than just signing the module? If you > > have the signature, the build is just as reproducible as before. > I think we may have different ideas about what reproducibility means. > When I say reproducible I mean *anyone* with the right tools installed > can reproduce the binary packages (.deb) from the source package (.dsc > and tarballs). > > The signing key obviously isn't available to everyone, so the source > package has to include detached signatures prepared outside of the > package build process. But we can't put them in the linux source > package, because that results in a dependency loop. So, given these requirements, what do you think now about supporting detached signatures? I spoke at greater length about what I'm trying to do at Linuxwochen Wien; see http://meetings-archive.debian.net/pub/debian-meetings/2016/mini-debconf-vienna/webm/Secure_Boot_vs_the_Debian_linux_package.webm#t=595 from about 9'55" to 17'30". Ben. > > > > > > > > > > > We could attach the signatures when building this second package or at > > > installation time, but that leads to duplication of all modules, > > > either in the archive or on users' systems. > > > > > > To avoid this, add support to libkmod for concatenating modules with > > > detached signatures (files with the '.sig' extension) at load time. > > this has the drawback that finit_module() can't be used. > So does module compression, but it's still a supported option. > > [...] > > > > > > > > + /* Try to open a detached signature. If it's missing, that's OK. */ > > > + if (asprintf(&sig_filename, "%s.sig", filename) < 0) { > > > + err = -errno; > > > + goto error; > > > + } > > > + file->sig_fd = open(sig_filename, O_RDONLY|O_CLOEXEC); > > > + if (file->sig_fd < 0 && errno != ENOENT) { > > > + err = -errno; > > > + goto error; > > > + } > > This can't really work if the module is being loaded uncompressed (I > > think nowadays we can even add support for compressed modules... > > Rusty, any input here?). > > > > When the module is being directly loaded, the direct flag gets set so > > kmod_module_insert_module() knows it can try to use finit_module(). > > Since you have an external signature what would happen is that we > > would load the signature, but try to load the module in the kernel > > without it. > It does work. I changed load_reg() to disable direct loading.when > there's a detached signature. > > Ben. > > > > > I'm still not convinced the split module + signature is actually a good thing. > > > > > > Lucas De Marchi
On Wed, Apr 13, 2016 at 7:00 AM, Ben Hutchings <ben@decadent.org.uk> wrote: > On Wed, 2016-04-13 at 01:05 -0300, Lucas De Marchi wrote: >> Hi, >> >> CC'ing Rusty >> >> On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings <ben@decadent.org.uk> wrote: >> > >> > Debian will not sign modules during the kernel package build, as this >> > conflicts with the goal of reproducible builds. Instead, we will >> > generate detached signatures offline and include them in a second >> > package. >> Is this a decision already? It doesn't look as a good reason - you >> would already need to provide a signing key (CONFIG_MODULE_SIG_KEY) >> anyway for this to work. How is leaving the module signature in >> another package be any better than just signing the module? If you >> have the signature, the build is just as reproducible as before. > > I think we may have different ideas about what reproducibility means. > When I say reproducible I mean *anyone* with the right tools installed > can reproduce the binary packages (.deb) from the source package (.dsc > and tarballs). > > The signing key obviously isn't available to everyone, so the source > package has to include detached signatures prepared outside of the And how is this signature prepared? Since it needs the compiled module it would be a matter of changing the compiler, even minor version, to invalidate the argument of reproducible build. It seems very fragile to me. > package build process. But we can't put them in the linux source > package, because that results in a dependency loop. > >> > >> > We could attach the signatures when building this second package or at >> > installation time, but that leads to duplication of all modules, >> > either in the archive or on users' systems. >> > >> > To avoid this, add support to libkmod for concatenating modules with >> > detached signatures (files with the '.sig' extension) at load time. >> this has the drawback that finit_module() can't be used. > > So does module compression, but it's still a supported option. This is easily fixed by teaching the kernel to handle the fd as a compressed file. The kernel already has the routines to uncompress them anyway. Supporting detached signatures means it can't be fixed anymore since we will have to use init_module() rather than finit_module(). Lucas De Marchi -- To unsubscribe from this list: send the line "unsubscribe linux-modules" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On May 21, Lucas De Marchi <lucas.de.marchi@gmail.com> wrote: > And how is this signature prepared? Since it needs the compiled > module it would be a matter of changing the compiler, even minor > version, to invalidate the argument of reproducible build. It seems > very fragile to me. But this is the whole point of reproducible builds: knowing that if you start from the same sources and build enviroment, and distributions do, then you will get the same results. Building gcc is reproducibile as well... Reproducibility with a different compiler has never been a goal.
On Sat, 2016-05-21 at 15:31 -0300, Lucas De Marchi wrote: > On Wed, Apr 13, 2016 at 7:00 AM, Ben Hutchings <ben@decadent.org.uk> wrote: > > > > On Wed, 2016-04-13 at 01:05 -0300, Lucas De Marchi wrote: > > > > > > Hi, > > > > > > CC'ing Rusty > > > > > > On Mon, Apr 4, 2016 at 9:32 PM, Ben Hutchings wrote: > > > > > > > > > > > > Debian will not sign modules during the kernel package build, as this > > > > conflicts with the goal of reproducible builds. Instead, we will > > > > generate detached signatures offline and include them in a second > > > > package. > > > Is this a decision already? It doesn't look as a good reason - you > > > would already need to provide a signing key (CONFIG_MODULE_SIG_KEY) > > > anyway for this to work. How is leaving the module signature in > > > another package be any better than just signing the module? If you > > > have the signature, the build is just as reproducible as before. > > I think we may have different ideas about what reproducibility means. > > When I say reproducible I mean *anyone* with the right tools installed > > can reproduce the binary packages (.deb) from the source package (.dsc > > and tarballs). > > > > The signing key obviously isn't available to everyone, so the source > > package has to include detached signatures prepared outside of the > And how is this signature prepared? Since it needs the compiled > module it would be a matter of changing the compiler, even minor > version, to invalidate the argument of reproducible build. It seems > very fragile to me. The versions of build tools have to be recorded: https://reproducible-builds.org/docs/formal-definition/ https://wiki.debian.org/ReproducibleBuilds/BuildinfoSpecification > > package build process. But we can't put them in the linux source > > package, because that results in a dependency loop. > > > > > > > > > > > > > > > > > We could attach the signatures when building this second package or at > > > > installation time, but that leads to duplication of all modules, > > > > either in the archive or on users' systems. > > > > > > > > To avoid this, add support to libkmod for concatenating modules with > > > > detached signatures (files with the '.sig' extension) at load time. > > > this has the drawback that finit_module() can't be used. > > So does module compression, but it's still a supported option. > This is easily fixed by teaching the kernel to handle the fd as a > compressed file. This sounds speculative. > The kernel already has the routines to uncompress > them anyway. Supporting detached signatures means it can't be fixed > anymore since we will have to use init_module() rather than > finit_module(). Why does that matter? init_module() isn't deprecated. Ben.
I'm withdrawing this patch for reasons explained in http://lists.debian.org/1464525520.2762.80.camel@decadent.org.uk Ben. -- Ben Hutchings Editing code like this is akin to sticking plasters on the bleeding stump of a severed limb. - me, 29 June 1999
On Sun, May 29, 2016 at 9:48 AM, Ben Hutchings <ben@decadent.org.uk> wrote: > I'm withdrawing this patch for reasons explained in > http://lists.debian.org/1464525520.2762.80.camel@decadent.org.uk quoting some parts: > This is blocked on upstream acceptance in kmod, and it's not clear > whether that's ever going to happen." I'm more against the impact of how this is implemented, not against the idea of reproducible builds you are pursuing. From the points you raised there: > 1. Attach module signatures at installation time, in a subdirectory. > Change kmod to prefer this subdirectory (this is purely a > configuration change). It would also be possible to check during > installation that signatures match the installed unsigned modules, > and if not then abort and leave any older signed modules in place. Yep, this is a mere change to depmod.d config files. > 2. Attach module signatures at package build time, making the > linux-image-signed packages provide/conflict/replace the > corresponding linux-image packages. For architectures with > signed modules, udebs would be built from linux-signed and not > from linux. very reasonable, too. Lucas De Marchi -- To unsubscribe from this list: send the line "unsubscribe linux-modules" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/libkmod/libkmod-file.c b/libkmod/libkmod-file.c index 5eeba6a912a2..cb1da3c9e2ae 100644 --- a/libkmod/libkmod-file.c +++ b/libkmod/libkmod-file.c @@ -52,6 +52,7 @@ struct kmod_file { gzFile gzf; #endif int fd; + int sig_fd; bool direct; off_t size; void *memory; @@ -60,6 +61,37 @@ struct kmod_file { struct kmod_elf *elf; }; +static int append_detached_sig(struct kmod_file *file, size_t buf_size) +{ + struct stat st; + ssize_t read_size; + + if (file->sig_fd < 0) + return 0; + + if (fstat(file->sig_fd, &st) < 0) + return -errno; + + /* Grow the buffer if necessary */ + if ((size_t)st.st_size > buf_size - file->size) { + void *tmp = realloc(file->memory, file->size + st.st_size); + if (tmp == NULL) + return -errno; + file->memory = tmp; + } + + read_size = read(file->sig_fd, (char *)file->memory + file->size, + st.st_size); + if (read_size < 0) + return -errno; + if (read_size != st.st_size) + return -EINVAL; + + file->size += read_size; + + return 0; +} + #ifdef ENABLE_XZ static void xz_uncompress_belch(struct kmod_file *file, lzma_ret ret) { @@ -144,6 +176,7 @@ static int load_xz(struct kmod_file *file) { lzma_stream strm = LZMA_STREAM_INIT; lzma_ret lzret; + size_t buf_size; int ret; lzret = lzma_stream_decoder(&strm, UINT64_MAX, LZMA_CONCATENATED); @@ -155,7 +188,14 @@ static int load_xz(struct kmod_file *file) return -EINVAL; } ret = xz_uncompress(&strm, file); + buf_size = file->size + strm.avail_out; lzma_end(&strm); + + if (!ret) { + ret = append_detached_sig(file, buf_size); + if (ret) + free(file->memory); + } return ret; } @@ -214,6 +254,11 @@ static int load_zlib(struct kmod_file *file) file->memory = p; file->size = did; + + err = append_detached_sig(file, total); + if (err) + goto error; + p = NULL; return 0; @@ -254,18 +299,50 @@ static int load_reg(struct kmod_file *file) if (fstat(file->fd, &st) < 0) return -errno; - file->size = st.st_size; - file->memory = mmap(NULL, file->size, PROT_READ, MAP_PRIVATE, - file->fd, 0); - if (file->memory == MAP_FAILED) - return -errno; - file->direct = true; + if (file->sig_fd < 0) { + file->size = st.st_size; + file->memory = mmap(NULL, file->size, PROT_READ, MAP_PRIVATE, + file->fd, 0); + if (file->memory == MAP_FAILED) + return -errno; + file->direct = true; + } else { + size_t plain_size = st.st_size, sig_size; + _cleanup_free_ unsigned char *p = NULL; + ssize_t ret; + + if (fstat(file->sig_fd, &st) < 0) + return -errno; + sig_size = st.st_size; + + p = malloc(plain_size + sig_size); + if (!p) + return -errno; + + ret = read(file->fd, p, plain_size); + if (ret < 0) + return -errno; + if ((size_t)ret != plain_size) + return -EINVAL; + file->memory = p; + file->size = plain_size; + + ret = append_detached_sig(file, plain_size + sig_size); + if (ret) + return ret; + + p = NULL; + } + return 0; } static void unload_reg(struct kmod_file *file) { - munmap(file->memory, file->size); + if (file->direct) + munmap(file->memory, file->size); + else + free(file->memory); } static const struct file_ops reg_ops = { @@ -285,6 +362,7 @@ struct kmod_file *kmod_file_open(const struct kmod_ctx *ctx, const char *filename) { struct kmod_file *file = calloc(1, sizeof(struct kmod_file)); + char *sig_filename = NULL; const struct comp_type *itr; size_t magic_size_max = 0; int err; @@ -292,12 +370,25 @@ struct kmod_file *kmod_file_open(const struct kmod_ctx *ctx, if (file == NULL) return NULL; + file->sig_fd = -1; + file->fd = open(filename, O_RDONLY|O_CLOEXEC); if (file->fd < 0) { err = -errno; goto error; } + /* Try to open a detached signature. If it's missing, that's OK. */ + if (asprintf(&sig_filename, "%s.sig", filename) < 0) { + err = -errno; + goto error; + } + file->sig_fd = open(sig_filename, O_RDONLY|O_CLOEXEC); + if (file->sig_fd < 0 && errno != ENOENT) { + err = -errno; + goto error; + } + for (itr = comp_types; itr->ops.load != NULL; itr++) { if (magic_size_max < itr->magic_size) magic_size_max = itr->magic_size; @@ -336,7 +427,10 @@ struct kmod_file *kmod_file_open(const struct kmod_ctx *ctx, err = file->ops->load(file); file->ctx = ctx; error: + free(sig_filename); if (err < 0) { + if (file->sig_fd >= 0) + close(file->sig_fd); if (file->fd >= 0) close(file->fd); free(file); @@ -373,6 +467,8 @@ void kmod_file_unref(struct kmod_file *file) kmod_elf_unref(file->elf); file->ops->unload(file); + if (file->sig_fd >= 0) + close(file->sig_fd); if (file->fd >= 0) close(file->fd); free(file);
Debian will not sign modules during the kernel package build, as this conflicts with the goal of reproducible builds. Instead, we will generate detached signatures offline and include them in a second package. We could attach the signatures when building this second package or at installation time, but that leads to duplication of all modules, either in the archive or on users' systems. To avoid this, add support to libkmod for concatenating modules with detached signatures (files with the '.sig' extension) at load time. Signed-off-by: Ben Hutchings <ben@decadent.org.uk> --- v2: Fix syntax error in the xz case Missed this because I didn't realise the Debian package disables gzip and xz support. Ben. libkmod/libkmod-file.c | 110 +++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 103 insertions(+), 7 deletions(-)