From patchwork Wed May 15 21:35:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Fernandes X-Patchwork-Id: 10945361 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 45D0076 for ; Wed, 15 May 2019 21:36:13 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 338FE2873E for ; Wed, 15 May 2019 21:36:13 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2690328748; Wed, 15 May 2019 21:36:13 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 195602873E for ; Wed, 15 May 2019 21:36:11 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727142AbfEOVgK (ORCPT ); Wed, 15 May 2019 17:36:10 -0400 Received: from mail-pg1-f195.google.com ([209.85.215.195]:42173 "EHLO mail-pg1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727021AbfEOVgK (ORCPT ); Wed, 15 May 2019 17:36:10 -0400 Received: by mail-pg1-f195.google.com with SMTP id 145so418989pgg.9 for ; Wed, 15 May 2019 14:36:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=VkH12zRhE8bVix5ewWJwVnDU/5Q47hhlmO4uv3QOaIE=; b=kGO1Xsyc63DMPKfIrpQrhAvr0k0HEPAk9NYI/7CZkXpM6gbD5xV1wUF6vBxeiHpC1B RieDPoOytOZeQXYGYjpzeJvIOtv2M1aIjuadHASU58SMx0Pyq36J9BXX9O72cH06D1gC DDiX0UNjQ8BGU/AmNoEgN9kmbQue8NwU/yy4U= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=VkH12zRhE8bVix5ewWJwVnDU/5Q47hhlmO4uv3QOaIE=; b=nLDLX53rXHNE1akGo/JVm6SFSxRKwHLe+ayc3733Ic8CdbDMjc3nY1fItF4JlD/UpZ 8COkhAf17MqjtG8p1MlMjJ5/YwpvAJlZHX3GGn0DVpbaq9j4plXQhCQDfYJS1JTdvqjN lhBeA38a+qw+7d1qpMTJI6eI2GuFglM8YSQiYKPcdK4NaLZ+2BLmDsJRUNyYA7eDsWqk 2mSOJv4PWQ+PYi28obcgZ07Hkz8r+pn3ShcF6uEFgdbnh4EfZHPrJv+JR2PiVU9K7w18 5sQSBKfzw2mGlNlNo0OdaqVfy8lSJ7zZNW3HtNPv5CGpXxrkgb9RrX+ZGJlmAXHFgHfQ 1OrQ== X-Gm-Message-State: APjAAAVuoSnh7RWqiPy+guIiVb2yqXNleDOEM0aoPTWcsBGFG+9TXthV af4HwUyQ4q9W6btnsG22qrTGnA== X-Google-Smtp-Source: APXvYqzCqqy4RvGfmptPBRowriykLFvDEdJVeVAGrgf/YTZWfqTKcxSa5FLRJE9fRuAVVrhGuaTEHw== X-Received: by 2002:a65:5647:: with SMTP id m7mr45645191pgs.348.1557956169356; Wed, 15 May 2019 14:36:09 -0700 (PDT) Received: from joelaf.cam.corp.google.com ([2620:15c:6:12:9c46:e0da:efbf:69cc]) by smtp.gmail.com with ESMTPSA id q4sm3695279pgb.39.2019.05.15.14.36.05 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 15 May 2019 14:36:08 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org Cc: "Joel Fernandes (Google)" , Steven Rostedt , Adrian Ratiu , Alexei Starovoitov , Andrew Morton , atishp04@gmail.com, bpf@vger.kernel.org, Brendan Gregg , Brendan Gregg , dancol@google.com, Daniel Borkmann , Dan Williams , dietmar.eggemann@arm.com, duyuchao , gregkh@linuxfoundation.org, Guenter Roeck , Jonathan Corbet , Karim Yaghmour , Kees Cook , kernel-team@android.com, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-trace-devel@vger.kernel.org, Manjo Raja Rao , Masahiro Yamada , Masami Hiramatsu , =?utf-8?q?Micha=C5=82_Gregorczyk?= , Michal Gregorczyk , Mohammad Husain , Olof Johansson , qais.yousef@arm.com, rdunlap@infradead.org, Shuah Khan , Srinivas Ramana , Tamir Carmeli , yhs@fb.com Subject: [PATCH v4 1/2] kheaders: Move from proc to sysfs Date: Wed, 15 May 2019 17:35:51 -0400 Message-Id: <20190515213552.203737-2-joel@joelfernandes.org> X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog In-Reply-To: <20190515213552.203737-1-joel@joelfernandes.org> References: <20190515213552.203737-1-joel@joelfernandes.org> MIME-Version: 1.0 Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The kheaders archive consisting of the kernel headers used for compiling bpf programs is in /proc. However there is concern that moving it here will make it permanent. Let us move it to /sys/kernel as discussed [1]. [1] https://lore.kernel.org/patchwork/patch/1067310/#1265969 Suggested-by: Steven Rostedt Signed-off-by: Joel Fernandes (Google) --- This patch applies on top of the previous patch that was applied to the driver tree: https://lore.kernel.org/patchwork/patch/1067310/ v2->v3: Fixed sysfs file mode nit (Greg). v1->v2: Fixed some kconfig nits (Masami). init/Kconfig | 17 +++++---- kernel/Makefile | 4 +-- kernel/{gen_ikh_data.sh => gen_kheaders.sh} | 2 +- kernel/kheaders.c | 40 +++++++++------------ 4 files changed, 27 insertions(+), 36 deletions(-) rename kernel/{gen_ikh_data.sh => gen_kheaders.sh} (98%) diff --git a/init/Kconfig b/init/Kconfig index 8b9ffe236e4f..16a7540d60c8 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -579,15 +579,14 @@ config IKCONFIG_PROC This option enables access to the kernel configuration file through /proc/config.gz. -config IKHEADERS_PROC - tristate "Enable kernel header artifacts through /proc/kheaders.tar.xz" - depends on PROC_FS - help - This option enables access to the kernel header and other artifacts that - are generated during the build process. These can be used to build eBPF - tracing programs, or similar programs. If you build the headers as a - module, a module called kheaders.ko is built which can be loaded on-demand - to get access to the headers. +config IKHEADERS + tristate "Enable kernel headers through /sys/kernel/kheaders.tar.xz" + depends on SYSFS + help + This option enables access to the in-kernel headers that are generated during + the build process. These can be used to build eBPF tracing programs, + or similar programs. If you build the headers as a module, a module called + kheaders.ko is built which can be loaded on-demand to get access to headers. config LOG_BUF_SHIFT int "Kernel log buffer size (16 => 64KB, 17 => 128KB)" diff --git a/kernel/Makefile b/kernel/Makefile index 33824f0385b3..a8d923b5481b 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -71,7 +71,7 @@ obj-$(CONFIG_UTS_NS) += utsname.o obj-$(CONFIG_USER_NS) += user_namespace.o obj-$(CONFIG_PID_NS) += pid_namespace.o obj-$(CONFIG_IKCONFIG) += configs.o -obj-$(CONFIG_IKHEADERS_PROC) += kheaders.o +obj-$(CONFIG_IKHEADERS) += kheaders.o obj-$(CONFIG_SMP) += stop_machine.o obj-$(CONFIG_KPROBES_SANITY_TEST) += test_kprobes.o obj-$(CONFIG_AUDIT) += audit.o auditfilter.o @@ -127,7 +127,7 @@ $(obj)/config_data.gz: $(KCONFIG_CONFIG) FORCE $(obj)/kheaders.o: $(obj)/kheaders_data.tar.xz quiet_cmd_genikh = CHK $(obj)/kheaders_data.tar.xz -cmd_genikh = $(CONFIG_SHELL) $(srctree)/kernel/gen_ikh_data.sh $@ +cmd_genikh = $(CONFIG_SHELL) $(srctree)/kernel/gen_kheaders.sh $@ $(obj)/kheaders_data.tar.xz: FORCE $(call cmd,genikh) diff --git a/kernel/gen_ikh_data.sh b/kernel/gen_kheaders.sh similarity index 98% rename from kernel/gen_ikh_data.sh rename to kernel/gen_kheaders.sh index 591a94f7b387..581b83534587 100755 --- a/kernel/gen_ikh_data.sh +++ b/kernel/gen_kheaders.sh @@ -2,7 +2,7 @@ # SPDX-License-Identifier: GPL-2.0 # This script generates an archive consisting of kernel headers -# for CONFIG_IKHEADERS_PROC. +# for CONFIG_IKHEADERS. set -e spath="$(dirname "$(readlink -f "$0")")" kroot="$spath/.." diff --git a/kernel/kheaders.c b/kernel/kheaders.c index 70ae6052920d..8f69772af77b 100644 --- a/kernel/kheaders.c +++ b/kernel/kheaders.c @@ -8,9 +8,8 @@ #include #include -#include +#include #include -#include /* * Define kernel_headers_data and kernel_headers_data_end, within which the @@ -31,39 +30,32 @@ extern char kernel_headers_data; extern char kernel_headers_data_end; static ssize_t -ikheaders_read_current(struct file *file, char __user *buf, - size_t len, loff_t *offset) +ikheaders_read(struct file *file, struct kobject *kobj, + struct bin_attribute *bin_attr, + char *buf, loff_t off, size_t len) { - return simple_read_from_buffer(buf, len, offset, - &kernel_headers_data, - &kernel_headers_data_end - - &kernel_headers_data); + memcpy(buf, &kernel_headers_data + off, len); + return len; } -static const struct file_operations ikheaders_file_ops = { - .read = ikheaders_read_current, - .llseek = default_llseek, +static struct bin_attribute kheaders_attr __ro_after_init = { + .attr = { + .name = "kheaders.tar.xz", + .mode = 0444, + }, + .read = &ikheaders_read, }; static int __init ikheaders_init(void) { - struct proc_dir_entry *entry; - - /* create the current headers file */ - entry = proc_create("kheaders.tar.xz", S_IRUGO, NULL, - &ikheaders_file_ops); - if (!entry) - return -ENOMEM; - - proc_set_size(entry, - &kernel_headers_data_end - - &kernel_headers_data); - return 0; + kheaders_attr.size = (&kernel_headers_data_end - + &kernel_headers_data); + return sysfs_create_bin_file(kernel_kobj, &kheaders_attr); } static void __exit ikheaders_cleanup(void) { - remove_proc_entry("kheaders.tar.xz", NULL); + sysfs_remove_bin_file(kernel_kobj, &kheaders_attr); } module_init(ikheaders_init); From patchwork Wed May 15 21:35:52 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joel Fernandes X-Patchwork-Id: 10945365 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9F6931515 for ; Wed, 15 May 2019 21:36:19 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8CB3A2873E for ; Wed, 15 May 2019 21:36:19 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7EF90288D0; Wed, 15 May 2019 21:36:19 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-8.0 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 074CC28741 for ; Wed, 15 May 2019 21:36:19 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727231AbfEOVgP (ORCPT ); Wed, 15 May 2019 17:36:15 -0400 Received: from mail-pl1-f195.google.com ([209.85.214.195]:44545 "EHLO mail-pl1-f195.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727275AbfEOVgO (ORCPT ); Wed, 15 May 2019 17:36:14 -0400 Received: by mail-pl1-f195.google.com with SMTP id c5so474717pll.11 for ; Wed, 15 May 2019 14:36:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=joelfernandes.org; s=google; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=zjB9LsovF8lpor6w+CVg9dpdpQizlQADdXWnWKh0+Rg=; b=VYyAWbWi7W7y+gL1aVVELgMHHzDwCnQoRhq01306PoaSmD2u7UqvAOGVFr/KZMuRCm Ad56tj2pg1vt1yOuEEChvUVluDdPThEqtHcqeTOoLtCbGuWm6Ayjuh8w7xMROHdTGo2S wbtlJxG2lTEdzsemWMiOsQy/Y140eT1YBdVko= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=zjB9LsovF8lpor6w+CVg9dpdpQizlQADdXWnWKh0+Rg=; b=NOng7HjJ3jH3LhTRzNZMDJK2qyLbZuUCXm80avJTFgsrOENwBTLpffus91YKln+Dr5 arZ9s/pThCNcqY4Raq3LXGlYkpYuUELF79SuLOVFn9gc5kuxoaOyZkVjlsWfsur9VVGS mPgV07qVHfClxP/v3IduHyk/aKGeg4f6R7ys3M1ba/IhvoIfl4hcMDoVJFXVg846hdNX 7eUlBJUnD/FoxmNpmAe8nRd/M5BVHgYHL4K/m5JiwVLCSgRhJbOeaI3tYnc9buWM/utm +GJp9uAsWeKkx3kPgYXW+p4M+5msm2wGG8uACDC54V47aLhuYGP/OAyangKcDeRmsGy/ Q7AA== X-Gm-Message-State: APjAAAX1wVA5gkVk+FXkx+Lu8V9cHp6FnY2/pL27SXBYvhJPUFBclowq xiKhPVmZQuUZWUzPQmAIOpeQgw== X-Google-Smtp-Source: APXvYqxHN70es4b0dCS2R1CKfqsJmikP3jLaCFw6mu2/8pQd3QzrWRXNHoRuaFwwglq5MBBI8QPEZA== X-Received: by 2002:a17:902:bc85:: with SMTP id bb5mr46246086plb.310.1557956173660; Wed, 15 May 2019 14:36:13 -0700 (PDT) Received: from joelaf.cam.corp.google.com ([2620:15c:6:12:9c46:e0da:efbf:69cc]) by smtp.gmail.com with ESMTPSA id q4sm3695279pgb.39.2019.05.15.14.36.09 (version=TLS1_3 cipher=AEAD-AES256-GCM-SHA384 bits=256/256); Wed, 15 May 2019 14:36:12 -0700 (PDT) From: "Joel Fernandes (Google)" To: linux-kernel@vger.kernel.org Cc: "Joel Fernandes (Google)" , Linus Torvalds , Adrian Ratiu , Alexei Starovoitov , Andrew Morton , atishp04@gmail.com, bpf@vger.kernel.org, Brendan Gregg , Brendan Gregg , dancol@google.com, Daniel Borkmann , Dan Williams , dietmar.eggemann@arm.com, duyuchao , gregkh@linuxfoundation.org, Guenter Roeck , Jonathan Corbet , Karim Yaghmour , Kees Cook , kernel-team@android.com, linux-doc@vger.kernel.org, linux-kselftest@vger.kernel.org, linux-trace-devel@vger.kernel.org, Manjo Raja Rao , Masahiro Yamada , Masami Hiramatsu , =?utf-8?q?Micha=C5=82_Gregorczyk?= , Michal Gregorczyk , Mohammad Husain , Olof Johansson , qais.yousef@arm.com, rdunlap@infradead.org, rostedt@goodmis.org, Shuah Khan , Srinivas Ramana , Tamir Carmeli , yhs@fb.com Subject: [PATCH v4 2/2] kheaders: Do not regenerate archive if config is not changed Date: Wed, 15 May 2019 17:35:52 -0400 Message-Id: <20190515213552.203737-3-joel@joelfernandes.org> X-Mailer: git-send-email 2.21.0.1020.gf2820cf01a-goog In-Reply-To: <20190515213552.203737-1-joel@joelfernandes.org> References: <20190515213552.203737-1-joel@joelfernandes.org> MIME-Version: 1.0 Sender: linux-kselftest-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Linus reported an issue that doing an allmodconfig was causing the kheaders archive to be regenerated even though the config is the same. This patch fixes the issue by ignoring the config-related header files for "knowing when to regenerate based on timestamps". Instead, if the CONFIG_X_Y option really changes, then we there are the include/config/X/Y.h which will already tells us "if a config really changed". So we don't really need these files for regeneration detection anyway, and ignoring them fixes Linus's issue. Reported-by: Linus Torvalds Signed-off-by: Joel Fernandes (Google) --- kernel/gen_kheaders.sh | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/kernel/gen_kheaders.sh b/kernel/gen_kheaders.sh index 581b83534587..9a34e1d9bd7f 100755 --- a/kernel/gen_kheaders.sh +++ b/kernel/gen_kheaders.sh @@ -31,9 +31,8 @@ arch/$SRCARCH/include/ # This block is useful for debugging the incremental builds. # Uncomment it for debugging. -# iter=1 -# if [ ! -f /tmp/iter ]; then echo 1 > /tmp/iter; -# else; iter=$(($(cat /tmp/iter) + 1)); fi +# if [ ! -f /tmp/iter ]; then iter=1; echo 1 > /tmp/iter; +# else iter=$(($(cat /tmp/iter) + 1)); echo $iter > /tmp/iter; fi # find $src_file_list -type f | xargs ls -lR > /tmp/src-ls-$iter # find $obj_file_list -type f | xargs ls -lR > /tmp/obj-ls-$iter @@ -43,10 +42,18 @@ arch/$SRCARCH/include/ pushd $kroot > /dev/null src_files_md5="$(find $src_file_list -type f | grep -v "include/generated/compile.h" | + grep -v "include/generated/autoconf.h" | + grep -v "include/config/auto.conf" | + grep -v "include/config/auto.conf.cmd" | + grep -v "include/config/tristate.conf" | xargs ls -lR | md5sum | cut -d ' ' -f1)" popd > /dev/null obj_files_md5="$(find $obj_file_list -type f | grep -v "include/generated/compile.h" | + grep -v "include/generated/autoconf.h" | + grep -v "include/config/auto.conf" | + grep -v "include/config/auto.conf.cmd" | + grep -v "include/config/tristate.conf" | xargs ls -lR | md5sum | cut -d ' ' -f1)" if [ -f $tarfile ]; then tarfile_md5="$(md5sum $tarfile | cut -d ' ' -f1)"; fi @@ -82,7 +89,7 @@ find $cpio_dir -type f -print0 | tar -Jcf $tarfile -C $cpio_dir/ . > /dev/null -echo "$src_files_md5" > kernel/kheaders.md5 +echo "$src_files_md5" > kernel/kheaders.md5 echo "$obj_files_md5" >> kernel/kheaders.md5 echo "$(md5sum $tarfile | cut -d ' ' -f1)" >> kernel/kheaders.md5