diff mbox series

[v7,1/2] kprobes: Implement trampoline memory allocator for tracing

Message ID 20240326134616.7691-1-jarkko@kernel.org (mailing list archive)
State New, archived
Headers show
Series [v7,1/2] kprobes: Implement trampoline memory allocator for tracing | expand

Commit Message

Jarkko Sakkinen March 26, 2024, 1:46 p.m. UTC
Tracing with kprobes while running a monolithic kernel is currently
impossible because CONFIG_KPROBES depends on CONFIG_MODULES.

Introduce alloc_execmem() and free_execmem() for allocating executable
memory. If an arch implements these functions, it can mark this up with
the HAVE_ALLOC_EXECMEM kconfig flag.

The second new kconfig flag is ALLOC_EXECMEM, which can be selected if
either MODULES is selected or HAVE_ALLOC_EXECMEM is support by the arch. If
HAVE_ALLOC_EXECMEM is not supported by an arch, module_alloc() and
module_memfree() are used as a fallback, thus retaining backwards
compatibility to earlier kernel versions.

This will allow architecture to enable kprobes traces without requiring
to enable module.

The support can be implemented with four easy steps:

1. Implement alloc_execmem().
2. Implement free_execmem().
3. Edit arch/<arch>/Makefile.
4. Set HAVE_ALLOC_EXECMEM in arch/<arch>/Kconfig.

Link: https://lore.kernel.org/all/20240325115632.04e37297491cadfbbf382767@kernel.org/
Suggested-by: Masami Hiramatsu <mhiramat@kernel.org>
Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
---
v7:
- Use "depends on" for ALLOC_EXECMEM instead of "select"
- Reduced and narrowed CONFIG_MODULES checks further in kprobes.c.
v6:
- Use null pointer for notifiers and register the module notifier only if
  IS_ENABLED(CONFIG_MODULES) is set.
- Fixed typo in the commit message and wrote more verbose description
  of the feature.
v5:
- alloc_execmem() was missing GFP_KERNEL parameter. The patch set did
  compile because 2/2 had the fixup (leaked there when rebasing the
  patch set).
v4:
- Squashed a couple of unrequired CONFIG_MODULES checks.
- See https://lore.kernel.org/all/D034M18D63EC.2Y11D954YSZYK@kernel.org/
v3:
- A new patch added.
- For IS_DEFINED() I need advice as I could not really find that many
  locations where it would be applicable.
---
 arch/Kconfig                | 17 +++++++++++-
 include/linux/execmem.h     | 13 +++++++++
 kernel/kprobes.c            | 53 ++++++++++++++++++++++---------------
 kernel/trace/trace_kprobe.c | 15 +++++++++--
 4 files changed, 73 insertions(+), 25 deletions(-)
 create mode 100644 include/linux/execmem.h

Comments

Christophe Leroy March 28, 2024, 12:45 p.m. UTC | #1
Le 26/03/2024 à 14:46, Jarkko Sakkinen a écrit :
> Tracing with kprobes while running a monolithic kernel is currently
> impossible because CONFIG_KPROBES depends on CONFIG_MODULES.
> 
> Introduce alloc_execmem() and free_execmem() for allocating executable
> memory. If an arch implements these functions, it can mark this up with
> the HAVE_ALLOC_EXECMEM kconfig flag.
> 
> The second new kconfig flag is ALLOC_EXECMEM, which can be selected if
> either MODULES is selected or HAVE_ALLOC_EXECMEM is support by the arch. If
> HAVE_ALLOC_EXECMEM is not supported by an arch, module_alloc() and
> module_memfree() are used as a fallback, thus retaining backwards
> compatibility to earlier kernel versions.
> 
> This will allow architecture to enable kprobes traces without requiring
> to enable module.
> 
> The support can be implemented with four easy steps:
> 
> 1. Implement alloc_execmem().
> 2. Implement free_execmem().
> 3. Edit arch/<arch>/Makefile.
> 4. Set HAVE_ALLOC_EXECMEM in arch/<arch>/Kconfig.
> 
> Link: https://lore.kernel.org/all/20240325115632.04e37297491cadfbbf382767@kernel.org/
> Suggested-by: Masami Hiramatsu <mhiramat@kernel.org>
> Signed-off-by: Jarkko Sakkinen <jarkko@kernel.org>
> ---
> v7:
> - Use "depends on" for ALLOC_EXECMEM instead of "select"
> - Reduced and narrowed CONFIG_MODULES checks further in kprobes.c.
> v6:
> - Use null pointer for notifiers and register the module notifier only if
>    IS_ENABLED(CONFIG_MODULES) is set.
> - Fixed typo in the commit message and wrote more verbose description
>    of the feature.
> v5:
> - alloc_execmem() was missing GFP_KERNEL parameter. The patch set did
>    compile because 2/2 had the fixup (leaked there when rebasing the
>    patch set).
> v4:
> - Squashed a couple of unrequired CONFIG_MODULES checks.
> - See https://lore.kernel.org/all/D034M18D63EC.2Y11D954YSZYK@kernel.org/
> v3:
> - A new patch added.
> - For IS_DEFINED() I need advice as I could not really find that many
>    locations where it would be applicable.
> ---
>   arch/Kconfig                | 17 +++++++++++-
>   include/linux/execmem.h     | 13 +++++++++
>   kernel/kprobes.c            | 53 ++++++++++++++++++++++---------------
>   kernel/trace/trace_kprobe.c | 15 +++++++++--
>   4 files changed, 73 insertions(+), 25 deletions(-)
>   create mode 100644 include/linux/execmem.h
> 
> diff --git a/arch/Kconfig b/arch/Kconfig
> index a5af0edd3eb8..5e9735f60f3c 100644
> --- a/arch/Kconfig
> +++ b/arch/Kconfig
> @@ -52,8 +52,8 @@ config GENERIC_ENTRY
>   
>   config KPROBES
>   	bool "Kprobes"
> -	depends on MODULES
>   	depends on HAVE_KPROBES
> +	depends on ALLOC_EXECMEM
>   	select KALLSYMS
>   	select TASKS_RCU if PREEMPTION
>   	help
> @@ -215,6 +215,21 @@ config HAVE_OPTPROBES
>   config HAVE_KPROBES_ON_FTRACE
>   	bool
>   
> +config HAVE_ALLOC_EXECMEM
> +	bool
> +	help
> +	  Architectures that select this option are capable of allocating trampoline
> +	  executable memory for tracing subsystems, indepedently of the kernel module
> +	  subsystem.
> +
> +config ALLOC_EXECMEM
> +	bool "Executable (trampoline) memory allocation"

Why make it user selectable ? Previously I was able to select KPROBES as 
soon as MODULES was selected. Now I will have to first select 
ALLOC_EXECMEM in addition ? What is the added value of allowing the user 
to disable it ?

> +	default y
> +	depends on MODULES || HAVE_ALLOC_EXECMEM
> +	help
> +	  Select this for executable (trampoline) memory. Can be enabled when either
> +	  module allocator or arch-specific allocator is available.
> +
>   config ARCH_CORRECT_STACKTRACE_ON_KRETPROBE
>   	bool
>   	help
> diff --git a/include/linux/execmem.h b/include/linux/execmem.h
> new file mode 100644
> index 000000000000..ae2ff151523a
> --- /dev/null
> +++ b/include/linux/execmem.h
> @@ -0,0 +1,13 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +#ifndef _LINUX_EXECMEM_H
> +#define _LINUX_EXECMEM_H
> +

It should include moduleloader.h otherwise the user of alloc_execmem() 
must include both this header and moduleloader.h to use alloc_execmem()

> +#ifdef CONFIG_HAVE_ALLOC_EXECMEM
> +void *alloc_execmem(unsigned long size, gfp_t gfp);
> +void free_execmem(void *region);
> +#else
> +#define alloc_execmem(size, gfp)	module_alloc(size)

Then gfp is silently ignored in the case. Is that expected ?

> +#define free_execmem(region)		module_memfree(region)
> +#endif
> +
> +#endif /* _LINUX_EXECMEM_H */
> diff --git a/kernel/kprobes.c b/kernel/kprobes.c
> index 9d9095e81792..13bef5de315c 100644
> --- a/kernel/kprobes.c
> +++ b/kernel/kprobes.c
> @@ -44,6 +44,7 @@
>   #include <asm/cacheflush.h>
>   #include <asm/errno.h>
>   #include <linux/uaccess.h>
> +#include <linux/execmem.h>
>   
>   #define KPROBE_HASH_BITS 6
>   #define KPROBE_TABLE_SIZE (1 << KPROBE_HASH_BITS)
> @@ -113,17 +114,17 @@ enum kprobe_slot_state {
>   void __weak *alloc_insn_page(void)
>   {
>   	/*
> -	 * Use module_alloc() so this page is within +/- 2GB of where the
> +	 * Use alloc_execmem() so this page is within +/- 2GB of where the

This 2G stuff is x86 specific, might be different value on other arches. 
Can we remove ?

>   	 * kernel image and loaded module images reside. This is required
>   	 * for most of the architectures.
>   	 * (e.g. x86-64 needs this to handle the %rip-relative fixups.)
>   	 */
> -	return module_alloc(PAGE_SIZE);
> +	return alloc_execmem(PAGE_SIZE, GFP_KERNEL);
>   }
>   
>   static void free_insn_page(void *page)
>   {
> -	module_memfree(page);
> +	free_execmem(page);
>   }
>   
>   struct kprobe_insn_cache kprobe_insn_slots = {
> @@ -1592,6 +1593,7 @@ static int check_kprobe_address_safe(struct kprobe *p,
>   			goto out;
>   		}
>   
> +#ifdef CONFIG_MODULES
>   		/*
>   		 * If the module freed '.init.text', we couldn't insert
>   		 * kprobes in there.
> @@ -1602,7 +1604,9 @@ static int check_kprobe_address_safe(struct kprobe *p,
>   			*probed_mod = NULL;
>   			ret = -ENOENT;
>   		}
> +#endif /* CONFIG_MODULES */
>   	}
> +
>   out:
>   	preempt_enable();
>   	jump_label_unlock();
> @@ -2482,24 +2486,6 @@ int kprobe_add_area_blacklist(unsigned long start, unsigned long end)
>   	return 0;
>   }
>   
> -/* Remove all symbols in given area from kprobe blacklist */
> -static void kprobe_remove_area_blacklist(unsigned long start, unsigned long end)
> -{
> -	struct kprobe_blacklist_entry *ent, *n;
> -
> -	list_for_each_entry_safe(ent, n, &kprobe_blacklist, list) {
> -		if (ent->start_addr < start || ent->start_addr >= end)
> -			continue;
> -		list_del(&ent->list);
> -		kfree(ent);
> -	}
> -}
> -
> -static void kprobe_remove_ksym_blacklist(unsigned long entry)
> -{
> -	kprobe_remove_area_blacklist(entry, entry + 1);
> -}
> -
>   int __weak arch_kprobe_get_kallsym(unsigned int *symnum, unsigned long *value,
>   				   char *type, char *sym)
>   {
> @@ -2564,6 +2550,25 @@ static int __init populate_kprobe_blacklist(unsigned long *start,
>   	return ret ? : arch_populate_kprobe_blacklist();
>   }
>   
> +#ifdef CONFIG_MODULES
> +/* Remove all symbols in given area from kprobe blacklist */
> +static void kprobe_remove_area_blacklist(unsigned long start, unsigned long end)
> +{
> +	struct kprobe_blacklist_entry *ent, *n;
> +
> +	list_for_each_entry_safe(ent, n, &kprobe_blacklist, list) {
> +		if (ent->start_addr < start || ent->start_addr >= end)
> +			continue;
> +		list_del(&ent->list);
> +		kfree(ent);
> +	}
> +}
> +
> +static void kprobe_remove_ksym_blacklist(unsigned long entry)
> +{
> +	kprobe_remove_area_blacklist(entry, entry + 1);
> +}
> +
>   static void add_module_kprobe_blacklist(struct module *mod)
>   {
>   	unsigned long start, end;
> @@ -2660,6 +2665,9 @@ static int kprobes_module_callback(struct notifier_block *nb,
>   	mutex_unlock(&kprobe_mutex);
>   	return NOTIFY_DONE;
>   }
> +#else
> +#define kprobes_module_callback	(NULL)

Can you use a static inline instead ? It allows typechecking at build.

> +#endif /* CONFIG_MODULES */
>   
>   static struct notifier_block kprobe_module_nb = {
>   	.notifier_call = kprobes_module_callback,
> @@ -2724,7 +2732,8 @@ static int __init init_kprobes(void)
>   	err = arch_init_kprobes();
>   	if (!err)
>   		err = register_die_notifier(&kprobe_exceptions_nb);
> -	if (!err)
> +
> +	if (!err && IS_ENABLED(CONFIG_MODULES))
>   		err = register_module_notifier(&kprobe_module_nb);
>   
>   	kprobes_initialized = (err == 0);
> diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
> index c4c6e0e0068b..13ea0d0994d5 100644
> --- a/kernel/trace/trace_kprobe.c
> +++ b/kernel/trace/trace_kprobe.c
> @@ -111,6 +111,7 @@ static nokprobe_inline bool trace_kprobe_within_module(struct trace_kprobe *tk,
>   	return strncmp(module_name(mod), name, len) == 0 && name[len] == ':';
>   }
>   
> +#ifdef CONFIG_MODULES
>   static nokprobe_inline bool trace_kprobe_module_exist(struct trace_kprobe *tk)
>   {
>   	char *p;
> @@ -129,6 +130,9 @@ static nokprobe_inline bool trace_kprobe_module_exist(struct trace_kprobe *tk)
>   
>   	return ret;
>   }
> +#else
> +#define trace_kprobe_module_exist(tk) false /* aka a module never exists */

same, can it be a static inline instead ?

> +#endif /* CONFIG_MODULES */
>   
>   static bool trace_kprobe_is_busy(struct dyn_event *ev)
>   {
> @@ -670,6 +674,7 @@ static int register_trace_kprobe(struct trace_kprobe *tk)
>   	return ret;
>   }
>   
> +#ifdef CONFIG_MODULES
>   /* Module notifier call back, checking event on the module */
>   static int trace_kprobe_module_callback(struct notifier_block *nb,
>   				       unsigned long val, void *data)
> @@ -699,6 +704,9 @@ static int trace_kprobe_module_callback(struct notifier_block *nb,
>   
>   	return NOTIFY_DONE;
>   }
> +#else
> +#define trace_kprobe_module_callback (NULL)

Same

> +#endif /* CONFIG_MODULES */
>   
>   static struct notifier_block trace_kprobe_module_nb = {
>   	.notifier_call = trace_kprobe_module_callback,
> @@ -1897,8 +1905,11 @@ static __init int init_kprobe_trace_early(void)
>   	if (ret)
>   		return ret;
>   
> -	if (register_module_notifier(&trace_kprobe_module_nb))
> -		return -EINVAL;
> +	if (IS_ENABLED(CONFIG_MODULES)) {
> +		ret = register_module_notifier(&trace_kprobe_module_nb);
> +		if (ret)
> +			return -EINVAL;
> +	}

No need for that, register_module_notifier() returns 0 when 
CONFIG_MODULES is not set.

>   
>   	return 0;
>   }
diff mbox series

Patch

diff --git a/arch/Kconfig b/arch/Kconfig
index a5af0edd3eb8..5e9735f60f3c 100644
--- a/arch/Kconfig
+++ b/arch/Kconfig
@@ -52,8 +52,8 @@  config GENERIC_ENTRY
 
 config KPROBES
 	bool "Kprobes"
-	depends on MODULES
 	depends on HAVE_KPROBES
+	depends on ALLOC_EXECMEM
 	select KALLSYMS
 	select TASKS_RCU if PREEMPTION
 	help
@@ -215,6 +215,21 @@  config HAVE_OPTPROBES
 config HAVE_KPROBES_ON_FTRACE
 	bool
 
+config HAVE_ALLOC_EXECMEM
+	bool
+	help
+	  Architectures that select this option are capable of allocating trampoline
+	  executable memory for tracing subsystems, indepedently of the kernel module
+	  subsystem.
+
+config ALLOC_EXECMEM
+	bool "Executable (trampoline) memory allocation"
+	default y
+	depends on MODULES || HAVE_ALLOC_EXECMEM
+	help
+	  Select this for executable (trampoline) memory. Can be enabled when either
+	  module allocator or arch-specific allocator is available.
+
 config ARCH_CORRECT_STACKTRACE_ON_KRETPROBE
 	bool
 	help
diff --git a/include/linux/execmem.h b/include/linux/execmem.h
new file mode 100644
index 000000000000..ae2ff151523a
--- /dev/null
+++ b/include/linux/execmem.h
@@ -0,0 +1,13 @@ 
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_EXECMEM_H
+#define _LINUX_EXECMEM_H
+
+#ifdef CONFIG_HAVE_ALLOC_EXECMEM
+void *alloc_execmem(unsigned long size, gfp_t gfp);
+void free_execmem(void *region);
+#else
+#define alloc_execmem(size, gfp)	module_alloc(size)
+#define free_execmem(region)		module_memfree(region)
+#endif
+
+#endif /* _LINUX_EXECMEM_H */
diff --git a/kernel/kprobes.c b/kernel/kprobes.c
index 9d9095e81792..13bef5de315c 100644
--- a/kernel/kprobes.c
+++ b/kernel/kprobes.c
@@ -44,6 +44,7 @@ 
 #include <asm/cacheflush.h>
 #include <asm/errno.h>
 #include <linux/uaccess.h>
+#include <linux/execmem.h>
 
 #define KPROBE_HASH_BITS 6
 #define KPROBE_TABLE_SIZE (1 << KPROBE_HASH_BITS)
@@ -113,17 +114,17 @@  enum kprobe_slot_state {
 void __weak *alloc_insn_page(void)
 {
 	/*
-	 * Use module_alloc() so this page is within +/- 2GB of where the
+	 * Use alloc_execmem() so this page is within +/- 2GB of where the
 	 * kernel image and loaded module images reside. This is required
 	 * for most of the architectures.
 	 * (e.g. x86-64 needs this to handle the %rip-relative fixups.)
 	 */
-	return module_alloc(PAGE_SIZE);
+	return alloc_execmem(PAGE_SIZE, GFP_KERNEL);
 }
 
 static void free_insn_page(void *page)
 {
-	module_memfree(page);
+	free_execmem(page);
 }
 
 struct kprobe_insn_cache kprobe_insn_slots = {
@@ -1592,6 +1593,7 @@  static int check_kprobe_address_safe(struct kprobe *p,
 			goto out;
 		}
 
+#ifdef CONFIG_MODULES
 		/*
 		 * If the module freed '.init.text', we couldn't insert
 		 * kprobes in there.
@@ -1602,7 +1604,9 @@  static int check_kprobe_address_safe(struct kprobe *p,
 			*probed_mod = NULL;
 			ret = -ENOENT;
 		}
+#endif /* CONFIG_MODULES */
 	}
+
 out:
 	preempt_enable();
 	jump_label_unlock();
@@ -2482,24 +2486,6 @@  int kprobe_add_area_blacklist(unsigned long start, unsigned long end)
 	return 0;
 }
 
-/* Remove all symbols in given area from kprobe blacklist */
-static void kprobe_remove_area_blacklist(unsigned long start, unsigned long end)
-{
-	struct kprobe_blacklist_entry *ent, *n;
-
-	list_for_each_entry_safe(ent, n, &kprobe_blacklist, list) {
-		if (ent->start_addr < start || ent->start_addr >= end)
-			continue;
-		list_del(&ent->list);
-		kfree(ent);
-	}
-}
-
-static void kprobe_remove_ksym_blacklist(unsigned long entry)
-{
-	kprobe_remove_area_blacklist(entry, entry + 1);
-}
-
 int __weak arch_kprobe_get_kallsym(unsigned int *symnum, unsigned long *value,
 				   char *type, char *sym)
 {
@@ -2564,6 +2550,25 @@  static int __init populate_kprobe_blacklist(unsigned long *start,
 	return ret ? : arch_populate_kprobe_blacklist();
 }
 
+#ifdef CONFIG_MODULES
+/* Remove all symbols in given area from kprobe blacklist */
+static void kprobe_remove_area_blacklist(unsigned long start, unsigned long end)
+{
+	struct kprobe_blacklist_entry *ent, *n;
+
+	list_for_each_entry_safe(ent, n, &kprobe_blacklist, list) {
+		if (ent->start_addr < start || ent->start_addr >= end)
+			continue;
+		list_del(&ent->list);
+		kfree(ent);
+	}
+}
+
+static void kprobe_remove_ksym_blacklist(unsigned long entry)
+{
+	kprobe_remove_area_blacklist(entry, entry + 1);
+}
+
 static void add_module_kprobe_blacklist(struct module *mod)
 {
 	unsigned long start, end;
@@ -2660,6 +2665,9 @@  static int kprobes_module_callback(struct notifier_block *nb,
 	mutex_unlock(&kprobe_mutex);
 	return NOTIFY_DONE;
 }
+#else
+#define kprobes_module_callback	(NULL)
+#endif /* CONFIG_MODULES */
 
 static struct notifier_block kprobe_module_nb = {
 	.notifier_call = kprobes_module_callback,
@@ -2724,7 +2732,8 @@  static int __init init_kprobes(void)
 	err = arch_init_kprobes();
 	if (!err)
 		err = register_die_notifier(&kprobe_exceptions_nb);
-	if (!err)
+
+	if (!err && IS_ENABLED(CONFIG_MODULES))
 		err = register_module_notifier(&kprobe_module_nb);
 
 	kprobes_initialized = (err == 0);
diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c
index c4c6e0e0068b..13ea0d0994d5 100644
--- a/kernel/trace/trace_kprobe.c
+++ b/kernel/trace/trace_kprobe.c
@@ -111,6 +111,7 @@  static nokprobe_inline bool trace_kprobe_within_module(struct trace_kprobe *tk,
 	return strncmp(module_name(mod), name, len) == 0 && name[len] == ':';
 }
 
+#ifdef CONFIG_MODULES
 static nokprobe_inline bool trace_kprobe_module_exist(struct trace_kprobe *tk)
 {
 	char *p;
@@ -129,6 +130,9 @@  static nokprobe_inline bool trace_kprobe_module_exist(struct trace_kprobe *tk)
 
 	return ret;
 }
+#else
+#define trace_kprobe_module_exist(tk) false /* aka a module never exists */
+#endif /* CONFIG_MODULES */
 
 static bool trace_kprobe_is_busy(struct dyn_event *ev)
 {
@@ -670,6 +674,7 @@  static int register_trace_kprobe(struct trace_kprobe *tk)
 	return ret;
 }
 
+#ifdef CONFIG_MODULES
 /* Module notifier call back, checking event on the module */
 static int trace_kprobe_module_callback(struct notifier_block *nb,
 				       unsigned long val, void *data)
@@ -699,6 +704,9 @@  static int trace_kprobe_module_callback(struct notifier_block *nb,
 
 	return NOTIFY_DONE;
 }
+#else
+#define trace_kprobe_module_callback (NULL)
+#endif /* CONFIG_MODULES */
 
 static struct notifier_block trace_kprobe_module_nb = {
 	.notifier_call = trace_kprobe_module_callback,
@@ -1897,8 +1905,11 @@  static __init int init_kprobe_trace_early(void)
 	if (ret)
 		return ret;
 
-	if (register_module_notifier(&trace_kprobe_module_nb))
-		return -EINVAL;
+	if (IS_ENABLED(CONFIG_MODULES)) {
+		ret = register_module_notifier(&trace_kprobe_module_nb);
+		if (ret)
+			return -EINVAL;
+	}
 
 	return 0;
 }