| Message ID | 155544804466.1032396.13418949511615676665.stgit@dwillia2-desk3.amr.corp.intel.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | init: Initialize jump labels before command line option parsing | expand |
On Tue, 16 Apr 2019 13:54:04 -0700 Dan Williams <dan.j.williams@intel.com> wrote: > When a module option, or core kernel argument, toggles a static-key it > requires jump labels to be initialized early. While x86, PowerPC, and > ARM64 arrange for jump_label_init() to be called before parse_args(), > ARM does not. > > Kernel command line: rdinit=/sbin/init page_alloc.shuffle=1 panic=-1 console=ttyAMA0,115200 page_alloc.shuffle=1 > ------------[ cut here ]------------ > WARNING: CPU: 0 PID: 0 at ./include/linux/jump_label.h:303 > page_alloc_shuffle+0x12c/0x1ac > static_key_enable(): static key 'page_alloc_shuffle_key+0x0/0x4' used > before call to jump_label_init() > Modules linked in: > CPU: 0 PID: 0 Comm: swapper Not tainted > 5.1.0-rc4-next-20190410-00003-g3367c36ce744 #1 > Hardware name: ARM Integrator/CP (Device Tree) > [<c0011c68>] (unwind_backtrace) from [<c000ec48>] (show_stack+0x10/0x18) > [<c000ec48>] (show_stack) from [<c07e9710>] (dump_stack+0x18/0x24) > [<c07e9710>] (dump_stack) from [<c001bb1c>] (__warn+0xe0/0x108) > [<c001bb1c>] (__warn) from [<c001bb88>] (warn_slowpath_fmt+0x44/0x6c) > [<c001bb88>] (warn_slowpath_fmt) from [<c0b0c4a8>] > (page_alloc_shuffle+0x12c/0x1ac) > [<c0b0c4a8>] (page_alloc_shuffle) from [<c0b0c550>] (shuffle_store+0x28/0x48) > [<c0b0c550>] (shuffle_store) from [<c003e6a0>] (parse_args+0x1f4/0x350) > [<c003e6a0>] (parse_args) from [<c0ac3c00>] (start_kernel+0x1c0/0x488) > > Move the fallback call to jump_label_init() to occur before > parse_args(). The redundant calls to jump_label_init() in other archs > are left intact in case they have static key toggling use cases that are > even earlier than option parsing. Has it been confirmed that this fixes mm-shuffle-initial-free-memory-to-improve-memory-side-cache-utilization.patch on beaglebone-black?
On Tue, Apr 16, 2019 at 4:44 PM Andrew Morton <akpm@linux-foundation.org> wrote: > > On Tue, 16 Apr 2019 13:54:04 -0700 Dan Williams <dan.j.williams@intel.com> wrote: > > > When a module option, or core kernel argument, toggles a static-key it > > requires jump labels to be initialized early. While x86, PowerPC, and > > ARM64 arrange for jump_label_init() to be called before parse_args(), > > ARM does not. > > > > Kernel command line: rdinit=/sbin/init page_alloc.shuffle=1 panic=-1 console=ttyAMA0,115200 page_alloc.shuffle=1 > > ------------[ cut here ]------------ > > WARNING: CPU: 0 PID: 0 at ./include/linux/jump_label.h:303 > > page_alloc_shuffle+0x12c/0x1ac > > static_key_enable(): static key 'page_alloc_shuffle_key+0x0/0x4' used > > before call to jump_label_init() > > Modules linked in: > > CPU: 0 PID: 0 Comm: swapper Not tainted > > 5.1.0-rc4-next-20190410-00003-g3367c36ce744 #1 > > Hardware name: ARM Integrator/CP (Device Tree) > > [<c0011c68>] (unwind_backtrace) from [<c000ec48>] (show_stack+0x10/0x18) > > [<c000ec48>] (show_stack) from [<c07e9710>] (dump_stack+0x18/0x24) > > [<c07e9710>] (dump_stack) from [<c001bb1c>] (__warn+0xe0/0x108) > > [<c001bb1c>] (__warn) from [<c001bb88>] (warn_slowpath_fmt+0x44/0x6c) > > [<c001bb88>] (warn_slowpath_fmt) from [<c0b0c4a8>] > > (page_alloc_shuffle+0x12c/0x1ac) > > [<c0b0c4a8>] (page_alloc_shuffle) from [<c0b0c550>] (shuffle_store+0x28/0x48) > > [<c0b0c550>] (shuffle_store) from [<c003e6a0>] (parse_args+0x1f4/0x350) > > [<c003e6a0>] (parse_args) from [<c0ac3c00>] (start_kernel+0x1c0/0x488) > > > > Move the fallback call to jump_label_init() to occur before > > parse_args(). The redundant calls to jump_label_init() in other archs > > are left intact in case they have static key toggling use cases that are > > even earlier than option parsing. > > Has it been confirmed that this fixes > mm-shuffle-initial-free-memory-to-improve-memory-side-cache-utilization.patch > on beaglebone-black? This only fixes dynamically enabling the shuffling on 32-bit ARM. Guenter happened to run without the mm-only 'force-enable-always' patch and when he went to use the command line option to enable it he hit the jump-label warning. The original beaglebone-black failure was something different and avoided this case because the jump-label was never used. I am otherwise unable to recreate the failure on either the original failing -next, nor on v5.1-rc5 plus the latest state of the patches. I need from someone who is actually still seeing the failure so they can compare with the configuration that is working for me. For reference that's the Yocto beaglebone-black defconfig: https://github.com/jumpnow/meta-bbb/blob/thud/recipes-kernel/linux/linux-stable-5.0/beaglebone/defconfig
On Tue, Apr 16, 2019 at 5:04 PM Dan Williams <dan.j.williams@intel.com> wrote: > > On Tue, Apr 16, 2019 at 4:44 PM Andrew Morton <akpm@linux-foundation.org> wrote: > > > > On Tue, 16 Apr 2019 13:54:04 -0700 Dan Williams <dan.j.williams@intel.com> wrote: > > > > > When a module option, or core kernel argument, toggles a static-key it > > > requires jump labels to be initialized early. While x86, PowerPC, and > > > ARM64 arrange for jump_label_init() to be called before parse_args(), > > > ARM does not. > > > > > > Kernel command line: rdinit=/sbin/init page_alloc.shuffle=1 panic=-1 console=ttyAMA0,115200 page_alloc.shuffle=1 > > > ------------[ cut here ]------------ > > > WARNING: CPU: 0 PID: 0 at ./include/linux/jump_label.h:303 > > > page_alloc_shuffle+0x12c/0x1ac > > > static_key_enable(): static key 'page_alloc_shuffle_key+0x0/0x4' used > > > before call to jump_label_init() > > > Modules linked in: > > > CPU: 0 PID: 0 Comm: swapper Not tainted > > > 5.1.0-rc4-next-20190410-00003-g3367c36ce744 #1 > > > Hardware name: ARM Integrator/CP (Device Tree) > > > [<c0011c68>] (unwind_backtrace) from [<c000ec48>] (show_stack+0x10/0x18) > > > [<c000ec48>] (show_stack) from [<c07e9710>] (dump_stack+0x18/0x24) > > > [<c07e9710>] (dump_stack) from [<c001bb1c>] (__warn+0xe0/0x108) > > > [<c001bb1c>] (__warn) from [<c001bb88>] (warn_slowpath_fmt+0x44/0x6c) > > > [<c001bb88>] (warn_slowpath_fmt) from [<c0b0c4a8>] > > > (page_alloc_shuffle+0x12c/0x1ac) > > > [<c0b0c4a8>] (page_alloc_shuffle) from [<c0b0c550>] (shuffle_store+0x28/0x48) > > > [<c0b0c550>] (shuffle_store) from [<c003e6a0>] (parse_args+0x1f4/0x350) > > > [<c003e6a0>] (parse_args) from [<c0ac3c00>] (start_kernel+0x1c0/0x488) > > > > > > Move the fallback call to jump_label_init() to occur before > > > parse_args(). The redundant calls to jump_label_init() in other archs > > > are left intact in case they have static key toggling use cases that are > > > even earlier than option parsing. > > > > Has it been confirmed that this fixes > > mm-shuffle-initial-free-memory-to-improve-memory-side-cache-utilization.patch > > on beaglebone-black? > > This only fixes dynamically enabling the shuffling on 32-bit ARM. > Guenter happened to run without the mm-only 'force-enable-always' > patch and when he went to use the command line option to enable it he > hit the jump-label warning. > For my part I have not seen the original failure; it seems that the kernelci logs are no longer present. As such, I neither know how it looks like nor how to (try to) reproduce it. I just thought it might be worthwhile to run the patch through my boot tests to see if anything pops up. From the feedback I got, though, it sounded like the failure is/was very omap2 specific, so I would not be able to reproduce it anyway. Guenter > The original beaglebone-black failure was something different and > avoided this case because the jump-label was never used. > > I am otherwise unable to recreate the failure on either the original > failing -next, nor on v5.1-rc5 plus the latest state of the patches. I > need from someone who is actually still seeing the failure so they can > compare with the configuration that is working for me. For reference > that's the Yocto beaglebone-black defconfig: > > https://github.com/jumpnow/meta-bbb/blob/thud/recipes-kernel/linux/linux-stable-5.0/beaglebone/defconfig
On Tue, 16 Apr 2019 17:39:03 -0700 Guenter Roeck <groeck@google.com> wrote: > > > Has it been confirmed that this fixes > > > mm-shuffle-initial-free-memory-to-improve-memory-side-cache-utilization.patch > > > on beaglebone-black? > > > > This only fixes dynamically enabling the shuffling on 32-bit ARM. > > Guenter happened to run without the mm-only 'force-enable-always' > > patch and when he went to use the command line option to enable it he > > hit the jump-label warning. > > > > For my part I have not seen the original failure; it seems that the > kernelci logs are no longer present. As such, I neither know how it > looks like nor how to (try to) reproduce it. I just thought it might > be worthwhile to run the patch through my boot tests to see if > anything pops up. From the feedback I got, though, it sounded like the > failure is/was very omap2 specific, so I would not be able to > reproduce it anyway. hm. Maybe we forge ahead and see if someone hits the issue who can work with us on resolving it. It sounds like the affected population will be quite small. But still, ugh :(
diff --git a/init/main.c b/init/main.c index 598e278b46f7..7d4025d665eb 100644 --- a/init/main.c +++ b/init/main.c @@ -582,6 +582,8 @@ asmlinkage __visible void __init start_kernel(void) page_alloc_init(); pr_notice("Kernel command line: %s\n", boot_command_line); + /* parameters may set static keys */ + jump_label_init(); parse_early_param(); after_dashes = parse_args("Booting kernel", static_command_line, __start___param, @@ -591,8 +593,6 @@ asmlinkage __visible void __init start_kernel(void) parse_args("Setting init args", after_dashes, NULL, 0, -1, -1, NULL, set_init_arg); - jump_label_init(); - /* * These use large bootmem allocations and must precede * kmem_cache_init()
When a module option, or core kernel argument, toggles a static-key it requires jump labels to be initialized early. While x86, PowerPC, and ARM64 arrange for jump_label_init() to be called before parse_args(), ARM does not. Kernel command line: rdinit=/sbin/init page_alloc.shuffle=1 panic=-1 console=ttyAMA0,115200 page_alloc.shuffle=1 ------------[ cut here ]------------ WARNING: CPU: 0 PID: 0 at ./include/linux/jump_label.h:303 page_alloc_shuffle+0x12c/0x1ac static_key_enable(): static key 'page_alloc_shuffle_key+0x0/0x4' used before call to jump_label_init() Modules linked in: CPU: 0 PID: 0 Comm: swapper Not tainted 5.1.0-rc4-next-20190410-00003-g3367c36ce744 #1 Hardware name: ARM Integrator/CP (Device Tree) [<c0011c68>] (unwind_backtrace) from [<c000ec48>] (show_stack+0x10/0x18) [<c000ec48>] (show_stack) from [<c07e9710>] (dump_stack+0x18/0x24) [<c07e9710>] (dump_stack) from [<c001bb1c>] (__warn+0xe0/0x108) [<c001bb1c>] (__warn) from [<c001bb88>] (warn_slowpath_fmt+0x44/0x6c) [<c001bb88>] (warn_slowpath_fmt) from [<c0b0c4a8>] (page_alloc_shuffle+0x12c/0x1ac) [<c0b0c4a8>] (page_alloc_shuffle) from [<c0b0c550>] (shuffle_store+0x28/0x48) [<c0b0c550>] (shuffle_store) from [<c003e6a0>] (parse_args+0x1f4/0x350) [<c003e6a0>] (parse_args) from [<c0ac3c00>] (start_kernel+0x1c0/0x488) Move the fallback call to jump_label_init() to occur before parse_args(). The redundant calls to jump_label_init() in other archs are left intact in case they have static key toggling use cases that are even earlier than option parsing. Reported-by: Guenter Roeck <groeck@google.com> Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> Cc: Andrew Morton <akpm@linux-foundation.org> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: Mike Rapoport <rppt@linux.ibm.com> Signed-off-by: Dan Williams <dan.j.williams@intel.com> --- init/main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)