| Message ID | 20250212162151.1599059-1-longman@redhat.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | [v2] kasan: Don't call find_vm_area() in RT kernel | expand |
On 2/12/25 5:21 PM, Waiman Long wrote: > The following bug report appeared with a test run in a RT debug kernel. > > [ 3359.353842] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48 > [ 3359.353848] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 140605, name: kunit_try_catch > [ 3359.353853] preempt_count: 1, expected: 0 > : > [ 3359.353933] Call trace: > : > [ 3359.353955] rt_spin_lock+0x70/0x140 > [ 3359.353959] find_vmap_area+0x84/0x168 > [ 3359.353963] find_vm_area+0x1c/0x50 > [ 3359.353966] print_address_description.constprop.0+0x2a0/0x320 > [ 3359.353972] print_report+0x108/0x1f8 > [ 3359.353976] kasan_report+0x90/0xc8 > [ 3359.353980] __asan_load1+0x60/0x70 > > Commit e30a0361b851 ("kasan: make report_lock a raw spinlock") > changes report_lock to a raw_spinlock_t to avoid a similar RT problem. > The print_address_description() function is called with report_lock > acquired and interrupt disabled. However, the find_vm_area() function > still needs to acquire a spinlock_t which becomes a sleeping lock in > the RT kernel. IOW, we can't call find_vm_area() in a RT kernel and > changing report_lock to a raw_spinlock_t is not enough to completely > solve this RT kernel problem. > > Fix this bug report by skipping the find_vm_area() call in this case > and just print out the address as is. > > For !RT kernel, follow the example set in commit 0cce06ba859a > ("debugobjects,locking: Annotate debug_object_fill_pool() wait type > violation") and use DEFINE_WAIT_OVERRIDE_MAP() to avoid a spinlock_t > inside raw_spinlock_t warning. > > Fixes: e30a0361b851 ("kasan: make report_lock a raw spinlock") > Signed-off-by: Waiman Long <longman@redhat.com> > --- > mm/kasan/report.c | 47 ++++++++++++++++++++++++++++++++++------------- > 1 file changed, 34 insertions(+), 13 deletions(-) > > [v2] Encapsulate the change into a new > kasan_print_vmalloc_info_ret_page() helper > Not exactly what I had i mind, but this way is fine too. Acked-by: Andrey Ryabinin <ryabinin.a.a@gmail.com>
On Wed, Feb 12, 2025 at 5:22 PM Waiman Long <longman@redhat.com> wrote: > > The following bug report appeared with a test run in a RT debug kernel. > > [ 3359.353842] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48 > [ 3359.353848] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 140605, name: kunit_try_catch > [ 3359.353853] preempt_count: 1, expected: 0 > : > [ 3359.353933] Call trace: > : > [ 3359.353955] rt_spin_lock+0x70/0x140 > [ 3359.353959] find_vmap_area+0x84/0x168 > [ 3359.353963] find_vm_area+0x1c/0x50 > [ 3359.353966] print_address_description.constprop.0+0x2a0/0x320 > [ 3359.353972] print_report+0x108/0x1f8 > [ 3359.353976] kasan_report+0x90/0xc8 > [ 3359.353980] __asan_load1+0x60/0x70 > > Commit e30a0361b851 ("kasan: make report_lock a raw spinlock") > changes report_lock to a raw_spinlock_t to avoid a similar RT problem. > The print_address_description() function is called with report_lock > acquired and interrupt disabled. However, the find_vm_area() function > still needs to acquire a spinlock_t which becomes a sleeping lock in > the RT kernel. IOW, we can't call find_vm_area() in a RT kernel and > changing report_lock to a raw_spinlock_t is not enough to completely > solve this RT kernel problem. > > Fix this bug report by skipping the find_vm_area() call in this case > and just print out the address as is. > > For !RT kernel, follow the example set in commit 0cce06ba859a > ("debugobjects,locking: Annotate debug_object_fill_pool() wait type > violation") and use DEFINE_WAIT_OVERRIDE_MAP() to avoid a spinlock_t > inside raw_spinlock_t warning. Would it be possible to get lockdep to allow taking spinlock_t inside raw_spinlock_t instead of annotating the callers for the !RT case? Or is this a rare thing for this to be allowed on !RT? > > Fixes: e30a0361b851 ("kasan: make report_lock a raw spinlock") > Signed-off-by: Waiman Long <longman@redhat.com> > --- > mm/kasan/report.c | 47 ++++++++++++++++++++++++++++++++++------------- > 1 file changed, 34 insertions(+), 13 deletions(-) > > [v2] Encapsulate the change into a new > kasan_print_vmalloc_info_ret_page() helper > > diff --git a/mm/kasan/report.c b/mm/kasan/report.c > index 3fe77a360f1c..9580ac3f3203 100644 > --- a/mm/kasan/report.c > +++ b/mm/kasan/report.c > @@ -370,6 +370,38 @@ static inline bool init_task_stack_addr(const void *addr) > sizeof(init_thread_union.stack)); > } > > +/* > + * RT kernel cannot call find_vm_area() in atomic context. For !RT kernel, > + * prevent spinlock_t inside raw_spinlock_t warning by raising wait-type > + * to WAIT_SLEEP. > + * > + * Return: page pointer or NULL > + */ > +static inline struct page *kasan_print_vmalloc_info_ret_page(void *addr) No need for the kasan_ prefix: this is a static function. (Also the _ret_* suffix is something I've never seen before in the kernel context, but I don't mind it.) > +{ > + if (!IS_ENABLED(CONFIG_PREEMPT_RT)) { > + static DEFINE_WAIT_OVERRIDE_MAP(vmalloc_map, LD_WAIT_SLEEP); > + struct page *page = NULL; > + struct vm_struct *va; > + > + lock_map_acquire_try(&vmalloc_map); > + va = find_vm_area(addr); > + if (va) { > + pr_err("The buggy address belongs to the virtual mapping at\n" > + " [%px, %px) created by:\n" > + " %pS\n", > + va->addr, va->addr + va->size, va->caller); > + pr_err("\n"); > + > + page = vmalloc_to_page(addr); > + } > + lock_map_release(&vmalloc_map); > + return page; > + } > + pr_err("The buggy address %px belongs to a vmalloc virtual mapping\n", addr); > + return NULL; > +} > + > static void print_address_description(void *addr, u8 tag, > struct kasan_report_info *info) > { > @@ -398,19 +430,8 @@ static void print_address_description(void *addr, u8 tag, > pr_err("\n"); > } > > - if (is_vmalloc_addr(addr)) { > - struct vm_struct *va = find_vm_area(addr); > - > - if (va) { > - pr_err("The buggy address belongs to the virtual mapping at\n" > - " [%px, %px) created by:\n" > - " %pS\n", > - va->addr, va->addr + va->size, va->caller); > - pr_err("\n"); > - > - page = vmalloc_to_page(addr); > - } > - } > + if (is_vmalloc_addr(addr)) > + page = kasan_print_vmalloc_info_ret_page(addr); > > if (page) { > pr_err("The buggy address belongs to the physical page:\n"); > -- > 2.48.1 >
On 2/12/25 8:48 PM, Andrey Konovalov wrote: > On Wed, Feb 12, 2025 at 5:22 PM Waiman Long <longman@redhat.com> wrote: >> The following bug report appeared with a test run in a RT debug kernel. >> >> [ 3359.353842] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48 >> [ 3359.353848] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 140605, name: kunit_try_catch >> [ 3359.353853] preempt_count: 1, expected: 0 >> : >> [ 3359.353933] Call trace: >> : >> [ 3359.353955] rt_spin_lock+0x70/0x140 >> [ 3359.353959] find_vmap_area+0x84/0x168 >> [ 3359.353963] find_vm_area+0x1c/0x50 >> [ 3359.353966] print_address_description.constprop.0+0x2a0/0x320 >> [ 3359.353972] print_report+0x108/0x1f8 >> [ 3359.353976] kasan_report+0x90/0xc8 >> [ 3359.353980] __asan_load1+0x60/0x70 >> >> Commit e30a0361b851 ("kasan: make report_lock a raw spinlock") >> changes report_lock to a raw_spinlock_t to avoid a similar RT problem. >> The print_address_description() function is called with report_lock >> acquired and interrupt disabled. However, the find_vm_area() function >> still needs to acquire a spinlock_t which becomes a sleeping lock in >> the RT kernel. IOW, we can't call find_vm_area() in a RT kernel and >> changing report_lock to a raw_spinlock_t is not enough to completely >> solve this RT kernel problem. >> >> Fix this bug report by skipping the find_vm_area() call in this case >> and just print out the address as is. >> >> For !RT kernel, follow the example set in commit 0cce06ba859a >> ("debugobjects,locking: Annotate debug_object_fill_pool() wait type >> violation") and use DEFINE_WAIT_OVERRIDE_MAP() to avoid a spinlock_t >> inside raw_spinlock_t warning. > Would it be possible to get lockdep to allow taking spinlock_t inside > raw_spinlock_t instead of annotating the callers for the !RT case? Or > is this a rare thing for this to be allowed on !RT? Lockdep currently issues warnings for taking spinlock_t inside raw_spinlock_t because it is not allowed in RT. Test coverage of RT kernels is likely less than !RT kernel and so less bug of this kind will be caught. By making !RT doing the same check, we increase coverage. However, we do allow override in the !RT case, but it has to be done on a case-by-case basis. Currently we only do that for debugging code, not the code that will be used in production kernel yet. Cheers, Longman
On 2/12/25 8:48 PM, Andrey Konovalov wrote: > On Wed, Feb 12, 2025 at 5:22 PM Waiman Long <longman@redhat.com> wrote: >> The following bug report appeared with a test run in a RT debug kernel. >> >> [ 3359.353842] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48 >> [ 3359.353848] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 140605, name: kunit_try_catch >> [ 3359.353853] preempt_count: 1, expected: 0 >> : >> [ 3359.353933] Call trace: >> : >> [ 3359.353955] rt_spin_lock+0x70/0x140 >> [ 3359.353959] find_vmap_area+0x84/0x168 >> [ 3359.353963] find_vm_area+0x1c/0x50 >> [ 3359.353966] print_address_description.constprop.0+0x2a0/0x320 >> [ 3359.353972] print_report+0x108/0x1f8 >> [ 3359.353976] kasan_report+0x90/0xc8 >> [ 3359.353980] __asan_load1+0x60/0x70 >> >> Commit e30a0361b851 ("kasan: make report_lock a raw spinlock") >> changes report_lock to a raw_spinlock_t to avoid a similar RT problem. >> The print_address_description() function is called with report_lock >> acquired and interrupt disabled. However, the find_vm_area() function >> still needs to acquire a spinlock_t which becomes a sleeping lock in >> the RT kernel. IOW, we can't call find_vm_area() in a RT kernel and >> changing report_lock to a raw_spinlock_t is not enough to completely >> solve this RT kernel problem. >> >> Fix this bug report by skipping the find_vm_area() call in this case >> and just print out the address as is. >> >> For !RT kernel, follow the example set in commit 0cce06ba859a >> ("debugobjects,locking: Annotate debug_object_fill_pool() wait type >> violation") and use DEFINE_WAIT_OVERRIDE_MAP() to avoid a spinlock_t >> inside raw_spinlock_t warning. > Would it be possible to get lockdep to allow taking spinlock_t inside > raw_spinlock_t instead of annotating the callers for the !RT case? Or > is this a rare thing for this to be allowed on !RT? > >> Fixes: e30a0361b851 ("kasan: make report_lock a raw spinlock") >> Signed-off-by: Waiman Long <longman@redhat.com> >> --- >> mm/kasan/report.c | 47 ++++++++++++++++++++++++++++++++++------------- >> 1 file changed, 34 insertions(+), 13 deletions(-) >> >> [v2] Encapsulate the change into a new >> kasan_print_vmalloc_info_ret_page() helper >> >> diff --git a/mm/kasan/report.c b/mm/kasan/report.c >> index 3fe77a360f1c..9580ac3f3203 100644 >> --- a/mm/kasan/report.c >> +++ b/mm/kasan/report.c >> @@ -370,6 +370,38 @@ static inline bool init_task_stack_addr(const void *addr) >> sizeof(init_thread_union.stack)); >> } >> >> +/* >> + * RT kernel cannot call find_vm_area() in atomic context. For !RT kernel, >> + * prevent spinlock_t inside raw_spinlock_t warning by raising wait-type >> + * to WAIT_SLEEP. >> + * >> + * Return: page pointer or NULL >> + */ >> +static inline struct page *kasan_print_vmalloc_info_ret_page(void *addr) > No need for the kasan_ prefix: this is a static function. (Also the > _ret_* suffix is something I've never seen before in the kernel > context, but I don't mind it.) Sorry for missing that. Yes, I can remove the prefix. Will post a v3. Cheers, Longman
diff --git a/mm/kasan/report.c b/mm/kasan/report.c index 3fe77a360f1c..9580ac3f3203 100644 --- a/mm/kasan/report.c +++ b/mm/kasan/report.c @@ -370,6 +370,38 @@ static inline bool init_task_stack_addr(const void *addr) sizeof(init_thread_union.stack)); } +/* + * RT kernel cannot call find_vm_area() in atomic context. For !RT kernel, + * prevent spinlock_t inside raw_spinlock_t warning by raising wait-type + * to WAIT_SLEEP. + * + * Return: page pointer or NULL + */ +static inline struct page *kasan_print_vmalloc_info_ret_page(void *addr) +{ + if (!IS_ENABLED(CONFIG_PREEMPT_RT)) { + static DEFINE_WAIT_OVERRIDE_MAP(vmalloc_map, LD_WAIT_SLEEP); + struct page *page = NULL; + struct vm_struct *va; + + lock_map_acquire_try(&vmalloc_map); + va = find_vm_area(addr); + if (va) { + pr_err("The buggy address belongs to the virtual mapping at\n" + " [%px, %px) created by:\n" + " %pS\n", + va->addr, va->addr + va->size, va->caller); + pr_err("\n"); + + page = vmalloc_to_page(addr); + } + lock_map_release(&vmalloc_map); + return page; + } + pr_err("The buggy address %px belongs to a vmalloc virtual mapping\n", addr); + return NULL; +} + static void print_address_description(void *addr, u8 tag, struct kasan_report_info *info) { @@ -398,19 +430,8 @@ static void print_address_description(void *addr, u8 tag, pr_err("\n"); } - if (is_vmalloc_addr(addr)) { - struct vm_struct *va = find_vm_area(addr); - - if (va) { - pr_err("The buggy address belongs to the virtual mapping at\n" - " [%px, %px) created by:\n" - " %pS\n", - va->addr, va->addr + va->size, va->caller); - pr_err("\n"); - - page = vmalloc_to_page(addr); - } - } + if (is_vmalloc_addr(addr)) + page = kasan_print_vmalloc_info_ret_page(addr); if (page) { pr_err("The buggy address belongs to the physical page:\n");
The following bug report appeared with a test run in a RT debug kernel. [ 3359.353842] BUG: sleeping function called from invalid context at kernel/locking/spinlock_rt.c:48 [ 3359.353848] in_atomic(): 1, irqs_disabled(): 1, non_block: 0, pid: 140605, name: kunit_try_catch [ 3359.353853] preempt_count: 1, expected: 0 : [ 3359.353933] Call trace: : [ 3359.353955] rt_spin_lock+0x70/0x140 [ 3359.353959] find_vmap_area+0x84/0x168 [ 3359.353963] find_vm_area+0x1c/0x50 [ 3359.353966] print_address_description.constprop.0+0x2a0/0x320 [ 3359.353972] print_report+0x108/0x1f8 [ 3359.353976] kasan_report+0x90/0xc8 [ 3359.353980] __asan_load1+0x60/0x70 Commit e30a0361b851 ("kasan: make report_lock a raw spinlock") changes report_lock to a raw_spinlock_t to avoid a similar RT problem. The print_address_description() function is called with report_lock acquired and interrupt disabled. However, the find_vm_area() function still needs to acquire a spinlock_t which becomes a sleeping lock in the RT kernel. IOW, we can't call find_vm_area() in a RT kernel and changing report_lock to a raw_spinlock_t is not enough to completely solve this RT kernel problem. Fix this bug report by skipping the find_vm_area() call in this case and just print out the address as is. For !RT kernel, follow the example set in commit 0cce06ba859a ("debugobjects,locking: Annotate debug_object_fill_pool() wait type violation") and use DEFINE_WAIT_OVERRIDE_MAP() to avoid a spinlock_t inside raw_spinlock_t warning. Fixes: e30a0361b851 ("kasan: make report_lock a raw spinlock") Signed-off-by: Waiman Long <longman@redhat.com> --- mm/kasan/report.c | 47 ++++++++++++++++++++++++++++++++++------------- 1 file changed, 34 insertions(+), 13 deletions(-) [v2] Encapsulate the change into a new kasan_print_vmalloc_info_ret_page() helper