| Message ID | 20210727040021.21371-2-Kuan-Ying.Lee@mediatek.com (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | kasan, mm: reset tag when access metadata | expand |
+Cc Catalin On Tue, 27 Jul 2021 at 06:00, Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> wrote: > > Hardware tag-based KASAN doesn't use compiler instrumentation, we > can not use kasan_disable_current() to ignore tag check. > > Thus, we need to reset tags when accessing metadata. > > Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> This looks reasonable, but the patch title is not saying this is kmemleak, nor does the description say what the problem is. What problem did you encounter? Was it a false positive? Perhaps this should have been "kmemleak, kasan: reset pointer tags to avoid false positives" ? > --- > mm/kmemleak.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/mm/kmemleak.c b/mm/kmemleak.c > index 228a2fbe0657..73d46d16d575 100644 > --- a/mm/kmemleak.c > +++ b/mm/kmemleak.c > @@ -290,7 +290,7 @@ static void hex_dump_object(struct seq_file *seq, > warn_or_seq_printf(seq, " hex dump (first %zu bytes):\n", len); > kasan_disable_current(); > warn_or_seq_hex_dump(seq, DUMP_PREFIX_NONE, HEX_ROW_SIZE, > - HEX_GROUP_SIZE, ptr, len, HEX_ASCII); > + HEX_GROUP_SIZE, kasan_reset_tag((void *)ptr), len, HEX_ASCII); > kasan_enable_current(); > } > > @@ -1171,7 +1171,7 @@ static bool update_checksum(struct kmemleak_object *object) > > kasan_disable_current(); > kcsan_disable_current(); > - object->checksum = crc32(0, (void *)object->pointer, object->size); > + object->checksum = crc32(0, kasan_reset_tag((void *)object->pointer), object->size); > kasan_enable_current(); > kcsan_enable_current(); > > @@ -1246,7 +1246,7 @@ static void scan_block(void *_start, void *_end, > break; > > kasan_disable_current(); > - pointer = *ptr; > + pointer = *(unsigned long *)kasan_reset_tag((void *)ptr); > kasan_enable_current(); > > untagged_ptr = (unsigned long)kasan_reset_tag((void *)pointer); > -- > 2.18.0 > > -- > You received this message because you are subscribed to the Google Groups "kasan-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an email to kasan-dev+unsubscribe@googlegroups.com. > To view this discussion on the web visit https://groups.google.com/d/msgid/kasan-dev/20210727040021.21371-2-Kuan-Ying.Lee%40mediatek.com.
On Tue, 2021-07-27 at 09:10 +0200, Marco Elver wrote: > +Cc Catalin > > On Tue, 27 Jul 2021 at 06:00, Kuan-Ying Lee < > Kuan-Ying.Lee@mediatek.com> wrote: > > > > Hardware tag-based KASAN doesn't use compiler instrumentation, we > > can not use kasan_disable_current() to ignore tag check. > > > > Thus, we need to reset tags when accessing metadata. > > > > Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> > > This looks reasonable, but the patch title is not saying this is > kmemleak, nor does the description say what the problem is. What > problem did you encounter? Was it a false positive? kmemleak would scan kernel memory to check memory leak. When it scans on the invalid slab and dereference, the issue will occur like below. So I think we should reset the tag before scanning. # echo scan > /sys/kernel/debug/kmemleak [ 151.905804] ================================================================== [ 151.907120] BUG: KASAN: out-of-bounds in scan_block+0x58/0x170 [ 151.908773] Read at addr f7ff0000c0074eb0 by task kmemleak/138 [ 151.909656] Pointer tag: [f7], memory tag: [fe] [ 151.910195] [ 151.910876] CPU: 7 PID: 138 Comm: kmemleak Not tainted 5.14.0-rc2- 00001-g8cae8cd89f05-dirty #134 [ 151.912085] Hardware name: linux,dummy-virt (DT) [ 151.912868] Call trace: [ 151.913211] dump_backtrace+0x0/0x1b0 [ 151.913796] show_stack+0x1c/0x30 [ 151.914248] dump_stack_lvl+0x68/0x84 [ 151.914778] print_address_description+0x7c/0x2b4 [ 151.915340] kasan_report+0x138/0x38c [ 151.915804] __do_kernel_fault+0x190/0x1c4 [ 151.916386] do_tag_check_fault+0x78/0x90 [ 151.916856] do_mem_abort+0x44/0xb4 [ 151.917308] el1_abort+0x40/0x60 [ 151.917754] el1h_64_sync_handler+0xb4/0xd0 [ 151.918270] el1h_64_sync+0x78/0x7c [ 151.918714] scan_block+0x58/0x170 [ 151.919157] scan_gray_list+0xdc/0x1a0 [ 151.919626] kmemleak_scan+0x2ac/0x560 [ 151.920129] kmemleak_scan_thread+0xb0/0xe0 [ 151.920635] kthread+0x154/0x160 [ 151.921115] ret_from_fork+0x10/0x18 [ 151.921717] [ 151.922077] Allocated by task 0: [ 151.922523] kasan_save_stack+0x2c/0x60 [ 151.923099] __kasan_kmalloc+0xec/0x104 [ 151.923502] __kmalloc+0x224/0x3c4 [ 151.924172] __register_sysctl_paths+0x200/0x290 [ 151.924709] register_sysctl_table+0x2c/0x40 [ 151.925175] sysctl_init+0x20/0x34 [ 151.925665] proc_sys_init+0x3c/0x48 [ 151.926136] proc_root_init+0x80/0x9c [ 151.926547] start_kernel+0x648/0x6a4 [ 151.926987] __primary_switched+0xc0/0xc8 [ 151.927557] [ 151.927994] Freed by task 0: [ 151.928340] kasan_save_stack+0x2c/0x60 [ 151.928766] kasan_set_track+0x2c/0x40 [ 151.929173] kasan_set_free_info+0x44/0x54 [ 151.929568] ____kasan_slab_free.constprop.0+0x150/0x1b0 [ 151.930063] __kasan_slab_free+0x14/0x20 [ 151.930449] slab_free_freelist_hook+0xa4/0x1fc [ 151.930924] kfree+0x1e8/0x30c [ 151.931285] put_fs_context+0x124/0x220 [ 151.931731] vfs_kern_mount.part.0+0x60/0xd4 [ 151.932280] kern_mount+0x24/0x4c [ 151.932686] bdev_cache_init+0x70/0x9c [ 151.933122] vfs_caches_init+0xdc/0xf4 [ 151.933578] start_kernel+0x638/0x6a4 [ 151.934014] __primary_switched+0xc0/0xc8 [ 151.934478] [ 151.934757] The buggy address belongs to the object at ffff0000c0074e00 [ 151.934757] which belongs to the cache kmalloc-256 of size 256 [ 151.935744] The buggy address is located 176 bytes inside of [ 151.935744] 256-byte region [ffff0000c0074e00, ffff0000c0074f00) [ 151.936702] The buggy address belongs to the page: [ 151.937378] page:(____ptrval____) refcount:1 mapcount:0 mapping:0000000000000000 index:0x0 pfn:0x100074 [ 151.938682] head:(____ptrval____) order:2 compound_mapcount:0 compound_pincount:0 [ 151.939440] flags: 0xbfffc0000010200(slab|head|node=0|zone=2|lastcpupid=0xffff|kasantag=0x 0) [ 151.940886] raw: 0bfffc0000010200 0000000000000000 dead000000000122 f5ff0000c0002300 [ 151.941634] raw: 0000000000000000 0000000000200020 00000001ffffffff 0000000000000000 [ 151.942353] page dumped because: kasan: bad access detected [ 151.942923] [ 151.943214] Memory state around the buggy address: [ 151.943896] ffff0000c0074c00: f0 f0 f0 f0 f0 f0 f0 f0 f0 fe fe fe fe fe fe fe [ 151.944857] ffff0000c0074d00: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe [ 151.945892] >ffff0000c0074e00: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 fe fe fe fe fe [ 151.946407] ^ [ 151.946939] ffff0000c0074f00: fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe fe [ 151.947445] ffff0000c0075000: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb [ 151.947999] ================================================================== [ 151.948524] Disabling lock debugging due to kernel taint [ 156.434569] kmemleak: 181 new suspected memory leaks (see /sys/kernel/debug/kmemleak) > > Perhaps this should have been "kmemleak, kasan: reset pointer tags to > avoid false positives" ? Thanks for the suggestions. But I think it doesn't belong to false positive becuase scan block touched invalid metadata certainly. Maybe "kmemleak, kasan: reset tags when scanning block"? > > > --- > > mm/kmemleak.c | 6 +++--- > > 1 file changed, 3 insertions(+), 3 deletions(-) > > > > diff --git a/mm/kmemleak.c b/mm/kmemleak.c > > index 228a2fbe0657..73d46d16d575 100644 > > --- a/mm/kmemleak.c > > +++ b/mm/kmemleak.c > > @@ -290,7 +290,7 @@ static void hex_dump_object(struct seq_file > > *seq, > > warn_or_seq_printf(seq, " hex dump (first %zu bytes):\n", > > len); > > kasan_disable_current(); > > warn_or_seq_hex_dump(seq, DUMP_PREFIX_NONE, HEX_ROW_SIZE, > > - HEX_GROUP_SIZE, ptr, len, HEX_ASCII); > > + HEX_GROUP_SIZE, kasan_reset_tag((void > > *)ptr), len, HEX_ASCII); > > kasan_enable_current(); > > } > > > > @@ -1171,7 +1171,7 @@ static bool update_checksum(struct > > kmemleak_object *object) > > > > kasan_disable_current(); > > kcsan_disable_current(); > > - object->checksum = crc32(0, (void *)object->pointer, > > object->size); > > + object->checksum = crc32(0, kasan_reset_tag((void *)object- > > >pointer), object->size); > > kasan_enable_current(); > > kcsan_enable_current(); > > > > @@ -1246,7 +1246,7 @@ static void scan_block(void *_start, void > > *_end, > > break; > > > > kasan_disable_current(); > > - pointer = *ptr; > > + pointer = *(unsigned long *)kasan_reset_tag((void > > *)ptr); > > kasan_enable_current(); > > > > untagged_ptr = (unsigned long)kasan_reset_tag((void > > *)pointer); > > -- > > 2.18.0 > > > > -- > > You received this message because you are subscribed to the Google > > Groups "kasan-dev" group. > > To unsubscribe from this group and stop receiving emails from it, > > send an email to kasan-dev+unsubscribe@googlegroups.com. > > To view this discussion on the web visit > > https://urldefense.com/v3/__https://groups.google.com/d/msgid/kasan-dev/20210727040021.21371-2-Kuan-Ying.Lee*40mediatek.com__;JQ!!CTRNKA9wMg0ARbw!wNP4ZkYDM7Xvs9xfzKwYuG1X2h9zFqST8_Vm2jSvZUl9BiS8SPFMTvMp3VAPKCnuWELL7Q$ > > .
On Tue, 27 Jul 2021 at 10:32, Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> wrote: > > On Tue, 2021-07-27 at 09:10 +0200, Marco Elver wrote: > > +Cc Catalin > > > > On Tue, 27 Jul 2021 at 06:00, Kuan-Ying Lee < > > Kuan-Ying.Lee@mediatek.com> wrote: > > > > > > Hardware tag-based KASAN doesn't use compiler instrumentation, we > > > can not use kasan_disable_current() to ignore tag check. > > > > > > Thus, we need to reset tags when accessing metadata. > > > > > > Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> > > > > This looks reasonable, but the patch title is not saying this is > > kmemleak, nor does the description say what the problem is. What > > problem did you encounter? Was it a false positive? > > kmemleak would scan kernel memory to check memory leak. > When it scans on the invalid slab and dereference, the issue > will occur like below. Please also add this info to commit message. > So I think we should reset the tag before scanning. > > # echo scan > /sys/kernel/debug/kmemleak > [ 151.905804] > ================================================================== > [ 151.907120] BUG: KASAN: out-of-bounds in scan_block+0x58/0x170 > [ 151.908773] Read at addr f7ff0000c0074eb0 by task kmemleak/138 > [ 151.909656] Pointer tag: [f7], memory tag: [fe] > [ 151.910195] > [ 151.910876] CPU: 7 PID: 138 Comm: kmemleak Not tainted 5.14.0-rc2- > 00001-g8cae8cd89f05-dirty #134 > [ 151.912085] Hardware name: linux,dummy-virt (DT) > [ 151.912868] Call trace: > [ 151.913211] dump_backtrace+0x0/0x1b0 > [ 151.913796] show_stack+0x1c/0x30 > [ 151.914248] dump_stack_lvl+0x68/0x84 > [ 151.914778] print_address_description+0x7c/0x2b4 > [ 151.915340] kasan_report+0x138/0x38c > [ 151.915804] __do_kernel_fault+0x190/0x1c4 > [ 151.916386] do_tag_check_fault+0x78/0x90 > [ 151.916856] do_mem_abort+0x44/0xb4 > [ 151.917308] el1_abort+0x40/0x60 > [ 151.917754] el1h_64_sync_handler+0xb4/0xd0 > [ 151.918270] el1h_64_sync+0x78/0x7c > [ 151.918714] scan_block+0x58/0x170 > [ 151.919157] scan_gray_list+0xdc/0x1a0 > [ 151.919626] kmemleak_scan+0x2ac/0x560 > [ 151.920129] kmemleak_scan_thread+0xb0/0xe0 > [ 151.920635] kthread+0x154/0x160 > [ 151.921115] ret_from_fork+0x10/0x18 > [ 151.921717] > [ 151.922077] Allocated by task 0: > [ 151.922523] kasan_save_stack+0x2c/0x60 > [ 151.923099] __kasan_kmalloc+0xec/0x104 > [ 151.923502] __kmalloc+0x224/0x3c4 > [ 151.924172] __register_sysctl_paths+0x200/0x290 > [ 151.924709] register_sysctl_table+0x2c/0x40 > [ 151.925175] sysctl_init+0x20/0x34 > [ 151.925665] proc_sys_init+0x3c/0x48 > [ 151.926136] proc_root_init+0x80/0x9c > [ 151.926547] start_kernel+0x648/0x6a4 > [ 151.926987] __primary_switched+0xc0/0xc8 > [ 151.927557] > [ 151.927994] Freed by task 0: > [ 151.928340] kasan_save_stack+0x2c/0x60 > [ 151.928766] kasan_set_track+0x2c/0x40 > [ 151.929173] kasan_set_free_info+0x44/0x54 > [ 151.929568] ____kasan_slab_free.constprop.0+0x150/0x1b0 > [ 151.930063] __kasan_slab_free+0x14/0x20 > [ 151.930449] slab_free_freelist_hook+0xa4/0x1fc > [ 151.930924] kfree+0x1e8/0x30c > [ 151.931285] put_fs_context+0x124/0x220 > [ 151.931731] vfs_kern_mount.part.0+0x60/0xd4 > [ 151.932280] kern_mount+0x24/0x4c > [ 151.932686] bdev_cache_init+0x70/0x9c > [ 151.933122] vfs_caches_init+0xdc/0xf4 > [ 151.933578] start_kernel+0x638/0x6a4 > [ 151.934014] __primary_switched+0xc0/0xc8 > [ 151.934478] > [ 151.934757] The buggy address belongs to the object at > ffff0000c0074e00 > [ 151.934757] which belongs to the cache kmalloc-256 of size 256 > [ 151.935744] The buggy address is located 176 bytes inside of > [ 151.935744] 256-byte region [ffff0000c0074e00, ffff0000c0074f00) > [ 151.936702] The buggy address belongs to the page: > [ 151.937378] page:(____ptrval____) refcount:1 mapcount:0 > mapping:0000000000000000 index:0x0 pfn:0x100074 > [ 151.938682] head:(____ptrval____) order:2 compound_mapcount:0 > compound_pincount:0 > [ 151.939440] flags: > 0xbfffc0000010200(slab|head|node=0|zone=2|lastcpupid=0xffff|kasantag=0x > 0) > [ 151.940886] raw: 0bfffc0000010200 0000000000000000 dead000000000122 > f5ff0000c0002300 > [ 151.941634] raw: 0000000000000000 0000000000200020 00000001ffffffff > 0000000000000000 > [ 151.942353] page dumped because: kasan: bad access detected > [ 151.942923] > [ 151.943214] Memory state around the buggy address: > [ 151.943896] ffff0000c0074c00: f0 f0 f0 f0 f0 f0 f0 f0 f0 fe fe fe > fe fe fe fe > [ 151.944857] ffff0000c0074d00: fe fe fe fe fe fe fe fe fe fe fe fe > fe fe fe fe > [ 151.945892] >ffff0000c0074e00: f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 f7 fe > fe fe fe fe > [ 151.946407] ^ > [ 151.946939] ffff0000c0074f00: fe fe fe fe fe fe fe fe fe fe fe fe > fe fe fe fe > [ 151.947445] ffff0000c0075000: fb fb fb fb fb fb fb fb fb fb fb fb > fb fb fb fb > [ 151.947999] > ================================================================== > [ 151.948524] Disabling lock debugging due to kernel taint > [ 156.434569] kmemleak: 181 new suspected memory leaks (see > /sys/kernel/debug/kmemleak) > > > > > Perhaps this should have been "kmemleak, kasan: reset pointer tags to > > avoid false positives" ? > > Thanks for the suggestions. > But I think it doesn't belong to false > positive becuase scan block > touched invalid metadata certainly. It's how kmemleak works, so we knowingly access invalid memory, and for all intents and purposes it's a false positive. > Maybe "kmemleak, kasan: reset tags when scanning block"? That's fine. Thanks, -- Marco
On Tue, Jul 27, 2021 at 04:32:02PM +0800, Kuan-Ying Lee wrote: > On Tue, 2021-07-27 at 09:10 +0200, Marco Elver wrote: > > +Cc Catalin > > > > On Tue, 27 Jul 2021 at 06:00, Kuan-Ying Lee < > > Kuan-Ying.Lee@mediatek.com> wrote: > > > > > > Hardware tag-based KASAN doesn't use compiler instrumentation, we > > > can not use kasan_disable_current() to ignore tag check. > > > > > > Thus, we need to reset tags when accessing metadata. > > > > > > Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> > > > > This looks reasonable, but the patch title is not saying this is > > kmemleak, nor does the description say what the problem is. What > > problem did you encounter? Was it a false positive? > > kmemleak would scan kernel memory to check memory leak. > When it scans on the invalid slab and dereference, the issue > will occur like below. > > So I think we should reset the tag before scanning. > > # echo scan > /sys/kernel/debug/kmemleak > [ 151.905804] > ================================================================== > [ 151.907120] BUG: KASAN: out-of-bounds in scan_block+0x58/0x170 > [ 151.908773] Read at addr f7ff0000c0074eb0 by task kmemleak/138 > [ 151.909656] Pointer tag: [f7], memory tag: [fe] It would be interesting to find out why the tag doesn't match. Kmemleak should in principle only scan valid objects that have been allocated and the pointer can be safely dereferenced. 0xfe is KASAN_TAG_INVALID, so it either goes past the size of the object (into the red zone) or it still accesses the object after it was marked as freed but before being released from kmemleak. With slab, looking at __cache_free(), it calls kasan_slab_free() before ___cache_free() -> kmemleak_free_recursive(), so the second scenario is possible. With slub, however, slab_free_hook() first releases the object from kmemleak before poisoning it. Based on the stack dump, you are using slub, so it may be that kmemleak goes into the object red zones. I'd like this clarified before blindly resetting the tag.
On Tue, 2021-07-27 at 20:22 +0100, Catalin Marinas wrote: > On Tue, Jul 27, 2021 at 04:32:02PM +0800, Kuan-Ying Lee wrote: > > On Tue, 2021-07-27 at 09:10 +0200, Marco Elver wrote: > > > +Cc Catalin > > > > > > On Tue, 27 Jul 2021 at 06:00, Kuan-Ying Lee < > > > Kuan-Ying.Lee@mediatek.com> wrote: > > > > > > > > Hardware tag-based KASAN doesn't use compiler instrumentation, > > > > we > > > > can not use kasan_disable_current() to ignore tag check. > > > > > > > > Thus, we need to reset tags when accessing metadata. > > > > > > > > Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> > > > > > > This looks reasonable, but the patch title is not saying this is > > > kmemleak, nor does the description say what the problem is. What > > > problem did you encounter? Was it a false positive? > > > > kmemleak would scan kernel memory to check memory leak. > > When it scans on the invalid slab and dereference, the issue > > will occur like below. > > > > So I think we should reset the tag before scanning. > > > > # echo scan > /sys/kernel/debug/kmemleak > > [ 151.905804] > > ================================================================== > > [ 151.907120] BUG: KASAN: out-of-bounds in scan_block+0x58/0x170 > > [ 151.908773] Read at addr f7ff0000c0074eb0 by task kmemleak/138 > > [ 151.909656] Pointer tag: [f7], memory tag: [fe] > > It would be interesting to find out why the tag doesn't match. > Kmemleak > should in principle only scan valid objects that have been allocated > and > the pointer can be safely dereferenced. 0xfe is KASAN_TAG_INVALID, so > it > either goes past the size of the object (into the red zone) or it > still > accesses the object after it was marked as freed but before being > released from kmemleak. > > With slab, looking at __cache_free(), it calls kasan_slab_free() > before > ___cache_free() -> kmemleak_free_recursive(), so the second scenario > is > possible. With slub, however, slab_free_hook() first releases the > object > from kmemleak before poisoning it. Based on the stack dump, you are > using slub, so it may be that kmemleak goes into the object red > zones. > > I'd like this clarified before blindly resetting the tag. This kasan issue only happened on hardware tag-based kasan mode. Because kasan_disable_current() works for generic and sw tag-based kasan. HW tag-based kasan depends on slub so slab will not hit this issue. I think we can just check if HW tag-based kasan is enabled or not and decide to reset the tag as below. if (kasan_has_integrated_init()) // slub case, hw-tag kasan pointer = *(unsigned long *)kasan_reset_tag((void *)ptr); else pointer = *ptr; // slab Is this better or any other suggestions? Any suggestion is appreciated. Thanks, Kuan-Ying Lee >
On Wed, 28 Jul 2021 at 13:05, Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> wrote: > > On Tue, 2021-07-27 at 20:22 +0100, Catalin Marinas wrote: > > On Tue, Jul 27, 2021 at 04:32:02PM +0800, Kuan-Ying Lee wrote: > > > On Tue, 2021-07-27 at 09:10 +0200, Marco Elver wrote: > > > > +Cc Catalin > > > > > > > > On Tue, 27 Jul 2021 at 06:00, Kuan-Ying Lee < > > > > Kuan-Ying.Lee@mediatek.com> wrote: > > > > > > > > > > Hardware tag-based KASAN doesn't use compiler instrumentation, > > > > > we > > > > > can not use kasan_disable_current() to ignore tag check. > > > > > > > > > > Thus, we need to reset tags when accessing metadata. > > > > > > > > > > Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> > > > > > > > > This looks reasonable, but the patch title is not saying this is > > > > kmemleak, nor does the description say what the problem is. What > > > > problem did you encounter? Was it a false positive? > > > > > > kmemleak would scan kernel memory to check memory leak. > > > When it scans on the invalid slab and dereference, the issue > > > will occur like below. > > > > > > So I think we should reset the tag before scanning. > > > > > > # echo scan > /sys/kernel/debug/kmemleak > > > [ 151.905804] > > > ================================================================== > > > [ 151.907120] BUG: KASAN: out-of-bounds in scan_block+0x58/0x170 > > > [ 151.908773] Read at addr f7ff0000c0074eb0 by task kmemleak/138 > > > [ 151.909656] Pointer tag: [f7], memory tag: [fe] > > > > It would be interesting to find out why the tag doesn't match. > > Kmemleak > > should in principle only scan valid objects that have been allocated > > and > > the pointer can be safely dereferenced. 0xfe is KASAN_TAG_INVALID, so > > it > > either goes past the size of the object (into the red zone) or it > > still > > accesses the object after it was marked as freed but before being > > released from kmemleak. > > > > With slab, looking at __cache_free(), it calls kasan_slab_free() > > before > > ___cache_free() -> kmemleak_free_recursive(), so the second scenario > > is > > possible. With slub, however, slab_free_hook() first releases the > > object > > from kmemleak before poisoning it. Based on the stack dump, you are > > using slub, so it may be that kmemleak goes into the object red > > zones. > > > > I'd like this clarified before blindly resetting the tag. > > This kasan issue only happened on hardware tag-based kasan mode. > Because kasan_disable_current() works for generic and sw tag-based > kasan. > > HW tag-based kasan depends on slub so slab will not hit this > issue. > I think we can just check if HW tag-based kasan is enabled or not > and decide to reset the tag as below. > > if (kasan_has_integrated_init()) // slub case, hw-tag kasan > pointer = *(unsigned long *)kasan_reset_tag((void *)ptr); > else > pointer = *ptr; // slab This is redundant. kasan_reset_tag() is a noop if !IS_ENABLED(CONFIG_KASAN_HW_TAGS). > Is this better or any other suggestions? > Any suggestion is appreciated. The current version is fine. But I think Catalin's point about why kmemleak accesses the data in the first place still deserves some investigation. Could it be a race between free and kmemleak scan?
On Tue, Jul 27, 2021 at 9:22 PM Catalin Marinas <catalin.marinas@arm.com> wrote: > > On Tue, Jul 27, 2021 at 04:32:02PM +0800, Kuan-Ying Lee wrote: > > On Tue, 2021-07-27 at 09:10 +0200, Marco Elver wrote: > > > +Cc Catalin > > > > > > On Tue, 27 Jul 2021 at 06:00, Kuan-Ying Lee < > > > Kuan-Ying.Lee@mediatek.com> wrote: > > > > > > > > Hardware tag-based KASAN doesn't use compiler instrumentation, we > > > > can not use kasan_disable_current() to ignore tag check. > > > > > > > > Thus, we need to reset tags when accessing metadata. > > > > > > > > Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> > > > > > > This looks reasonable, but the patch title is not saying this is > > > kmemleak, nor does the description say what the problem is. What > > > problem did you encounter? Was it a false positive? > > > > kmemleak would scan kernel memory to check memory leak. > > When it scans on the invalid slab and dereference, the issue > > will occur like below. > > > > So I think we should reset the tag before scanning. > > > > # echo scan > /sys/kernel/debug/kmemleak > > [ 151.905804] > > ================================================================== > > [ 151.907120] BUG: KASAN: out-of-bounds in scan_block+0x58/0x170 > > [ 151.908773] Read at addr f7ff0000c0074eb0 by task kmemleak/138 > > [ 151.909656] Pointer tag: [f7], memory tag: [fe] > > It would be interesting to find out why the tag doesn't match. Kmemleak > should in principle only scan valid objects that have been allocated and > the pointer can be safely dereferenced. 0xfe is KASAN_TAG_INVALID, so it > either goes past the size of the object (into the red zone) or it still > accesses the object after it was marked as freed but before being > released from kmemleak. > > With slab, looking at __cache_free(), it calls kasan_slab_free() before > ___cache_free() -> kmemleak_free_recursive(), so the second scenario is > possible. With slub, however, slab_free_hook() first releases the object > from kmemleak before poisoning it. Based on the stack dump, you are > using slub, so it may be that kmemleak goes into the object red zones. > > I'd like this clarified before blindly resetting the tag. AFAIK, kmemleak scans the whole object including the leftover redzone for kmalloc-allocated objects. Looking at the report, there are 11 0xf7 granules, which amounts to 176 bytes, and the object is allocated from the kmalloc-256 cache. So when kmemleak accesses the last 256-176 bytes, it causes faults, as those are marked with KASAN_KMALLOC_REDZONE == KASAN_TAG_INVALID == 0xfe. Generally, resetting tags in kasan_disable/enable_current() section should be fine to suppress MTE faults, provided those sections had been added correctly in the first place.
On Fri, Jul 30, 2021 at 04:57:20PM +0200, Andrey Konovalov wrote: > On Tue, Jul 27, 2021 at 9:22 PM Catalin Marinas <catalin.marinas@arm.com> wrote: > > > > On Tue, Jul 27, 2021 at 04:32:02PM +0800, Kuan-Ying Lee wrote: > > > On Tue, 2021-07-27 at 09:10 +0200, Marco Elver wrote: > > > > +Cc Catalin > > > > > > > > On Tue, 27 Jul 2021 at 06:00, Kuan-Ying Lee < > > > > Kuan-Ying.Lee@mediatek.com> wrote: > > > > > > > > > > Hardware tag-based KASAN doesn't use compiler instrumentation, we > > > > > can not use kasan_disable_current() to ignore tag check. > > > > > > > > > > Thus, we need to reset tags when accessing metadata. > > > > > > > > > > Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> > > > > > > > > This looks reasonable, but the patch title is not saying this is > > > > kmemleak, nor does the description say what the problem is. What > > > > problem did you encounter? Was it a false positive? > > > > > > kmemleak would scan kernel memory to check memory leak. > > > When it scans on the invalid slab and dereference, the issue > > > will occur like below. > > > > > > So I think we should reset the tag before scanning. > > > > > > # echo scan > /sys/kernel/debug/kmemleak > > > [ 151.905804] > > > ================================================================== > > > [ 151.907120] BUG: KASAN: out-of-bounds in scan_block+0x58/0x170 > > > [ 151.908773] Read at addr f7ff0000c0074eb0 by task kmemleak/138 > > > [ 151.909656] Pointer tag: [f7], memory tag: [fe] > > > > It would be interesting to find out why the tag doesn't match. Kmemleak > > should in principle only scan valid objects that have been allocated and > > the pointer can be safely dereferenced. 0xfe is KASAN_TAG_INVALID, so it > > either goes past the size of the object (into the red zone) or it still > > accesses the object after it was marked as freed but before being > > released from kmemleak. > > > > With slab, looking at __cache_free(), it calls kasan_slab_free() before > > ___cache_free() -> kmemleak_free_recursive(), so the second scenario is > > possible. With slub, however, slab_free_hook() first releases the object > > from kmemleak before poisoning it. Based on the stack dump, you are > > using slub, so it may be that kmemleak goes into the object red zones. > > > > I'd like this clarified before blindly resetting the tag. > > AFAIK, kmemleak scans the whole object including the leftover redzone > for kmalloc-allocated objects. > > Looking at the report, there are 11 0xf7 granules, which amounts to > 176 bytes, and the object is allocated from the kmalloc-256 cache. So > when kmemleak accesses the last 256-176 bytes, it causes faults, as > those are marked with KASAN_KMALLOC_REDZONE == KASAN_TAG_INVALID == > 0xfe. > > Generally, resetting tags in kasan_disable/enable_current() section > should be fine to suppress MTE faults, provided those sections had > been added correctly in the first place. Thanks for the explanation, the patch makes sense. FWIW: Acked-by: Catalin Marinas <catalin.marinas@arm.com>
On Fri, 2021-07-30 at 16:57 +0200, Andrey Konovalov wrote: > On Tue, Jul 27, 2021 at 9:22 PM Catalin Marinas < > catalin.marinas@arm.com> wrote: > > > > On Tue, Jul 27, 2021 at 04:32:02PM +0800, Kuan-Ying Lee wrote: > > > On Tue, 2021-07-27 at 09:10 +0200, Marco Elver wrote: > > > > +Cc Catalin > > > > > > > > On Tue, 27 Jul 2021 at 06:00, Kuan-Ying Lee < > > > > Kuan-Ying.Lee@mediatek.com> wrote: > > > > > > > > > > Hardware tag-based KASAN doesn't use compiler > > > > > instrumentation, we > > > > > can not use kasan_disable_current() to ignore tag check. > > > > > > > > > > Thus, we need to reset tags when accessing metadata. > > > > > > > > > > Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> > > > > > > > > This looks reasonable, but the patch title is not saying this > > > > is > > > > kmemleak, nor does the description say what the problem is. > > > > What > > > > problem did you encounter? Was it a false positive? > > > > > > kmemleak would scan kernel memory to check memory leak. > > > When it scans on the invalid slab and dereference, the issue > > > will occur like below. > > > > > > So I think we should reset the tag before scanning. > > > > > > # echo scan > /sys/kernel/debug/kmemleak > > > [ 151.905804] > > > ================================================================= > > > = > > > [ 151.907120] BUG: KASAN: out-of-bounds in scan_block+0x58/0x170 > > > [ 151.908773] Read at addr f7ff0000c0074eb0 by task kmemleak/138 > > > [ 151.909656] Pointer tag: [f7], memory tag: [fe] > > > > It would be interesting to find out why the tag doesn't match. > > Kmemleak > > should in principle only scan valid objects that have been > > allocated and > > the pointer can be safely dereferenced. 0xfe is KASAN_TAG_INVALID, > > so it > > either goes past the size of the object (into the red zone) or it > > still > > accesses the object after it was marked as freed but before being > > released from kmemleak. > > > > With slab, looking at __cache_free(), it calls kasan_slab_free() > > before > > ___cache_free() -> kmemleak_free_recursive(), so the second > > scenario is > > possible. With slub, however, slab_free_hook() first releases the > > object > > from kmemleak before poisoning it. Based on the stack dump, you are > > using slub, so it may be that kmemleak goes into the object red > > zones. > > > > I'd like this clarified before blindly resetting the tag. > > AFAIK, kmemleak scans the whole object including the leftover redzone > for kmalloc-allocated objects. > > Looking at the report, there are 11 0xf7 granules, which amounts to > 176 bytes, and the object is allocated from the kmalloc-256 cache. So > when kmemleak accesses the last 256-176 bytes, it causes faults, as > those are marked with KASAN_KMALLOC_REDZONE == KASAN_TAG_INVALID == > 0xfe. > > Generally, resetting tags in kasan_disable/enable_current() section > should be fine to suppress MTE faults, provided those sections had > been added correctly in the first place. Thanks Andrey for explanation. I will refine commit and upload v2. Thanks.
diff --git a/mm/kmemleak.c b/mm/kmemleak.c index 228a2fbe0657..73d46d16d575 100644 --- a/mm/kmemleak.c +++ b/mm/kmemleak.c @@ -290,7 +290,7 @@ static void hex_dump_object(struct seq_file *seq, warn_or_seq_printf(seq, " hex dump (first %zu bytes):\n", len); kasan_disable_current(); warn_or_seq_hex_dump(seq, DUMP_PREFIX_NONE, HEX_ROW_SIZE, - HEX_GROUP_SIZE, ptr, len, HEX_ASCII); + HEX_GROUP_SIZE, kasan_reset_tag((void *)ptr), len, HEX_ASCII); kasan_enable_current(); } @@ -1171,7 +1171,7 @@ static bool update_checksum(struct kmemleak_object *object) kasan_disable_current(); kcsan_disable_current(); - object->checksum = crc32(0, (void *)object->pointer, object->size); + object->checksum = crc32(0, kasan_reset_tag((void *)object->pointer), object->size); kasan_enable_current(); kcsan_enable_current(); @@ -1246,7 +1246,7 @@ static void scan_block(void *_start, void *_end, break; kasan_disable_current(); - pointer = *ptr; + pointer = *(unsigned long *)kasan_reset_tag((void *)ptr); kasan_enable_current(); untagged_ptr = (unsigned long)kasan_reset_tag((void *)pointer);
Hardware tag-based KASAN doesn't use compiler instrumentation, we can not use kasan_disable_current() to ignore tag check. Thus, we need to reset tags when accessing metadata. Signed-off-by: Kuan-Ying Lee <Kuan-Ying.Lee@mediatek.com> --- mm/kmemleak.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)