| Message ID | 20250305230007.590178-1-vinicius.gomes@intel.com (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | e87ca16e99118ab4e130a41bdf12abbf6a87656c |
| Headers | show |
| Series | [v1] dmaengine: dmatest: Fix dmatest waiting less when interrupted | expand |
On 3/5/25 4:00 PM, Vinicius Costa Gomes wrote: > Change the "wait for operation finish" logic to take interrupts into > account. > > When using dmatest with idxd DMA engine, it's possible that during > longer tests, the interrupt notifying the finish of an operation > happens during wait_event_freezable_timeout(), which causes dmatest to > cleanup all the resources, some of which might still be in use. > > This fix ensures that the wait logic correctly handles interrupts, > preventing premature cleanup of resources. > > Reported-by: kernel test robot <oliver.sang@intel.com> > Closes: https://lore.kernel.org/oe-lkp/202502171134.8c403348-lkp@intel.com > Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com> Reviewed-by: Dave Jiang <dave.jiang@intel.com> > --- > drivers/dma/dmatest.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/drivers/dma/dmatest.c b/drivers/dma/dmatest.c > index 91b2fbc0b864..d891dfca358e 100644 > --- a/drivers/dma/dmatest.c > +++ b/drivers/dma/dmatest.c > @@ -841,9 +841,9 @@ static int dmatest_func(void *data) > } else { > dma_async_issue_pending(chan); > > - wait_event_freezable_timeout(thread->done_wait, > - done->done, > - msecs_to_jiffies(params->timeout)); > + wait_event_timeout(thread->done_wait, > + done->done, > + msecs_to_jiffies(params->timeout)); > > status = dma_async_is_tx_complete(chan, cookie, NULL, > NULL);
On Wed, 05 Mar 2025 15:00:06 -0800, Vinicius Costa Gomes wrote: > Change the "wait for operation finish" logic to take interrupts into > account. > > When using dmatest with idxd DMA engine, it's possible that during > longer tests, the interrupt notifying the finish of an operation > happens during wait_event_freezable_timeout(), which causes dmatest to > cleanup all the resources, some of which might still be in use. > > [...] Applied, thanks! [1/1] dmaengine: dmatest: Fix dmatest waiting less when interrupted commit: e87ca16e99118ab4e130a41bdf12abbf6a87656c Best regards,
Hi, Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: > Change the "wait for operation finish" logic to take interrupts into > account. > > When using dmatest with idxd DMA engine, it's possible that during > longer tests, the interrupt notifying the finish of an operation > happens during wait_event_freezable_timeout(), which causes dmatest to > cleanup all the resources, some of which might still be in use. > > This fix ensures that the wait logic correctly handles interrupts, > preventing premature cleanup of resources. > > Reported-by: kernel test robot <oliver.sang@intel.com> > Closes: https://lore.kernel.org/oe-lkp/202502171134.8c403348-lkp@intel.com Given the report at the URL above I'm struggling to follow the rationale for this change. It looks like a use-after-free in idxd while resetting/unbinding the device, and I can't see how changing whether dmatest threads perform freezeable waits would change this. Note the idxd code emits a couple of warnings about inconsistent state before the UAF: [ 81.023244][ T1644] idxd dsa0: Active wq 0 on disable wq0.0. [ 81.040447][ T1644] idxd 0000:6a:01.0: Clients has claim on wq 0: 1 Here is the bad access: BUG: KASAN: slab-use-after-free in idxd_dma_complete_txd+0x418/0x510 [idxd] Write of size 4 at addr ff11000134978114 by task kworker/118:1/1644 CPU: 118 UID: 0 PID: 1644 Comm: kworker/118:1 Tainted: G S 6.13.0-rc1-00054-g98d187a98903 #1 Tainted: [S]=CPU_OUT_OF_SPEC Hardware name: Intel Corporation D50DNP1SBB/D50DNP1SBB, BIOS SE5C7411.86B.8118.D04.2206151341 06/15/2022 Workqueue: 0000:6a:01.0 idxd_device_flr [idxd] Call Trace: <TASK> dump_stack_lvl+0x4f/0x70 print_address_description+0x2c/0x3a0 print_report+0xb9/0x280 kasan_report+0xaa/0xe0 idxd_dma_complete_txd+0x418/0x510 [idxd] idxd_flush_pending_descs+0x4a8/0x7e0 [idxd] idxd_wq_free_irq+0xcd/0x330 [idxd] idxd_drv_disable_wq+0x125/0x2d0 [idxd] idxd_dmaengine_drv_remove+0x1fd/0x2f0 [idxd] device_release_driver_internal+0x36d/0x530 idxd_device_drv_remove+0xa0/0x240 [idxd] device_release_driver_internal+0x36d/0x530 idxd_reset_done+0x600/0x770 [idxd] pci_reset_function+0x1c9/0x230 idxd_device_flr+0x34/0x90 [idxd] process_one_work+0x676/0x1000 worker_thread+0x710/0xf40 kthread+0x2d4/0x3c0 ret_from_fork+0x2d/0x70 ret_from_fork_asm+0x1a/0x30 </TASK> Here is the allocation site: Allocated by task 3664: kasan_save_stack+0x1c/0x40 kasan_save_track+0x10/0x30 __kasan_kmalloc+0x8b/0x90 idxd_dmaengine_drv_probe+0x2eb/0x860 [idxd] really_probe+0x1e0/0x920 __driver_probe_device+0x18c/0x3d0 device_driver_attach+0xae/0x1b0 bind_store+0xc9/0x140 kernfs_fop_write_iter+0x2e6/0x4c0 And here is where the memory was released earlier: Freed by task 1644: kasan_save_stack+0x1c/0x40 kasan_save_track+0x10/0x30 kasan_save_free_info+0x37/0x60 __kasan_slab_free+0x33/0x40 kfree+0xef/0x3e0 idxd_dmaengine_drv_remove+0x1cb/0x2f0 [idxd] device_release_driver_internal+0x36d/0x530 idxd_device_drv_remove+0xa0/0x240 [idxd] device_release_driver_internal+0x36d/0x530 idxd_reset_done+0x600/0x770 [idxd] pci_reset_function+0x1c9/0x230 idxd_device_flr+0x34/0x90 [idxd] process_one_work+0x676/0x1000 worker_thread+0x710/0xf40 kthread+0x2d4/0x3c0 ret_from_fork+0x2d/0x70 > Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com> > --- > drivers/dma/dmatest.c | 6 +++--- > 1 file changed, 3 insertions(+), 3 deletions(-) > > diff --git a/drivers/dma/dmatest.c b/drivers/dma/dmatest.c > index 91b2fbc0b864..d891dfca358e 100644 > --- a/drivers/dma/dmatest.c > +++ b/drivers/dma/dmatest.c > @@ -841,9 +841,9 @@ static int dmatest_func(void *data) > } else { > dma_async_issue_pending(chan); > > - wait_event_freezable_timeout(thread->done_wait, > - done->done, > - msecs_to_jiffies(params->timeout)); > + wait_event_timeout(thread->done_wait, > + done->done, > + msecs_to_jiffies(params->timeout)); > > status = dma_async_is_tx_complete(chan, cookie, NULL, > NULL); > -- > 2.48.1
Hi Nathan, Nathan Lynch <nathan.lynch@amd.com> writes: > Hi, > > Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >> Change the "wait for operation finish" logic to take interrupts into >> account. >> >> When using dmatest with idxd DMA engine, it's possible that during >> longer tests, the interrupt notifying the finish of an operation >> happens during wait_event_freezable_timeout(), which causes dmatest to >> cleanup all the resources, some of which might still be in use. >> >> This fix ensures that the wait logic correctly handles interrupts, >> preventing premature cleanup of resources. >> >> Reported-by: kernel test robot <oliver.sang@intel.com> >> Closes: https://lore.kernel.org/oe-lkp/202502171134.8c403348-lkp@intel.com > > Given the report at the URL above I'm struggling to follow the rationale > for this change. It looks like a use-after-free in idxd while > resetting/unbinding the device, and I can't see how changing whether > dmatest threads perform freezeable waits would change this. > I think that the short version is that the reproducition script triggers different problems on different platforms/configurations. The solution I proposed fixes a problem I was seeing on a SPR system, on a GNR system (that I was only able to get later) I see something more similar to this particular splat (currently working on the fix). Seeing this question, I realize that I should have added a note to the commit detailing this. So I am planning on proposing two (this and another) fixes for the same report, hoping that it's not that confusing/unusual. Cheers,
Hi Vinicius, Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: > Nathan Lynch <nathan.lynch@amd.com> writes: >> Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >>> Change the "wait for operation finish" logic to take interrupts into >>> account. >>> >>> When using dmatest with idxd DMA engine, it's possible that during >>> longer tests, the interrupt notifying the finish of an operation >>> happens during wait_event_freezable_timeout(), which causes dmatest to >>> cleanup all the resources, some of which might still be in use. >>> >>> This fix ensures that the wait logic correctly handles interrupts, >>> preventing premature cleanup of resources. >>> >>> Reported-by: kernel test robot <oliver.sang@intel.com> >>> Closes: https://lore.kernel.org/oe-lkp/202502171134.8c403348-lkp@intel.com >> >> Given the report at the URL above I'm struggling to follow the rationale >> for this change. It looks like a use-after-free in idxd while >> resetting/unbinding the device, and I can't see how changing whether >> dmatest threads perform freezeable waits would change this. >> > > I think that the short version is that the reproducition script triggers > different problems on different platforms/configurations. > > The solution I proposed fixes a problem I was seeing on a SPR system, on > a GNR system (that I was only able to get later) I see something more similar > to this particular splat (currently working on the fix). > > Seeing this question, I realize that I should have added a note to the > commit detailing this. > > So I am planning on proposing two (this and another) fixes for the same > report, hoping that it's not that confusing/unusual. I'm still confused... why is wait_event_freezable_timeout() the wrong API for dmatest to use, and how could changing it to wait_event_timeout() cause it to "take interrupts into account" that it didn't before? AFAIK the only change made here is that dmatest threads effectively become unfreezeable, which is contrary to prior authors' intentions: commit 981ed70d8e4f ("dmatest: make dmatest threads freezable") commit adfa543e7314 ("dmatest: don't use set_freezable_with_signal()")
Nathan Lynch <nathan.lynch@amd.com> writes: > Hi Vinicius, > > Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >> Nathan Lynch <nathan.lynch@amd.com> writes: >>> Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >>>> Change the "wait for operation finish" logic to take interrupts into >>>> account. >>>> >>>> When using dmatest with idxd DMA engine, it's possible that during >>>> longer tests, the interrupt notifying the finish of an operation >>>> happens during wait_event_freezable_timeout(), which causes dmatest to >>>> cleanup all the resources, some of which might still be in use. >>>> >>>> This fix ensures that the wait logic correctly handles interrupts, >>>> preventing premature cleanup of resources. >>>> >>>> Reported-by: kernel test robot <oliver.sang@intel.com> >>>> Closes: https://lore.kernel.org/oe-lkp/202502171134.8c403348-lkp@intel.com >>> >>> Given the report at the URL above I'm struggling to follow the rationale >>> for this change. It looks like a use-after-free in idxd while >>> resetting/unbinding the device, and I can't see how changing whether >>> dmatest threads perform freezeable waits would change this. >>> >> >> I think that the short version is that the reproducition script triggers >> different problems on different platforms/configurations. >> >> The solution I proposed fixes a problem I was seeing on a SPR system, on >> a GNR system (that I was only able to get later) I see something more similar >> to this particular splat (currently working on the fix). >> >> Seeing this question, I realize that I should have added a note to the >> commit detailing this. >> >> So I am planning on proposing two (this and another) fixes for the same >> report, hoping that it's not that confusing/unusual. > > I'm still confused... why is wait_event_freezable_timeout() the wrong > API for dmatest to use, and how could changing it to > wait_event_timeout() cause it to "take interrupts into account" that it > didn't before? > My understanding (and testing) is that wait_event_timeout() will block for the duration even in the face of interrupts, 'freezable' will not. > AFAIK the only change made here is that dmatest threads effectively > become unfreezeable, which is contrary to prior authors' intentions: > > commit 981ed70d8e4f ("dmatest: make dmatest threads freezable") > commit adfa543e7314 ("dmatest: don't use set_freezable_with_signal()") > Cheers,
Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: > Nathan Lynch <nathan.lynch@amd.com> writes: > >> Hi Vinicius, >> >> Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >>> Nathan Lynch <nathan.lynch@amd.com> writes: >>>> Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >>>>> Change the "wait for operation finish" logic to take interrupts into >>>>> account. >>>>> >>>>> When using dmatest with idxd DMA engine, it's possible that during >>>>> longer tests, the interrupt notifying the finish of an operation >>>>> happens during wait_event_freezable_timeout(), which causes dmatest to >>>>> cleanup all the resources, some of which might still be in use. >>>>> >>>>> This fix ensures that the wait logic correctly handles interrupts, >>>>> preventing premature cleanup of resources. >>>>> >>>>> Reported-by: kernel test robot <oliver.sang@intel.com> >>>>> Closes: https://lore.kernel.org/oe-lkp/202502171134.8c403348-lkp@intel.com >>>> >>>> Given the report at the URL above I'm struggling to follow the rationale >>>> for this change. It looks like a use-after-free in idxd while >>>> resetting/unbinding the device, and I can't see how changing whether >>>> dmatest threads perform freezeable waits would change this. >>>> >>> >>> I think that the short version is that the reproducition script triggers >>> different problems on different platforms/configurations. >>> >>> The solution I proposed fixes a problem I was seeing on a SPR system, on >>> a GNR system (that I was only able to get later) I see something more similar >>> to this particular splat (currently working on the fix). >>> >>> Seeing this question, I realize that I should have added a note to the >>> commit detailing this. >>> >>> So I am planning on proposing two (this and another) fixes for the same >>> report, hoping that it's not that confusing/unusual. >> >> I'm still confused... why is wait_event_freezable_timeout() the wrong >> API for dmatest to use, and how could changing it to >> wait_event_timeout() cause it to "take interrupts into account" that it >> didn't before? >> > > My understanding (and testing) is that wait_event_timeout() will block > for the duration even in the face of interrupts, 'freezable' will not. They have different behaviors with respect to *signals* and the wake_up() variant used, but not device interrupts. dmatest_callback() employs wake_up_all(), which means this change introduces no beneficial difference in the wakeup behavior. The dmatest thread gets woken on receipt of the completion interrupt either way. And to reiterate, the change regresses the combination of dmatest and the task freezer, which is a use case people have cared about, apparently. >> AFAIK the only change made here is that dmatest threads effectively >> become unfreezeable, which is contrary to prior authors' intentions: >> >> commit 981ed70d8e4f ("dmatest: make dmatest threads freezable") >> commit adfa543e7314 ("dmatest: don't use set_freezable_with_signal()")
Nathan Lynch <nathan.lynch@amd.com> writes: > Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >> Nathan Lynch <nathan.lynch@amd.com> writes: >> >>> Hi Vinicius, >>> >>> Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >>>> Nathan Lynch <nathan.lynch@amd.com> writes: >>>>> Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >>>>>> Change the "wait for operation finish" logic to take interrupts into >>>>>> account. >>>>>> >>>>>> When using dmatest with idxd DMA engine, it's possible that during >>>>>> longer tests, the interrupt notifying the finish of an operation >>>>>> happens during wait_event_freezable_timeout(), which causes dmatest to >>>>>> cleanup all the resources, some of which might still be in use. >>>>>> >>>>>> This fix ensures that the wait logic correctly handles interrupts, >>>>>> preventing premature cleanup of resources. >>>>>> >>>>>> Reported-by: kernel test robot <oliver.sang@intel.com> >>>>>> Closes: https://lore.kernel.org/oe-lkp/202502171134.8c403348-lkp@intel.com >>>>> >>>>> Given the report at the URL above I'm struggling to follow the rationale >>>>> for this change. It looks like a use-after-free in idxd while >>>>> resetting/unbinding the device, and I can't see how changing whether >>>>> dmatest threads perform freezeable waits would change this. >>>>> >>>> >>>> I think that the short version is that the reproducition script triggers >>>> different problems on different platforms/configurations. >>>> >>>> The solution I proposed fixes a problem I was seeing on a SPR system, on >>>> a GNR system (that I was only able to get later) I see something more similar >>>> to this particular splat (currently working on the fix). >>>> >>>> Seeing this question, I realize that I should have added a note to the >>>> commit detailing this. >>>> >>>> So I am planning on proposing two (this and another) fixes for the same >>>> report, hoping that it's not that confusing/unusual. >>> >>> I'm still confused... why is wait_event_freezable_timeout() the wrong >>> API for dmatest to use, and how could changing it to >>> wait_event_timeout() cause it to "take interrupts into account" that it >>> didn't before? >>> >> >> My understanding (and testing) is that wait_event_timeout() will block >> for the duration even in the face of interrupts, 'freezable' will not. > > They have different behaviors with respect to *signals* and the > wake_up() variant used, but not device interrupts. > Ah! That's something that I wasn't considering. That it could be something other than interrupts that were unblocking wait_event_*(). > dmatest_callback() employs wake_up_all(), which means this change > introduces no beneficial difference in the wakeup behavior. The dmatest > thread gets woken on receipt of the completion interrupt either way. > > And to reiterate, the change regresses the combination of dmatest and > the task freezer, which is a use case people have cared about, > apparently. > If this change in behavior causes a regression for others, glad to send a revert and find another solution. >>> AFAIK the only change made here is that dmatest threads effectively >>> become unfreezeable, which is contrary to prior authors' intentions: >>> >>> commit 981ed70d8e4f ("dmatest: make dmatest threads freezable") >>> commit adfa543e7314 ("dmatest: don't use set_freezable_with_signal()") Cheers,
Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: > Nathan Lynch <nathan.lynch@amd.com> writes: >> Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >>> My understanding (and testing) is that wait_event_timeout() will block >>> for the duration even in the face of interrupts, 'freezable' will not. >> >> They have different behaviors with respect to *signals* and the >> wake_up() variant used, but not device interrupts. >> > > Ah! That's something that I wasn't considering. That it could be > something other than interrupts that were unblocking wait_event_*(). Well, I doubt it would be a signal in this case. Maybe you've experienced timeouts? >> dmatest_callback() employs wake_up_all(), which means this change >> introduces no beneficial difference in the wakeup behavior. The dmatest >> thread gets woken on receipt of the completion interrupt either way. >> >> And to reiterate, the change regresses the combination of dmatest and >> the task freezer, which is a use case people have cared about, >> apparently. >> > > If this change in behavior causes a regression for others, glad to send > a revert and find another solution. Thanks - yes it should be reverted or dropped IMO.
Nathan Lynch <nathan.lynch@amd.com> writes: > Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >> Nathan Lynch <nathan.lynch@amd.com> writes: >>> Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >>>> My understanding (and testing) is that wait_event_timeout() will block >>>> for the duration even in the face of interrupts, 'freezable' will not. >>> >>> They have different behaviors with respect to *signals* and the >>> wake_up() variant used, but not device interrupts. >>> >> >> Ah! That's something that I wasn't considering. That it could be >> something other than interrupts that were unblocking wait_event_*(). > > Well, I doubt it would be a signal in this case. Maybe you've > experienced timeouts? > What I was seeing is that the wait_event_freezable_timeout() was finishing early, 'done->done' was false, and so dmatest considers it a timeout, and cleans everything up. Later the pending operation finishes, resulting in the warning below, because it everything was already cleaned up. Changing the way the waiting was done fixed the issue. (perhaps approaching from the other direction, "why is it finishing early?", might be better) The log I was seeing is here: [ 2677.607890] dmatest: Added 1 threads using dma8chan0 [ 2677.608063] dmatest: Started 1 threads using dma8chan0 [ 2677.623910] dmatest: dma8chan0-copy0: result #777: 'test timed out' with src_off=0x1 dst_off=0x4 len=0x1c (0) [ 2677.623917] dmatest: dma8chan0-copy0: summary 777 tests, 1 failures 50323.83 iops 777 KB/s (0) [ 2677.624356] ------------[ cut here ]------------ [ 2677.624360] dmatest: Kernel memory may be corrupted!! [ 2677.624385] WARNING: CPU: 142 PID: 38729 at drivers/dma/dmatest.c:450 dmatest_callback+0x30/0x40 [dmatest] [ 2677.624391] Modules linked in: dmatest(-) idxd idxd_bus xt_conntrack nft_chain_nat xt_MASQUERADE nf_nat nf_conntrack_netlink nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 xfrm_user xfrm_algo xt_addrtype nft_compat nf_tables br_netfilter bridge stp llc overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common i10nm_edac skx_edac_common nfit x86_pkg_temp_thermal intel_powerclamp coretemp snd_hda_codec_realtek binfmt_misc snd_hda_codec_generic snd_hda_scodec_component kvm_intel snd_hda_intel dax_hmem snd_intel_dspcfg snd_intel_sdw_acpi cxl_acpi cxl_port snd_hda_codec snd_hda_core kvm snd_hwdep ast drm_client_lib drm_shmem_helper rapl ipmi_ssif drm_kms_helper cxl_core snd_pcm isst_if_mmio isst_if_mbox_pci intel_cstate einj i2c_algo_bit qat_4xxx snd_timer isst_if_common i2c_i801 nls_iso8859_1 intel_qat snd mei_me crc8 i2c_mux authenc soundcore i2c_smbus mei i2c_ismt acpi_power_meter ipmi_si acpi_ipmi ipmi_devintf ipmi_msghandler acpi_pad joydev input_leds mac_hid sch_fq_codel dm_multipath [ 2677.624453] drm efi_pstore nfnetlink dmi_sysfs ip_tables x_tables autofs4 btrfs blake2b_generic zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq raid1 raid0 hid_generic nvme ghash_clmulni_intel usbhid ahci sha512_ssse3 sha256_ssse3 hid nvme_core igc sha1_ssse3 libahci wmi pinctrl_emmitsburg aesni_intel crypto_simd cryptd [last unloaded: dmatest] [ 2677.624482] CPU: 142 UID: 0 PID: 38729 Comm: irq/112-idxd-po Not tainted 6.14.0-rc1+ #11 [ 2677.624484] Hardware name: Intel Corporation ArcherCity/ArcherCity, BIOS EGSDCRB1.SYS.0037.P02.2305261001 05/26/2023 [ 2677.624485] RIP: 0010:dmatest_callback+0x30/0x40 [dmatest] [ 2677.624488] Code: 44 00 00 80 7f 10 00 75 15 c6 07 01 48 8b 7f 08 31 c9 31 d2 be 03 00 00 00 e9 5c 20 03 f6 48 c7 c7 00 22 d6 c0 e8 30 9a f9 f5 <0f> 0b c3 cc cc cc cc 66 0f 1f 84 00 00 00 00 00 90 90 90 90 90 90 [ 2677.624489] RSP: 0018:ff574666a8037db8 EFLAGS: 00010282 [ 2677.624491] RAX: 0000000000000000 RBX: ff4a4a2b1bb8a600 RCX: ff4a4a2e6ff21a48 [ 2677.624492] RDX: 0000000000000027 RSI: 0000000000000027 RDI: 0000000000000001 [ 2677.624493] RBP: 0000000000000001 R08: 0000000000000000 R09: 0000000000000003 [ 2677.624494] R10: ff574666a8037c58 R11: ff4a4a327fec9fe8 R12: ff4a4a2b1bb8a620 [ 2677.624495] R13: ff4a4a2b1bb8a600 R14: ffffffffffffff88 R15: ffffffffffffff88 [ 2677.624496] FS: 0000000000000000(0000) GS:ff4a4a2e6ff00000(0000) knlGS:0000000000000000 [ 2677.624498] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 2677.624499] CR2: 00007f71f9e543f0 CR3: 0000000159530003 CR4: 0000000000f73ef0 [ 2677.624500] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 2677.624501] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400 [ 2677.624502] PKRU: 55555554 [ 2677.624503] Call Trace: [ 2677.624505] <TASK> [ 2677.624507] ? __warn+0x91/0x190 [ 2677.624511] ? dmatest_callback+0x30/0x40 [dmatest] [ 2677.624515] ? report_bug+0x164/0x190 [ 2677.624521] ? handle_bug+0x58/0x90 [ 2677.624524] ? exc_invalid_op+0x17/0x70 [ 2677.624526] ? asm_exc_invalid_op+0x1a/0x20 [ 2677.624532] ? dmatest_callback+0x30/0x40 [dmatest] [ 2677.624534] ? dmatest_callback+0x30/0x40 [dmatest] [ 2677.624536] idxd_dma_complete_txd+0x151/0x160 [idxd] [ 2677.624544] idxd_wq_thread+0x1c0/0x2b0 [idxd] [ 2677.624549] irq_thread_fn+0x20/0x60 [ 2677.624554] irq_thread+0x18b/0x2c0 [ 2677.624556] ? __pfx_irq_thread_fn+0x10/0x10 [ 2677.624559] ? __pfx_irq_thread_dtor+0x10/0x10 [ 2677.624562] ? __pfx_irq_thread+0x10/0x10 [ 2677.624565] kthread+0x100/0x210 [ 2677.624569] ? __pfx_kthread+0x10/0x10 [ 2677.624572] ret_from_fork+0x31/0x50 [ 2677.624575] ? __pfx_kthread+0x10/0x10 [ 2677.624577] ret_from_fork_asm+0x1a/0x30 [ 2677.624584] </TASK> [ 2677.624585] irq event stamp: 18951 [ 2677.624586] hardirqs last enabled at (18957): [<ffffffffb6d9c5ee>] __up_console_sem+0x5e/0x70 [ 2677.624589] hardirqs last disabled at (18962): [<ffffffffb6d9c5d3>] __up_console_sem+0x43/0x70 [ 2677.624591] softirqs last enabled at (1084): [<ffffffffb6cde835>] handle_softirqs+0x315/0x3f0 [ 2677.624594] softirqs last disabled at (1067): [<ffffffffb6cde9cc>] __irq_exit_rcu+0xac/0xd0 [ 2677.624596] ---[ end trace 0000000000000000 ]--- >>> dmatest_callback() employs wake_up_all(), which means this change >>> introduces no beneficial difference in the wakeup behavior. The dmatest >>> thread gets woken on receipt of the completion interrupt either way. >>> >>> And to reiterate, the change regresses the combination of dmatest and >>> the task freezer, which is a use case people have cared about, >>> apparently. >>> >> >> If this change in behavior causes a regression for others, glad to send >> a revert and find another solution. > > Thanks - yes it should be reverted or dropped IMO. Here's what I am thinking, I'll work on this a few days and see if I can find an alternative solution and send the revert together with the fix. If I can't find another solution in a few days, I'll propose the revert anyway. Cheers,
Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: > Nathan Lynch <nathan.lynch@amd.com> writes: > >> Vinicius Costa Gomes <vinicius.gomes@intel.com> writes: >>> Nathan Lynch <nathan.lynch@amd.com> writes: >>>> dmatest_callback() employs wake_up_all(), which means this change >>>> introduces no beneficial difference in the wakeup behavior. The dmatest >>>> thread gets woken on receipt of the completion interrupt either way. >>>> >>>> And to reiterate, the change regresses the combination of dmatest and >>>> the task freezer, which is a use case people have cared about, >>>> apparently. >>>> >>> >>> If this change in behavior causes a regression for others, glad to send >>> a revert and find another solution. >> >> Thanks - yes it should be reverted or dropped IMO. > > Here's what I am thinking, I'll work on this a few days and see if I can > find an alternative solution and send the revert together with the fix. > If I can't find another solution in a few days, I'll propose the revert > anyway. Just checking on this - I see this regression is in Linus's master branch now.
diff --git a/drivers/dma/dmatest.c b/drivers/dma/dmatest.c index 91b2fbc0b864..d891dfca358e 100644 --- a/drivers/dma/dmatest.c +++ b/drivers/dma/dmatest.c @@ -841,9 +841,9 @@ static int dmatest_func(void *data) } else { dma_async_issue_pending(chan); - wait_event_freezable_timeout(thread->done_wait, - done->done, - msecs_to_jiffies(params->timeout)); + wait_event_timeout(thread->done_wait, + done->done, + msecs_to_jiffies(params->timeout)); status = dma_async_is_tx_complete(chan, cookie, NULL, NULL);
Change the "wait for operation finish" logic to take interrupts into account. When using dmatest with idxd DMA engine, it's possible that during longer tests, the interrupt notifying the finish of an operation happens during wait_event_freezable_timeout(), which causes dmatest to cleanup all the resources, some of which might still be in use. This fix ensures that the wait logic correctly handles interrupts, preventing premature cleanup of resources. Reported-by: kernel test robot <oliver.sang@intel.com> Closes: https://lore.kernel.org/oe-lkp/202502171134.8c403348-lkp@intel.com Signed-off-by: Vinicius Costa Gomes <vinicius.gomes@intel.com> --- drivers/dma/dmatest.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)