| Message ID | 20220824154358.117-1-shiraz.saleem@intel.com (mailing list archive) |
|---|---|
| State | Accepted |
| Headers | show |
| Series | [for-rc] RDMA/irdma: Fix drain SQ hang with no completion | expand |
On Wed, Aug 24, 2022 at 10:43:59AM -0500, Shiraz Saleem wrote: > SW generated completions for outstanding WRs posted on SQ > after QP is in error target the wrong CQ. This causes the > ib_drain_sq to hang with no completion. > > Fix this to generate completions on the right CQ. > > [ 863.969340] INFO: task kworker/u52:2:671 blocked for more than 122 seconds. > [ 863.979224] Not tainted 5.14.0-130.el9.x86_64 #1 > [ 863.986588] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. > [ 863.996997] task:kworker/u52:2 state:D stack: 0 pid: 671 ppid: 2 flags:0x00004000 > [ 864.007272] Workqueue: xprtiod xprt_autoclose [sunrpc] > [ 864.014056] Call Trace: > [ 864.017575] __schedule+0x206/0x580 > [ 864.022296] schedule+0x43/0xa0 > [ 864.026736] schedule_timeout+0x115/0x150 > [ 864.032185] __wait_for_common+0x93/0x1d0 > [ 864.037717] ? usleep_range_state+0x90/0x90 > [ 864.043368] __ib_drain_sq+0xf6/0x170 [ib_core] > [ 864.049371] ? __rdma_block_iter_next+0x80/0x80 [ib_core] > [ 864.056240] ib_drain_sq+0x66/0x70 [ib_core] > [ 864.062003] rpcrdma_xprt_disconnect+0x82/0x3b0 [rpcrdma] > [ 864.069365] ? xprt_prepare_transmit+0x5d/0xc0 [sunrpc] > [ 864.076386] xprt_rdma_close+0xe/0x30 [rpcrdma] > [ 864.082593] xprt_autoclose+0x52/0x100 [sunrpc] > [ 864.088718] process_one_work+0x1e8/0x3c0 > [ 864.094170] worker_thread+0x50/0x3b0 > [ 864.099109] ? rescuer_thread+0x370/0x370 > [ 864.104473] kthread+0x149/0x170 > [ 864.109022] ? set_kthread_struct+0x40/0x40 > [ 864.114713] ret_from_fork+0x22/0x30 > > Fixes: 81091d7696ae ("RDMA/irdma: Add SW mechanism to generate completions on error") > Reported-by: Kamal Heib <kamalheib1@gmail.com> > Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com> > --- > drivers/infiniband/hw/irdma/utils.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > Thanks, applied.
diff --git a/drivers/infiniband/hw/irdma/utils.c b/drivers/infiniband/hw/irdma/utils.c index fdf4cc8..c8b9235 100644 --- a/drivers/infiniband/hw/irdma/utils.c +++ b/drivers/infiniband/hw/irdma/utils.c @@ -2598,7 +2598,7 @@ void irdma_generate_flush_completions(struct irdma_qp *iwqp) spin_unlock_irqrestore(&iwqp->lock, flags2); spin_unlock_irqrestore(&iwqp->iwscq->lock, flags1); if (compl_generated) - irdma_comp_handler(iwqp->iwrcq); + irdma_comp_handler(iwqp->iwscq); } else { spin_unlock_irqrestore(&iwqp->iwscq->lock, flags1); mod_delayed_work(iwqp->iwdev->cleanup_wq, &iwqp->dwork_flush,
SW generated completions for outstanding WRs posted on SQ after QP is in error target the wrong CQ. This causes the ib_drain_sq to hang with no completion. Fix this to generate completions on the right CQ. [ 863.969340] INFO: task kworker/u52:2:671 blocked for more than 122 seconds. [ 863.979224] Not tainted 5.14.0-130.el9.x86_64 #1 [ 863.986588] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [ 863.996997] task:kworker/u52:2 state:D stack: 0 pid: 671 ppid: 2 flags:0x00004000 [ 864.007272] Workqueue: xprtiod xprt_autoclose [sunrpc] [ 864.014056] Call Trace: [ 864.017575] __schedule+0x206/0x580 [ 864.022296] schedule+0x43/0xa0 [ 864.026736] schedule_timeout+0x115/0x150 [ 864.032185] __wait_for_common+0x93/0x1d0 [ 864.037717] ? usleep_range_state+0x90/0x90 [ 864.043368] __ib_drain_sq+0xf6/0x170 [ib_core] [ 864.049371] ? __rdma_block_iter_next+0x80/0x80 [ib_core] [ 864.056240] ib_drain_sq+0x66/0x70 [ib_core] [ 864.062003] rpcrdma_xprt_disconnect+0x82/0x3b0 [rpcrdma] [ 864.069365] ? xprt_prepare_transmit+0x5d/0xc0 [sunrpc] [ 864.076386] xprt_rdma_close+0xe/0x30 [rpcrdma] [ 864.082593] xprt_autoclose+0x52/0x100 [sunrpc] [ 864.088718] process_one_work+0x1e8/0x3c0 [ 864.094170] worker_thread+0x50/0x3b0 [ 864.099109] ? rescuer_thread+0x370/0x370 [ 864.104473] kthread+0x149/0x170 [ 864.109022] ? set_kthread_struct+0x40/0x40 [ 864.114713] ret_from_fork+0x22/0x30 Fixes: 81091d7696ae ("RDMA/irdma: Add SW mechanism to generate completions on error") Reported-by: Kamal Heib <kamalheib1@gmail.com> Signed-off-by: Shiraz Saleem <shiraz.saleem@intel.com> --- drivers/infiniband/hw/irdma/utils.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)