| Message ID | 162a5429342dfd5e16881f4360c0ef858fd39cbd.1455741336.git.swise@chelsio.com (mailing list archive) |
|---|---|
| State | Accepted |
| Headers | show |
Looks good,
Reviewed-by: Sagi Grimberg <sagig@mellanox.com>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Feb 17, 2016 at 08:15:42AM -0800, Steve Wise wrote: > Add completion objects, named sq_drained and rq_drained, to the c4iw_qp > struct. The queue-specific completion object is signaled when the last > CQE is drained from the CQ for that queue. > > Add c4iw_drain_sq() to block until qp->rq_drained is completed. > > Add c4iw_drain_rq() to block until qp->sq_drained is completed. Hi Steve, My knowledge is limited in this driver and I have no illusion that question sounds naive, but why did you add these functions to this specific driver? Do the other drivers need such update too? Thanks -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
> On Wed, Feb 17, 2016 at 08:15:42AM -0800, Steve Wise wrote: > > Add completion objects, named sq_drained and rq_drained, to the c4iw_qp > > struct. The queue-specific completion object is signaled when the last > > CQE is drained from the CQ for that queue. > > > > Add c4iw_drain_sq() to block until qp->rq_drained is completed. > > > > Add c4iw_drain_rq() to block until qp->sq_drained is completed. > > Hi Steve, > My knowledge is limited in this driver and I have no illusion that > question sounds naive, but why did you add these functions to this > specific driver? Do the other drivers need such update too? > > Thanks Hey Leon, I added these so iSER and other storage protocols that are or will be using the drain logic will work on cxgb4. IB devices should not need device-specific drain logic since the common core drain works for IB (and I assume RoCE). iWARP devices, however, require device-specific logic. Other iwarp drivers are iw_cxgb3 and iw_nes. Steve. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
>> Hi Steve, >> My knowledge is limited in this driver and I have no illusion that >> question sounds naive, but why did you add these functions to this >> specific driver? Do the other drivers need such update too? >> >> Thanks > > Hey Leon, > > I added these so iSER and other storage protocols that are or will be using the drain logic will work on cxgb4. IB devices should > not need device-specific drain logic since the common core drain works for IB (and I assume RoCE). iWARP devices, however, require > device-specific logic. Other iwarp drivers are iw_cxgb3 and iw_nes. Yes, this is iWARP specific. Steve was kind enough to fix chelsio drivers and we need someone to pick up nes so ULPs will be able to run reliably over it. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Wed, Mar 09, 2016 at 05:28:47PM +0200, Sagi Grimberg wrote: > > >>Hi Steve, > >>My knowledge is limited in this driver and I have no illusion that > >>question sounds naive, but why did you add these functions to this > >>specific driver? Do the other drivers need such update too? > >> > >>Thanks > > > >Hey Leon, > > > >I added these so iSER and other storage protocols that are or will be using the drain logic will work on cxgb4. IB devices should > >not need device-specific drain logic since the common core drain works for IB (and I assume RoCE). iWARP devices, however, require > >device-specific logic. Other iwarp drivers are iw_cxgb3 and iw_nes. > > Yes, this is iWARP specific. Steve was kind enough to fix chelsio > drivers and we need someone to pick up nes so ULPs will be able to run > reliably over it. Steve and Sagi, Thank you for clarification. I see that nes driver is supported. Is it still relevant? NETEFFECT IWARP RNIC DRIVER (IW_NES) M: Faisal Latif <faisal.latif@intel.com> L: linux-rdma@vger.kernel.org W: http://www.intel.com/Products/Server/Adapters/Server-Cluster/Server-Cluster-overview.htm S: Supported F: drivers/infiniband/hw/nes/ -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
ping On Thu, Mar 10, 2016 at 07:05:58AM +0200, Leon Romanovsky wrote: > On Wed, Mar 09, 2016 at 05:28:47PM +0200, Sagi Grimberg wrote: > > > > Yes, this is iWARP specific. Steve was kind enough to fix chelsio > > drivers and we need someone to pick up nes so ULPs will be able to run > > reliably over it. > > Steve and Sagi, > Thank you for clarification. > > I see that nes driver is supported. Is it still relevant? > > NETEFFECT IWARP RNIC DRIVER (IW_NES) > M: Faisal Latif <faisal.latif@intel.com> > L: linux-rdma@vger.kernel.org > W: http://www.intel.com/Products/Server/Adapters/Server-Cluster/Server-Cluster-overview.htm > S: Supported > F: drivers/infiniband/hw/nes/ -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
On Thu, Mar 24, 2016 at 12:46:59PM +0200, Leon Romanovsky wrote: > ping > > On Thu, Mar 10, 2016 at 07:05:58AM +0200, Leon Romanovsky wrote: > > On Wed, Mar 09, 2016 at 05:28:47PM +0200, Sagi Grimberg wrote: > > > > > > Yes, this is iWARP specific. Steve was kind enough to fix chelsio > > > drivers and we need someone to pick up nes so ULPs will be able to run > > > reliably over it. > > > > Steve and Sagi, > > Thank you for clarification. > > > > I see that nes driver is supported. Is it still relevant? > > Sorry as I was on vacation and will get the patch for nes ready. Thanks Faisal -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/drivers/infiniband/hw/cxgb4/cq.c b/drivers/infiniband/hw/cxgb4/cq.c index cf21df4..b4eeb78 100644 --- a/drivers/infiniband/hw/cxgb4/cq.c +++ b/drivers/infiniband/hw/cxgb4/cq.c @@ -815,8 +815,15 @@ static int c4iw_poll_cq_one(struct c4iw_cq *chp, struct ib_wc *wc) } } out: - if (wq) + if (wq) { + if (unlikely(qhp->attr.state != C4IW_QP_STATE_RTS)) { + if (t4_sq_empty(wq)) + complete(&qhp->sq_drained); + if (t4_rq_empty(wq)) + complete(&qhp->rq_drained); + } spin_unlock(&qhp->lock); + } return ret; } diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h index fb2de75..7c6a6e1 100644 --- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h +++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h @@ -476,6 +476,8 @@ struct c4iw_qp { wait_queue_head_t wait; struct timer_list timer; int sq_sig_all; + struct completion rq_drained; + struct completion sq_drained; }; static inline struct c4iw_qp *to_c4iw_qp(struct ib_qp *ibqp) @@ -1016,6 +1018,8 @@ extern int c4iw_wr_log; extern int db_fc_threshold; extern int db_coalescing_threshold; extern int use_dsgl; +void c4iw_drain_rq(struct ib_qp *qp); +void c4iw_drain_sq(struct ib_qp *qp); #endif diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c index ec04272..104662d 100644 --- a/drivers/infiniband/hw/cxgb4/provider.c +++ b/drivers/infiniband/hw/cxgb4/provider.c @@ -564,6 +564,8 @@ int c4iw_register_device(struct c4iw_dev *dev) dev->ibdev.get_protocol_stats = c4iw_get_mib; dev->ibdev.uverbs_abi_ver = C4IW_UVERBS_ABI_VERSION; dev->ibdev.get_port_immutable = c4iw_port_immutable; + dev->ibdev.drain_sq = c4iw_drain_sq; + dev->ibdev.drain_rq = c4iw_drain_rq; dev->ibdev.iwcm = kmalloc(sizeof(struct iw_cm_verbs), GFP_KERNEL); if (!dev->ibdev.iwcm) diff --git a/drivers/infiniband/hw/cxgb4/qp.c b/drivers/infiniband/hw/cxgb4/qp.c index e99345e..7b1b1e8 100644 --- a/drivers/infiniband/hw/cxgb4/qp.c +++ b/drivers/infiniband/hw/cxgb4/qp.c @@ -1697,6 +1697,8 @@ struct ib_qp *c4iw_create_qp(struct ib_pd *pd, struct ib_qp_init_attr *attrs, qhp->attr.max_ird = 0; qhp->sq_sig_all = attrs->sq_sig_type == IB_SIGNAL_ALL_WR; spin_lock_init(&qhp->lock); + init_completion(&qhp->sq_drained); + init_completion(&qhp->rq_drained); mutex_init(&qhp->mutex); init_waitqueue_head(&qhp->wait); atomic_set(&qhp->refcnt, 1); @@ -1888,3 +1890,17 @@ int c4iw_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, init_attr->sq_sig_type = qhp->sq_sig_all ? IB_SIGNAL_ALL_WR : 0; return 0; } + +void c4iw_drain_sq(struct ib_qp *ibqp) +{ + struct c4iw_qp *qp = to_c4iw_qp(ibqp); + + wait_for_completion(&qp->sq_drained); +} + +void c4iw_drain_rq(struct ib_qp *ibqp) +{ + struct c4iw_qp *qp = to_c4iw_qp(ibqp); + + wait_for_completion(&qp->rq_drained); +}
Add completion objects, named sq_drained and rq_drained, to the c4iw_qp struct. The queue-specific completion object is signaled when the last CQE is drained from the CQ for that queue. Add c4iw_drain_sq() to block until qp->rq_drained is completed. Add c4iw_drain_rq() to block until qp->sq_drained is completed. Signed-off-by: Steve Wise <swise@opengridcomputing.com> --- drivers/infiniband/hw/cxgb4/cq.c | 9 ++++++++- drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 4 ++++ drivers/infiniband/hw/cxgb4/provider.c | 2 ++ drivers/infiniband/hw/cxgb4/qp.c | 16 ++++++++++++++++ 4 files changed, 30 insertions(+), 1 deletion(-)