Message ID | 0-v2-53c22d5c1405+33-rdma_connect_locking_jgg@nvidia.com (mailing list archive) |
---|---|
State | Accepted |
Commit | 53c22d5c1405aa0764142a97bbb65779e605023d |
Delegated to: | Jason Gunthorpe |
Headers | show |
Series | [rdma,v2] RDMA: Add rdma_connect_locked() | expand |
On 10/27/2020 2:20 PM, Jason Gunthorpe wrote: > There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the > handler triggers a completion and another thread does rdma_connect() or > the handler directly calls rdma_connect(). > > In all cases rdma_connect() needs to hold the handler_mutex, but when > handler's are invoked this is already held by the core code. This causes > ULPs using the 2nd method to deadlock. > > Provide a rdma_connect_locked() and have all ULPs call it from their > handlers. > > Link: https://lore.kernel.org/r/0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com > Reported-and-tested-by: Guoqing Jiang <guoqing.jiang@cloud.ionos.com> > Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state") > Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com> > Acked-by: Jack Wang <jinpu.wang@cloud.ionos.com> > Reviewed-by: Christoph Hellwig <hch@lst.de> > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> > --- > drivers/infiniband/core/cma.c | 40 +++++++++++++++++++++--- > drivers/infiniband/ulp/iser/iser_verbs.c | 2 +- > drivers/infiniband/ulp/rtrs/rtrs-clt.c | 4 +-- > drivers/nvme/host/rdma.c | 4 +-- > include/rdma/rdma_cm.h | 14 ++------- > net/rds/ib_cm.c | 5 +-- > 6 files changed, 46 insertions(+), 23 deletions(-) > > v2: > - Remove extra code from nvme (Chao) > - Fix long lines (CH) > > I've applied this version to rdma-rc - expecting to get these ULPs unbroken for rc2 > release > > Thanks, > Jason > iser and nvme/rdma looks good to me, Reviewed-by: Max Gurtovoy <mgurtovoy@nvidia.com>
On Tue, Oct 27, 2020 at 09:20:36AM -0300, Jason Gunthorpe wrote: > There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the > handler triggers a completion and another thread does rdma_connect() or > the handler directly calls rdma_connect(). > > In all cases rdma_connect() needs to hold the handler_mutex, but when > handler's are invoked this is already held by the core code. This causes > ULPs using the 2nd method to deadlock. > > Provide a rdma_connect_locked() and have all ULPs call it from their > handlers. > > Link: https://lore.kernel.org/r/0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com > Reported-and-tested-by: Guoqing Jiang <guoqing.jiang@cloud.ionos.com> > Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state") > Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com> > Acked-by: Jack Wang <jinpu.wang@cloud.ionos.com> > Reviewed-by: Christoph Hellwig <hch@lst.de> > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> > --- > drivers/infiniband/core/cma.c | 40 +++++++++++++++++++++--- > drivers/infiniband/ulp/iser/iser_verbs.c | 2 +- > drivers/infiniband/ulp/rtrs/rtrs-clt.c | 4 +-- > drivers/nvme/host/rdma.c | 4 +-- > include/rdma/rdma_cm.h | 14 ++------- > net/rds/ib_cm.c | 5 +-- > 6 files changed, 46 insertions(+), 23 deletions(-) > > v2: > - Remove extra code from nvme (Chao) > - Fix long lines (CH) > > I've applied this version to rdma-rc - expecting to get these ULPs unbroken for rc2 > release > > Thanks, > Jason > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c > index 7c2ab1f2fbea37..193c8902b9db26 100644 > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > @@ -405,10 +405,10 @@ static int cma_comp_exch(struct rdma_id_private *id_priv, > /* > * The FSM uses a funny double locking where state is protected by both > * the handler_mutex and the spinlock. State is not allowed to change > - * away from a handler_mutex protected value without also holding > + * to/from a handler_mutex protected value without also holding > * handler_mutex. > */ > - if (comp == RDMA_CM_CONNECT) > + if (comp == RDMA_CM_CONNECT || exch == RDMA_CM_CONNECT) > lockdep_assert_held(&id_priv->handler_mutex); > > spin_lock_irqsave(&id_priv->lock, flags); > @@ -4038,13 +4038,21 @@ static int cma_connect_iw(struct rdma_id_private *id_priv, > return ret; > } > > -int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) > +/** > + * rdma_connect_locked - Initiate an active connection request. > + * @id: Connection identifier to connect. > + * @conn_param: Connection information used for connected QPs. > + * > + * Same as rdma_connect() but can only be called from the > + * RDMA_CM_EVENT_ROUTE_RESOLVED handler callback. > + */ > +int rdma_connect_locked(struct rdma_cm_id *id, > + struct rdma_conn_param *conn_param) > { > struct rdma_id_private *id_priv = > container_of(id, struct rdma_id_private, id); > int ret; > > - mutex_lock(&id_priv->handler_mutex); > if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) { > ret = -EINVAL; > goto err_unlock; Not a big deal, but his label is not correct anymore. Thanks
On Tue, Oct 27, 2020 at 03:19:36PM +0200, Leon Romanovsky wrote: > > +int rdma_connect_locked(struct rdma_cm_id *id, > > + struct rdma_conn_param *conn_param) > > { > > struct rdma_id_private *id_priv = > > container_of(id, struct rdma_id_private, id); > > int ret; > > > > - mutex_lock(&id_priv->handler_mutex); > > if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) { > > ret = -EINVAL; > > goto err_unlock; > > Not a big deal, but his label is not correct anymore. Oh, yep diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 193c8902b9db26..f58d19881524dc 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -4053,10 +4053,8 @@ int rdma_connect_locked(struct rdma_cm_id *id, container_of(id, struct rdma_id_private, id); int ret; - if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) { - ret = -EINVAL; - goto err_unlock; - } + if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) + return -EINVAL; if (!id->qp) { id_priv->qp_num = conn_param->qp_num; @@ -4078,7 +4076,6 @@ int rdma_connect_locked(struct rdma_cm_id *id, return 0; err_state: cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED); -err_unlock: return ret; } EXPORT_SYMBOL(rdma_connect_locked);
Reviewed-by: Sagi Grimberg <sagi@grimberg.me>
On 10/27/2020 2:20 PM, Jason Gunthorpe wrote: > There are two flows for handling RDMA_CM_EVENT_ROUTE_RESOLVED, either the > handler triggers a completion and another thread does rdma_connect() or > the handler directly calls rdma_connect(). > > In all cases rdma_connect() needs to hold the handler_mutex, but when > handler's are invoked this is already held by the core code. This causes > ULPs using the 2nd method to deadlock. > > Provide a rdma_connect_locked() and have all ULPs call it from their > handlers. > > Link: https://lore.kernel.org/r/0-v1-75e124dbad74+b05-rdma_connect_locking_jgg@nvidia.com > Reported-and-tested-by: Guoqing Jiang <guoqing.jiang@cloud.ionos.com> > Fixes: 2a7cec538169 ("RDMA/cma: Fix locking for the RDMA_CM_CONNECT state") > Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com> > Acked-by: Jack Wang <jinpu.wang@cloud.ionos.com> > Reviewed-by: Christoph Hellwig <hch@lst.de> > Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> > --- > drivers/infiniband/core/cma.c | 40 +++++++++++++++++++++--- > drivers/infiniband/ulp/iser/iser_verbs.c | 2 +- > drivers/infiniband/ulp/rtrs/rtrs-clt.c | 4 +-- > drivers/nvme/host/rdma.c | 4 +-- > include/rdma/rdma_cm.h | 14 ++------- > net/rds/ib_cm.c | 5 +-- > 6 files changed, 46 insertions(+), 23 deletions(-) > > v2: > - Remove extra code from nvme (Chao) > - Fix long lines (CH) > > I've applied this version to rdma-rc - expecting to get these ULPs unbroken for rc2 > release > > Thanks, > Jason > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c > index 7c2ab1f2fbea37..193c8902b9db26 100644 > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > @@ -405,10 +405,10 @@ static int cma_comp_exch(struct rdma_id_private *id_priv, > /* > * The FSM uses a funny double locking where state is protected by both > * the handler_mutex and the spinlock. State is not allowed to change > - * away from a handler_mutex protected value without also holding > + * to/from a handler_mutex protected value without also holding > * handler_mutex. > */ > - if (comp == RDMA_CM_CONNECT) > + if (comp == RDMA_CM_CONNECT || exch == RDMA_CM_CONNECT) > lockdep_assert_held(&id_priv->handler_mutex); > > spin_lock_irqsave(&id_priv->lock, flags); > @@ -4038,13 +4038,21 @@ static int cma_connect_iw(struct rdma_id_private *id_priv, > return ret; > } > > -int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) > +/** > + * rdma_connect_locked - Initiate an active connection request. > + * @id: Connection identifier to connect. > + * @conn_param: Connection information used for connected QPs. > + * > + * Same as rdma_connect() but can only be called from the > + * RDMA_CM_EVENT_ROUTE_RESOLVED handler callback. > + */ > +int rdma_connect_locked(struct rdma_cm_id *id, > + struct rdma_conn_param *conn_param) > { > struct rdma_id_private *id_priv = > container_of(id, struct rdma_id_private, id); > int ret; > > - mutex_lock(&id_priv->handler_mutex); You need to delete the mutex_unlock in success path too. > if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) { > ret = -EINVAL; > goto err_unlock; > @@ -4071,6 +4079,30 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) > err_state: > cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED); > err_unlock: > + return ret; > +} > +EXPORT_SYMBOL(rdma_connect_locked); > + > +/** > + * rdma_connect - Initiate an active connection request. > + * @id: Connection identifier to connect. > + * @conn_param: Connection information used for connected QPs. > + * > + * Users must have resolved a route for the rdma_cm_id to connect with by having > + * called rdma_resolve_route before calling this routine. > + * > + * This call will either connect to a remote QP or obtain remote QP information > + * for unconnected rdma_cm_id's. The actual operation is based on the > + * rdma_cm_id's port space. > + */ > +int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) > +{ > + struct rdma_id_private *id_priv = > + container_of(id, struct rdma_id_private, id); > + int ret; > + > + mutex_lock(&id_priv->handler_mutex); > + ret = rdma_connect_locked(id, conn_param); > mutex_unlock(&id_priv->handler_mutex); > return ret; > } > diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c > index 2f3ebc0a75d924..2bd18b00689341 100644 > --- a/drivers/infiniband/ulp/iser/iser_verbs.c > +++ b/drivers/infiniband/ulp/iser/iser_verbs.c > @@ -620,7 +620,7 @@ static void iser_route_handler(struct rdma_cm_id *cma_id) > conn_param.private_data = (void *)&req_hdr; > conn_param.private_data_len = sizeof(struct iser_cm_hdr); > > - ret = rdma_connect(cma_id, &conn_param); > + ret = rdma_connect_locked(cma_id, &conn_param); > if (ret) { > iser_err("failure connecting: %d\n", ret); > goto failure; > diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c > index 776e89231c52f7..f298adc02acba2 100644 > --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c > +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c > @@ -1674,9 +1674,9 @@ static int rtrs_rdma_route_resolved(struct rtrs_clt_con *con) > uuid_copy(&msg.sess_uuid, &sess->s.uuid); > uuid_copy(&msg.paths_uuid, &clt->paths_uuid); > > - err = rdma_connect(con->c.cm_id, ¶m); > + err = rdma_connect_locked(con->c.cm_id, ¶m); > if (err) > - rtrs_err(clt, "rdma_connect(): %d\n", err); > + rtrs_err(clt, "rdma_connect_locked(): %d\n", err); > > return err; > } > diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c > index aad829a2b50d0f..8bbc48cc45dc1d 100644 > --- a/drivers/nvme/host/rdma.c > +++ b/drivers/nvme/host/rdma.c > @@ -1890,10 +1890,10 @@ static int nvme_rdma_route_resolved(struct nvme_rdma_queue *queue) > priv.hsqsize = cpu_to_le16(queue->ctrl->ctrl.sqsize); > } > > - ret = rdma_connect(queue->cm_id, ¶m); > + ret = rdma_connect_locked(queue->cm_id, ¶m); > if (ret) { > dev_err(ctrl->ctrl.device, > - "rdma_connect failed (%d).\n", ret); > + "rdma_connect_locked failed (%d).\n", ret); > goto out_destroy_queue_ib; > } > > diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h > index c672ae1da26bb5..32a67af18415d6 100644 > --- a/include/rdma/rdma_cm.h > +++ b/include/rdma/rdma_cm.h > @@ -227,19 +227,9 @@ void rdma_destroy_qp(struct rdma_cm_id *id); > int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr, > int *qp_attr_mask); > > -/** > - * rdma_connect - Initiate an active connection request. > - * @id: Connection identifier to connect. > - * @conn_param: Connection information used for connected QPs. > - * > - * Users must have resolved a route for the rdma_cm_id to connect with > - * by having called rdma_resolve_route before calling this routine. > - * > - * This call will either connect to a remote QP or obtain remote QP > - * information for unconnected rdma_cm_id's. The actual operation is > - * based on the rdma_cm_id's port space. > - */ > int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param); > +int rdma_connect_locked(struct rdma_cm_id *id, > + struct rdma_conn_param *conn_param); > > int rdma_connect_ece(struct rdma_cm_id *id, struct rdma_conn_param *conn_param, > struct rdma_ucm_ece *ece); > diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c > index 06603dd1c8aa38..b36b60668b1da9 100644 > --- a/net/rds/ib_cm.c > +++ b/net/rds/ib_cm.c > @@ -956,9 +956,10 @@ int rds_ib_cm_initiate_connect(struct rdma_cm_id *cm_id, bool isv6) > rds_ib_cm_fill_conn_param(conn, &conn_param, &dp, > conn->c_proposed_version, > UINT_MAX, UINT_MAX, isv6); > - ret = rdma_connect(cm_id, &conn_param); > + ret = rdma_connect_locked(cm_id, &conn_param); > if (ret) > - rds_ib_conn_error(conn, "rdma_connect failed (%d)\n", ret); > + rds_ib_conn_error(conn, "rdma_connect_locked failed (%d)\n", > + ret); > > out: > /* Beware - returning non-zero tells the rdma_cm to destroy
On Wed, Oct 28, 2020 at 11:19:14AM +0200, Maor Gottlieb wrote: > > + struct rdma_conn_param *conn_param) > > { > > struct rdma_id_private *id_priv = > > container_of(id, struct rdma_id_private, id); > > int ret; > > - mutex_lock(&id_priv->handler_mutex); > > You need to delete the mutex_unlock in success path too. Gaaaaah. Just goes to prove I shouldn't write patches with a child on my lap :\ diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index f58d19881524dc..a77750b8954db0 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -4072,7 +4072,6 @@ int rdma_connect_locked(struct rdma_cm_id *id, ret = -ENOSYS; if (ret) goto err_state; - mutex_unlock(&id_priv->handler_mutex); return 0; err_state: cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED); Thanks, Jason
On 10/28/2020 2:14 PM, Jason Gunthorpe wrote: > On Wed, Oct 28, 2020 at 11:19:14AM +0200, Maor Gottlieb wrote: >>> + struct rdma_conn_param *conn_param) >>> { >>> struct rdma_id_private *id_priv = >>> container_of(id, struct rdma_id_private, id); >>> int ret; >>> - mutex_lock(&id_priv->handler_mutex); >> You need to delete the mutex_unlock in success path too. > Gaaaaah. Just goes to prove I shouldn't write patches with a child on > my lap :\ > > diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c > index f58d19881524dc..a77750b8954db0 100644 > --- a/drivers/infiniband/core/cma.c > +++ b/drivers/infiniband/core/cma.c > @@ -4072,7 +4072,6 @@ int rdma_connect_locked(struct rdma_cm_id *id, > ret = -ENOSYS; > if (ret) > goto err_state; > - mutex_unlock(&id_priv->handler_mutex); > return 0; > err_state: > cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED); > > Thanks, > Jason Reviewed-by: Maor Gottlieb <maorg@nvidia.com>
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 7c2ab1f2fbea37..193c8902b9db26 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -405,10 +405,10 @@ static int cma_comp_exch(struct rdma_id_private *id_priv, /* * The FSM uses a funny double locking where state is protected by both * the handler_mutex and the spinlock. State is not allowed to change - * away from a handler_mutex protected value without also holding + * to/from a handler_mutex protected value without also holding * handler_mutex. */ - if (comp == RDMA_CM_CONNECT) + if (comp == RDMA_CM_CONNECT || exch == RDMA_CM_CONNECT) lockdep_assert_held(&id_priv->handler_mutex); spin_lock_irqsave(&id_priv->lock, flags); @@ -4038,13 +4038,21 @@ static int cma_connect_iw(struct rdma_id_private *id_priv, return ret; } -int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) +/** + * rdma_connect_locked - Initiate an active connection request. + * @id: Connection identifier to connect. + * @conn_param: Connection information used for connected QPs. + * + * Same as rdma_connect() but can only be called from the + * RDMA_CM_EVENT_ROUTE_RESOLVED handler callback. + */ +int rdma_connect_locked(struct rdma_cm_id *id, + struct rdma_conn_param *conn_param) { struct rdma_id_private *id_priv = container_of(id, struct rdma_id_private, id); int ret; - mutex_lock(&id_priv->handler_mutex); if (!cma_comp_exch(id_priv, RDMA_CM_ROUTE_RESOLVED, RDMA_CM_CONNECT)) { ret = -EINVAL; goto err_unlock; @@ -4071,6 +4079,30 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) err_state: cma_comp_exch(id_priv, RDMA_CM_CONNECT, RDMA_CM_ROUTE_RESOLVED); err_unlock: + return ret; +} +EXPORT_SYMBOL(rdma_connect_locked); + +/** + * rdma_connect - Initiate an active connection request. + * @id: Connection identifier to connect. + * @conn_param: Connection information used for connected QPs. + * + * Users must have resolved a route for the rdma_cm_id to connect with by having + * called rdma_resolve_route before calling this routine. + * + * This call will either connect to a remote QP or obtain remote QP information + * for unconnected rdma_cm_id's. The actual operation is based on the + * rdma_cm_id's port space. + */ +int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param) +{ + struct rdma_id_private *id_priv = + container_of(id, struct rdma_id_private, id); + int ret; + + mutex_lock(&id_priv->handler_mutex); + ret = rdma_connect_locked(id, conn_param); mutex_unlock(&id_priv->handler_mutex); return ret; } diff --git a/drivers/infiniband/ulp/iser/iser_verbs.c b/drivers/infiniband/ulp/iser/iser_verbs.c index 2f3ebc0a75d924..2bd18b00689341 100644 --- a/drivers/infiniband/ulp/iser/iser_verbs.c +++ b/drivers/infiniband/ulp/iser/iser_verbs.c @@ -620,7 +620,7 @@ static void iser_route_handler(struct rdma_cm_id *cma_id) conn_param.private_data = (void *)&req_hdr; conn_param.private_data_len = sizeof(struct iser_cm_hdr); - ret = rdma_connect(cma_id, &conn_param); + ret = rdma_connect_locked(cma_id, &conn_param); if (ret) { iser_err("failure connecting: %d\n", ret); goto failure; diff --git a/drivers/infiniband/ulp/rtrs/rtrs-clt.c b/drivers/infiniband/ulp/rtrs/rtrs-clt.c index 776e89231c52f7..f298adc02acba2 100644 --- a/drivers/infiniband/ulp/rtrs/rtrs-clt.c +++ b/drivers/infiniband/ulp/rtrs/rtrs-clt.c @@ -1674,9 +1674,9 @@ static int rtrs_rdma_route_resolved(struct rtrs_clt_con *con) uuid_copy(&msg.sess_uuid, &sess->s.uuid); uuid_copy(&msg.paths_uuid, &clt->paths_uuid); - err = rdma_connect(con->c.cm_id, ¶m); + err = rdma_connect_locked(con->c.cm_id, ¶m); if (err) - rtrs_err(clt, "rdma_connect(): %d\n", err); + rtrs_err(clt, "rdma_connect_locked(): %d\n", err); return err; } diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c index aad829a2b50d0f..8bbc48cc45dc1d 100644 --- a/drivers/nvme/host/rdma.c +++ b/drivers/nvme/host/rdma.c @@ -1890,10 +1890,10 @@ static int nvme_rdma_route_resolved(struct nvme_rdma_queue *queue) priv.hsqsize = cpu_to_le16(queue->ctrl->ctrl.sqsize); } - ret = rdma_connect(queue->cm_id, ¶m); + ret = rdma_connect_locked(queue->cm_id, ¶m); if (ret) { dev_err(ctrl->ctrl.device, - "rdma_connect failed (%d).\n", ret); + "rdma_connect_locked failed (%d).\n", ret); goto out_destroy_queue_ib; } diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h index c672ae1da26bb5..32a67af18415d6 100644 --- a/include/rdma/rdma_cm.h +++ b/include/rdma/rdma_cm.h @@ -227,19 +227,9 @@ void rdma_destroy_qp(struct rdma_cm_id *id); int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr, int *qp_attr_mask); -/** - * rdma_connect - Initiate an active connection request. - * @id: Connection identifier to connect. - * @conn_param: Connection information used for connected QPs. - * - * Users must have resolved a route for the rdma_cm_id to connect with - * by having called rdma_resolve_route before calling this routine. - * - * This call will either connect to a remote QP or obtain remote QP - * information for unconnected rdma_cm_id's. The actual operation is - * based on the rdma_cm_id's port space. - */ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param); +int rdma_connect_locked(struct rdma_cm_id *id, + struct rdma_conn_param *conn_param); int rdma_connect_ece(struct rdma_cm_id *id, struct rdma_conn_param *conn_param, struct rdma_ucm_ece *ece); diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c index 06603dd1c8aa38..b36b60668b1da9 100644 --- a/net/rds/ib_cm.c +++ b/net/rds/ib_cm.c @@ -956,9 +956,10 @@ int rds_ib_cm_initiate_connect(struct rdma_cm_id *cm_id, bool isv6) rds_ib_cm_fill_conn_param(conn, &conn_param, &dp, conn->c_proposed_version, UINT_MAX, UINT_MAX, isv6); - ret = rdma_connect(cm_id, &conn_param); + ret = rdma_connect_locked(cm_id, &conn_param); if (ret) - rds_ib_conn_error(conn, "rdma_connect failed (%d)\n", ret); + rds_ib_conn_error(conn, "rdma_connect_locked failed (%d)\n", + ret); out: /* Beware - returning non-zero tells the rdma_cm to destroy