From patchwork Tue Apr 22 13:16:37 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Bart Van Assche X-Patchwork-Id: 4031821 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork1.web.kernel.org (Postfix) with ESMTP id 2EBB09F1F4 for ; Tue, 22 Apr 2014 13:17:02 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 0AD06201DE for ; Tue, 22 Apr 2014 13:16:58 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id BB940201EF for ; Tue, 22 Apr 2014 13:16:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752026AbaDVNQs (ORCPT ); Tue, 22 Apr 2014 09:16:48 -0400 Received: from smtp03.stone-is.org ([87.238.162.6]:55406 "EHLO smtpgw.stone-is.be" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755864AbaDVNQm (ORCPT ); Tue, 22 Apr 2014 09:16:42 -0400 Received: from localhost (unknown [127.0.0.1]) by smtpgw.stone-is.be (Postfix) with ESMTP id 63C62335783; Tue, 22 Apr 2014 13:16:40 +0000 (UTC) Received: from smtpgw.stone-is.be ([127.0.0.1]) by localhost (smtpgw.stone-is.be [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id TkJS5F8sWSCZ; Tue, 22 Apr 2014 15:16:38 +0200 (CEST) Received: from vz19.stone-is.net (vz19.stone-is.net [87.238.162.57]) by smtpgw.stone-is.be (Postfix) with ESMTP id 6BFAC3357B7; Tue, 22 Apr 2014 15:16:38 +0200 (CEST) X-No-Relay: not in my network X-No-Relay: not in my network X-No-Relay: not in my network Received: from [192.168.1.117] (178-119-65-67.access.telenet.be [178.119.65.67]) by vz19.stone-is.net (Postfix) with ESMTPSA id 1A44E8922E; Tue, 22 Apr 2014 15:16:38 +0200 (CEST) Message-ID: <53566BB5.5030203@acm.org> Date: Tue, 22 Apr 2014 15:16:37 +0200 From: Bart Van Assche User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.3.0 MIME-Version: 1.0 To: Roland Dreier CC: Sean Hefty , linux-rdma Subject: IB/cma: Make timeout dependent on the subnet timeout X-Enigmail-Version: 1.6 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-7.5 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The default RDMA/CM timeout and retry values are too large for small IB networks and make the SRP initiator reconnect mechanism unnecessary slow. Hence make the CM timeout dependent on the subnet timeout in IB networks. Signed-off-by: Bart Van Assche Cc: Sean Hefty --- drivers/infiniband/core/cma.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c index 199958d..95528e1 100644 --- a/drivers/infiniband/core/cma.c +++ b/drivers/infiniband/core/cma.c @@ -60,7 +60,6 @@ MODULE_AUTHOR("Sean Hefty"); MODULE_DESCRIPTION("Generic RDMA CM Agent"); MODULE_LICENSE("Dual BSD/GPL"); -#define CMA_CM_RESPONSE_TIMEOUT 20 #define CMA_MAX_CM_RETRIES 15 #define CMA_CM_MRA_SETTING (IB_CM_MRA_FLAG_DELAY | 24) #define CMA_IBOE_PACKET_LIFETIME 18 @@ -2728,6 +2727,15 @@ out: return ret; } +static u8 cma_get_ib_subnet_timeout(struct rdma_cm_id *id) +{ + struct ib_port_attr attr; + int ret; + + ret = ib_query_port(id->device, id->port_num, &attr); + return ret == 0 ? attr.subnet_timeout : 18; +} + static int cma_resolve_ib_udp(struct rdma_id_private *id_priv, struct rdma_conn_param *conn_param) { @@ -2735,6 +2743,7 @@ static int cma_resolve_ib_udp(struct rdma_id_private *id_priv, struct ib_cm_id *id; void *private_data; int offset, ret; + u8 cm_response_timeout = cma_get_ib_subnet_timeout(&id_priv->id) + 2; memset(&req, 0, sizeof req); offset = cma_user_data_offset(id_priv); @@ -2771,7 +2780,7 @@ static int cma_resolve_ib_udp(struct rdma_id_private *id_priv, req.path = id_priv->id.route.path_rec; req.service_id = rdma_get_service_id(&id_priv->id, cma_dst_addr(id_priv)); - req.timeout_ms = 1 << (CMA_CM_RESPONSE_TIMEOUT - 8); + req.timeout_ms = 1 << max(cm_response_timeout - 8, 0); req.max_cm_retries = CMA_MAX_CM_RETRIES; ret = ib_send_cm_sidr_req(id_priv->cm_id.ib, &req); @@ -2792,6 +2801,7 @@ static int cma_connect_ib(struct rdma_id_private *id_priv, void *private_data; struct ib_cm_id *id; int offset, ret; + u8 cm_response_timeout = cma_get_ib_subnet_timeout(&id_priv->id) + 2; memset(&req, 0, sizeof req); offset = cma_user_data_offset(id_priv); @@ -2839,8 +2849,8 @@ static int cma_connect_ib(struct rdma_id_private *id_priv, req.flow_control = conn_param->flow_control; req.retry_count = min_t(u8, 7, conn_param->retry_count); req.rnr_retry_count = min_t(u8, 7, conn_param->rnr_retry_count); - req.remote_cm_response_timeout = CMA_CM_RESPONSE_TIMEOUT; - req.local_cm_response_timeout = CMA_CM_RESPONSE_TIMEOUT; + req.remote_cm_response_timeout = cm_response_timeout; + req.local_cm_response_timeout = cm_response_timeout; req.max_cm_retries = CMA_MAX_CM_RETRIES; req.srq = id_priv->srq ? 1 : 0;