From patchwork Fri Dec 3 16:49:53 2010 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Heinz X-Patchwork-Id: 378551 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by demeter1.kernel.org (8.14.4/8.14.3) with ESMTP id oB3GngDj001100 for ; Fri, 3 Dec 2010 16:49:59 GMT Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752026Ab0LCQt6 (ORCPT ); Fri, 3 Dec 2010 11:49:58 -0500 Received: from avexcashub1.qlogic.com ([198.70.193.61]:58000 "EHLO avexcashub1.qlogic.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751759Ab0LCQt6 (ORCPT ); Fri, 3 Dec 2010 11:49:58 -0500 Received: from avexcashub2.qlogic.org (10.1.4.116) by avexcashub1.qlogic.org (10.1.4.161) with Microsoft SMTP Server (TLS) id 8.1.436.0; Fri, 3 Dec 2010 08:49:58 -0800 Received: from MNEXCASHUB2.qlogic.org (10.33.2.104) by avexcashub2.qlogic.org (10.1.4.162) with Microsoft SMTP Server (TLS) id 8.1.436.0; Fri, 3 Dec 2010 08:49:57 -0800 Received: from MNEXMB1.qlogic.org ([fe80::8516:7839:9549:6996]) by MNEXCASHUB2.qlogic.org ([::1]) with mapi; Fri, 3 Dec 2010 10:49:56 -0600 From: Mike Heinz To: "linux-rdma@vger.kernel.org" Date: Fri, 3 Dec 2010 10:49:53 -0600 Subject: [PATCH V4] SA Busy Handling Thread-Topic: [PATCH V4] SA Busy Handling Thread-Index: AcuTChutgLmL2jB9SlWPpEn2zWDyWg== Message-ID: <4C2744E8AD2982428C5BFE523DF8CDCB4A208DF3AF@MNEXMB1.qlogic.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: yes X-MS-TNEF-Correlator: acceptlanguage: en-US MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Greylist: IP, sender and recipient auto-whitelisted, not delayed by milter-greylist-4.2.3 (demeter1.kernel.org [140.211.167.41]); Fri, 03 Dec 2010 16:49:59 +0000 (UTC) diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c index 64e660c..2e103ed 100644 --- a/drivers/infiniband/core/mad.c +++ b/drivers/infiniband/core/mad.c @@ -54,6 +54,10 @@ MODULE_PARM_DESC(send_queue_size, "Size of send queue in number of work requests module_param_named(recv_queue_size, mad_recvq_size, int, 0444); MODULE_PARM_DESC(recv_queue_size, "Size of receive queue in number of work requests"); +int mad_wait_on_busy = 0; +module_param_named(treat_busy_as_timeout, mad_wait_on_busy, int, 0444); +MODULE_PARM_DESC(treat_busy_as_timeout, "When true, treat BUSY responses as if they were timeouts."); + static struct kmem_cache *ib_mad_cache; static struct list_head ib_mad_port_list; @@ -1120,6 +1124,7 @@ int ib_post_send_mad(struct ib_mad_send_buf *send_buf, mad_send_wr->timeout = msecs_to_jiffies(send_buf->timeout_ms); mad_send_wr->max_retries = send_buf->retries; mad_send_wr->retries_left = send_buf->retries; + mad_send_wr->wait_on_busy = send_buf->wait_on_busy || mad_wait_on_busy; send_buf->retries = 0; /* Reference for work request to QP + response */ mad_send_wr->refcount = 1 + (mad_send_wr->timeout > 0); @@ -1828,6 +1833,9 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, /* Complete corresponding request */ if (ib_response_mad(mad_recv_wc->recv_buf.mad)) { + u16 busy = be16_to_cpu(mad_recv_wc->recv_buf.mad->mad_hdr.status) & + IB_MGMT_MAD_STATUS_BUSY; + spin_lock_irqsave(&mad_agent_priv->lock, flags); mad_send_wr = ib_find_send_mad(mad_agent_priv, mad_recv_wc); if (!mad_send_wr) { @@ -1836,6 +1844,20 @@ static void ib_mad_complete_recv(struct ib_mad_agent_private *mad_agent_priv, deref_mad_agent(mad_agent_priv); return; } + + printk(KERN_DEBUG PFX "Completing recv %p: busy = %d, retries_left = %d, wait_on_busy = %d\n", + mad_send_wr, busy, mad_send_wr->retries_left, mad_send_wr->wait_on_busy); + + if (busy && mad_send_wr->retries_left && mad_send_wr->wait_on_busy && + (mad_recv_wc->recv_buf.mad->mad_hdr.method != IB_MGMT_METHOD_TRAP_REPRESS)) { + /* Just let the query timeout and have it requeued later */ + spin_unlock_irqrestore(&mad_agent_priv->lock, flags); + ib_free_recv_mad(mad_recv_wc); + deref_mad_agent(mad_agent_priv); + printk(KERN_INFO PFX "SA/SM responded MAD_STATUS_BUSY. Allowing request to time out.\n"); + return; + } + ib_mark_mad_done(mad_send_wr); spin_unlock_irqrestore(&mad_agent_priv->lock, flags); diff --git a/drivers/infiniband/core/mad_priv.h b/drivers/infiniband/core/mad_priv.h index 8b4df0a..b6477cf 100644 --- a/drivers/infiniband/core/mad_priv.h +++ b/drivers/infiniband/core/mad_priv.h @@ -135,6 +135,7 @@ struct ib_mad_send_wr_private { unsigned long timeout; int max_retries; int retries_left; + int wait_on_busy; int retry; int refcount; enum ib_wc_status status; diff --git a/include/rdma/ib_mad.h b/include/rdma/ib_mad.h index d3b9401..fadc2c3 100644 --- a/include/rdma/ib_mad.h +++ b/include/rdma/ib_mad.h @@ -77,6 +77,15 @@ #define IB_MGMT_MAX_METHODS 128 +/* MAD Status field bit masks */ +#define IB_MGMT_MAD_STATUS_SUCCESS 0x0000 +#define IB_MGMT_MAD_STATUS_BUSY 0x0001 +#define IB_MGMT_MAD_STATUS_REDIRECT_REQD 0x0002 +#define IB_MGMT_MAD_STATUS_BAD_VERSION 0x0004 +#define IB_MGMT_MAD_STATUS_UNSUPPORTED_METHOD 0x0008 +#define IB_MGMT_MAD_STATUS_UNSUPPORTED_METHOD_ATTRIB 0x000c +#define IB_MGMT_MAD_STATUS_INVALID_ATTRIB_VALUE 0x001c + /* RMPP information */ #define IB_MGMT_RMPP_VERSION 1 @@ -246,6 +255,7 @@ struct ib_mad_send_buf { int seg_count; int seg_size; int timeout_ms; + int wait_on_busy; int retries; };