From patchwork Mon Jun 23 22:39:58 2014 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 4405261 Return-Path: X-Original-To: patchwork-linux-rdma@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id 05953BEEAA for ; Mon, 23 Jun 2014 22:40:06 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 3081B2018E for ; Mon, 23 Jun 2014 22:40:05 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 4C84C20225 for ; Mon, 23 Jun 2014 22:40:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752427AbaFWWkB (ORCPT ); Mon, 23 Jun 2014 18:40:01 -0400 Received: from mail-ig0-f180.google.com ([209.85.213.180]:55817 "EHLO mail-ig0-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750962AbaFWWkA (ORCPT ); Mon, 23 Jun 2014 18:40:00 -0400 Received: by mail-ig0-f180.google.com with SMTP id h18so3617278igc.13 for ; Mon, 23 Jun 2014 15:40:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:subject:from:to:date:message-id:in-reply-to:references :user-agent:mime-version:content-type:content-transfer-encoding; bh=hi/7NXxjVrSAZHL8/SgB3iqRTXtYTNZ6ZRvha3FAk64=; b=fxuxuCoXRMuQx/h/DcnZJgVkQ+w2yfbUrPMF4eCqlPINYels05Lf7Iu9ExeADLs0wr 9kaFRDSucGiQ2JMFeCmLeMGqJNMS83ldofZY+lw5I64VlD96drxBhxcnA678SliQcEmp v7zmYRdKDc0Y/9NV/jxSJu975+dBI4tdzIAf/l1j+tt24tXr40Ol/BngBchpci8R+2Zz IcxnT9Gd5J+2YhW/VbSA920UCLEcFXIajWY/OC2T79tmbKXl8CcXhVGPvoy4LyaQJr4l JngiRKSMz+XeooS+cpd7yuZrxq505yc0MfU/RXLoHRVEuEbZZEQVpF+Uv6CSHIFzGcyd Ip7A== X-Received: by 10.42.244.201 with SMTP id lr9mr25371995icb.2.1403563199944; Mon, 23 Jun 2014 15:39:59 -0700 (PDT) Received: from manet.1015granger.net ([2604:8800:100:81fc:82ee:73ff:fe43:d64f]) by mx.google.com with ESMTPSA id 2sm36387725igs.17.2014.06.23.15.39.59 for (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Jun 2014 15:39:59 -0700 (PDT) Subject: [PATCH v1 07/13] xprtrdma: Encode Work Request opcode in wc->wr_id From: Chuck Lever To: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Date: Mon, 23 Jun 2014 18:39:58 -0400 Message-ID: <20140623223958.1634.60529.stgit@manet.1015granger.net> In-Reply-To: <20140623223201.1634.83888.stgit@manet.1015granger.net> References: <20140623223201.1634.83888.stgit@manet.1015granger.net> User-Agent: StGit/0.17.1-3-g7d0f MIME-Version: 1.0 Sender: linux-rdma-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-rdma@vger.kernel.org X-Spam-Status: No, score=-6.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, RCVD_IN_DNSWL_HI,T_DKIM_INVALID,T_RP_MATCHES_RCVD,UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The wc->opcode field is unreliable when a completion fails. Up until now, the completion handler has ignored unsuccessful completions, so that didn't matter to xprtrdma. In a subsequent patch, however, the send CQ handler will need to know which Work Request opcode is completing, even if for error completions. xprtrdma posts three Work Request opcodes via the send queue: SEND, FAST_REG_MR, and LOCAL_INV: For SEND, wc->wr_id is zero. Those completions are ignored. The other two plant a pointer to an rpcrdma_mw in wc->wr_id. Make the low-order bit indicate which of FAST_REG_MR or LOCAL_INV is being done. Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/verbs.c | 19 +++++++++++-------- net/sunrpc/xprtrdma/xprt_rdma.h | 2 ++ 2 files changed, 13 insertions(+), 8 deletions(-) -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index e8ed81c..cef67fd 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -145,20 +145,22 @@ rpcrdma_cq_async_error_upcall(struct ib_event *event, void *context) static void rpcrdma_sendcq_process_wc(struct ib_wc *wc) { - struct rpcrdma_mw *frmr = (struct rpcrdma_mw *)(unsigned long)wc->wr_id; + unsigned long wrid = wc->wr_id; + struct rpcrdma_mw *mw; + int fastreg; - dprintk("RPC: %s: frmr %p status %X opcode %d\n", - __func__, frmr, wc->status, wc->opcode); + dprintk("RPC: %s: wr_id %lx status %X opcode %d\n", + __func__, wrid, wc->status, wc->opcode); - if (wc->wr_id == 0ULL) + if (wrid == 0) return; if (wc->status != IB_WC_SUCCESS) return; - if (wc->opcode == IB_WC_FAST_REG_MR) - frmr->r.frmr.fr_state = FRMR_IS_VALID; - else if (wc->opcode == IB_WC_LOCAL_INV) - frmr->r.frmr.fr_state = FRMR_IS_INVALID; + fastreg = test_and_clear_bit(RPCRDMA_BIT_FASTREG, &wrid); + mw = (struct rpcrdma_mw *)wrid; + + mw->r.frmr.fr_state = fastreg ? FRMR_IS_VALID : FRMR_IS_INVALID; } static int @@ -1538,6 +1540,7 @@ rpcrdma_register_frmr_external(struct rpcrdma_mr_seg *seg, /* Prepare FRMR WR */ memset(&frmr_wr, 0, sizeof frmr_wr); frmr_wr.wr_id = (unsigned long)(void *)seg1->mr_chunk.rl_mw; + frmr_wr.wr_id |= (u64)1 << RPCRDMA_BIT_FASTREG; frmr_wr.opcode = IB_WR_FAST_REG_MR; frmr_wr.send_flags = IB_SEND_SIGNALED; frmr_wr.wr.fast_reg.iova_start = seg1->mr_dma; diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index 28c8570..6b5d243 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -177,6 +177,8 @@ struct rpcrdma_mw { struct list_head mw_list; }; +#define RPCRDMA_BIT_FASTREG (0) + /* * struct rpcrdma_req -- structure central to the request/reply sequence. *