From patchwork Fri Jun 21 13:32:42 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Simon Derr X-Patchwork-Id: 2763061 Return-Path: X-Original-To: patchwork-v9fs-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.19.201]) by patchwork2.web.kernel.org (Postfix) with ESMTP id CDE11C0AB1 for ; Fri, 21 Jun 2013 14:05:20 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 9D6FD20210 for ; Fri, 21 Jun 2013 14:05:19 +0000 (UTC) Received: from lists.sourceforge.net (lists.sourceforge.net [216.34.181.88]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 577CA201E2 for ; Fri, 21 Jun 2013 14:05:13 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=sfs-ml-4.v29.ch3.sourceforge.com) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Uq1xe-0002zQ-HC; Fri, 21 Jun 2013 14:05:10 +0000 Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-4.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1Uq1xb-0002zJ-Dn for v9fs-developer@lists.sourceforge.net; Fri, 21 Jun 2013 14:05:07 +0000 X-ACL-Warn: Received: from ecfrec.frec.bull.fr ([129.183.4.8]) by sog-mx-2.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1Uq1xT-00007d-6E for v9fs-developer@lists.sourceforge.net; Fri, 21 Jun 2013 14:05:07 +0000 Received: from atlas.frec.bull.fr (atlas.frec.bull.fr [129.183.91.13]) by ecfrec.frec.bull.fr (Postfix) with ESMTP id AEF386F5F6; Fri, 21 Jun 2013 15:33:03 +0200 (CEST) Received: by atlas.frec.bull.fr (Postfix, from userid 15269) id 50A2B38004E; Fri, 21 Jun 2013 15:33:03 +0200 (CEST) From: Simon Derr To: v9fs-developer@lists.sourceforge.net Date: Fri, 21 Jun 2013 15:32:42 +0200 Message-Id: <1371821563-5784-10-git-send-email-simon.derr@bull.net> X-Mailer: git-send-email 1.7.2.2 In-Reply-To: <1371821563-5784-1-git-send-email-simon.derr@bull.net> References: <1371821563-5784-1-git-send-email-simon.derr@bull.net> X-Spam-Score: -1.5 (-) X-Headers-End: 1Uq1xT-00007d-6E Cc: simon.derr@bull.net Subject: [V9fs-developer] [PATCH 09/10] 9P/RDMA: count posted buffers without a pending request X-BeenThere: v9fs-developer@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: v9fs-developer-bounces@lists.sourceforge.net X-Spam-Status: No, score=-8.4 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RP_MATCHES_RCVD, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP In rdma_request(): If an error occurs between posting the recv and the send, there will be a reply context posted without a pending request. Since there is no way to "un-post" it, we remember it and skip post_recv() for the next request. Signed-off-by: Simon Derr --- net/9p/trans_rdma.c | 31 ++++++++++++++++++++++++++++++- 1 files changed, 30 insertions(+), 1 deletions(-) diff --git a/net/9p/trans_rdma.c b/net/9p/trans_rdma.c index 926e72d..8f68df5 100644 --- a/net/9p/trans_rdma.c +++ b/net/9p/trans_rdma.c @@ -74,6 +74,8 @@ * @sq_sem: Semaphore for the SQ * @rq_depth: The depth of the Receive Queue. * @rq_sem: Semaphore for the RQ + * @excess_rc : Amount of posted Receive Contexts without a pending request. + * See rdma_request() * @addr: The remote peer's address * @req_lock: Protects the active request list * @cm_done: Completion event for connection management tracking @@ -99,6 +101,7 @@ struct p9_trans_rdma { struct semaphore sq_sem; int rq_depth; struct semaphore rq_sem; + atomic_t excess_rc; struct sockaddr_in addr; spinlock_t req_lock; @@ -426,6 +429,26 @@ static int rdma_request(struct p9_client *client, struct p9_req_t *req) struct p9_rdma_context *c = NULL; struct p9_rdma_context *rpl_context = NULL; + /* When an error occurs between posting the recv and the send, + * there will be a receive context posted without a pending request. + * Since there is no way to "un-post" it, we remember it and skip + * post_recv() for the next request. + * So here, + * see if we are this `next request' and need to absorb an excess rc. + * If yes, then drop and free our own, and do not recv_post(). + **/ + if (unlikely(atomic_read(&rdma->excess_rc) > 0)) { + if ((atomic_sub_return(1, &rdma->excess_rc) >= 0)) { + /* Got one ! */ + kfree(req->rc); + req->rc = NULL; + goto dont_need_post_recv; + } else { + /* We raced and lost. */ + atomic_inc(&rdma->excess_rc); + } + } + /* Allocate an fcall for the reply */ rpl_context = kmalloc(sizeof *rpl_context, GFP_NOFS); if (!rpl_context) { @@ -451,10 +474,10 @@ static int rdma_request(struct p9_client *client, struct p9_req_t *req) p9_debug(P9_DEBUG_FCALL, "POST RECV failed\n"); goto recv_error; } - /* remove posted receive buffer from request structure */ req->rc = NULL; +dont_need_post_recv: /* Post the request */ c = kmalloc(sizeof *c, GFP_NOFS); if (!c) { @@ -499,6 +522,11 @@ static int rdma_request(struct p9_client *client, struct p9_req_t *req) send_error: kfree(c); p9_debug(P9_DEBUG_ERROR, "Error %d in rdma_request()\n", err); + + /* Ach. + * We did recv_post(), but not send. We have one recv_post in excess. + */ + atomic_inc(&rdma->excess_rc); return err; /* Handle errors that happened during or while preparing post_recv(): */ @@ -549,6 +577,7 @@ static struct p9_trans_rdma *alloc_rdma(struct p9_rdma_opts *opts) init_completion(&rdma->cm_done); sema_init(&rdma->sq_sem, rdma->sq_depth); sema_init(&rdma->rq_sem, rdma->rq_depth); + atomic_set(&rdma->excess_rc, 0); return rdma; }