From patchwork Wed Aug 2 20:10:18 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Long Li X-Patchwork-Id: 9877475 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 202F2603B4 for ; Wed, 2 Aug 2017 20:12:45 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 099932885C for ; Wed, 2 Aug 2017 20:12:45 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 081DF28854; Wed, 2 Aug 2017 20:12:45 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 357402885C for ; Wed, 2 Aug 2017 20:12:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753080AbdHBUM2 (ORCPT ); Wed, 2 Aug 2017 16:12:28 -0400 Received: from a2nlsmtp01-03.prod.iad2.secureserver.net ([198.71.225.37]:50818 "EHLO a2nlsmtp01-03.prod.iad2.secureserver.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752983AbdHBUMZ (ORCPT ); Wed, 2 Aug 2017 16:12:25 -0400 Received: from linuxonhyperv.com ([107.180.71.197]) by : HOSTING RELAY : with SMTP id czzBdALQeP2LIczzBdEdmx; Wed, 02 Aug 2017 13:11:23 -0700 x-originating-ip: 107.180.71.197 Received: from longli by linuxonhyperv.com with local (Exim 4.89) (envelope-from ) id 1dczzB-0005GO-Me; Wed, 02 Aug 2017 13:11:17 -0700 From: Long Li To: Steve French , linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-kernel@vger.kernel.org Cc: Long Li Subject: [[PATCH v1] 07/37] [CIFS] SMBD: Implement receive buffer for handling SMBD response Date: Wed, 2 Aug 2017 13:10:18 -0700 Message-Id: <1501704648-20159-8-git-send-email-longli@exchange.microsoft.com> X-Mailer: git-send-email 1.7.1 In-Reply-To: <1501704648-20159-1-git-send-email-longli@exchange.microsoft.com> References: <1501704648-20159-1-git-send-email-longli@exchange.microsoft.com> X-CMAE-Envelope: MS4wfGduCUkJf2BUYUUpOCHsBXfUc79VqQ9Ios+E1oIjNsgHeXnyjXVjcQsiyImEmwb55ekvE4YkwD7IhrnKkuK2k39BphzAYxx4GP0moqiQCXJ4kd6hs6EV nXdsIHZOrDu6TQXx/hnFNpQHUhrKv514ehmz4SyC1sTLjqwlW4bhGdH/HuWMeLRGw6atwhqUmdc84pmLU+SA52X9Kgvc+PKER+DoZXVAIayBoAkE/Jaakeiq t0f3JRMrXkj18EW0Ovi8OdXRL7lHYkp46l6+3QEQ+SQOh/z6AK+Q0KMKBoslvXi4jytmRazwBbAcFyvC0YFkkh897uRiu3H4KR2+2i6wktM= Sender: linux-cifs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-cifs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Long Li Create recevie buffers to handling SMBD response messages. Each SMBD response message is assocated with a receive buffer, where the message payload is received. The number of receive buffer is determine after negotiating SMBD transport with the server. Signed-off-by: Long Li --- fs/cifs/cifsrdma.c | 89 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/cifs/cifsrdma.h | 3 ++ 2 files changed, 92 insertions(+) diff --git a/fs/cifs/cifsrdma.c b/fs/cifs/cifsrdma.c index 1636304..b3e43b1 100644 --- a/fs/cifs/cifsrdma.c +++ b/fs/cifs/cifsrdma.c @@ -54,6 +54,14 @@ #include "cifsrdma.h" +static struct cifs_rdma_response* get_receive_buffer( + struct cifs_rdma_info *info); +static void put_receive_buffer( + struct cifs_rdma_info *info, + struct cifs_rdma_response *response); +static int allocate_receive_buffers(struct cifs_rdma_info *info, int num_buf); +static void destroy_receive_buffers(struct cifs_rdma_info *info); + /* * Per RDMA transport connection parameters * as defined in [MS-SMBD] 3.1.1.1 @@ -280,6 +288,85 @@ static int cifs_rdma_ia_open( return rc; } +/* + * Receive buffer operations. + * For each remote send, we need to post a receive. The receive buffers are + * pre-allocated in advance. + */ +static struct cifs_rdma_response* get_receive_buffer(struct cifs_rdma_info *info) +{ + struct cifs_rdma_response *ret = NULL; + unsigned long flags; + + spin_lock_irqsave(&info->receive_queue_lock, flags); + if (!list_empty(&info->receive_queue)) { + ret = list_first_entry( + &info->receive_queue, + struct cifs_rdma_response, list); + list_del(&ret->list); + info->count_receive_buffer--; + info->count_get_receive_buffer++; + } + spin_unlock_irqrestore(&info->receive_queue_lock, flags); + + return ret; +} + +static void put_receive_buffer( + struct cifs_rdma_info *info, struct cifs_rdma_response *response) +{ + unsigned long flags; + + ib_dma_unmap_single(info->id->device, response->sge.addr, + response->sge.length, DMA_FROM_DEVICE); + + spin_lock_irqsave(&info->receive_queue_lock, flags); + list_add_tail(&response->list, &info->receive_queue); + info->count_receive_buffer++; + info->count_put_receive_buffer++; + spin_unlock_irqrestore(&info->receive_queue_lock, flags); +} + +static int allocate_receive_buffers(struct cifs_rdma_info *info, int num_buf) +{ + int i; + struct cifs_rdma_response *response; + + INIT_LIST_HEAD(&info->receive_queue); + spin_lock_init(&info->receive_queue_lock); + + for (i=0; iresponse_mempool, GFP_KERNEL); + if (!response) + goto allocate_failed; + + response->info = info; + list_add_tail(&response->list, &info->receive_queue); + info->count_receive_buffer++; + } + + return 0; + +allocate_failed: + while (!list_empty(&info->receive_queue)) { + response = list_first_entry( + &info->receive_queue, + struct cifs_rdma_response, list); + list_del(&response->list); + info->count_receive_buffer--; + + mempool_free(response, info->response_mempool); + } + return -ENOMEM; +} + +static void destroy_receive_buffers(struct cifs_rdma_info *info) +{ + struct cifs_rdma_response *response; + while ((response = get_receive_buffer(info))) + mempool_free(response, info->response_mempool); +} + struct cifs_rdma_info* cifs_create_rdma_session( struct TCP_Server_Info *server, struct sockaddr *dstaddr) { @@ -383,6 +470,8 @@ struct cifs_rdma_info* cifs_create_rdma_session( info->response_mempool = mempool_create(info->receive_credit_max, mempool_alloc_slab, mempool_free_slab, info->response_cache); + + allocate_receive_buffers(info, info->receive_credit_max); out2: rdma_destroy_id(info->id); diff --git a/fs/cifs/cifsrdma.h b/fs/cifs/cifsrdma.h index 41ae61a..78ce2bf 100644 --- a/fs/cifs/cifsrdma.h +++ b/fs/cifs/cifsrdma.h @@ -59,6 +59,9 @@ struct cifs_rdma_info { atomic_t receive_credits; atomic_t receive_credit_target; + struct list_head receive_queue; + spinlock_t receive_queue_lock; + // response pool for RDMA receive struct kmem_cache *response_cache; mempool_t *response_mempool;