From patchwork Mon Oct 2 02:30:20 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Long Li X-Patchwork-Id: 9980147 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7892E60365 for ; Mon, 2 Oct 2017 02:35:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 61F47289B3 for ; Mon, 2 Oct 2017 02:35:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 56B8B289B6; Mon, 2 Oct 2017 02:35:18 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9EF80289B3 for ; Mon, 2 Oct 2017 02:35:17 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752692AbdJBCeI (ORCPT ); Sun, 1 Oct 2017 22:34:08 -0400 Received: from a2nlsmtp01-05.prod.iad2.secureserver.net ([198.71.225.49]:56472 "EHLO a2nlsmtp01-05.prod.iad2.secureserver.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751913AbdJBCbz (ORCPT ); Sun, 1 Oct 2017 22:31:55 -0400 Received: from linuxonhyperv.com ([107.180.71.197]) by : HOSTING RELAY : with SMTP id yqVIdUn05irtSyqVIddSnE; Sun, 01 Oct 2017 19:30:53 -0700 x-originating-ip: 107.180.71.197 Received: from longli by linuxonhyperv.com with local (Exim 4.89) (envelope-from ) id 1dyqVH-0000xc-W1; Sun, 01 Oct 2017 19:30:44 -0700 From: Long Li To: Steve French , linux-cifs@vger.kernel.org, samba-technical@lists.samba.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, Christoph Hellwig , Tom Talpey , Matthew Wilcox Cc: Long Li Subject: [Patch v4 12/22] CIFS: SMBD: Implement function to receive data via RDMA receive Date: Sun, 1 Oct 2017 19:30:20 -0700 Message-Id: <20171002023030.3582-13-longli@exchange.microsoft.com> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20171002023030.3582-1-longli@exchange.microsoft.com> References: <20171002023030.3582-1-longli@exchange.microsoft.com> X-CMAE-Envelope: MS4wfOCh6/L6LjJ58E+Hs6Nz4jBgaXYG70FYgn0xmtQX8hKrywxqOb1choxSDSaRHzHFS2CiWVGEXU4FnultTSyTpslOo1WVAngFOCpV5GoED7BSxvtP5Qx8 36t0/SSl7isbie6qRkeA3u6on2ac6Sd4Fg/tWcr0Ja5UFEUHZSr0u6OHWUzzReoD2Db/WdNtkExuMbSz066BF1cgmunrC/STCSJFqlnOwvJQb7YiHvXL0UXj 70Y/7zlNwLW5rm4VN73Xv6OG++H9Na1v1IouWN1wA51WZTwX5qgeoEAFFqdFPkaA3p8o/BaGi1HRxHwksulH+n8NIe7gKqNL/Ze/qeT483GySKMFhKao+Sb/ UfCk+CVFGgVBWqEMeD9CqbMxbVsIsHX9EVu6+M2m6XS80ofZPdm15Vp0/HalU9OYA2DNyNhQr6Fm65ZOLlIoJ0ZmfcFs/dIrcFxVhivCPzxjU/Ru73IyxzII cwu6lvSWLggxnNnP Sender: linux-cifs-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-cifs@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Long Li On the receive path, the transport maintains receive buffers and a reassembly queue for transferring payload via RDMA recv. There is data copy in the transport on recv when it copies the payload to upper layer. The transport recognizes the RFC1002 header length use in the SMB upper layer payloads in CIFS. Because this length is mainly used for TCP and not applicable to RDMA, it is handled as a out-of-band information and is never sent over the wire, and the trasnport behaves like TCP to upper layer by processing and exposing the length correctly on data payloads. Signed-off-by: Long Li --- fs/cifs/smbdirect.c | 229 ++++++++++++++++++++++++++++++++++++++++++++++++++++ fs/cifs/smbdirect.h | 6 ++ 2 files changed, 235 insertions(+) diff --git a/fs/cifs/smbdirect.c b/fs/cifs/smbdirect.c index cb129c2..b9be9d6 100644 --- a/fs/cifs/smbdirect.c +++ b/fs/cifs/smbdirect.c @@ -200,6 +200,8 @@ static void smbd_destroy_rdma_work(struct work_struct *work) log_rdma_event(INFO, "wait for all recv to finish\n"); wake_up_interruptible(&info->wait_reassembly_queue); + wait_event(info->wait_smbd_recv_pending, + info->smbd_recv_pending == 0); log_rdma_event(INFO, "wait for all send posted to IB to finish\n"); wait_event(info->wait_send_pending, @@ -1678,6 +1680,9 @@ struct smbd_connection *_smbd_get_connection( queue_delayed_work(info->workqueue, &info->idle_timer_work, info->keep_alive_interval*HZ); + init_waitqueue_head(&info->wait_smbd_recv_pending); + info->smbd_recv_pending = 0; + init_waitqueue_head(&info->wait_send_pending); atomic_set(&info->send_pending, 0); @@ -1744,3 +1749,227 @@ struct smbd_connection *smbd_get_connection( } return ret; } + +/* + * Receive data from receive reassembly queue + * All the incoming data packets are placed in reassembly queue + * buf: the buffer to read data into + * size: the length of data to read + * return value: actual data read + * Note: this implementation copies the data from reassebmly queue to receive + * buffers used by upper layer. This is not the optimal code path. A better way + * to do it is to not have upper layer allocate its receive buffers but rather + * borrow the buffer from reassembly queue, and return it after data is + * consumed. But this will require more changes to upper layer code, and also + * need to consider packet boundaries while they still being reassembled. + */ +int smbd_recv_buf(struct smbd_connection *info, char *buf, unsigned int size) +{ + struct smbd_response *response; + struct smbd_data_transfer *data_transfer; + int to_copy, to_read, data_read, offset; + u32 data_length, remaining_data_length, data_offset; + int rc; + unsigned long flags; + +again: + if (info->transport_status != SMBD_CONNECTED) { + log_read(ERR, "disconnected\n"); + return -ENODEV; + } + + /* + * No need to hold the reassembly queue lock all the time as we are + * the only one reading from the front of the queue. The transport + * may add more entries to the back of the queeu at the same time + */ + log_read(INFO, "size=%d info->reassembly_data_length=%d\n", size, + info->reassembly_data_length); + if (info->reassembly_data_length >= size) { + unsigned long long t1 = rdtsc(); + int queue_length; + int queue_removed = 0; + + /* + * Need to make sure reassembly_data_length is read before + * reading reassembly_queue_length and calling + * _get_first_reassembly. This call is lock free + * as we never read at the end of the queue which are being + * updated in SOFTIRQ as more data is received + */ + virt_rmb(); + queue_length = info->reassembly_queue_length; + data_read = 0; + to_read = size; + offset = info->first_entry_offset; + while (data_read < size) { + response = _get_first_reassembly(info); + data_transfer = smbd_response_payload(response); + data_length = le32_to_cpu(data_transfer->data_length); + remaining_data_length = + le32_to_cpu( + data_transfer->remaining_data_length); + data_offset = le32_to_cpu(data_transfer->data_offset); + + /* + * The upper layer expects RFC1002 length at the + * beginning of the payload. Return it to indicate + * the total length of the packet. This minimize the + * change to upper layer packet processing logic. This + * will be eventually remove when an intermediate + * transport layer is added + */ + if (response->first_segment && size == 4) { + unsigned int rfc1002_len = + data_length + remaining_data_length; + *((__be32 *)buf) = cpu_to_be32(rfc1002_len); + data_read = 4; + response->first_segment = false; + log_read(INFO, "returning rfc1002 length %d\n", + rfc1002_len); + goto read_rfc1002_done; + } + + to_copy = min_t(int, data_length - offset, to_read); + memcpy( + buf + data_read, + (char *)data_transfer + data_offset + offset, + to_copy); + + /* move on to the next buffer? */ + if (to_copy == data_length - offset) { + queue_length--; + /* + * No need to lock if we are not at the + * end of the queue + */ + if (!queue_length) + spin_lock_irqsave( + &info->reassembly_queue_lock, + flags); + list_del(&response->list); + queue_removed++; + if (!queue_length) + spin_unlock_irqrestore( + &info->reassembly_queue_lock, + flags); + + info->count_reassembly_queue--; + info->count_dequeue_reassembly_queue++; + put_receive_buffer(info, response, true); + offset = 0; + log_read(INFO, "put_receive_buffer offset=0\n"); + } else + offset += to_copy; + + to_read -= to_copy; + data_read += to_copy; + + log_read(INFO, "_get_first_reassembly memcpy %d bytes " + "data_transfer_length-offset=%d after that " + "to_read=%d data_read=%d offset=%d\n", + to_copy, data_length - offset, + to_read, data_read, offset); + } + + spin_lock_irqsave(&info->reassembly_queue_lock, flags); + info->reassembly_data_length -= data_read; + info->reassembly_queue_length -= queue_removed; + spin_unlock_irqrestore(&info->reassembly_queue_lock, flags); + + info->first_entry_offset = offset; + log_read(INFO, "returning to thread data_read=%d " + "reassembly_data_length=%d first_entry_offset=%d\n", + data_read, info->reassembly_data_length, + info->first_entry_offset); +read_rfc1002_done: + profiling_add_histogram(rdtsc() - t1, info->smbd_recv_cycles); + return data_read; + } + + log_read(INFO, "wait_event on more data\n"); + rc = wait_event_interruptible( + info->wait_reassembly_queue, + info->reassembly_data_length >= size || + info->transport_status != SMBD_CONNECTED); + /* Don't return any data if interrupted */ + if (rc) + return -ENODEV; + + goto again; +} + +/* + * Receive a page from receive reassembly queue + * page: the page to read data into + * to_read: the length of data to read + * return value: actual data read + */ +int smbd_recv_page(struct smbd_connection *info, + struct page *page, unsigned int to_read) +{ + int ret; + char *to_address; + + /* make sure we have the page ready for read */ + ret = wait_event_interruptible( + info->wait_reassembly_queue, + info->reassembly_data_length >= to_read || + info->transport_status != SMBD_CONNECTED); + if (ret) + return 0; + + /* now we can read from reassembly queue and not sleep */ + to_address = kmap_atomic(page); + + log_read(INFO, "reading from page=%p address=%p to_read=%d\n", + page, to_address, to_read); + + ret = smbd_recv_buf(info, to_address, to_read); + kunmap_atomic(to_address); + + return ret; +} + +/* + * Receive data from transport + * msg: a msghdr point to the buffer, can be ITER_KVEC or ITER_BVEC + * return: total bytes read, or 0. SMB Direct will not do partial read. + */ +int smbd_recv(struct smbd_connection *info, struct msghdr *msg) +{ + char *buf; + struct page *page; + unsigned int to_read; + int rc; + + info->smbd_recv_pending++; + + switch (msg->msg_iter.type) { + case READ | ITER_KVEC: + buf = msg->msg_iter.kvec->iov_base; + to_read = msg->msg_iter.kvec->iov_len; + rc = smbd_recv_buf(info, buf, to_read); + break; + + case READ | ITER_BVEC: + page = msg->msg_iter.bvec->bv_page; + to_read = msg->msg_iter.bvec->bv_len; + rc = smbd_recv_page(info, page, to_read); + break; + + default: + /* It's a bug in upper layer to get there */ + cifs_dbg(VFS, "CIFS: invalid msg type %d\n", + msg->msg_iter.type); + rc = -EIO; + } + + info->smbd_recv_pending--; + wake_up(&info->wait_smbd_recv_pending); + + /* SMBDirect will read it all or nothing */ + if (rc > 0) + msg->msg_iter.count = 0; + return rc; +} diff --git a/fs/cifs/smbdirect.h b/fs/cifs/smbdirect.h index d14a484..26614fa 100644 --- a/fs/cifs/smbdirect.h +++ b/fs/cifs/smbdirect.h @@ -88,6 +88,9 @@ struct smbd_connection { int fragment_reassembly_remaining; /* Activity accoutning */ + /* Pending reqeusts issued from upper layer */ + int smbd_recv_pending; + wait_queue_head_t wait_smbd_recv_pending; atomic_t send_pending; wait_queue_head_t wait_send_pending; @@ -255,6 +258,9 @@ int smbd_reconnect(struct TCP_Server_Info *server); /* Destroy SMBDirect session */ void smbd_destroy(struct smbd_connection *info); +/* Interface for carrying upper layer I/O through send/recv */ +int smbd_recv(struct smbd_connection *info, struct msghdr *msg); + void profiling_display_histogram( struct seq_file *m, unsigned long long array[]); #endif