From patchwork Wed Mar 13 01:05:18 2013 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alex Elder X-Patchwork-Id: 2260541 Return-Path: X-Original-To: patchwork-ceph-devel@patchwork.kernel.org Delivered-To: patchwork-process-083081@patchwork1.kernel.org Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by patchwork1.kernel.org (Postfix) with ESMTP id 3473D3FCF6 for ; Wed, 13 Mar 2013 01:05:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932439Ab3CMBFW (ORCPT ); Tue, 12 Mar 2013 21:05:22 -0400 Received: from mail-ob0-f179.google.com ([209.85.214.179]:34567 "EHLO mail-ob0-f179.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932103Ab3CMBFW (ORCPT ); Tue, 12 Mar 2013 21:05:22 -0400 Received: by mail-ob0-f179.google.com with SMTP id un3so491701obb.24 for ; Tue, 12 Mar 2013 18:05:21 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:content-type:content-transfer-encoding :x-gm-message-state; bh=KrkyuAB3oN/Wmc8gSxT9EpERynD+D59TucPRXT648xY=; b=OPa3b1uyYTOkP9obWY1pwXNAtfPY1hwX4nyVbFVtwv3HzOdoue4fQI/QsbkH3/AQfx kYBjsl96xjQ+ILk00Kw75Eha7PNJ2I0dieozxYjfmQHzsMVke1/p7mAO4xfchFXV5Abt O0+le0nBDsLcPGEFh+otarOZcupNfypdPR0A8F6AIjMLdFyHrdGuBnC3m+jm8AAxWwFw diShHmOLkrV76o+CdvToDg1Nk+NvSoxzr0MH9poTlG//7MO9S0/uR0EcotEY9NwVe7g+ 9JXPNDuwc22QC69d0Wn8wjELyhnjXkSekqdZl4a2rph594bjo+I6bZ7/hxaOFd3kxNLe vU+A== X-Received: by 10.60.14.226 with SMTP id s2mr13956030oec.124.1363136721635; Tue, 12 Mar 2013 18:05:21 -0700 (PDT) Received: from [172.22.22.4] (c-71-195-31-37.hsd1.mn.comcast.net. [71.195.31.37]) by mx.google.com with ESMTPS id ka6sm23721353obb.3.2013.03.12.18.05.19 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 12 Mar 2013 18:05:20 -0700 (PDT) Message-ID: <513FD0CE.6000800@inktank.com> Date: Tue, 12 Mar 2013 20:05:18 -0500 From: Alex Elder User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130221 Thunderbird/17.0.3 MIME-Version: 1.0 To: ceph-devel@vger.kernel.org Subject: [PATCH 1/4] libceph: use cursor for bio reads References: <513FD092.2030106@inktank.com> In-Reply-To: <513FD092.2030106@inktank.com> X-Gm-Message-State: ALoCoQm0kbJUUW7WC2W7x9fb7dm8lriqfb6o6yjxtTqyIsSm+fdPmwBjyyG3RihhXo/QE6AO3IT2 Sender: ceph-devel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: ceph-devel@vger.kernel.org Replace the use of the information in con->in_msg_pos for incoming bio data. The old in_msg_pos and the new cursor mechanism do basically the same thing, just slightly differently. The main functional difference is that in_msg_pos keeps track of the length of the complete bio list, and assumed it was fully consumed when that many bytes had been transferred. The cursor does not assume a length, it simply consumes all bytes in the bio list. Because the only user of bio data is the rbd client, and because the length of a bio list provided by rbd client always matches the number of bytes in the list, both ways of tracking length are equivalent. In addition, for in_msg_pos the initial bio vector is selected as the initial value of the bio->bi_idx, while the cursor assumes this is zero. Again, the rbd client always passes 0 as the initial index so the effect is the same. Other than that, they basically match: in_msg_pos cursor ---------- ------ bio_iter bio bio_seg vec_index page_pos page_offset The in_msg_pos field is initialized by a call to init_bio_iter(). The bio cursor is initialized by ceph_msg_data_cursor_init(). Both now happen in the same spot, in prepare_message_data(). The in_msg_pos field is advanced by a call to in_msg_pos_next(), which updates page_pos and calls iter_bio_next() to move to the next bio vector, or to the next bio in the list. The cursor is advanced by ceph_msg_data_advance(). That isn't currently happening so add a call to that in in_msg_pos_next(). Finally, the next piece of data to use for a read is determined by a bunch of lines in read_partial_message_bio(). Those can be replaced by an equivalent ceph_msg_data_bio_next() call. Since the read path has no need to know whether the next piece of a bio to use is the last, allow it to pass a null pointer as its last_piece argument to ceph_msg_data_bio_next(). This partially resolves: http://tracker.ceph.com/issues/4428 Signed-off-by: Alex Elder --- net/ceph/messenger.c | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) @@ -2225,23 +2229,14 @@ static int read_partial_message_bio(struct ceph_connection *con, unsigned int data_len, bool do_datacrc) { struct ceph_msg *msg = con->in_msg; - struct ceph_msg_pos *msg_pos = &con->in_msg_pos; - struct bio_vec *bv; struct page *page; size_t page_offset; size_t length; - unsigned int left; int ret; BUG_ON(!msg); - BUG_ON(!msg->b.bio_iter); - bv = bio_iovec_idx(msg->b.bio_iter, msg->b.bio_seg); - page = bv->bv_page; - page_offset = bv->bv_offset + msg_pos->page_pos; - BUG_ON(msg_pos->data_pos >= data_len); - left = data_len - msg_pos->data_pos; - BUG_ON(msg_pos->page_pos >= bv->bv_len); - length = min_t(unsigned int, bv->bv_len - msg_pos->page_pos, left); + + page = ceph_msg_data_next(&msg->b, &page_offset, &length, NULL); ret = ceph_tcp_recvpage(con->sock, page, page_offset, length); if (ret <= 0) diff --git a/net/ceph/messenger.c b/net/ceph/messenger.c index 07ace1d..1b003b4 100644 --- a/net/ceph/messenger.c +++ b/net/ceph/messenger.c @@ -1450,6 +1450,10 @@ static void in_msg_pos_next(struct ceph_connection *con, size_t len, msg_pos->data_pos += received; msg_pos->page_pos += received; +#ifdef CONFIG_BLOCK + if (ceph_msg_has_bio(msg)) + (void) ceph_msg_data_advance(&msg->b, received); +#endif /* CONFIG_BLOCK */ if (received < len) return;