From patchwork Thu Nov 15 08:52:50 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10683727 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9E68A17F3 for ; Thu, 15 Nov 2018 08:54:16 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8F57F29B7E for ; Thu, 15 Nov 2018 08:54:16 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 820412B678; Thu, 15 Nov 2018 08:54:16 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D072E29B7E for ; Thu, 15 Nov 2018 08:54:14 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DE86B6B0271; Thu, 15 Nov 2018 03:54:13 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id DBEB96B0272; Thu, 15 Nov 2018 03:54:13 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CD4FA6B0273; Thu, 15 Nov 2018 03:54:13 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f197.google.com (mail-qk1-f197.google.com [209.85.222.197]) by kanga.kvack.org (Postfix) with ESMTP id 9F8C86B0271 for ; Thu, 15 Nov 2018 03:54:13 -0500 (EST) Received: by mail-qk1-f197.google.com with SMTP id d196so43704414qkb.6 for ; Thu, 15 Nov 2018 00:54:13 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=KtxO1h8lTIMO6x3g4219zciNDYDpEDTAic2TXSE5EAk=; b=d5eaA57MjGHZi0cKlXdfYnnN5rkr/4ffZGjRt9Z4WxYX7Du49zxrx85l+8NBIiiSrr 7bgQrjaF+lr+fBksUimkYu0FKZ4qzKZ9MV1lFR/t3vc9HyX6gR75imt7qujQzny+Rzpb 7UBxFnb3PoxZmKAimst0BCE4GtphJXQMng/Y9kNGxdC+BRZ9Ti7aGJhn2kgpKhfIHo1T 2VMjrXVhO/25QdgFNK5Izw/SzlgfN7bVO+6LJT12V8/9qTJbLAdpS2NurXNm7wsmO1Xp RgKq5tNEqu7I2n36AX5fM6g7z8CXKvW0DM11YeHJiokR5GzQoU1eAvoN4MzFI/83FF6/ s49w== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AGRZ1gKtkTlpQHE4Hkg6dg9wMUSrIiDn31h2IUUpRf2fNRZRKWmcEL/1 vygF/dqR5ekNqBpGcIbYntKRqcqtkahdpHVp2n5ZlXRV1Rd4Vb/fN5uVo1LRqaVppH9/DvIR0Cg W7Hh16kQMtVY3w9j4Hc9AQug/caPHzlzljPrM/gNpIzPCycIUmggqLiNxYFQtpN2xQQ== X-Received: by 2002:a37:474b:: with SMTP id u72mr5049503qka.106.1542272053376; Thu, 15 Nov 2018 00:54:13 -0800 (PST) X-Google-Smtp-Source: AJdET5dUgkN4CK0ngj1tH9/gmNkJr9CZmwUftur9ifE+1QzPhLvSAxYisFRnlTvcMDCBrOjlnXGO X-Received: by 2002:a37:474b:: with SMTP id u72mr5049479qka.106.1542272052773; Thu, 15 Nov 2018 00:54:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1542272052; cv=none; d=google.com; s=arc-20160816; b=c0v9la/HkzOaPaWOS2CQoWSqqWiQyAwvexwUYH8k/zbBh11wrQoCNY4bjdGRP2hRrb zEJ2eqTKjOTkzdFK0nzlQJi+La7e40G1gZOPf+wdMfxIsesLzsAnaVlP34H0KJB3IVjH bNMQyE7do24EESX4cq5Nad7kyqeSSHfQBfb8BjWYFkogCybJSxPG201TFy9oHQ0cgU8v b5GBV2QMUCudixo/HPlw3b7l2YJx/uGoLr5VNdhyaW0Qe9CxR2Ow6EXcHDn2won4KuZK KTwmnslZuGPoSZH3ahrVu1hMFRc7eaURYC3EojEg2Vr01d7LdtBRqDnNzzLS7RADly7Z OlVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=KtxO1h8lTIMO6x3g4219zciNDYDpEDTAic2TXSE5EAk=; b=Jmde5F6TkTK6R/R8rmgmxmP9sHO7iy7Hp3+xx+eCXDwbMgvQv2rw98nvKsGzL/WJSs mvvse0xdOBH44XZDnnCxFhUNFbUuXGM9KwL+rDcoT/cWKDJI+PIuB4gAeOwgaICTVOyz UpJvGQlnuAkIPxRCGC8+fLvYHJfJI3VzZsFhDHFvC97I4N90Gulasah2ythJmHWa2RMr 8zNODblPmEyfJ1aRBFynjYxR6CIHqJ8pFr0gYFdZ0wb6MJf3/l3sGn/ZQFLZUbeljFbV tbU+sZ9dot6Q6LXW7pFIM0aJuena6C46roRRezm6SleI1ixBCCoqyr3x2JOAOrj9pmOX iykQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id n69si11784014qkn.55.2018.11.15.00.54.12 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Nov 2018 00:54:12 -0800 (PST) Received-SPF: pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx01.intmail.prod.int.phx2.redhat.com [10.5.11.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 5067AA4051; Thu, 15 Nov 2018 08:54:11 +0000 (UTC) Received: from localhost (ovpn-8-23.pek2.redhat.com [10.72.8.23]) by smtp.corp.redhat.com (Postfix) with ESMTP id 738D6600C3; Thu, 15 Nov 2018 08:54:09 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Ming Lei , Dave Chinner , Kent Overstreet , Mike Snitzer , dm-devel@redhat.com, Alexander Viro , linux-fsdevel@vger.kernel.org, Shaohua Li , linux-raid@vger.kernel.org, linux-erofs@lists.ozlabs.org, David Sterba , linux-btrfs@vger.kernel.org, "Darrick J . Wong" , linux-xfs@vger.kernel.org, Gao Xiang , Christoph Hellwig , Theodore Ts'o , linux-ext4@vger.kernel.org, Coly Li , linux-bcache@vger.kernel.org, Boaz Harrosh , Bob Peterson , cluster-devel@redhat.com Subject: [PATCH V10 03/19] block: use bio_for_each_bvec() to compute multi-page bvec count Date: Thu, 15 Nov 2018 16:52:50 +0800 Message-Id: <20181115085306.9910-4-ming.lei@redhat.com> In-Reply-To: <20181115085306.9910-1-ming.lei@redhat.com> References: <20181115085306.9910-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.11 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Thu, 15 Nov 2018 08:54:11 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP First it is more efficient to use bio_for_each_bvec() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many multi-page bvecs there are in the bio. Secondly once bio_for_each_bvec() is used, the bvec may need to be splitted because its length can be very longer than max segment size, so we have to split the big bvec into several segments. Thirdly when splitting multi-page bvec into segments, the max segment limit may be reached, so the bio split need to be considered under this situation too. Cc: Dave Chinner Cc: Kent Overstreet Cc: Mike Snitzer Cc: dm-devel@redhat.com Cc: Alexander Viro Cc: linux-fsdevel@vger.kernel.org Cc: Shaohua Li Cc: linux-raid@vger.kernel.org Cc: linux-erofs@lists.ozlabs.org Cc: David Sterba Cc: linux-btrfs@vger.kernel.org Cc: Darrick J. Wong Cc: linux-xfs@vger.kernel.org Cc: Gao Xiang Cc: Christoph Hellwig Cc: Theodore Ts'o Cc: linux-ext4@vger.kernel.org Cc: Coly Li Cc: linux-bcache@vger.kernel.org Cc: Boaz Harrosh Cc: Bob Peterson Cc: cluster-devel@redhat.com Signed-off-by: Ming Lei --- block/blk-merge.c | 90 ++++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 76 insertions(+), 14 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index 91b2af332a84..6f7deb94a23f 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -160,6 +160,62 @@ static inline unsigned get_max_io_size(struct request_queue *q, return sectors; } +/* + * Split the bvec @bv into segments, and update all kinds of + * variables. + */ +static bool bvec_split_segs(struct request_queue *q, struct bio_vec *bv, + unsigned *nsegs, unsigned *last_seg_size, + unsigned *front_seg_size, unsigned *sectors) +{ + bool need_split = false; + unsigned len = bv->bv_len; + unsigned total_len = 0; + unsigned new_nsegs = 0, seg_size = 0; + + if ((*nsegs >= queue_max_segments(q)) || !len) + return need_split; + + /* + * Multipage bvec may be too big to hold in one segment, + * so the current bvec has to be splitted as multiple + * segments. + */ + while (new_nsegs + *nsegs < queue_max_segments(q)) { + seg_size = min(queue_max_segment_size(q), len); + + new_nsegs++; + total_len += seg_size; + len -= seg_size; + + if ((queue_virt_boundary(q) && ((bv->bv_offset + + total_len) & queue_virt_boundary(q))) || !len) + break; + } + + /* split in the middle of the bvec */ + if (len) + need_split = true; + + /* update front segment size */ + if (!*nsegs) { + unsigned first_seg_size = seg_size; + + if (new_nsegs > 1) + first_seg_size = queue_max_segment_size(q); + if (*front_seg_size < first_seg_size) + *front_seg_size = first_seg_size; + } + + /* update other varibles */ + *last_seg_size = seg_size; + *nsegs += new_nsegs; + if (sectors) + *sectors += total_len >> 9; + + return need_split; +} + static struct bio *blk_bio_segment_split(struct request_queue *q, struct bio *bio, struct bio_set *bs, @@ -173,7 +229,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, struct bio *new = NULL; const unsigned max_sectors = get_max_io_size(q, bio); - bio_for_each_segment(bv, bio, iter) { + bio_for_each_bvec(bv, bio, iter) { /* * If the queue doesn't support SG gaps and adding this * offset would create a gap, disallow it. @@ -188,8 +244,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, */ if (nsegs < queue_max_segments(q) && sectors < max_sectors) { - nsegs++; - sectors = max_sectors; + /* split in the middle of bvec */ + bv.bv_len = (max_sectors - sectors) << 9; + bvec_split_segs(q, &bv, &nsegs, + &seg_size, + &front_seg_size, + §ors); } goto split; } @@ -214,11 +274,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, if (nsegs == 1 && seg_size > front_seg_size) front_seg_size = seg_size; - nsegs++; bvprv = bv; bvprvp = &bvprv; - seg_size = bv.bv_len; - sectors += bv.bv_len >> 9; + + if (bvec_split_segs(q, &bv, &nsegs, &seg_size, + &front_seg_size, §ors)) + goto split; } @@ -296,6 +357,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, struct bio_vec bv, bvprv = { NULL }; int cluster, prev = 0; unsigned int seg_size, nr_phys_segs; + unsigned front_seg_size = bio->bi_seg_front_size; struct bio *fbio, *bbio; struct bvec_iter iter; @@ -316,7 +378,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, seg_size = 0; nr_phys_segs = 0; for_each_bio(bio) { - bio_for_each_segment(bv, bio, iter) { + bio_for_each_bvec(bv, bio, iter) { /* * If SG merging is disabled, each bio vector is * a segment @@ -336,20 +398,20 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, continue; } new_segment: - if (nr_phys_segs == 1 && seg_size > - fbio->bi_seg_front_size) - fbio->bi_seg_front_size = seg_size; + if (nr_phys_segs == 1 && seg_size > front_seg_size) + front_seg_size = seg_size; - nr_phys_segs++; bvprv = bv; prev = 1; - seg_size = bv.bv_len; + bvec_split_segs(q, &bv, &nr_phys_segs, &seg_size, + &front_seg_size, NULL); } bbio = bio; } - if (nr_phys_segs == 1 && seg_size > fbio->bi_seg_front_size) - fbio->bi_seg_front_size = seg_size; + if (nr_phys_segs == 1 && seg_size > front_seg_size) + front_seg_size = seg_size; + fbio->bi_seg_front_size = front_seg_size; if (seg_size > bbio->bi_seg_back_size) bbio->bi_seg_back_size = seg_size;