From patchwork Fri Jan 11 11:01:15 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10757705 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0AE4A1515 for ; Fri, 11 Jan 2019 11:03:40 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F0C3A29C07 for ; Fri, 11 Jan 2019 11:03:39 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E4AB229C0E; Fri, 11 Jan 2019 11:03:39 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=unavailable version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4EB9D29C07 for ; Fri, 11 Jan 2019 11:03:39 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6A8828E0009; Fri, 11 Jan 2019 06:03:38 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 657F78E0001; Fri, 11 Jan 2019 06:03:38 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56F278E0009; Fri, 11 Jan 2019 06:03:38 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qt1-f197.google.com (mail-qt1-f197.google.com [209.85.160.197]) by kanga.kvack.org (Postfix) with ESMTP id 2FD2F8E0001 for ; Fri, 11 Jan 2019 06:03:38 -0500 (EST) Received: by mail-qt1-f197.google.com with SMTP id n39so15928591qtn.18 for ; Fri, 11 Jan 2019 03:03:38 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=QdLd93Jb3gVSIaMuMlz5f3nxe2ES1ehL7zzHVfeXmV8=; b=QDq2SXScw2677rF+B0DtGX0Eqf7Ut75JoWUAxUNMS/o7VGUWwRUK7ub/MxGuyMSnbp ASW5G0vodth/TC+R7aKZ5uUqwN7My0+zVVOuKptiK+2kkVN05OJn/SAM9rEm4bBnnx2d n5gnk5zhtkKPFHS6fJKLFlSF3nu+gZp0izficBfaS8auf+JFJPYPEg5bRqBG0y3ohtTG ornz+JSq+3O4DRMpikjKPdysLTN7vf/w7UTuq2wbDGoVjViwVWeZK/dKugwbW99tYOv/ KHWIq/twNcCbBlciPQ2UsyoaAJTScOAwNb2wjqQWPXBV8JRaMrBgPi2WzeU/6BqZRBdJ E4/g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukepJbBi37nwSjuOOgnpTyAufb+aJM2jrn5LJ0Hb5FHi0crPHYFl REgqlBbmd9jG+T+f8hWRLeMDBexYlBOzNzo6rnpHYdW+wDZIaj/aoir/ughM4kujRz4WZLq0DS0 4PbUTa9a0P2QhKnzzbeKcsBBYt9kObl3ZPi3hNkEZ34WHiNNPbjfAEQ+MPLtj1F87ag== X-Received: by 2002:a0c:afd1:: with SMTP id t17mr13314509qvc.93.1547204617957; Fri, 11 Jan 2019 03:03:37 -0800 (PST) X-Google-Smtp-Source: ALg8bN5K35BPiId59YbSJuzLTfSkieWw08UpNZxXhjaXGBGulNSwO+lL0r6gwfyeui+YG65QU5mi X-Received: by 2002:a0c:afd1:: with SMTP id t17mr13314470qvc.93.1547204617316; Fri, 11 Jan 2019 03:03:37 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1547204617; cv=none; d=google.com; s=arc-20160816; b=J1NG1uh+1UDYce7rwjxm+olKoRgAK83qmRoG7nq+pDTTr2i1Cpuq8QL3w+1v62Yv/t AW27Du6TLbdeTJ/VFW4aPV6T1agsiEUlJD2KeQmGge+iLIMAUrGr9mhzTOtJdhSJh1WR vC1qSJEKLyfA+rsuTnhyursgCzlsFKyXPA5OWTDN7i3S1WssKE8gUjjlmasywVdOQN0c JN4OOffgh0J0w55OEkFYZHySnCG0NgjKg7hD4qpN2uF7HOkjPYivTaqOFwiACkIzUCRu hg68C+kgOmImSS4C84O/9f9thdoU8WFuiCe6djuK6tnsbYCahGVPVBNSfpb2hnCdpfxG dvGw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=QdLd93Jb3gVSIaMuMlz5f3nxe2ES1ehL7zzHVfeXmV8=; b=n2QycnzlFMeWROYiDyyJ3dQxb+fUR+lw/DA/L17ZVhGO+158REPxQLMhIsVziAsbsX l8WtCyyLlt0nPJrfjOlbg3EcsvW+nWaJd4LT6igPI01zABVBJDZUHDPRMIYzshEGvB/0 wsaTjgkSCat1q0rD1qHLGITPyPxUwj7FiRFDWgvrvLCfa7KwfNygEFVeq7Y0ytCijr+C NvIpGFJRSvi+12C8j4e76m1VUtqY+RlDI1GeA8sksuqhpK8+pf/ZjjfKtvcy8ZjFdcA6 W83iwRKygxuXBiaR1xfOj1lxQUZPhKeeGtfk4n7LRKIyKVzd6sT9/7f/hEUlrGLY/m5d qwYg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id k124si1342834qkd.2.2019.01.11.03.03.37 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 11 Jan 2019 03:03:37 -0800 (PST) Received-SPF: pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx07.intmail.prod.int.phx2.redhat.com [10.5.11.22]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 40A661F8EC; Fri, 11 Jan 2019 11:03:36 +0000 (UTC) Received: from localhost (ovpn-8-28.pek2.redhat.com [10.72.8.28]) by smtp.corp.redhat.com (Postfix) with ESMTP id 6DA3B1059580; Fri, 11 Jan 2019 11:03:26 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Theodore Ts'o , Omar Sandoval , Sagi Grimberg , Dave Chinner , Kent Overstreet , Mike Snitzer , dm-devel@redhat.com, Alexander Viro , linux-fsdevel@vger.kernel.org, linux-raid@vger.kernel.org, David Sterba , linux-btrfs@vger.kernel.org, "Darrick J . Wong" , linux-xfs@vger.kernel.org, Gao Xiang , Christoph Hellwig , linux-ext4@vger.kernel.org, Coly Li , linux-bcache@vger.kernel.org, Boaz Harrosh , Bob Peterson , cluster-devel@redhat.com, Ming Lei Subject: [PATCH V13 07/19] block: use bio_for_each_bvec() to compute multi-page bvec count Date: Fri, 11 Jan 2019 19:01:15 +0800 Message-Id: <20190111110127.21664-8-ming.lei@redhat.com> In-Reply-To: <20190111110127.21664-1-ming.lei@redhat.com> References: <20190111110127.21664-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.84 on 10.5.11.22 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Fri, 11 Jan 2019 11:03:36 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP First it is more efficient to use bio_for_each_bvec() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many multi-page bvecs there are in the bio. Secondly once bio_for_each_bvec() is used, the bvec may need to be splitted because its length can be very longer than max segment size, so we have to split the big bvec into several segments. Thirdly when splitting multi-page bvec into segments, the max segment limit may be reached, so the bio split need to be considered under this situation too. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-merge.c | 99 ++++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 79 insertions(+), 20 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index f85d878f313d..abe1c89c1253 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -161,6 +161,69 @@ static inline unsigned get_max_io_size(struct request_queue *q, return sectors; } +static unsigned get_max_segment_size(struct request_queue *q, + unsigned offset) +{ + unsigned long mask = queue_segment_boundary(q); + + return min_t(unsigned long, mask - (mask & offset) + 1, + queue_max_segment_size(q)); +} + +/* + * Split the bvec @bv into segments, and update all kinds of + * variables. + */ +static bool bvec_split_segs(struct request_queue *q, struct bio_vec *bv, + unsigned *nsegs, unsigned *last_seg_size, + unsigned *front_seg_size, unsigned *sectors) +{ + unsigned len = bv->bv_len; + unsigned total_len = 0; + unsigned new_nsegs = 0, seg_size = 0; + + /* + * Multi-page bvec may be too big to hold in one segment, so the + * current bvec has to be splitted as multiple segments. + */ + while (len && new_nsegs + *nsegs < queue_max_segments(q)) { + seg_size = get_max_segment_size(q, bv->bv_offset + total_len); + seg_size = min(seg_size, len); + + new_nsegs++; + total_len += seg_size; + len -= seg_size; + + if ((bv->bv_offset + total_len) & queue_virt_boundary(q)) + break; + } + + if (!new_nsegs) + return !!len; + + /* update front segment size */ + if (!*nsegs) { + unsigned first_seg_size; + + if (new_nsegs == 1) + first_seg_size = get_max_segment_size(q, bv->bv_offset); + else + first_seg_size = queue_max_segment_size(q); + + if (*front_seg_size < first_seg_size) + *front_seg_size = first_seg_size; + } + + /* update other varibles */ + *last_seg_size = seg_size; + *nsegs += new_nsegs; + if (sectors) + *sectors += total_len >> 9; + + /* split in the middle of the bvec if len != 0 */ + return !!len; +} + static struct bio *blk_bio_segment_split(struct request_queue *q, struct bio *bio, struct bio_set *bs, @@ -174,7 +237,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, struct bio *new = NULL; const unsigned max_sectors = get_max_io_size(q, bio); - bio_for_each_segment(bv, bio, iter) { + bio_for_each_bvec(bv, bio, iter) { /* * If the queue doesn't support SG gaps and adding this * offset would create a gap, disallow it. @@ -189,8 +252,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, */ if (nsegs < queue_max_segments(q) && sectors < max_sectors) { - nsegs++; - sectors = max_sectors; + /* split in the middle of bvec */ + bv.bv_len = (max_sectors - sectors) << 9; + bvec_split_segs(q, &bv, &nsegs, + &seg_size, + &front_seg_size, + §ors); } goto split; } @@ -212,14 +279,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, if (nsegs == queue_max_segments(q)) goto split; - if (nsegs == 1 && seg_size > front_seg_size) - front_seg_size = seg_size; - - nsegs++; bvprv = bv; bvprvp = &bvprv; - seg_size = bv.bv_len; - sectors += bv.bv_len >> 9; + + if (bvec_split_segs(q, &bv, &nsegs, &seg_size, + &front_seg_size, §ors)) + goto split; } @@ -233,8 +298,6 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, bio = new; } - if (nsegs == 1 && seg_size > front_seg_size) - front_seg_size = seg_size; bio->bi_seg_front_size = front_seg_size; if (seg_size > bio->bi_seg_back_size) bio->bi_seg_back_size = seg_size; @@ -297,6 +360,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, struct bio_vec bv, bvprv = { NULL }; int prev = 0; unsigned int seg_size, nr_phys_segs; + unsigned front_seg_size = bio->bi_seg_front_size; struct bio *fbio, *bbio; struct bvec_iter iter; @@ -316,7 +380,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, seg_size = 0; nr_phys_segs = 0; for_each_bio(bio) { - bio_for_each_segment(bv, bio, iter) { + bio_for_each_bvec(bv, bio, iter) { /* * If SG merging is disabled, each bio vector is * a segment @@ -336,20 +400,15 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, continue; } new_segment: - if (nr_phys_segs == 1 && seg_size > - fbio->bi_seg_front_size) - fbio->bi_seg_front_size = seg_size; - - nr_phys_segs++; bvprv = bv; prev = 1; - seg_size = bv.bv_len; + bvec_split_segs(q, &bv, &nr_phys_segs, &seg_size, + &front_seg_size, NULL); } bbio = bio; } - if (nr_phys_segs == 1 && seg_size > fbio->bi_seg_front_size) - fbio->bi_seg_front_size = seg_size; + fbio->bi_seg_front_size = front_seg_size; if (seg_size > bbio->bi_seg_back_size) bbio->bi_seg_back_size = seg_size;