From patchwork Mon Jan 21 08:17:53 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 10773173 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B6CEF13B4 for ; Mon, 21 Jan 2019 08:20:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A607529EE6 for ; Mon, 21 Jan 2019 08:20:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 9936C29EEA; Mon, 21 Jan 2019 08:20:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F086729EE6 for ; Mon, 21 Jan 2019 08:20:20 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 230758E0030; Mon, 21 Jan 2019 03:20:20 -0500 (EST) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 1B63F8E0025; Mon, 21 Jan 2019 03:20:20 -0500 (EST) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 080C08E0030; Mon, 21 Jan 2019 03:20:20 -0500 (EST) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from mail-qk1-f198.google.com (mail-qk1-f198.google.com [209.85.222.198]) by kanga.kvack.org (Postfix) with ESMTP id CD3708E0025 for ; Mon, 21 Jan 2019 03:20:19 -0500 (EST) Received: by mail-qk1-f198.google.com with SMTP id y27so18499038qkj.21 for ; Mon, 21 Jan 2019 00:20:19 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-original-authentication-results:x-gm-message-state:from:to:cc :subject:date:message-id:in-reply-to:references; bh=WCwWGna91+tAg/4Zdp/1JFxnNkzZtaFKiqbTCiVBFTo=; b=Np9mFIX9T0VmnY0dEyfr1i6dMoMQSCQOPJop6Pyy8ujWhupm6q/ojkAXRH+koLwhfR NaAgl+y8O/NH5Wal7NnyKKGTzaCKjmzus4BYHN6mnelR6S1uGWNS45DuMV1wHjq3e4b3 pVAILXy/AOc98KDqtVhaP5+KaLnNai77/eKcwJOfdX27RegwAanE1Sg06NXJpeDWmBb/ VOnwfbeKNwWk7leocUCDrs1LfEmznnlDAUPI13UmcRKxW/wTAkULjEUjRy6q2Sh6XlDo 98FCheLOE8/RumLmsH9z1N1xaP3or5prXH/2KntMrBR9MqtKs61xNirDRkAmw3MbtvPy f00g== X-Original-Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com X-Gm-Message-State: AJcUukcR4ASBvAQehNa/qaKmBnmlBzIC3hv5MfOfPRjBIIfZQJa0mDIx bQOBL5GgjfBYr0fkqLKCpVo4xZs6RiRwimO6ynwgQkY9qRILau2AGKjpZIg9WGlY4BoWoZXdZOS o6rlWmz9Lb9UUqUxWGujxm2jMHqTHgI+mV3TJBB9nKRKgvdVCp8uB6UQP+uWXhEy5Tw== X-Received: by 2002:a0c:eb41:: with SMTP id c1mr25696445qvq.211.1548058819593; Mon, 21 Jan 2019 00:20:19 -0800 (PST) X-Google-Smtp-Source: ALg8bN7ntN5NBgv/iEoJPNK0nMfYhFidgV4Q5TCCvDDjrwKiiDTMbDXSDWIXoc1Q/ktYbKWYc/u7 X-Received: by 2002:a0c:eb41:: with SMTP id c1mr25696415qvq.211.1548058818855; Mon, 21 Jan 2019 00:20:18 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548058818; cv=none; d=google.com; s=arc-20160816; b=nLZy63vY189GJRxSdxjrCf6/JhKhI2DRUjaFh6NT99RCxPImrU96eHcbw+bmCX/96e vL/mS2zcRhAsVVaxZQnGtFtVCHjUDCFJkTCh5+UQDr0WSNRM1hd6P2hsupCpaNj4gSny iUE8gRUsf2EsFwaOmUKbgqmInFn5n55K4Jgy2qHwapeHOs2+3nkCqktOG53aEsP1LpLj sE6Sry4C7RqN7qElfYoEe9kOrWpwnB5Bznbjlw8rAEeCkVCX7v3TN8xUKW0SiZHGm5HY fbwr7eUW5ONt56wj6wJ2VHov7webUhw8d9zKObAVaJjC5FaOSLVzjdm3a8QYcg9eBeUZ mZNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from; bh=WCwWGna91+tAg/4Zdp/1JFxnNkzZtaFKiqbTCiVBFTo=; b=P16o+sfn8I1gIi+rjc48084xbmwElrr/rFsB/pv67YfjB5fiWMZklQl7zAeKrxpYEf 7hj+DOJB+cIeIbcqhqDQTvwg60cxFR659dVls8E+ZELk3l9WumGEfClUrH161Pd/OZZu gZxvZCuSB8gP8IlhfeXMviMBSPecpddSMlgaX+bL7XPILawPAe27dmqLLe2/4VAbqg+l UOfXJemhc/fxwLZq6bbUGz7J27034sFVxPFvugCiBZRO0r+8cyPZL4n/pqlcM5x0LgZ7 1mbgxjcOpQomzCs3ShvinMYPP7xoRqoEFZvrlWPIGwb0mlCnno8G990yEvthdXwEZ1D1 73sA== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from mx1.redhat.com (mx1.redhat.com. [209.132.183.28]) by mx.google.com with ESMTPS id l28si1396822qtb.119.2019.01.21.00.20.18 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 21 Jan 2019 00:20:18 -0800 (PST) Received-SPF: pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) client-ip=209.132.183.28; Authentication-Results: mx.google.com; spf=pass (google.com: domain of ming.lei@redhat.com designates 209.132.183.28 as permitted sender) smtp.mailfrom=ming.lei@redhat.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: from smtp.corp.redhat.com (int-mx06.intmail.prod.int.phx2.redhat.com [10.5.11.16]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 98B4013822B; Mon, 21 Jan 2019 08:20:17 +0000 (UTC) Received: from localhost (ovpn-8-22.pek2.redhat.com [10.72.8.22]) by smtp.corp.redhat.com (Postfix) with ESMTP id 47C2A5C54B; Mon, 21 Jan 2019 08:19:55 +0000 (UTC) From: Ming Lei To: Jens Axboe Cc: linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Theodore Ts'o , Omar Sandoval , Sagi Grimberg , Dave Chinner , Kent Overstreet , Mike Snitzer , dm-devel@redhat.com, Alexander Viro , linux-fsdevel@vger.kernel.org, linux-raid@vger.kernel.org, David Sterba , linux-btrfs@vger.kernel.org, "Darrick J . Wong" , linux-xfs@vger.kernel.org, Gao Xiang , Christoph Hellwig , linux-ext4@vger.kernel.org, Coly Li , linux-bcache@vger.kernel.org, Boaz Harrosh , Bob Peterson , cluster-devel@redhat.com, Ming Lei Subject: [PATCH V14 06/18] block: use bio_for_each_mp_bvec() to compute multi-page bvec count Date: Mon, 21 Jan 2019 16:17:53 +0800 Message-Id: <20190121081805.32727-7-ming.lei@redhat.com> In-Reply-To: <20190121081805.32727-1-ming.lei@redhat.com> References: <20190121081805.32727-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.16 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.38]); Mon, 21 Jan 2019 08:20:18 +0000 (UTC) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: X-Virus-Scanned: ClamAV using ClamSMTP First it is more efficient to use bio_for_each_mp_bvec() in both blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how many multi-page bvecs there are in the bio. Secondly once bio_for_each_mp_bvec() is used, the bvec may need to be splitted because its length can be very longer than max segment size, so we have to split the big bvec into several segments. Thirdly when splitting multi-page bvec into segments, the max segment limit may be reached, so the bio split need to be considered under this situation too. Reviewed-by: Christoph Hellwig Reviewed-by: Omar Sandoval Signed-off-by: Ming Lei --- block/blk-merge.c | 103 +++++++++++++++++++++++++++++++++++++++++++----------- 1 file changed, 83 insertions(+), 20 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index f85d878f313d..2dfc30d8bc77 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -161,6 +161,73 @@ static inline unsigned get_max_io_size(struct request_queue *q, return sectors; } +static unsigned get_max_segment_size(struct request_queue *q, + unsigned offset) +{ + unsigned long mask = queue_segment_boundary(q); + + /* default segment boundary mask means no boundary limit */ + if (mask == BLK_SEG_BOUNDARY_MASK) + return queue_max_segment_size(q); + + return min_t(unsigned long, mask - (mask & offset) + 1, + queue_max_segment_size(q)); +} + +/* + * Split the bvec @bv into segments, and update all kinds of + * variables. + */ +static bool bvec_split_segs(struct request_queue *q, struct bio_vec *bv, + unsigned *nsegs, unsigned *last_seg_size, + unsigned *front_seg_size, unsigned *sectors) +{ + unsigned len = bv->bv_len; + unsigned total_len = 0; + unsigned new_nsegs = 0, seg_size = 0; + + /* + * Multi-page bvec may be too big to hold in one segment, so the + * current bvec has to be splitted as multiple segments. + */ + while (len && new_nsegs + *nsegs < queue_max_segments(q)) { + seg_size = get_max_segment_size(q, bv->bv_offset + total_len); + seg_size = min(seg_size, len); + + new_nsegs++; + total_len += seg_size; + len -= seg_size; + + if ((bv->bv_offset + total_len) & queue_virt_boundary(q)) + break; + } + + if (!new_nsegs) + return !!len; + + /* update front segment size */ + if (!*nsegs) { + unsigned first_seg_size; + + if (new_nsegs == 1) + first_seg_size = get_max_segment_size(q, bv->bv_offset); + else + first_seg_size = queue_max_segment_size(q); + + if (*front_seg_size < first_seg_size) + *front_seg_size = first_seg_size; + } + + /* update other varibles */ + *last_seg_size = seg_size; + *nsegs += new_nsegs; + if (sectors) + *sectors += total_len >> 9; + + /* split in the middle of the bvec if len != 0 */ + return !!len; +} + static struct bio *blk_bio_segment_split(struct request_queue *q, struct bio *bio, struct bio_set *bs, @@ -174,7 +241,7 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, struct bio *new = NULL; const unsigned max_sectors = get_max_io_size(q, bio); - bio_for_each_segment(bv, bio, iter) { + bio_for_each_mp_bvec(bv, bio, iter) { /* * If the queue doesn't support SG gaps and adding this * offset would create a gap, disallow it. @@ -189,8 +256,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, */ if (nsegs < queue_max_segments(q) && sectors < max_sectors) { - nsegs++; - sectors = max_sectors; + /* split in the middle of bvec */ + bv.bv_len = (max_sectors - sectors) << 9; + bvec_split_segs(q, &bv, &nsegs, + &seg_size, + &front_seg_size, + §ors); } goto split; } @@ -212,14 +283,12 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, if (nsegs == queue_max_segments(q)) goto split; - if (nsegs == 1 && seg_size > front_seg_size) - front_seg_size = seg_size; - - nsegs++; bvprv = bv; bvprvp = &bvprv; - seg_size = bv.bv_len; - sectors += bv.bv_len >> 9; + + if (bvec_split_segs(q, &bv, &nsegs, &seg_size, + &front_seg_size, §ors)) + goto split; } @@ -233,8 +302,6 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, bio = new; } - if (nsegs == 1 && seg_size > front_seg_size) - front_seg_size = seg_size; bio->bi_seg_front_size = front_seg_size; if (seg_size > bio->bi_seg_back_size) bio->bi_seg_back_size = seg_size; @@ -297,6 +364,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, struct bio_vec bv, bvprv = { NULL }; int prev = 0; unsigned int seg_size, nr_phys_segs; + unsigned front_seg_size = bio->bi_seg_front_size; struct bio *fbio, *bbio; struct bvec_iter iter; @@ -316,7 +384,7 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, seg_size = 0; nr_phys_segs = 0; for_each_bio(bio) { - bio_for_each_segment(bv, bio, iter) { + bio_for_each_mp_bvec(bv, bio, iter) { /* * If SG merging is disabled, each bio vector is * a segment @@ -336,20 +404,15 @@ static unsigned int __blk_recalc_rq_segments(struct request_queue *q, continue; } new_segment: - if (nr_phys_segs == 1 && seg_size > - fbio->bi_seg_front_size) - fbio->bi_seg_front_size = seg_size; - - nr_phys_segs++; bvprv = bv; prev = 1; - seg_size = bv.bv_len; + bvec_split_segs(q, &bv, &nr_phys_segs, &seg_size, + &front_seg_size, NULL); } bbio = bio; } - if (nr_phys_segs == 1 && seg_size > fbio->bi_seg_front_size) - fbio->bi_seg_front_size = seg_size; + fbio->bi_seg_front_size = front_seg_size; if (seg_size > bbio->bi_seg_back_size) bbio->bi_seg_back_size = seg_size;