From patchwork Fri Jun 10 11:07:38 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 9169483 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 7AD0860573 for ; Fri, 10 Jun 2016 11:08:04 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 6433F1FF45 for ; Fri, 10 Jun 2016 11:08:04 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 58D6B2835A; Fri, 10 Jun 2016 11:08:04 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 493FD1FF45 for ; Fri, 10 Jun 2016 11:08:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750710AbcFJLIB (ORCPT ); Fri, 10 Jun 2016 07:08:01 -0400 Received: from mail-pa0-f67.google.com ([209.85.220.67]:36641 "EHLO mail-pa0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751833AbcFJLIA (ORCPT ); Fri, 10 Jun 2016 07:08:00 -0400 Received: by mail-pa0-f67.google.com with SMTP id fg1so4902587pad.3; Fri, 10 Jun 2016 04:08:00 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:from:to:cc:subject:date:message-id; bh=DExLnvFDhg/AoimPFtfMvMATvWt6wJp883+mwhvo+6M=; b=G+UmJHP3OgpOTrkyAtjunm6qK/z4trM43Eu7tJIzd+asUDQvDy4V4t7CI+0hu6azL9 xDiu5j2doIVRJSwNPYOFNWBLc+6z/VfuNm5MmB3H8BC0mDJt8Bhfusdul0c0BFjK1YL5 d0Th4Tp6KzhnMZaqTDs6+cx4YAjzUGfLQ9Y6hBBySToxacnuyIXW+lAgTm9Sd/Rgdbmp MmreY2iXunK+lth4pds272EgPMZB3S9/Y1hQvW8MEYd7KjzexLFKLCgmwzmaf/dxh2iM SLHOPNBMmhvXJcHlVrkYC1ORy+9ZLgExrOL9MuTB/U8N1WIWDlhbZrGEQ9ZrJCIQQSUg PPkg== X-Gm-Message-State: ALyK8tJuZceeOmOuMmi4wZyKVUb7JO7FTNOX2a3g5cAkxxPlv95QBP5hi/j0I6Zfb2d23w== X-Received: by 10.66.144.228 with SMTP id sp4mr1759291pab.107.1465556879543; Fri, 10 Jun 2016 04:07:59 -0700 (PDT) Received: from localhost ([45.35.47.137]) by smtp.gmail.com with ESMTPSA id tb7sm16934291pab.21.2016.06.10.04.07.58 (version=TLS1_2 cipher=AES128-SHA bits=128/128); Fri, 10 Jun 2016 04:07:58 -0700 (PDT) From: Ming Lei To: Jens Axboe , linux-kernel@vger.kernel.org Cc: linux-block@vger.kernel.org, Christoph Hellwig , Kent Overstreet , Ming Lei , stable@vger.kernel.org (4.3+), Shaohua Li , Jens Axboe Subject: [PATCH v2] block: make sure big bio is splitted into at most 256 bvecs Date: Fri, 10 Jun 2016 19:07:38 +0800 Message-Id: <1465556858-30949-1-git-send-email-ming.lei@canonical.com> X-Mailer: git-send-email 1.9.1 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP After arbitrary bio size is supported, the incoming bio may be very big. We have to split the bio into small bios so that each holds at most BIO_MAX_PAGES bvecs for safety reason, such as bio_clone(). This patch fixes the following kernel crash: > [ 172.660142] BUG: unable to handle kernel NULL pointer dereference at 0000000000000028 > [ 172.660229] IP: [] bio_trim+0xf/0x2a > [ 172.660289] PGD 7faf3e067 PUD 7f9279067 PMD 0 > [ 172.660399] Oops: 0000 [#1] SMP > [...] > [ 172.664780] Call Trace: > [ 172.664813] [] ? raid1_make_request+0x2e8/0xad7 [raid1] > [ 172.664846] [] ? blk_queue_split+0x377/0x3d4 > [ 172.664880] [] ? md_make_request+0xf6/0x1e9 [md_mod] > [ 172.664912] [] ? generic_make_request+0xb5/0x155 > [ 172.664947] [] ? prio_io+0x85/0x95 [bcache] > [ 172.664981] [] ? register_cache_set+0x355/0x8d0 [bcache] > [ 172.665016] [] ? register_bcache+0x1006/0x1174 [bcache] The issue can be reproduced by the following steps: - create one raid1 over two virtio-blk - build bcache device over the above raid1 and another cache device and bucket size is set as 2Mbytes - set cache mode as writeback - run random write over ext4 on the bcache device Fixes: 54efd50(block: make generic_make_request handle arbitrarily sized bios) Reported-by: Sebastian Roesner Reported-by: Eric Wheeler Cc: stable@vger.kernel.org (4.3+) Cc: Shaohua Li Acked-by: Kent Overstreet Signed-off-by: Ming Lei --- V2: - don't mark as REQ_NOMERGE in case the bio is splitted for reaching the limit of bvecs count V1: - Kent pointed out that using max io size can't cover the case of non-full bvecs/pages block/blk-merge.c | 35 ++++++++++++++++++++++++++++++++--- 1 file changed, 32 insertions(+), 3 deletions(-) diff --git a/block/blk-merge.c b/block/blk-merge.c index c265348..839529b 100644 --- a/block/blk-merge.c +++ b/block/blk-merge.c @@ -85,7 +85,8 @@ static inline unsigned get_max_io_size(struct request_queue *q, static struct bio *blk_bio_segment_split(struct request_queue *q, struct bio *bio, struct bio_set *bs, - unsigned *segs) + unsigned *segs, + bool *no_merge) { struct bio_vec bv, bvprv, *bvprvp = NULL; struct bvec_iter iter; @@ -94,9 +95,34 @@ static struct bio *blk_bio_segment_split(struct request_queue *q, bool do_split = true; struct bio *new = NULL; const unsigned max_sectors = get_max_io_size(q, bio); + unsigned bvecs = 0; + + *no_merge = true; bio_for_each_segment(bv, bio, iter) { /* + * With arbitrary bio size, the incoming bio may be very + * big. We have to split the bio into small bios so that + * each holds at most BIO_MAX_PAGES bvecs because + * bio_clone() can fail to allocate big bvecs. + * + * It should have been better to apply the limit per + * request queue in which bio_clone() is involved, + * instead of globally. The biggest blocker is + * bio_clone() in bio bounce. + * + * If bio is splitted by this reason, we should allow + * to continue bios merging. + * + * TODO: deal with bio bounce's bio_clone() gracefully + * and convert the global limit into per-queue limit. + */ + if (bvecs++ >= BIO_MAX_PAGES) { + *no_merge = false; + goto split; + } + + /* * If the queue doesn't support SG gaps and adding this * offset would create a gap, disallow it. */ @@ -171,13 +197,15 @@ void blk_queue_split(struct request_queue *q, struct bio **bio, { struct bio *split, *res; unsigned nsegs; + bool no_merge_for_split = true; if (bio_op(*bio) == REQ_OP_DISCARD) split = blk_bio_discard_split(q, *bio, bs, &nsegs); else if (bio_op(*bio) == REQ_OP_WRITE_SAME) split = blk_bio_write_same_split(q, *bio, bs, &nsegs); else - split = blk_bio_segment_split(q, *bio, q->bio_split, &nsegs); + split = blk_bio_segment_split(q, *bio, q->bio_split, &nsegs, + &no_merge_for_split); /* physical segments can be figured out during splitting */ res = split ? split : *bio; @@ -186,7 +214,8 @@ void blk_queue_split(struct request_queue *q, struct bio **bio, if (split) { /* there isn't chance to merge the splitted bio */ - split->bi_rw |= REQ_NOMERGE; + if (no_merge_for_split) + split->bi_rw |= REQ_NOMERGE; bio_chain(split, *bio); trace_block_split(q, split, (*bio)->bi_iter.bi_sector);