From patchwork Mon Mar 19 00:36:19 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Michael Lyle X-Patchwork-Id: 10291465 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 2C066600F6 for ; Mon, 19 Mar 2018 00:36:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1DF0428EDF for ; Mon, 19 Mar 2018 00:36:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 128EB28F56; Mon, 19 Mar 2018 00:36:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8EA4F28F29 for ; Mon, 19 Mar 2018 00:36:53 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754795AbeCSAgw (ORCPT ); Sun, 18 Mar 2018 20:36:52 -0400 Received: from mail-pl0-f67.google.com ([209.85.160.67]:36251 "EHLO mail-pl0-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754685AbeCSAgs (ORCPT ); Sun, 18 Mar 2018 20:36:48 -0400 Received: by mail-pl0-f67.google.com with SMTP id 61-v6so9232746plf.3 for ; Sun, 18 Mar 2018 17:36:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=lyle-org.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=/6ll8AIBpnm7GSU1Mw3VsLW7gRyUd9qrjTjlChxSH4I=; b=RJLM/mVX4cgwGh0Fn5mXVG7tobWImtPIIQ9FTv9m2kEu0i9ivm6R2O1cUG8BZctdHv 7K6U/5qRulMa0yf9sqJH+XKmhLF+uMtmzH3Dbrc5CMGmFQ4UQlZ2z4g3kmAfoO7aqMpw cN3zmSY2GcZMHpAT7jsPsk2oE87ArvSCoSQm2UyV+sxO/5DHqTYLwdigVm6L00tD7agc 6ThUCcwL7Thsf5AgvQd0ZcdhfL7s5n2SSOKtCszsrqLgGHQthTV6WyjJO9GyNOVrsCQf daVw6pUMJphYKsmKAfC0rgzGCAAsE1TTllGFCfLAyilymHPLgDRT3tpxvpGOiCoHTm00 Lo6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=/6ll8AIBpnm7GSU1Mw3VsLW7gRyUd9qrjTjlChxSH4I=; b=a9n3bqqzwrPbcIquTmorknm0QKKvQlpUhPdJQf/CE+9RKSiDaj9SG99KAHzD58eCNi pueRvO2/uz2ss9LcqJXKtUPaGmTvuJ0ll7l35u6Q4re4B/dGssBN8Ooq2gSAkiOKr0a2 dkTG+2Fq5ftEZwkWWNtHykjkL8dSpMlegHM4Ldun8V+EjyBVqyhPUFLSy947nJem5Rdw U34fC6KF54SVa3CLSjRN9bC77t+SRGvNDWy2vJIGjt3tHhyS7EwMKCtegUsIitosOzAP dC6BuZnOI7LvCgqxpyuPJt9BZ4QUVEAS3cCSV8kJA3WsSDmxIMZ8W+G/lEFD0LxntSt3 jEfQ== X-Gm-Message-State: AElRT7F9+SZ/Wzg6w47/sQdAob32v7xZT52Edux8qizteXzkkCe3TB4d LQrFhWM+a3+f6FYuMPgyRSMcxw== X-Google-Smtp-Source: AG47ELuyZaFQ7B+UBl2akFXi7LLgoEgl4WEYzykHqwGQH6Ew4R2p7rSe1LIkmvoctf+wFQE/MnSflA== X-Received: by 2002:a17:902:8c8e:: with SMTP id t14-v6mr1078870plo.206.1521419807438; Sun, 18 Mar 2018 17:36:47 -0700 (PDT) Received: from midnight.lan (2600-6c52-6200-09b7-0000-0000-0000-0d66.dhcp6.chtrptr.net. [2600:6c52:6200:9b7::d66]) by smtp.gmail.com with ESMTPSA id a17sm27674857pfc.122.2018.03.18.17.36.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 18 Mar 2018 17:36:46 -0700 (PDT) From: Michael Lyle To: linux-bcache@vger.kernel.org, linux-block@vger.kernel.org Cc: axboe@fb.com, Tang Junhui Subject: [for-4.17 06/20] bcache: fix inaccurate io state for detached bcache devices Date: Sun, 18 Mar 2018 17:36:19 -0700 Message-Id: <20180319003633.27225-7-mlyle@lyle.org> X-Mailer: git-send-email 2.14.1 In-Reply-To: <20180319003633.27225-1-mlyle@lyle.org> References: <20180319003633.27225-1-mlyle@lyle.org> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: Tang Junhui When we run IO in a detached device, and run iostat to shows IO status, normally it will show like bellow (Omitted some fields): Device: ... avgrq-sz avgqu-sz await r_await w_await svctm %util sdd ... 15.89 0.53 1.82 0.20 2.23 1.81 52.30 bcache0 ... 15.89 115.42 0.00 0.00 0.00 2.40 69.60 but after IO stopped, there are still very big avgqu-sz and %util values as bellow: Device: ... avgrq-sz avgqu-sz await r_await w_await svctm %util bcache0 ... 0 5326.32 0.00 0.00 0.00 0.00 100.10 The reason for this issue is that, only generic_start_io_acct() called and no generic_end_io_acct() called for detached device in cached_dev_make_request(). See the code: //start generic_start_io_acct() generic_start_io_acct(q, rw, bio_sectors(bio), &d->disk->part0); if (cached_dev_get(dc)) { //will callback generic_end_io_acct() } else { //will not call generic_end_io_acct() } This patch calls generic_end_io_acct() in the end of IO for detached devices, so we can show IO state correctly. (Modified to use GFP_NOIO in kzalloc() by Coly Li) Changelog: v2: fix typo. v1: the initial version. Signed-off-by: Tang Junhui Reviewed-by: Coly Li Reviewed-by: Hannes Reinecke Reviewed-by: Michael Lyle --- drivers/md/bcache/request.c | 58 +++++++++++++++++++++++++++++++++++++++------ 1 file changed, 51 insertions(+), 7 deletions(-) diff --git a/drivers/md/bcache/request.c b/drivers/md/bcache/request.c index 7aca308bee5b..5c8ae69c8502 100644 --- a/drivers/md/bcache/request.c +++ b/drivers/md/bcache/request.c @@ -986,6 +986,55 @@ static void cached_dev_nodata(struct closure *cl) continue_at(cl, cached_dev_bio_complete, NULL); } +struct detached_dev_io_private { + struct bcache_device *d; + unsigned long start_time; + bio_end_io_t *bi_end_io; + void *bi_private; +}; + +static void detached_dev_end_io(struct bio *bio) +{ + struct detached_dev_io_private *ddip; + + ddip = bio->bi_private; + bio->bi_end_io = ddip->bi_end_io; + bio->bi_private = ddip->bi_private; + + generic_end_io_acct(ddip->d->disk->queue, + bio_data_dir(bio), + &ddip->d->disk->part0, ddip->start_time); + + kfree(ddip); + + bio->bi_end_io(bio); +} + +static void detached_dev_do_request(struct bcache_device *d, struct bio *bio) +{ + struct detached_dev_io_private *ddip; + struct cached_dev *dc = container_of(d, struct cached_dev, disk); + + /* + * no need to call closure_get(&dc->disk.cl), + * because upper layer had already opened bcache device, + * which would call closure_get(&dc->disk.cl) + */ + ddip = kzalloc(sizeof(struct detached_dev_io_private), GFP_NOIO); + ddip->d = d; + ddip->start_time = jiffies; + ddip->bi_end_io = bio->bi_end_io; + ddip->bi_private = bio->bi_private; + bio->bi_end_io = detached_dev_end_io; + bio->bi_private = ddip; + + if ((bio_op(bio) == REQ_OP_DISCARD) && + !blk_queue_discard(bdev_get_queue(dc->bdev))) + bio->bi_end_io(bio); + else + generic_make_request(bio); +} + /* Cached devices - read & write stuff */ static blk_qc_t cached_dev_make_request(struct request_queue *q, @@ -1028,13 +1077,8 @@ static blk_qc_t cached_dev_make_request(struct request_queue *q, else cached_dev_read(dc, s); } - } else { - if ((bio_op(bio) == REQ_OP_DISCARD) && - !blk_queue_discard(bdev_get_queue(dc->bdev))) - bio_endio(bio); - else - generic_make_request(bio); - } + } else + detached_dev_do_request(d, bio); return BLK_QC_T_NONE; }