From patchwork Sat Nov 27 10:10:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12642175 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FE18C4332F for ; Sat, 27 Nov 2021 10:00:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354162AbhK0KEE (ORCPT ); Sat, 27 Nov 2021 05:04:04 -0500 Received: from szxga02-in.huawei.com ([45.249.212.188]:27304 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353731AbhK0KCE (ORCPT ); Sat, 27 Nov 2021 05:02:04 -0500 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4J1Rs02W9fzbj4D; Sat, 27 Nov 2021 17:58:44 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Sat, 27 Nov 2021 17:58:48 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Sat, 27 Nov 2021 17:58:47 +0800 From: Yu Kuai To: , , CC: , , , , Subject: [PATCH 1/4] Revert "blk-throttle: remove tg_drain_bios" Date: Sat, 27 Nov 2021 18:10:56 +0800 Message-ID: <20211127101059.477405-2-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211127101059.477405-1-yukuai3@huawei.com> References: <20211127101059.477405-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This reverts commit 32e3374304c7c317c05a61f3ddc315dbd46424f2. Signed-off-by: Yu Kuai --- block/blk-throttle.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 39bb6e68a9a2..230e300c5856 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2257,6 +2257,28 @@ void blk_throtl_bio_endio(struct bio *bio) } #endif +/* + * Dispatch all bios from all children tg's queued on @parent_sq. On + * return, @parent_sq is guaranteed to not have any active children tg's + * and all bios from previously active tg's are on @parent_sq->bio_lists[]. + */ +static void tg_drain_bios(struct throtl_service_queue *parent_sq) +{ + struct throtl_grp *tg; + + while ((tg = throtl_rb_first(parent_sq))) { + struct throtl_service_queue *sq = &tg->service_queue; + struct bio *bio; + + throtl_dequeue_tg(tg); + + while ((bio = throtl_peek_queued(&sq->queued[READ]))) + tg_dispatch_one_bio(tg, bio_data_dir(bio)); + while ((bio = throtl_peek_queued(&sq->queued[WRITE]))) + tg_dispatch_one_bio(tg, bio_data_dir(bio)); + } +} + int blk_throtl_init(struct request_queue *q) { struct throtl_data *td; From patchwork Sat Nov 27 10:10:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12642181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 366D7C433F5 for ; Sat, 27 Nov 2021 10:01:03 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354346AbhK0KEP (ORCPT ); Sat, 27 Nov 2021 05:04:15 -0500 Received: from szxga03-in.huawei.com ([45.249.212.189]:28181 "EHLO szxga03-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353795AbhK0KCN (ORCPT ); Sat, 27 Nov 2021 05:02:13 -0500 Received: from dggemv704-chm.china.huawei.com (unknown [172.30.72.57]) by szxga03-in.huawei.com (SkyGuard) with ESMTP id 4J1Rq346b6z8vg2; Sat, 27 Nov 2021 17:57:03 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by dggemv704-chm.china.huawei.com (10.3.19.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Sat, 27 Nov 2021 17:58:49 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Sat, 27 Nov 2021 17:58:48 +0800 From: Yu Kuai To: , , CC: , , , , Subject: [PATCH 2/4] blk-throtl: don't warn in tg_drain_bios() Date: Sat, 27 Nov 2021 18:10:57 +0800 Message-ID: <20211127101059.477405-3-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211127101059.477405-1-yukuai3@huawei.com> References: <20211127101059.477405-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org tg_drain_bios() will iterate until throtl_rb_first() return NULL, don't warn in such situation. Signed-off-by: Yu Kuai --- block/blk-throttle.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 230e300c5856..25822c88bea1 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -497,12 +497,13 @@ static void throtl_pd_free(struct blkg_policy_data *pd) } static struct throtl_grp * -throtl_rb_first(struct throtl_service_queue *parent_sq) +throtl_rb_first(struct throtl_service_queue *parent_sq, bool warn) { struct rb_node *n; n = rb_first_cached(&parent_sq->pending_tree); - WARN_ON_ONCE(!n); + if (warn) + WARN_ON_ONCE(!n); if (!n) return NULL; return rb_entry_tg(n); @@ -520,7 +521,7 @@ static void update_min_dispatch_time(struct throtl_service_queue *parent_sq) { struct throtl_grp *tg; - tg = throtl_rb_first(parent_sq); + tg = throtl_rb_first(parent_sq, true); if (!tg) return; @@ -1089,7 +1090,7 @@ static int throtl_select_dispatch(struct throtl_service_queue *parent_sq) if (!parent_sq->nr_pending) break; - tg = throtl_rb_first(parent_sq); + tg = throtl_rb_first(parent_sq, true); if (!tg) break; @@ -2266,7 +2267,7 @@ static void tg_drain_bios(struct throtl_service_queue *parent_sq) { struct throtl_grp *tg; - while ((tg = throtl_rb_first(parent_sq))) { + while ((tg = throtl_rb_first(parent_sq, false))) { struct throtl_service_queue *sq = &tg->service_queue; struct bio *bio; From patchwork Sat Nov 27 10:10:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12642177 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 554B4C433F5 for ; Sat, 27 Nov 2021 10:00:58 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354209AbhK0KEI (ORCPT ); Sat, 27 Nov 2021 05:04:08 -0500 Received: from szxga08-in.huawei.com ([45.249.212.255]:28110 "EHLO szxga08-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353783AbhK0KCG (ORCPT ); Sat, 27 Nov 2021 05:02:06 -0500 Received: from dggemv703-chm.china.huawei.com (unknown [172.30.72.53]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4J1Rp56tk8z1DH9w; Sat, 27 Nov 2021 17:56:13 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by dggemv703-chm.china.huawei.com (10.3.19.46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Sat, 27 Nov 2021 17:58:50 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Sat, 27 Nov 2021 17:58:49 +0800 From: Yu Kuai To: , , CC: , , , , Subject: [PATCH 3/4] blk-throtl: introduce blk_throtl_cancel_bios() Date: Sat, 27 Nov 2021 18:10:58 +0800 Message-ID: <20211127101059.477405-4-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211127101059.477405-1-yukuai3@huawei.com> References: <20211127101059.477405-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This function is used to cancel all throttled bios. Noted this modification is mainly from revertion of commit b77412372b68 ("blk-throttle: remove blk_throtl_drain"). Signed-off-by: Yu Kuai --- block/blk-throttle.c | 37 +++++++++++++++++++++++++++++++++++++ block/blk-throttle.h | 2 ++ 2 files changed, 39 insertions(+) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 25822c88bea1..b31ae8a2c8b5 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2280,6 +2280,43 @@ static void tg_drain_bios(struct throtl_service_queue *parent_sq) } } +/** + * blk_throtl_cancel_bios - cancel throttled bios + * @q: request_queue to cancel throttled bios for + * + * This function is called to error all currently throttled bios on @q. + */ +void blk_throtl_cancel_bios(struct request_queue *q) +{ + struct throtl_data *td = q->td; + struct blkcg_gq *blkg; + struct cgroup_subsys_state *pos_css; + struct bio *bio; + int rw; + + rcu_read_lock(); + + /* + * Drain each tg while doing post-order walk on the blkg tree, so + * that all bios are propagated to td->service_queue. It'd be + * better to walk service_queue tree directly but blkg walk is + * easier. + */ + blkg_for_each_descendant_post(blkg, pos_css, td->queue->root_blkg) + tg_drain_bios(&blkg_to_tg(blkg)->service_queue); + + /* finally, transfer bios from top-level tg's into the td */ + tg_drain_bios(&td->service_queue); + + rcu_read_unlock(); + + /* all bios now should be in td->service_queue, cancel them */ + for (rw = READ; rw <= WRITE; rw++) + while ((bio = throtl_pop_queued(&td->service_queue.queued[rw], + NULL))) + bio_io_error(bio); +} + int blk_throtl_init(struct request_queue *q) { struct throtl_data *td; diff --git a/block/blk-throttle.h b/block/blk-throttle.h index 175f03abd9e4..9d67d5139954 100644 --- a/block/blk-throttle.h +++ b/block/blk-throttle.h @@ -160,12 +160,14 @@ static inline void blk_throtl_exit(struct request_queue *q) { } static inline void blk_throtl_register_queue(struct request_queue *q) { } static inline void blk_throtl_charge_bio_split(struct bio *bio) { } static inline bool blk_throtl_bio(struct bio *bio) { return false; } +#define blk_throtl_cancel_bios(q) do { } while (0) #else /* CONFIG_BLK_DEV_THROTTLING */ int blk_throtl_init(struct request_queue *q); void blk_throtl_exit(struct request_queue *q); void blk_throtl_register_queue(struct request_queue *q); void blk_throtl_charge_bio_split(struct bio *bio); bool __blk_throtl_bio(struct bio *bio); +void blk_throtl_cancel_bios(struct request_queue *q); static inline bool blk_throtl_bio(struct bio *bio) { struct throtl_grp *tg = blkg_to_tg(bio->bi_blkg); From patchwork Sat Nov 27 10:10:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12642179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB20AC433EF for ; Sat, 27 Nov 2021 10:00:59 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1354222AbhK0KEI (ORCPT ); Sat, 27 Nov 2021 05:04:08 -0500 Received: from szxga01-in.huawei.com ([45.249.212.187]:31917 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1353793AbhK0KCG (ORCPT ); Sat, 27 Nov 2021 05:02:06 -0500 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4J1Rs268WmzcbMg; Sat, 27 Nov 2021 17:58:46 +0800 (CST) Received: from kwepemm600009.china.huawei.com (7.193.23.164) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Sat, 27 Nov 2021 17:58:50 +0800 Received: from huawei.com (10.175.127.227) by kwepemm600009.china.huawei.com (7.193.23.164) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2308.20; Sat, 27 Nov 2021 17:58:49 +0800 From: Yu Kuai To: , , CC: , , , , Subject: [PATCH 4/4] block: cancel all throttled bios in del_gendisk() Date: Sat, 27 Nov 2021 18:10:59 +0800 Message-ID: <20211127101059.477405-5-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20211127101059.477405-1-yukuai3@huawei.com> References: <20211127101059.477405-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems702-chm.china.huawei.com (10.3.19.179) To kwepemm600009.china.huawei.com (7.193.23.164) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Throttled bios can't be issued after del_gendisk() is done, thus it's better to cancel them immediately rather than waiting for throttle is done. For example, if user thread is throttled with low bps while it's issuing large io, and the device is deleted. The user thread will wait for a long time for io to return. Signed-off-by: Yu Kuai --- block/genhd.c | 2 ++ 1 file changed, 2 insertions(+) diff --git a/block/genhd.c b/block/genhd.c index 8e9cbf23c510..24fa3356d164 100644 --- a/block/genhd.c +++ b/block/genhd.c @@ -28,6 +28,7 @@ #include "blk.h" #include "blk-rq-qos.h" +#include "blk-throttle.h" static struct kobject *block_depr; @@ -619,6 +620,7 @@ void del_gendisk(struct gendisk *disk) blk_mq_freeze_queue_wait(q); + blk_throtl_cancel_bios(q); rq_qos_exit(q); blk_sync_queue(q); blk_flush_integrity();