From patchwork Wed Sep 22 12:51:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12510445 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A190AC433FE for ; Wed, 22 Sep 2021 12:41:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 88F4661100 for ; Wed, 22 Sep 2021 12:41:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236038AbhIVMnN (ORCPT ); Wed, 22 Sep 2021 08:43:13 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:16372 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235945AbhIVMnM (ORCPT ); Wed, 22 Sep 2021 08:43:12 -0400 Received: from dggemv703-chm.china.huawei.com (unknown [172.30.72.56]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4HDyVc728mzR5Z7; Wed, 22 Sep 2021 20:37:28 +0800 (CST) Received: from dggema762-chm.china.huawei.com (10.1.198.204) by dggemv703-chm.china.huawei.com (10.3.19.46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.8; Wed, 22 Sep 2021 20:41:40 +0800 Received: from huawei.com (10.175.127.227) by dggema762-chm.china.huawei.com (10.1.198.204) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.8; Wed, 22 Sep 2021 20:41:40 +0800 From: Yu Kuai To: , CC: , , , , Subject: [PATCH 1/4] Revert "blk-throttle: remove tg_drain_bios" Date: Wed, 22 Sep 2021 20:51:12 +0800 Message-ID: <20210922125115.381752-2-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210922125115.381752-1-yukuai3@huawei.com> References: <20210922125115.381752-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggema762-chm.china.huawei.com (10.1.198.204) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This reverts commit 32e3374304c7c317c05a61f3ddc315dbd46424f2. Signed-off-by: Yu Kuai --- block/blk-throttle.c | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 7c4e7993ba97..43dee985170b 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2412,6 +2412,28 @@ void blk_throtl_bio_endio(struct bio *bio) } #endif +/* + * Dispatch all bios from all children tg's queued on @parent_sq. On + * return, @parent_sq is guaranteed to not have any active children tg's + * and all bios from previously active tg's are on @parent_sq->bio_lists[]. + */ +static void tg_drain_bios(struct throtl_service_queue *parent_sq) +{ + struct throtl_grp *tg; + + while ((tg = throtl_rb_first(parent_sq))) { + struct throtl_service_queue *sq = &tg->service_queue; + struct bio *bio; + + throtl_dequeue_tg(tg); + + while ((bio = throtl_peek_queued(&sq->queued[READ]))) + tg_dispatch_one_bio(tg, bio_data_dir(bio)); + while ((bio = throtl_peek_queued(&sq->queued[WRITE]))) + tg_dispatch_one_bio(tg, bio_data_dir(bio)); + } +} + int blk_throtl_init(struct request_queue *q) { struct throtl_data *td; From patchwork Wed Sep 22 12:51:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12510447 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D2AD3C433F5 for ; Wed, 22 Sep 2021 12:41:45 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id BE86161131 for ; Wed, 22 Sep 2021 12:41:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236042AbhIVMnN (ORCPT ); Wed, 22 Sep 2021 08:43:13 -0400 Received: from szxga02-in.huawei.com ([45.249.212.188]:9903 "EHLO szxga02-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235962AbhIVMnN (ORCPT ); Wed, 22 Sep 2021 08:43:13 -0400 Received: from dggemv711-chm.china.huawei.com (unknown [172.30.72.57]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4HDyVD5hKfz8yjl; Wed, 22 Sep 2021 20:37:08 +0800 (CST) Received: from dggema762-chm.china.huawei.com (10.1.198.204) by dggemv711-chm.china.huawei.com (10.1.198.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.8; Wed, 22 Sep 2021 20:41:41 +0800 Received: from huawei.com (10.175.127.227) by dggema762-chm.china.huawei.com (10.1.198.204) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.8; Wed, 22 Sep 2021 20:41:40 +0800 From: Yu Kuai To: , CC: , , , , Subject: [PATCH 2/4] blk-throtl: don't warn in tg_drain_bios() Date: Wed, 22 Sep 2021 20:51:13 +0800 Message-ID: <20210922125115.381752-3-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210922125115.381752-1-yukuai3@huawei.com> References: <20210922125115.381752-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggema762-chm.china.huawei.com (10.1.198.204) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org tg_drain_bios() will iterate until throtl_rb_first() return NULL, don't warn in such situation. Signed-off-by: Yu Kuai --- block/blk-throttle.c | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 43dee985170b..3892ead7a0b5 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -644,12 +644,13 @@ static void throtl_pd_free(struct blkg_policy_data *pd) } static struct throtl_grp * -throtl_rb_first(struct throtl_service_queue *parent_sq) +throtl_rb_first(struct throtl_service_queue *parent_sq, bool warn) { struct rb_node *n; n = rb_first_cached(&parent_sq->pending_tree); - WARN_ON_ONCE(!n); + if (warn) + WARN_ON_ONCE(!n); if (!n) return NULL; return rb_entry_tg(n); @@ -667,7 +668,7 @@ static void update_min_dispatch_time(struct throtl_service_queue *parent_sq) { struct throtl_grp *tg; - tg = throtl_rb_first(parent_sq); + tg = throtl_rb_first(parent_sq, true); if (!tg) return; @@ -1236,7 +1237,7 @@ static int throtl_select_dispatch(struct throtl_service_queue *parent_sq) if (!parent_sq->nr_pending) break; - tg = throtl_rb_first(parent_sq); + tg = throtl_rb_first(parent_sq, true); if (!tg) break; @@ -2421,7 +2422,7 @@ static void tg_drain_bios(struct throtl_service_queue *parent_sq) { struct throtl_grp *tg; - while ((tg = throtl_rb_first(parent_sq))) { + while ((tg = throtl_rb_first(parent_sq, false))) { struct throtl_service_queue *sq = &tg->service_queue; struct bio *bio; From patchwork Wed Sep 22 12:51:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12510449 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5F222C43217 for ; Wed, 22 Sep 2021 12:41:46 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 4813761156 for ; Wed, 22 Sep 2021 12:41:46 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236049AbhIVMnO (ORCPT ); Wed, 22 Sep 2021 08:43:14 -0400 Received: from szxga08-in.huawei.com ([45.249.212.255]:16229 "EHLO szxga08-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235986AbhIVMnN (ORCPT ); Wed, 22 Sep 2021 08:43:13 -0400 Received: from dggemv704-chm.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4HDyZ76MvWz1DH7j; Wed, 22 Sep 2021 20:40:31 +0800 (CST) Received: from dggema762-chm.china.huawei.com (10.1.198.204) by dggemv704-chm.china.huawei.com (10.3.19.47) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.8; Wed, 22 Sep 2021 20:41:41 +0800 Received: from huawei.com (10.175.127.227) by dggema762-chm.china.huawei.com (10.1.198.204) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.8; Wed, 22 Sep 2021 20:41:41 +0800 From: Yu Kuai To: , CC: , , , , Subject: [PATCH 3/4] blk-throtl: introduce blk_throtl_cancel_bios() Date: Wed, 22 Sep 2021 20:51:14 +0800 Message-ID: <20210922125115.381752-4-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210922125115.381752-1-yukuai3@huawei.com> References: <20210922125115.381752-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggema762-chm.china.huawei.com (10.1.198.204) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org This function is used to cancel all throttled bios when queue is dying. Noted this modification is mainly from revertion of commit b77412372b68 ("blk-throttle: remove blk_throtl_drain"). Signed-off-by: Yu Kuai --- block/blk-throttle.c | 39 +++++++++++++++++++++++++++++++++++++++ block/blk.h | 2 ++ 2 files changed, 41 insertions(+) diff --git a/block/blk-throttle.c b/block/blk-throttle.c index 3892ead7a0b5..4227dcf2dd3b 100644 --- a/block/blk-throttle.c +++ b/block/blk-throttle.c @@ -2435,6 +2435,45 @@ static void tg_drain_bios(struct throtl_service_queue *parent_sq) } } +/** + * blk_throtl_cancel_bios - cancel throttled bios + * @q: request_queue to cancel throttled bios for + * + * This function is called when queue is dying, error all currently + * throttled bios on @q so that user threads that are waiting for the bios + * can exit. + */ +void blk_throtl_cancel_bios(struct request_queue *q) +{ + struct throtl_data *td = q->td; + struct blkcg_gq *blkg; + struct cgroup_subsys_state *pos_css; + struct bio *bio; + int rw; + + rcu_read_lock(); + + /* + * Drain each tg while doing post-order walk on the blkg tree, so + * that all bios are propagated to td->service_queue. It'd be + * better to walk service_queue tree directly but blkg walk is + * easier. + */ + blkg_for_each_descendant_post(blkg, pos_css, td->queue->root_blkg) + tg_drain_bios(&blkg_to_tg(blkg)->service_queue); + + /* finally, transfer bios from top-level tg's into the td */ + tg_drain_bios(&td->service_queue); + + rcu_read_unlock(); + + /* all bios now should be in td->service_queue, cancel them */ + for (rw = READ; rw <= WRITE; rw++) + while ((bio = throtl_pop_queued(&td->service_queue.queued[rw], + NULL))) + bio_io_error(bio); +} + int blk_throtl_init(struct request_queue *q) { struct throtl_data *td; diff --git a/block/blk.h b/block/blk.h index 7d2a0ba7ed21..c407581a3a19 100644 --- a/block/blk.h +++ b/block/blk.h @@ -287,12 +287,14 @@ int create_task_io_context(struct task_struct *task, gfp_t gfp_mask, int node); * Internal throttling interface */ #ifdef CONFIG_BLK_DEV_THROTTLING +extern void blk_throtl_cancel_bios(struct request_queue *q); extern int blk_throtl_init(struct request_queue *q); extern void blk_throtl_exit(struct request_queue *q); extern void blk_throtl_register_queue(struct request_queue *q); extern void blk_throtl_charge_bio_split(struct bio *bio); bool blk_throtl_bio(struct bio *bio); #else /* CONFIG_BLK_DEV_THROTTLING */ +static inline void blk_throtl_cancel_bios(struct request_queue *q) { } static inline int blk_throtl_init(struct request_queue *q) { return 0; } static inline void blk_throtl_exit(struct request_queue *q) { } static inline void blk_throtl_register_queue(struct request_queue *q) { } From patchwork Wed Sep 22 12:51:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Kuai X-Patchwork-Id: 12510451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0AEB9C4332F for ; Wed, 22 Sep 2021 12:41:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E81C461131 for ; Wed, 22 Sep 2021 12:41:47 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236051AbhIVMnQ (ORCPT ); Wed, 22 Sep 2021 08:43:16 -0400 Received: from szxga01-in.huawei.com ([45.249.212.187]:20000 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236030AbhIVMnO (ORCPT ); Wed, 22 Sep 2021 08:43:14 -0400 Received: from dggemv703-chm.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4HDyVf31fVzbm3t; Wed, 22 Sep 2021 20:37:30 +0800 (CST) Received: from dggema762-chm.china.huawei.com (10.1.198.204) by dggemv703-chm.china.huawei.com (10.3.19.46) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.1.2308.8; Wed, 22 Sep 2021 20:41:42 +0800 Received: from huawei.com (10.175.127.227) by dggema762-chm.china.huawei.com (10.1.198.204) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256_P256) id 15.1.2308.8; Wed, 22 Sep 2021 20:41:41 +0800 From: Yu Kuai To: , CC: , , , , Subject: [PATCH 4/4] block: cancel all throttled bios in blk_cleanup_queue() Date: Wed, 22 Sep 2021 20:51:15 +0800 Message-ID: <20210922125115.381752-5-yukuai3@huawei.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210922125115.381752-1-yukuai3@huawei.com> References: <20210922125115.381752-1-yukuai3@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.127.227] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To dggema762-chm.china.huawei.com (10.1.198.204) X-CFilter-Loop: Reflected Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org Throttled bios can't be issued after queue is dying in blk_cleanup_queue(), thus it's better to cancel them immediately rather than waiting for throttle is done. For example, if user thread is throttled with low bps while is issuing large io, and the device is deleted. The user thread will wait for a long time for io to return. Signed-off-by: Yu Kuai --- block/blk-core.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/block/blk-core.c b/block/blk-core.c index 5454db2fa263..356a56318068 100644 --- a/block/blk-core.c +++ b/block/blk-core.c @@ -389,6 +389,9 @@ void blk_cleanup_queue(struct request_queue *q) blk_queue_flag_set(QUEUE_FLAG_DEAD, q); + if (q->root_blkg) + blk_throtl_cancel_bios(q); + /* for synchronous bio-based driver finish in-flight integrity i/o */ blk_flush_integrity();