From patchwork Wed May 3 17:40:37 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9710229 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 6C66C60351 for ; Wed, 3 May 2017 17:40:51 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 616D128630 for ; Wed, 3 May 2017 17:40:51 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5602428679; Wed, 3 May 2017 17:40:51 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.4 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, RCVD_IN_DNSWL_HI, RCVD_IN_SORBS_SPAM autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9F5CE2866B for ; Wed, 3 May 2017 17:40:50 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753471AbdECRkr (ORCPT ); Wed, 3 May 2017 13:40:47 -0400 Received: from mail-io0-f170.google.com ([209.85.223.170]:33510 "EHLO mail-io0-f170.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752950AbdECRkk (ORCPT ); Wed, 3 May 2017 13:40:40 -0400 Received: by mail-io0-f170.google.com with SMTP id p24so13216606ioi.0 for ; Wed, 03 May 2017 10:40:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=subject:to:references:cc:from:message-id:date:user-agent :mime-version:in-reply-to:content-transfer-encoding; bh=SLKKjp1XH3+ZSdQO7Wvni9GsAju/aM7GZn5qYhlJUJw=; b=gbhqYoSxmIZ1vS+YtyyOXEs65hFvfcWwtgo02Gi7bpzaWFbgBAeMtvdoYIyFn+pynY l7Jj9dEifLRyQmQYOs7HiMh68gFEokvoYqBf/tUlbBfggWNfxy7BNwbyJKkk9euBMoWK v0g/ganyv/RhNGAYH/6jWQ0+gmV4cLr4cH04rSp/6pxnDEZTOg/Y1Sn4gioW6qankpa+ mbKlw31OVzKLl+29DjcqwzHes8dClE29H2Xh6Y8HnpPmsc16ZvUeRvfcKgxPrqydHABh x7Cy0h8UWGYMxUmfp0gEiBZAB/8enIYRjNRo3Xkzy/zFhnylMctCTcg9uLJZrWIe6kw4 aK5w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:cc:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=SLKKjp1XH3+ZSdQO7Wvni9GsAju/aM7GZn5qYhlJUJw=; b=rshhoeMNShB+8Sa2V/9a17CZUDrCfwynH9bdoJueYs+WcQI3bFiAxPT4F+ockGOgFk j60nEgyyLP2Lx1xysOg0OTHTVCCgAPFSNWcC9sgiSAyUpbAWYmhz1APusFs6QXKvX2kT 73i04ZRJ74rbq+VmKJcLvOlxRJ1qYr4XTzXPMoTjfOIrMf2T1NSdnW59iRmP+PoHWZpj DQUfH6GUdGwF8eLsqRpCM7v6t0egigahQNnn+mtc6BxVmt5aSb0KW9yKJqwXRjJD/Ll0 vB7aHxtIQDr6t6SUSrSuT7S9JnFdKN5ZY9Jlq8mnvc3nspomi35utJCF9eYYrZFi9pLw PdXQ== X-Gm-Message-State: AN3rC/7vKSZoo4bmnrnMrVZV87njFfNI0I56wKTHvAzSWbUgIkaHXQWY Yi0GrBlMYBl7aMLAdtE= X-Received: by 10.107.59.17 with SMTP id i17mr4683793ioa.96.1493833239242; Wed, 03 May 2017 10:40:39 -0700 (PDT) Received: from [192.168.1.154] ([216.160.245.98]) by smtp.gmail.com with ESMTPSA id 139sm9620501ioz.2.2017.05.03.10.40.38 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 May 2017 10:40:38 -0700 (PDT) Subject: Re: [PATCH 0/4] blk-mq: support to use hw tag for scheduling To: Bart Van Assche , "ming.lei@redhat.com" References: <20170428151539.25514-1-ming.lei@redhat.com> <839682da-f375-8eab-d6f5-fcf1457150f1@fb.com> <20170503040303.GA20187@ming.t460p> <370fbeb6-d832-968a-2759-47f16b866551@kernel.dk> <20170503150351.GA7927@ming.t460p> <31bb973e-d9cf-9454-58fd-4893701088c5@kernel.dk> <20170503153808.GB7927@ming.t460p> <20170503165201.GB9706@ming.t460p> <20170503170315.GD9706@ming.t460p> <24ff7ca6-73d6-f8a5-b7d9-e92d0bfdb4b0@kernel.dk> <1493831722.3901.19.camel@sandisk.com> <1493832921.3901.22.camel@sandisk.com> Cc: "hch@infradead.org" , "linux-block@vger.kernel.org" , "osandov@fb.com" From: Jens Axboe Message-ID: <63d4a989-25ac-5ab7-ec5d-e92ebb7976d5@kernel.dk> Date: Wed, 3 May 2017 11:40:37 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: <1493832921.3901.22.camel@sandisk.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On 05/03/2017 11:35 AM, Bart Van Assche wrote: > On Wed, 2017-05-03 at 11:24 -0600, Jens Axboe wrote: >> diff --git a/drivers/block/mtip32xx/mtip32xx.c b/drivers/block/mtip32xx/mtip32xx.c >> index 3a779a4f5653..33b5d1382c45 100644 >> --- a/drivers/block/mtip32xx/mtip32xx.c >> +++ b/drivers/block/mtip32xx/mtip32xx.c >> [ ... ] >> @@ -4009,7 +4009,7 @@ static int mtip_block_remove(struct driver_data *dd) >> dd->disk->disk_name); >> >> blk_freeze_queue_start(dd->queue); >> - blk_mq_stop_hw_queues(dd->queue); >> + blk_mq_stop_hw_queues(dd->queue, true); >> blk_mq_tagset_busy_iter(&dd->tags, mtip_no_dev_cleanup, dd); >> >> /* > > Hello Jens, > > How about replacing the blk_freeze_queue_start() and blk_mq_stop_hw_queues() > calls by a single call to blk_set_queue_dying()? I think that would be more > appropriate in the context of mtip_block_remove(). Looking at all of the drivers, it's somewhat of a mess and a lot of it looks like it's not even needed. I'm going to propose that we do the below first, since it fixes the immediate regression and makes us no worse than 4.11 already was. Then we can do a round two of improving the situation, but I'd prefer not to turn the immediate fix into a huge round of fixes and driver touching. From 66e22c6402af135cdc9913d2e298107629b02cdb Mon Sep 17 00:00:00 2001 From: Jens Axboe Date: Wed, 3 May 2017 11:08:14 -0600 Subject: [PATCH] blk-mq: don't use sync workqueue flushing from drivers A previous commit introduced the sync flush, which we need from internal callers like blk_mq_quiesce_queue(). However, we also call the stop helpers from drivers, particularly from ->queue_rq() when we have to stop processing for a bit. We can't block from those locations, and we don't have to guarantee that we're fully flushed. Fixes: 9f993737906b ("blk-mq: unify hctx delayed_run_work and run_work") Signed-off-by: Jens Axboe Reviewed-by: Bart Van Assche --- block/blk-mq.c | 25 ++++++++++++++++++++----- 1 file changed, 20 insertions(+), 5 deletions(-) diff --git a/block/blk-mq.c b/block/blk-mq.c index e339247a2570..dec70ca0aafd 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -41,6 +41,7 @@ static LIST_HEAD(all_q_list); static void blk_mq_poll_stats_start(struct request_queue *q); static void blk_mq_poll_stats_fn(struct blk_stat_callback *cb); +static void __blk_mq_stop_hw_queues(struct request_queue *q, bool sync); static int blk_mq_poll_stats_bkt(const struct request *rq) { @@ -166,7 +167,7 @@ void blk_mq_quiesce_queue(struct request_queue *q) unsigned int i; bool rcu = false; - blk_mq_stop_hw_queues(q); + __blk_mq_stop_hw_queues(q, true); queue_for_each_hw_ctx(q, hctx, i) { if (hctx->flags & BLK_MQ_F_BLOCKING) @@ -1218,20 +1219,34 @@ bool blk_mq_queue_stopped(struct request_queue *q) } EXPORT_SYMBOL(blk_mq_queue_stopped); -void blk_mq_stop_hw_queue(struct blk_mq_hw_ctx *hctx) +static void __blk_mq_stop_hw_queue(struct blk_mq_hw_ctx *hctx, bool sync) { - cancel_delayed_work_sync(&hctx->run_work); + if (sync) + cancel_delayed_work_sync(&hctx->run_work); + else + cancel_delayed_work(&hctx->run_work); + set_bit(BLK_MQ_S_STOPPED, &hctx->state); } + +void blk_mq_stop_hw_queue(struct blk_mq_hw_ctx *hctx) +{ + __blk_mq_stop_hw_queue(hctx, false); +} EXPORT_SYMBOL(blk_mq_stop_hw_queue); -void blk_mq_stop_hw_queues(struct request_queue *q) +void __blk_mq_stop_hw_queues(struct request_queue *q, bool sync) { struct blk_mq_hw_ctx *hctx; int i; queue_for_each_hw_ctx(q, hctx, i) - blk_mq_stop_hw_queue(hctx); + __blk_mq_stop_hw_queue(hctx, sync); +} + +void blk_mq_stop_hw_queues(struct request_queue *q) +{ + __blk_mq_stop_hw_queues(q, false); } EXPORT_SYMBOL(blk_mq_stop_hw_queues);