From patchwork Fri Mar 8 17:40:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10845223 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1F7731515 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 017042E516 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E5A242ED60; Fri, 8 Mar 2019 17:39:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 88E202E516 for ; Fri, 8 Mar 2019 17:39:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726617AbfCHRjf (ORCPT ); Fri, 8 Mar 2019 12:39:35 -0500 Received: from mga01.intel.com ([192.55.52.88]:33276 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726544AbfCHRjf (ORCPT ); Fri, 8 Mar 2019 12:39:35 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Mar 2019 09:39:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,456,1544515200"; d="scan'208";a="132430639" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by orsmga003.jf.intel.com with ESMTP; 08 Mar 2019 09:39:35 -0800 From: Keith Busch To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, Jens Axboe Cc: Christoph Hellwig , Sagi Grimberg , Keith Busch Subject: [PATCH 1/5] blk-mq: Export reading mq request state Date: Fri, 8 Mar 2019 10:40:02 -0700 Message-Id: <20190308174006.5032-1-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Drivers may need to know the state of their requets. Signed-off-by: Keith Busch --- block/blk-mq.h | 9 --------- include/linux/blkdev.h | 9 +++++++++ 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/block/blk-mq.h b/block/blk-mq.h index c11353a3749d..99ab7e472e62 100644 --- a/block/blk-mq.h +++ b/block/blk-mq.h @@ -128,15 +128,6 @@ extern void blk_mq_hctx_kobj_init(struct blk_mq_hw_ctx *hctx); void blk_mq_release(struct request_queue *q); -/** - * blk_mq_rq_state() - read the current MQ_RQ_* state of a request - * @rq: target request. - */ -static inline enum mq_rq_state blk_mq_rq_state(struct request *rq) -{ - return READ_ONCE(rq->state); -} - static inline struct blk_mq_ctx *__blk_mq_get_ctx(struct request_queue *q, unsigned int cpu) { diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h index faed9d9eb84c..db113aee48bb 100644 --- a/include/linux/blkdev.h +++ b/include/linux/blkdev.h @@ -241,6 +241,15 @@ struct request { struct request *next_rq; }; +/** + * blk_mq_rq_state() - read the current MQ_RQ_* state of a request + * @rq: target request. + */ +static inline enum mq_rq_state blk_mq_rq_state(struct request *rq) +{ + return READ_ONCE(rq->state); +} + static inline bool blk_op_is_scsi(unsigned int op) { return op == REQ_OP_SCSI_IN || op == REQ_OP_SCSI_OUT; From patchwork Fri Mar 8 17:40:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10845225 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 343BB180E for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1CD702E571 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 10F282EE47; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id A985B2E571 for ; Fri, 8 Mar 2019 17:39:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726544AbfCHRjg (ORCPT ); Fri, 8 Mar 2019 12:39:36 -0500 Received: from mga01.intel.com ([192.55.52.88]:33276 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726581AbfCHRjf (ORCPT ); Fri, 8 Mar 2019 12:39:35 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Mar 2019 09:39:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,456,1544515200"; d="scan'208";a="132430643" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by orsmga003.jf.intel.com with ESMTP; 08 Mar 2019 09:39:35 -0800 From: Keith Busch To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, Jens Axboe Cc: Christoph Hellwig , Sagi Grimberg , Keith Busch Subject: [PATCH 2/5] blk-mq: Export iterating queue requests Date: Fri, 8 Mar 2019 10:40:03 -0700 Message-Id: <20190308174006.5032-2-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20190308174006.5032-1-keith.busch@intel.com> References: <20190308174006.5032-1-keith.busch@intel.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A driver may need to iterate a particular queue's tagged request rather than the whole tagset. Signed-off-by: Keith Busch --- block/blk-mq-tag.c | 1 + block/blk-mq-tag.h | 2 -- include/linux/blk-mq.h | 2 ++ 3 files changed, 3 insertions(+), 2 deletions(-) diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index a4931fc7be8a..a4ba91b332b0 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -407,6 +407,7 @@ void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, } blk_queue_exit(q); } +EXPORT_SYMBOL(blk_mq_queue_tag_busy_iter); static int bt_alloc(struct sbitmap_queue *bt, unsigned int depth, bool round_robin, int node) diff --git a/block/blk-mq-tag.h b/block/blk-mq-tag.h index 61deab0b5a5a..5af7ff94b400 100644 --- a/block/blk-mq-tag.h +++ b/block/blk-mq-tag.h @@ -33,8 +33,6 @@ extern int blk_mq_tag_update_depth(struct blk_mq_hw_ctx *hctx, struct blk_mq_tags **tags, unsigned int depth, bool can_grow); extern void blk_mq_tag_wakeup_all(struct blk_mq_tags *tags, bool); -void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, - void *priv); static inline struct sbq_wait_state *bt_wait_ptr(struct sbitmap_queue *bt, struct blk_mq_hw_ctx *hctx) diff --git a/include/linux/blk-mq.h b/include/linux/blk-mq.h index b0c814bcc7e3..a64b3fdce0b0 100644 --- a/include/linux/blk-mq.h +++ b/include/linux/blk-mq.h @@ -321,6 +321,8 @@ bool blk_mq_run_hw_queue(struct blk_mq_hw_ctx *hctx, bool async); void blk_mq_run_hw_queues(struct request_queue *q, bool async); void blk_mq_tagset_busy_iter(struct blk_mq_tag_set *tagset, busy_tag_iter_fn *fn, void *priv); +void blk_mq_queue_tag_busy_iter(struct request_queue *q, busy_iter_fn *fn, + void *priv); void blk_mq_freeze_queue(struct request_queue *q); void blk_mq_unfreeze_queue(struct request_queue *q); void blk_freeze_queue_start(struct request_queue *q); From patchwork Fri Mar 8 17:40:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10845229 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id F16711575 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D91002E571 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C9B022E516; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 32AF12E516 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726720AbfCHRjg (ORCPT ); Fri, 8 Mar 2019 12:39:36 -0500 Received: from mga01.intel.com ([192.55.52.88]:33276 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726581AbfCHRjg (ORCPT ); Fri, 8 Mar 2019 12:39:36 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Mar 2019 09:39:35 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,456,1544515200"; d="scan'208";a="132430646" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by orsmga003.jf.intel.com with ESMTP; 08 Mar 2019 09:39:35 -0800 From: Keith Busch To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, Jens Axboe Cc: Christoph Hellwig , Sagi Grimberg , Keith Busch Subject: [PATCH 3/5] blk-mq: Iterate tagset over all requests Date: Fri, 8 Mar 2019 10:40:04 -0700 Message-Id: <20190308174006.5032-3-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20190308174006.5032-1-keith.busch@intel.com> References: <20190308174006.5032-1-keith.busch@intel.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP Let the driver's iterator callback decide what request state it should handle. While there may be other uses, the intention here is to allow drivers to terminate requests that have entered a dying queue without needing to unquiesce that queue. It also turns out no current callback wanted to see COMPLETED requests that were being returned before, so this also fixes that for all existing callback handlers. Signed-off-by: Keith Busch --- block/blk-mq-tag.c | 12 ++++++------ drivers/block/mtip32xx/mtip32xx.c | 6 ++++++ drivers/block/nbd.c | 2 ++ drivers/block/skd_main.c | 4 ++++ drivers/nvme/host/core.c | 2 ++ drivers/nvme/host/fc.c | 2 ++ 6 files changed, 22 insertions(+), 6 deletions(-) diff --git a/block/blk-mq-tag.c b/block/blk-mq-tag.c index a4ba91b332b0..012f6d8e9631 100644 --- a/block/blk-mq-tag.c +++ b/block/blk-mq-tag.c @@ -288,7 +288,7 @@ static bool bt_tags_iter(struct sbitmap *bitmap, unsigned int bitnr, void *data) * test and set the bit before assining ->rqs[]. */ rq = tags->rqs[bitnr]; - if (rq && blk_mq_request_started(rq)) + if (rq) return iter_data->fn(rq, iter_data->data, reserved); return true; @@ -299,7 +299,7 @@ static bool bt_tags_iter(struct sbitmap *bitmap, unsigned int bitnr, void *data) * @tags: Tag map to iterate over. * @bt: sbitmap to examine. This is either the breserved_tags member * or the bitmap_tags member of struct blk_mq_tags. - * @fn: Pointer to the function that will be called for each started + * @fn: Pointer to the function that will be called for each * request. @fn will be called as follows: @fn(rq, @data, * @reserved) where rq is a pointer to a request. Return true * to continue iterating tags, false to stop. @@ -322,9 +322,9 @@ static void bt_tags_for_each(struct blk_mq_tags *tags, struct sbitmap_queue *bt, } /** - * blk_mq_all_tag_busy_iter - iterate over all started requests in a tag map + * blk_mq_all_tag_busy_iter - iterate over all requests in a tag map * @tags: Tag map to iterate over. - * @fn: Pointer to the function that will be called for each started + * @fn: Pointer to the function that will be called for each * request. @fn will be called as follows: @fn(rq, @priv, * reserved) where rq is a pointer to a request. 'reserved' * indicates whether or not @rq is a reserved request. Return @@ -340,9 +340,9 @@ static void blk_mq_all_tag_busy_iter(struct blk_mq_tags *tags, } /** - * blk_mq_tagset_busy_iter - iterate over all started requests in a tag set + * blk_mq_tagset_busy_iter - iterate over all requests in a tag set * @tagset: Tag set to iterate over. - * @fn: Pointer to the function that will be called for each started + * @fn: Pointer to the function that will be called for each * request. @fn will be called as follows: @fn(rq, @priv, * reserved) where rq is a pointer to a request. 'reserved' * indicates whether or not @rq is a reserved request. Return diff --git a/drivers/block/mtip32xx/mtip32xx.c b/drivers/block/mtip32xx/mtip32xx.c index 9a6f40cd8df6..88df90ee0a3c 100644 --- a/drivers/block/mtip32xx/mtip32xx.c +++ b/drivers/block/mtip32xx/mtip32xx.c @@ -2690,6 +2690,8 @@ static bool mtip_abort_cmd(struct request *req, void *data, bool reserved) struct mtip_cmd *cmd = blk_mq_rq_to_pdu(req); struct driver_data *dd = data; + if (blk_mq_rq_state(req) != MQ_RQ_IN_FLIGHT) + return true; dbg_printk(MTIP_DRV_NAME " Aborting request, tag = %d\n", req->tag); clear_bit(req->tag, dd->port->cmds_to_issue); @@ -2702,6 +2704,8 @@ static bool mtip_queue_cmd(struct request *req, void *data, bool reserved) { struct driver_data *dd = data; + if (blk_mq_rq_state(req) != MQ_RQ_IN_FLIGHT) + return true; set_bit(req->tag, dd->port->cmds_to_issue); blk_abort_request(req); return true; @@ -3852,6 +3856,8 @@ static bool mtip_no_dev_cleanup(struct request *rq, void *data, bool reserv) { struct mtip_cmd *cmd = blk_mq_rq_to_pdu(rq); + if (blk_mq_rq_state(rq) != MQ_RQ_IN_FLIGHT) + return true; cmd->status = BLK_STS_IOERR; blk_mq_complete_request(rq); return true; diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c index 90ba9f4c03f3..4811746f189b 100644 --- a/drivers/block/nbd.c +++ b/drivers/block/nbd.c @@ -739,6 +739,8 @@ static bool nbd_clear_req(struct request *req, void *data, bool reserved) { struct nbd_cmd *cmd = blk_mq_rq_to_pdu(req); + if (blk_mq_rq_state(req) != MQ_RQ_IN_FLIGHT) + return true; cmd->status = BLK_STS_IOERR; blk_mq_complete_request(req); return true; diff --git a/drivers/block/skd_main.c b/drivers/block/skd_main.c index 7d3ad6c22ee5..530d14e6b97b 100644 --- a/drivers/block/skd_main.c +++ b/drivers/block/skd_main.c @@ -387,6 +387,8 @@ static bool skd_inc_in_flight(struct request *rq, void *data, bool reserved) { int *count = data; + if (blk_mq_rq_state(rq) != MQ_RQ_IN_FLIGHT) + return true; count++; return true; } @@ -1899,6 +1901,8 @@ static bool skd_recover_request(struct request *req, void *data, bool reserved) struct skd_device *const skdev = data; struct skd_request_context *skreq = blk_mq_rq_to_pdu(req); + if (blk_mq_rq_state(req) != MQ_RQ_IN_FLIGHT) + return true; if (skreq->state != SKD_REQ_STATE_BUSY) return true; diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 1597efae6af8..cc5d9a83d5af 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -284,6 +284,8 @@ EXPORT_SYMBOL_GPL(nvme_complete_rq); bool nvme_cancel_request(struct request *req, void *data, bool reserved) { + if (blk_mq_rq_state(req) != MQ_RQ_IN_FLIGHT) + return true; dev_dbg_ratelimited(((struct nvme_ctrl *) data)->device, "Cancelling I/O %d", req->tag); diff --git a/drivers/nvme/host/fc.c b/drivers/nvme/host/fc.c index b29b12498a1a..0deba51575b0 100644 --- a/drivers/nvme/host/fc.c +++ b/drivers/nvme/host/fc.c @@ -2373,6 +2373,8 @@ nvme_fc_terminate_exchange(struct request *req, void *data, bool reserved) struct nvme_fc_ctrl *ctrl = to_fc_ctrl(nctrl); struct nvme_fc_fcp_op *op = blk_mq_rq_to_pdu(req); + if (blk_mq_rq_state(req) != MQ_RQ_IN_FLIGHT) + return true; __nvme_fc_abort_op(ctrl, op); return true; } From patchwork Fri Mar 8 17:40:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10845227 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D92AD1515 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF62A2E571 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B37BC2ED91; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 613542E571 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726721AbfCHRjg (ORCPT ); Fri, 8 Mar 2019 12:39:36 -0500 Received: from mga01.intel.com ([192.55.52.88]:33276 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726581AbfCHRjg (ORCPT ); Fri, 8 Mar 2019 12:39:36 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Mar 2019 09:39:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,456,1544515200"; d="scan'208";a="132430649" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by orsmga003.jf.intel.com with ESMTP; 08 Mar 2019 09:39:36 -0800 From: Keith Busch To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, Jens Axboe Cc: Christoph Hellwig , Sagi Grimberg , Keith Busch Subject: [PATCH 4/5] nvme: Fail dead namespace's entered requests Date: Fri, 8 Mar 2019 10:40:05 -0700 Message-Id: <20190308174006.5032-4-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20190308174006.5032-1-keith.busch@intel.com> References: <20190308174006.5032-1-keith.busch@intel.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP End the entered requests on a quieced queue directly rather than flush them through the low level driver's queue_rq(). Signed-off-by: Keith Busch --- drivers/nvme/host/core.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index cc5d9a83d5af..7095406bb293 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -94,6 +94,13 @@ static void nvme_put_subsystem(struct nvme_subsystem *subsys); static void nvme_remove_invalid_namespaces(struct nvme_ctrl *ctrl, unsigned nsid); +static bool nvme_fail_request(struct blk_mq_hw_ctx *hctx, struct request *req, + void *data, bool reserved) +{ + blk_mq_end_request(req, BLK_STS_IOERR); + return true; +} + static void nvme_set_queue_dying(struct nvme_ns *ns) { /* @@ -104,8 +111,7 @@ static void nvme_set_queue_dying(struct nvme_ns *ns) return; revalidate_disk(ns->disk); blk_set_queue_dying(ns->queue); - /* Forcibly unquiesce queues to avoid blocking dispatch */ - blk_mq_unquiesce_queue(ns->queue); + blk_mq_queue_tag_busy_iter(ns->queue, nvme_fail_request, NULL); } static void nvme_queue_scan(struct nvme_ctrl *ctrl) From patchwork Fri Mar 8 17:40:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10845231 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 7B7DD1869 for ; Fri, 8 Mar 2019 17:39:38 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 632632E516 for ; Fri, 8 Mar 2019 17:39:38 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 577962E571; Fri, 8 Mar 2019 17:39:38 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E0D5A2ED60 for ; Fri, 8 Mar 2019 17:39:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726759AbfCHRjh (ORCPT ); Fri, 8 Mar 2019 12:39:37 -0500 Received: from mga01.intel.com ([192.55.52.88]:33276 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726581AbfCHRjh (ORCPT ); Fri, 8 Mar 2019 12:39:37 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga003.jf.intel.com ([10.7.209.27]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 08 Mar 2019 09:39:36 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.58,456,1544515200"; d="scan'208";a="132430653" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by orsmga003.jf.intel.com with ESMTP; 08 Mar 2019 09:39:36 -0800 From: Keith Busch To: linux-nvme@lists.infradead.org, linux-block@vger.kernel.org, Jens Axboe Cc: Christoph Hellwig , Sagi Grimberg , Keith Busch Subject: [PATCH 5/5] nvme/pci: Remove queue IO flushing hack Date: Fri, 8 Mar 2019 10:40:06 -0700 Message-Id: <20190308174006.5032-5-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20190308174006.5032-1-keith.busch@intel.com> References: <20190308174006.5032-1-keith.busch@intel.com> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The nvme driver checked the queue state on every IO so this path could be used to flush out entered requests to a failed completion. The code even mentions in comments that we shouldn't have to do this, so let's not do it. Use the blk-mq's tag iterator to end all entered requests when the queue isn't going to be restarted so the IO path doesn't have to deal with these conditions. Signed-off-by: Keith Busch Signed-off-by: Keith Busch --- drivers/nvme/host/pci.c | 45 +++++++++++++++++++++++++++++---------------- 1 file changed, 29 insertions(+), 16 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index f54718b63637..398c6333cf77 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -918,13 +918,6 @@ static blk_status_t nvme_queue_rq(struct blk_mq_hw_ctx *hctx, struct nvme_command cmnd; blk_status_t ret; - /* - * We should not need to do this, but we're still using this to - * ensure we can drain requests on a dying queue. - */ - if (unlikely(!test_bit(NVMEQ_ENABLED, &nvmeq->flags))) - return BLK_STS_IOERR; - ret = nvme_setup_cmd(ns, req, &cmnd); if (ret) return ret; @@ -1403,10 +1396,6 @@ static int nvme_suspend_queue(struct nvme_queue *nvmeq) { if (!test_and_clear_bit(NVMEQ_ENABLED, &nvmeq->flags)) return 1; - - /* ensure that nvme_queue_rq() sees NVMEQ_ENABLED cleared */ - mb(); - nvmeq->dev->online_queues--; if (!nvmeq->qid && nvmeq->dev->ctrl.admin_q) blk_mq_quiesce_queue(nvmeq->dev->ctrl.admin_q); @@ -1616,15 +1605,29 @@ static const struct blk_mq_ops nvme_mq_ops = { .poll = nvme_poll, }; +static bool nvme_fail_queue_request(struct request *req, void *data, bool reserved) +{ + struct nvme_iod *iod = blk_mq_rq_to_pdu(req); + struct nvme_queue *nvmeq = iod->nvmeq; + + if (!test_bit(NVMEQ_ENABLED, &nvmeq->flags)) + blk_mq_end_request(req, BLK_STS_IOERR); + return true; +} + static void nvme_dev_remove_admin(struct nvme_dev *dev) { if (dev->ctrl.admin_q && !blk_queue_dying(dev->ctrl.admin_q)) { /* * If the controller was reset during removal, it's possible - * user requests may be waiting on a stopped queue. Start the - * queue to flush these to completion. + * user requests may be waiting on a stopped queue. End all + * entered requests after preventing new requests from + * entering. */ - blk_mq_unquiesce_queue(dev->ctrl.admin_q); + blk_set_queue_dying(dev->ctrl.admin_q); + blk_mq_tagset_busy_iter(&dev->admin_tagset, + nvme_fail_queue_request, + NULL); blk_cleanup_queue(dev->ctrl.admin_q); blk_mq_free_tag_set(&dev->admin_tagset); } @@ -2435,6 +2438,14 @@ static void nvme_pci_disable(struct nvme_dev *dev) } } +static void nvme_fail_requests(struct nvme_dev *dev) +{ + if (!dev->ctrl.tagset) + return; + blk_mq_tagset_busy_iter(dev->ctrl.tagset, nvme_fail_queue_request, + NULL); +} + static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown) { bool dead = true; @@ -2475,11 +2486,11 @@ static void nvme_dev_disable(struct nvme_dev *dev, bool shutdown) /* * The driver will not be starting up queues again if shutting down so - * must flush all entered requests to their failed completion to avoid + * must end all entered requests to their failed completion to avoid * deadlocking blk-mq hot-cpu notifier. */ if (shutdown) - nvme_start_queues(&dev->ctrl); + nvme_fail_requests(dev); mutex_unlock(&dev->shutdown_lock); } @@ -2624,6 +2635,8 @@ static void nvme_reset_work(struct work_struct *work) nvme_remove_namespaces(&dev->ctrl); new_state = NVME_CTRL_ADMIN_ONLY; } else { + /* Fail requests that entered an hctx that no longer exists */ + nvme_fail_requests(dev); nvme_start_queues(&dev->ctrl); nvme_wait_freeze(&dev->ctrl); /* hit this only when allocate tagset fails */