From patchwork Sat May 20 05:34:15 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ming Lei X-Patchwork-Id: 9738461 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 658386034C for ; Sat, 20 May 2017 05:34:37 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 44933285A0 for ; Sat, 20 May 2017 05:34:37 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 381DE285A2; Sat, 20 May 2017 05:34:37 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B331B285A0 for ; Sat, 20 May 2017 05:34:36 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756140AbdETFef (ORCPT ); Sat, 20 May 2017 01:34:35 -0400 Received: from mx1.redhat.com ([209.132.183.28]:60276 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753797AbdETFef (ORCPT ); Sat, 20 May 2017 01:34:35 -0400 Received: from smtp.corp.redhat.com (int-mx02.intmail.prod.int.phx2.redhat.com [10.5.11.12]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 4920980F7D; Sat, 20 May 2017 05:34:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mx1.redhat.com 4920980F7D Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; dmarc=none (p=none dis=none) header.from=redhat.com Authentication-Results: ext-mx03.extmail.prod.ext.phx2.redhat.com; spf=pass smtp.mailfrom=ming.lei@redhat.com DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.redhat.com 4920980F7D Received: from ming.t460p (ovpn-12-23.pek2.redhat.com [10.72.12.23]) by smtp.corp.redhat.com (Postfix) with ESMTPS id CBD4F17A76; Sat, 20 May 2017 05:34:19 +0000 (UTC) Date: Sat, 20 May 2017 13:34:15 +0800 From: Ming Lei To: Jens Axboe , Keith Busch , Christoph Hellwig , Sagi Grimberg Cc: Zhang Yi , stable@vger.kernel.org, linux-block@vger.kernel.org, linux-nvme@lists.infradead.org, Johannes Thumshirn Subject: Re: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list() Message-ID: <20170520053409.GA22317@ming.t460p> References: <20170520035605.21785-1-ming.lei@redhat.com> <20170520035605.21785-3-ming.lei@redhat.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20170520035605.21785-3-ming.lei@redhat.com> User-Agent: Mutt/1.8.0 (2017-02-23) X-Scanned-By: MIMEDefang 2.79 on 10.5.11.12 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.27]); Sat, 20 May 2017 05:34:34 +0000 (UTC) Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP On Sat, May 20, 2017 at 11:56:04AM +0800, Ming Lei wrote: > NVMe may add request into requeue list simply and not kick off the > requeue if hw queues are stopped. Then blk_mq_abort_requeue_list() > is called in both nvme_kill_queues() and nvme_ns_remove() for > dealing with this issue. > > Unfortunately blk_mq_abort_requeue_list() is absolutely a > race maker, for example, one request may be requeued during > the aborting. So this patch just calls blk_mq_kick_requeue_list() in > nvme_kill_queues() to handle this issue like what nvme_start_queues() > does. Now all requests in requeue list when queues are stopped will be > handled by blk_mq_kick_requeue_list() when queues are restarted, either > in nvme_start_queues() or in nvme_kill_queues(). > > Cc: stable@vger.kernel.org > Reported-by: Zhang Yi > Signed-off-by: Ming Lei > --- > drivers/nvme/host/core.c | 5 +++-- > 1 file changed, 3 insertions(+), 2 deletions(-) > > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c > index ce0d96913ee6..9ce0a0a16984 100644 > --- a/drivers/nvme/host/core.c > +++ b/drivers/nvme/host/core.c > @@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns) > if (ns->ndev) > nvme_nvm_unregister_sysfs(ns); > del_gendisk(ns->disk); > - blk_mq_abort_requeue_list(ns->queue); > blk_cleanup_queue(ns->queue); > } > > @@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl) > continue; > revalidate_disk(ns->disk); > blk_set_queue_dying(ns->queue); > - blk_mq_abort_requeue_list(ns->queue); > > /* > * We have to force to start queues for avoiding hang > @@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl) > * be stopped forever from now on. > */ > blk_mq_start_hw_queues(ns->queue); > + > + /* draining requests in requeue list */ > + blk_mq_kick_requeue_list(q); oops, the above line causes build failure, please take the following one. Reviewed-by: Christoph Hellwig Reviewed-by: Johannes Thumshirn --- From e70cfe2edacf278e3e6605f8c08e01ebf65bff01 Mon Sep 17 00:00:00 2001 From: Ming Lei Date: Wed, 17 May 2017 09:02:07 +0800 Subject: [PATCH v2 2/3] nvme: avoid to use blk_mq_abort_requeue_list() NVMe may add request into requeue list simply and not kick off the requeue if hw queues are stopped. Then blk_mq_abort_requeue_list() is called in both nvme_kill_queues() and nvme_ns_remove() for dealing with this issue. Unfortunately blk_mq_abort_requeue_list() is absolutely a race maker, for example, one request may be requeued during the aborting. So this patch just calls blk_mq_kick_requeue_list() in nvme_kill_queues() to handle this issue like what nvme_start_queues() does. Now all requests in requeue list when queues are stopped will be handled by blk_mq_kick_requeue_list() when queues are restarted, either in nvme_start_queues() or in nvme_kill_queues(). Cc: stable@vger.kernel.org Reported-by: Zhang Yi Signed-off-by: Ming Lei --- drivers/nvme/host/core.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index ce0d96913ee6..e344f5d7c1bc 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -2098,7 +2098,6 @@ static void nvme_ns_remove(struct nvme_ns *ns) if (ns->ndev) nvme_nvm_unregister_sysfs(ns); del_gendisk(ns->disk); - blk_mq_abort_requeue_list(ns->queue); blk_cleanup_queue(ns->queue); } @@ -2436,7 +2435,6 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl) continue; revalidate_disk(ns->disk); blk_set_queue_dying(ns->queue); - blk_mq_abort_requeue_list(ns->queue); /* * We have to force to start queues for avoiding hang @@ -2444,6 +2442,9 @@ void nvme_kill_queues(struct nvme_ctrl *ctrl) * be stopped forever from now on. */ blk_mq_start_hw_queues(ns->queue); + + /* draining requests in requeue list */ + blk_mq_kick_requeue_list(ns->queue); } mutex_unlock(&ctrl->namespaces_mutex); }