From patchwork Thu Jan 3 22:50:31 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Keith Busch X-Patchwork-Id: 10747833 X-Patchwork-Delegate: bhelgaas@google.com Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0E56613B5 for ; Thu, 3 Jan 2019 22:52:53 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id F29392237D for ; Thu, 3 Jan 2019 22:52:52 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E6C0324BFE; Thu, 3 Jan 2019 22:52:52 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-7.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 810A72237D for ; Thu, 3 Jan 2019 22:52:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728584AbfACWwv (ORCPT ); Thu, 3 Jan 2019 17:52:51 -0500 Received: from mga18.intel.com ([134.134.136.126]:45205 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726529AbfACWwu (ORCPT ); Thu, 3 Jan 2019 17:52:50 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 03 Jan 2019 14:52:50 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.56,436,1539673200"; d="scan'208";a="307303389" Received: from unknown (HELO localhost.lm.intel.com) ([10.232.112.69]) by fmsmga006.fm.intel.com with ESMTP; 03 Jan 2019 14:52:49 -0800 From: Keith Busch To: Jens Axboe , Christoph Hellwig , Sagi Grimberg , Ming Lei , linux-nvme@lists.infradead.org, Bjorn Helgaas , linux-pci@vger.kernel.org Cc: Keith Busch Subject: [PATCHv2 2/4] nvme-pci: Distribute io queue types after creation Date: Thu, 3 Jan 2019 15:50:31 -0700 Message-Id: <20190103225033.11249-3-keith.busch@intel.com> X-Mailer: git-send-email 2.13.6 In-Reply-To: <20190103225033.11249-1-keith.busch@intel.com> References: <20190103225033.11249-1-keith.busch@intel.com> Sender: linux-pci-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-pci@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The dev->io_queues types were set based on the results of the nvme set feature "number of queues" and the IRQ allocation. This result does not mean we're going to successfully allocate and create those IO queues, though. A failure there will cause blk-mq to have NULL hctx's because the map's nr_hw_queues accounts for more queues than were actually created. Adjust the io_queue types after we've created them when we have less than originally desired. Fixes: 3b6592f70ad7b ("nvme: utilize two queue maps, one for reads and one for writes") Signed-off-by: Keith Busch Reviewed-by: Ming Lei --- drivers/nvme/host/pci.c | 46 ++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 40 insertions(+), 6 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 98332d0a80f0..1481bb6d9c42 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -1733,6 +1733,30 @@ static int nvme_pci_configure_admin_queue(struct nvme_dev *dev) return result; } +static void nvme_distribute_queues(struct nvme_dev *dev, unsigned int io_queues) +{ + unsigned int irq_queues, this_p_queues = dev->io_queues[HCTX_TYPE_POLL], + this_w_queues = dev->io_queues[HCTX_TYPE_DEFAULT]; + + if (!io_queues) { + dev->io_queues[HCTX_TYPE_POLL] = 0; + dev->io_queues[HCTX_TYPE_DEFAULT] = 0; + dev->io_queues[HCTX_TYPE_READ] = 0; + return; + } + + if (this_p_queues >= io_queues) + this_p_queues = io_queues - 1; + irq_queues = io_queues - this_p_queues; + + if (this_w_queues > irq_queues) + this_w_queues = irq_queues; + + dev->io_queues[HCTX_TYPE_POLL] = this_p_queues; + dev->io_queues[HCTX_TYPE_DEFAULT] = this_w_queues; + dev->io_queues[HCTX_TYPE_READ] = irq_queues - this_w_queues; +} + static int nvme_create_io_queues(struct nvme_dev *dev) { unsigned i, max, rw_queues; @@ -1761,6 +1785,13 @@ static int nvme_create_io_queues(struct nvme_dev *dev) break; } + /* + * If we've created less than expected io queues, redistribute the + * dev->io_queues[] types accordingly. + */ + if (dev->online_queues - 1 != dev->max_qid) + nvme_distribute_queues(dev, dev->online_queues - 1); + /* * Ignore failing Create SQ/CQ commands, we can continue with less * than the desired amount of queues, and even a controller without @@ -2185,11 +2216,6 @@ static int nvme_setup_io_queues(struct nvme_dev *dev) result = max(result - 1, 1); dev->max_qid = result + dev->io_queues[HCTX_TYPE_POLL]; - dev_info(dev->ctrl.device, "%d/%d/%d default/read/poll queues\n", - dev->io_queues[HCTX_TYPE_DEFAULT], - dev->io_queues[HCTX_TYPE_READ], - dev->io_queues[HCTX_TYPE_POLL]); - /* * Should investigate if there's a performance win from allocating * more queues than interrupt vectors; it might allow the submission @@ -2203,7 +2229,15 @@ static int nvme_setup_io_queues(struct nvme_dev *dev) return result; } set_bit(NVMEQ_ENABLED, &adminq->flags); - return nvme_create_io_queues(dev); + result = nvme_create_io_queues(dev); + + if (!result) + dev_info(dev->ctrl.device, "%d/%d/%d default/read/poll queues\n", + dev->io_queues[HCTX_TYPE_DEFAULT], + dev->io_queues[HCTX_TYPE_READ], + dev->io_queues[HCTX_TYPE_POLL]); + return result; + } static void nvme_del_queue_end(struct request *req, blk_status_t error)