From patchwork Tue Jun 13 17:15:23 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9784419 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 085606038E for ; Tue, 13 Jun 2017 17:15:55 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E251E2857E for ; Tue, 13 Jun 2017 17:15:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D749F28676; Tue, 13 Jun 2017 17:15:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EE8062857E for ; Tue, 13 Jun 2017 17:15:52 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753642AbdFMRPo (ORCPT ); Tue, 13 Jun 2017 13:15:44 -0400 Received: from mail-io0-f174.google.com ([209.85.223.174]:33441 "EHLO mail-io0-f174.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753604AbdFMRPl (ORCPT ); Tue, 13 Jun 2017 13:15:41 -0400 Received: by mail-io0-f174.google.com with SMTP id t87so57865673ioe.0 for ; Tue, 13 Jun 2017 10:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=cwZOn+gU74LcvSCk2jZmd5l6SdczEcPwUcN26FqomqU=; b=yja2GtPReUSDKb2Kg+C8z28pi1mNUEBvHFwTADTJi6zFf83PaxVKSZ1BSOi/Bvt5DE m+aCxssGJwzQfyyCZaUCnPi8WHCJe48IeINNBlAKkc4kklDHY9RN1h/t6+9w49fbZ8zd qwRBigGxs7+NnoU8GcZHE8e9TxIrQrYXpRSorBMbHpZOOOKjwZe+aFkOTV7OHbvmncVA 99XQPiikabEpej/RsjyJTaNIRoF8ap9E1B815h2EIkX/30IrcxsjtpPlVXgUFTVJluby Fu8M6hpduyjFa6gGl8iBLcCFtmL8ZZp+Bb7suKXKK4K62d0UsPbrSPH4r5Rx7qxCswwV n7og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=cwZOn+gU74LcvSCk2jZmd5l6SdczEcPwUcN26FqomqU=; b=cn5U29pktiwOR2ILzYtfqU4QC2LRUiGtjfLQTKnziH06sJ3ynqQu1mcZB5KiTbCtso o8fDvusxjStl94tnKrs4Lr4TVpOvxjjD7PYRXtHdPOf4uVhqmgDGyAaBlz/hvO8SV4qd dxvxouUY+K9Dt5H9kzah7Z3j9wr4a9ZfELKfzzq19ZxiFSvH0pqQgTxnqxg1t2ntZoV0 JF3n/Z/4MD7VvkBtsKmIY/UFjnNGCFSQUmjqlyqglrkiS1MIqLhYbnR9MBSTAq+20j5q m6A4cJbxFZAKPieMN54tk3W4qdo1+FQ0ivmpdNGXKT6j1Odx5WH6h9xq8gCXtfogs+Nt BGIw== X-Gm-Message-State: AKS2vOwsuz7FfTpDaJSdjd9ren949Ylior4AmcvMVfoiO84ImAwgJRkd y880M2GQDv1MTOqPtAQutw== X-Received: by 10.107.3.106 with SMTP id 103mr1365407iod.82.1497374140461; Tue, 13 Jun 2017 10:15:40 -0700 (PDT) Received: from localhost.localdomain ([216.160.245.98]) by smtp.gmail.com with ESMTPSA id y188sm6217526ity.9.2017.06.13.10.15.39 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Jun 2017 10:15:40 -0700 (PDT) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Cc: Jens Axboe Subject: [PATCH 11/11] nvme: add support for streams and directives Date: Tue, 13 Jun 2017 11:15:23 -0600 Message-Id: <1497374123-15286-12-git-send-email-axboe@kernel.dk> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1497374123-15286-1-git-send-email-axboe@kernel.dk> References: <1497374123-15286-1-git-send-email-axboe@kernel.dk> Sender: linux-block-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-block@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This adds support for Directives in NVMe, particular for the Streams directive. We default to allocating 4 streams per name space, but it is configurable with the 'streams_per_ns' module option. If a write stream is set in a write, flag is as such before sending it to the device. Some debug stuff in this patch, dumping streams ID params when we load nvme. Signed-off-by: Jens Axboe --- drivers/nvme/host/core.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/nvme/host/nvme.h | 1 + include/linux/nvme.h | 48 ++++++++++++++++++ 3 files changed, 173 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 903d5813023a..81225e7d4176 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -65,6 +65,10 @@ static bool force_apst; module_param(force_apst, bool, 0644); MODULE_PARM_DESC(force_apst, "allow APST for newly enumerated devices even if quirked off"); +static char streams_per_ns = 4; +module_param(streams_per_ns, byte, 0644); +MODULE_PARM_DESC(streams_per_ns, "if available, allocate this many streams per NS"); + static LIST_HEAD(nvme_ctrl_list); static DEFINE_SPINLOCK(dev_list_lock); @@ -351,6 +355,15 @@ static inline void nvme_setup_rw(struct nvme_ns *ns, struct request *req, cmnd->rw.slba = cpu_to_le64(nvme_block_nr(ns, blk_rq_pos(req))); cmnd->rw.length = cpu_to_le16((blk_rq_bytes(req) >> ns->lba_shift) - 1); + if (req_op(req) == REQ_OP_WRITE) { + if (bio_stream_valid(req->bio) && ns->streams) { + unsigned stream = bio_stream(req->bio) & 0xffff; + + control |= NVME_RW_DTYPE_STREAMS; + dsmgmt |= (stream << 16); + } + } + if (ns->ms) { switch (ns->pi_type) { case NVME_NS_DPS_PI_TYPE3: @@ -1073,6 +1086,109 @@ static int nvme_revalidate_disk(struct gendisk *disk) return 0; } +static int nvme_enable_streams(struct nvme_ns *ns) +{ + struct nvme_command c; + + memset(&c, 0, sizeof(c)); + + c.directive.opcode = nvme_admin_directive_send; + c.directive.nsid = cpu_to_le32(ns->ns_id); + c.directive.doper = NVME_DIR_SND_ID_OP_ENABLE; + c.directive.dtype = NVME_DIR_IDENTIFY; + c.directive.tdtype = NVME_DIR_STREAMS; + c.directive.endir = NVME_DIR_ENDIR; + + return nvme_submit_sync_cmd(ns->ctrl->admin_q, &c, NULL, 0); +} + +static int nvme_streams_params(struct nvme_ns *ns) +{ + struct nvme_ctrl *ctrl = ns->ctrl; + struct streams_directive_params s; + struct nvme_command c; + int ret; + + memset(&c, 0, sizeof(c)); + memset(&s, 0, sizeof(s)); + + c.directive.opcode = nvme_admin_directive_recv; + c.directive.nsid = cpu_to_le32(ns->ns_id); + c.directive.numd = sizeof(s); + c.directive.doper = NVME_DIR_RCV_ST_OP_PARAM; + c.directive.dtype = NVME_DIR_STREAMS; + + ret = nvme_submit_sync_cmd(ctrl->admin_q, &c, &s, sizeof(s)); + if (ret) + return ret; + + s.msl = le16_to_cpu(s.msl); + s.nssa = le16_to_cpu(s.nssa); + s.nsso = le16_to_cpu(s.nsso); + s.sws = le32_to_cpu(s.sws); + s.sgs = le16_to_cpu(s.sgs); + s.nsa = le16_to_cpu(s.nsa); + s.nso = le16_to_cpu(s.nso); + + dev_info(ctrl->device, "streams: msl=%u, nssa=%u, nsso=%u, sws=%u " + "sgs=%u, nsa=%u, nso=%u\n", s.msl, s.nssa, + s.nsso, s.sws, s.sgs, s.nsa, s.nso); + return 0; +} + +static int nvme_streams_allocate(struct nvme_ns *ns, unsigned int streams) +{ + struct nvme_command c; + + memset(&c, 0, sizeof(c)); + + c.directive.opcode = nvme_admin_directive_recv; + c.directive.nsid = cpu_to_le32(ns->ns_id); + c.directive.doper = NVME_DIR_RCV_ST_OP_RESOURCE; + c.directive.dtype = NVME_DIR_STREAMS; + c.directive.endir = streams; + + return nvme_submit_sync_cmd(ns->ctrl->admin_q, &c, NULL, 0); +} + +static int nvme_streams_deallocate(struct nvme_ns *ns) +{ + struct nvme_command c; + + memset(&c, 0, sizeof(c)); + + c.directive.opcode = nvme_admin_directive_send; + c.directive.nsid = cpu_to_le32(ns->ns_id); + c.directive.doper = NVME_DIR_SND_ST_OP_REL_RSC; + c.directive.dtype = NVME_DIR_STREAMS; + + return nvme_submit_sync_cmd(ns->ctrl->admin_q, &c, NULL, 0); +} + +static void nvme_config_streams(struct nvme_ns *ns) +{ + int ret; + + ret = nvme_enable_streams(ns); + if (ret) + return; + + ret = nvme_streams_params(ns); + if (ret) + return; + + ret = nvme_streams_allocate(ns, streams_per_ns); + if (ret) + return; + + ret = nvme_streams_params(ns); + if (ret) + return; + + ns->streams = true; + dev_info(ns->ctrl->device, "successfully enabled streams\n"); +} + static char nvme_pr_type(enum pr_type type) { switch (type) { @@ -1606,6 +1722,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl) ctrl->sgls = le32_to_cpu(id->sgls); ctrl->kas = le16_to_cpu(id->kas); + if (ctrl->oacs & NVME_CTRL_OACS_DIRECTIVES) + dev_info(ctrl->dev, "supports directives\n"); + ctrl->npss = id->npss; prev_apsta = ctrl->apsta; if (ctrl->quirks & NVME_QUIRK_NO_APST) { @@ -2060,6 +2179,9 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid) goto out_free_id; } + if (ctrl->oacs & NVME_CTRL_OACS_DIRECTIVES) + nvme_config_streams(ns); + disk = alloc_disk_node(0, node); if (!disk) goto out_free_id; @@ -2112,6 +2234,8 @@ static void nvme_ns_remove(struct nvme_ns *ns) &nvme_ns_attr_group); if (ns->ndev) nvme_nvm_unregister_sysfs(ns); + if (ns->streams) + nvme_streams_deallocate(ns); del_gendisk(ns->disk); blk_cleanup_queue(ns->queue); } diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 9d6a070d4391..c2d8d23c90de 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -195,6 +195,7 @@ struct nvme_ns { int lba_shift; u16 ms; bool ext; + bool streams; u8 pi_type; unsigned long flags; diff --git a/include/linux/nvme.h b/include/linux/nvme.h index b625bacf37ef..8b2f5b140134 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -245,6 +245,7 @@ enum { NVME_CTRL_ONCS_WRITE_ZEROES = 1 << 3, NVME_CTRL_VWC_PRESENT = 1 << 0, NVME_CTRL_OACS_SEC_SUPP = 1 << 0, + NVME_CTRL_OACS_DIRECTIVES = 1 << 5, NVME_CTRL_OACS_DBBUF_SUPP = 1 << 7, }; @@ -295,6 +296,19 @@ enum { }; enum { + NVME_DIR_IDENTIFY = 0x00, + NVME_DIR_STREAMS = 0x01, + NVME_DIR_SND_ID_OP_ENABLE = 0x01, + NVME_DIR_SND_ST_OP_REL_ID = 0x01, + NVME_DIR_SND_ST_OP_REL_RSC = 0x02, + NVME_DIR_RCV_ID_OP_PARAM = 0x01, + NVME_DIR_RCV_ST_OP_PARAM = 0x01, + NVME_DIR_RCV_ST_OP_STATUS = 0x02, + NVME_DIR_RCV_ST_OP_RESOURCE = 0x03, + NVME_DIR_ENDIR = 0x01, +}; + +enum { NVME_NS_FEAT_THIN = 1 << 0, NVME_NS_FLBAS_LBA_MASK = 0xf, NVME_NS_FLBAS_META_EXT = 0x10, @@ -535,6 +549,7 @@ enum { NVME_RW_PRINFO_PRCHK_APP = 1 << 11, NVME_RW_PRINFO_PRCHK_GUARD = 1 << 12, NVME_RW_PRINFO_PRACT = 1 << 13, + NVME_RW_DTYPE_STREAMS = 1 << 4, }; struct nvme_dsm_cmd { @@ -604,6 +619,8 @@ enum nvme_admin_opcode { nvme_admin_download_fw = 0x11, nvme_admin_ns_attach = 0x15, nvme_admin_keep_alive = 0x18, + nvme_admin_directive_send = 0x19, + nvme_admin_directive_recv = 0x1a, nvme_admin_dbbuf = 0x7C, nvme_admin_format_nvm = 0x80, nvme_admin_security_send = 0x81, @@ -756,6 +773,24 @@ struct nvme_get_log_page_command { __u32 rsvd14[2]; }; +struct nvme_directive_cmd { + __u8 opcode; + __u8 flags; + __u16 command_id; + __le32 nsid; + __u64 rsvd2[2]; + union nvme_data_ptr dptr; + __le32 numd; + __u8 doper; + __u8 dtype; + __le16 dspec; + __u8 endir; + __u8 tdtype; + __u16 rsvd15; + + __u32 rsvd16[3]; +}; + /* * Fabrics subcommands. */ @@ -886,6 +921,18 @@ struct nvme_dbbuf { __u32 rsvd12[6]; }; +struct streams_directive_params { + __u16 msl; + __u16 nssa; + __u16 nsso; + __u8 rsvd[10]; + __u32 sws; + __u16 sgs; + __u16 nsa; + __u16 nso; + __u8 rsvd2[6]; +}; + struct nvme_command { union { struct nvme_common_command common; @@ -906,6 +953,7 @@ struct nvme_command { struct nvmf_property_set_command prop_set; struct nvmf_property_get_command prop_get; struct nvme_dbbuf dbbuf; + struct nvme_directive_cmd directive; }; };