From patchwork Wed Jun 14 04:01:59 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jens Axboe X-Patchwork-Id: 9785305 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id DB5EC60325 for ; Wed, 14 Jun 2017 04:02:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id E4C0A2844E for ; Wed, 14 Jun 2017 04:02:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id D9D332858D; Wed, 14 Jun 2017 04:02:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_HI autolearn=unavailable version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 1E80F2844E for ; Wed, 14 Jun 2017 04:02:29 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750728AbdFNEC0 (ORCPT ); Wed, 14 Jun 2017 00:02:26 -0400 Received: from mail-pf0-f173.google.com ([209.85.192.173]:32787 "EHLO mail-pf0-f173.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750849AbdFNECT (ORCPT ); Wed, 14 Jun 2017 00:02:19 -0400 Received: by mail-pf0-f173.google.com with SMTP id 83so77501376pfr.0 for ; Tue, 13 Jun 2017 21:02:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kernel-dk.20150623.gappssmtp.com; s=20150623; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=3rBfb2Efz6a8RSo0m+bWhorpL8bCwHlWIHfhX+srvlw=; b=E+V1r2JRxDm85CYso2qyaP/JGcnAFZhGMu4YruW3s2+pb0/n1eJK7vEbQn2ua+AmGu 36ekgdOX6sYTP1B+ytuJIY1+K0bdt6JcT6c6jff3do5NeizR26j3ToqKp4Sh3tW0lqi9 lZXWhMZ4zcNrE5PokLMdy5sseMvT7ogplCkQzSMe+77aNFksJ1//6pLM8A4Hkra3vKRf HFXpod0g4vKfSq4h7mcp4txZgYqawmZpdgYdU7T0KTPpS6g8+pSryMyQfOiZXb/pRBaj KWqkWLFj0bnQSa9sNBLSpulqJW+qr7iAsjMuUeRGbDLvSSBnSzg6wS2BIropsxvhhlzo SrJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=3rBfb2Efz6a8RSo0m+bWhorpL8bCwHlWIHfhX+srvlw=; b=obOWRYe9ovG2nXeUnnHRoFwHw6Pk8qgsrjbOC3SrMfj7WMPSDTpJh4XOB1nFreyvWa vH6u4l3orhVCmn7opHCGp7OXLVQZc3s3WpoiSBwN3n9SvVqIy6QuI3o+8cvhBTSnSe4v rlyMwnktQFu4/FVVKFCKfE3sLe7KTpy4LDV2H4QioxVpKNjxwM0JwvG6IgKvNh0rR9a1 hGiTujBAQAlFq92BT9vIPfg6ngK5tMKn8VXB+JC1HSPrAcsVjIh0A2WBwa6aLzeub8AP R8zqPO3389L8jkJHpkR7pL7cadmiVewtkzyVC1Ou9VVrx9MJHb8x+updRwL5q3mpQ4Ol ALeQ== X-Gm-Message-State: AKS2vOz1DFJmEZlpcoaZBaNBVMbtW5fQ9Ri4E1HuG7KRLgDTkaptyT3+ Bf/WBdew4UysN+2OHWgD4A== X-Received: by 10.99.163.110 with SMTP id v46mr2441761pgn.210.1497412938303; Tue, 13 Jun 2017 21:02:18 -0700 (PDT) Received: from localhost.localdomain (66.29.164.166.static.utbb.net. [66.29.164.166]) by smtp.gmail.com with ESMTPSA id f8sm30914083pfc.14.2017.06.13.21.02.16 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 13 Jun 2017 21:02:17 -0700 (PDT) From: Jens Axboe To: linux-fsdevel@vger.kernel.org, linux-block@vger.kernel.org Cc: adilger@dilger.ca, Jens Axboe Subject: [PATCH 10/10] nvme: add support for streams and directives Date: Tue, 13 Jun 2017 22:01:59 -0600 Message-Id: <1497412919-19400-11-git-send-email-axboe@kernel.dk> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1497412919-19400-1-git-send-email-axboe@kernel.dk> References: <1497412919-19400-1-git-send-email-axboe@kernel.dk> Sender: linux-fsdevel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-fsdevel@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP This adds support for Directives in NVMe, particular for the Streams directive. Support for Directives is a new feature in NVMe 1.3. It allows a user to pass in information about where to store the data, so that it the device can do so most effiently. If an application is managing and writing data with different life times, mixing differently retentioned data onto the same locations on flash can cause write amplification to grow. This, in turn, will reduce performance and life time of the device. We default to allocating 4 streams per name space, but it is configurable with the 'streams_per_ns' module option. If a write stream is set in a write, flag is as such before sending it to the device. Some debug stuff in this patch, dumping streams ID params when we load nvme. Signed-off-by: Jens Axboe --- drivers/nvme/host/core.c | 124 +++++++++++++++++++++++++++++++++++++++++++++++ drivers/nvme/host/nvme.h | 1 + include/linux/nvme.h | 48 ++++++++++++++++++ 3 files changed, 173 insertions(+) diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 903d5813023a..d7cbd050ddf4 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -65,6 +65,10 @@ static bool force_apst; module_param(force_apst, bool, 0644); MODULE_PARM_DESC(force_apst, "allow APST for newly enumerated devices even if quirked off"); +static char streams_per_ns = 4; +module_param(streams_per_ns, byte, 0644); +MODULE_PARM_DESC(streams_per_ns, "if available, allocate this many streams per NS"); + static LIST_HEAD(nvme_ctrl_list); static DEFINE_SPINLOCK(dev_list_lock); @@ -351,6 +355,15 @@ static inline void nvme_setup_rw(struct nvme_ns *ns, struct request *req, cmnd->rw.slba = cpu_to_le64(nvme_block_nr(ns, blk_rq_pos(req))); cmnd->rw.length = cpu_to_le16((blk_rq_bytes(req) >> ns->lba_shift) - 1); + if (req_op(req) == REQ_OP_WRITE) { + if (bio_stream_valid(req->bio) && ns->nr_streams) { + unsigned stream = bio_stream(req->bio) & 0xffff; + + control |= NVME_RW_DTYPE_STREAMS; + dsmgmt |= ((stream % (ns->nr_streams + 1)) << 16); + } + } + if (ns->ms) { switch (ns->pi_type) { case NVME_NS_DPS_PI_TYPE3: @@ -1073,6 +1086,109 @@ static int nvme_revalidate_disk(struct gendisk *disk) return 0; } +static int nvme_enable_streams(struct nvme_ns *ns) +{ + struct nvme_command c; + + memset(&c, 0, sizeof(c)); + + c.directive.opcode = nvme_admin_directive_send; + c.directive.nsid = cpu_to_le32(ns->ns_id); + c.directive.doper = NVME_DIR_SND_ID_OP_ENABLE; + c.directive.dtype = NVME_DIR_IDENTIFY; + c.directive.tdtype = NVME_DIR_STREAMS; + c.directive.endir = NVME_DIR_ENDIR; + + return nvme_submit_sync_cmd(ns->ctrl->admin_q, &c, NULL, 0); +} + +static int nvme_streams_params(struct nvme_ns *ns) +{ + struct nvme_ctrl *ctrl = ns->ctrl; + struct streams_directive_params s; + struct nvme_command c; + int ret; + + memset(&c, 0, sizeof(c)); + memset(&s, 0, sizeof(s)); + + c.directive.opcode = nvme_admin_directive_recv; + c.directive.nsid = cpu_to_le32(ns->ns_id); + c.directive.numd = sizeof(s); + c.directive.doper = NVME_DIR_RCV_ST_OP_PARAM; + c.directive.dtype = NVME_DIR_STREAMS; + + ret = nvme_submit_sync_cmd(ctrl->admin_q, &c, &s, sizeof(s)); + if (ret) + return ret; + + s.msl = le16_to_cpu(s.msl); + s.nssa = le16_to_cpu(s.nssa); + s.nsso = le16_to_cpu(s.nsso); + s.sws = le32_to_cpu(s.sws); + s.sgs = le16_to_cpu(s.sgs); + s.nsa = le16_to_cpu(s.nsa); + s.nso = le16_to_cpu(s.nso); + + dev_info(ctrl->device, "streams: msl=%u, nssa=%u, nsso=%u, sws=%u " + "sgs=%u, nsa=%u, nso=%u\n", s.msl, s.nssa, + s.nsso, s.sws, s.sgs, s.nsa, s.nso); + return 0; +} + +static int nvme_streams_allocate(struct nvme_ns *ns, unsigned int streams) +{ + struct nvme_command c; + + memset(&c, 0, sizeof(c)); + + c.directive.opcode = nvme_admin_directive_recv; + c.directive.nsid = cpu_to_le32(ns->ns_id); + c.directive.doper = NVME_DIR_RCV_ST_OP_RESOURCE; + c.directive.dtype = NVME_DIR_STREAMS; + c.directive.endir = streams; + + return nvme_submit_sync_cmd(ns->ctrl->admin_q, &c, NULL, 0); +} + +static int nvme_streams_deallocate(struct nvme_ns *ns) +{ + struct nvme_command c; + + memset(&c, 0, sizeof(c)); + + c.directive.opcode = nvme_admin_directive_send; + c.directive.nsid = cpu_to_le32(ns->ns_id); + c.directive.doper = NVME_DIR_SND_ST_OP_REL_RSC; + c.directive.dtype = NVME_DIR_STREAMS; + + return nvme_submit_sync_cmd(ns->ctrl->admin_q, &c, NULL, 0); +} + +static void nvme_config_streams(struct nvme_ns *ns) +{ + int ret; + + ret = nvme_enable_streams(ns); + if (ret) + return; + + ret = nvme_streams_params(ns); + if (ret) + return; + + ret = nvme_streams_allocate(ns, streams_per_ns); + if (ret) + return; + + ret = nvme_streams_params(ns); + if (ret) + return; + + ns->nr_streams = streams_per_ns; + dev_info(ns->ctrl->device, "successfully enabled streams\n"); +} + static char nvme_pr_type(enum pr_type type) { switch (type) { @@ -1606,6 +1722,9 @@ int nvme_init_identify(struct nvme_ctrl *ctrl) ctrl->sgls = le32_to_cpu(id->sgls); ctrl->kas = le16_to_cpu(id->kas); + if (ctrl->oacs & NVME_CTRL_OACS_DIRECTIVES) + dev_info(ctrl->dev, "supports directives\n"); + ctrl->npss = id->npss; prev_apsta = ctrl->apsta; if (ctrl->quirks & NVME_QUIRK_NO_APST) { @@ -2060,6 +2179,9 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, unsigned nsid) goto out_free_id; } + if (ctrl->oacs & NVME_CTRL_OACS_DIRECTIVES) + nvme_config_streams(ns); + disk = alloc_disk_node(0, node); if (!disk) goto out_free_id; @@ -2112,6 +2234,8 @@ static void nvme_ns_remove(struct nvme_ns *ns) &nvme_ns_attr_group); if (ns->ndev) nvme_nvm_unregister_sysfs(ns); + if (ns->nr_streams) + nvme_streams_deallocate(ns); del_gendisk(ns->disk); blk_cleanup_queue(ns->queue); } diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 9d6a070d4391..dc87c8284259 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -192,6 +192,7 @@ struct nvme_ns { u8 uuid[16]; unsigned ns_id; + unsigned nr_streams; int lba_shift; u16 ms; bool ext; diff --git a/include/linux/nvme.h b/include/linux/nvme.h index b625bacf37ef..8b2f5b140134 100644 --- a/include/linux/nvme.h +++ b/include/linux/nvme.h @@ -245,6 +245,7 @@ enum { NVME_CTRL_ONCS_WRITE_ZEROES = 1 << 3, NVME_CTRL_VWC_PRESENT = 1 << 0, NVME_CTRL_OACS_SEC_SUPP = 1 << 0, + NVME_CTRL_OACS_DIRECTIVES = 1 << 5, NVME_CTRL_OACS_DBBUF_SUPP = 1 << 7, }; @@ -295,6 +296,19 @@ enum { }; enum { + NVME_DIR_IDENTIFY = 0x00, + NVME_DIR_STREAMS = 0x01, + NVME_DIR_SND_ID_OP_ENABLE = 0x01, + NVME_DIR_SND_ST_OP_REL_ID = 0x01, + NVME_DIR_SND_ST_OP_REL_RSC = 0x02, + NVME_DIR_RCV_ID_OP_PARAM = 0x01, + NVME_DIR_RCV_ST_OP_PARAM = 0x01, + NVME_DIR_RCV_ST_OP_STATUS = 0x02, + NVME_DIR_RCV_ST_OP_RESOURCE = 0x03, + NVME_DIR_ENDIR = 0x01, +}; + +enum { NVME_NS_FEAT_THIN = 1 << 0, NVME_NS_FLBAS_LBA_MASK = 0xf, NVME_NS_FLBAS_META_EXT = 0x10, @@ -535,6 +549,7 @@ enum { NVME_RW_PRINFO_PRCHK_APP = 1 << 11, NVME_RW_PRINFO_PRCHK_GUARD = 1 << 12, NVME_RW_PRINFO_PRACT = 1 << 13, + NVME_RW_DTYPE_STREAMS = 1 << 4, }; struct nvme_dsm_cmd { @@ -604,6 +619,8 @@ enum nvme_admin_opcode { nvme_admin_download_fw = 0x11, nvme_admin_ns_attach = 0x15, nvme_admin_keep_alive = 0x18, + nvme_admin_directive_send = 0x19, + nvme_admin_directive_recv = 0x1a, nvme_admin_dbbuf = 0x7C, nvme_admin_format_nvm = 0x80, nvme_admin_security_send = 0x81, @@ -756,6 +773,24 @@ struct nvme_get_log_page_command { __u32 rsvd14[2]; }; +struct nvme_directive_cmd { + __u8 opcode; + __u8 flags; + __u16 command_id; + __le32 nsid; + __u64 rsvd2[2]; + union nvme_data_ptr dptr; + __le32 numd; + __u8 doper; + __u8 dtype; + __le16 dspec; + __u8 endir; + __u8 tdtype; + __u16 rsvd15; + + __u32 rsvd16[3]; +}; + /* * Fabrics subcommands. */ @@ -886,6 +921,18 @@ struct nvme_dbbuf { __u32 rsvd12[6]; }; +struct streams_directive_params { + __u16 msl; + __u16 nssa; + __u16 nsso; + __u8 rsvd[10]; + __u32 sws; + __u16 sgs; + __u16 nsa; + __u16 nso; + __u8 rsvd2[6]; +}; + struct nvme_command { union { struct nvme_common_command common; @@ -906,6 +953,7 @@ struct nvme_command { struct nvmf_property_set_command prop_set; struct nvmf_property_get_command prop_get; struct nvme_dbbuf dbbuf; + struct nvme_directive_cmd directive; }; };