From patchwork Thu Jun 22 02:16:13 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Uma Krishnan X-Patchwork-Id: 9803337 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 80DE460329 for ; Thu, 22 Jun 2017 02:16:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 73600223C7 for ; Thu, 22 Jun 2017 02:16:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 679D028531; Thu, 22 Jun 2017 02:16:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.9 required=2.0 tests=BAYES_00,RCVD_IN_DNSWL_HI autolearn=ham version=3.3.1 Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8B18922B27 for ; Thu, 22 Jun 2017 02:16:24 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752374AbdFVCQY (ORCPT ); Wed, 21 Jun 2017 22:16:24 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:58296 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752360AbdFVCQX (ORCPT ); Wed, 21 Jun 2017 22:16:23 -0400 Received: from pps.filterd (m0098393.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.20/8.16.0.20) with SMTP id v5M2EYI5044261 for ; Wed, 21 Jun 2017 22:16:23 -0400 Received: from e12.ny.us.ibm.com (e12.ny.us.ibm.com [129.33.205.202]) by mx0a-001b2d01.pphosted.com with ESMTP id 2b7uxfyhvh-1 (version=TLSv1.2 cipher=AES256-SHA bits=256 verify=NOT) for ; Wed, 21 Jun 2017 22:16:22 -0400 Received: from localhost by e12.ny.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 21 Jun 2017 22:16:21 -0400 Received: from b01cxnp22036.gho.pok.ibm.com (9.57.198.26) by e12.ny.us.ibm.com (146.89.104.199) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; Wed, 21 Jun 2017 22:16:18 -0400 Received: from b01ledav003.gho.pok.ibm.com (b01ledav003.gho.pok.ibm.com [9.57.199.108]) by b01cxnp22036.gho.pok.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id v5M2GGHx58786000; Thu, 22 Jun 2017 02:16:16 GMT Received: from b01ledav003.gho.pok.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3FFC2B2046; Wed, 21 Jun 2017 22:13:49 -0400 (EDT) Received: from p8tul1-build.aus.stglabs.ibm.com (unknown [9.3.141.206]) by b01ledav003.gho.pok.ibm.com (Postfix) with ESMTP id 6D52CB204E; Wed, 21 Jun 2017 22:13:48 -0400 (EDT) From: Uma Krishnan To: linux-scsi@vger.kernel.org, James Bottomley , "Martin K. Petersen" , "Matthew R. Ochs" , "Manoj N. Kumar" Cc: linuxppc-dev@lists.ozlabs.org, Ian Munsie , Andrew Donnellan , Frederic Barrat , Christophe Lombard Subject: [PATCH 13/17] cxlflash: Support LUN provisioning Date: Wed, 21 Jun 2017 21:16:13 -0500 X-Mailer: git-send-email 2.1.0 In-Reply-To: <1498097563-8680-1-git-send-email-ukrishn@linux.vnet.ibm.com> References: <1498097563-8680-1-git-send-email-ukrishn@linux.vnet.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 17062202-0048-0000-0000-000001AE9018 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00007269; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000214; SDB=6.00878073; UDB=6.00437482; IPR=6.00658213; BA=6.00005434; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015913; XFM=3.00000015; UTC=2017-06-22 02:16:20 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17062202-0049-0000-0000-0000419BE8F5 Message-Id: <1498097773-9151-1-git-send-email-ukrishn@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2017-06-22_01:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1706220037 Sender: linux-scsi-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-scsi@vger.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP From: "Matthew R. Ochs" Adopt the SISLite AFU LUN provisioning capability to allow future CXL Flash adapters the ability to better manage storage. Update the SISLite header with the changes necessary to support LUN provision operations and create a host ioctl interface for user LUN management software. Also update the cxlflash documentation to describe this new host ioctl. Signed-off-by: Matthew R. Ochs Signed-off-by: Uma Krishnan --- Documentation/powerpc/cxlflash.txt | 31 +++++++++++ drivers/scsi/cxlflash/common.h | 5 ++ drivers/scsi/cxlflash/main.c | 107 ++++++++++++++++++++++++++++++++++++- drivers/scsi/cxlflash/main.h | 5 ++ drivers/scsi/cxlflash/sislite.h | 22 +++++++- include/uapi/scsi/cxlflash_ioctl.h | 31 +++++++++-- 6 files changed, 192 insertions(+), 9 deletions(-) diff --git a/Documentation/powerpc/cxlflash.txt b/Documentation/powerpc/cxlflash.txt index ee67021..2d6297b 100644 --- a/Documentation/powerpc/cxlflash.txt +++ b/Documentation/powerpc/cxlflash.txt @@ -382,3 +382,34 @@ CXL Flash Driver Host IOCTLs The structure definitions for these IOCTLs are available in: uapi/scsi/cxlflash_ioctl.h + +HT_CXLFLASH_LUN_PROVISION +------------------------- + This ioctl is used to create and delete persistent LUNs on cxlflash + devices that lack an external LUN management interface. It is only + valid when used with AFUs that support the LUN provision capability. + + When sufficient space is available, LUNs can be created by specifying + the target port to host the LUN and a desired size in 4K blocks. Upon + success, the LUN ID and WWID of the created LUN will be returned and + the SCSI bus can be scanned to detect the change in LUN topology. Note + that partial allocations are not supported. Should a creation fail due + to a space issue, the target port can be queried for its current LUN + geometry. + + To remove a LUN, the device must first be disassociated from the Linux + SCSI subsystem. The LUN deletion can then be initiated by specifying a + target port and LUN ID. Upon success, the LUN geometry associated with + the port will be updated to reflect new number of provisioned LUNs and + available capacity. + + To query the LUN geometry of a port, the target port is specified and + upon success, the following information is presented: + + - Maximum number of provisioned LUNs allowed for the port + - Current number of provisioned LUNs for the port + - Maximum total capacity of provisioned LUNs for the port (4K blocks) + - Current total capacity of provisioned LUNs for the port (4K blocks) + + With this information, the number of available LUNs and capacity can be + can be calculated. diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h index c96526e..5810724 100644 --- a/drivers/scsi/cxlflash/common.h +++ b/drivers/scsi/cxlflash/common.h @@ -262,6 +262,11 @@ static inline bool afu_has_cap(struct afu *afu, u64 cap) return afu_cap & cap; } +static inline bool afu_is_lun_provision(struct afu *afu) +{ + return afu_has_cap(afu, SISL_INTVER_CAP_LUN_PROVISION); +} + static inline bool afu_is_sq_cmd_mode(struct afu *afu) { return afu_has_cap(afu, SISL_INTVER_CAP_SQ_CMD_MODE); diff --git a/drivers/scsi/cxlflash/main.c b/drivers/scsi/cxlflash/main.c index be468ed1..1279293 100644 --- a/drivers/scsi/cxlflash/main.c +++ b/drivers/scsi/cxlflash/main.c @@ -3227,14 +3227,105 @@ static int cxlflash_chr_open(struct inode *inode, struct file *file) static char *decode_hioctl(int cmd) { switch (cmd) { - default: - return "UNKNOWN"; + case HT_CXLFLASH_LUN_PROVISION: + return __stringify_1(HT_CXLFLASH_LUN_PROVISION); } return "UNKNOWN"; } /** + * cxlflash_lun_provision() - host LUN provisioning handler + * @cfg: Internal structure associated with the host. + * @arg: Kernel copy of userspace ioctl data structure. + * + * Return: 0 on success, -errno on failure + */ +static int cxlflash_lun_provision(struct cxlflash_cfg *cfg, + struct ht_cxlflash_lun_provision *lunprov) +{ + struct afu *afu = cfg->afu; + struct device *dev = &cfg->dev->dev; + struct sisl_ioarcb rcb; + struct sisl_ioasa asa; + __be64 __iomem *fc_port_regs; + u16 port = lunprov->port; + u16 scmd = lunprov->hdr.subcmd; + u16 type; + u64 reg; + u64 size; + u64 lun_id; + int rc = 0; + + if (!afu_is_lun_provision(afu)) { + rc = -ENOTSUPP; + goto out; + } + + if (port >= cfg->num_fc_ports) { + rc = -EINVAL; + goto out; + } + + switch (scmd) { + case HT_CXLFLASH_LUN_PROVISION_SUBCMD_CREATE_LUN: + type = SISL_AFU_LUN_PROVISION_CREATE; + size = lunprov->size; + lun_id = 0; + break; + case HT_CXLFLASH_LUN_PROVISION_SUBCMD_DELETE_LUN: + type = SISL_AFU_LUN_PROVISION_DELETE; + size = 0; + lun_id = lunprov->lun_id; + break; + case HT_CXLFLASH_LUN_PROVISION_SUBCMD_QUERY_PORT: + fc_port_regs = get_fc_port_regs(cfg, port); + + reg = readq_be(&fc_port_regs[FC_MAX_NUM_LUNS / 8]); + lunprov->max_num_luns = reg; + reg = readq_be(&fc_port_regs[FC_CUR_NUM_LUNS / 8]); + lunprov->cur_num_luns = reg; + reg = readq_be(&fc_port_regs[FC_MAX_CAP_PORT / 8]); + lunprov->max_cap_port = reg; + reg = readq_be(&fc_port_regs[FC_CUR_CAP_PORT / 8]); + lunprov->cur_cap_port = reg; + + goto out; + default: + rc = -EINVAL; + goto out; + } + + memset(&rcb, 0, sizeof(rcb)); + memset(&asa, 0, sizeof(asa)); + rcb.req_flags = SISL_REQ_FLAGS_AFU_CMD; + rcb.lun_id = lun_id; + rcb.msi = SISL_MSI_RRQ_UPDATED; + rcb.timeout = MC_LUN_PROV_TIMEOUT; + rcb.ioasa = &asa; + + rcb.cdb[0] = SISL_AFU_CMD_LUN_PROVISION; + rcb.cdb[1] = type; + rcb.cdb[2] = port; + put_unaligned_be64(size, &rcb.cdb[8]); + + rc = send_afu_cmd(afu, &rcb); + if (rc) { + dev_err(dev, "%s: send_afu_cmd failed rc=%d asc=%08x afux=%x\n", + __func__, rc, asa.ioasc, asa.afu_extra); + goto out; + } + + if (scmd == HT_CXLFLASH_LUN_PROVISION_SUBCMD_CREATE_LUN) { + lunprov->lun_id = (u64)asa.lunid_hi << 32 | asa.lunid_lo; + memcpy(lunprov->wwid, asa.wwid, sizeof(lunprov->wwid)); + } +out: + dev_dbg(dev, "%s: returning rc=%d\n", __func__, rc); + return rc; +} + +/** * cxlflash_chr_ioctl() - character device IOCTL handler * @file: File pointer for this device. * @cmd: IOCTL command. @@ -3270,6 +3361,8 @@ static long cxlflash_chr_ioctl(struct file *file, unsigned int cmd, size_t size; hioctl ioctl; } ioctl_tbl[] = { /* NOTE: order matters here */ + { sizeof(struct ht_cxlflash_lun_provision), + (hioctl)cxlflash_lun_provision }, }; /* Hold read semaphore so we can drain if needed */ @@ -3279,6 +3372,16 @@ static long cxlflash_chr_ioctl(struct file *file, unsigned int cmd, __func__, cmd, idx, sizeof(ioctl_tbl)); switch (cmd) { + case HT_CXLFLASH_LUN_PROVISION: + known_ioctl = true; + idx = _IOC_NR(HT_CXLFLASH_LUN_PROVISION) - _IOC_NR(cmd); + size = ioctl_tbl[idx].size; + do_ioctl = ioctl_tbl[idx].ioctl; + + if (likely(do_ioctl)) + break; + + /* fall through */ default: rc = -EINVAL; goto out; diff --git a/drivers/scsi/cxlflash/main.h b/drivers/scsi/cxlflash/main.h index 44c86c2..6ae8a11 100644 --- a/drivers/scsi/cxlflash/main.h +++ b/drivers/scsi/cxlflash/main.h @@ -41,6 +41,10 @@ /* FC defines */ #define FC_MTIP_CMDCONFIG 0x010 #define FC_MTIP_STATUS 0x018 +#define FC_MAX_NUM_LUNS 0x080 /* Max LUNs host can provision for port */ +#define FC_CUR_NUM_LUNS 0x088 /* Cur number LUNs provisioned for port */ +#define FC_MAX_CAP_PORT 0x090 /* Max capacity all LUNs for port (4K blocks) */ +#define FC_CUR_CAP_PORT 0x098 /* Cur capacity all LUNs for port (4K blocks) */ #define FC_PNAME 0x300 #define FC_CONFIG 0x320 @@ -63,6 +67,7 @@ /* AFU command timeout values */ #define MC_AFU_SYNC_TIMEOUT 5 /* 5 secs */ +#define MC_LUN_PROV_TIMEOUT 5 /* 5 secs */ /* AFU command room retry limit */ #define MC_ROOM_RETRY_CNT 10 diff --git a/drivers/scsi/cxlflash/sislite.h b/drivers/scsi/cxlflash/sislite.h index 483710a..c216fee 100644 --- a/drivers/scsi/cxlflash/sislite.h +++ b/drivers/scsi/cxlflash/sislite.h @@ -73,6 +73,10 @@ struct sisl_ioarcb { u32 rsvd1; u8 cdb[16]; /* must be in big endian */ #define SISL_AFU_CMD_SYNC 0xC0 /* AFU sync command */ +#define SISL_AFU_CMD_LUN_PROVISION 0xD0 /* AFU LUN provision command */ + +#define SISL_AFU_LUN_PROVISION_CREATE 0x00 /* LUN provision create type */ +#define SISL_AFU_LUN_PROVISION_DELETE 0x01 /* LUN provision delete type */ union { u64 reserved; /* Reserved for IOARRIN mode */ @@ -158,6 +162,7 @@ struct sisl_rc { }; #define SISL_SENSE_DATA_LEN 20 /* Sense data length */ +#define SISL_WWID_DATA_LEN 16 /* WWID data length */ /* * IOASA: 64 bytes & must follow IOARCB, min 16 byte alignment required, @@ -169,7 +174,12 @@ struct sisl_ioasa { u32 ioasc; #define SISL_IOASC_GOOD_COMPLETION 0x00000000U }; - u32 resid; + + union { + u32 resid; + u32 lunid_hi; + }; + u8 port; u8 afu_extra; /* when afu_rc=0x04, 0x14, 0x31 (_xxx_DMA_ERR): @@ -192,7 +202,14 @@ struct sisl_ioasa { u8 scsi_extra; u8 fc_extra; - u8 sense_data[SISL_SENSE_DATA_LEN]; + + union { + u8 sense_data[SISL_SENSE_DATA_LEN]; + struct { + u32 lunid_lo; + u8 wwid[SISL_WWID_DATA_LEN]; + }; + }; /* These fields are defined by the SISlite architecture for the * host to use as they see fit for their implementation. @@ -394,6 +411,7 @@ struct sisl_global_regs { #define SISL_INTVER_CAP_SQ_CMD_MODE 0x400000000000ULL #define SISL_INTVER_CAP_RESERVED_CMD_MODE_A 0x200000000000ULL #define SISL_INTVER_CAP_RESERVED_CMD_MODE_B 0x100000000000ULL +#define SISL_INTVER_CAP_LUN_PROVISION 0x080000000000ULL }; #define CXLFLASH_NUM_FC_PORTS_PER_BANK 2 /* fixed # of ports per bank */ diff --git a/include/uapi/scsi/cxlflash_ioctl.h b/include/uapi/scsi/cxlflash_ioctl.h index 87e1f63..ad79c34 100644 --- a/include/uapi/scsi/cxlflash_ioctl.h +++ b/include/uapi/scsi/cxlflash_ioctl.h @@ -202,14 +202,34 @@ union cxlflash_ioctls { #define HT_CXLFLASH_VERSION_0 0 struct ht_cxlflash_hdr { - __u16 version; /* Version data */ - __u16 subcmd; /* Sub-command */ - __u16 rsvd[2]; /* Reserved for future use */ - __u64 flags; /* Input flags */ - __u64 return_flags; /* Returned flags */ + __u16 version; /* Version data */ + __u16 subcmd; /* Sub-command */ + __u16 rsvd[2]; /* Reserved for future use */ + __u64 flags; /* Input flags */ + __u64 return_flags; /* Returned flags */ +}; + +#define HT_CXLFLASH_LUN_PROVISION_SUBCMD_CREATE_LUN 0x0001 +#define HT_CXLFLASH_LUN_PROVISION_SUBCMD_DELETE_LUN 0x0002 +#define HT_CXLFLASH_LUN_PROVISION_SUBCMD_QUERY_PORT 0x0003 +#define HT_CXLFLASH_LUN_PROVISION_WWID_LEN 16 + +struct ht_cxlflash_lun_provision { + struct ht_cxlflash_hdr hdr; /* Common fields */ + __u16 port; /* Target port for provision request */ + __u16 reserved16[3]; /* Reserved for future use */ + __u64 size; /* Size of LUN (4K blocks) */ + __u64 lun_id; /* SCSI LUN ID */ + __u8 wwid[HT_CXLFLASH_LUN_PROVISION_WWID_LEN]; /* Page83 WWID, NAA-6 */ + __u64 max_num_luns; /* Maximum number of LUNs provisioned */ + __u64 cur_num_luns; /* Current number of LUNs provisioned */ + __u64 max_cap_port; /* Total capacity for port (4K blocks) */ + __u64 cur_cap_port; /* Current capacity for port (4K blocks) */ + __u64 reserved[8]; /* Reserved for future use */ }; union cxlflash_ht_ioctls { + struct ht_cxlflash_lun_provision lun_provision; }; #define MAX_HT_CXLFLASH_IOCTL_SZ (sizeof(union cxlflash_ht_ioctls)) @@ -218,6 +238,7 @@ union cxlflash_ht_ioctls { * CXL Flash host ioctls start at the top of the reserved CXL_MAGIC * region (0xBF) and grow downwards. */ +#define HT_CXLFLASH_LUN_PROVISION CXL_IOWR(0xBF, ht_cxlflash_lun_provision) #endif /* ifndef _CXLFLASH_IOCTL_H */