From patchwork Wed Jan 22 23:50:32 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947710 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 351B61CAA86 for ; Wed, 22 Jan 2025 23:52:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589924; cv=none; b=gvyUtPOfiBGQIgwY2cG3QGo4tHOmzAU0EGrBYaAfQ5tt+6xY5U/XWObwQRqj6Tu49cNxuoi8dQMuGQjnw+40A0enSJCmOZZNsOKk+BagtVnQkmG9QgE42HJSbNQcBvZqnB1dsL1PG51Z+aR/1XgNOZcHL8a57xVoXNsKcZML+bc= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589924; c=relaxed/simple; bh=OHRiJKJZnBiMKzsEF+/ldQ+2nonAmTCGVTyjqoZkr64=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=n9Sp6iKdLY56LKalfnhjH419FZdgpOAUQJmz3fQoPxGWQS4yFYK8HWRgCfAsV+b7sP+yVCV+NWnovcd8D2ZF9B0JvztKwi32SO9tO+jgxNtDswLNSQCmVjVAkA9wr/f5kKUR9ojNO6uvTXSm79O50PgrJCiuWkvua7yNxyuo8rM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7E70CC4CED6; Wed, 22 Jan 2025 23:52:03 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 01/19] cxl: Refactor user ioctl command path from mds to mailbox Date: Wed, 22 Jan 2025 16:50:32 -0700 Message-ID: <20250122235159.2716036-2-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 With 'struct cxl_mailbox' context introduced, the helper functions cxl_query_cmd() and cxl_send_cmd() can take a cxl_mailbox directly rather than a cxl_memdev parameter. Refactor to use cxl_mailbox directly. Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams Signed-off-by: Dave Jiang --- drivers/cxl/core/core.h | 6 ++- drivers/cxl/core/mbox.c | 89 +++++++++++++++++++-------------------- drivers/cxl/core/memdev.c | 22 +++++++--- drivers/cxl/cxlmem.h | 40 ------------------ include/cxl/mailbox.h | 41 +++++++++++++++++- 5 files changed, 102 insertions(+), 96 deletions(-) diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 800466f96a68..23761340e65c 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -4,6 +4,8 @@ #ifndef __CXL_CORE_H__ #define __CXL_CORE_H__ +#include + extern const struct device_type cxl_nvdimm_bridge_type; extern const struct device_type cxl_nvdimm_type; extern const struct device_type cxl_pmu_type; @@ -65,9 +67,9 @@ static inline void cxl_region_exit(void) struct cxl_send_command; struct cxl_mem_query_commands; -int cxl_query_cmd(struct cxl_memdev *cxlmd, +int cxl_query_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mem_query_commands __user *q); -int cxl_send_cmd(struct cxl_memdev *cxlmd, struct cxl_send_command __user *s); +int cxl_send_cmd(struct cxl_mailbox *cxl_mailbox, struct cxl_send_command __user *s); void __iomem *devm_cxl_iomap_block(struct device *dev, resource_size_t addr, resource_size_t length); diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 548564c770c0..bdb8f060f2c1 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -349,40 +349,40 @@ static bool cxl_payload_from_user_allowed(u16 opcode, void *payload_in) return true; } -static int cxl_mbox_cmd_ctor(struct cxl_mbox_cmd *mbox, - struct cxl_memdev_state *mds, u16 opcode, +static int cxl_mbox_cmd_ctor(struct cxl_mbox_cmd *mbox_cmd, + struct cxl_mailbox *cxl_mbox, u16 opcode, size_t in_size, size_t out_size, u64 in_payload) { - struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; - *mbox = (struct cxl_mbox_cmd) { + *mbox_cmd = (struct cxl_mbox_cmd) { .opcode = opcode, .size_in = in_size, }; if (in_size) { - mbox->payload_in = vmemdup_user(u64_to_user_ptr(in_payload), - in_size); - if (IS_ERR(mbox->payload_in)) - return PTR_ERR(mbox->payload_in); + mbox_cmd->payload_in = vmemdup_user(u64_to_user_ptr(in_payload), + in_size); + if (IS_ERR(mbox_cmd->payload_in)) + return PTR_ERR(mbox_cmd->payload_in); - if (!cxl_payload_from_user_allowed(opcode, mbox->payload_in)) { - dev_dbg(mds->cxlds.dev, "%s: input payload not allowed\n", + if (!cxl_payload_from_user_allowed(opcode, + mbox_cmd->payload_in)) { + dev_dbg(cxl_mbox->host, "%s: input payload not allowed\n", cxl_mem_opcode_to_name(opcode)); - kvfree(mbox->payload_in); + kvfree(mbox_cmd->payload_in); return -EBUSY; } } /* Prepare to handle a full payload for variable sized output */ if (out_size == CXL_VARIABLE_PAYLOAD) - mbox->size_out = cxl_mbox->payload_size; + mbox_cmd->size_out = cxl_mbox->payload_size; else - mbox->size_out = out_size; + mbox_cmd->size_out = out_size; - if (mbox->size_out) { - mbox->payload_out = kvzalloc(mbox->size_out, GFP_KERNEL); - if (!mbox->payload_out) { - kvfree(mbox->payload_in); + if (mbox_cmd->size_out) { + mbox_cmd->payload_out = kvzalloc(mbox_cmd->size_out, GFP_KERNEL); + if (!mbox_cmd->payload_out) { + kvfree(mbox_cmd->payload_in); return -ENOMEM; } } @@ -397,10 +397,8 @@ static void cxl_mbox_cmd_dtor(struct cxl_mbox_cmd *mbox) static int cxl_to_mem_cmd_raw(struct cxl_mem_command *mem_cmd, const struct cxl_send_command *send_cmd, - struct cxl_memdev_state *mds) + struct cxl_mailbox *cxl_mbox) { - struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; - if (send_cmd->raw.rsvd) return -EINVAL; @@ -415,7 +413,7 @@ static int cxl_to_mem_cmd_raw(struct cxl_mem_command *mem_cmd, if (!cxl_mem_raw_command_allowed(send_cmd->raw.opcode)) return -EPERM; - dev_WARN_ONCE(mds->cxlds.dev, true, "raw command path used\n"); + dev_WARN_ONCE(cxl_mbox->host, true, "raw command path used\n"); *mem_cmd = (struct cxl_mem_command) { .info = { @@ -431,7 +429,7 @@ static int cxl_to_mem_cmd_raw(struct cxl_mem_command *mem_cmd, static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd, const struct cxl_send_command *send_cmd, - struct cxl_memdev_state *mds) + struct cxl_mailbox *cxl_mbox) { struct cxl_mem_command *c = &cxl_mem_commands[send_cmd->id]; const struct cxl_command_info *info = &c->info; @@ -446,11 +444,11 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd, return -EINVAL; /* Check that the command is enabled for hardware */ - if (!test_bit(info->id, mds->enabled_cmds)) + if (!test_bit(info->id, cxl_mbox->enabled_cmds)) return -ENOTTY; /* Check that the command is not claimed for exclusive kernel use */ - if (test_bit(info->id, mds->exclusive_cmds)) + if (test_bit(info->id, cxl_mbox->exclusive_cmds)) return -EBUSY; /* Check the input buffer is the expected size */ @@ -479,7 +477,7 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd, /** * cxl_validate_cmd_from_user() - Check fields for CXL_MEM_SEND_COMMAND. * @mbox_cmd: Sanitized and populated &struct cxl_mbox_cmd. - * @mds: The driver data for the operation + * @cxl_mbox: CXL mailbox context * @send_cmd: &struct cxl_send_command copied in from userspace. * * Return: @@ -494,10 +492,9 @@ static int cxl_to_mem_cmd(struct cxl_mem_command *mem_cmd, * safe to send to the hardware. */ static int cxl_validate_cmd_from_user(struct cxl_mbox_cmd *mbox_cmd, - struct cxl_memdev_state *mds, + struct cxl_mailbox *cxl_mbox, const struct cxl_send_command *send_cmd) { - struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; struct cxl_mem_command mem_cmd; int rc; @@ -514,24 +511,23 @@ static int cxl_validate_cmd_from_user(struct cxl_mbox_cmd *mbox_cmd, /* Sanitize and construct a cxl_mem_command */ if (send_cmd->id == CXL_MEM_COMMAND_ID_RAW) - rc = cxl_to_mem_cmd_raw(&mem_cmd, send_cmd, mds); + rc = cxl_to_mem_cmd_raw(&mem_cmd, send_cmd, cxl_mbox); else - rc = cxl_to_mem_cmd(&mem_cmd, send_cmd, mds); + rc = cxl_to_mem_cmd(&mem_cmd, send_cmd, cxl_mbox); if (rc) return rc; /* Sanitize and construct a cxl_mbox_cmd */ - return cxl_mbox_cmd_ctor(mbox_cmd, mds, mem_cmd.opcode, + return cxl_mbox_cmd_ctor(mbox_cmd, cxl_mbox, mem_cmd.opcode, mem_cmd.info.size_in, mem_cmd.info.size_out, send_cmd->in.payload); } -int cxl_query_cmd(struct cxl_memdev *cxlmd, +int cxl_query_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_mem_query_commands __user *q) { - struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); - struct device *dev = &cxlmd->dev; + struct device *dev = cxl_mbox->host; struct cxl_mem_command *cmd; u32 n_commands; int j = 0; @@ -552,9 +548,9 @@ int cxl_query_cmd(struct cxl_memdev *cxlmd, cxl_for_each_cmd(cmd) { struct cxl_command_info info = cmd->info; - if (test_bit(info.id, mds->enabled_cmds)) + if (test_bit(info.id, cxl_mbox->enabled_cmds)) info.flags |= CXL_MEM_COMMAND_FLAG_ENABLED; - if (test_bit(info.id, mds->exclusive_cmds)) + if (test_bit(info.id, cxl_mbox->exclusive_cmds)) info.flags |= CXL_MEM_COMMAND_FLAG_EXCLUSIVE; if (copy_to_user(&q->commands[j++], &info, sizeof(info))) @@ -569,7 +565,7 @@ int cxl_query_cmd(struct cxl_memdev *cxlmd, /** * handle_mailbox_cmd_from_user() - Dispatch a mailbox command for userspace. - * @mds: The driver data for the operation + * @cxl_mbox: The mailbox context for the operation. * @mbox_cmd: The validated mailbox command. * @out_payload: Pointer to userspace's output payload. * @size_out: (Input) Max payload size to copy out. @@ -590,13 +586,12 @@ int cxl_query_cmd(struct cxl_memdev *cxlmd, * * See cxl_send_cmd(). */ -static int handle_mailbox_cmd_from_user(struct cxl_memdev_state *mds, +static int handle_mailbox_cmd_from_user(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *mbox_cmd, u64 out_payload, s32 *size_out, u32 *retval) { - struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; - struct device *dev = mds->cxlds.dev; + struct device *dev = cxl_mbox->host; int rc; dev_dbg(dev, @@ -633,10 +628,9 @@ static int handle_mailbox_cmd_from_user(struct cxl_memdev_state *mds, return rc; } -int cxl_send_cmd(struct cxl_memdev *cxlmd, struct cxl_send_command __user *s) +int cxl_send_cmd(struct cxl_mailbox *cxl_mbox, struct cxl_send_command __user *s) { - struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); - struct device *dev = &cxlmd->dev; + struct device *dev = cxl_mbox->host; struct cxl_send_command send; struct cxl_mbox_cmd mbox_cmd; int rc; @@ -646,11 +640,11 @@ int cxl_send_cmd(struct cxl_memdev *cxlmd, struct cxl_send_command __user *s) if (copy_from_user(&send, s, sizeof(send))) return -EFAULT; - rc = cxl_validate_cmd_from_user(&mbox_cmd, mds, &send); + rc = cxl_validate_cmd_from_user(&mbox_cmd, cxl_mbox, &send); if (rc) return rc; - rc = handle_mailbox_cmd_from_user(mds, &mbox_cmd, send.out.payload, + rc = handle_mailbox_cmd_from_user(cxl_mbox, &mbox_cmd, send.out.payload, &send.out.size, &send.retval); if (rc) return rc; @@ -724,6 +718,7 @@ static int cxl_xfer_log(struct cxl_memdev_state *mds, uuid_t *uuid, */ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel) { + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; struct cxl_cel_entry *cel_entry; const int cel_entries = size / sizeof(*cel_entry); struct device *dev = mds->cxlds.dev; @@ -737,7 +732,7 @@ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel) int enabled = 0; if (cmd) { - set_bit(cmd->info.id, mds->enabled_cmds); + set_bit(cmd->info.id, cxl_mbox->enabled_cmds); enabled++; } @@ -807,6 +802,7 @@ static const uuid_t log_uuid[] = { */ int cxl_enumerate_cmds(struct cxl_memdev_state *mds) { + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; struct cxl_mbox_get_supported_logs *gsl; struct device *dev = mds->cxlds.dev; struct cxl_mem_command *cmd; @@ -845,7 +841,7 @@ int cxl_enumerate_cmds(struct cxl_memdev_state *mds) /* In case CEL was bogus, enable some default commands. */ cxl_for_each_cmd(cmd) if (cmd->flags & CXL_CMD_FLAG_FORCE_ENABLE) - set_bit(cmd->info.id, mds->enabled_cmds); + set_bit(cmd->info.id, cxl_mbox->enabled_cmds); /* Found the required CEL */ rc = 0; @@ -1448,6 +1444,7 @@ struct cxl_memdev_state *cxl_memdev_state_create(struct device *dev) mutex_init(&mds->event.log_lock); mds->cxlds.dev = dev; mds->cxlds.reg_map.host = dev; + mds->cxlds.cxl_mbox.host = dev; mds->cxlds.reg_map.resource = CXL_RESOURCE_NONE; mds->cxlds.type = CXL_DEVTYPE_CLASSMEM; mds->ram_perf.qos_class = CXL_QOS_CLASS_INVALID; diff --git a/drivers/cxl/core/memdev.c b/drivers/cxl/core/memdev.c index ae3dfcbe8938..2e2e035abdaa 100644 --- a/drivers/cxl/core/memdev.c +++ b/drivers/cxl/core/memdev.c @@ -564,9 +564,11 @@ EXPORT_SYMBOL_NS_GPL(is_cxl_memdev, "CXL"); void set_exclusive_cxl_commands(struct cxl_memdev_state *mds, unsigned long *cmds) { + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + down_write(&cxl_memdev_rwsem); - bitmap_or(mds->exclusive_cmds, mds->exclusive_cmds, cmds, - CXL_MEM_COMMAND_ID_MAX); + bitmap_or(cxl_mbox->exclusive_cmds, cxl_mbox->exclusive_cmds, + cmds, CXL_MEM_COMMAND_ID_MAX); up_write(&cxl_memdev_rwsem); } EXPORT_SYMBOL_NS_GPL(set_exclusive_cxl_commands, "CXL"); @@ -579,9 +581,11 @@ EXPORT_SYMBOL_NS_GPL(set_exclusive_cxl_commands, "CXL"); void clear_exclusive_cxl_commands(struct cxl_memdev_state *mds, unsigned long *cmds) { + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + down_write(&cxl_memdev_rwsem); - bitmap_andnot(mds->exclusive_cmds, mds->exclusive_cmds, cmds, - CXL_MEM_COMMAND_ID_MAX); + bitmap_andnot(cxl_mbox->exclusive_cmds, cxl_mbox->exclusive_cmds, + cmds, CXL_MEM_COMMAND_ID_MAX); up_write(&cxl_memdev_rwsem); } EXPORT_SYMBOL_NS_GPL(clear_exclusive_cxl_commands, "CXL"); @@ -656,11 +660,14 @@ static struct cxl_memdev *cxl_memdev_alloc(struct cxl_dev_state *cxlds, static long __cxl_memdev_ioctl(struct cxl_memdev *cxlmd, unsigned int cmd, unsigned long arg) { + struct cxl_memdev_state *mds = to_cxl_memdev_state(cxlmd->cxlds); + struct cxl_mailbox *cxl_mbox = &mds->cxlds.cxl_mbox; + switch (cmd) { case CXL_MEM_QUERY_COMMANDS: - return cxl_query_cmd(cxlmd, (void __user *)arg); + return cxl_query_cmd(cxl_mbox, (void __user *)arg); case CXL_MEM_SEND_COMMAND: - return cxl_send_cmd(cxlmd, (void __user *)arg); + return cxl_send_cmd(cxl_mbox, (void __user *)arg); default: return -ENOTTY; } @@ -994,10 +1001,11 @@ static void cxl_remove_fw_upload(void *fwl) int devm_cxl_setup_fw_upload(struct device *host, struct cxl_memdev_state *mds) { struct cxl_dev_state *cxlds = &mds->cxlds; + struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; struct device *dev = &cxlds->cxlmd->dev; struct fw_upload *fwl; - if (!test_bit(CXL_MEM_COMMAND_ID_GET_FW_INFO, mds->enabled_cmds)) + if (!test_bit(CXL_MEM_COMMAND_ID_GET_FW_INFO, cxl_mbox->enabled_cmds)) return 0; fwl = firmware_upload_register(THIS_MODULE, dev, dev_name(dev), diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 2a25d1957ddb..a0a49809cd76 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -106,42 +106,6 @@ static inline struct cxl_ep *cxl_ep_load(struct cxl_port *port, return xa_load(&port->endpoints, (unsigned long)&cxlmd->dev); } -/** - * struct cxl_mbox_cmd - A command to be submitted to hardware. - * @opcode: (input) The command set and command submitted to hardware. - * @payload_in: (input) Pointer to the input payload. - * @payload_out: (output) Pointer to the output payload. Must be allocated by - * the caller. - * @size_in: (input) Number of bytes to load from @payload_in. - * @size_out: (input) Max number of bytes loaded into @payload_out. - * (output) Number of bytes generated by the device. For fixed size - * outputs commands this is always expected to be deterministic. For - * variable sized output commands, it tells the exact number of bytes - * written. - * @min_out: (input) internal command output payload size validation - * @poll_count: (input) Number of timeouts to attempt. - * @poll_interval_ms: (input) Time between mailbox background command polling - * interval timeouts. - * @return_code: (output) Error code returned from hardware. - * - * This is the primary mechanism used to send commands to the hardware. - * All the fields except @payload_* correspond exactly to the fields described in - * Command Register section of the CXL 2.0 8.2.8.4.5. @payload_in and - * @payload_out are written to, and read from the Command Payload Registers - * defined in CXL 2.0 8.2.8.4.8. - */ -struct cxl_mbox_cmd { - u16 opcode; - void *payload_in; - void *payload_out; - size_t size_in; - size_t size_out; - size_t min_out; - int poll_count; - int poll_interval_ms; - u16 return_code; -}; - /* * Per CXL 3.0 Section 8.2.8.4.5.1 */ @@ -461,8 +425,6 @@ static inline struct cxl_dev_state *mbox_to_cxlds(struct cxl_mailbox *cxl_mbox) * @lsa_size: Size of Label Storage Area * (CXL 2.0 8.2.9.5.1.1 Identify Memory Device) * @firmware_version: Firmware version for the memory device. - * @enabled_cmds: Hardware commands found enabled in CEL. - * @exclusive_cmds: Commands that are kernel-internal only * @total_bytes: sum of all possible capacities * @volatile_only_bytes: hard volatile capacity * @persistent_only_bytes: hard persistent capacity @@ -485,8 +447,6 @@ struct cxl_memdev_state { struct cxl_dev_state cxlds; size_t lsa_size; char firmware_version[0x10]; - DECLARE_BITMAP(enabled_cmds, CXL_MEM_COMMAND_ID_MAX); - DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX); u64 total_bytes; u64 volatile_only_bytes; u64 persistent_only_bytes; diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index bacd111e75f1..cc894f07a435 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -3,12 +3,49 @@ #ifndef __CXL_MBOX_H__ #define __CXL_MBOX_H__ #include +#include -struct cxl_mbox_cmd; +/** + * struct cxl_mbox_cmd - A command to be submitted to hardware. + * @opcode: (input) The command set and command submitted to hardware. + * @payload_in: (input) Pointer to the input payload. + * @payload_out: (output) Pointer to the output payload. Must be allocated by + * the caller. + * @size_in: (input) Number of bytes to load from @payload_in. + * @size_out: (input) Max number of bytes loaded into @payload_out. + * (output) Number of bytes generated by the device. For fixed size + * outputs commands this is always expected to be deterministic. For + * variable sized output commands, it tells the exact number of bytes + * written. + * @min_out: (input) internal command output payload size validation + * @poll_count: (input) Number of timeouts to attempt. + * @poll_interval_ms: (input) Time between mailbox background command polling + * interval timeouts. + * @return_code: (output) Error code returned from hardware. + * + * This is the primary mechanism used to send commands to the hardware. + * All the fields except @payload_* correspond exactly to the fields described in + * Command Register section of the CXL 2.0 8.2.8.4.5. @payload_in and + * @payload_out are written to, and read from the Command Payload Registers + * defined in CXL 2.0 8.2.8.4.8. + */ +struct cxl_mbox_cmd { + u16 opcode; + void *payload_in; + void *payload_out; + size_t size_in; + size_t size_out; + size_t min_out; + int poll_count; + int poll_interval_ms; + u16 return_code; +}; /** * struct cxl_mailbox - context for CXL mailbox operations * @host: device that hosts the mailbox + * @enabled_cmds: mailbox commands that are enabled by the driver + * @exclusive_cmds: mailbox commands that are exclusive to the kernel * @payload_size: Size of space for payload * (CXL 3.1 8.2.8.4.3 Mailbox Capabilities Register) * @mbox_mutex: mutex protects device mailbox and firmware @@ -17,6 +54,8 @@ struct cxl_mbox_cmd; */ struct cxl_mailbox { struct device *host; + DECLARE_BITMAP(enabled_cmds, CXL_MEM_COMMAND_ID_MAX); + DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX); size_t payload_size; struct mutex mbox_mutex; /* lock to protect mailbox context */ struct rcuwait mbox_wait; From patchwork Wed Jan 22 23:50:33 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947711 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 983F31CAA98 for ; Wed, 22 Jan 2025 23:52:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589925; cv=none; b=XHJYE6bSxVjBiBjlU+VdOk/4h2DxfZtGPNHpVRYlpvQP3/KE+l/hHqblxD5v5nLjf0aWB3uPzBkd+7ewzb9GQZN/tbnColqCW3+OidSxhSyrW0h4MdsfuEVvwnfuVfDIHZZO0Npo8Tt0J2Afqviicu2J2VYjYeqWXL0rLZIUR/M= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589925; c=relaxed/simple; bh=EV39FRmte87bFtxXj+S/b5LWCv8dzhXqKmB+NThGIdI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qBVgCrY4GpfqwcOlc3N6xr6me560v9R7t6oNKa6XndREUGa9Rw05t1Ojfib0O15tzom5JQWNJ9hpsSOc9mnALQQw5e4Kly4sdEaCAU7fTUTYUy2JQRsXA7uWWcAI6lKNfXMABdpfntVyvEbNiOkKAc86gxuB21eDiY13mF6uqqM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id E37F9C4CEE4; Wed, 22 Jan 2025 23:52:04 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 02/19] cxl: Add skeletal features driver Date: Wed, 22 Jan 2025 16:50:33 -0700 Message-ID: <20250122235159.2716036-3-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add the basic bits of a features driver to handle all CXL feature related services. The driver is expected to handle all CXL mailbox feature command related operations. Suggested-by: Dan Williams Signed-off-by: Dave Jiang --- drivers/cxl/Kconfig | 11 +++++ drivers/cxl/Makefile | 3 ++ drivers/cxl/core/Makefile | 1 + drivers/cxl/core/core.h | 1 + drivers/cxl/core/features.c | 71 +++++++++++++++++++++++++++ drivers/cxl/core/port.c | 3 ++ drivers/cxl/cxl.h | 1 + drivers/cxl/features.c | 44 +++++++++++++++++ drivers/cxl/pci.c | 19 +++++++ include/cxl/features.h | 23 +++++++++ include/cxl/mailbox.h | 3 ++ tools/testing/cxl/Kbuild | 7 +++ tools/testing/cxl/cxl_features_test.c | 6 +++ 13 files changed, 193 insertions(+) create mode 100644 drivers/cxl/core/features.c create mode 100644 drivers/cxl/features.c create mode 100644 include/cxl/features.h create mode 100644 tools/testing/cxl/cxl_features_test.c diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 876469e23f7a..9a6ffd81ac0e 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -146,4 +146,15 @@ config CXL_REGION_INVALIDATION_TEST If unsure, or if this kernel is meant for production environments, say N. +config CXL_FEATURES + tristate "CXL: Features support" + default CXL_BUS + help + Enable CXL features support that are tied to a CXL mailbox. + The support for features including the feature mailbox + commands and also FWCTL support of the commands via user + space. + + If unsure say 'y'. + endif diff --git a/drivers/cxl/Makefile b/drivers/cxl/Makefile index 2caa90fa4bf2..4696fc218df4 100644 --- a/drivers/cxl/Makefile +++ b/drivers/cxl/Makefile @@ -7,15 +7,18 @@ # - 'mem' and 'pmem' before endpoint drivers so that memdevs are # immediately enabled # - 'pci' last, also mirrors the hardware enumeration hierarchy +# - 'features' comes after pci device is enumerated obj-y += core/ obj-$(CONFIG_CXL_PORT) += cxl_port.o obj-$(CONFIG_CXL_ACPI) += cxl_acpi.o obj-$(CONFIG_CXL_PMEM) += cxl_pmem.o obj-$(CONFIG_CXL_MEM) += cxl_mem.o obj-$(CONFIG_CXL_PCI) += cxl_pci.o +obj-$(CONFIG_CXL_FEATURES) += cxl_features.o cxl_port-y := port.o cxl_acpi-y := acpi.o cxl_pmem-y := pmem.o security.o cxl_mem-y := mem.o cxl_pci-y := pci.o +cxl_features-y := features.o diff --git a/drivers/cxl/core/Makefile b/drivers/cxl/core/Makefile index 9259bcc6773c..73b6348afd67 100644 --- a/drivers/cxl/core/Makefile +++ b/drivers/cxl/core/Makefile @@ -14,5 +14,6 @@ cxl_core-y += pci.o cxl_core-y += hdm.o cxl_core-y += pmu.o cxl_core-y += cdat.o +cxl_core-y += features.o cxl_core-$(CONFIG_TRACING) += trace.o cxl_core-$(CONFIG_CXL_REGION) += region.o diff --git a/drivers/cxl/core/core.h b/drivers/cxl/core/core.h index 23761340e65c..e8a3df226643 100644 --- a/drivers/cxl/core/core.h +++ b/drivers/cxl/core/core.h @@ -9,6 +9,7 @@ extern const struct device_type cxl_nvdimm_bridge_type; extern const struct device_type cxl_nvdimm_type; extern const struct device_type cxl_pmu_type; +extern const struct device_type cxl_features_type; extern struct attribute_group cxl_base_attribute_group; diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c new file mode 100644 index 000000000000..eb6eb191a32e --- /dev/null +++ b/drivers/cxl/core/features.c @@ -0,0 +1,71 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2024-2025 Intel Corporation. All rights reserved. */ +#include +#include "cxl.h" +#include "core.h" + +#define CXL_FEATURE_MAX_DEVS 65536 +static DEFINE_IDA(cxl_features_ida); + +static void cxl_features_release(struct device *dev) +{ + struct cxl_features *features = to_cxl_features(dev); + + ida_free(&cxl_features_ida, features->id); + kfree(features); +} + +static void remove_features_dev(void *dev) +{ + device_unregister(dev); +} + +const struct device_type cxl_features_type = { + .name = "features", + .release = cxl_features_release, +}; +EXPORT_SYMBOL_NS_GPL(cxl_features_type, "CXL"); + +struct cxl_features *cxl_features_alloc(struct cxl_mailbox *cxl_mbox, + struct device *parent) +{ + struct device *dev; + int rc; + + struct cxl_features *features __free(kfree) = + kzalloc(sizeof(*features), GFP_KERNEL); + if (!features) + return ERR_PTR(-ENOMEM); + + rc = ida_alloc_max(&cxl_features_ida, CXL_FEATURE_MAX_DEVS - 1, + GFP_KERNEL); + if (rc < 0) + return ERR_PTR(rc); + + features->id = rc; + features->cxl_mbox = cxl_mbox; + dev = &features->dev; + device_initialize(dev); + device_set_pm_not_required(dev); + dev->parent = parent; + dev->bus = &cxl_bus_type; + dev->type = &cxl_features_type; + rc = dev_set_name(dev, "features%d", features->id); + if (rc) + goto err; + + rc = device_add(dev); + if (rc) + goto err; + + rc = devm_add_action_or_reset(parent, remove_features_dev, dev); + if (rc) + goto err; + + return no_free_ptr(features); + +err: + put_device(dev); + return ERR_PTR(rc); +} +EXPORT_SYMBOL_NS_GPL(cxl_features_alloc, "CXL"); diff --git a/drivers/cxl/core/port.c b/drivers/cxl/core/port.c index 78a5c2c25982..cc53a597cae6 100644 --- a/drivers/cxl/core/port.c +++ b/drivers/cxl/core/port.c @@ -74,6 +74,9 @@ static int cxl_device_id(const struct device *dev) return CXL_DEVICE_REGION; if (dev->type == &cxl_pmu_type) return CXL_DEVICE_PMU; + if (dev->type == &cxl_features_type) + return CXL_DEVICE_FEATURES; + return 0; } diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index f6015f24ad38..ee29d1a1c8df 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -855,6 +855,7 @@ void cxl_driver_unregister(struct cxl_driver *cxl_drv); #define CXL_DEVICE_PMEM_REGION 7 #define CXL_DEVICE_DAX_REGION 8 #define CXL_DEVICE_PMU 9 +#define CXL_DEVICE_FEATURES 10 #define MODULE_ALIAS_CXL(type) MODULE_ALIAS("cxl:t" __stringify(type) "*") #define CXL_MODALIAS_FMT "cxl:t%d" diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c new file mode 100644 index 000000000000..644add26975f --- /dev/null +++ b/drivers/cxl/features.c @@ -0,0 +1,44 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* Copyright(c) 2024,2025 Intel Corporation. All rights reserved. */ +#include +#include +#include +#include + +#include "cxl.h" + +static int cxl_features_probe(struct device *dev) +{ + struct cxl_features *features = to_cxl_features(dev); + struct cxl_features_state *cfs __free(kfree) = + kzalloc(sizeof(*cfs), GFP_KERNEL); + + if (!cfs) + return -ENOMEM; + + cfs->features = features; + dev_set_drvdata(dev, no_free_ptr(cfs)); + + return 0; +} + +static void cxl_features_remove(struct device *dev) +{ + struct cxl_features_state *cfs = dev_get_drvdata(dev); + + kfree(cfs); +} + +static struct cxl_driver cxl_features_driver = { + .name = "cxl_features", + .probe = cxl_features_probe, + .remove = cxl_features_remove, + .id = CXL_DEVICE_FEATURES, +}; + +module_cxl_driver(cxl_features_driver); + +MODULE_DESCRIPTION("CXL: Features"); +MODULE_LICENSE("GPL"); +MODULE_IMPORT_NS("CXL"); +MODULE_ALIAS_CXL(CXL_DEVICE_FEATURES); diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c index 6d94ff4a4f1a..eb68dd3f8b21 100644 --- a/drivers/cxl/pci.c +++ b/drivers/cxl/pci.c @@ -386,6 +386,21 @@ static int cxl_pci_mbox_send(struct cxl_mailbox *cxl_mbox, return rc; } +static int cxl_pci_setup_features(struct cxl_memdev_state *mds) +{ + struct cxl_dev_state *cxlds = &mds->cxlds; + struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; + struct cxl_features *features; + + features = cxl_features_alloc(cxl_mbox, cxlds->dev); + if (IS_ERR(features)) + return PTR_ERR(features); + + cxl_mbox->features = features; + + return 0; +} + static int cxl_pci_setup_mailbox(struct cxl_memdev_state *mds, bool irq_avail) { struct cxl_dev_state *cxlds = &mds->cxlds; @@ -980,6 +995,10 @@ static int cxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id) if (rc) return rc; + rc = cxl_pci_setup_features(mds); + if (rc) + return rc; + rc = cxl_set_timestamp(mds); if (rc) return rc; diff --git a/include/cxl/features.h b/include/cxl/features.h new file mode 100644 index 000000000000..b92da1e92780 --- /dev/null +++ b/include/cxl/features.h @@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ +/* Copyright(c) 2024-2025 Intel Corporation. */ +#ifndef __CXL_FEATURES_H__ +#define __CXL_FEATURES_H__ + +struct cxl_mailbox; + +struct cxl_features { + int id; + struct device dev; + struct cxl_mailbox *cxl_mbox; +}; +#define to_cxl_features(dev) container_of(dev, struct cxl_features, dev) + +struct cxl_features_state { + struct cxl_features *features; + int num_features; +}; + +struct cxl_features *cxl_features_alloc(struct cxl_mailbox *cxl_mbox, + struct device *parent); + +#endif diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index cc894f07a435..6caab0d406ba 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -3,6 +3,7 @@ #ifndef __CXL_MBOX_H__ #define __CXL_MBOX_H__ #include +#include #include /** @@ -50,6 +51,7 @@ struct cxl_mbox_cmd { * (CXL 3.1 8.2.8.4.3 Mailbox Capabilities Register) * @mbox_mutex: mutex protects device mailbox and firmware * @mbox_wait: rcuwait for mailbox + * @features: pointer to cxl_features device * @mbox_send: @dev specific transport for transmitting mailbox commands */ struct cxl_mailbox { @@ -59,6 +61,7 @@ struct cxl_mailbox { size_t payload_size; struct mutex mbox_mutex; /* lock to protect mailbox context */ struct rcuwait mbox_wait; + struct cxl_features *features; int (*mbox_send)(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd); }; diff --git a/tools/testing/cxl/Kbuild b/tools/testing/cxl/Kbuild index b1256fee3567..8b62341bb5df 100644 --- a/tools/testing/cxl/Kbuild +++ b/tools/testing/cxl/Kbuild @@ -50,6 +50,12 @@ cxl_mem-y := $(CXL_SRC)/mem.o cxl_mem-y += config_check.o cxl_mem-y += cxl_mem_test.o +obj-m += cxl_features.o + +cxl_features-y := $(CXL_SRC)/features.o +cxl_features-y += config_check.o +cxl_features-y += cxl_features_test.o + obj-m += cxl_core.o cxl_core-y := $(CXL_CORE_SRC)/port.o @@ -61,6 +67,7 @@ cxl_core-y += $(CXL_CORE_SRC)/pci.o cxl_core-y += $(CXL_CORE_SRC)/hdm.o cxl_core-y += $(CXL_CORE_SRC)/pmu.o cxl_core-y += $(CXL_CORE_SRC)/cdat.o +cxl_core-y += $(CXL_CORE_SRC)/features.o cxl_core-$(CONFIG_TRACING) += $(CXL_CORE_SRC)/trace.o cxl_core-$(CONFIG_CXL_REGION) += $(CXL_CORE_SRC)/region.o cxl_core-y += config_check.o diff --git a/tools/testing/cxl/cxl_features_test.c b/tools/testing/cxl/cxl_features_test.c new file mode 100644 index 000000000000..b117977b5ab2 --- /dev/null +++ b/tools/testing/cxl/cxl_features_test.c @@ -0,0 +1,6 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2025 Intel Corporation. All rights reserved. */ + +#include "watermark.h" + +cxl_test_watermark(cxl_features); From patchwork Wed Jan 22 23:50:34 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947712 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 955EC1CDA3F for ; Wed, 22 Jan 2025 23:52:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589926; cv=none; b=kNoS7DDF3F2v906/sSUDywOrNuk1sL47SjMKA1oKZEC3dEpTiyAwOje5+ig1us1IoLjrllPaL7tKQXseck+yg1frR4owCNIe1Z1W5+feUTHL7yuQOmrQItBonzPrh+PnBrLcAOcQ3dEPRd71PdJ0r3bY5CDDtTtUWgT2L2ef+w0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589926; c=relaxed/simple; bh=Pm2JxZLpVDeZ1ooGqG9Ivm6dKr1gi6blJJW4Z7deD1A=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EYvj+PxAsEyl1Ks58jN8TIbM9w8dFPv/tI2fyzG5wOZ+HfOVcpgyXmD6Tcl9q+LvwMo+mEQrsAQnSenFB0k3FTDwr5a0zihEwh0FKzinMc+uDH5ya+7xESCeSKSK2BkBc97Wt9lKTiovqx97CBUAFNucQJEy3g302hD4dwpNaVc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5043FC4CED2; Wed, 22 Jan 2025 23:52:06 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 03/19] cxl: Enumerate feature commands Date: Wed, 22 Jan 2025 16:50:34 -0700 Message-ID: <20250122235159.2716036-4-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add feature commands enumeration code in order to detect and enumerate the 3 feature related commands "get supported features", "get feature", and "set feature". The enumeration will help determine whether the driver can issue any of the 3 commands to the device. Signed-off-by: Dave Jiang --- drivers/cxl/core/mbox.c | 41 +++++++++++++++++++++++++++++++++++++++++ drivers/cxl/cxlmem.h | 3 +++ include/cxl/features.h | 7 +++++++ include/cxl/mailbox.h | 1 + 4 files changed, 52 insertions(+) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index bdb8f060f2c1..5e21ff99d70f 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -38,6 +38,21 @@ static bool cxl_raw_allow_all; .flags = _flags, \ } +#define cxl_for_each_feature_cmd(cmd) \ + for ((cmd) = &cxl_feature_commands[0]; \ + ((cmd) - cxl_feature_commands) < ARRAY_SIZE(cxl_feature_commands); (cmd)++) + +#define CXL_FEATURE_CMD(_id, sin, sout, _flags) \ + [CXL_FEATURE_ID_##_id] = { \ + .info = { \ + .id = CXL_FEATURE_ID_##_id, \ + .size_in = sin, \ + .size_out = sout, \ + }, \ + .opcode = CXL_MBOX_OP_##_id, \ + .flags = _flags, \ + } + #define CXL_VARIABLE_PAYLOAD ~0U /* * This table defines the supported mailbox commands for the driver. This table @@ -69,6 +84,13 @@ static struct cxl_mem_command cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = { CXL_CMD(GET_TIMESTAMP, 0, 0x8, 0), }; +#define CXL_FEATURE_COMMAND_ID_MAX 3 +static struct cxl_mem_command cxl_feature_commands[CXL_FEATURE_COMMAND_ID_MAX] = { + CXL_FEATURE_CMD(GET_SUPPORTED_FEATURES, 0x8, CXL_VARIABLE_PAYLOAD, 0), + CXL_FEATURE_CMD(GET_FEATURE, 0xf, CXL_VARIABLE_PAYLOAD, 0), + CXL_FEATURE_CMD(SET_FEATURE, CXL_VARIABLE_PAYLOAD, 0, 0), +}; + /* * Commands that RAW doesn't permit. The rationale for each: * @@ -212,6 +234,17 @@ static struct cxl_mem_command *cxl_mem_find_command(u16 opcode) return NULL; } +static struct cxl_mem_command *cxl_find_feature_command(u16 opcode) +{ + struct cxl_mem_command *c; + + cxl_for_each_feature_cmd(c) + if (c->opcode == opcode) + return c; + + return NULL; +} + static const char *cxl_mem_opcode_to_name(u16 opcode) { struct cxl_mem_command *c; @@ -734,6 +767,14 @@ static void cxl_walk_cel(struct cxl_memdev_state *mds, size_t size, u8 *cel) if (cmd) { set_bit(cmd->info.id, cxl_mbox->enabled_cmds); enabled++; + } else { + struct cxl_mem_command *fcmd = + cxl_find_feature_command(opcode); + + if (fcmd) { + set_bit(fcmd->info.id, cxl_mbox->feature_cmds); + enabled++; + } } if (cxl_is_poison_command(opcode)) { diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index a0a49809cd76..55c55685cb39 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -490,6 +490,9 @@ enum cxl_opcode { CXL_MBOX_OP_GET_LOG_CAPS = 0x0402, CXL_MBOX_OP_CLEAR_LOG = 0x0403, CXL_MBOX_OP_GET_SUP_LOG_SUBLIST = 0x0405, + CXL_MBOX_OP_GET_SUPPORTED_FEATURES = 0x0500, + CXL_MBOX_OP_GET_FEATURE = 0x0501, + CXL_MBOX_OP_SET_FEATURE = 0x0502, CXL_MBOX_OP_IDENTIFY = 0x4000, CXL_MBOX_OP_GET_PARTITION_INFO = 0x4100, CXL_MBOX_OP_SET_PARTITION_INFO = 0x4101, diff --git a/include/cxl/features.h b/include/cxl/features.h index b92da1e92780..7a8be3c621a1 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -5,6 +5,13 @@ struct cxl_mailbox; +enum feature_cmds { + CXL_FEATURE_ID_GET_SUPPORTED_FEATURES = 0, + CXL_FEATURE_ID_GET_FEATURE, + CXL_FEATURE_ID_SET_FEATURE, + CXL_FEATURE_ID_MAX, +}; + struct cxl_features { int id; struct device dev; diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index 6caab0d406ba..263fc346aeb1 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -58,6 +58,7 @@ struct cxl_mailbox { struct device *host; DECLARE_BITMAP(enabled_cmds, CXL_MEM_COMMAND_ID_MAX); DECLARE_BITMAP(exclusive_cmds, CXL_MEM_COMMAND_ID_MAX); + DECLARE_BITMAP(feature_cmds, CXL_FEATURE_ID_MAX); size_t payload_size; struct mutex mbox_mutex; /* lock to protect mailbox context */ struct rcuwait mbox_wait; From patchwork Wed Jan 22 23:50:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947713 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 561AA1CAA98 for ; Wed, 22 Jan 2025 23:52:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589928; cv=none; b=vCiyPIt11YQGQGN1sjyf0cgtYpSc2i0d2Di3E/mqrw99Rp/kKl++z4ul3uzWfLe8D2xOBp91yVSG0YxsKCtR/NCDrA7UYSIoJ8yT4I0MMlqQI104JDyPcSXTuj6wdLHgxHlOT/zdM+MvGtJY2f8Tn0+UF15R3Aroj5Zwiz73R40= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589928; c=relaxed/simple; bh=pw6f0QVtZSevOI6RSE7JaKpmoyG4cFUt5Cs1EdxSu3Q=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=KzyseCGUxo5L4wmgPnu5b7k6AMjvLbPGJkRAS9Jt0rgH6XHEBsuolHm1j1qAoRSH7D1KKJ83gN+BmE8UNfLnJ/3TvAsq6CYHZCPdT4MIMuYwaK3PQYUWbhOpdOgn53RuW3WmKrW+YhhCyaZN24fo9OwE3j8+2tsluWX7V7/AhyY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id AFD7FC4CED2; Wed, 22 Jan 2025 23:52:07 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 04/19] cxl: Add Get Supported Features command for kernel usage Date: Wed, 22 Jan 2025 16:50:35 -0700 Message-ID: <20250122235159.2716036-5-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 CXL spec r3.1 8.2.9.6.1 Get Supported Features (Opcode 0500h) The command retrieve the list of supported device-specific features (identified by UUID) and general information about each Feature. The driver will retrieve the feature entries in order to make checks and provide information for the Get Feature and Set Feature command. One of the main piece of information retrieved are the effects a Set Feature command would have for a particular feature. Co-developed-by: Shiju Jose Signed-off-by: Shiju Jose Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- v1: - Change input param from cxlds to cxl_mbox - Move mbox_out declaration inline. (Jonathan) - Fix __counted_by() input. (Jonathan) - Return count for cxl_get_supported_features_count(). (Dan) - Remove goto from cxl_get_supported_features(). (Dan) - Return cxl_get_supported_feature_entry() directly. (Dan) - Drop user path enumeration. (Dan) - Move to the feature driver model. (Dan) - Add support for 0 feature data requested. - Add missing increment of feature data ptr during copy. --- drivers/cxl/core/features.c | 28 +++++++ drivers/cxl/core/mbox.c | 3 +- drivers/cxl/cxl.h | 2 + drivers/cxl/features.c | 146 +++++++++++++++++++++++++++++++++++- include/cxl/features.h | 32 ++++++++ 5 files changed, 209 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c index eb6eb191a32e..66a4b82910e6 100644 --- a/drivers/cxl/core/features.c +++ b/drivers/cxl/core/features.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: GPL-2.0-only /* Copyright(c) 2024-2025 Intel Corporation. All rights reserved. */ #include +#include #include "cxl.h" #include "core.h" @@ -69,3 +70,30 @@ struct cxl_features *cxl_features_alloc(struct cxl_mailbox *cxl_mbox, return ERR_PTR(rc); } EXPORT_SYMBOL_NS_GPL(cxl_features_alloc, "CXL"); + +struct cxl_feat_entry * +cxl_get_supported_feature_entry(struct cxl_features *features, + const uuid_t *feat_uuid) +{ + struct cxl_feat_entry *feat_entry; + struct cxl_features_state *cfs; + int count; + + cfs = dev_get_drvdata(&features->dev); + if (!cfs) + return ERR_PTR(-EOPNOTSUPP); + + if (!cfs->num_features) + return ERR_PTR(-ENOENT); + + /* Check CXL dev supports the feature */ + feat_entry = cfs->entries; + for (count = 0; count < cfs->num_features; + count++, feat_entry++) { + if (uuid_equal(&feat_entry->uuid, feat_uuid)) + return feat_entry; + } + + return ERR_PTR(-ENOENT); +} +EXPORT_SYMBOL_NS_GPL(cxl_get_supported_feature_entry, "CXL"); diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 5e21ff99d70f..0b4946205910 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -234,7 +234,7 @@ static struct cxl_mem_command *cxl_mem_find_command(u16 opcode) return NULL; } -static struct cxl_mem_command *cxl_find_feature_command(u16 opcode) +struct cxl_mem_command *cxl_find_feature_command(u16 opcode) { struct cxl_mem_command *c; @@ -244,6 +244,7 @@ static struct cxl_mem_command *cxl_find_feature_command(u16 opcode) return NULL; } +EXPORT_SYMBOL_NS_GPL(cxl_find_feature_command, "CXL"); static const char *cxl_mem_opcode_to_name(u16 opcode) { diff --git a/drivers/cxl/cxl.h b/drivers/cxl/cxl.h index ee29d1a1c8df..1284614d71d0 100644 --- a/drivers/cxl/cxl.h +++ b/drivers/cxl/cxl.h @@ -912,6 +912,8 @@ void cxl_coordinates_combine(struct access_coordinate *out, bool cxl_endpoint_decoder_reset_detected(struct cxl_port *port); +struct cxl_mem_command *cxl_find_feature_command(u16 opcode); + /* * Unit test builds overrides this to __weak, find the 'strong' version * of these symbols in tools/testing/cxl/. diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c index 644add26975f..a5949312a4ab 100644 --- a/drivers/cxl/features.c +++ b/drivers/cxl/features.c @@ -3,20 +3,164 @@ #include #include #include +#include #include #include "cxl.h" +#include "cxlmem.h" + +static void cxl_free_feature_entries(void *entries) +{ + kvfree(entries); +} + +static int cxl_get_supported_features_count(struct cxl_mailbox *cxl_mbox) +{ + struct cxl_mbox_get_sup_feats_out mbox_out; + struct cxl_mbox_get_sup_feats_in mbox_in; + struct cxl_mbox_cmd mbox_cmd; + int rc; + + memset(&mbox_in, 0, sizeof(mbox_in)); + mbox_in.count = cpu_to_le32(sizeof(mbox_out)); + memset(&mbox_out, 0, sizeof(mbox_out)); + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_GET_SUPPORTED_FEATURES, + .size_in = sizeof(mbox_in), + .payload_in = &mbox_in, + .size_out = sizeof(mbox_out), + .payload_out = &mbox_out, + .min_out = sizeof(mbox_out), + }; + rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); + if (rc < 0) + return rc; + + return le16_to_cpu(mbox_out.supported_feats); +} + +static int cxl_get_supported_features(struct cxl_features_state *cfs) +{ + int remain_feats, max_size, max_feats, start, rc, hdr_size; + struct cxl_mailbox *cxl_mbox = cfs->features->cxl_mbox; + int feat_size = sizeof(struct cxl_feat_entry); + struct cxl_mbox_get_sup_feats_in mbox_in; + struct cxl_feat_entry *entry; + struct cxl_mbox_cmd mbox_cmd; + struct cxl_mem_command *cmd; + int count; + + /* Get supported features is optional, need to check */ + cmd = cxl_find_feature_command(CXL_MBOX_OP_GET_SUPPORTED_FEATURES); + if (!cmd) + return -EOPNOTSUPP; + if (!test_bit(cmd->info.id, cxl_mbox->feature_cmds)) + return -EOPNOTSUPP; + + count = cxl_get_supported_features_count(cxl_mbox); + if (count == 0) + return 0; + if (count < 0) + return -ENXIO; + + struct cxl_feat_entry *entries __free(kvfree) = + kvmalloc(count * sizeof(*entries), GFP_KERNEL); + if (!entries) + return -ENOMEM; + + struct cxl_mbox_get_sup_feats_out *mbox_out __free(kvfree) = + kvmalloc(cxl_mbox->payload_size, GFP_KERNEL); + if (!mbox_out) + return -ENOMEM; + + hdr_size = sizeof(*mbox_out); + max_size = cxl_mbox->payload_size - hdr_size; + /* max feat entries that can fit in mailbox max payload size */ + max_feats = max_size / feat_size; + entry = entries; + + start = 0; + remain_feats = count; + do { + int retrieved, alloc_size, copy_feats; + int num_entries; + + if (remain_feats > max_feats) { + alloc_size = sizeof(*mbox_out) + max_feats * feat_size; + remain_feats = remain_feats - max_feats; + copy_feats = max_feats; + } else { + alloc_size = sizeof(*mbox_out) + remain_feats * feat_size; + copy_feats = remain_feats; + remain_feats = 0; + } + + memset(&mbox_in, 0, sizeof(mbox_in)); + mbox_in.count = cpu_to_le32(alloc_size); + mbox_in.start_idx = cpu_to_le16(start); + memset(mbox_out, 0, alloc_size); + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_GET_SUPPORTED_FEATURES, + .size_in = sizeof(mbox_in), + .payload_in = &mbox_in, + .size_out = alloc_size, + .payload_out = mbox_out, + .min_out = hdr_size, + }; + rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); + if (rc < 0) + return rc; + + if (mbox_cmd.size_out <= hdr_size) + return -ENXIO; + + /* + * Make sure retrieved out buffer is multiple of feature + * entries. + */ + retrieved = mbox_cmd.size_out - hdr_size; + if (retrieved % feat_size) + return -ENXIO; + + num_entries = le16_to_cpu(mbox_out->num_entries); + /* + * If the reported output entries * defined entry size != + * retrieved output bytes, then the output package is incorrect. + */ + if (num_entries * feat_size != retrieved) + return -ENXIO; + + memcpy(entry, mbox_out->ents, retrieved); + entry++; + /* + * If the number of output entries is less than expected, add the + * remaining entries to the next batch. + */ + remain_feats += copy_feats - num_entries; + start += num_entries; + } while (remain_feats); + + cfs->num_features = count; + cfs->entries = no_free_ptr(entries); + return devm_add_action_or_reset(&cfs->features->dev, + cxl_free_feature_entries, cfs->entries); +} static int cxl_features_probe(struct device *dev) { struct cxl_features *features = to_cxl_features(dev); + int rc; + struct cxl_features_state *cfs __free(kfree) = kzalloc(sizeof(*cfs), GFP_KERNEL); - if (!cfs) return -ENOMEM; cfs->features = features; + rc = cxl_get_supported_features(cfs); + if (rc) + return rc; + dev_set_drvdata(dev, no_free_ptr(cfs)); return 0; diff --git a/include/cxl/features.h b/include/cxl/features.h index 7a8be3c621a1..429b9782667c 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -3,6 +3,8 @@ #ifndef __CXL_FEATURES_H__ #define __CXL_FEATURES_H__ +#include + struct cxl_mailbox; enum feature_cmds { @@ -19,12 +21,42 @@ struct cxl_features { }; #define to_cxl_features(dev) container_of(dev, struct cxl_features, dev) +/* Get Supported Features (0x500h) CXL r3.1 8.2.9.6.1 */ +struct cxl_mbox_get_sup_feats_in { + __le32 count; + __le16 start_idx; + u8 reserved[2]; +} __packed; + +struct cxl_feat_entry { + uuid_t uuid; + __le16 id; + __le16 get_feat_size; + __le16 set_feat_size; + __le32 flags; + u8 get_feat_ver; + u8 set_feat_ver; + __le16 effects; + u8 reserved[18]; +} __packed; + +struct cxl_mbox_get_sup_feats_out { + __le16 num_entries; + __le16 supported_feats; + u8 reserved[4]; + struct cxl_feat_entry ents[] __counted_by_le(num_entries); +} __packed; + struct cxl_features_state { struct cxl_features *features; int num_features; + struct cxl_feat_entry *entries; }; struct cxl_features *cxl_features_alloc(struct cxl_mailbox *cxl_mbox, struct device *parent); +struct cxl_feat_entry * +cxl_get_supported_feature_entry(struct cxl_features *features, + const uuid_t *feat_uuid); #endif From patchwork Wed Jan 22 23:50:36 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947715 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 198721CD1EA for ; Wed, 22 Jan 2025 23:52:09 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589931; cv=none; b=q2v3vVfJ7G9nMW0MS7PV+zwVmNV1DrP9+toy2Inkulf6Yjweke6/Rzde+LaE5jCunDnwbBX/P/1s5XfD8cRYFQRM1LXyHv0MeYfQS5neabFRthd1SwDMq7mkYJmsVtxUsE5JrI86JUM0yZWnWqKxqHc4Oi13OdcA+zu/fhc2B6k= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589931; c=relaxed/simple; bh=ISTzco0bW0ZSJFe7b+RI2slRx3Zq5pCCaADI3rS07gs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=XRtqAlZPT4v3T3ygyFf+DiQhNYzxkIUiXOWhkyM4tNIeUaBVkVoJfdzmGpJ5epC8rGOdvjzSJhklSXAG8K6ZDTpR7rTKXaHvCyKq+s25QVfJG7RYU1avSXqJ7eJEbYExjpFRuSgn4H2qJlHbFKdHvrYdsfS6FOO5WpjGmxwP8wY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0F335C4CED2; Wed, 22 Jan 2025 23:52:09 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 05/19] cxl: Add features driver attribute to emit number of features supported Date: Wed, 22 Jan 2025 16:50:36 -0700 Message-ID: <20250122235159.2716036-6-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Enable sysfs attribute emission of the number of features supported by the driver/device. This is useful for userspace to determine the number of features to query for. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- drivers/cxl/features.c | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c index a5949312a4ab..0d1fc3da9e35 100644 --- a/drivers/cxl/features.c +++ b/drivers/cxl/features.c @@ -173,11 +173,38 @@ static void cxl_features_remove(struct device *dev) kfree(cfs); } +static ssize_t features_show(struct device *dev, struct device_attribute *attr, + char *buf) +{ + struct cxl_features_state *cfs = dev_get_drvdata(dev); + + if (!cfs) + return -ENOENT; + + return sysfs_emit(buf, "%d\n", cfs->num_features); +} + +static DEVICE_ATTR_RO(features); + +static struct attribute *cxl_features_attrs[] = { + &dev_attr_features.attr, + NULL +}; + +static struct attribute_group cxl_features_group = { + .attrs = cxl_features_attrs, +}; + +__ATTRIBUTE_GROUPS(cxl_features); + static struct cxl_driver cxl_features_driver = { .name = "cxl_features", .probe = cxl_features_probe, .remove = cxl_features_remove, .id = CXL_DEVICE_FEATURES, + .drv = { + .dev_groups = cxl_features_groups, + }, }; module_cxl_driver(cxl_features_driver); From patchwork Wed Jan 22 23:50:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947714 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 198181CBE96 for ; Wed, 22 Jan 2025 23:52:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589931; cv=none; b=BjM+CqBG8mpJ98MfG2auF+95nMtGB+LB3I1U94hHRUQlcgFhqSEjEr7k6LY0MOCQGNKssN28bU/wbZywCxI4YfgcmKANE3jnC9PLKftX9oH9SgpOMdsj9JJE8ER0NXcK04w4W1LhO90ttmowRN99V3AvtYr/gm9jcaGn4X3JO2Q= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589931; c=relaxed/simple; bh=XlrVulVRU1y5mors2YYUBLELyZs4OSL9ZK3PT0NpI40=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=aa1L0NWfwqS/JcJP3R9nJcgA6lOwWNYPfK7TvCruOgzGJMAN8eeJ7ZzqiQVscuZbfBxuDp8yofLAomOzbZMgcrwLmjjfpAXd+ierKclmOynGTJV3JwyCS+Tae9oN7TW3DXnZSE2N3g3aASDs9Ghfx50Oh1MWEoKZyZp5hkXXACc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 70904C4CED6; Wed, 22 Jan 2025 23:52:10 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 06/19] cxl/test: Add Get Supported Features mailbox command support Date: Wed, 22 Jan 2025 16:50:37 -0700 Message-ID: <20250122235159.2716036-7-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add cxl-test emulation of Get Supported Features mailbox command. Currently only adding a test feature with feature identifier of all f's for testing. Reviewed-by: Jonathan Cameron Acked-by: Dan Williams Signed-off-by: Dave Jiang Reviewed-by: Dan Williams --- v1: - Fix setting of num_entries endieness issue. (Jonathan) --- include/cxl/features.h | 23 ++++++++++ tools/testing/cxl/test/mem.c | 87 ++++++++++++++++++++++++++++++++++++ 2 files changed, 110 insertions(+) diff --git a/include/cxl/features.h b/include/cxl/features.h index 429b9782667c..8ff7d90932d6 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -28,6 +28,29 @@ struct cxl_mbox_get_sup_feats_in { u8 reserved[2]; } __packed; +/* CXL spec r3.2 Table 8-87 command effects */ +#define CXL_CMD_CONFIG_CHANGE_COLD_RESET BIT(0) +#define CXL_CMD_CONFIG_CHANGE_IMMEDIATE BIT(1) +#define CXL_CMD_DATA_CHANGE_IMMEDIATE BIT(2) +#define CXL_CMD_POLICY_CHANGE_IMMEDIATE BIT(3) +#define CXL_CMD_LOG_CHANGE_IMMEDIATE BIT(4) +#define CXL_CMD_SECURITY_STATE_CHANGE BIT(5) +#define CXL_CMD_BACKGROUND BIT(6) +#define CXL_CMD_BGCMD_ABORT_SUPPORTED BIT(7) +#define CXL_CMD_EFFECTS_EXTEND BIT(9) +#define CXL_CMD_CONFIG_CHANGE_CONV_RESET BIT(10) +#define CXL_CMD_CONFIG_CHANGE_CXL_RESET BIT(11) + +/* + * CXL spec r3.2 Table 8-109 + * Get Supported Features Supported Feature Entry + * Atribute Flags + */ +#define CXL_FEATURE_F_CHANGEABLE BIT(0) +#define CXL_FEATURE_F_PERSIST_FW_UPDATE BIT(4) +#define CXL_FEATURE_F_DEFAULT_SEL BIT(5) +#define CXL_FEATURE_F_SAVED_SEL BIT(6) + struct cxl_feat_entry { uuid_t uuid; __le16 id; diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 347c1e7b37bd..208337432bfe 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -44,6 +44,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_SUPPORTED_LOGS), .effect = CXL_CMD_EFFECT_NONE, }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_GET_SUPPORTED_FEATURES), + .effect = CXL_CMD_EFFECT_NONE, + }, { .opcode = cpu_to_le16(CXL_MBOX_OP_IDENTIFY), .effect = CXL_CMD_EFFECT_NONE, @@ -1337,6 +1341,67 @@ static int mock_activate_fw(struct cxl_mockmem_data *mdata, return -EINVAL; } +#define CXL_VENDOR_FEATURE_TEST \ + UUID_INIT(0xffffffff, 0xffff, 0xffff, 0xff, 0xff, 0xff, 0xff, 0xff, \ + 0xff, 0xff, 0xff) + +static void fill_feature_vendor_test(struct cxl_feat_entry *feat) +{ + feat->uuid = CXL_VENDOR_FEATURE_TEST; + feat->id = 0; + feat->get_feat_size = cpu_to_le16(0x4); + feat->set_feat_size = cpu_to_le16(0x4); + feat->flags = cpu_to_le32(CXL_FEATURE_F_CHANGEABLE | + CXL_FEATURE_F_DEFAULT_SEL | + CXL_FEATURE_F_SAVED_SEL); + feat->get_feat_ver = 1; + feat->set_feat_ver = 1; + feat->effects = cpu_to_le16(CXL_CMD_CONFIG_CHANGE_COLD_RESET | + CXL_CMD_EFFECTS_EXTEND); +} + +static int mock_get_supported_features(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_get_sup_feats_in *in = cmd->payload_in; + struct cxl_mbox_get_sup_feats_out *out = cmd->payload_out; + struct cxl_feat_entry *feat; + u16 start_idx, count; + + if (cmd->size_out < sizeof(*out)) { + cmd->return_code = CXL_MBOX_CMD_RC_PAYLOADLEN; + return -EINVAL; + } + + /* + * Current emulation only supports 1 feature + */ + start_idx = le16_to_cpu(in->start_idx); + if (start_idx != 0) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + count = le16_to_cpu(in->count); + if (count < sizeof(*out)) { + cmd->return_code = CXL_MBOX_CMD_RC_PAYLOADLEN; + return -EINVAL; + } + + out->supported_feats = cpu_to_le16(1); + cmd->return_code = 0; + if (count < sizeof(*out) + sizeof(*feat)) { + out->num_entries = 0; + return 0; + } + + out->num_entries = cpu_to_le16(1); + feat = out->ents; + fill_feature_vendor_test(feat); + + return 0; +} + static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, struct cxl_mbox_cmd *cmd) { @@ -1422,6 +1487,9 @@ static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, case CXL_MBOX_OP_ACTIVATE_FW: rc = mock_activate_fw(mdata, cmd); break; + case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: + rc = mock_get_supported_features(mdata, cmd); + break; default: break; } @@ -1469,6 +1537,21 @@ static int cxl_mock_mailbox_create(struct cxl_dev_state *cxlds) return 0; } +static int cxl_mock_setup_features(struct cxl_memdev_state *mds) +{ + struct cxl_dev_state *cxlds = &mds->cxlds; + struct cxl_mailbox *cxl_mbox = &cxlds->cxl_mbox; + struct cxl_features *features; + + features = cxl_features_alloc(cxl_mbox, cxlds->dev); + if (IS_ERR(features)) + return PTR_ERR(features); + + cxl_mbox->features = features; + + return 0; +} + static int cxl_mock_mem_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -1524,6 +1607,10 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) if (rc) return rc; + rc = cxl_mock_setup_features(mds); + if (rc) + dev_dbg(dev, "cxl_test: No features setup\n"); + rc = cxl_poison_state_init(mds); if (rc) return rc; From patchwork Wed Jan 22 23:50:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947716 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 99C561CD1EA for ; Wed, 22 Jan 2025 23:52:12 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589932; cv=none; b=QB9ICgoKKcxMeT3HSIMM/i8WIDlm942fatqOW5NcSIXXoPtwO66HSuX8W/UQzu+z/SHMcXaWSQVZGrD9aByBdoRrmm1UCTRLQmN5mr3xCgvzpkdXyFL4ptIEToLCLWnejL+sa2TWl0CZB27JoHas/wYJFI0Ai6/h/1sTrOuEVpw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589932; c=relaxed/simple; bh=e6XFUgkrU2imjzu+WJjfu5d7Yp1JoVBxvZLn0kKAjAk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EB12R3JjxZrVU3/uQV9XmVlmu4zn1ozaasSuBRLc5k/YMUpIYDg1RqFFj/dCVsLHTYgu2Gsesj4ALuDnxdMKzMxuoj5blw8z3aHNVs3MHob+JD8Y77tUDDq/RAJzJOQnFReQLe6sRI//ySkCpOiIOEgaQlKWpHfI+PxjYDRzV24= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id D0649C4CED6; Wed, 22 Jan 2025 23:52:11 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 07/19] cxl/mbox: Add GET_FEATURE mailbox command Date: Wed, 22 Jan 2025 16:50:38 -0700 Message-ID: <20250122235159.2716036-8-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Shiju Jose Add support for GET_FEATURE mailbox command. CXL spec 3.1 section 8.2.9.6 describes optional device specific features. The settings of a feature can be retrieved using Get Feature command. CXL spec 3.1 section 8.2.9.6.2 describes Get Feature command. Signed-off-by: Shiju Jose Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- v1: - pass in cxl_mbox instead of cxlds (Dan) - Move to the feature driver model. (Dan) --- drivers/cxl/core/features.c | 74 +++++++++++++++++++++++++++++++++++++ drivers/cxl/features.c | 6 +-- include/cxl/features.h | 27 ++++++++++++++ 3 files changed, 102 insertions(+), 5 deletions(-) diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c index 66a4b82910e6..ab9386b53a95 100644 --- a/drivers/cxl/core/features.c +++ b/drivers/cxl/core/features.c @@ -4,6 +4,7 @@ #include #include "cxl.h" #include "core.h" +#include "cxlmem.h" #define CXL_FEATURE_MAX_DEVS 65536 static DEFINE_IDA(cxl_features_ida); @@ -97,3 +98,76 @@ cxl_get_supported_feature_entry(struct cxl_features *features, return ERR_PTR(-ENOENT); } EXPORT_SYMBOL_NS_GPL(cxl_get_supported_feature_entry, "CXL"); + +bool cxl_feature_enabled(struct cxl_features_state *cfs, u16 opcode) +{ + struct cxl_mailbox *cxl_mbox = cfs->features->cxl_mbox; + struct cxl_mem_command *cmd; + + cmd = cxl_find_feature_command(opcode); + if (!cmd) + return false; + + return test_bit(cmd->info.id, cxl_mbox->feature_cmds); +} +EXPORT_SYMBOL_NS_GPL(cxl_feature_enabled, "CXL"); + +size_t cxl_get_feature(struct cxl_features *features, const uuid_t feat_uuid, + enum cxl_get_feat_selection selection, + void *feat_out, size_t feat_out_size, u16 offset, + u16 *return_code) +{ + size_t data_to_rd_size, size_out; + struct cxl_features_state *cfs; + struct cxl_mbox_get_feat_in pi; + struct cxl_mailbox *cxl_mbox; + struct cxl_mbox_cmd mbox_cmd; + size_t data_rcvd_size = 0; + int rc; + + if (return_code) + *return_code = CXL_MBOX_CMD_RC_INPUT; + + cfs = dev_get_drvdata(&features->dev); + if (!cfs) + return 0; + + if (!cxl_feature_enabled(cfs, CXL_MBOX_OP_GET_FEATURE)) + return 0; + + if (!feat_out || !feat_out_size) + return 0; + + cxl_mbox = features->cxl_mbox; + size_out = min(feat_out_size, cxl_mbox->payload_size); + pi.uuid = feat_uuid; + pi.selection = selection; + do { + data_to_rd_size = min(feat_out_size - data_rcvd_size, + cxl_mbox->payload_size); + pi.offset = cpu_to_le16(offset + data_rcvd_size); + pi.count = cpu_to_le16(data_to_rd_size); + + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_GET_FEATURE, + .size_in = sizeof(pi), + .payload_in = &pi, + .size_out = size_out, + .payload_out = feat_out + data_rcvd_size, + .min_out = data_to_rd_size, + }; + rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); + if (rc < 0 || !mbox_cmd.size_out) { + if (return_code) + *return_code = mbox_cmd.return_code; + return 0; + } + data_rcvd_size += mbox_cmd.size_out; + } while (data_rcvd_size < feat_out_size); + + if (return_code) + *return_code = CXL_MBOX_CMD_RC_SUCCESS; + + return data_rcvd_size; +} +EXPORT_SYMBOL_NS_GPL(cxl_get_feature, "CXL"); diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c index 0d1fc3da9e35..6b061bef3a6a 100644 --- a/drivers/cxl/features.c +++ b/drivers/cxl/features.c @@ -47,14 +47,10 @@ static int cxl_get_supported_features(struct cxl_features_state *cfs) struct cxl_mbox_get_sup_feats_in mbox_in; struct cxl_feat_entry *entry; struct cxl_mbox_cmd mbox_cmd; - struct cxl_mem_command *cmd; int count; /* Get supported features is optional, need to check */ - cmd = cxl_find_feature_command(CXL_MBOX_OP_GET_SUPPORTED_FEATURES); - if (!cmd) - return -EOPNOTSUPP; - if (!test_bit(cmd->info.id, cxl_mbox->feature_cmds)) + if (!cxl_feature_enabled(cfs, CXL_MBOX_OP_GET_SUPPORTED_FEATURES)) return -EOPNOTSUPP; count = cxl_get_supported_features_count(cxl_mbox); diff --git a/include/cxl/features.h b/include/cxl/features.h index 8ff7d90932d6..872edd0a4417 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -76,10 +76,37 @@ struct cxl_features_state { struct cxl_feat_entry *entries; }; +/* + * Get Feature CXL 3.1 Spec 8.2.9.6.2 + */ + +/* + * Get Feature input payload + * CXL rev 3.1 section 8.2.9.6.2 Table 8-99 + */ +enum cxl_get_feat_selection { + CXL_GET_FEAT_SEL_CURRENT_VALUE, + CXL_GET_FEAT_SEL_DEFAULT_VALUE, + CXL_GET_FEAT_SEL_SAVED_VALUE, + CXL_GET_FEAT_SEL_MAX +}; + +struct cxl_mbox_get_feat_in { + uuid_t uuid; + __le16 offset; + __le16 count; + u8 selection; +} __packed; + +bool cxl_feature_enabled(struct cxl_features_state *cfs, u16 opcode); struct cxl_features *cxl_features_alloc(struct cxl_mailbox *cxl_mbox, struct device *parent); struct cxl_feat_entry * cxl_get_supported_feature_entry(struct cxl_features *features, const uuid_t *feat_uuid); +size_t cxl_get_feature(struct cxl_features *features, const uuid_t feat_uuid, + enum cxl_get_feat_selection selection, + void *feat_out, size_t feat_out_size, u16 offset, + u16 *return_code); #endif From patchwork Wed Jan 22 23:50:39 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947717 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 72E421CEE9F for ; Wed, 22 Jan 2025 23:52:13 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589933; cv=none; b=jWXhMRfBEbyryYaYpKtxVpTZxK1MFrakAFml1rZVHhhozPWTwFe7kSy9FhXqseQ149jSCXmuLvtFyhbwBNc5wsz/MaumH3gx4UdD8MvLEMZ5oX63itJ30tg7ezHqdPjhvqfpF/ytOXP1/tvLzDxalSzlFg7fC11C/AkU89HmAcE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589933; c=relaxed/simple; bh=cnmqNZClW1YUNWFxOCsbvZ1yVnN0dnG2tyyDPjsyAVs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nLkKqxR4MZNkok5SRVvM9jhyfhlTN/gJO1B2LJrnFmr0f3h7kHzJpMNQxyHqQjLw70erd73yywvN3F9l0tXv84jIgBEq36crt/9KSYnfI6sLZk7cfs6wV2zfxHl8n3rZ1nPpOFouaqacJGR8gePiKVxrEjNPE+lcwVUtOpLibjQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 31FDDC4CED2; Wed, 22 Jan 2025 23:52:13 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 08/19] cxl/mbox: Add SET_FEATURE mailbox command Date: Wed, 22 Jan 2025 16:50:39 -0700 Message-ID: <20250122235159.2716036-9-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Shiju Jose Add support for SET_FEATURE mailbox command. CXL spec 3.1 section 8.2.9.6 describes optional device specific features. CXL devices supports features with changeable attributes. The settings of a feature can be optionally modified using Set Feature command. CXL spec 3.1 section 8.2.9.6.3 describes Set Feature command. Signed-off-by: Shiju Jose Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- v1: - CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER enum must be set to 0. --- drivers/cxl/core/features.c | 92 ++++++++++++++++++++++++++++++++++++ include/cxl/features.h | 32 +++++++++++++ tools/testing/cxl/test/mem.c | 1 + 3 files changed, 125 insertions(+) diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c index ab9386b53a95..932e82b52f90 100644 --- a/drivers/cxl/core/features.c +++ b/drivers/cxl/core/features.c @@ -171,3 +171,95 @@ size_t cxl_get_feature(struct cxl_features *features, const uuid_t feat_uuid, return data_rcvd_size; } EXPORT_SYMBOL_NS_GPL(cxl_get_feature, "CXL"); + +/* + * FEAT_DATA_MIN_PAYLOAD_SIZE - min extra number of bytes should be + * available in the mailbox for storing the actual feature data so that + * the feature data transfer would work as expected. + */ +#define FEAT_DATA_MIN_PAYLOAD_SIZE 10 +int cxl_set_feature(struct cxl_features *features, + const uuid_t feat_uuid, u8 feat_version, + void *feat_data, size_t feat_data_size, + u32 feat_flag, u16 offset, u16 *return_code) +{ + struct cxl_memdev_set_feat_pi { + struct cxl_mbox_set_feat_hdr hdr; + u8 feat_data[]; + } __packed; + size_t data_in_size, data_sent_size = 0; + struct cxl_features_state *cfs; + struct cxl_mbox_cmd mbox_cmd; + struct cxl_mailbox *cxl_mbox; + size_t hdr_size; + int rc = 0; + + if (return_code) + *return_code = CXL_MBOX_CMD_RC_INPUT; + + cfs = dev_get_drvdata(&features->dev); + if (!cfs) + return -EOPNOTSUPP; + + if (!cxl_feature_enabled(cfs, CXL_MBOX_OP_SET_FEATURE)) + return -EOPNOTSUPP; + + cxl_mbox = features->cxl_mbox; + + struct cxl_memdev_set_feat_pi *pi __free(kfree) = + kmalloc(cxl_mbox->payload_size, GFP_KERNEL); + pi->hdr.uuid = feat_uuid; + pi->hdr.version = feat_version; + feat_flag &= ~CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK; + feat_flag |= CXL_SET_FEAT_FLAG_DATA_SAVED_ACROSS_RESET; + hdr_size = sizeof(pi->hdr); + /* + * Check minimum mbox payload size is available for + * the feature data transfer. + */ + if (hdr_size + FEAT_DATA_MIN_PAYLOAD_SIZE > cxl_mbox->payload_size) + return -ENOMEM; + + if ((hdr_size + feat_data_size) <= cxl_mbox->payload_size) { + pi->hdr.flags = cpu_to_le32(feat_flag | + CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER); + data_in_size = feat_data_size; + } else { + pi->hdr.flags = cpu_to_le32(feat_flag | + CXL_SET_FEAT_FLAG_INITIATE_DATA_TRANSFER); + data_in_size = cxl_mbox->payload_size - hdr_size; + } + + do { + pi->hdr.offset = cpu_to_le16(offset + data_sent_size); + memcpy(pi->feat_data, feat_data + data_sent_size, data_in_size); + mbox_cmd = (struct cxl_mbox_cmd) { + .opcode = CXL_MBOX_OP_SET_FEATURE, + .size_in = hdr_size + data_in_size, + .payload_in = pi, + }; + rc = cxl_internal_send_cmd(cxl_mbox, &mbox_cmd); + if (rc < 0) { + if (return_code) + *return_code = mbox_cmd.return_code; + return rc; + } + + data_sent_size += data_in_size; + if (data_sent_size >= feat_data_size) { + if (return_code) + *return_code = CXL_MBOX_CMD_RC_SUCCESS; + return 0; + } + + if ((feat_data_size - data_sent_size) <= (cxl_mbox->payload_size - hdr_size)) { + data_in_size = feat_data_size - data_sent_size; + pi->hdr.flags = cpu_to_le32(feat_flag | + CXL_SET_FEAT_FLAG_FINISH_DATA_TRANSFER); + } else { + pi->hdr.flags = cpu_to_le32(feat_flag | + CXL_SET_FEAT_FLAG_CONTINUE_DATA_TRANSFER); + } + } while (true); +} +EXPORT_SYMBOL_NS_GPL(cxl_set_feature, "CXL"); diff --git a/include/cxl/features.h b/include/cxl/features.h index 872edd0a4417..370c8d57c728 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -98,6 +98,35 @@ struct cxl_mbox_get_feat_in { u8 selection; } __packed; +/* + * Set Feature CXL 3.1 Spec 8.2.9.6.3 + */ + +/* + * Set Feature input payload + * CXL rev 3.1 section 8.2.9.6.3 Table 8-101 + */ +/* Set Feature : Payload in flags */ +#define CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK GENMASK(2, 0) +enum cxl_set_feat_flag_data_transfer { + CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER = 0, + CXL_SET_FEAT_FLAG_INITIATE_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_CONTINUE_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_FINISH_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_ABORT_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_DATA_TRANSFER_MAX +}; + +#define CXL_SET_FEAT_FLAG_DATA_SAVED_ACROSS_RESET BIT(3) + +struct cxl_mbox_set_feat_hdr { + uuid_t uuid; + __le32 flags; + __le16 offset; + u8 version; + u8 rsvd[9]; +} __packed; + bool cxl_feature_enabled(struct cxl_features_state *cfs, u16 opcode); struct cxl_features *cxl_features_alloc(struct cxl_mailbox *cxl_mbox, struct device *parent); @@ -108,5 +137,8 @@ size_t cxl_get_feature(struct cxl_features *features, const uuid_t feat_uuid, enum cxl_get_feat_selection selection, void *feat_out, size_t feat_out_size, u16 offset, u16 *return_code); +int cxl_set_feature(struct cxl_features *features, const uuid_t feat_uuid, + u8 feat_version, void *feat_data, size_t feat_data_size, + u32 feat_flag, u16 offset, u16 *return_code); #endif diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 208337432bfe..2b111142b2c0 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include From patchwork Wed Jan 22 23:50:40 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947718 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C37071CEE9F for ; Wed, 22 Jan 2025 23:52:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589934; cv=none; b=sCJu6Vo37Xoh3tqnzygFBgLx68Nx+SL/c/IJ+Zk5g2ygzM/hWrqvZxNnJVrWtgfHYSTwE03MqKACeAanchQG4DHsEAkJlTGMdL9OExTdQckixwEe4pm6rs6vblqO4fRa5JyszIBs0PnoLAAZ77n9Tq1Zl4BRAKb+0dWxjhalIf4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589934; c=relaxed/simple; bh=fDnjnfCNGgIHUIMaIjv0kSDP+AwuyzQkxQn4WDWls2c=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UzfBfCmLOlQUvmHS8B9pAFD7rfDPzBsVmEYBo5hExmQM8Drn7l3KiuNsS9VrA1oEj7Q4xGQEkVdkRjO+dAhV+FjtRx480oIIygQX1h2qUxuAGkLulYadG6BRLZiDKuDsXehLpaJgmxT53T2pOAaoAh5jkO/EBS5Po2jy5bHKWBw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8DBB6C4CED2; Wed, 22 Jan 2025 23:52:14 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 09/19] cxl: Setup exclusive CXL features that are reserved for the kernel Date: Wed, 22 Jan 2025 16:50:40 -0700 Message-ID: <20250122235159.2716036-10-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Certain features will be exclusively used by components such as in kernel RAS driver. Setup an exclusion list that can be later filtered out before exposing to user space. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams --- drivers/cxl/core/features.c | 22 ++++++++++++++++++++++ drivers/cxl/features.c | 6 +++++- include/cxl/features.h | 34 ++++++++++++++++++++++++++++++++++ 3 files changed, 61 insertions(+), 1 deletion(-) diff --git a/drivers/cxl/core/features.c b/drivers/cxl/core/features.c index 932e82b52f90..c836b3a64561 100644 --- a/drivers/cxl/core/features.c +++ b/drivers/cxl/core/features.c @@ -6,6 +6,17 @@ #include "core.h" #include "cxlmem.h" +static const uuid_t cxl_exclusive_feats[] = { + CXL_FEAT_PATROL_SCRUB_UUID, + CXL_FEAT_ECS_UUID, + CXL_FEAT_SPPR_UUID, + CXL_FEAT_HPPR_UUID, + CXL_FEAT_CACHELINE_SPARING_UUID, + CXL_FEAT_ROW_SPARING_UUID, + CXL_FEAT_BANK_SPARING_UUID, + CXL_FEAT_RANK_SPARING_UUID, +}; + #define CXL_FEATURE_MAX_DEVS 65536 static DEFINE_IDA(cxl_features_ida); @@ -263,3 +274,14 @@ int cxl_set_feature(struct cxl_features *features, } while (true); } EXPORT_SYMBOL_NS_GPL(cxl_set_feature, "CXL"); + +bool is_cxl_feature_exclusive(struct cxl_feat_entry *entry) +{ + for (int i = 0; i < ARRAY_SIZE(cxl_exclusive_feats); i++) { + if (uuid_equal(&entry->uuid, &cxl_exclusive_feats[i])) + return true; + } + + return false; +} +EXPORT_SYMBOL_NS_GPL(is_cxl_feature_exclusive, "CXL"); diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c index 6b061bef3a6a..1c4bcd7465cb 100644 --- a/drivers/cxl/features.c +++ b/drivers/cxl/features.c @@ -47,6 +47,7 @@ static int cxl_get_supported_features(struct cxl_features_state *cfs) struct cxl_mbox_get_sup_feats_in mbox_in; struct cxl_feat_entry *entry; struct cxl_mbox_cmd mbox_cmd; + int user_feats = 0; int count; /* Get supported features is optional, need to check */ @@ -127,6 +128,8 @@ static int cxl_get_supported_features(struct cxl_features_state *cfs) return -ENXIO; memcpy(entry, mbox_out->ents, retrieved); + if (!is_cxl_feature_exclusive(entry)) + user_feats++; entry++; /* * If the number of output entries is less than expected, add the @@ -138,6 +141,7 @@ static int cxl_get_supported_features(struct cxl_features_state *cfs) cfs->num_features = count; cfs->entries = no_free_ptr(entries); + cfs->num_user_features = user_feats; return devm_add_action_or_reset(&cfs->features->dev, cxl_free_feature_entries, cfs->entries); } @@ -177,7 +181,7 @@ static ssize_t features_show(struct device *dev, struct device_attribute *attr, if (!cfs) return -ENOENT; - return sysfs_emit(buf, "%d\n", cfs->num_features); + return sysfs_emit(buf, "%d\n", cfs->num_user_features); } static DEVICE_ATTR_RO(features); diff --git a/include/cxl/features.h b/include/cxl/features.h index 370c8d57c728..f746358ba5f9 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -5,6 +5,38 @@ #include +#define CXL_FEAT_PATROL_SCRUB_UUID \ + UUID_INIT(0x96dad7d6, 0xfde8, 0x482b, 0xa7, 0x33, 0x75, 0x77, 0x4e, \ + 0x06, 0xdb, 0x8a) + +#define CXL_FEAT_ECS_UUID \ + UUID_INIT(0xe5b13f22, 0x2328, 0x4a14, 0xb8, 0xba, 0xb9, 0x69, 0x1e, \ + 0x89, 0x33, 0x86) + +#define CXL_FEAT_SPPR_UUID \ + UUID_INIT(0x892ba475, 0xfad8, 0x474e, 0x9d, 0x3e, 0x69, 0x2c, 0x91, \ + 0x75, 0x68, 0xbb) + +#define CXL_FEAT_HPPR_UUID \ + UUID_INIT(0x80ea4521, 0x786f, 0x4127, 0xaf, 0xb1, 0xec, 0x74, 0x59, \ + 0xfb, 0x0e, 0x24) + +#define CXL_FEAT_CACHELINE_SPARING_UUID \ + UUID_INIT(0x96C33386, 0x91dd, 0x44c7, 0x9e, 0xcb, 0xfd, 0xaf, 0x65, \ + 0x03, 0xba, 0xc4) + +#define CXL_FEAT_ROW_SPARING_UUID \ + UUID_INIT(0x450ebf67, 0xb135, 0x4f97, 0xa4, 0x98, 0xc2, 0xd5, 0x7f, \ + 0x27, 0x9b, 0xed) + +#define CXL_FEAT_BANK_SPARING_UUID \ + UUID_INIT(0x78b79636, 0x90ac, 0x4b64, 0xa4, 0xef, 0xfa, 0xac, 0x5d, \ + 0x18, 0xa8, 0x63) + +#define CXL_FEAT_RANK_SPARING_UUID \ + UUID_INIT(0x34dbaff5, 0x0552, 0x4281, 0x8f, 0x76, 0xda, 0x0b, 0x5e, \ + 0x7a, 0x76, 0xa7) + struct cxl_mailbox; enum feature_cmds { @@ -73,6 +105,7 @@ struct cxl_mbox_get_sup_feats_out { struct cxl_features_state { struct cxl_features *features; int num_features; + int num_user_features; struct cxl_feat_entry *entries; }; @@ -140,5 +173,6 @@ size_t cxl_get_feature(struct cxl_features *features, const uuid_t feat_uuid, int cxl_set_feature(struct cxl_features *features, const uuid_t feat_uuid, u8 feat_version, void *feat_data, size_t feat_data_size, u32 feat_flag, u16 offset, u16 *return_code); +bool is_cxl_feature_exclusive(struct cxl_feat_entry *entry); #endif From patchwork Wed Jan 22 23:50:41 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947719 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33AAF1CAA99 for ; Wed, 22 Jan 2025 23:52:16 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589936; cv=none; b=oPWQ2EjaU7aJkU5fsQNSeiicvqT+putbb8YjujVcfgAjzscHyirNn+fHAMRD2HP+5H26Hab9oOR/FlH6iZiow5CY9G9Es3kVJ5igR1j9YQ8IHPY8PctDxGo5bOS2WjPQahKOPh3S/qLMPeea+3RhP1GrDWB0pGjOWtPLBN5AhFI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589936; c=relaxed/simple; bh=2wx+Ja1wemk02GmHYRSORoWjuTt1Fvq5WIMtcCQP3d8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Zqmo9NFj+uxcmKnjib9KouBaqTzsVDkegS4iHv8ZS7iQung7yM/Ev5FGQDSEe77h9yhUTNTuzmQnyskDxux9vxR15MowNhglO8ZSORdhhO4deDwuGIxVw306zLik3EQ+3SmdB96S7FIK65lFLgQOT0KRPkrIKBNUzbNipPBuZqo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id E4E7EC4CED2; Wed, 22 Jan 2025 23:52:15 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 10/19] cxl: Add FWCTL support to the CXL features driver Date: Wed, 22 Jan 2025 16:50:41 -0700 Message-ID: <20250122235159.2716036-11-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add fwctl support code to allow sending of CXL feature commands from userspace through as ioctls via FWCTL. Provide initial setup bits. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- v1: - Add missing dev_set_name() (Jason) --- drivers/cxl/Kconfig | 1 + drivers/cxl/features.c | 44 ++++++++++++++++++++++++++++++++++++-- include/cxl/features.h | 2 ++ include/uapi/fwctl/fwctl.h | 1 + 4 files changed, 46 insertions(+), 2 deletions(-) diff --git a/drivers/cxl/Kconfig b/drivers/cxl/Kconfig index 9a6ffd81ac0e..5de06c3806ce 100644 --- a/drivers/cxl/Kconfig +++ b/drivers/cxl/Kconfig @@ -148,6 +148,7 @@ config CXL_REGION_INVALIDATION_TEST config CXL_FEATURES tristate "CXL: Features support" + depends on FWCTL default CXL_BUS help Enable CXL features support that are tied to a CXL mailbox. diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c index 1c4bcd7465cb..5a2b771586d3 100644 --- a/drivers/cxl/features.c +++ b/drivers/cxl/features.c @@ -9,6 +9,37 @@ #include "cxl.h" #include "cxlmem.h" +static int cxlctl_open_uctx(struct fwctl_uctx *uctx) +{ + return 0; +} + +static void cxlctl_close_uctx(struct fwctl_uctx *uctx) +{ +} + +static void *cxlctl_info(struct fwctl_uctx *uctx, size_t *length) +{ + /* Place holder */ + return ERR_PTR(-EOPNOTSUPP); +} + +static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, + void *rpc_in, size_t in_len, size_t *out_len) +{ + /* Place holder */ + return ERR_PTR(-EOPNOTSUPP); +} + +static const struct fwctl_ops cxlctl_ops = { + .device_type = FWCTL_DEVICE_TYPE_CXL, + .uctx_size = sizeof(struct cxl_features), + .open_uctx = cxlctl_open_uctx, + .close_uctx = cxlctl_close_uctx, + .info = cxlctl_info, + .fw_rpc = cxlctl_fw_rpc, +}; + static void cxl_free_feature_entries(void *entries) { kvfree(entries); @@ -146,13 +177,16 @@ static int cxl_get_supported_features(struct cxl_features_state *cfs) cxl_free_feature_entries, cfs->entries); } +DEFINE_FREE(cfs, struct cxl_features_state *, if (_T) fwctl_put(&_T->fwctl)) + static int cxl_features_probe(struct device *dev) { struct cxl_features *features = to_cxl_features(dev); int rc; - struct cxl_features_state *cfs __free(kfree) = - kzalloc(sizeof(*cfs), GFP_KERNEL); + struct cxl_features_state *cfs __free(cfs) = + fwctl_alloc_device(dev, &cxlctl_ops, + struct cxl_features_state, fwctl); if (!cfs) return -ENOMEM; @@ -161,6 +195,10 @@ static int cxl_features_probe(struct device *dev) if (rc) return rc; + rc = fwctl_register(&cfs->fwctl); + if (rc) + return rc; + dev_set_drvdata(dev, no_free_ptr(cfs)); return 0; @@ -170,6 +208,7 @@ static void cxl_features_remove(struct device *dev) { struct cxl_features_state *cfs = dev_get_drvdata(dev); + fwctl_unregister(&cfs->fwctl); kfree(cfs); } @@ -212,4 +251,5 @@ module_cxl_driver(cxl_features_driver); MODULE_DESCRIPTION("CXL: Features"); MODULE_LICENSE("GPL"); MODULE_IMPORT_NS("CXL"); +MODULE_IMPORT_NS("FWCTL"); MODULE_ALIAS_CXL(CXL_DEVICE_FEATURES); diff --git a/include/cxl/features.h b/include/cxl/features.h index f746358ba5f9..69697c069e63 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -4,6 +4,7 @@ #define __CXL_FEATURES_H__ #include +#include #define CXL_FEAT_PATROL_SCRUB_UUID \ UUID_INIT(0x96dad7d6, 0xfde8, 0x482b, 0xa7, 0x33, 0x75, 0x77, 0x4e, \ @@ -103,6 +104,7 @@ struct cxl_mbox_get_sup_feats_out { } __packed; struct cxl_features_state { + struct fwctl_device fwctl; struct cxl_features *features; int num_features; int num_user_features; diff --git a/include/uapi/fwctl/fwctl.h b/include/uapi/fwctl/fwctl.h index f9b27fb5c161..2c3817592308 100644 --- a/include/uapi/fwctl/fwctl.h +++ b/include/uapi/fwctl/fwctl.h @@ -43,6 +43,7 @@ enum { enum fwctl_device_type { FWCTL_DEVICE_TYPE_ERROR = 0, FWCTL_DEVICE_TYPE_MLX5 = 1, + FWCTL_DEVICE_TYPE_CXL, }; /** From patchwork Wed Jan 22 23:50:42 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947721 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E00E61CAA94 for ; Wed, 22 Jan 2025 23:52:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589938; cv=none; b=RFwnkWwMKL9GKgytExLXEJSHWd0Ja7LpgJ75gK/tELh0HU7Di+Jc9ftDl0gCJxk6qgJGivrn+ayK4gisO3npw1sZd3Pu7iqr+pI1yX7MV/sTO9WqujhO+2kNl66QQmZzdJZu6Wi3+0WyaiwNyNHNwRfxxQSCFeXt8l6O5vc5YCw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589938; c=relaxed/simple; bh=gguz5muXhSjTF6W7I8s/W/qMAdRTwcOeqz1w57KAmn8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=EEuygAW41my7YwNiWfLlBTD5zDsBZZbrXezKiuZUhNZQs9qhRGJ037x2HpXa9bPuX52Vo10I6NWsZT+MaxXdI+nxIjlwx1rPygNi8+AAN6r/SklamUGUfFpMTg4CEm1sAZPz29ZM8y29/lKtyIBl0fJwlDNJ80aFbi1JvFcd+TE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 48DDCC4CED2; Wed, 22 Jan 2025 23:52:17 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 11/19] cxl: Add support for get driver information Date: Wed, 22 Jan 2025 16:50:42 -0700 Message-ID: <20250122235159.2716036-12-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add definition for fwctl_ops->info() to return driver information. The function will return a mask of the feature mailbox commands supported by the fwctl char device. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- v1: - Add missed setting of *length for ->info(). - Use BIT() instead of enum directly. --- drivers/cxl/features.c | 30 ++++++++++++++++++++++++++++-- include/cxl/features.h | 7 ------- include/cxl/mailbox.h | 1 + include/uapi/fwctl/cxl.h | 31 +++++++++++++++++++++++++++++++ 4 files changed, 60 insertions(+), 9 deletions(-) create mode 100644 include/uapi/fwctl/cxl.h diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c index 5a2b771586d3..cc72e73ae8d6 100644 --- a/drivers/cxl/features.c +++ b/drivers/cxl/features.c @@ -9,6 +9,8 @@ #include "cxl.h" #include "cxlmem.h" +#define to_cxl_features_state(fwctl) container_of(fwctl, struct cxl_features_state, fwctl) + static int cxlctl_open_uctx(struct fwctl_uctx *uctx) { return 0; @@ -18,10 +20,34 @@ static void cxlctl_close_uctx(struct fwctl_uctx *uctx) { } +static void set_command_mask(struct cxl_features_state *cfs, u32 *mask) +{ + struct cxl_mailbox *cxl_mbox = cfs->features->cxl_mbox; + + *mask = 0; + if (test_bit(CXL_FEATURE_ID_GET_SUPPORTED_FEATURES, + cxl_mbox->feature_cmds)) + *mask |= BIT(CXL_FEATURE_ID_GET_SUPPORTED_FEATURES); + if (test_bit(CXL_FEATURE_ID_GET_FEATURE, cxl_mbox->feature_cmds)) + *mask |= BIT(CXL_FEATURE_ID_GET_FEATURE); + if (test_bit(CXL_FEATURE_ID_SET_FEATURE, cxl_mbox->feature_cmds)) + *mask |= BIT(CXL_FEATURE_ID_SET_FEATURE); +} + static void *cxlctl_info(struct fwctl_uctx *uctx, size_t *length) { - /* Place holder */ - return ERR_PTR(-EOPNOTSUPP); + struct fwctl_device *fwctl = uctx->fwctl; + struct cxl_features_state *cfs = to_cxl_features_state(fwctl); + struct fwctl_info_cxl *info; + + info = kzalloc(sizeof(*info), GFP_KERNEL); + if (!info) + return ERR_PTR(-ENOMEM); + + set_command_mask(cfs, &info->cmd_mask); + *length = sizeof(*info); + + return info; } static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, diff --git a/include/cxl/features.h b/include/cxl/features.h index 69697c069e63..e38ee777328d 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -40,13 +40,6 @@ struct cxl_mailbox; -enum feature_cmds { - CXL_FEATURE_ID_GET_SUPPORTED_FEATURES = 0, - CXL_FEATURE_ID_GET_FEATURE, - CXL_FEATURE_ID_SET_FEATURE, - CXL_FEATURE_ID_MAX, -}; - struct cxl_features { int id; struct device dev; diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index 263fc346aeb1..1157b19175a5 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -4,6 +4,7 @@ #define __CXL_MBOX_H__ #include #include +#include #include /** diff --git a/include/uapi/fwctl/cxl.h b/include/uapi/fwctl/cxl.h new file mode 100644 index 000000000000..79b822dbfafd --- /dev/null +++ b/include/uapi/fwctl/cxl.h @@ -0,0 +1,31 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (c) 2024, Intel Corporation + * + * These are definitions for the mailbox command interface of CXL subsystem. + */ +#ifndef _UAPI_FWCTL_CXL_H_ +#define _UAPI_FWCTL_CXL_H_ + +#include + +enum feature_cmds { + CXL_FEATURE_ID_GET_SUPPORTED_FEATURES = 0, + CXL_FEATURE_ID_GET_FEATURE, + CXL_FEATURE_ID_SET_FEATURE, + CXL_FEATURE_ID_MAX, +}; + +/** + * struct fwctl_info_cxl - ioctl(FWCTL_INFO) out_device_data + * @cmd_mask: Mask indicate which commands are supported based on 'enum feature_cmds' + * + * Return basic information about the FW interface available. + * + * nr_commands is number of hardware commands the driver supports. Use + * FWCTL_CMD_HW_INFO ioctl to request additional information. + */ +struct fwctl_info_cxl { + __u32 cmd_mask; +}; +#endif From patchwork Wed Jan 22 23:50:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947720 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D52411CAA99 for ; Wed, 22 Jan 2025 23:52:18 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589938; cv=none; b=a4SbzvfPTIbema7KAnKjlnEaP3/YsEOnafSF6Adl574ocHLIW4JJ+3Y2VK5A+d+MCcUYSXkkAaKws5JptiTP4124Dwn7We+OG08FVrLjGTKemThoGmqjpJlyFURFu21mBRBIfKAGeHvt0fejv4jFlTfMMnqxRISSr3QgWC+zymI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589938; c=relaxed/simple; bh=ggdnxkcsFMWwRKkLCmTBboYzCkMHzOS/V5VB1aA8cNI=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=CHBShaXC5b2R3BpQL7a0hE7X2LCmXJFt2jcN3W+ps8b1QqNcV81O6/qypUD/VkJg2olLNzzk9gTG/dAQfbzWSobW6f5s5+zJaTzrs4Ko2USCko9jaxam5xHR3h0NxZ2yBvTSgGk+3gDLgYFYzrenTD68HErsFp/0zhiPJBsNIwM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 9DB3EC4CED6; Wed, 22 Jan 2025 23:52:18 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 12/19] cxl: Move cxl_mem.h under uapi to cxl exclusive directory Date: Wed, 22 Jan 2025 16:50:43 -0700 Message-ID: <20250122235159.2716036-13-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In preparation of more cxl headers, create a cxl directory under uapi. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron --- MAINTAINERS | 1 - drivers/cxl/cxlmem.h | 2 +- include/cxl/mailbox.h | 2 +- include/uapi/{linux/cxl_mem.h => cxl/mem.h} | 0 4 files changed, 2 insertions(+), 3 deletions(-) rename include/uapi/{linux/cxl_mem.h => cxl/mem.h} (100%) diff --git a/MAINTAINERS b/MAINTAINERS index 877eb301b1e5..d5a38bdbf2c1 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -5775,7 +5775,6 @@ S: Maintained F: Documentation/driver-api/cxl F: drivers/cxl/ F: include/cxl/ -F: include/uapi/linux/cxl_mem.h F: tools/testing/cxl/ COMPUTE EXPRESS LINK PMU (CPMU) diff --git a/drivers/cxl/cxlmem.h b/drivers/cxl/cxlmem.h index 55c55685cb39..e82c84b2edb5 100644 --- a/drivers/cxl/cxlmem.h +++ b/drivers/cxl/cxlmem.h @@ -2,7 +2,7 @@ /* Copyright(c) 2020-2021 Intel Corporation. */ #ifndef __CXL_MEM_H__ #define __CXL_MEM_H__ -#include +#include #include #include #include diff --git a/include/cxl/mailbox.h b/include/cxl/mailbox.h index 1157b19175a5..9b7bd59f5fa4 100644 --- a/include/cxl/mailbox.h +++ b/include/cxl/mailbox.h @@ -4,8 +4,8 @@ #define __CXL_MBOX_H__ #include #include +#include #include -#include /** * struct cxl_mbox_cmd - A command to be submitted to hardware. diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/cxl/mem.h similarity index 100% rename from include/uapi/linux/cxl_mem.h rename to include/uapi/cxl/mem.h From patchwork Wed Jan 22 23:50:44 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947722 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A9D7F1C5D54 for ; Wed, 22 Jan 2025 23:52:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589940; cv=none; b=XglGju3DYILHSEpX1zI5GnSbaa8NLva73nPZzWTyvGixHjv21UpF6ogZVJ91/GZzH+KDMPCCgwd4n/rNFzLN2+FclkgxqwmdUbC3CH0TJv1BGhbcoXi22foeBgf+4mi6wxapDSTLn5oDt0FD8thVzYGCAgyAGc6gvCuwidZZcrE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589940; c=relaxed/simple; bh=OwMdasK9gC621SO4HdBVwzLw8nMNx8gWBSi/N7zzxRY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ma0fkMmNtVSyo62xcu/daTRd5KhY7wXT0MCrzjziUBAdZ9qmFeqyb86IKt4/0RA5bIoxjdYNqMUfIcIzPEnCbhaIxrLMQD7zP0+VEmBO+zdqOvzLidGWpIWPM9rH/sBOrp1naaL7fb/rWiGhHrE4vIn0emVq8Dg/7e77++fMHH0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0838AC4CED2; Wed, 22 Jan 2025 23:52:19 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 13/19] cxl: Move cxl feature command structs to user header Date: Wed, 22 Jan 2025 16:50:44 -0700 Message-ID: <20250122235159.2716036-14-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In perparation for cxl fwctl enabling, move data structures related to cxl feature commands to a user header file. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams --- v1: - Fixup user headers from 0day report --- include/cxl/features.h | 101 +------------------------------ include/uapi/cxl/features.h | 117 ++++++++++++++++++++++++++++++++++++ 2 files changed, 118 insertions(+), 100 deletions(-) create mode 100644 include/uapi/cxl/features.h diff --git a/include/cxl/features.h b/include/cxl/features.h index e38ee777328d..9f234659ca2d 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -5,6 +5,7 @@ #include #include +#include #define CXL_FEAT_PATROL_SCRUB_UUID \ UUID_INIT(0x96dad7d6, 0xfde8, 0x482b, 0xa7, 0x33, 0x75, 0x77, 0x4e, \ @@ -47,55 +48,6 @@ struct cxl_features { }; #define to_cxl_features(dev) container_of(dev, struct cxl_features, dev) -/* Get Supported Features (0x500h) CXL r3.1 8.2.9.6.1 */ -struct cxl_mbox_get_sup_feats_in { - __le32 count; - __le16 start_idx; - u8 reserved[2]; -} __packed; - -/* CXL spec r3.2 Table 8-87 command effects */ -#define CXL_CMD_CONFIG_CHANGE_COLD_RESET BIT(0) -#define CXL_CMD_CONFIG_CHANGE_IMMEDIATE BIT(1) -#define CXL_CMD_DATA_CHANGE_IMMEDIATE BIT(2) -#define CXL_CMD_POLICY_CHANGE_IMMEDIATE BIT(3) -#define CXL_CMD_LOG_CHANGE_IMMEDIATE BIT(4) -#define CXL_CMD_SECURITY_STATE_CHANGE BIT(5) -#define CXL_CMD_BACKGROUND BIT(6) -#define CXL_CMD_BGCMD_ABORT_SUPPORTED BIT(7) -#define CXL_CMD_EFFECTS_EXTEND BIT(9) -#define CXL_CMD_CONFIG_CHANGE_CONV_RESET BIT(10) -#define CXL_CMD_CONFIG_CHANGE_CXL_RESET BIT(11) - -/* - * CXL spec r3.2 Table 8-109 - * Get Supported Features Supported Feature Entry - * Atribute Flags - */ -#define CXL_FEATURE_F_CHANGEABLE BIT(0) -#define CXL_FEATURE_F_PERSIST_FW_UPDATE BIT(4) -#define CXL_FEATURE_F_DEFAULT_SEL BIT(5) -#define CXL_FEATURE_F_SAVED_SEL BIT(6) - -struct cxl_feat_entry { - uuid_t uuid; - __le16 id; - __le16 get_feat_size; - __le16 set_feat_size; - __le32 flags; - u8 get_feat_ver; - u8 set_feat_ver; - __le16 effects; - u8 reserved[18]; -} __packed; - -struct cxl_mbox_get_sup_feats_out { - __le16 num_entries; - __le16 supported_feats; - u8 reserved[4]; - struct cxl_feat_entry ents[] __counted_by_le(num_entries); -} __packed; - struct cxl_features_state { struct fwctl_device fwctl; struct cxl_features *features; @@ -104,57 +56,6 @@ struct cxl_features_state { struct cxl_feat_entry *entries; }; -/* - * Get Feature CXL 3.1 Spec 8.2.9.6.2 - */ - -/* - * Get Feature input payload - * CXL rev 3.1 section 8.2.9.6.2 Table 8-99 - */ -enum cxl_get_feat_selection { - CXL_GET_FEAT_SEL_CURRENT_VALUE, - CXL_GET_FEAT_SEL_DEFAULT_VALUE, - CXL_GET_FEAT_SEL_SAVED_VALUE, - CXL_GET_FEAT_SEL_MAX -}; - -struct cxl_mbox_get_feat_in { - uuid_t uuid; - __le16 offset; - __le16 count; - u8 selection; -} __packed; - -/* - * Set Feature CXL 3.1 Spec 8.2.9.6.3 - */ - -/* - * Set Feature input payload - * CXL rev 3.1 section 8.2.9.6.3 Table 8-101 - */ -/* Set Feature : Payload in flags */ -#define CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK GENMASK(2, 0) -enum cxl_set_feat_flag_data_transfer { - CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER = 0, - CXL_SET_FEAT_FLAG_INITIATE_DATA_TRANSFER, - CXL_SET_FEAT_FLAG_CONTINUE_DATA_TRANSFER, - CXL_SET_FEAT_FLAG_FINISH_DATA_TRANSFER, - CXL_SET_FEAT_FLAG_ABORT_DATA_TRANSFER, - CXL_SET_FEAT_FLAG_DATA_TRANSFER_MAX -}; - -#define CXL_SET_FEAT_FLAG_DATA_SAVED_ACROSS_RESET BIT(3) - -struct cxl_mbox_set_feat_hdr { - uuid_t uuid; - __le32 flags; - __le16 offset; - u8 version; - u8 rsvd[9]; -} __packed; - bool cxl_feature_enabled(struct cxl_features_state *cfs, u16 opcode); struct cxl_features *cxl_features_alloc(struct cxl_mailbox *cxl_mbox, struct device *parent); diff --git a/include/uapi/cxl/features.h b/include/uapi/cxl/features.h new file mode 100644 index 000000000000..18b74252058a --- /dev/null +++ b/include/uapi/cxl/features.h @@ -0,0 +1,117 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* + * Copyright (c) 2024,2025, Intel Corporation + * + * These are definitions for the mailbox command interface of CXL subsystem. + */ +#ifndef _UAPI_CXL_FEATURES_H_ +#define _UAPI_CXL_FEATURES_H_ + +#include +#ifndef __KERNEL__ +#include +#else +#include +#endif + +/* Get Supported Features (0x500h) CXL r3.1 8.2.9.6.1 */ +struct cxl_mbox_get_sup_feats_in { + __le32 count; + __le16 start_idx; + __u8 reserved[2]; +} __attribute__ ((__packed__)); + +/* CXL spec r3.2 Table 8-87 command effects */ +#define CXL_CMD_CONFIG_CHANGE_COLD_RESET BIT(0) +#define CXL_CMD_CONFIG_CHANGE_IMMEDIATE BIT(1) +#define CXL_CMD_DATA_CHANGE_IMMEDIATE BIT(2) +#define CXL_CMD_POLICY_CHANGE_IMMEDIATE BIT(3) +#define CXL_CMD_LOG_CHANGE_IMMEDIATE BIT(4) +#define CXL_CMD_SECURITY_STATE_CHANGE BIT(5) +#define CXL_CMD_BACKGROUND BIT(6) +#define CXL_CMD_BGCMD_ABORT_SUPPORTED BIT(7) +#define CXL_CMD_EFFECTS_EXTEND BIT(9) +#define CXL_CMD_CONFIG_CHANGE_CONV_RESET BIT(10) +#define CXL_CMD_CONFIG_CHANGE_CXL_RESET BIT(11) + +/* + * CXL spec r3.2 Table 8-109 + * Get Supported Features Supported Feature Entry + * Attribute Flags + */ +#define CXL_FEATURE_F_CHANGEABLE BIT(0) +#define CXL_FEATURE_F_PERSIST_FW_UPDATE BIT(4) +#define CXL_FEATURE_F_DEFAULT_SEL BIT(5) +#define CXL_FEATURE_F_SAVED_SEL BIT(6) + +struct cxl_feat_entry { + uuid_t uuid; + __le16 id; + __le16 get_feat_size; + __le16 set_feat_size; + __le32 flags; + __u8 get_feat_ver; + __u8 set_feat_ver; + __le16 effects; + __u8 reserved[18]; +} __attribute__ ((__packed__)); + +struct cxl_mbox_get_sup_feats_out { + __le16 num_entries; + __le16 supported_feats; + __u8 reserved[4]; + struct cxl_feat_entry ents[] __counted_by_le(num_entries); +} __attribute__ ((__packed__)); + +/* + * Get Feature CXL 3.1 Spec 8.2.9.6.2 + */ + +/* + * Get Feature input payload + * CXL rev 3.1 section 8.2.9.6.2 Table 8-99 + */ +enum cxl_get_feat_selection { + CXL_GET_FEAT_SEL_CURRENT_VALUE, + CXL_GET_FEAT_SEL_DEFAULT_VALUE, + CXL_GET_FEAT_SEL_SAVED_VALUE, + CXL_GET_FEAT_SEL_MAX +}; + +struct cxl_mbox_get_feat_in { + uuid_t uuid; + __le16 offset; + __le16 count; + __u8 selection; +} __attribute__ ((__packed__)); + +/* + * Set Feature CXL 3.1 Spec 8.2.9.6.3 + */ + +/* + * Set Feature input payload + * CXL rev 3.1 section 8.2.9.6.3 Table 8-101 + */ +/* Set Feature : Payload in flags */ +#define CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK GENMASK(2, 0) +enum cxl_set_feat_flag_data_transfer { + CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER = 0, + CXL_SET_FEAT_FLAG_INITIATE_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_CONTINUE_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_FINISH_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_ABORT_DATA_TRANSFER, + CXL_SET_FEAT_FLAG_DATA_TRANSFER_MAX +}; + +#define CXL_SET_FEAT_FLAG_DATA_SAVED_ACROSS_RESET BIT(3) + +struct cxl_mbox_set_feat_hdr { + uuid_t uuid; + __le32 flags; + __le16 offset; + __u8 version; + __u8 rsvd[9]; +} __attribute__ ((__packed__)); + +#endif From patchwork Wed Jan 22 23:50:45 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947723 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0FF471C5D54 for ; Wed, 22 Jan 2025 23:52:21 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589942; cv=none; b=Jzq5E5tVm8SuApD3DRj0JRTuqSwhFJ3DvlP0FFaWn9ylhQSdAAoRm0MsO0ETNd7ZptKs+4LR4tpuY1LbpHi44pS3YpbOiZuaOOTT0uJmnUsktVdh3pQcrOztIEwGlX+Od/PqD0eVV+tWmYEy8wfIprbcn+fpO/LSiXfqnFBjEP4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589942; c=relaxed/simple; bh=VXXSaRvuYDO2bdX4ogPUAXl+Tl0K9odGFbx4C7vrZt0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Ww0FUQy+fEVWw7xb91DIxWanZUXsD44ybs8K0vrmNkL6iX6bfHz5/J1CFMjQB4AY4NObDmPv23uG4rSNXXND8a0WJvBP8AJE26nSqeINV44oid9NX0PPQItN00CmoJPVaqVV0w9N9srjig3Z0JD2rbCZjLpJpkheVMz07mV/o+8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 66FABC4CED2; Wed, 22 Jan 2025 23:52:21 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 14/19] cxl: Add support for fwctl RPC command to enable CXL feature commands Date: Wed, 22 Jan 2025 16:50:45 -0700 Message-ID: <20250122235159.2716036-15-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 fwctl provides a fwctl_ops->fw_rpc() callback in order to issue ioctls to a device. The cxl fwctl driver will start by supporting the CXL feature commands: Get Supported Features, Get Feature, and Set Feature. The fw_rpc() callback provides 'enum fwctl_rpc_scope' parameter where it indicates the security scope of the call. The Get Supported Features and Get Feature calls can be executed with the scope of FWCTL_RPC_CONFIGRATION. The Set Feature call is gated by the effects of the feature reported by Get Supported Features call for the specific feature. Only "get supported features" is supported in this patch. Additional commands will be added in follow on patches. "Get supported features" will filter the features that are exclusive to the kernel and only report out features that are not kernel only. Signed-off-by: Dave Jiang --- drivers/cxl/core/mbox.c | 12 +++ drivers/cxl/features.c | 198 +++++++++++++++++++++++++++++++++++- include/cxl/features.h | 1 + include/uapi/cxl/features.h | 13 +++ include/uapi/fwctl/cxl.h | 29 ++++++ 5 files changed, 250 insertions(+), 3 deletions(-) diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c index 0b4946205910..f818e452dbca 100644 --- a/drivers/cxl/core/mbox.c +++ b/drivers/cxl/core/mbox.c @@ -246,6 +246,18 @@ struct cxl_mem_command *cxl_find_feature_command(u16 opcode) } EXPORT_SYMBOL_NS_GPL(cxl_find_feature_command, "CXL"); +struct cxl_mem_command *cxl_find_feature_command_by_id(u32 id) +{ + struct cxl_mem_command *c; + + cxl_for_each_feature_cmd(c) + if (c->info.id == id) + return c; + + return NULL; +} +EXPORT_SYMBOL_NS_GPL(cxl_find_feature_command_by_id, "CXL"); + static const char *cxl_mem_opcode_to_name(u16 opcode) { struct cxl_mem_command *c; diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c index cc72e73ae8d6..e65fc8479d21 100644 --- a/drivers/cxl/features.c +++ b/drivers/cxl/features.c @@ -50,11 +50,203 @@ static void *cxlctl_info(struct fwctl_uctx *uctx, size_t *length) return info; } +static struct cxl_feat_entry * +get_support_feature_info(struct cxl_features_state *cfs, + const struct fwctl_rpc_cxl *rpc_in) +{ + struct cxl_feat_entry *feat; + uuid_t uuid; + + if (rpc_in->op_size < sizeof(uuid)) + return ERR_PTR(-EINVAL); + + if (copy_from_user(&uuid, u64_to_user_ptr(rpc_in->in_payload), + sizeof(uuid))) + return ERR_PTR(-EFAULT); + + for (int i = 0; i < cfs->num_features; i++) { + feat = &cfs->entries[i]; + if (uuid_equal(&uuid, &feat->uuid)) + return feat; + } + + return ERR_PTR(-ENOENT); +} + +static void *cxlctl_get_supported_features(struct cxl_features_state *cfs, + const struct fwctl_rpc_cxl *rpc_in, + size_t *out_len) +{ + struct cxl_mbox_get_sup_feats_out *feat_out; + struct cxl_mbox_get_sup_feats_in feat_in; + struct cxl_feat_entry *saved, *pos; + int requested, copied; + size_t out_size; + u32 count; + u16 start; + + if (rpc_in->op_size != sizeof(feat_in)) + return ERR_PTR(-EINVAL); + + if (copy_from_user(&feat_in, u64_to_user_ptr(rpc_in->in_payload), + rpc_in->op_size)) + return ERR_PTR(-EFAULT); + + count = le32_to_cpu(feat_in.count); + start = le16_to_cpu(feat_in.start_idx); + requested = count / sizeof(*pos); + + /* + * Make sure that the total requested number of entries is not greater + * than the total number of supported features allowed for userspace. + */ + if (start >= cfs->num_user_features) + return ERR_PTR(-EINVAL); + + requested = min_t(int, requested, cfs->num_user_features - start); + + out_size = sizeof(struct fwctl_rpc_cxl_out) + sizeof(*feat_out) + + requested * sizeof(*pos); + + struct fwctl_rpc_cxl_out *rpc_out __free(kvfree) = + kvzalloc(out_size, GFP_KERNEL); + if (!rpc_out) + return ERR_PTR(-ENOMEM); + + rpc_out->size = sizeof(*feat_out) + requested * sizeof(*pos); + feat_out = (struct cxl_mbox_get_sup_feats_out *)rpc_out->payload; + if (requested == 0) { + feat_out->num_entries = cpu_to_le16(requested); + feat_out->supported_feats = cpu_to_le16(cfs->num_user_features); + rpc_out->retval = CXL_MBOX_CMD_RC_SUCCESS; + *out_len = out_size; + return no_free_ptr(rpc_out); + } + + pos = &feat_out->ents[0]; + saved = &cfs->entries[0]; + + copied = 0; + for (int i = 0; i < cfs->num_features; i++, saved++) { + if (is_cxl_feature_exclusive(saved)) + continue; + + memcpy(pos, saved, sizeof(*saved)); + copied++; + if (copied == requested) + break; + pos++; + } + + feat_out->num_entries = cpu_to_le16(requested); + feat_out->supported_feats = cpu_to_le16(cfs->num_user_features); + rpc_out->retval = CXL_MBOX_CMD_RC_SUCCESS; + *out_len = out_size; + + return no_free_ptr(rpc_out); +} + +static bool cxlctl_validate_set_features(struct cxl_features_state *cfs, + const struct fwctl_rpc_cxl *rpc_in, + enum fwctl_rpc_scope scope) +{ + struct cxl_feat_entry *feat; + u16 effects, mask; + u32 flags; + + feat = get_support_feature_info(cfs, rpc_in); + if (IS_ERR(feat)) + return false; + + /* Ensure that the attribute is changeable */ + flags = le32_to_cpu(feat->flags); + if (!(flags & CXL_FEATURE_F_CHANGEABLE)) + return false; + + effects = le16_to_cpu(feat->effects); + /* Currently no user background command support */ + if (effects & CXL_CMD_BACKGROUND) + return false; + + mask = CXL_CMD_CONFIG_CHANGE_IMMEDIATE | + CXL_CMD_DATA_CHANGE_IMMEDIATE | + CXL_CMD_POLICY_CHANGE_IMMEDIATE | + CXL_CMD_LOG_CHANGE_IMMEDIATE; + if (effects & mask && scope >= FWCTL_RPC_DEBUG_WRITE_FULL) + return true; + + /* These effects supported for all scope */ + if ((effects & CXL_CMD_CONFIG_CHANGE_COLD_RESET || + (effects & CXL_CMD_EFFECTS_EXTEND && + (effects & CXL_CMD_CONFIG_CHANGE_CONV_RESET || + effects & CXL_CMD_CONFIG_CHANGE_CXL_RESET))) && + scope >= FWCTL_RPC_DEBUG_WRITE) + return true; + + return false; +} + +static struct cxl_mem_command * +cxlctl_get_valid_hw_command(struct cxl_features_state *cfs, + const struct fwctl_rpc_cxl *rpc_in, + enum fwctl_rpc_scope scope) +{ + struct cxl_mem_command *cmd; + + if (!cfs->num_features) + return ERR_PTR(-EOPNOTSUPP); + + cmd = cxl_find_feature_command_by_id(rpc_in->command_id); + if (!cmd) + return ERR_PTR(-ENOENT); + + if (!cxl_feature_enabled(cfs, cmd->opcode)) + return ERR_PTR(-EPERM); + + switch (cmd->opcode) { + case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: + case CXL_MBOX_OP_GET_FEATURE: + if (scope >= FWCTL_RPC_CONFIGURATION) + return cmd; + break; + case CXL_MBOX_OP_SET_FEATURE: + if (cxlctl_validate_set_features(cfs, rpc_in, scope)) + return cmd; + break; + default: + return ERR_PTR(-EINVAL); + } + + return ERR_PTR(-EINVAL); +} + +static void *cxlctl_handle_commands(struct cxl_features_state *cfs, + const struct fwctl_rpc_cxl *rpc_in, + size_t *out_len, struct cxl_mem_command *cmd) +{ + switch (cmd->opcode) { + case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: + return cxlctl_get_supported_features(cfs, rpc_in, out_len); + case CXL_MBOX_OP_GET_FEATURE: + case CXL_MBOX_OP_SET_FEATURE: + default: + return ERR_PTR(-EOPNOTSUPP); + } +} + static void *cxlctl_fw_rpc(struct fwctl_uctx *uctx, enum fwctl_rpc_scope scope, - void *rpc_in, size_t in_len, size_t *out_len) + void *in, size_t in_len, size_t *out_len) { - /* Place holder */ - return ERR_PTR(-EOPNOTSUPP); + struct fwctl_device *fwctl = uctx->fwctl; + struct cxl_features_state *cfs = to_cxl_features_state(fwctl); + const struct fwctl_rpc_cxl *rpc_in = in; + struct cxl_mem_command *cmd; + + cmd = cxlctl_get_valid_hw_command(cfs, rpc_in, scope); + if (IS_ERR(cmd)) + return (void *)cmd; + + return cxlctl_handle_commands(cfs, rpc_in, out_len, cmd); } static const struct fwctl_ops cxlctl_ops = { diff --git a/include/cxl/features.h b/include/cxl/features.h index 9f234659ca2d..b5dacc1665c7 100644 --- a/include/cxl/features.h +++ b/include/cxl/features.h @@ -69,6 +69,7 @@ size_t cxl_get_feature(struct cxl_features *features, const uuid_t feat_uuid, int cxl_set_feature(struct cxl_features *features, const uuid_t feat_uuid, u8 feat_version, void *feat_data, size_t feat_data_size, u32 feat_flag, u16 offset, u16 *return_code); +struct cxl_mem_command *cxl_find_feature_command_by_id(u32 id); bool is_cxl_feature_exclusive(struct cxl_feat_entry *entry); #endif diff --git a/include/uapi/cxl/features.h b/include/uapi/cxl/features.h index 18b74252058a..55fa5b2a5d17 100644 --- a/include/uapi/cxl/features.h +++ b/include/uapi/cxl/features.h @@ -14,6 +14,19 @@ #include #endif +/* CXL spec r3.2 Table 8-87 command effects */ +#define CXL_CMD_CONFIG_CHANGE_COLD_RESET BIT(0) +#define CXL_CMD_CONFIG_CHANGE_IMMEDIATE BIT(1) +#define CXL_CMD_DATA_CHANGE_IMMEDIATE BIT(2) +#define CXL_CMD_POLICY_CHANGE_IMMEDIATE BIT(3) +#define CXL_CMD_LOG_CHANGE_IMMEDIATE BIT(4) +#define CXL_CMD_SECURITY_STATE_CHANGE BIT(5) +#define CXL_CMD_BACKGROUND BIT(6) +#define CXL_CMD_BGCMD_ABORT_SUPPORTED BIT(7) +#define CXL_CMD_EFFECTS_EXTEND BIT(9) +#define CXL_CMD_CONFIG_CHANGE_CONV_RESET BIT(10) +#define CXL_CMD_CONFIG_CHANGE_CXL_RESET BIT(11) + /* Get Supported Features (0x500h) CXL r3.1 8.2.9.6.1 */ struct cxl_mbox_get_sup_feats_in { __le32 count; diff --git a/include/uapi/fwctl/cxl.h b/include/uapi/fwctl/cxl.h index 79b822dbfafd..ad23c65ce724 100644 --- a/include/uapi/fwctl/cxl.h +++ b/include/uapi/fwctl/cxl.h @@ -28,4 +28,33 @@ enum feature_cmds { struct fwctl_info_cxl { __u32 cmd_mask; }; + +/** + * struct fwctl_rpc_cxl - ioctl(FWCTL_RPC) input for CXL + * @command_id: the defined command id by 'enum feature_cmds' + * @flags: Flags for the command (input). + * @op_size: Size of input payload. + * @reserved1: Reserved. Must be 0s. + * @in_payload: User address of the hardware op input structure + */ +struct fwctl_rpc_cxl { + __u32 command_id; + __u32 flags; + __u32 op_size; + __u32 reserved1; + __aligned_u64 in_payload; +}; + +/** + * struct fwctl_rpc_cxl_out - ioctl(FWCTL_RPC) output for CXL + * @size: Size of the output payload + * @retval: Return value from device + * @payload: Return data from device + */ +struct fwctl_rpc_cxl_out { + __u32 size; + __u32 retval; + __u8 payload[] __counted_by(size); +}; + #endif From patchwork Wed Jan 22 23:50:46 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947724 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E5111C5D54 for ; Wed, 22 Jan 2025 23:52:23 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589943; cv=none; b=lLnx5KgixsW6WMGgIHh6JPcGix+U2SFcTXjjQi2ikB05Zydsbd6RY3BREZPkuMuDbsq3YZfmkP9rrLsRvL8fUf1yfC/Z5pDjfRjKNAs/oO7ebqOzzcsrvuVGYBBVKyZIjkQ7TrpIjb2vLjvXpLtUvBeWfRmy5dzjORAu88csCFw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589943; c=relaxed/simple; bh=HlJAVLOEDjjWh2ADTL6fk5FrTEsD6dYTpGhEIuq4FXg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eaEStjGUNmrn/ISmDXkoHt1UM893XHK+qJ6L2TFGhtamazosbvJXTEosDvGtxyKwaUWaZhmCqlef218wUngf6uyOl1KvvAWdaSnVX2k1TaXkMNmA/D2AA4A3dqaia7etnjxkEVRb7GPZRdsbpr/PlgH+o1i/qbz1QsdgbeV9/5s= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id CDD3CC4CED2; Wed, 22 Jan 2025 23:52:22 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 15/19] cxl: Add support to handle user feature commands for get feature Date: Wed, 22 Jan 2025 16:50:46 -0700 Message-ID: <20250122235159.2716036-16-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add helper function to parse the user data from fwctl RPC ioctl and send the parsed input parameters to cxl_get_feature() call. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams --- drivers/cxl/features.c | 46 ++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c index e65fc8479d21..7c726f1512d0 100644 --- a/drivers/cxl/features.c +++ b/drivers/cxl/features.c @@ -146,6 +146,51 @@ static void *cxlctl_get_supported_features(struct cxl_features_state *cfs, return no_free_ptr(rpc_out); } +static void *cxlctl_get_feature(struct cxl_features_state *cfs, + const struct fwctl_rpc_cxl *rpc_in, + size_t *out_len) +{ + struct cxl_mbox_get_feat_in feat_in; + u16 offset, count, return_code; + size_t out_size = *out_len; + void *feat_out; + + if (rpc_in->op_size != sizeof(feat_in)) + return ERR_PTR(-EINVAL); + + if (copy_from_user(&feat_in, u64_to_user_ptr(rpc_in->in_payload), + rpc_in->op_size)) + return ERR_PTR(-EFAULT); + + offset = le16_to_cpu(feat_in.offset); + count = le16_to_cpu(feat_in.count); + + if (!count) + return ERR_PTR(-EINVAL); + + struct fwctl_rpc_cxl_out *rpc_out __free(kvfree) = + kvzalloc(out_size, GFP_KERNEL); + if (!rpc_out) + return ERR_PTR(-ENOMEM); + + feat_out = (void *)rpc_out->payload; + out_size = cxl_get_feature(cfs->features, feat_in.uuid, + feat_in.selection, feat_out, + count, offset, &return_code); + *out_len = sizeof(struct fwctl_rpc_cxl_out); + if (!out_size) { + rpc_out->size = 0; + rpc_out->retval = return_code; + return no_free_ptr(rpc_out); + } + + rpc_out->size = out_size; + rpc_out->retval = CXL_MBOX_CMD_RC_SUCCESS; + *out_len += out_size; + + return no_free_ptr(rpc_out); +} + static bool cxlctl_validate_set_features(struct cxl_features_state *cfs, const struct fwctl_rpc_cxl *rpc_in, enum fwctl_rpc_scope scope) @@ -228,6 +273,7 @@ static void *cxlctl_handle_commands(struct cxl_features_state *cfs, case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: return cxlctl_get_supported_features(cfs, rpc_in, out_len); case CXL_MBOX_OP_GET_FEATURE: + return cxlctl_get_feature(cfs, rpc_in, out_len); case CXL_MBOX_OP_SET_FEATURE: default: return ERR_PTR(-EOPNOTSUPP); From patchwork Wed Jan 22 23:50:47 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947725 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 711DE1C5D54 for ; Wed, 22 Jan 2025 23:52:24 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589944; cv=none; b=sIMe06ToZmVQSsh0hMxTiq5sluCBZ6flvuifnIrY71b0NC/E9ayXTxgDozslTUC1cb+wsNAW0d96PY4mXg40jFc55USJ3S9pDRpfY3cQ2iwEQeD0OYMBoHHNgYsvTLajlnR/0Gol9YREYORZMglDWD3Qt3mrNE1QPIW9gTvRzS4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589944; c=relaxed/simple; bh=38RanBwofiKJJw5GPskz1TGMTfcZlJjTEFtaRtc9zGY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=Z5l2qWelevq1HLapqQw0X/T6WkHe0iY+Kr8qt57NNarLRCzaWFuOXY0Dsj13rrAXKgMv9wFsrhCtDjLnGSmC0Uf7xhYVDH9igOltB4BF7y9Iy6Wnc/gThcs9v6/8Q5SHqzfbZAKTtRHE9DVIOrXzLyVNiIZGg94wLSGyoUHyCZY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3A919C4CED2; Wed, 22 Jan 2025 23:52:24 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 16/19] cxl: Add support to handle user feature commands for set feature Date: Wed, 22 Jan 2025 16:50:47 -0700 Message-ID: <20250122235159.2716036-17-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add helper function to parse the user data from fwctl RPC ioctl and send the parsed input parameters to cxl_set_feature() call. Signed-off-by: Dave Jiang Reviewed-by: Jonathan Cameron Reviewed-by: Dan Williams --- drivers/cxl/features.c | 48 +++++++++++++++++++++++++++++++++++++ include/uapi/cxl/features.h | 5 ++++ 2 files changed, 53 insertions(+) diff --git a/drivers/cxl/features.c b/drivers/cxl/features.c index 7c726f1512d0..4c5c93278e58 100644 --- a/drivers/cxl/features.c +++ b/drivers/cxl/features.c @@ -191,6 +191,53 @@ static void *cxlctl_get_feature(struct cxl_features_state *cfs, return no_free_ptr(rpc_out); } +static void *cxlctl_set_feature(struct cxl_features_state *cfs, + const struct fwctl_rpc_cxl *rpc_in, + size_t *out_len) +{ + size_t out_size, data_size; + u16 offset, return_code; + u32 flags; + int rc; + + if (rpc_in->op_size <= sizeof(struct cxl_mbox_set_feat_hdr)) + return ERR_PTR(-EINVAL); + + struct cxl_mbox_set_feat_in *feat_in __free(kvfree) = + kvzalloc(rpc_in->op_size, GFP_KERNEL); + if (!feat_in) + return ERR_PTR(-ENOMEM); + + if (copy_from_user(feat_in, u64_to_user_ptr(rpc_in->in_payload), + rpc_in->op_size)) + return ERR_PTR(-EFAULT); + + offset = le16_to_cpu(feat_in->hdr.offset); + flags = le32_to_cpu(feat_in->hdr.flags); + out_size = *out_len; + + struct fwctl_rpc_cxl_out *rpc_out __free(kvfree) = + kvzalloc(out_size, GFP_KERNEL); + if (!rpc_out) + return ERR_PTR(-ENOMEM); + + rpc_out->size = 0; + + data_size = rpc_in->op_size - sizeof(feat_in->hdr); + rc = cxl_set_feature(cfs->features, feat_in->hdr.uuid, + feat_in->hdr.version, feat_in->data, + data_size, flags, offset, &return_code); + if (rc) { + rpc_out->retval = return_code; + return no_free_ptr(rpc_out); + } + + rpc_out->retval = CXL_MBOX_CMD_RC_SUCCESS; + *out_len = sizeof(*rpc_out); + + return no_free_ptr(rpc_out); +} + static bool cxlctl_validate_set_features(struct cxl_features_state *cfs, const struct fwctl_rpc_cxl *rpc_in, enum fwctl_rpc_scope scope) @@ -275,6 +322,7 @@ static void *cxlctl_handle_commands(struct cxl_features_state *cfs, case CXL_MBOX_OP_GET_FEATURE: return cxlctl_get_feature(cfs, rpc_in, out_len); case CXL_MBOX_OP_SET_FEATURE: + return cxlctl_set_feature(cfs, rpc_in, out_len); default: return ERR_PTR(-EOPNOTSUPP); } diff --git a/include/uapi/cxl/features.h b/include/uapi/cxl/features.h index 55fa5b2a5d17..67ed8b08f7e9 100644 --- a/include/uapi/cxl/features.h +++ b/include/uapi/cxl/features.h @@ -127,4 +127,9 @@ struct cxl_mbox_set_feat_hdr { __u8 rsvd[9]; } __attribute__ ((__packed__)); +struct cxl_mbox_set_feat_in { + struct cxl_mbox_set_feat_hdr hdr; + __u8 data[]; +} __attribute__ ((__packed__)); + #endif From patchwork Wed Jan 22 23:50:48 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947726 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CE2F61CD1E4 for ; Wed, 22 Jan 2025 23:52:25 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589945; cv=none; b=RYpf8XqJpybHA92S0wgAIapPMOZdVxHkDQ8dxngnTlqyzAgwDfsjSlzSRJCAXuRErxJahFqAW7qoYC1noT3vKFbV3AHjW0nPceKs4fi6MQieBUX2xY5qE/dbdCUyhrwN0T4JnvOyNBZ9BPESwfrC6tB9ujF+JsomtlxBnp0QhKE= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589945; c=relaxed/simple; bh=X5NjDRcDvcbL/JwgOg2Afb8v9pAhVyRLi5mrQcgKoeU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=eNlk43QezcHx8YbdzyqyadiKx2JkPDrAHT55rYIPm6mgzAKCEy1q8aNRmhtaJrGVo9ktbPzDJtiUJaRXu+lLR4oIb2UZ4OIWJeJQNWPmmZ0slGsDsg69sNV4usRQS14yTpUzyQET0knsbcaGNDPPDToH4t1CiduvTq7nb4Ifswo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 90184C4CED2; Wed, 22 Jan 2025 23:52:25 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 17/19] cxl/test: Add Get Feature support to cxl_test Date: Wed, 22 Jan 2025 16:50:48 -0700 Message-ID: <20250122235159.2716036-18-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add emulation of Get Feature command to cxl_test. The feature for device patrol scrub is returned by the emulation code. This is the only feature currently supported by cxl_test. It returns the information for the device patrol scrub feature. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Reviewed-by: Dan Williams --- tools/testing/cxl/test/mem.c | 56 ++++++++++++++++++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 2b111142b2c0..5b7b0420e3a0 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -49,6 +49,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_SUPPORTED_FEATURES), .effect = CXL_CMD_EFFECT_NONE, }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_GET_FEATURE), + .effect = CXL_CMD_EFFECT_NONE, + }, { .opcode = cpu_to_le16(CXL_MBOX_OP_IDENTIFY), .effect = CXL_CMD_EFFECT_NONE, @@ -150,6 +154,10 @@ struct mock_event_store { u32 ev_status; }; +struct vendor_test_feat { + __le32 data; +} __packed; + struct cxl_mockmem_data { void *lsa; void *fw; @@ -166,6 +174,7 @@ struct cxl_mockmem_data { u8 event_buf[SZ_4K]; u64 timestamp; unsigned long sanitize_timeout; + struct vendor_test_feat test_feat; }; static struct mock_event_log *event_find_log(struct device *dev, int log_type) @@ -1361,6 +1370,44 @@ static void fill_feature_vendor_test(struct cxl_feat_entry *feat) CXL_CMD_EFFECTS_EXTEND); } +static int mock_get_test_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct vendor_test_feat *output = cmd->payload_out; + struct cxl_mbox_get_feat_in *input = cmd->payload_in; + u16 offset = le16_to_cpu(input->offset); + u16 count = le16_to_cpu(input->count); + u8 *ptr; + + if (offset > sizeof(*output)) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + if (offset + count > sizeof(*output)) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + ptr = (u8 *)&mdata->test_feat + offset; + memcpy((u8 *)output + offset, ptr, count); + + return 0; +} + +static int mock_get_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_get_feat_in *input = cmd->payload_in; + + if (uuid_equal(&input->uuid, &CXL_VENDOR_FEATURE_TEST)) + return mock_get_test_feature(mdata, cmd); + + cmd->return_code = CXL_MBOX_CMD_RC_UNSUPPORTED; + + return -EOPNOTSUPP; +} + static int mock_get_supported_features(struct cxl_mockmem_data *mdata, struct cxl_mbox_cmd *cmd) { @@ -1491,6 +1538,9 @@ static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, case CXL_MBOX_OP_GET_SUPPORTED_FEATURES: rc = mock_get_supported_features(mdata, cmd); break; + case CXL_MBOX_OP_GET_FEATURE: + rc = mock_get_feature(mdata, cmd); + break; default: break; } @@ -1553,6 +1603,11 @@ static int cxl_mock_setup_features(struct cxl_memdev_state *mds) return 0; } +static void cxl_mock_test_feat_init(struct cxl_mockmem_data *mdata) +{ + mdata->test_feat.data = cpu_to_le32(0xdeadbeef); +} + static int cxl_mock_mem_probe(struct platform_device *pdev) { struct device *dev = &pdev->dev; @@ -1644,6 +1699,7 @@ static int cxl_mock_mem_probe(struct platform_device *pdev) return rc; cxl_mem_get_event_records(mds, CXLDEV_EVENT_STATUS_ALL); + cxl_mock_test_feat_init(mdata); return 0; } From patchwork Wed Jan 22 23:50:49 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947727 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 915361CD1ED for ; Wed, 22 Jan 2025 23:52:27 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589947; cv=none; b=svje7gpkGU+Yy/QUpD+OMcakWVzi/EmO3z2E0nkMeGN8Fund8s3ySOdi9o9YUP5B/P090PnD+Zpx4NAngAt2TywhcfUG64mDenwVxIPVQKH8qTGA5otCHmcQgIpiIovXJvQIJ2AH6uDvvOExOJ0oWT1tiugLP4TuMdS83je2erU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589947; c=relaxed/simple; bh=KUy5UpMdcpjFl0z1dzwDYcR96s+vUAx0TNxrpykfiL8=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=tdwWbU/Mwo9SIeSP1YXu/Oc88TTFhXOlo7mLMJFhSd6K+Bk7fkBCTa7FtJ54Wi70MI2qMeZZwsP6hXuyU4BT5PyWfqeUyi7Lh+6+nCYHG5rAiwyQ6t2SDRpKsvtNWM9OVzdo1Bdj3yQMiyowwQcmpNE6y/V2ogOpT4aEPcLJOSE= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id E4C7DC4CED2; Wed, 22 Jan 2025 23:52:26 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 18/19] cxl/test: Add Set Feature support to cxl_test Date: Wed, 22 Jan 2025 16:50:49 -0700 Message-ID: <20250122235159.2716036-19-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add emulation to support Set Feature mailbox command to cxl_test. The only feature supported is the device patrol scrub feature. The set feature allows activation of patrol scrub for the cxl_test emulated device. The command does not support partial data transfer even though the spec allows it. This restriction is to reduce complexity of the emulation given the patrol scrub feature is very minimal. Reviewed-by: Jonathan Cameron Signed-off-by: Dave Jiang Reviewed-by: Dan Williams --- tools/testing/cxl/test/mem.c | 50 ++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/tools/testing/cxl/test/mem.c b/tools/testing/cxl/test/mem.c index 5b7b0420e3a0..f1d775968607 100644 --- a/tools/testing/cxl/test/mem.c +++ b/tools/testing/cxl/test/mem.c @@ -53,6 +53,10 @@ static struct cxl_cel_entry mock_cel[] = { .opcode = cpu_to_le16(CXL_MBOX_OP_GET_FEATURE), .effect = CXL_CMD_EFFECT_NONE, }, + { + .opcode = cpu_to_le16(CXL_MBOX_OP_SET_FEATURE), + .effect = cpu_to_le16(EFFECT(CONF_CHANGE_IMMEDIATE)), + }, { .opcode = cpu_to_le16(CXL_MBOX_OP_IDENTIFY), .effect = CXL_CMD_EFFECT_NONE, @@ -1408,6 +1412,49 @@ static int mock_get_feature(struct cxl_mockmem_data *mdata, return -EOPNOTSUPP; } +static int mock_set_test_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_set_feat_in *input = cmd->payload_in; + struct vendor_test_feat *test = (struct vendor_test_feat *)input->data; + u32 action; + + action = FIELD_GET(CXL_SET_FEAT_FLAG_DATA_TRANSFER_MASK, + le32_to_cpu(input->hdr.flags)); + /* + * While it is spec compliant to support other set actions, it is not + * necessary to add the complication in the emulation currently. Reject + * anything besides full xfer. + */ + if (action != CXL_SET_FEAT_FLAG_FULL_DATA_TRANSFER) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + /* Offset should be reserved when doing full transfer */ + if (input->hdr.offset) { + cmd->return_code = CXL_MBOX_CMD_RC_INPUT; + return -EINVAL; + } + + memcpy(&mdata->test_feat.data, &test->data, sizeof(u32)); + + return 0; +} + +static int mock_set_feature(struct cxl_mockmem_data *mdata, + struct cxl_mbox_cmd *cmd) +{ + struct cxl_mbox_set_feat_in *input = cmd->payload_in; + + if (uuid_equal(&input->hdr.uuid, &CXL_VENDOR_FEATURE_TEST)) + return mock_set_test_feature(mdata, cmd); + + cmd->return_code = CXL_MBOX_CMD_RC_UNSUPPORTED; + + return -EOPNOTSUPP; +} + static int mock_get_supported_features(struct cxl_mockmem_data *mdata, struct cxl_mbox_cmd *cmd) { @@ -1541,6 +1588,9 @@ static int cxl_mock_mbox_send(struct cxl_mailbox *cxl_mbox, case CXL_MBOX_OP_GET_FEATURE: rc = mock_get_feature(mdata, cmd); break; + case CXL_MBOX_OP_SET_FEATURE: + rc = mock_set_feature(mdata, cmd); + break; default: break; } From patchwork Wed Jan 22 23:50:50 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dave Jiang X-Patchwork-Id: 13947728 Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A2CE61CAA86 for ; Wed, 22 Jan 2025 23:52:28 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589948; cv=none; b=A81l8InAlbHXPNmqyJEnoEMUD36IVs87WP+Mu3P6bJGdwey7hl8pf8Y7EEL9RQmXc5OqGzAlt++E7yf9suC5GirKJ3ksGbBH2gc8RtCLnvayi3xid13rqqs5TilebEjjMb1VqYDPDmlZ2uljEMxQThGHa148aMzCQqQ8cw6khaI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737589948; c=relaxed/simple; bh=WYqzviXH/PktliwlNsAe+Brprt2iNB8TNcM+lDyC5gE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ZPVjGfQl0PslXK38/7fTCn832cWzS0k05krRYGOe6UZQh0FdWomSPTRPLyq5T4m/26HY/NAtntdPBJ0N8uXyxwHOPnj6g+oADyFHsGSUsfqVl2GmhgK+3l4OAE2Ws5zoemOO64mv34aXT6tyvuJZ5O637nZY7TtOeCT2jZIdjms= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 Received: by smtp.kernel.org (Postfix) with ESMTPSA id 566F4C4CED2; Wed, 22 Jan 2025 23:52:28 +0000 (UTC) From: Dave Jiang To: linux-cxl@vger.kernel.org Cc: dan.j.williams@intel.com, ira.weiny@intel.com, vishal.l.verma@intel.com, alison.schofield@intel.com, Jonathan.Cameron@huawei.com, dave@stgolabs.net, jgg@nvidia.com, shiju.jose@huawei.com Subject: [PATCH v1 19/19] fwctl/cxl: Add documentation to FWCTL CXL Date: Wed, 22 Jan 2025 16:50:50 -0700 Message-ID: <20250122235159.2716036-20-dave.jiang@intel.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: <20250122235159.2716036-1-dave.jiang@intel.com> References: <20250122235159.2716036-1-dave.jiang@intel.com> Precedence: bulk X-Mailing-List: linux-cxl@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add policy and operational documentation for FWCTL CXL. Signed-off-by: Dave Jiang --- .../userspace-api/fwctl/fwctl-cxl.rst | 81 +++++++++++++++++++ Documentation/userspace-api/fwctl/index.rst | 1 + 2 files changed, 82 insertions(+) create mode 100644 Documentation/userspace-api/fwctl/fwctl-cxl.rst diff --git a/Documentation/userspace-api/fwctl/fwctl-cxl.rst b/Documentation/userspace-api/fwctl/fwctl-cxl.rst new file mode 100644 index 000000000000..f774bf2ecb5f --- /dev/null +++ b/Documentation/userspace-api/fwctl/fwctl-cxl.rst @@ -0,0 +1,81 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================ +fwctl cxl driver +================ + +:Author: Dave Jiang + +Overview +======== + +The CXL spec defines a set of commands that can be issued to the mailbox of a +CXL device or switch. It also left room for vendor specific commands to be +issued to the mailbox as well. fwctl provides a path to issue a set of allowed +mailbox commands from user space to the device moderated by the kernel driver. + +While there are a large set of mailbox commands, only the feature related +commands will be allowed to be issued through fwctl initially. No background +commands will be supported at this time. + +CXL spec r3.1 8.2.9.6.1 Get Supported Features (Opcode 0500h) +CXL spec r3.1 8.2.9.6.2 Get Feature (Opcode 0501h) +CXL spec r3.1 8.2.9.6.3 Set Feature (Opcode 0502h) + +The "Get Supported Features" return data may be filtered by the kernel driver to +drop any features that are forbidden by the kernel or being exclusively used by +the kernel. The "Get Supported Features" command and the "Get Features" falls +under the fwctl policy of FWCTL_RPC_CONFIGURATION. + +For "Set Feature" command, the access policy currently is broken down into two +categories depending on the set feature effects reported by the device. If the +set feature will cause immediate change to the device, the fwctl access policy +must be FWCTL_RPC_DEBUG_WRITE_FULL. The effects for this level are +"immediate config change", "immediate data change", "immediate policy change", +or "immediate log change" for the set effects mask. If the effects are "config +change with cold reset" or "config change with conventional reset", then the +fwctl access policy must be FWCTL_RPC_DEBUG_WRITE or higher. + +fwctl cxl User API +================== + +.. kernel-doc:: include/uapi/fwctl/cxl.h +.. kernel-doc:: include/uapi/cxl/features.h + +1. Driver info query +-------------------- + +First step for the app is to issue the ioctl(FWCTL_CMD_INFO) in order to retrieve +the information from the driver. A ``struct fwctl_info`` needs to be filled out +with the ``fwctl_info.out_device_type`` set to ``FWCTL_DEVICE_TYPE_CXL``. The +return data should be ``struct fwctl_info_cxl`` that contains a command mask +that indicates which of the commands are supported by the driver. + +2. Send hardware command +------------------------ + +Next step is to send the 'Get Supported Features' command to the driver from +user space via ioctl(FWCTL_RPC). A ``struct fwctl_rpc_cxl`` should be pointed to +by ``fwctl_rpc.in``. ``struct fwctl_rpc_cxl.in_payload`` should point to +the hardware input structure that is defined by the CXL spec. ``fwctl_rpc.out`` +points to the buffer that contains a ``struct fwctl_rpc_cxl_out`` that includes +the hardware output data inlined as ``fwctl_rpc_cxl_out.payload``. This command +can be called twice. First time to retrieve the number of features supported. +A second time to retrieve the specific feature details. + +After getting the specific feature details, a get/set feature command can be +appropriately programmed and sent. For a "set feature" command, the retrieved +feature info contains an effects field that details the resulting +"set feature" command will trigger. That should inform the user whether +the system is configured to allowed the "set feature" command or not. + +It is recommended to take a look at CXL CLI test directory for a detailed +user code example on how to exercise this path. + +fwctl cxl Kernel API +==================== + +.. kernel-doc:: drivers/cxl/core/features.c +.. kernel-doc:: drivers/cxl/features.c + :export: +.. kernel-doc:: include/cxl/features.h diff --git a/Documentation/userspace-api/fwctl/index.rst b/Documentation/userspace-api/fwctl/index.rst index 06959fbf1547..d9d40a468a31 100644 --- a/Documentation/userspace-api/fwctl/index.rst +++ b/Documentation/userspace-api/fwctl/index.rst @@ -10,3 +10,4 @@ to securely construct and execute RPCs inside device firmware. :maxdepth: 1 fwctl + fwctl-cxl