From patchwork Tue Feb 22 19:56:03 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 12755866 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A07E3C433EF for ; Tue, 22 Feb 2022 19:52:39 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235328AbiBVTxE (ORCPT ); Tue, 22 Feb 2022 14:53:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53896 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235327AbiBVTxC (ORCPT ); Tue, 22 Feb 2022 14:53:02 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id C4F32C5580 for ; Tue, 22 Feb 2022 11:52:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645559555; x=1677095555; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=Y/VSSTrah+fp8sLz2UCfMe2qslgyf4HT2fdartoaU60=; b=LD5tXQKyR4vzrZGvakQqoNZNO/QjkBayJGttjPQxc6AgjwHKJHhn/Fhf wQVhV7psN7MhK/wI2q7cq3BnLwr9/icVxNXYi0pWHkUBND88xshBLqpMj NVHbeBr3rMn7UhZ0d3RxJSJSGYH2uOZoiUeybtXLINzGkY7UXTbxdHnLI I7ypUX7S54/Uyt2E1cgLd7bDawt0AoHWBiygDVpggbAf3/VlLRZpQq04b Q3j51sn+XxzzNXpT4bz22jdOGCincjKEHMWo6/egv8BITz5qc3lx2/F+x GcL0LAm5+fKAdyK7AFs1qg9YoMB/tdcgEB66q6Nj1E+63Pr96slfMDJVa w==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="250624953" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="250624953" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:35 -0800 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="683637800" Received: from alison-desk.jf.intel.com (HELO localhost) ([10.54.74.41]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:35 -0800 From: alison.schofield@intel.com To: Ben Widawsky , Dan Williams , Ira Weiny , Vishal Verma Cc: Alison Schofield , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Subject: [ndctl PATCH v6 1/6] libcxl: add GET_PARTITION_INFO mailbox command and accessors Date: Tue, 22 Feb 2022 11:56:03 -0800 Message-Id: <6cd7fffe1a95c9a1bc2239cb342067df564401a5.1645558189.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield The CXL PMEM provisioning model depends upon the values reported in the CXL GET_PARTITION_INFO mailbox command when changing the partitioning between volatile and persistent capacity. Add libcxl APIs to create a new GET_PARTITION_INFO mailbox command, the command output data structure (privately), and accessor APIs to return the fields in the partition info output. Per the CXL 2.0 specification, devices report partition capacities as multiples of 256MB. Define and use a capacity multiplier to convert the raw data into bytes for user consumption. Use byte format as the norm for all capacity values produced or consumed using CXL Mailbox commands. Signed-off-by: Alison Schofield Reviewed-by: Dan Williams --- Documentation/cxl/lib/libcxl.txt | 1 + cxl/lib/libcxl.c | 66 ++++++++++++++++++++++++++++++++ cxl/lib/libcxl.sym | 5 +++ cxl/lib/private.h | 10 +++++ cxl/libcxl.h | 5 +++ util/size.h | 1 + 6 files changed, 88 insertions(+) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index 4392b47e6ece..a6986abafce3 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -131,6 +131,7 @@ int cxl_memdev_read_label(struct cxl_memdev *memdev, void *buf, size_t length, size_t offset); int cxl_memdev_write_label(struct cxl_memdev *memdev, void *buf, size_t length, size_t offset); +struct cxl_cmd *cxl_cmd_new_get_partition(struct cxl_memdev *memdev); ---- diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index e0b443fb0aa4..4557a71de845 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -1985,6 +1985,11 @@ static int cxl_cmd_validate_status(struct cxl_cmd *cmd, u32 id) return 0; } +static uint64_t cxl_capacity_to_bytes(leint64_t size) +{ + return le64_to_cpu(size) * CXL_CAPACITY_MULTIPLIER; +} + /* Helpers for health_info fields (no endian conversion) */ #define cmd_get_field_u8(cmd, n, N, field) \ do { \ @@ -2371,6 +2376,67 @@ CXL_EXPORT ssize_t cxl_cmd_read_label_get_payload(struct cxl_cmd *cmd, return length; } +CXL_EXPORT struct cxl_cmd *cxl_cmd_new_get_partition(struct cxl_memdev *memdev) +{ + return cxl_cmd_new_generic(memdev, + CXL_MEM_COMMAND_ID_GET_PARTITION_INFO); +} + +static struct cxl_cmd_get_partition * +cmd_to_get_partition(struct cxl_cmd *cmd) +{ + if (cxl_cmd_validate_status(cmd, CXL_MEM_COMMAND_ID_GET_PARTITION_INFO)) + return NULL; + + if (!cmd) + return NULL; + return cmd->output_payload; +} + +CXL_EXPORT unsigned long long +cxl_cmd_partition_get_active_volatile_size(struct cxl_cmd *cmd) +{ + struct cxl_cmd_get_partition *c; + + c = cmd_to_get_partition(cmd); + if (!c) + return ULLONG_MAX; + return cxl_capacity_to_bytes(c->active_volatile); +} + +CXL_EXPORT unsigned long long +cxl_cmd_partition_get_active_persistent_size(struct cxl_cmd *cmd) +{ + struct cxl_cmd_get_partition *c; + + c = cmd_to_get_partition(cmd); + if (!c) + return ULLONG_MAX; + return cxl_capacity_to_bytes(c->active_persistent); +} + +CXL_EXPORT unsigned long long +cxl_cmd_partition_get_next_volatile_size(struct cxl_cmd *cmd) +{ + struct cxl_cmd_get_partition *c; + + c = cmd_to_get_partition(cmd); + if (!c) + return ULLONG_MAX; + return cxl_capacity_to_bytes(c->next_volatile); +} + +CXL_EXPORT unsigned long long +cxl_cmd_partition_get_next_persistent_size(struct cxl_cmd *cmd) +{ + struct cxl_cmd_get_partition *c; + + c = cmd_to_get_partition(cmd); + if (!c) + return ULLONG_MAX; + return cxl_capacity_to_bytes(c->next_persistent); +} + CXL_EXPORT int cxl_cmd_submit(struct cxl_cmd *cmd) { struct cxl_memdev *memdev = cmd->memdev; diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index e56a2bf96e22..509e62daad4b 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -155,4 +155,9 @@ global: cxl_dport_get_port; cxl_port_get_dport_by_memdev; cxl_dport_maps_memdev; + cxl_cmd_new_get_partition; + cxl_cmd_partition_get_active_volatile_size; + cxl_cmd_partition_get_active_persistent_size; + cxl_cmd_partition_get_next_volatile_size; + cxl_cmd_partition_get_next_persistent_size; } LIBCXL_1; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index f483c308a8b5..7f3a562a7c8e 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -7,6 +7,7 @@ #include #include #include +#include #define CXL_EXPORT __attribute__ ((visibility("default"))) @@ -185,6 +186,15 @@ struct cxl_cmd_get_health_info { le32 pmem_errors; } __attribute__((packed)); +struct cxl_cmd_get_partition { + le64 active_volatile; + le64 active_persistent; + le64 next_volatile; + le64 next_persistent; +} __attribute__((packed)); + +#define CXL_CAPACITY_MULTIPLIER SZ_256M + /* CXL 2.0 8.2.9.5.3 Byte 0 Health Status */ #define CXL_CMD_HEALTH_INFO_STATUS_MAINTENANCE_NEEDED_MASK BIT(0) #define CXL_CMD_HEALTH_INFO_STATUS_PERFORMANCE_DEGRADED_MASK BIT(1) diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 3b2293bbee84..2c0a8d199f0a 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -242,6 +242,11 @@ ssize_t cxl_cmd_read_label_get_payload(struct cxl_cmd *cmd, void *buf, unsigned int length); struct cxl_cmd *cxl_cmd_new_write_label(struct cxl_memdev *memdev, void *buf, unsigned int offset, unsigned int length); +struct cxl_cmd *cxl_cmd_new_get_partition(struct cxl_memdev *memdev); +unsigned long long cxl_cmd_partition_get_active_volatile_size(struct cxl_cmd *cmd); +unsigned long long cxl_cmd_partition_get_active_persistent_size(struct cxl_cmd *cmd); +unsigned long long cxl_cmd_partition_get_next_volatile_size(struct cxl_cmd *cmd); +unsigned long long cxl_cmd_partition_get_next_persistent_size(struct cxl_cmd *cmd); #ifdef __cplusplus } /* extern "C" */ diff --git a/util/size.h b/util/size.h index a0f3593df6b5..e72467f0f8b0 100644 --- a/util/size.h +++ b/util/size.h @@ -15,6 +15,7 @@ #define SZ_4M 0x00400000 #define SZ_16M 0x01000000 #define SZ_64M 0x04000000 +#define SZ_256M 0x10000000 #define SZ_1G 0x40000000 #define SZ_1T 0x10000000000ULL From patchwork Tue Feb 22 19:56:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 12755868 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B9E1C4332F for ; Tue, 22 Feb 2022 19:52:40 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235327AbiBVTxE (ORCPT ); Tue, 22 Feb 2022 14:53:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53898 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235318AbiBVTxC (ORCPT ); Tue, 22 Feb 2022 14:53:02 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31CB2C5581 for ; Tue, 22 Feb 2022 11:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645559557; x=1677095557; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=BTG5Smbf2n0RnE6NAt5Gg8jbLwJZPj7ZBnYfgHaACFI=; b=mJtCx4guzQUpwuHPyVeQGtUw42AA64ObqmUBdhI5iImeM31A6vM3KRsW BGozY3rO9KmzlNJQqdl9QObwB31JTGB/nu09yfc4qU/SAq1fwCCGfXUN9 +uf/JSd/svTErSSLz79OlbF5B2Ok3XG5w9JfgBWptqIrqUQb3wnfrf66R jWg0CGz2jTRkMHtxlZOsP2cH8o/wmmo8ddSWW5agZRuLs6D71zZzz7kqM QJGVJNFHGRv1GhlEU7qyjtpyr7TmDUGoIe9hFHJN6JwkFmTlwdsSDaC+m RlWUXR36JD7nFFT2v2hBZWLJumKgPuBz594jf94MZ9NT26PgNYon+pSnz g==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="250624954" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="250624954" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:36 -0800 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="683637807" Received: from alison-desk.jf.intel.com (HELO localhost) ([10.54.74.41]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:36 -0800 From: alison.schofield@intel.com To: Ben Widawsky , Dan Williams , Ira Weiny , Vishal Verma Cc: Alison Schofield , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Subject: [ndctl PATCH v6 2/6] libcxl: add accessors for capacity fields of the IDENTIFY command Date: Tue, 22 Feb 2022 11:56:04 -0800 Message-Id: <58dec40b15a68f134466f61421751994735e55c1.1645558189.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield The CXL PMEM provisioning model depends upon the values reported in the CXL IDENTIFY mailbox command when changing the partitioning between volatile and persistent capacity. Add accessors to the libcxl API to retrieve the total, volatile only, and persistent only capacities from the IDENTIFY command. The fields are specified in multiples of 256MB per the CXL 2.0 spec. Use the capacity multiplier to convert the raw data into bytes for user consumption. Signed-off-by: Alison Schofield Reviewed-by: Dan Williams --- cxl/lib/libcxl.c | 44 ++++++++++++++++++++++++++++++++++++++++++++ cxl/lib/libcxl.sym | 3 +++ cxl/libcxl.h | 3 +++ 3 files changed, 50 insertions(+) diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index 4557a71de845..9413384b4b3b 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -2277,6 +2277,17 @@ CXL_EXPORT struct cxl_cmd *cxl_cmd_new_identify(struct cxl_memdev *memdev) return cxl_cmd_new_generic(memdev, CXL_MEM_COMMAND_ID_IDENTIFY); } +static struct cxl_cmd_identify * +cmd_to_identify(struct cxl_cmd *cmd) +{ + if (cxl_cmd_validate_status(cmd, CXL_MEM_COMMAND_ID_IDENTIFY)) + return NULL; + + if (!cmd) + return NULL; + return cmd->output_payload; +} + CXL_EXPORT int cxl_cmd_identify_get_fw_rev(struct cxl_cmd *cmd, char *fw_rev, int fw_len) { @@ -2321,6 +2332,39 @@ CXL_EXPORT unsigned int cxl_cmd_identify_get_label_size(struct cxl_cmd *cmd) return le32_to_cpu(id->lsa_size); } +CXL_EXPORT unsigned long long +cxl_cmd_identify_get_total_size(struct cxl_cmd *cmd) +{ + struct cxl_cmd_identify *c; + + c = cmd_to_identify(cmd); + if (!c) + return ULLONG_MAX; + return cxl_capacity_to_bytes(c->total_capacity); +} + +CXL_EXPORT unsigned long long +cxl_cmd_identify_get_volatile_only_size(struct cxl_cmd *cmd) +{ + struct cxl_cmd_identify *c; + + c = cmd_to_identify(cmd); + if (!c) + return ULLONG_MAX; + return cxl_capacity_to_bytes(c->volatile_capacity); +} + +CXL_EXPORT unsigned long long +cxl_cmd_identify_get_persistent_only_size(struct cxl_cmd *cmd) +{ + struct cxl_cmd_identify *c; + + c = cmd_to_identify(cmd); + if (!c) + return ULLONG_MAX; + return cxl_capacity_to_bytes(c->persistent_capacity); +} + CXL_EXPORT struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev, int opcode) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 509e62daad4b..5ac6e9bbe981 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -160,4 +160,7 @@ global: cxl_cmd_partition_get_active_persistent_size; cxl_cmd_partition_get_next_volatile_size; cxl_cmd_partition_get_next_persistent_size; + cxl_cmd_identify_get_total_size; + cxl_cmd_identify_get_volatile_only_size; + cxl_cmd_identify_get_persistent_only_size; } LIBCXL_1; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 2c0a8d199f0a..6e18e843d3ea 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -201,6 +201,9 @@ int cxl_cmd_get_mbox_status(struct cxl_cmd *cmd); int cxl_cmd_get_out_size(struct cxl_cmd *cmd); struct cxl_cmd *cxl_cmd_new_identify(struct cxl_memdev *memdev); int cxl_cmd_identify_get_fw_rev(struct cxl_cmd *cmd, char *fw_rev, int fw_len); +unsigned long long cxl_cmd_identify_get_total_size(struct cxl_cmd *cmd); +unsigned long long cxl_cmd_identify_get_volatile_only_size(struct cxl_cmd *cmd); +unsigned long long cxl_cmd_identify_get_persistent_only_size(struct cxl_cmd *cmd); unsigned long long cxl_cmd_identify_get_partition_align(struct cxl_cmd *cmd); unsigned int cxl_cmd_identify_get_label_size(struct cxl_cmd *cmd); struct cxl_cmd *cxl_cmd_new_get_health_info(struct cxl_memdev *memdev); From patchwork Tue Feb 22 19:56:05 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 12755869 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8C576C433EF for ; Tue, 22 Feb 2022 19:52:43 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235318AbiBVTxG (ORCPT ); Tue, 22 Feb 2022 14:53:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53914 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233423AbiBVTxD (ORCPT ); Tue, 22 Feb 2022 14:53:03 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D569DC4E30 for ; Tue, 22 Feb 2022 11:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645559557; x=1677095557; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=i/f+JBwQs1IDAStzVhcNq+ZwWqHAE3YETRKnb3eAacA=; b=nkqBABY7FTpbjwaAkjALZzegsi+IKnusJ+fLVu2MZyVI4kAJGutjgEKO DghsIOTVJqjAZ2lDH2s55KMUFdIYIzROlrTQs3RH+H0Gtx65LdMpTM3jJ iCSrTsljf1psTieL7PhjNTcofGy+IZm0aJVMoQTAyM8P6SaEWy/o3S0H+ z9muxkw2Kbtmj6jnqepkUkYQvawan+TA7/xUVFczxh3GUodN6xy4URQTX Q1RS9hbYOLlSkrtSw2zMAGyxJ5b/stK00wyCbU2ua/pUZ54bi7YB+55hD q0ISmNRB2lg/iwiL5mh1X1hQmYDWWGwpwRD7YwKARUa/UORFk0pllVolP w==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="250624958" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="250624958" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:36 -0800 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="683637812" Received: from alison-desk.jf.intel.com (HELO localhost) ([10.54.74.41]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:36 -0800 From: alison.schofield@intel.com To: Ben Widawsky , Dan Williams , Ira Weiny , Vishal Verma Cc: Alison Schofield , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Subject: [ndctl PATCH v6 3/6] libcxl: return the partition alignment field in bytes Date: Tue, 22 Feb 2022 11:56:05 -0800 Message-Id: <6b937b09b61ddf95e069fd7acfda0c5bbb845be8.1645558189.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield Per the CXL specification, the partition alignment field reports the alignment value in multiples of 256MB. In the libcxl API, values for all capacity fields are defined to return bytes. Update the partition alignment accessor to return bytes so that it is in sync with other capacity related fields. Since this is early in the development cycle, the expectation is that no third party consumers of this library have come to depend on the encoded capacity field. If that is not the case, the original format can be restored, and a new _bytes version introduced. Signed-off-by: Alison Schofield Reviewed-by: Dan Williams --- cxl/lib/libcxl.c | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index 9413384b4b3b..c05c13c501ab 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -2308,15 +2308,12 @@ CXL_EXPORT int cxl_cmd_identify_get_fw_rev(struct cxl_cmd *cmd, char *fw_rev, CXL_EXPORT unsigned long long cxl_cmd_identify_get_partition_align( struct cxl_cmd *cmd) { - struct cxl_cmd_identify *id = - (struct cxl_cmd_identify *)cmd->send_cmd->out.payload; + struct cxl_cmd_identify *c; - if (cmd->send_cmd->id != CXL_MEM_COMMAND_ID_IDENTIFY) - return -EINVAL; - if (cmd->status < 0) - return cmd->status; - - return le64_to_cpu(id->partition_align); + c = cmd_to_identify(cmd); + if (!c) + return ULLONG_MAX; + return cxl_capacity_to_bytes(c->partition_align); } CXL_EXPORT unsigned int cxl_cmd_identify_get_label_size(struct cxl_cmd *cmd) From patchwork Tue Feb 22 19:56:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 12755867 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4D63CC43217 for ; Tue, 22 Feb 2022 19:52:41 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235333AbiBVTxF (ORCPT ); Tue, 22 Feb 2022 14:53:05 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53922 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235326AbiBVTxD (ORCPT ); Tue, 22 Feb 2022 14:53:03 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id DA0E8C558D for ; Tue, 22 Feb 2022 11:52:37 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645559557; x=1677095557; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=KmrMQTDb6cGTe/qnj9HgoX3TAd54sfcgXTDrSD8yIso=; b=Pl1npDNnMBOU+kAhWKihCBwXdyKecYlcVXFGfsltDI9rUczaYYG4k2Ku iPogWrZmr6tIBS57wSbWno/+YIAdqlutGOjd7AXp1DuQ2RmSmEl7zXTRu HmQzQ+Kc+cygSacRsq0DGFMgjWbkQduYdXBX2MKFrgfFFTHEuzgRKiaNI dPhbPBoM5cjp26GFOJQfJO2JvVPT6IukDeXkp1dPiNM8vwffQXnup7iy5 NzLVPX7CXf+VXTQixmjsGIBZxt6t+VzeV3fRl4BaCksn0rEhocF5+41RO S2PD8L1mpIciB+rqTNHkVphw+/HFo/SQlnGPn/K6+MdgCIz8n/H08qOcJ g==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="250624963" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="250624963" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:37 -0800 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="683637815" Received: from alison-desk.jf.intel.com (HELO localhost) ([10.54.74.41]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:37 -0800 From: alison.schofield@intel.com To: Ben Widawsky , Dan Williams , Ira Weiny , Vishal Verma Cc: Alison Schofield , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Subject: [ndctl PATCH v6 4/6] cxl: add memdev partition information to cxl-list Date: Tue, 22 Feb 2022 11:56:06 -0800 Message-Id: <70cc57379d2c49692036b1daa158a122aa19c126.1645558189.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield The CXL PMEM provisioning model depends upon the values reported in both the CXL IDENTIFY and GET_PARTITION_INFO mailbox commands when changing the partitioning between volatile and persistent capacity. Add an option to the 'cxl list' command to display partition information. Include the partitioning related fields from the IDENTIFY command: total, volatile_only, persistent_only, and partition_alignment sizes. When the partition_alignment size is greater than zero, indicating partitionable capacity, include the active and next size fields of GET_PARTITION_INFO. Example: "partition_info":{ "total_size":273535729664, "volatile_only_size":0, "persistent_only_size":0, "partition_alignment_size":268435456 "active_volatile_size":273535729664, "active_persistent_size":0, "next_volatile_size":0, "next_persistent_size":0, } Signed-off-by: Alison Schofield Reviewed-by: Dan Williams --- Documentation/cxl/cxl-list.txt | 23 +++++++ cxl/filter.c | 2 + cxl/filter.h | 1 + cxl/json.c | 120 +++++++++++++++++++++++++++++++++ cxl/list.c | 2 + util/json.h | 1 + 6 files changed, 149 insertions(+) diff --git a/Documentation/cxl/cxl-list.txt b/Documentation/cxl/cxl-list.txt index 90e6d9f9658b..f6aba0c5c1fa 100644 --- a/Documentation/cxl/cxl-list.txt +++ b/Documentation/cxl/cxl-list.txt @@ -196,6 +196,29 @@ OPTIONS } ] ---- +-I:: +--partition:: + Include partition information in the memdev listing. Example listing: +---- +# cxl list -m mem0 -I +[ + { + "memdev":"mem0", + "pmem_size":0, + "ram_size":273535729664, + "partition_info":{ + "total_size":273535729664, + "volatile_only_size":0, + "persistent_only_size":0, + "partition_alignment_size":268435456 + "active_volatile_size":273535729664, + "active_persistent_size":0, + "next_volatile_size":0, + "next_persistent_size":0, + } + } +] +---- -B:: --buses:: diff --git a/cxl/filter.c b/cxl/filter.c index 925bf3a1c62d..b3396426dda8 100644 --- a/cxl/filter.c +++ b/cxl/filter.c @@ -581,6 +581,8 @@ static unsigned long params_to_flags(struct cxl_filter_params *param) flags |= UTIL_JSON_HEALTH; if (param->targets) flags |= UTIL_JSON_TARGETS; + if (param->partition) + flags |= UTIL_JSON_PARTITION; return flags; } diff --git a/cxl/filter.h b/cxl/filter.h index 5deabb3d38fe..697b7779c08e 100644 --- a/cxl/filter.h +++ b/cxl/filter.h @@ -23,6 +23,7 @@ struct cxl_filter_params { bool idle; bool human; bool health; + bool partition; struct log_ctx ctx; }; diff --git a/cxl/json.c b/cxl/json.c index f3b536e1d4fa..fdc6f73a86c1 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -185,6 +185,121 @@ err_jobj: return NULL; } +/* + * Present complete view of memdev partition by presenting fields from + * both GET_PARTITION_INFO and IDENTIFY mailbox commands. + */ +static struct json_object *util_cxl_memdev_partition_to_json(struct cxl_memdev *memdev, + unsigned long flags) +{ + struct json_object *jobj = NULL; + struct json_object *jpart; + unsigned long long cap; + struct cxl_cmd *cmd; + int rc; + + jpart = json_object_new_object(); + if (!jpart) + return NULL; + if (!memdev) + goto err_jobj; + + /* Retrieve partition info in the IDENTIFY mbox cmd */ + cmd = cxl_cmd_new_identify(memdev); + if (!cmd) + goto err_jobj; + + rc = cxl_cmd_submit(cmd); + if (rc < 0) + goto err_identify; + rc = cxl_cmd_get_mbox_status(cmd); + if (rc != 0) + goto err_identify; + + cap = cxl_cmd_identify_get_total_size(cmd); + if (cap != ULLONG_MAX) { + jobj = util_json_object_size(cap, flags); + if (jobj) + json_object_object_add(jpart, "total_size", jobj); + } + cap = cxl_cmd_identify_get_volatile_only_size(cmd); + if (cap != ULLONG_MAX) { + jobj = util_json_object_size(cap, flags); + if (jobj) + json_object_object_add(jpart, + "volatile_only_size", jobj); + } + cap = cxl_cmd_identify_get_persistent_only_size(cmd); + if (cap != ULLONG_MAX) { + jobj = util_json_object_size(cap, flags); + if (jobj) + json_object_object_add(jpart, + "persistent_only_size", jobj); + } + cap = cxl_cmd_identify_get_partition_align(cmd); + jobj = util_json_object_size(cap, flags); + if (jobj) + json_object_object_add(jpart, "partition_alignment_size", jobj); + + cxl_cmd_unref(cmd); + + /* Return now if there is no partition info to get. */ + if (!cap) + return jpart; + + /* Retrieve partition info in GET_PARTITION_INFO mbox cmd */ + cmd = cxl_cmd_new_get_partition(memdev); + if (!cmd) + return jpart; + + rc = cxl_cmd_submit(cmd); + if (rc < 0) + goto err_get; + rc = cxl_cmd_get_mbox_status(cmd); + if (rc != 0) + goto err_get; + + cap = cxl_cmd_partition_get_active_volatile_size(cmd); + if (cap != ULLONG_MAX) { + jobj = util_json_object_size(cap, flags); + if (jobj) + json_object_object_add(jpart, + "active_volatile_size", jobj); + } + cap = cxl_cmd_partition_get_active_persistent_size(cmd); + if (cap != ULLONG_MAX) { + jobj = util_json_object_size(cap, flags); + if (jobj) + json_object_object_add(jpart, + "active_persistent_size", jobj); + } + cap = cxl_cmd_partition_get_next_volatile_size(cmd); + if (cap != ULLONG_MAX) { + jobj = util_json_object_size(cap, flags); + if (jobj) + json_object_object_add(jpart, + "next_volatile_size", jobj); + } + cap = cxl_cmd_partition_get_next_persistent_size(cmd); + if (cap != ULLONG_MAX) { + jobj = util_json_object_size(cap, flags); + if (jobj) + json_object_object_add(jpart, + "next_persistent_size", jobj); + } + +err_get: + cxl_cmd_unref(cmd); + return jpart; + +err_identify: + cxl_cmd_unref(cmd); + +err_jobj: + json_object_put(jpart); + return NULL; +} + struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev, unsigned long flags) { @@ -239,6 +354,11 @@ struct json_object *util_cxl_memdev_to_json(struct cxl_memdev *memdev, json_object_object_add(jdev, "state", jobj); } + if (flags & UTIL_JSON_PARTITION) { + jobj = util_cxl_memdev_partition_to_json(memdev, flags); + if (jobj) + json_object_object_add(jdev, "partition_info", jobj); + } return jdev; } diff --git a/cxl/list.c b/cxl/list.c index de96ff91fb5d..1e9d441190a0 100644 --- a/cxl/list.c +++ b/cxl/list.c @@ -48,6 +48,8 @@ static const struct option options[] = { "use human friendly number formats "), OPT_BOOLEAN('H', "health", ¶m.health, "include memory device health information "), + OPT_BOOLEAN('I', "partition", ¶m.partition, + "include memory device partition information "), #ifdef ENABLE_DEBUG OPT_BOOLEAN(0, "debug", &debug, "debug list walk"), #endif diff --git a/util/json.h b/util/json.h index e026df1e1519..73bb9f0075cf 100644 --- a/util/json.h +++ b/util/json.h @@ -19,6 +19,7 @@ enum util_json_flags { UTIL_JSON_DAX_MAPPINGS = (1 << 9), UTIL_JSON_HEALTH = (1 << 10), UTIL_JSON_TARGETS = (1 << 11), + UTIL_JSON_PARTITION = (1 << 12), }; void util_display_json_array(FILE *f_out, struct json_object *jarray, From patchwork Tue Feb 22 19:56:07 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 12755870 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 794FFC433FE for ; Tue, 22 Feb 2022 19:52:44 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233423AbiBVTxG (ORCPT ); Tue, 22 Feb 2022 14:53:06 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53946 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235329AbiBVTxE (ORCPT ); Tue, 22 Feb 2022 14:53:04 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 5B628C558F for ; Tue, 22 Feb 2022 11:52:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645559558; x=1677095558; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=sTwS3zyfONq/hobAVvA2o56Qgrn6SYc0iF1csV4Jw7c=; b=YrtLP3kTDDok/HRSb9i72QYUwt5fLk37ZHZ4tXdGJfnHYuUHaQhcID36 np/VYXJUFgFnTBYnia7cb/hIkBqFyLxx8qVOVOk6CDRH2+27sMGrCvW8S lHpmJaYfa99sFel99/O/P7AgxHHZXZ8soOlh48zGvJOzuW93GBVrpRKxo lGIXmCLGtcgT8lXOUHrnLkxR3ILOCdxDnkeYOCXHLHavQkgF7L8w15+UI xJu68Yy3lHZ3v6uGmQTxh8S3/0/7vKA0gxDTx1cZhqEzP3NSbregO4VmA Fe2twozC5qfB1mw53p6MpjQwLQSS93uLlfmpA4+Rfiurzj/NZqJtKnAVp g==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="250624966" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="250624966" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:38 -0800 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="683637818" Received: from alison-desk.jf.intel.com (HELO localhost) ([10.54.74.41]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:37 -0800 From: alison.schofield@intel.com To: Ben Widawsky , Dan Williams , Ira Weiny , Vishal Verma Cc: Alison Schofield , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Subject: [ndctl PATCH v6 5/6] libcxl: add interfaces for SET_PARTITION_INFO mailbox command Date: Tue, 22 Feb 2022 11:56:07 -0800 Message-Id: <978c1cf78f3dd22f6070e51a241bc63cac9297de.1645558189.git.alison.schofield@intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield The CXL PMEM provisioning model depends upon the CXL mailbox command SET_PARTITION_INFO to change a device's partitioning between volatile and persistent capacity. Add interfaces to libcxl to allocate and send a SET_PARTITION_INFO mailbox command as defined in the CXL 2.0 specification. Signed-off-by: Alison Schofield Reviewed-by: Dan Williams --- Documentation/cxl/lib/libcxl.txt | 11 +++++++++++ cxl/lib/libcxl.c | 28 ++++++++++++++++++++++++++++ cxl/lib/libcxl.sym | 2 ++ cxl/lib/private.h | 8 ++++++++ cxl/libcxl.h | 10 ++++++++++ 5 files changed, 59 insertions(+) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index a6986abafce3..7b223cbcac3f 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -132,6 +132,8 @@ int cxl_memdev_read_label(struct cxl_memdev *memdev, void *buf, size_t length, int cxl_memdev_write_label(struct cxl_memdev *memdev, void *buf, size_t length, size_t offset); struct cxl_cmd *cxl_cmd_new_get_partition(struct cxl_memdev *memdev); +struct cxl_cmd *cxl_cmd_new_set_partition(struct cxl_memdev *memdev, + unsigned long long volatile_size); ---- @@ -148,6 +150,8 @@ this sub-class of interfaces, there are: a CXL standard opcode. See the potential command ids in /usr/include/linux/cxl_mem.h. + * 'cxl_cmd__set_' interfaces that set specific fields in a cxl_cmd + * 'cxl_cmd_submit' which submits the command via ioctl() * 'cxl_cmd__get_' interfaces that get specific fields out of the @@ -167,6 +171,13 @@ cxl_memdev{read,write,zero}_label() are helpers for marshaling multiple label access commands over an arbitrary extent of the device's label area. +cxl_cmd_partition_set_mode() supports selecting NEXTBOOT or IMMEDIATE +mode. When CXL_SETPART_IMMEDIATE mode is set, it is the caller’s +responsibility to avoid immediate changes to partitioning when the +device is in use. When CXL_SETPART_NEXTBOOT mode is set, the change +in partitioning shall become the “next” configuration, to become +active on the next device reset. + BUSES ----- The CXL Memory space is CPU and Device coherent. The address ranges that diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index c05c13c501ab..daa2bbc5a299 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -2478,6 +2478,34 @@ cxl_cmd_partition_get_next_persistent_size(struct cxl_cmd *cmd) return cxl_capacity_to_bytes(c->next_persistent); } +CXL_EXPORT int cxl_cmd_partition_set_mode(struct cxl_cmd *cmd, + enum cxl_setpartition_mode mode) +{ + struct cxl_cmd_set_partition *setpart = cmd->input_payload; + + if (mode == CXL_SETPART_IMMEDIATE) + setpart->flags = CXL_CMD_SET_PARTITION_FLAG_IMMEDIATE; + else + setpart->flags = !CXL_CMD_SET_PARTITION_FLAG_IMMEDIATE; + + return 0; +} + +CXL_EXPORT struct cxl_cmd *cxl_cmd_new_set_partition(struct cxl_memdev *memdev, + unsigned long long volatile_size) +{ + struct cxl_cmd_set_partition *setpart; + struct cxl_cmd *cmd; + + cmd = cxl_cmd_new_generic(memdev, + CXL_MEM_COMMAND_ID_SET_PARTITION_INFO); + + setpart = cmd->input_payload; + setpart->volatile_size = cpu_to_le64(volatile_size) + / CXL_CAPACITY_MULTIPLIER; + return cmd; +} + CXL_EXPORT int cxl_cmd_submit(struct cxl_cmd *cmd) { struct cxl_memdev *memdev = cmd->memdev; diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 5ac6e9bbe981..aab1112a91d8 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -163,4 +163,6 @@ global: cxl_cmd_identify_get_total_size; cxl_cmd_identify_get_volatile_only_size; cxl_cmd_identify_get_persistent_only_size; + cxl_cmd_new_set_partition; + cxl_cmd_partition_set_mode; } LIBCXL_1; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index 7f3a562a7c8e..c6d88f7140f2 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -195,6 +195,14 @@ struct cxl_cmd_get_partition { #define CXL_CAPACITY_MULTIPLIER SZ_256M +struct cxl_cmd_set_partition { + le64 volatile_size; + u8 flags; +} __attribute__((packed)); + +/* CXL 2.0 8.2.9.5.2 Set Partition Info */ +#define CXL_CMD_SET_PARTITION_FLAG_IMMEDIATE BIT(0) + /* CXL 2.0 8.2.9.5.3 Byte 0 Health Status */ #define CXL_CMD_HEALTH_INFO_STATUS_MAINTENANCE_NEEDED_MASK BIT(0) #define CXL_CMD_HEALTH_INFO_STATUS_PERFORMANCE_DEGRADED_MASK BIT(1) diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 6e18e843d3ea..0063d31ab398 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -250,6 +250,16 @@ unsigned long long cxl_cmd_partition_get_active_volatile_size(struct cxl_cmd *cm unsigned long long cxl_cmd_partition_get_active_persistent_size(struct cxl_cmd *cmd); unsigned long long cxl_cmd_partition_get_next_volatile_size(struct cxl_cmd *cmd); unsigned long long cxl_cmd_partition_get_next_persistent_size(struct cxl_cmd *cmd); +struct cxl_cmd *cxl_cmd_new_set_partition(struct cxl_memdev *memdev, + unsigned long long volatile_size); + +enum cxl_setpartition_mode { + CXL_SETPART_NEXTBOOT, + CXL_SETPART_IMMEDIATE, +}; + +int cxl_cmd_partition_set_mode(struct cxl_cmd *cmd, + enum cxl_setpartition_mode mode); #ifdef __cplusplus } /* extern "C" */ From patchwork Tue Feb 22 19:56:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Alison Schofield X-Patchwork-Id: 12755871 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A1E2C4332F for ; Tue, 22 Feb 2022 19:52:45 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235335AbiBVTxI (ORCPT ); Tue, 22 Feb 2022 14:53:08 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53956 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235331AbiBVTxE (ORCPT ); Tue, 22 Feb 2022 14:53:04 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id A402FC5595 for ; Tue, 22 Feb 2022 11:52:38 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1645559558; x=1677095558; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ee1UdJrfOmyH51HvJJPxJDtcPSYBf631vBoDQWhI4xA=; b=PAp51jnoP2plgSl1BQ58FT2l7gjuOzUXeOx0tvF0GfPr+DEF1Pl+Qikf ZVC8Uxlg7DZobVqkoNMWWuSjXvN+KsadsZzQHsCcmL9xAbvW2DnyLJZLS abwLMFKTRbrTTCH4tfJ3ExxI8qsG6u5CD+bhjmKU7g3w7cAAoE01ET9TC D+JW+0tQvUh9t4lQqCG+mqqnYecNV+x3bO8JzWkngrB21sAsd0WTXSuiQ ghlMxhfcghB7akI7ik8Ht8nC9CEI/Z+KN1DLcCpdCsvDAVBVjs84M4j+f BA+sKQsRBoVztnXMphfhx+caEwL2lBGjNyEmQ0qossHibr7nRFNKzJT1A w==; X-IronPort-AV: E=McAfee;i="6200,9189,10266"; a="250624967" X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="250624967" Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:38 -0800 X-IronPort-AV: E=Sophos;i="5.88,387,1635231600"; d="scan'208";a="683637821" Received: from alison-desk.jf.intel.com (HELO localhost) ([10.54.74.41]) by fmsmga001-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 22 Feb 2022 11:52:38 -0800 From: alison.schofield@intel.com To: Ben Widawsky , Dan Williams , Ira Weiny , Vishal Verma Cc: Alison Schofield , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Subject: [ndctl PATCH v6 6/6] cxl: add command 'cxl set-partition' Date: Tue, 22 Feb 2022 11:56:08 -0800 Message-Id: X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 Precedence: bulk List-ID: X-Mailing-List: linux-cxl@vger.kernel.org From: Alison Schofield CXL devices may support both volatile and persistent memory capacity. The amount of device capacity set aside for each type is typically established at the factory, but some devices also allow for dynamic re-partitioning. Add a command for this purpose. usage: cxl set-partition [..] [] -v, --verbose turn on debug -S, --serial use serial numbers to id memdevs -t, --type 'pmem' or 'volatile' (Default: 'pmem') -s, --size size in bytes (Default: all available capacity) -a, --align auto-align --size per device's requirement Signed-off-by: Alison Schofield Reviewed-by: Dan Williams --- Documentation/cxl/cxl-set-partition.txt | 68 ++++++++ Documentation/cxl/meson.build | 1 + cxl/builtin.h | 1 + cxl/cxl.c | 1 + cxl/memdev.c | 206 ++++++++++++++++++++++++ 5 files changed, 277 insertions(+) create mode 100644 Documentation/cxl/cxl-set-partition.txt diff --git a/Documentation/cxl/cxl-set-partition.txt b/Documentation/cxl/cxl-set-partition.txt new file mode 100644 index 000000000000..1e548af77da2 --- /dev/null +++ b/Documentation/cxl/cxl-set-partition.txt @@ -0,0 +1,68 @@ +// SPDX-License-Identifier: GPL-2.0 + +cxl-set-partition(1) +==================== + +NAME +---- +cxl-set-partition - set the partitioning between volatile and persistent capacity on a CXL memdev + +SYNOPSIS +-------- +[verse] +'cxl set-partition [ [..] []' + +DESCRIPTION +----------- +CXL devices may support both volatile and persistent memory capacity. +The amount of device capacity set aside for each type is typically +established at the factory, but some devices also allow for dynamic +re-partitioning. + +Use this command to partition a device into volatile and persistent +capacity. The change in partitioning becomes the “next” configuration, +to become active on the next device reset. + +Use "cxl list -m -I" to examine the partitioning capabilities +of a device. A partition_alignment_size value of zero means there is +no available capacity and therefore the partitions cannot be changed. + +Using this command to change the size of the persistent capacity shall +result in the loss of data stored. + +OPTIONS +------- +:: +include::memdev-option.txt[] + +-t:: +--type=:: + Type of partition, 'pmem' or 'volatile', to modify. + Default: 'pmem' + +-s:: +--size=:: + Size of the partition in bytes. Size must align to the + devices alignment requirement. Use 'cxl list -m -I' + to find 'partition_alignment_size', or, use the --align option. + Default: All available capacity is assigned to . + +-a:: +--align:: + Select this option to allow the automatic alignment of --size + to meet device alignment requirements. When using this option, + specify the minimum --size of the --type partition needed. When + this option is omitted, the command fails if --size is not + properly aligned. Use 'cxl list -m -I' to examine the + partition_alignment_size. + +-v:: + Turn on verbose debug messages in the library (if libcxl was built with + logging and debug enabled). + +include::../copyright.txt[] + +SEE ALSO +-------- +linkcxl:cxl-list[1], +CXL-2.0 8.2.9.5.2 diff --git a/Documentation/cxl/meson.build b/Documentation/cxl/meson.build index 96f4666a828d..e927644a3826 100644 --- a/Documentation/cxl/meson.build +++ b/Documentation/cxl/meson.build @@ -34,6 +34,7 @@ cxl_manpages = [ 'cxl-disable-memdev.txt', 'cxl-enable-port.txt', 'cxl-disable-port.txt', + 'cxl-set-partition.txt', ] foreach man : cxl_manpages diff --git a/cxl/builtin.h b/cxl/builtin.h index 3123d5e0d4ed..7bbad98f67ac 100644 --- a/cxl/builtin.h +++ b/cxl/builtin.h @@ -14,4 +14,5 @@ int cmd_disable_memdev(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_enable_memdev(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_disable_port(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_enable_port(int argc, const char **argv, struct cxl_ctx *ctx); +int cmd_set_partition(int argc, const char **argv, struct cxl_ctx *ctx); #endif /* _CXL_BUILTIN_H_ */ diff --git a/cxl/cxl.c b/cxl/cxl.c index c20c5693fd2b..ab4bbeccaa76 100644 --- a/cxl/cxl.c +++ b/cxl/cxl.c @@ -68,6 +68,7 @@ static struct cmd_struct commands[] = { { "enable-memdev", .c_fn = cmd_enable_memdev }, { "disable-port", .c_fn = cmd_disable_port }, { "enable-port", .c_fn = cmd_enable_port }, + { "set-partition", .c_fn = cmd_set_partition }, }; int main(int argc, const char **argv) diff --git a/cxl/memdev.c b/cxl/memdev.c index 90b33e1b4195..91d914db5af6 100644 --- a/cxl/memdev.c +++ b/cxl/memdev.c @@ -6,11 +6,14 @@ #include #include #include +#include +#include #include #include #include #include +#include "json.h" #include "filter.h" struct action_context { @@ -26,10 +29,18 @@ static struct parameters { bool verbose; bool serial; bool force; + bool align; + const char *type; + const char *size; } param; static struct log_ctx ml; +enum cxl_setpart_type { + CXL_SETPART_PMEM, + CXL_SETPART_VOLATILE, +}; + #define BASE_OPTIONS() \ OPT_BOOLEAN('v',"verbose", ¶m.verbose, "turn on debug"), \ OPT_BOOLEAN('S', "serial", ¶m.serial, "use serial numbers to id memdevs") @@ -51,6 +62,14 @@ OPT_UINTEGER('O', "offset", ¶m.offset, \ OPT_BOOLEAN('f', "force", ¶m.force, \ "DANGEROUS: override active memdev safety checks") +#define SET_PARTITION_OPTIONS() \ +OPT_STRING('t', "type", ¶m.type, "type", \ + "'pmem' or 'volatile' (Default: 'pmem')"), \ +OPT_STRING('s', "size", ¶m.size, "size", \ + "size in bytes (Default: all available capacity)"), \ +OPT_BOOLEAN('a', "align", ¶m.align, \ + "auto-align --size per device's requirement") + static const struct option read_options[] = { BASE_OPTIONS(), LABEL_OPTIONS(), @@ -82,6 +101,12 @@ static const struct option enable_options[] = { OPT_END(), }; +static const struct option set_partition_options[] = { + BASE_OPTIONS(), + SET_PARTITION_OPTIONS(), + OPT_END(), +}; + static int action_disable(struct cxl_memdev *memdev, struct action_context *actx) { if (!cxl_memdev_is_enabled(memdev)) @@ -209,6 +234,176 @@ out: return rc; } +static unsigned long long +partition_align(const char *devname, enum cxl_setpart_type type, + unsigned long long volatile_size, unsigned long long alignment, + unsigned long long available) +{ + if (IS_ALIGNED(volatile_size, alignment)) + return volatile_size; + + if (!param.align) { + log_err(&ml, "%s: size %lld is not partition aligned %lld\n", + devname, volatile_size, alignment); + return ULLONG_MAX; + } + + /* Align based on partition type to fulfill users size request */ + if (type == CXL_SETPART_PMEM) + volatile_size = ALIGN_DOWN(volatile_size, alignment); + else + volatile_size = ALIGN(volatile_size, alignment); + + /* Fail if the align pushes size over the available limit. */ + if (volatile_size > available) { + log_err(&ml, "%s: aligned partition size %lld exceeds available size %lld\n", + devname, volatile_size, available); + volatile_size = ULLONG_MAX; + } + + return volatile_size; +} + +static unsigned long long +param_size_to_volatile_size(const char *devname, enum cxl_setpart_type type, + unsigned long long size, unsigned long long available) +{ + /* User omits size option. Apply all available capacity to type. */ + if (size == ULLONG_MAX) { + if (type == CXL_SETPART_PMEM) + return 0; + return available; + } + + /* User includes a size option. Apply it to type */ + if (size > available) { + log_err(&ml, "%s: %lld exceeds available capacity %lld\n", + devname, size, available); + return ULLONG_MAX; + } + if (type == CXL_SETPART_PMEM) + return available - size; + return size; +} + +/* + * Return the volatile_size to use in the CXL set paritition + * command, or ULLONG_MAX if unable to validate the partition + * request. + */ +static unsigned long long +validate_partition(struct cxl_memdev *memdev, enum cxl_setpart_type type, + unsigned long long size) +{ + unsigned long long total_cap, volatile_only, persistent_only; + const char *devname = cxl_memdev_get_devname(memdev); + unsigned long long volatile_size = ULLONG_MAX; + unsigned long long available, alignment; + struct cxl_cmd *cmd; + int rc; + + cmd = cxl_cmd_new_identify(memdev); + if (!cmd) + return ULLONG_MAX; + rc = cxl_cmd_submit(cmd); + if (rc < 0) + goto out; + rc = cxl_cmd_get_mbox_status(cmd); + if (rc != 0) + goto out; + + alignment = cxl_cmd_identify_get_partition_align(cmd); + if (alignment == 0) { + log_err(&ml, "%s: no available capacity\n", devname); + goto out; + } + + /* Calculate the actual available capacity */ + total_cap = cxl_cmd_identify_get_total_size(cmd); + volatile_only = cxl_cmd_identify_get_volatile_only_size(cmd); + persistent_only = cxl_cmd_identify_get_persistent_only_size(cmd); + available = total_cap - volatile_only - persistent_only; + + /* Translate the users size request into an aligned volatile_size */ + volatile_size = param_size_to_volatile_size(devname, type, size, + available); + if (volatile_size == ULLONG_MAX) + goto out; + + volatile_size = partition_align(devname, type, volatile_size, alignment, + available); + +out: + cxl_cmd_unref(cmd); + return volatile_size; +} + +static int action_setpartition(struct cxl_memdev *memdev, + struct action_context *actx) +{ + const char *devname = cxl_memdev_get_devname(memdev); + enum cxl_setpart_type type = CXL_SETPART_PMEM; + unsigned long long size = ULLONG_MAX; + struct json_object *jmemdev; + struct cxl_cmd *cmd; + int rc; + + if (param.type) { + if (strcmp(param.type, "pmem") == 0) + /* default */; + else if (strcmp(param.type, "volatile") == 0) + type = CXL_SETPART_VOLATILE; + else { + log_err(&ml, "invalid type '%s'\n", param.type); + return -EINVAL; + } + } + + if (param.size) { + size = parse_size64(param.size); + if (size == ULLONG_MAX) { + log_err(&ml, "%s: failed to parse size option '%s'\n", + devname, param.size); + return -EINVAL; + } + } + + size = validate_partition(memdev, type, size); + if (size == ULLONG_MAX) + return -EINVAL; + + cmd = cxl_cmd_new_set_partition(memdev, size); + if (!cmd) { + rc = -ENXIO; + goto out_err; + } + + rc = cxl_cmd_submit(cmd); + if (rc < 0) { + log_err(&ml, "cmd submission failed: %s\n", strerror(-rc)); + goto out_cmd; + } + + rc = cxl_cmd_get_mbox_status(cmd); + if (rc != 0) { + log_err(&ml, "%s: mbox status: %d\n", __func__, rc); + rc = -ENXIO; + } + +out_cmd: + cxl_cmd_unref(cmd); +out_err: + if (rc) + log_err(&ml, "%s error: %s\n", devname, strerror(-rc)); + + jmemdev = util_cxl_memdev_to_json(memdev, UTIL_JSON_PARTITION); + if (jmemdev) + printf("%s\n", json_object_to_json_string_ext(jmemdev, + JSON_C_TO_STRING_PRETTY)); + + return rc; +} + static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, int (*action)(struct cxl_memdev *memdev, struct action_context *actx), @@ -398,3 +593,14 @@ int cmd_enable_memdev(int argc, const char **argv, struct cxl_ctx *ctx) count > 1 ? "s" : ""); return count >= 0 ? 0 : EXIT_FAILURE; } + +int cmd_set_partition(int argc, const char **argv, struct cxl_ctx *ctx) +{ + int count = memdev_action(argc, argv, ctx, action_setpartition, + set_partition_options, + "cxl set-partition [..] []"); + log_info(&ml, "set_partition %d mem%s\n", count >= 0 ? count : 0, + count > 1 ? "s" : ""); + + return count >= 0 ? 0 : EXIT_FAILURE; +}