From patchwork Thu Jul 14 17:01:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918272 Received: from mga06.intel.com (mga06b.intel.com [134.134.136.31]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B8E294C76 for ; Thu, 14 Jul 2022 17:02:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818122; x=1689354122; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=EpKSZRUg6s+nHr80Ig7EQknua3MGIHFjqj++oJFMvrE=; b=bLVtHL0q3wC78PQXVU5dcg0eI5DSpqu38ARX0GxnovuCjKVWG1FMLvZJ 8WQzSp+TbQFhAFBTVqEAHsNuKlZSF7uUHs0wTnNs3OAruLelgmxS1KxRz v9mgIBCaHH97JjW4z1HIzrybSD5yQMllUguS6jPnLMKk7RrY0+XgBSlGp mUDq3zGrd8XoBjdn+BrHhir8z0rmrLF6R5HdceJblBrfdLewrIFw+bjC+ vsBENlYgwRWCUXe/5e3j1UUVVAFnzSYUYloPtZnWrvSIvNZIulERLZV+V gvvngQU3RDwVYibMs5UxjJEHZhbAUTq3Q9DUCb8sLr4/tR9yvD/dvLCk7 Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="347254264" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="347254264" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga104.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:01:53 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="772693773" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:01:53 -0700 Subject: [ndctl PATCH v2 01/12] cxl/list: Reformat option list From: Dan Williams To: vishal.l.verma@intel.com Cc: Davidlohr Bueso , alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:01:52 -0700 Message-ID: <165781811294.1555691.6271986101970794441.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Cleanup some spurious spaces and let clang-format re-layout the options. Reviewed-by: Davidlohr Bueso Signed-off-by: Dan Williams --- cxl/list.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/cxl/list.c b/cxl/list.c index 940782d33a10..1b5f58328047 100644 --- a/cxl/list.c +++ b/cxl/list.c @@ -36,8 +36,7 @@ static const struct option options[] = { "filter by CXL endpoint device name(s)"), OPT_BOOLEAN('E', "endpoints", ¶m.endpoints, "include CXL endpoint info"), - OPT_STRING('d', "decoder", ¶m.decoder_filter, - "decoder device name", + OPT_STRING('d', "decoder", ¶m.decoder_filter, "decoder device name", "filter by CXL decoder device name(s) / class"), OPT_BOOLEAN('D', "decoders", ¶m.decoders, "include CXL decoder info"), @@ -45,11 +44,11 @@ static const struct option options[] = { "include CXL target data with decoders or ports"), OPT_BOOLEAN('i', "idle", ¶m.idle, "include disabled devices"), OPT_BOOLEAN('u', "human", ¶m.human, - "use human friendly number formats "), + "use human friendly number formats"), OPT_BOOLEAN('H', "health", ¶m.health, - "include memory device health information "), + "include memory device health information"), OPT_BOOLEAN('I', "partition", ¶m.partition, - "include memory device partition information "), + "include memory device partition information"), #ifdef ENABLE_DEBUG OPT_BOOLEAN(0, "debug", &debug, "debug list walk"), #endif From patchwork Thu Jul 14 17:01:58 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918275 Received: from mga07.intel.com (mga07.intel.com [134.134.136.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 1E78F6002 for ; Thu, 14 Jul 2022 17:02:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818134; x=1689354134; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=AeWJ82o6BwFn1NlqdqKUFKk+NqL3Dqc98exzk9j2WHE=; b=X2AtudmRmr7N4tss+jn5iKTk7ia781oIceBlz1fTETSef2ywEZ7lw7i+ N1bHoT2+aJsOv1azuazuBZa3s6T8YKEwMfFoiKScSWWYbPCMWkjNMP7qm xa+Yvj7q9LM1eq2cQLLGtWslXrHDT7aBz7Y3rHn1EqSIsWUOFGh9HTZvC OAN9PZzZtBGwTNHW/ju8FWJcLBo4wY9MXzVv1O94wVSS6bEVibXnfKXki kw9eosxMKhj4D335wsn4R+5WV3fPQwv/8W6232Esm3KBuMaiCdk6CHAC8 6/uS1jluGmf9xGGLI1o+77B80ionSoCxh0xT9ffAO+3IHHch47ASKYbvc A==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="349541992" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="349541992" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga105.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:01:59 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="685643797" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:01:58 -0700 Subject: [ndctl PATCH v2 02/12] cxl/list: Emit endpoint decoders filtered by memdev From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:01:58 -0700 Message-ID: <165781811836.1555691.1997564050287016121.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 For example, dump all the endpoint decoders from memdev 'mem8'. cxl list -Di -m 8 -d endpoint Signed-off-by: Dan Williams --- cxl/filter.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/cxl/filter.c b/cxl/filter.c index 66fd7420144a..2f88a9d2f398 100644 --- a/cxl/filter.c +++ b/cxl/filter.c @@ -428,7 +428,9 @@ util_cxl_decoder_filter_by_memdev(struct cxl_decoder *decoder, const char *ident, const char *serial) { struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + struct cxl_endpoint *endpoint; struct cxl_memdev *memdev; + struct cxl_port *port; if (!ident && !serial) return decoder; @@ -438,6 +440,12 @@ util_cxl_decoder_filter_by_memdev(struct cxl_decoder *decoder, continue; if (cxl_decoder_get_target_by_memdev(decoder, memdev)) return decoder; + port = cxl_decoder_get_port(decoder); + if (!port || !cxl_port_is_endpoint(port)) + continue; + endpoint = cxl_port_to_endpoint(port); + if (cxl_endpoint_get_memdev(endpoint) == memdev) + return decoder; } return NULL; From patchwork Thu Jul 14 17:02:04 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918274 Received: from mga12.intel.com (mga12.intel.com [192.55.52.136]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EF6C36002 for ; Thu, 14 Jul 2022 17:02:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818126; x=1689354126; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NgVTcO/N8lNlztGxQoim/NbcYQnweRCnSkSjx/KcjhI=; b=VfV1q8AnSpUVewbj/ganq7q/v4JupmkxArjtcpIPSpfrblqIqY3Oug1k wSbygV4UqUZpx3TKF5rJ5+3GdnnAfvESFjALjvbMbkMd8Kljfh+2mDik3 P5SkdjVmpdTvepPUl7sahaUsBJePBwe/WMyVAQ/tmO7yLpcNWT2UZq19+ JqnUwlOIGKr0KQ3vsaqef7a8ciIHnZE37mFIt194bN5UVFmxiqbpw9RFj NIoRdpU88fmMQkU66/20IfAYmLS6yKSPnu6Vg/bnPoCGHHDVHNClHfvR6 0ZKJWSI5MglGHJtYTm+YbsT0Kin1yTB5CnHPONtrvXO869D2DiUuCeVP7 A==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="265364058" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="265364058" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:04 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="772693904" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:04 -0700 Subject: [ndctl PATCH v2 03/12] cxl/list: Hide 0s in disabled decoder listings From: Dan Williams To: vishal.l.verma@intel.com Cc: Davidlohr Bueso , alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:04 -0700 Message-ID: <165781812427.1555691.5252994293073680408.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Trim some redundant information from decoder listings when they are disabled. Reviewed-by: Davidlohr Bueso Signed-off-by: Dan Williams --- cxl/json.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/cxl/json.c b/cxl/json.c index fdc6f73a86c1..a213fdad55fd 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -442,7 +442,7 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, const char *devname = cxl_decoder_get_devname(decoder); struct cxl_port *port = cxl_decoder_get_port(decoder); struct json_object *jdecoder, *jobj; - u64 val; + u64 val, size; jdecoder = json_object_new_object(); if (!jdecoder) @@ -452,21 +452,21 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, if (jobj) json_object_object_add(jdecoder, "decoder", jobj); + size = cxl_decoder_get_size(decoder); val = cxl_decoder_get_resource(decoder); - if (val < ULLONG_MAX) { + if (size && val < ULLONG_MAX) { jobj = util_json_object_hex(val, flags); if (jobj) json_object_object_add(jdecoder, "resource", jobj); } - val = cxl_decoder_get_size(decoder); - if (val < ULLONG_MAX) { - jobj = util_json_object_size(val, flags); + if (size && size < ULLONG_MAX) { + jobj = util_json_object_size(size, flags); if (jobj) json_object_object_add(jdecoder, "size", jobj); } - if (val == 0) { + if (size == 0) { jobj = json_object_new_string("disabled"); if (jobj) json_object_object_add(jdecoder, "state", jobj); From patchwork Thu Jul 14 17:02:09 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918280 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0DB6A6002 for ; Thu, 14 Jul 2022 17:02:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818179; x=1689354179; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=NR3xEc9IRdzsHzkHzpnmoYhBLcYxBL8o1b4hDwqsDX8=; b=b14al9Hlq6g+t4XK7+mVKpCv7K1B3Nf6lFgfqNBTTPaaJlwGfz33e2GB 1F+mqY0WvUYnPwprNri2aMAHu+adrvGjuxqNELhs8+UveJofGcB9Yn32l 1T7bd6zp+E1+XbfhyQk7FapzflWVCcYi1hT+LsjfQki1LApxDw+eBD/19 SOaKGyRDx17+o8bpUT5BeGVsagl2FggsKXpUpHaVT9plEayCMBxveLhpM RwHQq+TVMKk/ZLsfNV0YvbkG+LBs4UKftHXPsukGZSjfA/RrFdYgUnSHo 8BYCTJy+/CF3IDQ02XbEm4JRoQNubbped52P2ncILvpJA2spAR1IO/RFI g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="284331672" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="284331672" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:10 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="663862091" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga004-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:10 -0700 Subject: [ndctl PATCH v2 04/12] cxl/list: Add DPA span to endpoint decoder listings From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:09 -0700 Message-ID: <165781812967.1555691.4685129673233918478.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Optionally include in decoder listings the device local address space for endpoint decoders with active / allocated capacity. Signed-off-by: Dan Williams --- Documentation/cxl/lib/libcxl.txt | 2 ++ cxl/json.c | 18 ++++++++++++++++ cxl/lib/libcxl.c | 43 +++++++++++++++++++++++++++++++++++++- cxl/lib/libcxl.sym | 6 +++++ cxl/lib/private.h | 2 ++ cxl/libcxl.h | 2 ++ 6 files changed, 72 insertions(+), 1 deletion(-) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index f8f0e668ab59..2aef489e8e12 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -392,6 +392,8 @@ more CXL decoder objects. ---- unsigned long long cxl_decoder_get_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder); +unsigned long long cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder); +unsigned long long cxl_decoder_get_dpa_size(struct cxl_decoder *decoder); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); int cxl_decoder_get_id(struct cxl_decoder *decoder); int cxl_decoder_get_nr_targets(struct cxl_decoder *decoder); diff --git a/cxl/json.c b/cxl/json.c index a213fdad55fd..3f52d3bbff45 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -472,6 +472,24 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, json_object_object_add(jdecoder, "state", jobj); } + if (cxl_port_is_endpoint(port)) { + size = cxl_decoder_get_dpa_size(decoder); + val = cxl_decoder_get_dpa_resource(decoder); + if (size && val < ULLONG_MAX) { + jobj = util_json_object_hex(val, flags); + if (jobj) + json_object_object_add(jdecoder, "dpa_resource", + jobj); + } + + if (size && size < ULLONG_MAX) { + jobj = util_json_object_size(size, flags); + if (jobj) + json_object_object_add(jdecoder, "dpa_size", + jobj); + } + } + if (cxl_port_is_root(port) && cxl_decoder_is_mem_capable(decoder)) { if (cxl_decoder_is_pmem_capable(decoder)) { jobj = json_object_new_boolean(true); diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index c988ce2ddea9..f36edcfc735a 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -955,8 +955,19 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) decoder->size = strtoull(buf, NULL, 0); switch (port->type) { - case CXL_PORT_SWITCH: case CXL_PORT_ENDPOINT: + sprintf(path, "%s/dpa_resource", cxldecoder_base); + if (sysfs_read_attr(ctx, path, buf) < 0) + decoder->dpa_resource = ULLONG_MAX; + else + decoder->dpa_resource = strtoull(buf, NULL, 0); + sprintf(path, "%s/dpa_size", cxldecoder_base); + if (sysfs_read_attr(ctx, path, buf) < 0) + decoder->dpa_size = ULLONG_MAX; + else + decoder->dpa_size = strtoull(buf, NULL, 0); + + case CXL_PORT_SWITCH: decoder->pmem_capable = true; decoder->volatile_capable = true; decoder->mem_capable = true; @@ -1113,6 +1124,36 @@ CXL_EXPORT unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder) return decoder->size; } +CXL_EXPORT unsigned long long +cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return ULLONG_MAX; + } + + return decoder->dpa_resource; +} + +CXL_EXPORT unsigned long long +cxl_decoder_get_dpa_size(struct cxl_decoder *decoder) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return ULLONG_MAX; + } + + return decoder->dpa_size; +} + CXL_EXPORT enum cxl_decoder_target_type cxl_decoder_get_target_type(struct cxl_decoder *decoder) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index dffcb60b8dd0..8e2fc75557f9 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -167,3 +167,9 @@ global: cxl_cmd_new_set_partition; cxl_cmd_partition_set_mode; } LIBCXL_1; + +LIBCXL_3 { +global: + cxl_decoder_get_dpa_resource; + cxl_decoder_get_dpa_size; +} LIBCXL_2; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index c6d88f7140f2..24a2ae6787be 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -101,6 +101,8 @@ struct cxl_decoder { struct cxl_ctx *ctx; u64 start; u64 size; + u64 dpa_resource; + u64 dpa_size; void *dev_buf; size_t buf_len; char *dev_path; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 0007f4d9bcee..76aebe3efda8 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -129,6 +129,8 @@ struct cxl_decoder *cxl_decoder_get_first(struct cxl_port *port); struct cxl_decoder *cxl_decoder_get_next(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder); +unsigned long long cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder); +unsigned long long cxl_decoder_get_dpa_size(struct cxl_decoder *decoder); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); struct cxl_target *cxl_decoder_get_target_by_memdev(struct cxl_decoder *decoder, struct cxl_memdev *memdev); From patchwork Thu Jul 14 17:02:15 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918276 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C1ECF6002 for ; Thu, 14 Jul 2022 17:02:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818144; x=1689354144; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=eEjIKuScu5PsNnJ6b00Knr0ddNcwkSwfM5aF9xtCMgY=; b=LmhMzCOsYvbDheTAeg0JmQgZhfmz7ymf1OIAamkaBDHwTHXZkD/iNaJr jAw8qV06VuVS+T/hBVXBNpfmQBo1HB+ADuWlVuiY2uYORv5IlFbG97fjZ dKpzbjd2FSA3NlhgDbNd/X4ZY+V06s0Ze9t7I8rs5vW1F699qXTToQZyX s/NPMvxoL0RHbk3HNk5dY7XasEQ5PzMrp1YGsB5voUgwaVLNr8oNcPKux Wl+FXTGuA3deuYyYqYTTh4M7D3X1OhEeG6cO4M31uqITWs04YHn1W9KNK r0zmY6u1DBZN9esW4y9QsEax2XNgVdfTIs3za2C873u1Ptp8SsMuxY7Z8 g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="286316722" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="286316722" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:17 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="685643981" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga003-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:16 -0700 Subject: [ndctl PATCH v2 05/12] ccan/list: Import latest list helpers From: Dan Williams To: vishal.l.verma@intel.com Cc: Ira Weiny , alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:15 -0700 Message-ID: <165781813572.1555691.15909358688944168922.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Pick up the definition of list_add_{before,after} and other updates from ccan at commit 52b86922f846 ("ccan/base64: fix GCC warning."). Reported-by: Ira Weiny Signed-off-by: Dan Williams Reviewed-by: Ira Weiny --- ccan/list/list.h | 258 +++++++++++++++++++++++++++++++++++++++++++++------- ndctl/lib/inject.c | 1 util/list.h | 40 -------- 3 files changed, 222 insertions(+), 77 deletions(-) delete mode 100644 util/list.h diff --git a/ccan/list/list.h b/ccan/list/list.h index 3ebd1b23dc0f..15f5fb7b34eb 100644 --- a/ccan/list/list.h +++ b/ccan/list/list.h @@ -95,8 +95,8 @@ struct list_node *list_check_node(const struct list_node *n, #define list_debug(h, loc) list_check((h), loc) #define list_debug_node(n, loc) list_check_node((n), loc) #else -#define list_debug(h, loc) (h) -#define list_debug_node(n, loc) (n) +#define list_debug(h, loc) ((void)loc, h) +#define list_debug_node(n, loc) ((void)loc, n) #endif /** @@ -111,7 +111,7 @@ struct list_node *list_check_node(const struct list_node *n, * Example: * static struct list_head my_list = LIST_HEAD_INIT(my_list); */ -#define LIST_HEAD_INIT(name) { { &name.n, &name.n } } +#define LIST_HEAD_INIT(name) { { &(name).n, &(name).n } } /** * LIST_HEAD - define and initialize an empty list_head @@ -145,6 +145,48 @@ static inline void list_head_init(struct list_head *h) h->n.next = h->n.prev = &h->n; } +/** + * list_node_init - initialize a list_node + * @n: the list_node to link to itself. + * + * You don't need to use this normally! But it lets you list_del(@n) + * safely. + */ +static inline void list_node_init(struct list_node *n) +{ + n->next = n->prev = n; +} + +/** + * list_add_after - add an entry after an existing node in a linked list + * @h: the list_head to add the node to (for debugging) + * @p: the existing list_node to add the node after + * @n: the new list_node to add to the list. + * + * The existing list_node must already be a member of the list. + * The new list_node does not need to be initialized; it will be overwritten. + * + * Example: + * struct child c1, c2, c3; + * LIST_HEAD(h); + * + * list_add_tail(&h, &c1.list); + * list_add_tail(&h, &c3.list); + * list_add_after(&h, &c1.list, &c2.list); + */ +#define list_add_after(h, p, n) list_add_after_(h, p, n, LIST_LOC) +static inline void list_add_after_(struct list_head *h, + struct list_node *p, + struct list_node *n, + const char *abortstr) +{ + n->next = p->next; + n->prev = p; + p->next->prev = n; + p->next = n; + (void)list_debug(h, abortstr); +} + /** * list_add - add an entry at the start of a linked list. * @h: the list_head to add the node to @@ -163,10 +205,34 @@ static inline void list_add_(struct list_head *h, struct list_node *n, const char *abortstr) { - n->next = h->n.next; - n->prev = &h->n; - h->n.next->prev = n; - h->n.next = n; + list_add_after_(h, &h->n, n, abortstr); +} + +/** + * list_add_before - add an entry before an existing node in a linked list + * @h: the list_head to add the node to (for debugging) + * @p: the existing list_node to add the node before + * @n: the new list_node to add to the list. + * + * The existing list_node must already be a member of the list. + * The new list_node does not need to be initialized; it will be overwritten. + * + * Example: + * list_head_init(&h); + * list_add_tail(&h, &c1.list); + * list_add_tail(&h, &c3.list); + * list_add_before(&h, &c3.list, &c2.list); + */ +#define list_add_before(h, p, n) list_add_before_(h, p, n, LIST_LOC) +static inline void list_add_before_(struct list_head *h, + struct list_node *p, + struct list_node *n, + const char *abortstr) +{ + n->next = p; + n->prev = p->prev; + p->prev->next = n; + p->prev = n; (void)list_debug(h, abortstr); } @@ -185,11 +251,7 @@ static inline void list_add_tail_(struct list_head *h, struct list_node *n, const char *abortstr) { - n->next = &h->n; - n->prev = h->n.prev; - h->n.prev->next = n; - h->n.prev = n; - (void)list_debug(h, abortstr); + list_add_before_(h, &h->n, n, abortstr); } /** @@ -229,6 +291,21 @@ static inline bool list_empty_nodebug(const struct list_head *h) } #endif +/** + * list_empty_nocheck - is a list empty? + * @h: the list_head + * + * If the list is empty, returns true. This doesn't perform any + * debug check for list consistency, so it can be called without + * locks, racing with the list being modified. This is ok for + * checks where an incorrect result is not an issue (optimized + * bail out path for example). + */ +static inline bool list_empty_nocheck(const struct list_head *h) +{ + return h->n.next == &h->n; +} + /** * list_del - delete an entry from an (unknown) linked list. * @n: the list_node to delete from the list. @@ -237,7 +314,7 @@ static inline bool list_empty_nodebug(const struct list_head *h) * another list, but not deleted again. * * See also: - * list_del_from() + * list_del_from(), list_del_init() * * Example: * list_del(&child->list); @@ -255,6 +332,27 @@ static inline void list_del_(struct list_node *n, const char* abortstr) #endif } +/** + * list_del_init - delete a node, and reset it so it can be deleted again. + * @n: the list_node to be deleted. + * + * list_del(@n) or list_del_init() again after this will be safe, + * which can be useful in some cases. + * + * See also: + * list_del_from(), list_del() + * + * Example: + * list_del_init(&child->list); + * parent->num_children--; + */ +#define list_del_init(n) list_del_init_(n, LIST_LOC) +static inline void list_del_init_(struct list_node *n, const char *abortstr) +{ + list_del_(n, abortstr); + list_node_init(n); +} + /** * list_del_from - delete an entry from a known linked list. * @h: the list_head the node is in. @@ -285,6 +383,39 @@ static inline void list_del_from(struct list_head *h, struct list_node *n) list_del(n); } +/** + * list_swap - swap out an entry from an (unknown) linked list for a new one. + * @o: the list_node to replace from the list. + * @n: the list_node to insert in place of the old one. + * + * Note that this leaves @o in an undefined state; it can be added to + * another list, but not deleted/swapped again. + * + * See also: + * list_del() + * + * Example: + * struct child x1, x2; + * LIST_HEAD(xh); + * + * list_add(&xh, &x1.list); + * list_swap(&x1.list, &x2.list); + */ +#define list_swap(o, n) list_swap_(o, n, LIST_LOC) +static inline void list_swap_(struct list_node *o, + struct list_node *n, + const char* abortstr) +{ + (void)list_debug_node(o, abortstr); + *n = *o; + n->next->prev = n; + n->prev->next = n; +#ifdef CCAN_LIST_DEBUG + /* Catch use-after-del. */ + o->next = o->prev = NULL; +#endif +} + /** * list_entry - convert a list_node back into the structure containing it. * @n: the list_node @@ -406,9 +537,29 @@ static inline const void *list_tail_(const struct list_head *h, size_t off) * printf("Name: %s\n", child->name); */ #define list_for_each_rev(h, i, member) \ - for (i = container_of_var(list_debug(h, LIST_LOC)->n.prev, i, member); \ - &i->member != &(h)->n; \ - i = container_of_var(i->member.prev, i, member)) + list_for_each_rev_off(h, i, list_off_var_(i, member)) + +/** + * list_for_each_rev_safe - iterate through a list backwards, + * maybe during deletion + * @h: the list_head + * @i: the structure containing the list_node + * @nxt: the structure containing the list_node + * @member: the list_node member of the structure + * + * This is a convenient wrapper to iterate @i over the entire list backwards. + * It's a for loop, so you can break and continue as normal. The extra + * variable * @nxt is used to hold the next element, so you can delete @i + * from the list. + * + * Example: + * struct child *next; + * list_for_each_rev_safe(&parent->children, child, next, list) { + * printf("Name: %s\n", child->name); + * } + */ +#define list_for_each_rev_safe(h, i, nxt, member) \ + list_for_each_rev_safe_off(h, i, nxt, list_off_var_(i, member)) /** * list_for_each_safe - iterate through a list, maybe during deletion @@ -422,7 +573,6 @@ static inline const void *list_tail_(const struct list_head *h, size_t off) * @nxt is used to hold the next element, so you can delete @i from the list. * * Example: - * struct child *next; * list_for_each_safe(&parent->children, child, next, list) { * list_del(&child->list); * parent->num_children--; @@ -537,10 +687,28 @@ static inline void list_prepend_list_(struct list_head *to, list_head_init(from); } +/* internal macros, do not use directly */ +#define list_for_each_off_dir_(h, i, off, dir) \ + for (i = list_node_to_off_(list_debug(h, LIST_LOC)->n.dir, \ + (off)); \ + list_node_from_off_((void *)i, (off)) != &(h)->n; \ + i = list_node_to_off_(list_node_from_off_((void *)i, (off))->dir, \ + (off))) + +#define list_for_each_safe_off_dir_(h, i, nxt, off, dir) \ + for (i = list_node_to_off_(list_debug(h, LIST_LOC)->n.dir, \ + (off)), \ + nxt = list_node_to_off_(list_node_from_off_(i, (off))->dir, \ + (off)); \ + list_node_from_off_(i, (off)) != &(h)->n; \ + i = nxt, \ + nxt = list_node_to_off_(list_node_from_off_(i, (off))->dir, \ + (off))) + /** * list_for_each_off - iterate through a list of memory regions. * @h: the list_head - * @i: the pointer to a memory region wich contains list node data. + * @i: the pointer to a memory region which contains list node data. * @off: offset(relative to @i) at which list node data resides. * * This is a low-level wrapper to iterate @i over the entire list, used to @@ -548,12 +716,12 @@ static inline void list_prepend_list_(struct list_head *to, * so you can break and continue as normal. * * WARNING! Being the low-level macro that it is, this wrapper doesn't know - * nor care about the type of @i. The only assumtion made is that @i points + * nor care about the type of @i. The only assumption made is that @i points * to a chunk of memory that at some @offset, relative to @i, contains a - * properly filled `struct node_list' which in turn contains pointers to - * memory chunks and it's turtles all the way down. Whith all that in mind + * properly filled `struct list_node' which in turn contains pointers to + * memory chunks and it's turtles all the way down. With all that in mind * remember that given the wrong pointer/offset couple this macro will - * happilly churn all you memory untill SEGFAULT stops it, in other words + * happily churn all you memory until SEGFAULT stops it, in other words * caveat emptor. * * It is worth mentioning that one of legitimate use-cases for that wrapper @@ -567,17 +735,24 @@ static inline void list_prepend_list_(struct list_head *to, * printf("Name: %s\n", child->name); */ #define list_for_each_off(h, i, off) \ - for (i = list_node_to_off_(list_debug(h, LIST_LOC)->n.next, \ - (off)); \ - list_node_from_off_((void *)i, (off)) != &(h)->n; \ - i = list_node_to_off_(list_node_from_off_((void *)i, (off))->next, \ - (off))) + list_for_each_off_dir_((h),(i),(off),next) + +/** + * list_for_each_rev_off - iterate through a list of memory regions backwards + * @h: the list_head + * @i: the pointer to a memory region which contains list node data. + * @off: offset(relative to @i) at which list node data resides. + * + * See list_for_each_off for details + */ +#define list_for_each_rev_off(h, i, off) \ + list_for_each_off_dir_((h),(i),(off),prev) /** * list_for_each_safe_off - iterate through a list of memory regions, maybe * during deletion * @h: the list_head - * @i: the pointer to a memory region wich contains list node data. + * @i: the pointer to a memory region which contains list node data. * @nxt: the structure containing the list_node * @off: offset(relative to @i) at which list node data resides. * @@ -590,15 +765,26 @@ static inline void list_prepend_list_(struct list_head *to, * printf("Name: %s\n", child->name); */ #define list_for_each_safe_off(h, i, nxt, off) \ - for (i = list_node_to_off_(list_debug(h, LIST_LOC)->n.next, \ - (off)), \ - nxt = list_node_to_off_(list_node_from_off_(i, (off))->next, \ - (off)); \ - list_node_from_off_(i, (off)) != &(h)->n; \ - i = nxt, \ - nxt = list_node_to_off_(list_node_from_off_(i, (off))->next, \ - (off))) + list_for_each_safe_off_dir_((h),(i),(nxt),(off),next) +/** + * list_for_each_rev_safe_off - iterate backwards through a list of + * memory regions, maybe during deletion + * @h: the list_head + * @i: the pointer to a memory region which contains list node data. + * @nxt: the structure containing the list_node + * @off: offset(relative to @i) at which list node data resides. + * + * For details see `list_for_each_rev_off' and `list_for_each_rev_safe' + * descriptions. + * + * Example: + * list_for_each_rev_safe_off(&parent->children, child, + * next, offsetof(struct child, list)) + * printf("Name: %s\n", child->name); + */ +#define list_for_each_rev_safe_off(h, i, nxt, off) \ + list_for_each_safe_off_dir_((h),(i),(nxt),(off),prev) /* Other -off variants. */ #define list_entry_off(n, type, off) \ diff --git a/ndctl/lib/inject.c b/ndctl/lib/inject.c index d61c02c176e2..3486ffefc40a 100644 --- a/ndctl/lib/inject.c +++ b/ndctl/lib/inject.c @@ -2,7 +2,6 @@ // Copyright (C) 2014-2020, Intel Corporation. All rights reserved. #include #include -#include #include #include #include diff --git a/util/list.h b/util/list.h deleted file mode 100644 index 1ea9c59b9f0c..000000000000 --- a/util/list.h +++ /dev/null @@ -1,40 +0,0 @@ -/* SPDX-License-Identifier: GPL-2.0 */ -/* Copyright (C) 2015-2020 Intel Corporation. All rights reserved. */ -#ifndef _NDCTL_LIST_H_ -#define _NDCTL_LIST_H_ - -#include - -/** - * list_add_after - add an entry after the given node in the linked list. - * @h: the list_head to add the node to - * @l: the list_node after which to add to - * @n: the list_node to add to the list. - * - * The list_node does not need to be initialized; it will be overwritten. - * Example: - * struct child *child = malloc(sizeof(*child)); - * - * child->name = "geoffrey"; - * list_add_after(&parent->children, &child1->list, &child->list); - * parent->num_children++; - */ -#define list_add_after(h, l, n) list_add_after_(h, l, n, LIST_LOC) -static inline void list_add_after_(struct list_head *h, - struct list_node *l, - struct list_node *n, - const char *abortstr) -{ - if (l->next == &h->n) { - /* l is the last element, this becomes a list_add_tail */ - list_add_tail(h, n); - return; - } - n->next = l->next; - n->prev = l; - l->next->prev = n; - l->next = n; - (void)list_debug(h, abortstr); -} - -#endif /* _NDCTL_LIST_H_ */ From patchwork Thu Jul 14 17:02:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918277 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3F39F6002 for ; Thu, 14 Jul 2022 17:02:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818156; x=1689354156; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=8pHDEH3ziJVotJ1oYBiooCwlZBWAUamzVCenaWfGOWU=; b=aiftfanPxMMj6MMkQnf4psmp2ALZfiyRrp3rxWr4vqrDq6aXZtzsFVth i38oatHLRxq8O2of11rGjeu+IRnymcy1Q9SFSnCXhRe5m3vUac4SPHTVE MwMy+VcrIiSIbQashaHUOiPfWQGavO7f+7Hc0XHl5A4bi1H2afLK2hzgv D66h11M4xKXbhCfp3ClO5jf6pMWgE/IiZlM3/LedBloGt/w/mGIOAKNOn GGapSFwrKienW7aAs12nCUipzknc0tokR/OFDQw00Ou0NGTTo9wTqPBZg fKYQpq3Plv8ZSzrcLSWLIQByGNc4eJqxT7CfkpwZEOf5Y9+X0zgFQ1+Rs w==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="268602718" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="268602718" Received: from fmsmga006.fm.intel.com ([10.253.24.20]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:31 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="842214200" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga006-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:22 -0700 Subject: [ndctl PATCH v2 06/12] cxl/lib: Maintain decoders in id order From: Dan Williams To: vishal.l.verma@intel.com Cc: Davidlohr Bueso , alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:21 -0700 Message-ID: <165781814167.1555691.14895625637451030942.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Given that decoder instance order is fundamental to the DPA translation sequence for endpoint decoders, enforce that cxl_decoder_for_each() returns decoders in instance order. Otherwise, they show up in readddir() order which is not predictable. Add a list_add_sorted() to generically handle inserting into a sorted list. Reviewed-by: Davidlohr Bueso Signed-off-by: Dan Williams --- cxl/lib/libcxl.c | 8 +++++++- util/list.h | 39 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 46 insertions(+), 1 deletion(-) create mode 100644 util/list.h diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index f36edcfc735a..e4c5d3819e88 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -19,6 +19,7 @@ #include #include +#include #include #include #include @@ -908,6 +909,11 @@ cxl_endpoint_get_memdev(struct cxl_endpoint *endpoint) return NULL; } +static int decoder_id_cmp(struct cxl_decoder *d1, struct cxl_decoder *d2) +{ + return d1->id - d2->id; +} + static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) { const char *devname = devpath_to_devname(cxldecoder_base); @@ -1049,7 +1055,7 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) return decoder_dup; } - list_add(&port->decoders, &decoder->list); + list_add_sorted(&port->decoders, decoder, list, decoder_id_cmp); free(path); return decoder; diff --git a/util/list.h b/util/list.h new file mode 100644 index 000000000000..cb7727123ea8 --- /dev/null +++ b/util/list.h @@ -0,0 +1,39 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2022 Intel Corporation. All rights reserved. */ +#ifndef _NDCTL_LIST_H_ +#define _NDCTL_LIST_H_ + +#include + +#define list_add_sorted(head, n, node, cmp) \ + do { \ + struct list_head *__head = (head); \ + typeof(n) __iter, __next; \ + typeof(n) __new = (n); \ + \ + if (list_empty(__head)) { \ + list_add(__head, &__new->node); \ + break; \ + } \ + \ + list_for_each (__head, __iter, node) { \ + if (cmp(__new, __iter) < 0) { \ + list_add_before(__head, &__iter->node, \ + &__new->node); \ + break; \ + } \ + __next = list_next(__head, __iter, node); \ + if (!__next) { \ + list_add_after(__head, &__iter->node, \ + &__new->node); \ + break; \ + } \ + if (cmp(__new, __next) < 0) { \ + list_add_before(__head, &__next->node, \ + &__new->node); \ + break; \ + } \ + } \ + } while (0) + +#endif /* _NDCTL_LIST_H_ */ From patchwork Thu Jul 14 17:02:27 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918282 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 33C616002 for ; Thu, 14 Jul 2022 17:03:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818191; x=1689354191; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=z2SMMm41uIRd5DJN5abWRnM6MsMByqAH5tJ7qm43uuc=; b=EMIx7pQycska+Mrtt3wUSc92fxxthYMg0tjs8Lfs1mLhZvEyiwFYp9qb /lUOM9OK155cOtUFhuM1XsqzBJ5O1HsBQFRoIOM2HzdGDuJiHWC9iG0Vo TH+RXcCCbedV7WUXQSBW0qiwYZzuntS50t6aVoIw+fMl7q54vB6bCsXao s8wtAk16AvYzHfskoFyWulc9EjYik7ZogFfo808Jocz2ia2t1BDgDBsCb cZyLnmuEYUU4TnXdot0Ze4KOLopBNIA5MBhQsHS/MkfPs++SoIaxoW9uG QQoE7ZVzd8NSy+xy+vXP+iqv+tbkdHU7jcfMTy478wyrJntlLlhUI6YcA A==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="284331794" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="284331794" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:28 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="653953199" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:27 -0700 Subject: [ndctl PATCH v2 07/12] cxl/memdev: Fix json for multi-device partitioning From: Dan Williams To: vishal.l.verma@intel.com Cc: Alison Schofield , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:27 -0700 Message-ID: <165781814737.1555691.889129128205037941.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In the case when someone partitions several devices at once, collect all the affected memdevs into a json array. With the move to use util_display_json_array() that also requires a set of flags to be specifiied. Apply the UTIL_JSON_HUMAN flag for all interactive command result output to bring this command in line with other tools. Cc: Alison Schofield Signed-off-by: Dan Williams --- cxl/memdev.c | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-) diff --git a/cxl/memdev.c b/cxl/memdev.c index 91d914db5af6..9fcd8ae5724b 100644 --- a/cxl/memdev.c +++ b/cxl/memdev.c @@ -19,6 +19,7 @@ struct action_context { FILE *f_out; FILE *f_in; + struct json_object *jdevs; }; static struct parameters { @@ -339,12 +340,13 @@ out: } static int action_setpartition(struct cxl_memdev *memdev, - struct action_context *actx) + struct action_context *actx) { const char *devname = cxl_memdev_get_devname(memdev); enum cxl_setpart_type type = CXL_SETPART_PMEM; unsigned long long size = ULLONG_MAX; struct json_object *jmemdev; + unsigned long flags; struct cxl_cmd *cmd; int rc; @@ -396,10 +398,12 @@ out_err: if (rc) log_err(&ml, "%s error: %s\n", devname, strerror(-rc)); - jmemdev = util_cxl_memdev_to_json(memdev, UTIL_JSON_PARTITION); - if (jmemdev) - printf("%s\n", json_object_to_json_string_ext(jmemdev, - JSON_C_TO_STRING_PRETTY)); + flags = UTIL_JSON_PARTITION; + if (actx->f_out == stdout && isatty(1)) + flags |= UTIL_JSON_HUMAN; + jmemdev = util_cxl_memdev_to_json(memdev, flags); + if (actx->jdevs && jmemdev) + json_object_array_add(actx->jdevs, jmemdev); return rc; } @@ -446,6 +450,9 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, err++; } + if (action == action_setpartition) + actx.jdevs = json_object_new_array(); + if (err == argc) { usage_with_options(u, options); return -EINVAL; @@ -528,6 +535,15 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, if (actx.f_in != stdin) fclose(actx.f_in); + if (actx.jdevs) { + unsigned long flags = 0; + + if (actx.f_out == stdout && isatty(1)) + flags |= UTIL_JSON_HUMAN; + util_display_json_array(actx.f_out, actx.jdevs, flags); + } + + out_close_fout: if (actx.f_out != stdout) fclose(actx.f_out); From patchwork Thu Jul 14 17:02:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918281 Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C8A7E6002 for ; Thu, 14 Jul 2022 17:03:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818182; x=1689354182; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=R/HiYanKlcDyL1XhX9Xghwwfcblt81s6EZC3V86Gj74=; b=YPw3TinMXy+VgrL1dOV+ohHolLIEF4MI03gLfndNO1YaYI+Z0lyZwjou 73ui4hG6fDBjl/D0dhh1treDo0IK0vtUguhcRyhzIuwu1QcI2r1cGEo/+ M0PCBX7ZqLHkFKnBA66VmYngHKnMSg8vWE1Hc6NiUyEimtQHwk6aVcURw voQag3SPTA5UD4HPTyCDWuL9JKLFDaPUBZPt18XQwgUbfixkjs+DKor69 BcuvTvCGAkQw9F6z29y62YhoLwgj7/vHM2FMnrKEU7+jg6UUp+x1wIgqt vSi1B3IxfV0QNhJQbtT6cincTOcbUq96LZ7U4+0fcBjpNmxJwzZiLyu0k w==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="283132515" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="283132515" Received: from fmsmga008.fm.intel.com ([10.253.24.58]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:33 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="653953217" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by fmsmga008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:33 -0700 Subject: [ndctl PATCH v2 08/12] cxl/list: Emit 'mode' for endpoint decoder objects From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:33 -0700 Message-ID: <165781815306.1555691.17541956592287631419.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 The 'mode' property of an endpoint decoder indicates the access properties of the DPA (device physical address) mapped into HPA (host physical address) by the decoder. Where the modes are 'none' (decoder-disabled), 'ram' (voltaile memory), 'pmem' (persistent memory), and 'mixed' (an unexpected, but possible, case where the decoder straddles a mode / partition boundary). Signed-off-by: Dan Williams --- Documentation/cxl/lib/libcxl.txt | 9 +++++++++ cxl/json.c | 8 ++++++++ cxl/lib/libcxl.c | 30 ++++++++++++++++++++++++++++++ cxl/lib/libcxl.sym | 1 + cxl/lib/private.h | 1 + cxl/libcxl.h | 23 +++++++++++++++++++++++ 6 files changed, 72 insertions(+) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index 2aef489e8e12..90fe33887821 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -405,6 +405,15 @@ enum cxl_decoder_target_type { }; cxl_decoder_get_target_type(struct cxl_decoder *decoder); + +enum cxl_decoder_mode { + CXL_DECODER_MODE_NONE, + CXL_DECODER_MODE_MIXED, + CXL_DECODER_MODE_PMEM, + CXL_DECODER_MODE_RAM, +}; +enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); + bool cxl_decoder_is_pmem_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_volatile_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_mem_capable(struct cxl_decoder *decoder); diff --git a/cxl/json.c b/cxl/json.c index 3f52d3bbff45..ae9c8126f14f 100644 --- a/cxl/json.c +++ b/cxl/json.c @@ -473,6 +473,8 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, } if (cxl_port_is_endpoint(port)) { + enum cxl_decoder_mode mode = cxl_decoder_get_mode(decoder); + size = cxl_decoder_get_dpa_size(decoder); val = cxl_decoder_get_dpa_resource(decoder); if (size && val < ULLONG_MAX) { @@ -488,6 +490,12 @@ struct json_object *util_cxl_decoder_to_json(struct cxl_decoder *decoder, json_object_object_add(jdecoder, "dpa_size", jobj); } + + if (mode > CXL_DECODER_MODE_NONE) { + jobj = json_object_new_string(cxl_decoder_mode_name(mode)); + if (jobj) + json_object_object_add(jdecoder, "mode", jobj); + } } if (cxl_port_is_root(port) && cxl_decoder_is_mem_capable(decoder)) { diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index e4c5d3819e88..a4709175678d 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -960,6 +960,21 @@ static void *add_cxl_decoder(void *parent, int id, const char *cxldecoder_base) else decoder->size = strtoull(buf, NULL, 0); + sprintf(path, "%s/mode", cxldecoder_base); + if (sysfs_read_attr(ctx, path, buf) == 0) { + if (strcmp(buf, "ram") == 0) + decoder->mode = CXL_DECODER_MODE_RAM; + else if (strcmp(buf, "pmem") == 0) + decoder->mode = CXL_DECODER_MODE_PMEM; + else if (strcmp(buf, "mixed") == 0) + decoder->mode = CXL_DECODER_MODE_MIXED; + else if (strcmp(buf, "none") == 0) + decoder->mode = CXL_DECODER_MODE_NONE; + else + decoder->mode = CXL_DECODER_MODE_MIXED; + } else + decoder->mode = CXL_DECODER_MODE_NONE; + switch (port->type) { case CXL_PORT_ENDPOINT: sprintf(path, "%s/dpa_resource", cxldecoder_base); @@ -1160,6 +1175,21 @@ cxl_decoder_get_dpa_size(struct cxl_decoder *decoder) return decoder->dpa_size; } +CXL_EXPORT enum cxl_decoder_mode +cxl_decoder_get_mode(struct cxl_decoder *decoder) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return CXL_DECODER_MODE_NONE; + } + + return decoder->mode; +} + CXL_EXPORT enum cxl_decoder_target_type cxl_decoder_get_target_type(struct cxl_decoder *decoder) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 8e2fc75557f9..88c5a7edd33e 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -172,4 +172,5 @@ LIBCXL_3 { global: cxl_decoder_get_dpa_resource; cxl_decoder_get_dpa_size; + cxl_decoder_get_mode; } LIBCXL_2; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index 24a2ae6787be..f6d4573757fd 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -108,6 +108,7 @@ struct cxl_decoder { char *dev_path; int nr_targets; int id; + enum cxl_decoder_mode mode; bool pmem_capable; bool volatile_capable; bool mem_capable; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 76aebe3efda8..1436dc4601cc 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -127,10 +127,33 @@ struct cxl_dport *cxl_port_get_dport_by_memdev(struct cxl_port *port, struct cxl_decoder; struct cxl_decoder *cxl_decoder_get_first(struct cxl_port *port); struct cxl_decoder *cxl_decoder_get_next(struct cxl_decoder *decoder); +struct cxl_decoder *cxl_decoder_get_last(struct cxl_port *port); +struct cxl_decoder *cxl_decoder_get_prev(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_dpa_size(struct cxl_decoder *decoder); +enum cxl_decoder_mode { + CXL_DECODER_MODE_NONE, + CXL_DECODER_MODE_MIXED, + CXL_DECODER_MODE_PMEM, + CXL_DECODER_MODE_RAM, +}; +static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) +{ + static const char *names[] = { + [CXL_DECODER_MODE_NONE] = "none", + [CXL_DECODER_MODE_MIXED] = "mixed", + [CXL_DECODER_MODE_PMEM] = "pmem", + [CXL_DECODER_MODE_RAM] = "ram", + }; + + if (mode < CXL_DECODER_MODE_NONE || mode > CXL_DECODER_MODE_RAM) + mode = CXL_DECODER_MODE_NONE; + return names[mode]; +} + +enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); struct cxl_target *cxl_decoder_get_target_by_memdev(struct cxl_decoder *decoder, struct cxl_memdev *memdev); From patchwork Thu Jul 14 17:02:38 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918284 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 699B66006 for ; Thu, 14 Jul 2022 17:03:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818194; x=1689354194; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=d6ApsR6hlkYxBPeMgixmyfChQYOVY7n3l02N3ta4pdo=; b=jUpEHwEeyafuyPMYPizFpVaWdksqlyMlBHQ2sKAV3Iu2cv96IzTDo527 ddJk2ytDul9IE4b7uZ9DmHaMenI7YtpRWGq133AK8Uk4nws88/FvR60nj nvBfvL+zW3kZRWpReMALJAIW8qpCv0UoOk4KmOdf8xVryfjrtV5+9/pe1 KmeyMlv6nOdaBUGgKTLMj7WckwBC4jU0b5OLi6NVF332UaskS+M0e2ToJ pizS2o166bZT+28uL2lgNULIy68vO0hzB/WUXWG7818LujCg2pWzM8ui6 dvxB1ZeaXMn1fKqp+ZbTkGRv5vj2XMxrSDlhcczZ7JldyDf4OpNLrVBVO w==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="284331850" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="284331850" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:39 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="593438298" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga007-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:39 -0700 Subject: [ndctl PATCH v2 09/12] cxl/set-partition: Accept 'ram' as an alias for 'volatile' From: Dan Williams To: vishal.l.verma@intel.com Cc: Alison Schofield , Davidlohr Bueso , nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:38 -0700 Message-ID: <165781815878.1555691.12251226240559355924.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 'ram' is a more convenient shorthand for volatile memory. Cc: Alison Schofield Reviewed-by: Davidlohr Bueso Signed-off-by: Dan Williams --- Documentation/cxl/cxl-set-partition.txt | 2 +- cxl/memdev.c | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/Documentation/cxl/cxl-set-partition.txt b/Documentation/cxl/cxl-set-partition.txt index 1e548af77da2..f0126daf808b 100644 --- a/Documentation/cxl/cxl-set-partition.txt +++ b/Documentation/cxl/cxl-set-partition.txt @@ -37,7 +37,7 @@ include::memdev-option.txt[] -t:: --type=:: - Type of partition, 'pmem' or 'volatile', to modify. + Type of partition, 'pmem' or 'ram' (volatile), to modify. Default: 'pmem' -s:: diff --git a/cxl/memdev.c b/cxl/memdev.c index 9fcd8ae5724b..1cecad2dba4b 100644 --- a/cxl/memdev.c +++ b/cxl/memdev.c @@ -65,7 +65,7 @@ OPT_BOOLEAN('f', "force", ¶m.force, \ #define SET_PARTITION_OPTIONS() \ OPT_STRING('t', "type", ¶m.type, "type", \ - "'pmem' or 'volatile' (Default: 'pmem')"), \ + "'pmem' or 'ram' (volatile) (Default: 'pmem')"), \ OPT_STRING('s', "size", ¶m.size, "size", \ "size in bytes (Default: all available capacity)"), \ OPT_BOOLEAN('a', "align", ¶m.align, \ @@ -355,6 +355,8 @@ static int action_setpartition(struct cxl_memdev *memdev, /* default */; else if (strcmp(param.type, "volatile") == 0) type = CXL_SETPART_VOLATILE; + else if (strcmp(param.type, "ram") == 0) + type = CXL_SETPART_VOLATILE; else { log_err(&ml, "invalid type '%s'\n", param.type); return -EINVAL; From patchwork Thu Jul 14 17:02:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918278 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 3C6976003 for ; Thu, 14 Jul 2022 17:02:45 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818165; x=1689354165; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=u7l+qE05EyPGBjp2Bg9ejGd8ChU7Kpr9hP8IMycY+WY=; b=UFsUgmidm0iTOAiCFJNrTfFA51qaWb0+RcX82WcutQ0LHTSuLaXeBbhp j3MHpHkGOV6iF14O0gkejbLljFmlhO0bqOk1/MA52Av5s1FBFNL/BgHv1 34+yQbfWpq0a/ryI9J3sY+5RlDWKx4G/ZsYxNzXJ/GBAfjixqD66/Xu65 IU0xtUUMuyz1SVGiIWO4MS7AclcFd0FGKQle67Ku1V11kL0nl9s8BEwvy UeagfXI9M70r1SMCgW+Gb8OaHQOKt+aQOw8KpRfh0a/I2dJaWy0epZTsS goCOaRfmtsVMWMHY8mBIGwJVIvNsyj/nxH9ccAFS6ci0SYIF4wEA5strl A==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="268602796" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="268602796" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:44 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="772694169" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:44 -0700 Subject: [ndctl PATCH v2 10/12] cxl/memdev: Add {reserve,free}-dpa commands From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:44 -0700 Message-ID: <165781816425.1555691.17958897857798325111.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add helper commands for managing allocations of DPA (device physical address) capacity on a set of CXL memory devices. The main convenience this command affords is automatically picking the next decoder to allocate per-memdev. For example, to allocate 256MiB from all endpoints that are covered by a given root decoder, and collect those resulting endpoint-decoders into an array: readarray -t mem < <(cxl list -M -d $decoder | jq -r ".[].memdev") readarray -t endpoint < <(cxl reserve-dpa -t pmem ${mem[*]} -s $((256<<20)) | jq -r ".[] | .decoder.decoder") Signed-off-by: Dan Williams --- .clang-format | 1 Documentation/cxl/cxl-free-dpa.txt | 53 ++++++ Documentation/cxl/cxl-reserve-dpa.txt | 67 ++++++++ Documentation/cxl/lib/libcxl.txt | 2 Documentation/cxl/meson.build | 2 cxl/builtin.h | 2 cxl/cxl.c | 2 cxl/filter.c | 4 cxl/filter.h | 2 cxl/lib/libcxl.c | 86 ++++++++++ cxl/lib/libcxl.sym | 4 cxl/libcxl.h | 9 + cxl/memdev.c | 280 +++++++++++++++++++++++++++++++++ 13 files changed, 511 insertions(+), 3 deletions(-) create mode 100644 Documentation/cxl/cxl-free-dpa.txt create mode 100644 Documentation/cxl/cxl-reserve-dpa.txt diff --git a/.clang-format b/.clang-format index 6aabcb68e6a9..7254a1b15738 100644 --- a/.clang-format +++ b/.clang-format @@ -81,6 +81,7 @@ ForEachMacros: - 'cxl_bus_foreach' - 'cxl_port_foreach' - 'cxl_decoder_foreach' + - 'cxl_decoder_foreach_reverse' - 'cxl_target_foreach' - 'cxl_dport_foreach' - 'cxl_endpoint_foreach' diff --git a/Documentation/cxl/cxl-free-dpa.txt b/Documentation/cxl/cxl-free-dpa.txt new file mode 100644 index 000000000000..73fb048fdee3 --- /dev/null +++ b/Documentation/cxl/cxl-free-dpa.txt @@ -0,0 +1,53 @@ +// SPDX-License-Identifier: GPL-2.0 + +cxl-free-dpa(1) +=============== + +NAME +---- +cxl-free-dpa - release device-physical address space + +SYNOPSIS +-------- +[verse] +'cxl free-dpa' [..] [] + +The CXL region provisioning process proceeds in multiple steps. One of +the steps is identifying and reserving the DPA span that each member of +the interleave-set (region) contributes in advance of attaching that +allocation to a region. For development, test, and debug purposes this +command is a helper to find the last allocated decoder on a device and +zero-out / free its DPA allocation. + +OPTIONS +------- +:: +include::memdev-option.txt[] + +-d:: +--decoder:: + Specify the decoder to free. The CXL specification + mandates that DPA must be released in the reverse order it was + allocated. See linkcxl:cxl-reserve-dpa[1] + +-t:: +--type:: + Constrain the search for "last allocated decoder" to decoders targeting + the given partition. + +-f:: +--force:: + The kernel enforces CXL DPA ordering constraints on deallocation events, + and the tool anticipates those and fails operations that are expected to + fail without sending them to the kernel. For test purposes, continue to + attempt "expected to fail" operations to exercise the driver. + +-v:: + Turn on verbose debug messages in the library (if libcxl was built with + logging and debug enabled). + +include::../copyright.txt[] + +SEE ALSO +-------- +linkcxl:cxl-reserve-dpa[1] diff --git a/Documentation/cxl/cxl-reserve-dpa.txt b/Documentation/cxl/cxl-reserve-dpa.txt new file mode 100644 index 000000000000..560daf0cb277 --- /dev/null +++ b/Documentation/cxl/cxl-reserve-dpa.txt @@ -0,0 +1,67 @@ +// SPDX-License-Identifier: GPL-2.0 + +cxl-reserve-dpa(1) +================== + +NAME +---- +cxl-reserve-dpa - allocate device-physical address space + +SYNOPSIS +-------- +[verse] +'cxl reserve-dpa' [..] [] + +The CXL region provisioning process proceeds in multiple steps. One of +the steps is identifying and reserving the DPA span that each member of +the interleave-set (region) contributes in advance of attaching that +allocation to a region. For development, test, and debug purposes this +command is a helper to find the next available decoder on endpoint +(memdev) and mark a span of DPA as busy. + +OPTIONS +------- +:: +include::memdev-option.txt[] + +-d:: +--decoder:: + Specify the decoder to attempt the allocation. The CXL specification + mandates that allocations must be ordered by DPA and decoder instance. + I.e. the lowest DPA allocation on the device is covered by deocder0, and + the last / highest DPA allocation is covered by the last decoder. This + ordering is enforced by the kernel. By default the tool picks the 'next + available' decoder. + +-t:: +--type:: + Select the partition for the allocation. CXL devices implement a + partition that divdes 'ram' and 'pmem' capacity, where 'pmem' capacity + consumes the higher DPA capacity above the partition boundary. The type + defaults to 'pmem'. Note that given CXL DPA allocation constraints, once + any 'pmem' allocation is established then all remaining 'ram' capacity + becomes reserved (skipped). + +-f:: +--force:: + The kernel enforces CXL DPA allocation ordering constraints, and + the tool anticipates those and fails operations that are expected to + fail without sending them to the kernel. For test purposes, continue to + attempt "expected to fail" operations to exercise the driver. + +-s:: +--size:: + Specify the size of the allocation. This option supports the suffixes + "k" or "K" for KiB, "m" or "M" for MiB, "g" or "G" for GiB and "t" or + "T" for TiB. This defaults to "all available capacity of the specified + type". + +-v:: + Turn on verbose debug messages in the library (if libcxl was built with + logging and debug enabled). + +include::../copyright.txt[] + +SEE ALSO +-------- +linkcxl:cxl-free-dpa[1] diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index 90fe33887821..7a38ce4a54e2 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -394,6 +394,7 @@ unsigned long long cxl_decoder_get_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_size(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_dpa_resource(struct cxl_decoder *decoder); unsigned long long cxl_decoder_get_dpa_size(struct cxl_decoder *decoder); +int cxl_decoder_set_dpa_size(struct cxl_decoder *decoder, unsigned long long size); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); int cxl_decoder_get_id(struct cxl_decoder *decoder); int cxl_decoder_get_nr_targets(struct cxl_decoder *decoder); @@ -413,6 +414,7 @@ enum cxl_decoder_mode { CXL_DECODER_MODE_RAM, }; enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); +int cxl_decoder_set_mode(struct cxl_decoder *decoder, enum cxl_decoder_mode mode); bool cxl_decoder_is_pmem_capable(struct cxl_decoder *decoder); bool cxl_decoder_is_volatile_capable(struct cxl_decoder *decoder); diff --git a/Documentation/cxl/meson.build b/Documentation/cxl/meson.build index 974a5a41d169..d019dfc23659 100644 --- a/Documentation/cxl/meson.build +++ b/Documentation/cxl/meson.build @@ -36,6 +36,8 @@ cxl_manpages = [ 'cxl-disable-port.txt', 'cxl-disable-bus.txt', 'cxl-set-partition.txt', + 'cxl-reserve-dpa.txt', + 'cxl-free-dpa.txt', ] foreach man : cxl_manpages diff --git a/cxl/builtin.h b/cxl/builtin.h index a437bc314a30..9e6fc624e86c 100644 --- a/cxl/builtin.h +++ b/cxl/builtin.h @@ -12,6 +12,8 @@ int cmd_init_labels(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_check_labels(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_disable_memdev(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_enable_memdev(int argc, const char **argv, struct cxl_ctx *ctx); +int cmd_reserve_dpa(int argc, const char **argv, struct cxl_ctx *ctx); +int cmd_free_dpa(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_disable_port(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_enable_port(int argc, const char **argv, struct cxl_ctx *ctx); int cmd_set_partition(int argc, const char **argv, struct cxl_ctx *ctx); diff --git a/cxl/cxl.c b/cxl/cxl.c index aa4ce61b7c87..ef4cda9e7c23 100644 --- a/cxl/cxl.c +++ b/cxl/cxl.c @@ -66,6 +66,8 @@ static struct cmd_struct commands[] = { { "write-labels", .c_fn = cmd_write_labels }, { "disable-memdev", .c_fn = cmd_disable_memdev }, { "enable-memdev", .c_fn = cmd_enable_memdev }, + { "reserve-dpa", .c_fn = cmd_reserve_dpa }, + { "free-dpa", .c_fn = cmd_free_dpa }, { "disable-port", .c_fn = cmd_disable_port }, { "enable-port", .c_fn = cmd_enable_port }, { "set-partition", .c_fn = cmd_set_partition }, diff --git a/cxl/filter.c b/cxl/filter.c index 2f88a9d2f398..e5fab19ec47f 100644 --- a/cxl/filter.c +++ b/cxl/filter.c @@ -380,8 +380,8 @@ struct cxl_port *util_cxl_port_filter_by_memdev(struct cxl_port *port, return NULL; } -static struct cxl_decoder *util_cxl_decoder_filter(struct cxl_decoder *decoder, - const char *__ident) +struct cxl_decoder *util_cxl_decoder_filter(struct cxl_decoder *decoder, + const char *__ident) { struct cxl_port *port = cxl_decoder_get_port(decoder); int pid, did; diff --git a/cxl/filter.h b/cxl/filter.h index 955794366d5c..c913dafd85ba 100644 --- a/cxl/filter.h +++ b/cxl/filter.h @@ -50,6 +50,8 @@ struct cxl_target *util_cxl_target_filter_by_memdev(struct cxl_target *target, struct cxl_dport *util_cxl_dport_filter_by_memdev(struct cxl_dport *dport, const char *ident, const char *serial); +struct cxl_decoder *util_cxl_decoder_filter(struct cxl_decoder *decoder, + const char *__ident); int cxl_filter_walk(struct cxl_ctx *ctx, struct cxl_filter_params *param); bool cxl_filter_has(const char *needle, const char *__filter); #endif /* _CXL_UTIL_FILTER_H_ */ diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index a4709175678d..be6bc2c65483 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -1120,6 +1120,20 @@ CXL_EXPORT struct cxl_decoder *cxl_decoder_get_next(struct cxl_decoder *decoder) return list_next(&port->decoders, decoder, list); } +CXL_EXPORT struct cxl_decoder *cxl_decoder_get_last(struct cxl_port *port) +{ + cxl_decoders_init(port); + + return list_tail(&port->decoders, struct cxl_decoder, list); +} + +CXL_EXPORT struct cxl_decoder *cxl_decoder_get_prev(struct cxl_decoder *decoder) +{ + struct cxl_port *port = decoder->port; + + return list_prev(&port->decoders, decoder, list); +} + CXL_EXPORT struct cxl_ctx *cxl_decoder_get_ctx(struct cxl_decoder *decoder) { return decoder->ctx; @@ -1175,6 +1189,78 @@ cxl_decoder_get_dpa_size(struct cxl_decoder *decoder) return decoder->dpa_size; } +CXL_EXPORT int cxl_decoder_set_dpa_size(struct cxl_decoder *decoder, + unsigned long long size) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + char *path = decoder->dev_buf; + int len = decoder->buf_len, rc; + char buf[SYSFS_ATTR_SIZE]; + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return -EINVAL; + } + + if (snprintf(path, len, "%s/dpa_size", decoder->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", + cxl_decoder_get_devname(decoder)); + return -ENOMEM; + } + + sprintf(buf, "%#llx\n", size); + rc = sysfs_write_attr(ctx, path, buf); + if (rc < 0) + return rc; + + decoder->dpa_size = size; + return 0; +} + +CXL_EXPORT int cxl_decoder_set_mode(struct cxl_decoder *decoder, + enum cxl_decoder_mode mode) +{ + struct cxl_port *port = cxl_decoder_get_port(decoder); + struct cxl_ctx *ctx = cxl_decoder_get_ctx(decoder); + char *path = decoder->dev_buf; + int len = decoder->buf_len, rc; + char buf[SYSFS_ATTR_SIZE]; + + if (!cxl_port_is_endpoint(port)) { + err(ctx, "%s: not an endpoint decoder\n", + cxl_decoder_get_devname(decoder)); + return -EINVAL; + } + + switch (mode) { + case CXL_DECODER_MODE_PMEM: + sprintf(buf, "pmem"); + break; + case CXL_DECODER_MODE_RAM: + sprintf(buf, "ram"); + break; + default: + err(ctx, "%s: unsupported mode: %d\n", + cxl_decoder_get_devname(decoder), mode); + return -EINVAL; + } + + if (snprintf(path, len, "%s/mode", decoder->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", + cxl_decoder_get_devname(decoder)); + return -ENOMEM; + } + + rc = sysfs_write_attr(ctx, path, buf); + if (rc < 0) + return rc; + + decoder->mode = mode; + return 0; +} + CXL_EXPORT enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index 88c5a7edd33e..7712de0c5010 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -173,4 +173,8 @@ global: cxl_decoder_get_dpa_resource; cxl_decoder_get_dpa_size; cxl_decoder_get_mode; + cxl_decoder_get_last; + cxl_decoder_get_prev; + cxl_decoder_set_dpa_size; + cxl_decoder_set_mode; } LIBCXL_2; diff --git a/cxl/libcxl.h b/cxl/libcxl.h index 1436dc4601cc..33a216ee3a4c 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -139,6 +139,7 @@ enum cxl_decoder_mode { CXL_DECODER_MODE_PMEM, CXL_DECODER_MODE_RAM, }; + static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) { static const char *names[] = { @@ -154,6 +155,10 @@ static inline const char *cxl_decoder_mode_name(enum cxl_decoder_mode mode) } enum cxl_decoder_mode cxl_decoder_get_mode(struct cxl_decoder *decoder); +int cxl_decoder_set_mode(struct cxl_decoder *decoder, + enum cxl_decoder_mode mode); +int cxl_decoder_set_dpa_size(struct cxl_decoder *decoder, + unsigned long long size); const char *cxl_decoder_get_devname(struct cxl_decoder *decoder); struct cxl_target *cxl_decoder_get_target_by_memdev(struct cxl_decoder *decoder, struct cxl_memdev *memdev); @@ -182,6 +187,10 @@ bool cxl_decoder_is_locked(struct cxl_decoder *decoder); for (decoder = cxl_decoder_get_first(port); decoder != NULL; \ decoder = cxl_decoder_get_next(decoder)) +#define cxl_decoder_foreach_reverse(port, decoder) \ + for (decoder = cxl_decoder_get_last(port); decoder != NULL; \ + decoder = cxl_decoder_get_prev(decoder)) + struct cxl_target; struct cxl_target *cxl_target_get_first(struct cxl_decoder *decoder); struct cxl_target *cxl_target_get_next(struct cxl_target *target); diff --git a/cxl/memdev.c b/cxl/memdev.c index 1cecad2dba4b..e42f554326ec 100644 --- a/cxl/memdev.c +++ b/cxl/memdev.c @@ -33,6 +33,7 @@ static struct parameters { bool align; const char *type; const char *size; + const char *decoder_filter; } param; static struct log_ctx ml; @@ -71,6 +72,19 @@ OPT_STRING('s', "size", ¶m.size, "size", \ OPT_BOOLEAN('a', "align", ¶m.align, \ "auto-align --size per device's requirement") +#define RESERVE_DPA_OPTIONS() \ +OPT_STRING('s', "size", ¶m.size, "size", \ + "size in bytes (Default: all available capacity)") + +#define DPA_OPTIONS() \ +OPT_STRING('d', "decoder", ¶m.decoder_filter, \ + "decoder instance id", \ + "override the automatic decoder selection"), \ +OPT_STRING('t', "type", ¶m.type, "type", \ + "'pmem' or 'ram' (volatile) (Default: 'pmem')"), \ +OPT_BOOLEAN('f', "force", ¶m.force, \ + "Attempt 'expected to fail' operations") + static const struct option read_options[] = { BASE_OPTIONS(), LABEL_OPTIONS(), @@ -108,6 +122,242 @@ static const struct option set_partition_options[] = { OPT_END(), }; +static const struct option reserve_dpa_options[] = { + BASE_OPTIONS(), + RESERVE_DPA_OPTIONS(), + DPA_OPTIONS(), + OPT_END(), +}; + +static const struct option free_dpa_options[] = { + BASE_OPTIONS(), + DPA_OPTIONS(), + OPT_END(), +}; + +enum reserve_dpa_mode { + DPA_ALLOC, + DPA_FREE, +}; + +static int __reserve_dpa(struct cxl_memdev *memdev, + enum reserve_dpa_mode alloc_mode, + struct action_context *actx) +{ + struct cxl_decoder *decoder, *auto_target = NULL, *target = NULL; + struct cxl_endpoint *endpoint = cxl_memdev_get_endpoint(memdev); + const char *devname = cxl_memdev_get_devname(memdev); + unsigned long long avail_dpa, size; + enum cxl_decoder_mode mode; + struct cxl_port *port; + char buf[256]; + int rc; + + if (param.type) { + if (strcmp(param.type, "ram") == 0) + mode = CXL_DECODER_MODE_RAM; + else if (strcmp(param.type, "volatile") == 0) + mode = CXL_DECODER_MODE_RAM; + else if (strcmp(param.type, "ram") == 0) + mode = CXL_DECODER_MODE_RAM; + else if (strcmp(param.type, "pmem") == 0) + mode = CXL_DECODER_MODE_PMEM; + else { + log_err(&ml, "%s: unsupported type: %s\n", devname, + param.type); + return -EINVAL; + } + } else + mode = CXL_DECODER_MODE_RAM; + + if (!endpoint) { + log_err(&ml, "%s: CXL operation disabled\n", devname); + return -ENXIO; + } + + port = cxl_endpoint_get_port(endpoint); + + if (mode == CXL_DECODER_MODE_RAM) + avail_dpa = cxl_memdev_get_ram_size(memdev); + else + avail_dpa = cxl_memdev_get_pmem_size(memdev); + + cxl_decoder_foreach(port, decoder) { + size = cxl_decoder_get_dpa_size(decoder); + if (size == ULLONG_MAX) + continue; + if (cxl_decoder_get_mode(decoder) != mode) + continue; + + if (size > avail_dpa) { + log_err(&ml, "%s: capacity accounting error\n", + devname); + return -ENXIO; + } + avail_dpa -= size; + } + + if (!param.size) + if (alloc_mode == DPA_ALLOC) { + size = avail_dpa; + if (!avail_dpa) { + log_err(&ml, "%s: no available capacity\n", + devname); + return -ENOSPC; + } + } else + size = 0; + else { + size = parse_size64(param.size); + if (size == ULLONG_MAX) { + log_err(&ml, "%s: failed to parse size option '%s'\n", + devname, param.size); + return -EINVAL; + } + if (size > avail_dpa) { + log_err(&ml, "%s: '%s' exceeds available capacity\n", + devname, param.size); + if (!param.force) + return -ENOSPC; + } + } + + /* + * Find next free decoder, assumes cxl_decoder_foreach() is in + * hardware instance-id order + */ + if (alloc_mode == DPA_ALLOC) + cxl_decoder_foreach(port, decoder) { + /* first 0-dpa_size is our target */ + if (cxl_decoder_get_dpa_size(decoder) == 0) { + auto_target = decoder; + break; + } + } + else + cxl_decoder_foreach_reverse(port, decoder) { + /* nothing to free? */ + if (!cxl_decoder_get_dpa_size(decoder)) + continue; + /* + * Active decoders can't be freed, and by definition all + * previous decoders must also be active + */ + if (cxl_decoder_get_size(decoder)) + break; + /* first dpa_size > 0 + disabled decoder is our target */ + if (cxl_decoder_get_dpa_size(decoder) < ULLONG_MAX) { + auto_target = decoder; + break; + } + } + + if (param.decoder_filter) { + unsigned long id; + char *end; + + id = strtoul(param.decoder_filter, &end, 0); + /* allow for standalone ordinal decoder ids */ + if (*end == '\0') + rc = snprintf(buf, sizeof(buf), "decoder%d.%ld", + cxl_port_get_id(port), id); + else + rc = snprintf(buf, sizeof(buf), "%s", + param.decoder_filter); + + if (rc >= (int)sizeof(buf)) { + log_err(&ml, "%s: decoder filter '%s' too long\n", + devname, param.decoder_filter); + return -EINVAL; + } + + if (alloc_mode == DPA_ALLOC) + cxl_decoder_foreach(port, decoder) { + target = util_cxl_decoder_filter(decoder, buf); + if (target) + break; + } + else + cxl_decoder_foreach_reverse(port, decoder) { + target = util_cxl_decoder_filter(decoder, buf); + if (target) + break; + } + + if (!target) { + log_err(&ml, "%s: no match for decoder: '%s'\n", + devname, param.decoder_filter); + return -ENXIO; + } + + if (target != auto_target) { + log_err(&ml, "%s: %s is out of sequence\n", devname, + cxl_decoder_get_devname(target)); + if (!param.force) + return -EINVAL; + } + } + + if (!target) + target = auto_target; + + if (!target) { + log_err(&ml, "%s: no suitable decoder found\n", devname); + return -ENXIO; + } + + if (cxl_decoder_get_mode(target) != mode) { + rc = cxl_decoder_set_dpa_size(target, 0); + if (rc) { + log_err(&ml, + "%s: %s: failed to clear allocation to set mode\n", + devname, cxl_decoder_get_devname(target)); + return rc; + } + rc = cxl_decoder_set_mode(target, mode); + if (rc) { + log_err(&ml, "%s: %s: failed to set %s mode\n", devname, + cxl_decoder_get_devname(target), + mode == CXL_DECODER_MODE_PMEM ? "pmem" : "ram"); + return rc; + } + } + + rc = cxl_decoder_set_dpa_size(target, size); + if (rc) + log_err(&ml, "%s: %s: failed to set dpa allocation\n", devname, + cxl_decoder_get_devname(target)); + else { + struct json_object *jdev, *jdecoder; + unsigned long flags = 0; + + if (actx->f_out == stdout && isatty(1)) + flags |= UTIL_JSON_HUMAN; + jdev = util_cxl_memdev_to_json(memdev, flags); + jdecoder = util_cxl_decoder_to_json(target, flags); + if (!jdev || !jdecoder) { + json_object_put(jdev); + json_object_put(jdecoder); + } else { + json_object_object_add(jdev, "decoder", jdecoder); + json_object_array_add(actx->jdevs, jdev); + } + } + return rc; +} + +static int action_reserve_dpa(struct cxl_memdev *memdev, + struct action_context *actx) +{ + return __reserve_dpa(memdev, DPA_ALLOC, actx); +} + +static int action_free_dpa(struct cxl_memdev *memdev, + struct action_context *actx) +{ + return __reserve_dpa(memdev, DPA_FREE, actx); +} + static int action_disable(struct cxl_memdev *memdev, struct action_context *actx) { if (!cxl_memdev_is_enabled(memdev)) @@ -452,7 +702,8 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, err++; } - if (action == action_setpartition) + if (action == action_setpartition || action == action_reserve_dpa || + action == action_free_dpa) actx.jdevs = json_object_new_array(); if (err == argc) { @@ -495,6 +746,8 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, count = 0; for (i = 0; i < argc; i++) { + bool found = false; + cxl_memdev_foreach(ctx, memdev) { const char *memdev_filter = NULL; const char *serial_filter = NULL; @@ -507,6 +760,7 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, if (!util_cxl_memdev_filter(memdev, memdev_filter, serial_filter)) continue; + found = true; if (action == action_write) { single = memdev; @@ -519,6 +773,8 @@ static int memdev_action(int argc, const char **argv, struct cxl_ctx *ctx, else if (rc && !err) err = rc; } + if (!found) + log_info(&ml, "no memdev matches %s\n", argv[i]); } rc = err; @@ -622,3 +878,25 @@ int cmd_set_partition(int argc, const char **argv, struct cxl_ctx *ctx) return count >= 0 ? 0 : EXIT_FAILURE; } + +int cmd_reserve_dpa(int argc, const char **argv, struct cxl_ctx *ctx) +{ + int count = memdev_action( + argc, argv, ctx, action_reserve_dpa, reserve_dpa_options, + "cxl reserve-dpa [..] []"); + log_info(&ml, "reservation completed on %d mem device%s\n", + count >= 0 ? count : 0, count > 1 ? "s" : ""); + + return count >= 0 ? 0 : EXIT_FAILURE; +} + +int cmd_free_dpa(int argc, const char **argv, struct cxl_ctx *ctx) +{ + int count = memdev_action( + argc, argv, ctx, action_free_dpa, free_dpa_options, + "cxl free-dpa [..] []"); + log_info(&ml, "reservation release completed on %d mem device%s\n", + count >= 0 ? count : 0, count > 1 ? "s" : ""); + + return count >= 0 ? 0 : EXIT_FAILURE; +} From patchwork Thu Jul 14 17:02:49 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918279 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 42D316002 for ; Thu, 14 Jul 2022 17:02:50 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818170; x=1689354170; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ScYbThIZfT4n55eM8DUbm+cfiTaDRHUaL5wTJC8Cy8k=; b=MKa4J3sSEOUMVbejXDqZhWSAF+qm2S9apugEX6DF+c54kLSCijlpuGhQ 56GBIk/y/dQ0JZG4stncpe+xUla1BlnVXuFLRUtxLmhDU9OP1Va5L12mz t4mt/laPmxT0mH99+LjkOTxr2JqP2v51ic5uxC+y8K4A7m4tDL2+4IlHS P+LwfH2Gzy1Tq5pRAAImLRw7YQCSX5hgdHi4ki/IqBUYZVrI+cCFcdqjF x8JHA+rkbvqI2d+jwv7XYADg3Im1hKDp84dA4VMv3gOzOB78ln6oCF52G UMIBRZQh+jiSI0QJSSuizwssC2+DizwQ4/hHiHgcp99zaPXbOl6XiVLZg g==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="268602813" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="268602813" Received: from orsmga002.jf.intel.com ([10.7.209.21]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:50 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="596163896" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga002-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:49 -0700 Subject: [ndctl PATCH v2 11/12] cxl/test: Update CXL memory parameters From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:49 -0700 Message-ID: <165781816971.1555691.18362747345754213762.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 In support of testing CXL region configurations cxl_test changed the size of its root decoders and endpoints. Use the size of the first root decoder to determine if this is an updated kernel. Signed-off-by: Dan Williams --- test/cxl-topology.sh | 32 +++++++++++++++++++++----------- 1 file changed, 21 insertions(+), 11 deletions(-) diff --git a/test/cxl-topology.sh b/test/cxl-topology.sh index ff11614f4f14..2583005fef26 100644 --- a/test/cxl-topology.sh +++ b/test/cxl-topology.sh @@ -64,14 +64,9 @@ switch[2]=$(jq -r ".[] | .[\"ports:${bridge[1]}\"] | $port_sort | .[0].host" <<< switch[3]=$(jq -r ".[] | .[\"ports:${bridge[1]}\"] | $port_sort | .[1].host" <<< $json) -# check that all 8 cxl_test memdevs are enabled by default and have a -# pmem size of 256M -json=$($CXL list -b cxl_test -M) -count=$(jq "map(select(.pmem_size == $((256 << 20)))) | length" <<< $json) -((count == 8)) || err "$LINENO" - - # validate the expected properties of the 4 root decoders +# use the size of the first decoder to determine the cxl_test version / +# properties json=$($CXL list -b cxl_test -D -d root) port_id=${root:4} port_id_len=${#port_id} @@ -80,26 +75,41 @@ count=$(jq "[ $decoder_sort | .[0] | select(.volatile_capable == true) | select(.size == $((256 << 20))) | select(.nr_targets == 1) ] | length" <<< $json) -((count == 1)) || err "$LINENO" + +if [ $count -eq 1 ]; then + decoder_base_size=$((256 << 20)) + pmem_size=$((256 << 20)) +else + decoder_base_size=$((1 << 30)) + pmem_size=$((1 << 30)) +fi count=$(jq "[ $decoder_sort | .[1] | select(.volatile_capable == true) | - select(.size == $((512 << 20))) | + select(.size == $((decoder_base_size * 2))) | select(.nr_targets == 2) ] | length" <<< $json) ((count == 1)) || err "$LINENO" count=$(jq "[ $decoder_sort | .[2] | select(.pmem_capable == true) | - select(.size == $((256 << 20))) | + select(.size == $decoder_base_size) | select(.nr_targets == 1) ] | length" <<< $json) ((count == 1)) || err "$LINENO" count=$(jq "[ $decoder_sort | .[3] | select(.pmem_capable == true) | - select(.size == $((512 << 20))) | + select(.size == $((decoder_base_size * 2))) | select(.nr_targets == 2) ] | length" <<< $json) ((count == 1)) || err "$LINENO" + +# check that all 8 cxl_test memdevs are enabled by default and have a +# pmem size of 256M, or 1G +json=$($CXL list -b cxl_test -M) +count=$(jq "map(select(.pmem_size == $pmem_size)) | length" <<< $json) +((count == 8)) || err "$LINENO" + + # check that switch ports disappear after all of their memdevs have been # disabled, and return when the memdevs are enabled. for s in ${switch[@]} From patchwork Thu Jul 14 17:02:55 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Dan Williams X-Patchwork-Id: 12918283 Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C55BD6002 for ; Thu, 14 Jul 2022 17:03:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1657818192; x=1689354192; h=subject:from:to:cc:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=SXHtWpv2HNpUil832VHGH8Vd9M5eM5+D7bANZc7UQtE=; b=DmBMbNb/1qbFt//UHtH1HVzoRF7eCABjjCF358LS2rnb4sjuRnkNjTgJ 9gfwFRPDwT+j/U93NfHyrvq5tCaNGdK784bSgX8/5rckeOU7zhPZGjVtx 5hs38mmHH03dVIe/0NrUOswGF12YW7Bt+WE7T/UNKe/b/nWO0oJSwNRHT Hcpje5xQCF6R4wFTTY5Z5By/KM4IlzjtfaLwRZNc3Lbp8oN0eQx4va+xV 2/pqJ7j5EoqCqfxj54NimEW3yG2e7JWAgwCc9SDtyOhVOcI89tIOCMTia fIbwkMmVbbaT8qUGRM1qnPQMkXmtYHkntvGom2fACK+vQ0jkxv8wjt+95 Q==; X-IronPort-AV: E=McAfee;i="6400,9594,10408"; a="371889778" X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="371889778" Received: from orsmga005.jf.intel.com ([10.7.209.41]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:55 -0700 X-IronPort-AV: E=Sophos;i="5.92,271,1650956400"; d="scan'208";a="772694252" Received: from jlcone-mobl1.amr.corp.intel.com (HELO dwillia2-xfh.jf.intel.com) ([10.209.2.90]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 14 Jul 2022 10:02:55 -0700 Subject: [ndctl PATCH v2 12/12] cxl/test: Checkout region setup/teardown From: Dan Williams To: vishal.l.verma@intel.com Cc: alison.schofield@intel.com, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org Date: Thu, 14 Jul 2022 10:02:55 -0700 Message-ID: <165781817516.1555691.3557156570639615515.stgit@dwillia2-xfh.jf.intel.com> In-Reply-To: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> References: <165781810717.1555691.1411727384567016588.stgit@dwillia2-xfh.jf.intel.com> User-Agent: StGit/0.18-3-g996c Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Exercise the fundamental region provisioning sysfs mechanisms of discovering available DPA capacity, allocating DPA to a region, and programming HDM decoders. Signed-off-by: Dan Williams --- test/cxl-region-sysfs.sh | 122 ++++++++++++++++++++++++++++++++++++++++++++++ test/meson.build | 2 + 2 files changed, 124 insertions(+) create mode 100644 test/cxl-region-sysfs.sh diff --git a/test/cxl-region-sysfs.sh b/test/cxl-region-sysfs.sh new file mode 100644 index 000000000000..2582edb3f306 --- /dev/null +++ b/test/cxl-region-sysfs.sh @@ -0,0 +1,122 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (C) 2022 Intel Corporation. All rights reserved. + +. $(dirname $0)/common + +rc=1 + +set -ex + +trap 'err $LINENO' ERR + +check_prereq "jq" + +modprobe -r cxl_test +modprobe cxl_test +udevadm settle + +# THEORY OF OPERATION: Create a x8 interleave across the pmem capacity +# of the 8 endpoints defined by cxl_test, commit the decoders (which +# just stubs out the actual hardware programming aspect, but updates the +# driver state), and then tear it all down again. As with other cxl_test +# tests if the CXL topology in tools/testing/cxl/test/cxl.c ever changes +# then the paired update must be made to this test. + +# find the root decoder that spans both test host-bridges and support pmem +decoder=$($CXL list -b cxl_test -D -d root | jq -r ".[] | + select(.pmem_capable == true) | + select(.nr_targets == 2) | + .decoder") + +# find the memdevs mapped by that decoder +readarray -t mem < <($CXL list -M -d $decoder | jq -r ".[].memdev") + +# ask cxl reserve-dpa to allocate pmem capacity from each of those memdevs +readarray -t endpoint < <($CXL reserve-dpa -t pmem ${mem[*]} -s $((256<<20)) | + jq -r ".[] | .decoder.decoder") + +# instantiate an empty region +region=$(cat /sys/bus/cxl/devices/$decoder/create_pmem_region) +echo $region > /sys/bus/cxl/devices/$decoder/create_pmem_region +uuidgen > /sys/bus/cxl/devices/$region/uuid + +# setup interleave geometry +nr_targets=${#endpoint[@]} +echo $nr_targets > /sys/bus/cxl/devices/$region/interleave_ways +g=$(cat /sys/bus/cxl/devices/$decoder/interleave_granularity) +echo $g > /sys/bus/cxl/devices/$region/interleave_granularity +echo $((nr_targets * (256<<20))) > /sys/bus/cxl/devices/$region/size + +# grab the list of memdevs grouped by host-bridge interleave position +port_dev0=$($CXL list -T -d $decoder | jq -r ".[] | + .targets | .[] | select(.position == 0) | .target") +port_dev1=$($CXL list -T -d $decoder | jq -r ".[] | + .targets | .[] | select(.position == 1) | .target") +readarray -t mem_sort0 < <($CXL list -M -p $port_dev0 | jq -r ".[] | .memdev") +readarray -t mem_sort1 < <($CXL list -M -p $port_dev1 | jq -r ".[] | .memdev") + +# TODO: add a cxl list option to list memdevs in valid region provisioning +# order, hardcode for now. +mem_sort=() +mem_sort[0]=${mem_sort0[0]} +mem_sort[1]=${mem_sort1[0]} +mem_sort[2]=${mem_sort0[2]} +mem_sort[3]=${mem_sort1[2]} +mem_sort[4]=${mem_sort0[1]} +mem_sort[5]=${mem_sort1[1]} +mem_sort[6]=${mem_sort0[3]} +mem_sort[7]=${mem_sort1[3]} + +# TODO: use this alternative memdev ordering to validate a negative test for +# specifying invalid positions of memdevs +#mem_sort[2]=${mem_sort0[0]} +#mem_sort[1]=${mem_sort1[0]} +#mem_sort[0]=${mem_sort0[2]} +#mem_sort[3]=${mem_sort1[2]} +#mem_sort[4]=${mem_sort0[1]} +#mem_sort[5]=${mem_sort1[1]} +#mem_sort[6]=${mem_sort0[3]} +#mem_sort[7]=${mem_sort1[3]} + +# re-generate the list of endpoint decoders in region position programming order +endpoint=() +for i in ${mem_sort[@]} +do + readarray -O ${#endpoint[@]} -t endpoint < <($CXL list -Di -d endpoint -m $i | jq -r ".[] | + select(.mode == \"pmem\") | .decoder") +done + +# attach all endpoint decoders to the region +pos=0 +for i in ${endpoint[@]} +do + echo $i > /sys/bus/cxl/devices/$region/target$pos + pos=$((pos+1)) +done +echo "$region added ${#endpoint[@]} targets: ${endpoint[@]}" + +# walk up the topology and commit all decoders +echo 1 > /sys/bus/cxl/devices/$region/commit + +# walk down the topology and de-commit all decoders +echo 0 > /sys/bus/cxl/devices/$region/commit + +# remove endpoints from the region +pos=0 +for i in ${endpoint[@]} +do + echo "" > /sys/bus/cxl/devices/$region/target$pos + pos=$((pos+1)) +done + +# release DPA capacity +readarray -t endpoint < <($CXL free-dpa -t pmem ${mem[*]} | + jq -r ".[] | .decoder.decoder") +echo "$region released ${#endpoint[@]} targets: ${endpoint[@]}" + +# validate no WARN or lockdep report during the run +log=$(journalctl -r -k --since "-$((SECONDS+1))s") +grep -q "Call Trace" <<< $log && err "$LINENO" + +modprobe -r cxl_test diff --git a/test/meson.build b/test/meson.build index 210dcb0b5ff1..3203d9cea243 100644 --- a/test/meson.build +++ b/test/meson.build @@ -151,6 +151,7 @@ max_extent = find_program('max_available_extent_ns.sh') pfn_meta_errors = find_program('pfn-meta-errors.sh') track_uuid = find_program('track-uuid.sh') cxl_topo = find_program('cxl-topology.sh') +cxl_sysfs = find_program('cxl-region-sysfs.sh') tests = [ [ 'libndctl', libndctl, 'ndctl' ], @@ -176,6 +177,7 @@ tests = [ [ 'pfn-meta-errors.sh', pfn_meta_errors, 'ndctl' ], [ 'track-uuid.sh', track_uuid, 'ndctl' ], [ 'cxl-topology.sh', cxl_topo, 'cxl' ], + [ 'cxl-region-sysfs.sh', cxl_sysfs, 'cxl' ], ] if get_option('destructive').enabled()