From patchwork Fri Nov 15 18:46:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ira Weiny X-Patchwork-Id: 13876692 X-Patchwork-Delegate: iweiny@gmail.com Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 999421684AC for ; Fri, 15 Nov 2024 18:46:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.16 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731696407; cv=none; b=IB2E6La4cgzs16inA1y0i6eDm9XirIKvDOVQH7/gYeNhQJfsAexxQTPh4PhAe8GLYZioKt3R7LrlOnrdpjxNCVmZweosho+JYLJv9VmEXGqtY/gFzfF6c+Z62AU71q8DyI7FM95SmDFqvcxpdZA6GZzgQYF7lKQbIcqIBODLhL8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1731696407; c=relaxed/simple; bh=bee6YOxiEEbvEMLIRmtuV/fOI+H/2Oms2YLawlgg7js=; h=From:Date:Subject:MIME-Version:Content-Type:Message-Id:References: In-Reply-To:To:Cc; b=dk6Tzu5Uw7isESjYCHduagTojO01t4p0RRCju0avxoNSt4Z1WMjM+zJE6Cz5ocEfGsZVB9cHG3S6CGyzwY85tUErGXWtIsamZACG5q27GaHmpZQ1A5TOJO6yvQeYTLIe/9G/EpJ3QfchJvCFPDy8Y5P6X4ratofm8zvVJ+1PIeA= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=mG+NEU5u; arc=none smtp.client-ip=198.175.65.16 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mG+NEU5u" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1731696406; x=1763232406; h=from:date:subject:mime-version:content-transfer-encoding: message-id:references:in-reply-to:to:cc; bh=bee6YOxiEEbvEMLIRmtuV/fOI+H/2Oms2YLawlgg7js=; b=mG+NEU5ur9JllL2c0++evqliDvWNt9d2VeuNexJ3Uob0Uwj6nPk/7d9y XifXcHg2k8aTSeyhqOKs/rVWPlICyAN77EwrBPiCoehvXFtezW2S6aG6O UEAEY9VKaitWttjumTbYQYC0OyUTGDOvxkEXMnQWzsoilM+BhLjGTEyOz J1zbH+EZtXfaP41CqC/QzDjpaaqoPSK24AITTEItA7qMJvVeMfHil4PvL fpyjlejeVSiPFoMv5cftBMBbYWWcwR04jK2BzJMCB7hR0hp+YLJKrufvb Pp8PsLOmPKbP1/ubHrQByQI8Ar+gylwfOKQyjY9CDxNLISfBG9gdHDvF3 Q==; X-CSE-ConnectionGUID: a+f6hk8rQzCWB+E3vkrQkw== X-CSE-MsgGUID: 1R3RNtwbQ+ivpQEH5BTMaw== X-IronPort-AV: E=McAfee;i="6700,10204,11257"; a="31848487" X-IronPort-AV: E=Sophos;i="6.12,157,1728975600"; d="scan'208";a="31848487" Received: from orviesa008.jf.intel.com ([10.64.159.148]) by orvoesa108.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Nov 2024 10:46:39 -0800 X-CSE-ConnectionGUID: CVEQM3PJTP6ybiwVqegBkg== X-CSE-MsgGUID: eUJlIw0lQcmyblV6t+pfKg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.12,157,1728975600"; d="scan'208";a="89392921" Received: from ehanks-mobl1.amr.corp.intel.com (HELO localhost) ([10.125.108.112]) by orviesa008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Nov 2024 10:46:38 -0800 From: Ira Weiny Date: Fri, 15 Nov 2024 12:46:25 -0600 Subject: [ndctl PATCH v3 7/9] libcxl: Add extent functionality to DC regions Precedence: bulk X-Mailing-List: nvdimm@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Message-Id: <20241115-dcd-region2-v3-7-585d480ccdab@intel.com> References: <20241115-dcd-region2-v3-0-585d480ccdab@intel.com> In-Reply-To: <20241115-dcd-region2-v3-0-585d480ccdab@intel.com> To: Alison Schofield Cc: Vishal Verma , Jonathan Cameron , Fan Ni , Navneet Singh , Dan Williams , Dave Jiang , linux-cxl@vger.kernel.org, nvdimm@lists.linux.dev, Ira Weiny X-Mailer: b4 0.15-dev-2a633 X-Developer-Signature: v=1; a=ed25519-sha256; t=1731696382; l=9208; i=ira.weiny@intel.com; s=20221211; h=from:subject:message-id; bh=bee6YOxiEEbvEMLIRmtuV/fOI+H/2Oms2YLawlgg7js=; b=BdfY5YPJHPLrIlELQrItexXj0kOodXdTeU4l/qW0B9R2A54Sxz6YZyxQc5GkJvuWEuHqAwEnT m7Mo9QDw8+yDQ/tLXYsuMUj7rjyVBk6B5NYHxOXz/DePLVUdnOy5ETy X-Developer-Key: i=ira.weiny@intel.com; a=ed25519; pk=noldbkG+Wp1qXRrrkfY1QJpDf7QsOEthbOT7vm0PqsE= DCD regions have 0 or more extents. The ability to list those and their properties is useful to end users. Add extent scanning and reporting functionality to libcxl. Signed-off-by: Ira Weiny --- Changes: [djiang: report strerror() on opendir() error] [djiang: Fix up strtoull() error checking] [Alison: Enhance man page] [Alison: Enhance extent processing debug] [Alison: Fix up libcxl export symbols] [Alison/iweiny: Add documentation] --- Documentation/cxl/lib/libcxl.txt | 27 ++++++++ cxl/lib/libcxl.c | 138 +++++++++++++++++++++++++++++++++++++++ cxl/lib/libcxl.sym | 5 ++ cxl/lib/private.h | 11 ++++ cxl/libcxl.h | 11 ++++ 5 files changed, 192 insertions(+) diff --git a/Documentation/cxl/lib/libcxl.txt b/Documentation/cxl/lib/libcxl.txt index abca08fc81e6e84d176facafad6decae2f875880..0b53cf9a3a09a3e8c9059f796823b52d22d1077f 100644 --- a/Documentation/cxl/lib/libcxl.txt +++ b/Documentation/cxl/lib/libcxl.txt @@ -632,6 +632,33 @@ Regions now have a mode distinct from decoders. cxl_region_get_mode() is deprecated in favor of cxl_region_get_region_mode(). Dynamic capacity regions require the use of cxl_region_get_region_mode(). +EXTENTS +------- + +=== EXTENT: Enumeration +---- +struct cxl_region_extent; +struct cxl_region_extent *cxl_extent_get_first(struct cxl_region *region); +struct cxl_region_extent *cxl_extent_get_next(struct cxl_region_extent *extent); +#define cxl_extent_foreach(region, extent) \ + for (extent = cxl_extent_get_first(region); \ + extent != NULL; \ + extent = cxl_extent_get_next(extent)) + +---- + +=== EXTENT: Attributes +---- +unsigned long long cxl_extent_get_offset(struct cxl_region_extent *extent); +unsigned long long cxl_extent_get_length(struct cxl_region_extent *extent); +void cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag); +---- + +Extents represent available memory within a dynamic capacity region. Extent +objects are available for informational purposes to aid in allocation of +memory. + + include::../../copyright.txt[] SEE ALSO diff --git a/cxl/lib/libcxl.c b/cxl/lib/libcxl.c index df250db9dbacb2f0f34e8a592ce194159584fe4f..a029b14dcccf038b02b28d05df6f0dc71557df5e 100644 --- a/cxl/lib/libcxl.c +++ b/cxl/lib/libcxl.c @@ -568,6 +568,7 @@ static void *add_cxl_region(void *parent, int id, const char *cxlregion_base) region->ctx = ctx; region->decoder = decoder; list_head_init(®ion->mappings); + list_head_init(®ion->extents); region->dev_path = strdup(cxlregion_base); if (!region->dev_path) @@ -1178,6 +1179,143 @@ cxl_mapping_get_next(struct cxl_memdev_mapping *mapping) return list_next(®ion->mappings, mapping, list); } +static void cxl_extents_init(struct cxl_region *region) +{ + const char *devname = cxl_region_get_devname(region); + struct cxl_ctx *ctx = cxl_region_get_ctx(region); + char *extent_path, *dax_region_path; + struct dirent *de; + DIR *dir = NULL; + + if (region->extents_init) + return; + region->extents_init = 1; + + dax_region_path = calloc(1, strlen(region->dev_path) + 64); + if (!dax_region_path) { + err(ctx, "%s: allocation failure\n", devname); + return; + } + + extent_path = calloc(1, strlen(region->dev_path) + 100); + if (!extent_path) { + err(ctx, "%s: allocation failure\n", devname); + free(dax_region_path); + return; + } + + sprintf(dax_region_path, "%s/dax_region%d", + region->dev_path, region->id); + dir = opendir(dax_region_path); + if (!dir) { + err(ctx, "no extents found (%s): %s\n", + strerror(errno), dax_region_path); + free(extent_path); + free(dax_region_path); + return; + } + + while ((de = readdir(dir)) != NULL) { + struct cxl_region_extent *extent; + char buf[SYSFS_ATTR_SIZE]; + u64 offset, length; + int id, region_id; + + if (sscanf(de->d_name, "extent%d.%d", ®ion_id, &id) != 2) + continue; + + sprintf(extent_path, "%s/extent%d.%d/offset", + dax_region_path, region_id, id); + if (sysfs_read_attr(ctx, extent_path, buf) < 0) { + err(ctx, "%s: failed to read extent%d.%d/offset\n", + devname, region_id, id); + continue; + } + + offset = strtoull(buf, NULL, 0); + if (offset == ULLONG_MAX) { + err(ctx, "%s extent%d.%d: failed to read offset\n", + devname, region_id, id); + continue; + } + + sprintf(extent_path, "%s/extent%d.%d/length", + dax_region_path, region_id, id); + if (sysfs_read_attr(ctx, extent_path, buf) < 0) { + err(ctx, "%s: failed to read extent%d.%d/length\n", + devname, region_id, id); + continue; + } + + length = strtoull(buf, NULL, 0); + if (length == ULLONG_MAX) { + err(ctx, "%s extent%d.%d: failed to read length\n", + devname, region_id, id); + continue; + } + + sprintf(extent_path, "%s/extent%d.%d/tag", + dax_region_path, region_id, id); + buf[0] = '\0'; + if (sysfs_read_attr(ctx, extent_path, buf) != 0) + dbg(ctx, "%s extent%d.%d: failed to read tag\n", + devname, region_id, id); + + extent = calloc(1, sizeof(*extent)); + if (!extent) { + err(ctx, "%s extent%d.%d: allocation failure\n", + devname, region_id, id); + continue; + } + if (strlen(buf) && uuid_parse(buf, extent->tag) < 0) + err(ctx, "%s:%s\n", extent_path, buf); + extent->region = region; + extent->offset = offset; + extent->length = length; + + list_node_init(&extent->list); + list_add(®ion->extents, &extent->list); + dbg(ctx, "%s added extent%d.%d\n", devname, region_id, id); + } + free(dax_region_path); + free(extent_path); + closedir(dir); +} + +CXL_EXPORT struct cxl_region_extent * +cxl_extent_get_first(struct cxl_region *region) +{ + cxl_extents_init(region); + + return list_top(®ion->extents, struct cxl_region_extent, list); +} + +CXL_EXPORT struct cxl_region_extent * +cxl_extent_get_next(struct cxl_region_extent *extent) +{ + struct cxl_region *region = extent->region; + + return list_next(®ion->extents, extent, list); +} + +CXL_EXPORT unsigned long long +cxl_extent_get_offset(struct cxl_region_extent *extent) +{ + return extent->offset; +} + +CXL_EXPORT unsigned long long +cxl_extent_get_length(struct cxl_region_extent *extent) +{ + return extent->length; +} + +CXL_EXPORT void +cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag) +{ + memcpy(tag, extent->tag, sizeof(uuid_t)); +} + CXL_EXPORT struct cxl_decoder * cxl_mapping_get_decoder(struct cxl_memdev_mapping *mapping) { diff --git a/cxl/lib/libcxl.sym b/cxl/lib/libcxl.sym index fdb227789985443a13c72751bbd42ab383db5f97..d8e8dbc7e091792fe48faa4657ab7cf1d795efdd 100644 --- a/cxl/lib/libcxl.sym +++ b/cxl/lib/libcxl.sym @@ -295,4 +295,9 @@ global: cxl_memdev_get_dc_size; cxl_decoder_is_dc_capable; cxl_decoder_create_dc_region; + cxl_extent_get_first; + cxl_extent_get_next; + cxl_extent_get_offset; + cxl_extent_get_length; + cxl_extent_get_tag; } LIBECXL_8; diff --git a/cxl/lib/private.h b/cxl/lib/private.h index 3efa230bfb632e6c6048aadd18f799b07d4bdfd3..62278ec79963c198dcca490015e4c3f7621109b2 100644 --- a/cxl/lib/private.h +++ b/cxl/lib/private.h @@ -164,6 +164,7 @@ struct cxl_region { struct cxl_decoder *decoder; struct list_node list; int mappings_init; + int extents_init; struct cxl_ctx *ctx; void *dev_buf; size_t buf_len; @@ -179,6 +180,7 @@ struct cxl_region { struct daxctl_region *dax_region; struct kmod_module *module; struct list_head mappings; + struct list_head extents; }; struct cxl_memdev_mapping { @@ -188,6 +190,15 @@ struct cxl_memdev_mapping { struct list_node list; }; +#define CXL_REGION_EXTENT_TAG 0x10 +struct cxl_region_extent { + struct cxl_region *region; + u64 offset; + u64 length; + uuid_t tag; + struct list_node list; +}; + enum cxl_cmd_query_status { CXL_CMD_QUERY_NOT_RUN = 0, CXL_CMD_QUERY_OK, diff --git a/cxl/libcxl.h b/cxl/libcxl.h index d7f8a37816f236acd71fc834eae70a7a17a2721a..1d294ac0278d798214acb2f62e98aaaccaf60ea5 100644 --- a/cxl/libcxl.h +++ b/cxl/libcxl.h @@ -445,6 +445,17 @@ unsigned int cxl_mapping_get_position(struct cxl_memdev_mapping *mapping); mapping != NULL; \ mapping = cxl_mapping_get_next(mapping)) +struct cxl_region_extent; +struct cxl_region_extent *cxl_extent_get_first(struct cxl_region *region); +struct cxl_region_extent *cxl_extent_get_next(struct cxl_region_extent *extent); +#define cxl_extent_foreach(region, extent) \ + for (extent = cxl_extent_get_first(region); \ + extent != NULL; \ + extent = cxl_extent_get_next(extent)) +unsigned long long cxl_extent_get_offset(struct cxl_region_extent *extent); +unsigned long long cxl_extent_get_length(struct cxl_region_extent *extent); +void cxl_extent_get_tag(struct cxl_region_extent *extent, uuid_t tag); + struct cxl_cmd; const char *cxl_cmd_get_devname(struct cxl_cmd *cmd); struct cxl_cmd *cxl_cmd_new_raw(struct cxl_memdev *memdev, int opcode);