From patchwork Fri Jun 28 19:10:58 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023339 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8892B138B for ; Fri, 28 Jun 2019 19:11:20 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7584328834 for ; Fri, 28 Jun 2019 19:11:20 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 6689A2883B; Fri, 28 Jun 2019 19:11:20 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1E09928834 for ; Fri, 28 Jun 2019 19:11:20 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B6CEA2194EB78; Fri, 28 Jun 2019 12:11:19 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id C8A6D21296715 for ; Fri, 28 Jun 2019 12:11:17 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:16 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564721" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:16 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 01/13] libdaxctl: add interfaces to get ctx and check device state Date: Fri, 28 Jun 2019 13:10:58 -0600 Message-Id: <20190628191110.21428-2-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP In preparation for libdaxctl and daxctl to grow operational modes for DAX devices, add the following supporting APIs: daxctl_dev_get_ctx daxctl_dev_is_enabled Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl.c | 30 ++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 6 ++++++ daxctl/libdaxctl.h | 2 ++ 3 files changed, 38 insertions(+) diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index c2e3a52..70f896b 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -559,6 +559,36 @@ static void dax_regions_init(struct daxctl_ctx *ctx) } } +static int is_enabled(const char *drvpath) +{ + struct stat st; + + if (lstat(drvpath, &st) < 0 || !S_ISLNK(st.st_mode)) + return 0; + else + return 1; +} + +DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) +{ + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + char *path = dev->dev_buf; + int len = dev->buf_len; + + if (snprintf(path, len, "%s/driver", dev->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", + daxctl_dev_get_devname(dev)); + return 0; + } + + return is_enabled(path); +} + +DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) +{ + return dev->region->ctx; +} + DAXCTL_EXPORT struct daxctl_dev *daxctl_dev_get_first(struct daxctl_region *region) { dax_devices_init(region); diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 84d3a69..c4af9a7 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -50,3 +50,9 @@ LIBDAXCTL_5 { global: daxctl_region_get_path; } LIBDAXCTL_4; + +LIBDAXCTL_6 { +global: + daxctl_dev_get_ctx; + daxctl_dev_is_enabled; +} LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index 1d13ea2..e20ccb4 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -67,6 +67,8 @@ const char *daxctl_dev_get_devname(struct daxctl_dev *dev); int daxctl_dev_get_major(struct daxctl_dev *dev); int daxctl_dev_get_minor(struct daxctl_dev *dev); unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev); +struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev); +int daxctl_dev_is_enabled(struct daxctl_dev *dev); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ From patchwork Fri Jun 28 19:10:59 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023343 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 0CE71138D for ; Fri, 28 Jun 2019 19:11:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id EFF7028833 for ; Fri, 28 Jun 2019 19:11:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id E43662883B; Fri, 28 Jun 2019 19:11:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 32E6428833 for ; Fri, 28 Jun 2019 19:11:21 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 039282194D387; Fri, 28 Jun 2019 12:11:20 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id F38FA2128A64D for ; Fri, 28 Jun 2019 12:11:17 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564725" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:16 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 02/13] libdaxctl: add interfaces to enable/disable devices Date: Fri, 28 Jun 2019 13:10:59 -0600 Message-Id: <20190628191110.21428-3-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add new libdaxctl interfaces to disable a device_dax based device, and to enable it into the given mode. The modes available are 'devdax', and 'system-ram', where devdax is the normal device DAX mode used via a character device, and 'system-ram' uses the kernel's 'kmem' facility to hotplug the device making it usable as normal memory. This adds the following new interfaces: daxctl_dev_disable; daxctl_dev_enable_devdax; daxctl_dev_enable_ram; Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/Makefile.am | 3 +- daxctl/lib/libdaxctl-private.h | 15 +++ daxctl/lib/libdaxctl.c | 234 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 3 + daxctl/libdaxctl.h | 9 ++ 5 files changed, 263 insertions(+), 1 deletion(-) diff --git a/daxctl/lib/Makefile.am b/daxctl/lib/Makefile.am index d3d4852..9f0e444 100644 --- a/daxctl/lib/Makefile.am +++ b/daxctl/lib/Makefile.am @@ -16,7 +16,8 @@ libdaxctl_la_SOURCES =\ libdaxctl.c libdaxctl_la_LIBADD =\ - $(UUID_LIBS) + $(UUID_LIBS) \ + $(KMOD_LIBS) daxctl_modprobe_data_DATA = daxctl.conf diff --git a/daxctl/lib/libdaxctl-private.h b/daxctl/lib/libdaxctl-private.h index 4a462e7..e64d0a7 100644 --- a/daxctl/lib/libdaxctl-private.h +++ b/daxctl/lib/libdaxctl-private.h @@ -13,6 +13,8 @@ #ifndef _LIBDAXCTL_PRIVATE_H_ #define _LIBDAXCTL_PRIVATE_H_ +#include + #define DAXCTL_EXPORT __attribute__ ((visibility("default"))) enum dax_subsystem { @@ -26,6 +28,11 @@ static const char *dax_subsystems[] = { [DAX_BUS] = "/sys/bus/dax/devices", }; +static const char *dax_modules[] = { + [DAXCTL_DEV_MODE_DEVDAX] = "device_dax", + [DAXCTL_DEV_MODE_RAM] = "kmem", +}; + /** * struct daxctl_region - container for dax_devices */ @@ -53,6 +60,14 @@ struct daxctl_dev { char *dev_path; struct list_node list; unsigned long long size; + struct kmod_module *module; + struct kmod_list *kmod_list; struct daxctl_region *region; }; + +static inline int check_kmod(struct kmod_ctx *kmod_ctx) +{ + return kmod_ctx ? 0 : -ENXIO; +} + #endif /* _LIBDAXCTL_PRIVATE_H_ */ diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 70f896b..05f013b 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -46,6 +46,7 @@ struct daxctl_ctx { void *userdata; int regions_init; struct list_head regions; + struct kmod_ctx *kmod_ctx; }; /** @@ -84,20 +85,32 @@ DAXCTL_EXPORT void daxctl_set_userdata(struct daxctl_ctx *ctx, void *userdata) */ DAXCTL_EXPORT int daxctl_new(struct daxctl_ctx **ctx) { + struct kmod_ctx *kmod_ctx; struct daxctl_ctx *c; + int rc = 0; c = calloc(1, sizeof(struct daxctl_ctx)); if (!c) return -ENOMEM; + kmod_ctx = kmod_new(NULL, NULL); + if (check_kmod(kmod_ctx) != 0) { + rc = -ENXIO; + goto out; + } + c->refcount = 1; log_init(&c->ctx, "libdaxctl", "DAXCTL_LOG"); info(c, "ctx %p created\n", c); dbg(c, "log_priority=%d\n", c->ctx.log_priority); *ctx = c; list_head_init(&c->regions); + c->kmod_ctx = kmod_ctx; return 0; +out: + free(c); + return rc; } /** @@ -132,6 +145,7 @@ DAXCTL_EXPORT void daxctl_unref(struct daxctl_ctx *ctx) list_for_each_safe(&ctx->regions, region, _r, list) free_region(region, &ctx->regions); + kmod_unref(ctx->kmod_ctx); info(ctx, "context %p released\n", ctx); free(ctx); } @@ -189,6 +203,7 @@ static void free_dev(struct daxctl_dev *dev, struct list_head *head) { if (head) list_del_from(head, &dev->list); + kmod_module_unref_list(dev->kmod_list); free(dev->dev_buf); free(dev->dev_path); free(dev); @@ -306,6 +321,27 @@ DAXCTL_EXPORT struct daxctl_region *daxctl_new_region(struct daxctl_ctx *ctx, return region; } +static struct kmod_list *to_module_list(struct daxctl_ctx *ctx, + const char *alias) +{ + struct kmod_list *list = NULL; + int rc; + + if (!ctx->kmod_ctx || !alias) + return NULL; + if (alias[0] == 0) + return NULL; + + rc = kmod_module_new_from_lookup(ctx->kmod_ctx, alias, &list); + if (rc < 0 || !list) { + dbg(ctx, "failed to find modules for alias: %s %d list: %s\n", + alias, rc, list ? "populated" : "empty"); + return NULL; + } + + return list; +} + static void *add_dax_dev(void *parent, int id, const char *daxdev_base) { const char *devname = devpath_to_devname(daxdev_base); @@ -315,6 +351,7 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) struct daxctl_dev *dev, *dev_dup; char buf[SYSFS_ATTR_SIZE]; struct stat st; + int rc; if (!path) return NULL; @@ -346,6 +383,14 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) goto err_read; dev->buf_len = strlen(daxdev_base) + 50; + sprintf(path, "%s/modalias", daxdev_base); + rc = sysfs_read_attr(ctx, path, buf); + /* older kernels may be lack the modalias attribute */ + if (rc < 0 && rc != -ENOENT) + goto err_read; + if (rc == 0) + dev->kmod_list = to_module_list(ctx, buf); + daxctl_dev_foreach(region, dev_dup) if (dev_dup->id == dev->id) { free_dev(dev, NULL); @@ -569,6 +614,92 @@ static int is_enabled(const char *drvpath) return 1; } +static int daxctl_bind(struct daxctl_ctx *ctx, const char *devname, + const char *mod_name) +{ + DIR *dir; + int rc = 0; + char path[200]; + struct dirent *de; + const int len = sizeof(path); + + if (!devname) { + err(ctx, "missing devname\n"); + return -EINVAL; + } + + if (snprintf(path, len, "/sys/bus/dax/drivers") >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + dir = opendir(path); + if (!dir) { + err(ctx, "%s: opendir(\"%s\") failed\n", devname, path); + return -ENXIO; + } + + while ((de = readdir(dir)) != NULL) { + char *drv_path; + + if (de->d_ino == 0) + continue; + if (de->d_name[0] == '.') + continue; + if (strcmp(de->d_name, mod_name) != 0) + continue; + + if (asprintf(&drv_path, "%s/%s/new_id", path, de->d_name) < 0) { + err(ctx, "%s: path allocation failure\n", devname); + rc = -ENOMEM; + break; + } + rc = sysfs_write_attr_quiet(ctx, drv_path, devname); + free(drv_path); + + if (asprintf(&drv_path, "%s/%s/bind", path, de->d_name) < 0) { + err(ctx, "%s: path allocation failure\n", devname); + rc = -ENOMEM; + break; + } + rc = sysfs_write_attr_quiet(ctx, drv_path, devname); + free(drv_path); + break; + } + closedir(dir); + + if (rc) { + dbg(ctx, "%s: bind failed\n", devname); + return rc; + } + return 0; +} + +static int daxctl_unbind(struct daxctl_ctx *ctx, const char *devpath) +{ + const char *devname = devpath_to_devname(devpath); + char path[200]; + const int len = sizeof(path); + int rc; + + if (snprintf(path, len, "%s/driver/remove_id", devpath) >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + rc = sysfs_write_attr(ctx, path, devname); + if (rc) + return rc; + + if (snprintf(path, len, "%s/driver/unbind", devpath) >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + return sysfs_write_attr(ctx, path, devname); + +} + DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) { struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); @@ -584,6 +715,109 @@ DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) return is_enabled(path); } +static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev, + const char *mod_name) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + struct kmod_list *iter; + int rc = -ENXIO; + + if (dev->kmod_list == NULL) { + err(ctx, "%s: a modalias lookup list was not created\n", + devname); + return rc; + } + + kmod_list_foreach(iter, dev->kmod_list) { + struct kmod_module *mod = kmod_module_get_module(iter); + const char *name = kmod_module_get_name(mod); + + if (strcmp(name, mod_name) != 0) { + kmod_module_unref(mod); + continue; + } + dbg(ctx, "%s inserting module: %s\n", devname, name); + rc = kmod_module_probe_insert_module(mod, + KMOD_PROBE_APPLY_BLACKLIST, NULL, NULL, NULL, + NULL); + if (rc < 0) { + err(ctx, "%s: insert failure: %d\n", devname, rc); + return rc; + } + dev->module = mod; + } + + if (rc == -ENXIO) + err(ctx, "%s: Unable to find module: %s in alias list\n", + devname, mod_name); + return rc; +} + +static int daxctl_dev_enable(struct daxctl_dev *dev, enum daxctl_dev_mode mode) +{ + struct daxctl_region *region = daxctl_dev_get_region(dev); + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *mod_name = dax_modules[mode]; + int rc; + + if (daxctl_dev_is_enabled(dev)) + return 0; + + if (mode == DAXCTL_DEV_MODE_UNKNOWN || mod_name == NULL) { + err(ctx, "%s: Invalid mode: %d\n", devname, mode); + return -EINVAL; + } + + rc = daxctl_insert_kmod_for_mode(dev, mod_name); + if (rc) + return rc; + + rc = daxctl_bind(ctx, devname, mod_name); + if (!daxctl_dev_is_enabled(dev)) { + err(ctx, "%s: failed to enable\n", devname); + return rc ? rc : -ENXIO; + } + + region->devices_init = 0; + dax_devices_init(region); + rc = 0; + dbg(ctx, "%s: enabled\n", devname); + return rc; +} + +DAXCTL_EXPORT int daxctl_dev_enable_devdax(struct daxctl_dev *dev) +{ + return daxctl_dev_enable(dev, DAXCTL_DEV_MODE_DEVDAX); +} + +DAXCTL_EXPORT int daxctl_dev_enable_ram(struct daxctl_dev *dev) +{ + return daxctl_dev_enable(dev, DAXCTL_DEV_MODE_RAM); +} + +DAXCTL_EXPORT int daxctl_dev_disable(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + + if (!daxctl_dev_is_enabled(dev)) + return 0; + + daxctl_unbind(ctx, dev->dev_path); + + if (daxctl_dev_is_enabled(dev)) { + err(ctx, "%s: failed to disable\n", devname); + return -EBUSY; + } + + kmod_module_unref(dev->module); + dbg(ctx, "%s: disabled\n", devname); + + return 0; +} + DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) { return dev->region->ctx; diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index c4af9a7..19904a2 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -55,4 +55,7 @@ LIBDAXCTL_6 { global: daxctl_dev_get_ctx; daxctl_dev_is_enabled; + daxctl_dev_disable; + daxctl_dev_enable_devdax; + daxctl_dev_enable_ram; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index e20ccb4..b80488e 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -69,6 +69,9 @@ int daxctl_dev_get_minor(struct daxctl_dev *dev); unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev); struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev); int daxctl_dev_is_enabled(struct daxctl_dev *dev); +int daxctl_dev_disable(struct daxctl_dev *dev); +int daxctl_dev_enable_devdax(struct daxctl_dev *dev); +int daxctl_dev_enable_ram(struct daxctl_dev *dev); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ @@ -81,6 +84,12 @@ int daxctl_dev_is_enabled(struct daxctl_dev *dev); region != NULL; \ region = daxctl_region_get_next(region)) +enum daxctl_dev_mode { + DAXCTL_DEV_MODE_UNKNOWN, + DAXCTL_DEV_MODE_RAM, + DAXCTL_DEV_MODE_DEVDAX, +}; + #ifdef __cplusplus } /* extern "C" */ #endif From patchwork Fri Jun 28 19:11:00 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023345 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 30A9B15E6 for ; Fri, 28 Jun 2019 19:11:22 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2041728833 for ; Fri, 28 Jun 2019 19:11:22 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1471528837; Fri, 28 Jun 2019 19:11:22 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 93BB228834 for ; Fri, 28 Jun 2019 19:11:21 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 53A6621945DCD; Fri, 28 Jun 2019 12:11:20 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2BE2621959CB2 for ; Fri, 28 Jun 2019 12:11:18 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564728" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:17 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 03/13] libdaxctl: add an interface to retrieve the device resource Date: Fri, 28 Jun 2019 13:11:00 -0600 Message-Id: <20190628191110.21428-4-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin , Tony Luck Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add an interface to retrieve the 'resource' attribute for a dax device. Attempt to retrieve it as usual via sysfs, but since older kernels may be missing this attribute, as a fallback, attempt to retrieve it from /proc/iomem Cc: Dan Williams [fscanf format string problem and diagnosis] Reported-by: Tony Luck Signed-off-by: Vishal Verma --- Makefile.am | 3 ++- daxctl/lib/Makefile.am | 2 ++ daxctl/lib/libdaxctl-private.h | 1 + daxctl/lib/libdaxctl.c | 12 +++++++++++ daxctl/lib/libdaxctl.sym | 1 + daxctl/libdaxctl.h | 1 + util/iomem.c | 37 ++++++++++++++++++++++++++++++++++ util/iomem.h | 12 +++++++++++ 8 files changed, 68 insertions(+), 1 deletion(-) create mode 100644 util/iomem.c create mode 100644 util/iomem.h diff --git a/Makefile.am b/Makefile.am index df8797e..8d10a10 100644 --- a/Makefile.am +++ b/Makefile.am @@ -74,6 +74,7 @@ libutil_a_SOURCES = \ util/wrapper.c \ util/filter.c \ util/bitmap.c \ - util/abspath.c + util/abspath.c \ + util/iomem.c nobase_include_HEADERS = daxctl/libdaxctl.h diff --git a/daxctl/lib/Makefile.am b/daxctl/lib/Makefile.am index 9f0e444..7704b1b 100644 --- a/daxctl/lib/Makefile.am +++ b/daxctl/lib/Makefile.am @@ -9,6 +9,8 @@ lib_LTLIBRARIES = libdaxctl.la libdaxctl_la_SOURCES =\ ../libdaxctl.h \ libdaxctl-private.h \ + ../../util/iomem.c \ + ../../util/iomem.h \ ../../util/sysfs.c \ ../../util/sysfs.h \ ../../util/log.c \ diff --git a/daxctl/lib/libdaxctl-private.h b/daxctl/lib/libdaxctl-private.h index e64d0a7..eb7c1ec 100644 --- a/daxctl/lib/libdaxctl-private.h +++ b/daxctl/lib/libdaxctl-private.h @@ -59,6 +59,7 @@ struct daxctl_dev { size_t buf_len; char *dev_path; struct list_node list; + unsigned long long resource; unsigned long long size; struct kmod_module *module; struct kmod_list *kmod_list; diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 05f013b..5431063 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -24,6 +24,7 @@ #include #include +#include #include #include "libdaxctl-private.h" @@ -369,6 +370,12 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) dev->major = major(st.st_rdev); dev->minor = minor(st.st_rdev); + sprintf(path, "%s/resource", daxdev_base); + if (sysfs_read_attr(ctx, path, buf) == 0) + dev->resource = strtoull(buf, NULL, 0); + else + dev->resource = iomem_get_dev_resource(ctx, daxdev_base); + sprintf(path, "%s/size", daxdev_base); if (sysfs_read_attr(ctx, path, buf) < 0) goto err_read; @@ -878,6 +885,11 @@ DAXCTL_EXPORT int daxctl_dev_get_minor(struct daxctl_dev *dev) return dev->minor; } +DAXCTL_EXPORT unsigned long long daxctl_dev_get_resource(struct daxctl_dev *dev) +{ + return dev->resource; +} + DAXCTL_EXPORT unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev) { return dev->size; diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 19904a2..1692624 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -58,4 +58,5 @@ global: daxctl_dev_disable; daxctl_dev_enable_devdax; daxctl_dev_enable_ram; + daxctl_dev_get_resource; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index b80488e..7214cd3 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -66,6 +66,7 @@ int daxctl_dev_get_id(struct daxctl_dev *dev); const char *daxctl_dev_get_devname(struct daxctl_dev *dev); int daxctl_dev_get_major(struct daxctl_dev *dev); int daxctl_dev_get_minor(struct daxctl_dev *dev); +unsigned long long daxctl_dev_get_resource(struct daxctl_dev *dev); unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev); struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev); int daxctl_dev_is_enabled(struct daxctl_dev *dev); diff --git a/util/iomem.c b/util/iomem.c new file mode 100644 index 0000000..a3c23f5 --- /dev/null +++ b/util/iomem.c @@ -0,0 +1,37 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ + +#include +#include +#include +#include +#include +#include + +unsigned long long __iomem_get_dev_resource(struct log_ctx *ctx, + const char *devpath) +{ + const char *devname = devpath_to_devname(devpath); + FILE *fp = fopen("/proc/iomem", "r"); + unsigned long long res; + char name[256]; + + if (fp == NULL) { + log_err(ctx, "%s: open /proc/iomem: %s\n", devname, + strerror(errno)); + return 0; + } + + while (fscanf(fp, "%llx-%*x : %254[^\n]\n", &res, name) == 2) { + if (strcmp(name, devname) == 0) { + log_dbg(ctx, "%s: got resource via iomem: %#llx\n", + devname, res); + fclose(fp); + return res; + } + } + + log_dbg(ctx, "%s: not found in iomem\n", devname); + fclose(fp); + return 0; +} diff --git a/util/iomem.h b/util/iomem.h new file mode 100644 index 0000000..aaaf6a7 --- /dev/null +++ b/util/iomem.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ +#ifndef _NDCTL_IOMEM_H_ +#define _NDCTL_IOMEM_H_ + +struct log_ctx; +unsigned long long __iomem_get_dev_resource(struct log_ctx *ctx, + const char *path); + +#define iomem_get_dev_resource(c, p) __iomem_get_dev_resource(&(c)->ctx, (p)) + +#endif /* _NDCTL_IOMEM_H_ */ From patchwork Fri Jun 28 19:11:01 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023349 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 40D1E15E6 for ; Fri, 28 Jun 2019 19:11:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 3018B28833 for ; Fri, 28 Jun 2019 19:11:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 2430A28837; Fri, 28 Jun 2019 19:11:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 515DA28833 for ; Fri, 28 Jun 2019 19:11:24 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B488E212AB4F7; Fri, 28 Jun 2019 12:11:21 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id D65412129EBB7 for ; Fri, 28 Jun 2019 12:11:18 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564731" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:17 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 04/13] libdaxctl: add a 'daxctl_memory' object for memory based operations Date: Fri, 28 Jun 2019 13:11:01 -0600 Message-Id: <20190628191110.21428-5-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Introduce a new 'daxctl_memory' object, which will be used for operations related to managing dax devices in 'system-memory' modes. Add libdaxctl APIs to get the target_node of a DAX device, and to online, offline, and query the state of hotplugged memory sections associated with a given device. This adds the following new interfaces: daxctl_dev_get_target_node daxctl_dev_get_memory; daxctl_memory_get_dev; daxctl_memory_get_node_path; daxctl_memory_get_block_size; daxctl_memory_set_online daxctl_memory_set_offline daxctl_memory_is_online Cc: Pavel Tatashin Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl-private.h | 14 ++ daxctl/lib/libdaxctl.c | 333 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 8 + daxctl/libdaxctl.h | 10 + 4 files changed, 365 insertions(+) diff --git a/daxctl/lib/libdaxctl-private.h b/daxctl/lib/libdaxctl-private.h index eb7c1ec..673be0f 100644 --- a/daxctl/lib/libdaxctl-private.h +++ b/daxctl/lib/libdaxctl-private.h @@ -33,6 +33,11 @@ static const char *dax_modules[] = { [DAXCTL_DEV_MODE_RAM] = "kmem", }; +enum memory_state { + MEM_OFFLINE, + MEM_ONLINE, +}; + /** * struct daxctl_region - container for dax_devices */ @@ -64,8 +69,17 @@ struct daxctl_dev { struct kmod_module *module; struct kmod_list *kmod_list; struct daxctl_region *region; + struct daxctl_memory *mem; + int target_node; }; +struct daxctl_memory { +struct daxctl_dev *dev; + char *node_path; + unsigned long block_size; +}; + + static inline int check_kmod(struct kmod_ctx *kmod_ctx) { return kmod_ctx ? 0 : -ENXIO; diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 5431063..b22df50 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -200,6 +200,12 @@ DAXCTL_EXPORT void daxctl_region_get_uuid(struct daxctl_region *region, uuid_t u uuid_copy(uu, region->uuid); } +static void free_mem(struct daxctl_memory *mem) +{ + free(mem->node_path); + free(mem); +} + static void free_dev(struct daxctl_dev *dev, struct list_head *head) { if (head) @@ -207,6 +213,7 @@ static void free_dev(struct daxctl_dev *dev, struct list_head *head) kmod_module_unref_list(dev->kmod_list); free(dev->dev_buf); free(dev->dev_path); + free_mem(dev->mem); free(dev); } @@ -343,6 +350,44 @@ static struct kmod_list *to_module_list(struct daxctl_ctx *ctx, return list; } +static struct daxctl_memory *daxctl_dev_init_mem(struct daxctl_dev *dev) +{ + const char *size_path = "/sys/devices/system/memory/block_size_bytes"; + const char *node_base = "/sys/devices/system/node/node"; + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + struct daxctl_memory *mem; + char buf[SYSFS_ATTR_SIZE]; + int node_num; + + mem = calloc(1, sizeof(*mem)); + if (!mem) + return NULL; + + mem->dev = dev; + + /* + * Everything here is best-effort, we won't fail the device add + * for anything other than the ENOMEM case above. + */ + if (sysfs_read_attr(ctx, size_path, buf) == 0) { + mem->block_size = strtoul(buf, NULL, 16); + if (mem->block_size == 0 || mem->block_size == ULONG_MAX) { + err(ctx, "%s: Unable to determine memblock size: %s\n", + devname, strerror(errno)); + mem->block_size = 0; + } + } + + node_num = daxctl_dev_get_target_node(dev); + if (node_num >= 0) { + if (asprintf(&mem->node_path, "%s%d", node_base, node_num) < 0) + err(ctx, "%s: Unable to set node_path\n", devname); + } + + return mem; +} + static void *add_dax_dev(void *parent, int id, const char *daxdev_base) { const char *devname = devpath_to_devname(daxdev_base); @@ -398,6 +443,16 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) if (rc == 0) dev->kmod_list = to_module_list(ctx, buf); + sprintf(path, "%s/target_node", daxdev_base); + if (sysfs_read_attr(ctx, path, buf) == 0) + dev->target_node = strtol(buf, NULL, 0); + else + dev->target_node = -1; + + dev->mem = daxctl_dev_init_mem(dev); + if (!dev->mem) + goto err_read; + daxctl_dev_foreach(region, dev_dup) if (dev_dup->id == dev->id) { free_dev(dev, NULL); @@ -894,3 +949,281 @@ DAXCTL_EXPORT unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev) { return dev->size; } + +DAXCTL_EXPORT int daxctl_dev_get_target_node(struct daxctl_dev *dev) +{ + return dev->target_node; +} + +DAXCTL_EXPORT struct daxctl_memory *daxctl_dev_get_memory(struct daxctl_dev *dev) +{ + return dev->mem; +} + +DAXCTL_EXPORT struct daxctl_dev *daxctl_memory_get_dev(struct daxctl_memory *mem) +{ + return mem->dev; +} + +DAXCTL_EXPORT const char *daxctl_memory_get_node_path(struct daxctl_memory *mem) +{ + return mem->node_path; +} + +DAXCTL_EXPORT unsigned long daxctl_memory_get_block_size(struct daxctl_memory *mem) +{ + return mem->block_size; +} + +static int online_one_memblock(struct daxctl_dev *dev, char *path) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *mode = "online_movable"; + char buf[SYSFS_ATTR_SIZE]; + int rc; + + rc = sysfs_read_attr(ctx, path, buf); + if (rc) { + err(ctx, "%s: Failed to read %s: %s\n", + devname, path, strerror(-rc)); + return rc; + } + + /* + * if already online, possibly due to kernel config or a udev rule, + * there is nothing to do and we can skip over the memblock + */ + if (strncmp(buf, "online", 6) == 0) + return 1; + + rc = sysfs_write_attr_quiet(ctx, path, mode); + if (rc) + err(ctx, "%s: Failed to online %s: %s\n", + devname, path, strerror(-rc)); + return rc; +} + +static int offline_one_memblock(struct daxctl_dev *dev, char *path) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *mode = "offline"; + char buf[SYSFS_ATTR_SIZE]; + int rc; + + rc = sysfs_read_attr(ctx, path, buf); + if (rc) { + err(ctx, "%s: Failed to read %s: %s\n", + devname, path, strerror(-rc)); + return rc; + } + + /* if already offline, there is nothing to do */ + if (strncmp(buf, "offline", 6) == 0) + return 1; + + rc = sysfs_write_attr_quiet(ctx, path, mode); + if (rc) + err(ctx, "%s: Failed to offline %s: %s\n", + devname, path, strerror(-rc)); + return rc; +} + +static bool memblock_in_dev(struct daxctl_dev *dev, const char *memblock) +{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); + const char *mem_base = "/sys/devices/system/memory/"; + unsigned long long memblock_res, dev_start, dev_end; + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + unsigned long memblock_size; + char buf[SYSFS_ATTR_SIZE]; + unsigned long phys_index; + char *index_path; + + if (asprintf(&index_path, "%s/%s/phys_index", mem_base, memblock) < 0) + return false; + + if (sysfs_read_attr(ctx, index_path, buf) == 0) { + phys_index = strtoul(buf, NULL, 16); + if (phys_index == 0 || phys_index == ULONG_MAX) { + err(ctx, "%s: %s: Unable to determine phys_index: %s\n", + devname, memblock, strerror(errno)); + goto out_err; + } + } else { + err(ctx, "%s: %s: Unable to determine phys_index: %s\n", + devname, memblock, strerror(errno)); + goto out_err; + } + + dev_start = daxctl_dev_get_resource(dev); + if (!dev_start) { + err(ctx, "%s: Unable to determine resource\n", devname); + goto out_err; + } + dev_end = dev_start + daxctl_dev_get_size(dev); + + memblock_size = daxctl_memory_get_block_size(mem); + if (!memblock_size) { + err(ctx, "%s: Unable to determine memory block size\n", + devname); + goto out_err; + } + memblock_res = phys_index * memblock_size; + + if (memblock_res >= dev_start && memblock_res <= dev_end) { + free(index_path); + return true; + } + +out_err: + free(index_path); + return false; +} + +static int daxctl_memory_set_state(struct daxctl_memory *mem, + enum memory_state state) +{ + struct daxctl_dev *dev = daxctl_memory_get_dev(mem); + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *node_path; + int rc, changed = 0; + struct dirent *de; + DIR *node_dir; + + node_path = daxctl_memory_get_node_path(mem); + if (!node_path) { + err(ctx, "%s: Failed to get node_path\n", devname); + return -ENXIO; + } + + node_dir = opendir(node_path); + if (!node_dir) + return -errno; + + errno = 0; + while ((de = readdir(node_dir)) != NULL) { + char *mem_path; + + if (strncmp(de->d_name, "memory", 6) == 0) { + if (!memblock_in_dev(dev, de->d_name)) + continue; + rc = asprintf(&mem_path, "%s/%s/state", + node_path, de->d_name); + if (rc < 0) { + rc = -ENOMEM; + goto out_dir; + } + if (state == MEM_ONLINE) + rc = online_one_memblock(dev, mem_path); + else if (state == MEM_OFFLINE) + rc = offline_one_memblock(dev, mem_path); + else + rc = -EINVAL; + free(mem_path); + if (rc < 0) + goto out_dir; + if (rc == 0) + changed++; + } + errno = 0; + } + if (errno) { + rc = -errno; + goto out_dir; + } + rc = changed; + +out_dir: + closedir(node_dir); + return rc; +} + +DAXCTL_EXPORT int daxctl_memory_set_online(struct daxctl_memory *mem) +{ + return daxctl_memory_set_state(mem, MEM_ONLINE); +} + +DAXCTL_EXPORT int daxctl_memory_set_offline(struct daxctl_memory *mem) +{ + return daxctl_memory_set_state(mem, MEM_OFFLINE); +} + +static int memblock_is_online(struct daxctl_dev *dev, char *path) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + char buf[SYSFS_ATTR_SIZE]; + int rc; + + rc = sysfs_read_attr(ctx, path, buf); + if (rc) { + err(ctx, "%s: Failed to read %s: %s\n", + devname, path, strerror(-rc)); + return rc; + } + + if (strncmp(buf, "online", 6) == 0) + return 1; + + return 0; +} + +DAXCTL_EXPORT int daxctl_memory_is_online(struct daxctl_memory *mem) +{ + struct daxctl_dev *dev = daxctl_memory_get_dev(mem); + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + int rc, num_online = 0; + const char *node_path; + struct dirent *de; + DIR *node_dir; + + node_path = daxctl_memory_get_node_path(mem); + if (!node_path) { + err(ctx, "%s: Failed to get node_path\n", devname); + return -ENXIO; + } + + node_dir = opendir(node_path); + if (!node_dir) + return -errno; + + errno = 0; + while ((de = readdir(node_dir)) != NULL) { + char *mem_path; + + if (strncmp(de->d_name, "memory", 6) == 0) { + if (!memblock_in_dev(dev, de->d_name)) + continue; + rc = asprintf(&mem_path, "%s/%s/state", + node_path, de->d_name); + if (rc < 0) { + rc = -ENOMEM; + goto out_dir; + } + rc = memblock_is_online(dev, mem_path); + if (rc < 0) { + err(ctx, "%s: Unable to determine state: %s\n", + devname, mem_path); + goto out_dir; + } + if (rc > 0) + num_online++; + free(mem_path); + } + errno = 0; + } + if (errno) { + rc = -errno; + goto out_dir; + } + rc = num_online; + +out_dir: + closedir(node_dir); + return rc; +} diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 1692624..53eb700 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -59,4 +59,12 @@ global: daxctl_dev_enable_devdax; daxctl_dev_enable_ram; daxctl_dev_get_resource; + daxctl_dev_get_target_node; + daxctl_dev_get_memory; + daxctl_memory_get_dev; + daxctl_memory_get_node_path; + daxctl_memory_get_block_size; + daxctl_memory_set_online; + daxctl_memory_set_offline; + daxctl_memory_is_online; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index 7214cd3..a5a2bab 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -73,6 +73,16 @@ int daxctl_dev_is_enabled(struct daxctl_dev *dev); int daxctl_dev_disable(struct daxctl_dev *dev); int daxctl_dev_enable_devdax(struct daxctl_dev *dev); int daxctl_dev_enable_ram(struct daxctl_dev *dev); +int daxctl_dev_get_target_node(struct daxctl_dev *dev); + +struct daxctl_memory; +struct daxctl_memory *daxctl_dev_get_memory(struct daxctl_dev *dev); +struct daxctl_dev *daxctl_memory_get_dev(struct daxctl_memory *mem); +const char *daxctl_memory_get_node_path(struct daxctl_memory *mem); +unsigned long daxctl_memory_get_block_size(struct daxctl_memory *mem); +int daxctl_memory_set_online(struct daxctl_memory *mem); +int daxctl_memory_set_offline(struct daxctl_memory *mem); +int daxctl_memory_is_online(struct daxctl_memory *mem); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ From patchwork Fri Jun 28 19:11:02 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023347 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E2919138B for ; Fri, 28 Jun 2019 19:11:24 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id D231328834 for ; Fri, 28 Jun 2019 19:11:24 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id C63CE2883B; Fri, 28 Jun 2019 19:11:24 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 8C7B928834 for ; Fri, 28 Jun 2019 19:11:24 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 1DF8121945DE0; Fri, 28 Jun 2019 12:11:22 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2A05A212AAFE9 for ; Fri, 28 Jun 2019 12:11:19 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:18 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564734" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:18 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 05/13] daxctl/list: add target_node for device listings Date: Fri, 28 Jun 2019 13:11:02 -0600 Message-Id: <20190628191110.21428-6-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP The kernel provides a 'target_node' attribute for dax devices. When converting a dax device to the system-ram mode, the memory is hotplugged into this numa node. It would be helpful to print this in device listings so that it is easy for applications to detect the node to which the new memory belongs. Cc: Dan Williams Cc: Dave Hansen Signed-off-by: Vishal Verma --- util/json.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/util/json.c b/util/json.c index babdc8c..f521337 100644 --- a/util/json.c +++ b/util/json.c @@ -271,6 +271,7 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, { const char *devname = daxctl_dev_get_devname(dev); struct json_object *jdev, *jobj; + int node; jdev = json_object_new_object(); if (!devname || !jdev) @@ -284,6 +285,13 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, if (jobj) json_object_object_add(jdev, "size", jobj); + node = daxctl_dev_get_target_node(dev); + if (node >= 0) { + jobj = json_object_new_int(node); + if (jobj) + json_object_object_add(jdev, "target_node", jobj); + } + return jdev; } From patchwork Fri Jun 28 19:11:03 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023351 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 97513138D for ; Fri, 28 Jun 2019 19:11:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 87BF828833 for ; Fri, 28 Jun 2019 19:11:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 7BF5A2883B; Fri, 28 Jun 2019 19:11:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 1D98E28833 for ; Fri, 28 Jun 2019 19:11:27 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 52972212ABA21; Fri, 28 Jun 2019 12:11:22 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id B5A0D2194EB77 for ; Fri, 28 Jun 2019 12:11:19 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564738" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:18 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 06/13] libdaxctl: add an interface to get the mode for a dax device Date: Fri, 28 Jun 2019 13:11:03 -0600 Message-Id: <20190628191110.21428-7-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP In preparation for a reconfigure-device command, add an interface to retrieve the 'mode' of a dax device. This will allow the reconfigure-device command (and via daxctl_dev_to_json(), also daxctl-list) to print the mode in device listings via a 'daxctl-list' command or immediately after a mode change. Add a 'state' attribute to the json listings for devices, since a device could end up in a state where it is not bound to any driver, and hence, 'disabled'. The state attribute is only displayed for disabled devices. Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl.c | 37 +++++++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 1 + daxctl/libdaxctl.h | 1 + util/json.c | 17 +++++++++++++++++ 4 files changed, 56 insertions(+) diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index b22df50..9ff6235 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -12,6 +12,8 @@ */ #include #include +#include +#include #include #include #include @@ -880,6 +882,41 @@ DAXCTL_EXPORT int daxctl_dev_disable(struct daxctl_dev *dev) return 0; } +DAXCTL_EXPORT enum daxctl_dev_mode daxctl_dev_get_mode(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + int rc = DAXCTL_DEV_MODE_UNKNOWN; + char path[200]; + const int len = sizeof(path); + char *mod_path, *mod_base; + + if (!daxctl_dev_is_enabled(dev)) + return rc; + + if (snprintf(path, len, "%s/driver/module", dev->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + mod_path = realpath(path, NULL); + if (!mod_path) { + rc = -errno; + err(ctx, "%s: unable to determine module: %s\n", devname, + strerror(errno)); + return rc; + } + + mod_base = basename(mod_path); + if (strcmp(mod_base, dax_modules[DAXCTL_DEV_MODE_RAM]) == 0) + rc = DAXCTL_DEV_MODE_RAM; + else if (strcmp(mod_base, dax_modules[DAXCTL_DEV_MODE_DEVDAX]) == 0) + rc = DAXCTL_DEV_MODE_DEVDAX; + + free(mod_path); + return rc; +} + DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) { return dev->region->ctx; diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 53eb700..9ce5096 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -67,4 +67,5 @@ global: daxctl_memory_set_online; daxctl_memory_set_offline; daxctl_memory_is_online; + daxctl_dev_get_mode; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index a5a2bab..6d1c17d 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -74,6 +74,7 @@ int daxctl_dev_disable(struct daxctl_dev *dev); int daxctl_dev_enable_devdax(struct daxctl_dev *dev); int daxctl_dev_enable_ram(struct daxctl_dev *dev); int daxctl_dev_get_target_node(struct daxctl_dev *dev); +enum daxctl_dev_mode daxctl_dev_get_mode(struct daxctl_dev *dev); struct daxctl_memory; struct daxctl_memory *daxctl_dev_get_memory(struct daxctl_dev *dev); diff --git a/util/json.c b/util/json.c index f521337..1673f1d 100644 --- a/util/json.c +++ b/util/json.c @@ -271,6 +271,7 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, { const char *devname = daxctl_dev_get_devname(dev); struct json_object *jdev, *jobj; + enum daxctl_dev_mode mode; int node; jdev = json_object_new_object(); @@ -292,6 +293,22 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, json_object_object_add(jdev, "target_node", jobj); } + mode = daxctl_dev_get_mode(dev); + if (mode >= 0) { + if (mode == DAXCTL_DEV_MODE_RAM) + jobj = json_object_new_string("system-ram"); + else + jobj = json_object_new_string("devdax"); + if (jobj) + json_object_object_add(jdev, "mode", jobj); + } + + if (!daxctl_dev_is_enabled(dev)) { + jobj = json_object_new_string("disabled"); + if (jobj) + json_object_object_add(jdev, "state", jobj); + } + return jdev; } From patchwork Fri Jun 28 19:11:04 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023353 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 38B08138D for ; Fri, 28 Jun 2019 19:11:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 26F4928833 for ; Fri, 28 Jun 2019 19:11:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 1B9562883B; Fri, 28 Jun 2019 19:11:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 646CA28833 for ; Fri, 28 Jun 2019 19:11:28 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 976FD212ABA28; Fri, 28 Jun 2019 12:11:22 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 4EC8A21945DC7 for ; Fri, 28 Jun 2019 12:11:20 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564743" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:19 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 07/13] daxctl: add a new reconfigure-device command Date: Fri, 28 Jun 2019 13:11:04 -0600 Message-Id: <20190628191110.21428-8-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add a new command 'daxctl-reconfigure-device'. This is used to switch the mode of a dax device between regular 'device_dax' and 'system-memory'. The command also uses the memory hotplug sysfs interfaces to online the newly available memory when converting to 'system-ram', and to attempt to offline the memory when converting back to a DAX device. Cc: Pavel Tatashin Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/Makefile.am | 2 + daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c | 348 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 352 insertions(+) create mode 100644 daxctl/device.c diff --git a/daxctl/Makefile.am b/daxctl/Makefile.am index 94f73f9..66dcc7f 100644 --- a/daxctl/Makefile.am +++ b/daxctl/Makefile.am @@ -15,10 +15,12 @@ daxctl_SOURCES =\ daxctl.c \ list.c \ migrate.c \ + device.c \ ../util/json.c daxctl_LDADD =\ lib/libdaxctl.la \ ../libutil.a \ $(UUID_LIBS) \ + $(KMOD_LIBS) \ $(JSON_LIBS) diff --git a/daxctl/builtin.h b/daxctl/builtin.h index 00ef5e9..756ba2a 100644 --- a/daxctl/builtin.h +++ b/daxctl/builtin.h @@ -6,4 +6,5 @@ struct daxctl_ctx; int cmd_list(int argc, const char **argv, struct daxctl_ctx *ctx); int cmd_migrate(int argc, const char **argv, struct daxctl_ctx *ctx); +int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx); #endif /* _DAXCTL_BUILTIN_H_ */ diff --git a/daxctl/daxctl.c b/daxctl/daxctl.c index 2e41747..e1ba7b8 100644 --- a/daxctl/daxctl.c +++ b/daxctl/daxctl.c @@ -71,6 +71,7 @@ static struct cmd_struct commands[] = { { "list", .d_fn = cmd_list }, { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, + { "reconfigure-device", .d_fn = cmd_reconfig_device }, }; int main(int argc, const char **argv) diff --git a/daxctl/device.c b/daxctl/device.c new file mode 100644 index 0000000..a1fb698 --- /dev/null +++ b/daxctl/device.c @@ -0,0 +1,348 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static struct { + const char *dev; + const char *mode; + int region_id; + bool no_online; + bool do_offline; + bool human; + bool verbose; +} param = { + .region_id = -1, +}; + +static enum daxctl_dev_mode reconfig_mode = DAXCTL_DEV_MODE_UNKNOWN; +static unsigned long flags; + +enum device_action { + ACTION_RECONFIG, +}; + +#define BASE_OPTIONS() \ +OPT_INTEGER('r', "region", ¶m.region_id, "restrict to the given region"), \ +OPT_BOOLEAN('u', "human", ¶m.human, "use human friendly number formats"), \ +OPT_BOOLEAN('v', "verbose", ¶m.verbose, "emit more debug messages") + +#define RECONFIG_OPTIONS() \ +OPT_STRING('m', "mode", ¶m.mode, "mode", "mode to switch the device to"), \ +OPT_BOOLEAN('N', "no-online", ¶m.no_online, \ + "don't auto-online memory sections"), \ +OPT_BOOLEAN('O', "attempt-offline", ¶m.do_offline, \ + "attempt to offline memory sections") + +static const struct option reconfig_options[] = { + BASE_OPTIONS(), + RECONFIG_OPTIONS(), + OPT_END(), +}; + +static const char *parse_device_options(int argc, const char **argv, + enum device_action action, const struct option *options, + const char *usage, struct daxctl_ctx *ctx) +{ + const char * const u[] = { + usage, + NULL + }; + int i, rc = 0; + + argc = parse_options(argc, argv, options, u, 0); + + /* Handle action-agnostic non-option arguments */ + if (argc == 0) { + char *action_string; + + switch (action) { + case ACTION_RECONFIG: + action_string = "reconfigure"; + break; + default: + action_string = "<>"; + break; + } + fprintf(stderr, "specify a device to %s, or \"all\"\n", + action_string); + rc = -EINVAL; + } + for (i = 1; i < argc; i++) { + fprintf(stderr, "unknown extra parameter \"%s\"\n", argv[i]); + rc = -EINVAL; + } + + if (rc) { + usage_with_options(u, options); + return NULL; + } + + /* Handle action-agnostic options */ + if (param.verbose) + daxctl_set_log_priority(ctx, LOG_DEBUG); + if (param.human) + flags |= UTIL_JSON_HUMAN; + + /* Handle action-specific options */ + switch (action) { + case ACTION_RECONFIG: + if (!param.mode) { + fprintf(stderr, "error: a 'mode' option is required\n"); + usage_with_options(u, reconfig_options); + rc = -EINVAL; + } + if (strcmp(param.mode, "system-ram") == 0) { + reconfig_mode = DAXCTL_DEV_MODE_RAM; + if (param.do_offline) { + fprintf(stderr, + "can't --attempt-offline for system-ram mode\n"); + rc = -EINVAL; + } + } else if (strcmp(param.mode, "devdax") == 0) { + reconfig_mode = DAXCTL_DEV_MODE_DEVDAX; + if (param.no_online) { + fprintf(stderr, + "can't --no-online for devdax mode\n"); + rc = -EINVAL; + } + } + break; + } + if (rc) { + usage_with_options(u, options); + return NULL; + } + + return argv[0]; +} + +static int disable_devdax_device(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + enum daxctl_dev_mode mode; + int rc; + + mode = daxctl_dev_get_mode(dev); + if (mode < 0) { + fprintf(stderr, "%s: unable to determine current mode: %s\n", + devname, strerror(-mode)); + return mode; + } + if (mode == DAXCTL_DEV_MODE_RAM) { + fprintf(stderr, "%s is in system-ram mode, nothing to do\n", + devname); + return 1; + } + rc = daxctl_dev_disable(dev); + if (rc) { + fprintf(stderr, "%s: disable failed: %s\n", + daxctl_dev_get_devname(dev), strerror(-rc)); + return rc; + } + return 0; +} + +static int reconfig_mode_system_ram(struct daxctl_dev *dev) +{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); + const char *devname = daxctl_dev_get_devname(dev); + int rc, skip_enable = 0; + + if (daxctl_dev_is_enabled(dev)) { + rc = disable_devdax_device(dev); + if (rc < 0) + return rc; + if (rc > 0) + skip_enable = 1; + } + + if (!skip_enable) { + rc = daxctl_dev_enable_ram(dev); + if (rc) + return rc; + } + + if (param.no_online) + return 0; + + rc = daxctl_memory_set_online(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to online memory: %s\n", + devname, strerror(-rc)); + return rc; + } + if (param.verbose) + fprintf(stderr, "%s: onlined %d memory sections\n", + devname, rc); + + return 0; +} + +static int disable_system_ram_device(struct daxctl_dev *dev) +{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); + const char *devname = daxctl_dev_get_devname(dev); + enum daxctl_dev_mode mode; + int rc; + + mode = daxctl_dev_get_mode(dev); + if (mode < 0) { + fprintf(stderr, "%s: unable to determine current mode: %s\n", + devname, strerror(-mode)); + return mode; + } + if (mode == DAXCTL_DEV_MODE_DEVDAX) { + fprintf(stderr, "%s: already in devdax mode, nothing to do\n", + devname); + return 1; + } + + if (param.do_offline) { + rc = daxctl_memory_set_offline(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to offline memory: %s\n", + devname, strerror(-rc)); + return rc; + } + if (param.verbose) + fprintf(stderr, "%s: offlined %d memory sections\n", + devname, rc); + } + + rc = daxctl_memory_is_online(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to determine online state: %s\n", + devname, strerror(-rc)); + return rc; + } + if (rc > 0) { + if (param.verbose) { + fprintf(stderr, "%s: found %d memory sections online\n", + devname, rc); + fprintf(stderr, "%s: refusing to change modes\n", + devname); + } + return -EBUSY; + } + rc = daxctl_dev_disable(dev); + if (rc) { + fprintf(stderr, "%s: disable failed: %s\n", + daxctl_dev_get_devname(dev), strerror(-rc)); + return rc; + } + return 0; +} + +static int reconfig_mode_devdax(struct daxctl_dev *dev) +{ + int rc; + + if (daxctl_dev_is_enabled(dev)) { + rc = disable_system_ram_device(dev); + if (rc) + return rc; + } + + rc = daxctl_dev_enable_devdax(dev); + if (rc) + return rc; + + return 0; +} + +static int do_reconfig(struct daxctl_dev *dev, enum daxctl_dev_mode mode, + struct json_object *jdevs) +{ + struct json_object *jdev; + int rc = 0; + + switch (mode) { + case DAXCTL_DEV_MODE_RAM: + rc = reconfig_mode_system_ram(dev); + break; + case DAXCTL_DEV_MODE_DEVDAX: + rc = reconfig_mode_devdax(dev); + break; + default: + fprintf(stderr, "%s: unknown mode requested: %d\n", + daxctl_dev_get_devname(dev), mode); + rc = -EINVAL; + } + + jdev = util_daxctl_dev_to_json(dev, flags); + if (jdevs && jdev) + json_object_array_add(jdevs, jdev); + + return rc; +} + +static int do_xaction_device(const char *device, enum device_action action, + struct daxctl_ctx *ctx, int *processed) +{ + struct json_object *jdevs = json_object_new_array(); + struct daxctl_region *region; + struct daxctl_dev *dev; + int rc = -ENXIO; + + *processed = 0; + + daxctl_region_foreach(ctx, region) { + if (param.region_id >= 0 && param.region_id + != daxctl_region_get_id(region)) + continue; + + daxctl_dev_foreach(region, dev) { + if (!util_daxctl_dev_filter(dev, device)) + continue; + + switch (action) { + case ACTION_RECONFIG: + rc = do_reconfig(dev, reconfig_mode, jdevs); + if (rc == 0) + (*processed)++; + break; + default: + rc = -EINVAL; + break; + } + } + } + + /* + * jdevs is the containing json array for all devices we are reporting + * on. It therefore needs to be outside the region/device iterators, + * and passed in to the do_ functions to add their objects to + */ + if (jdevs) + util_display_json_array(stdout, jdevs, flags); + + return rc; +} + +int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx) +{ + char *usage = "daxctl reconfigure-device []"; + const char *device = parse_device_options(argc, argv, ACTION_RECONFIG, + reconfig_options, usage, ctx); + int processed, rc; + + rc = do_xaction_device(device, ACTION_RECONFIG, ctx, &processed); + if (rc < 0) + fprintf(stderr, "error reconfiguring devices: %s\n", + strerror(-rc)); + + fprintf(stderr, "reconfigured %d device%s\n", processed, + processed == 1 ? "" : "s"); + return rc; +} From patchwork Fri Jun 28 19:11:05 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023355 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5D19F138B for ; Fri, 28 Jun 2019 19:11:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 4DA5E28833 for ; Fri, 28 Jun 2019 19:11:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 4229328837; Fri, 28 Jun 2019 19:11:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C56AB28834 for ; Fri, 28 Jun 2019 19:11:28 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id DC9E4212ABA2D; Fri, 28 Jun 2019 12:11:24 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id BAFA62129EBB7 for ; Fri, 28 Jun 2019 12:11:20 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564748" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:20 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 08/13] Documentation/daxctl: add a man page for daxctl-reconfigure-device Date: Fri, 28 Jun 2019 13:11:05 -0600 Message-Id: <20190628191110.21428-9-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add a man page describing the new daxctl-reconfigure-device command. Cc: Pavel Tatashin Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- Documentation/daxctl/Makefile.am | 3 +- .../daxctl/daxctl-reconfigure-device.txt | 139 ++++++++++++++++++ 2 files changed, 141 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt diff --git a/Documentation/daxctl/Makefile.am b/Documentation/daxctl/Makefile.am index 6aba035..715fbad 100644 --- a/Documentation/daxctl/Makefile.am +++ b/Documentation/daxctl/Makefile.am @@ -28,7 +28,8 @@ endif man1_MANS = \ daxctl.1 \ daxctl-list.1 \ - daxctl-migrate-device-model.1 + daxctl-migrate-device-model.1 \ + daxctl-reconfigure-device.1 CLEANFILES = $(man1_MANS) diff --git a/Documentation/daxctl/daxctl-reconfigure-device.txt b/Documentation/daxctl/daxctl-reconfigure-device.txt new file mode 100644 index 0000000..fb2b36b --- /dev/null +++ b/Documentation/daxctl/daxctl-reconfigure-device.txt @@ -0,0 +1,139 @@ +// SPDX-License-Identifier: GPL-2.0 + +daxctl-reconfigure-device(1) +============================ + +NAME +---- +daxctl-reconfigure-device - Reconfigure a dax device into a different mode + +SYNOPSIS +-------- +[verse] +'daxctl reconfigure-device' [...] [] + +EXAMPLES +-------- + +* Reconfigure dax0.0 to system-ram mode, don't online the memory +---- +# daxctl reconfigure-device --mode=system-ram --no-online dax0.0 +[ + { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + } +] +---- + +* Reconfigure dax0.0 to devdax mode, attempt to offline the memory +---- +# daxctl reconfigure-device --human --mode=devdax --attempt-offline dax0.0 +{ + "chardev":"dax0.0", + "size":"15.63 GiB (16.78 GB)", + "target_node":2, + "mode":"devdax" +} +---- + +* Reconfigure all dax devices on region0 to system-ram mode +---- +# daxctl reconfigure-device --mode=system-ram --region=0 all +[ + { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + }, + { + "chardev":"dax0.1", + "size":16777216000, + "target_node":3, + "mode":"system-ram" + } +] +---- + +* Run a process called 'some-service' using numactl to restrict its cpu +nodes to '0' and '1', and memory allocations to node 2 (determined using +daxctl_dev_get_target_node() or 'daxctl list') +---- +# daxctl reconfigure-device --mode=system-ram --no-online dax0.0 +[ + { + "chardev":"dax0.0", + "size":16777216000, + "target_node":2, + "mode":"system-ram" + } +] + +# numactl --cpunodebind=0-1 --membind=2 -- some-service --opt1 --opt2 +---- + +DESCRIPTION +----------- + +Reconfigure the operational mode of a dax device. This can be used to convert +a regular 'devdax' mode device to the 'system-ram' mode which allows for the dax +range to be hot-plugged into the system as regular memory. + +NOTE: This is a destructive operation. Any data on the dax device *will* be +lost. + +NOTE: Device reconfiguration depends on the dax-bus device model. If dax-class is +in use (via the dax_pmem_compat driver), the reconfiguration will fail. See +linkdaxctl:daxctl-migrate-device-model[1] for more information. + +OPTIONS +------- +-r:: +--region=:: + Restrict the operation to devices belonging to the specified region(s). + A device-dax region is a contiguous range of memory that hosts one or + more /dev/daxX.Y devices, where X is the region id and Y is the device + instance id. + +-m:: +--mode=:: + Specify the mode to which the dax device(s) should be reconfigured. + - "system-ram": hotplug the device into system memory. + + - "devdax": switch to the normal "device dax" mode. This requires the + kernel to support hot-unplugging 'kmem' based memory. If this is not + available, a reboot is the only way to switch back to 'devdax' mode. + +-N:: +--no-online:: + By default, memory sections provided by system-ram devices will be + brought online automatically and immediately with the 'online_movable' + policy. Use this option to disable the automatic onlining behavior. + +-O:: +--attempt-offline:: + When converting from "system-ram" mode to "devdax", it is expected + that all the memory sections are first made offline. By default, + daxctl won't touch online memory. However with this option, attempt + to offline the memory on the NUMA node associated with the dax device + before converting it back to "devdax" mode. + +-u:: +--human:: + By default the command will output machine-friendly raw-integer + data. Instead, with this flag, numbers representing storage size + will be formatted as human readable strings with units, other + fields are converted to hexadecimal strings. + +-v:: +--verbose:: + Emit more debug messages + +include::../copyright.txt[] + +SEE ALSO +-------- +linkdaxctl:daxctl-list[1],daxctl-migrate-device-model[1] From patchwork Fri Jun 28 19:11:06 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023357 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C898F138D for ; Fri, 28 Jun 2019 19:11:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B766F28833 for ; Fri, 28 Jun 2019 19:11:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id AB91728837; Fri, 28 Jun 2019 19:11:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 306A328833 for ; Fri, 28 Jun 2019 19:11:30 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 3227F212ABA37; Fri, 28 Jun 2019 12:11:25 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 4A2742129EBB7 for ; Fri, 28 Jun 2019 12:11:21 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564751" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:20 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 09/13] daxctl: add commands to online and offline memory Date: Fri, 28 Jun 2019 13:11:06 -0600 Message-Id: <20190628191110.21428-10-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add two new commands: daxctl-online-memory daxctl-offline-memory to manage the state of hot-plugged memory from the system-ram mode for dax devices. This provides a way for the user to online/offline the memory as a separate step from the reconfiguration. Without this, a user that reconfigures a device into the system-ram mode with the --no-online option, would have no way to later online the memory, and would have to resort to shell scripting to online them manually via sysfs. Cc: Dan Williams Cc: Dave Hansen Signed-off-by: Vishal Verma --- daxctl/builtin.h | 2 + daxctl/daxctl.c | 2 + daxctl/device.c | 138 ++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 141 insertions(+), 1 deletion(-) diff --git a/daxctl/builtin.h b/daxctl/builtin.h index 756ba2a..f5a0147 100644 --- a/daxctl/builtin.h +++ b/daxctl/builtin.h @@ -7,4 +7,6 @@ struct daxctl_ctx; int cmd_list(int argc, const char **argv, struct daxctl_ctx *ctx); int cmd_migrate(int argc, const char **argv, struct daxctl_ctx *ctx); int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx); +int cmd_online_memory(int argc, const char **argv, struct daxctl_ctx *ctx); +int cmd_offline_memory(int argc, const char **argv, struct daxctl_ctx *ctx); #endif /* _DAXCTL_BUILTIN_H_ */ diff --git a/daxctl/daxctl.c b/daxctl/daxctl.c index e1ba7b8..1ab0732 100644 --- a/daxctl/daxctl.c +++ b/daxctl/daxctl.c @@ -72,6 +72,8 @@ static struct cmd_struct commands[] = { { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, { "reconfigure-device", .d_fn = cmd_reconfig_device }, + { "online-memory", .d_fn = cmd_online_memory }, + { "offline-memory", .d_fn = cmd_offline_memory }, }; int main(int argc, const char **argv) diff --git a/daxctl/device.c b/daxctl/device.c index a1fb698..22cf0c8 100644 --- a/daxctl/device.c +++ b/daxctl/device.c @@ -30,6 +30,8 @@ static unsigned long flags; enum device_action { ACTION_RECONFIG, + ACTION_ONLINE, + ACTION_OFFLINE, }; #define BASE_OPTIONS() \ @@ -50,6 +52,11 @@ static const struct option reconfig_options[] = { OPT_END(), }; +static const struct option memory_options[] = { + BASE_OPTIONS(), + OPT_END(), +}; + static const char *parse_device_options(int argc, const char **argv, enum device_action action, const struct option *options, const char *usage, struct daxctl_ctx *ctx) @@ -70,6 +77,12 @@ static const char *parse_device_options(int argc, const char **argv, case ACTION_RECONFIG: action_string = "reconfigure"; break; + case ACTION_ONLINE: + action_string = "online memory for"; + break; + case ACTION_OFFLINE: + action_string = "offline memory for"; + break; default: action_string = "<>"; break; @@ -118,6 +131,10 @@ static const char *parse_device_options(int argc, const char **argv, } } break; + case ACTION_ONLINE: + case ACTION_OFFLINE: + /* nothing special */ + break; } if (rc) { usage_with_options(u, options); @@ -287,10 +304,82 @@ static int do_reconfig(struct daxctl_dev *dev, enum daxctl_dev_mode mode, return rc; } +static int do_xline(struct daxctl_dev *dev, enum device_action action) +{ + struct daxctl_memory *mem = daxctl_dev_get_memory(dev); + const char *devname = daxctl_dev_get_devname(dev); + enum daxctl_dev_mode mode; + int rc, num_online; + + if (!daxctl_dev_is_enabled(dev)) { + fprintf(stderr, + "%s: memory operations not possible when disabled\n", + devname); + return -ENXIO; + } + + mode = daxctl_dev_get_mode(dev); + if (mode < 0) { + fprintf(stderr, "%s: unable to determine current mode: %s\n", + devname, strerror(-mode)); + return rc; + } + if (mode == DAXCTL_DEV_MODE_DEVDAX) { + fprintf(stderr, + "%s: memory operations are not applicable in devdax mode\n", + devname); + return -ENXIO; + } + + /* We are enabled, and in the correct mode. Proceed. */ + num_online = daxctl_memory_is_online(mem); + if (num_online < 0) { + fprintf(stderr, "%s: unable to determine online state: %s\n", + devname, strerror(-num_online)); + return num_online; + } + + switch (action) { + case ACTION_ONLINE: + if (num_online > 0) + fprintf(stderr, "%s: %d section%s already online\n", + devname, num_online, + num_online == 1 ? "" : "s"); + rc = daxctl_memory_set_online(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to online memory: %s\n", + devname, strerror(-rc)); + return rc; + } + fprintf(stderr, "%s: %d new section%s onlined\n", devname, rc, + rc == 1 ? "" : "s"); + break; + case ACTION_OFFLINE: + if (num_online == 0) { + fprintf(stderr, "%s: all sections already offline\n", + devname); + return 0; + } + rc = daxctl_memory_set_offline(mem); + if (rc < 0) { + fprintf(stderr, "%s: unable to offline memory: %s\n", + devname, strerror(-rc)); + return rc; + } + fprintf(stderr, "%s: %d section%s offlined\n", devname, rc, + rc == 1 ? "" : "s"); + break; + default: + fprintf(stderr, "%s: invalid action: %d\n", devname, action); + return -EINVAL; + } + return 0; +} + static int do_xaction_device(const char *device, enum device_action action, struct daxctl_ctx *ctx, int *processed) { - struct json_object *jdevs = json_object_new_array(); + struct json_object *jdevs = NULL; struct daxctl_region *region; struct daxctl_dev *dev; int rc = -ENXIO; @@ -308,10 +397,23 @@ static int do_xaction_device(const char *device, enum device_action action, switch (action) { case ACTION_RECONFIG: + /* reconfig needs jdevs, initialize it once */ + if (!jdevs) + jdevs = json_object_new_array(); rc = do_reconfig(dev, reconfig_mode, jdevs); if (rc == 0) (*processed)++; break; + case ACTION_ONLINE: + rc = do_xline(dev, action); + if (rc == 0) + (*processed)++; + break; + case ACTION_OFFLINE: + rc = do_xline(dev, action); + if (rc == 0) + (*processed)++; + break; default: rc = -EINVAL; break; @@ -346,3 +448,37 @@ int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx) processed == 1 ? "" : "s"); return rc; } + +int cmd_online_memory(int argc, const char **argv, struct daxctl_ctx *ctx) +{ + char *usage = "daxctl online-memory []"; + const char *device = parse_device_options(argc, argv, ACTION_ONLINE, + memory_options, usage, ctx); + int processed, rc; + + rc = do_xaction_device(device, ACTION_ONLINE, ctx, &processed); + if (rc < 0) + fprintf(stderr, "error onlining memory: %s\n", + strerror(-rc)); + + fprintf(stderr, "onlined memory for %d device%s\n", processed, + processed == 1 ? "" : "s"); + return rc; +} + +int cmd_offline_memory(int argc, const char **argv, struct daxctl_ctx *ctx) +{ + char *usage = "daxctl offline-memory []"; + const char *device = parse_device_options(argc, argv, ACTION_OFFLINE, + memory_options, usage, ctx); + int processed, rc; + + rc = do_xaction_device(device, ACTION_OFFLINE, ctx, &processed); + if (rc < 0) + fprintf(stderr, "error offlining memory: %s\n", + strerror(-rc)); + + fprintf(stderr, "offlined memory for %d device%s\n", processed, + processed == 1 ? "" : "s"); + return rc; +} From patchwork Fri Jun 28 19:11:07 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023359 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1D1C6138D for ; Fri, 28 Jun 2019 19:11:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0CB7728833 for ; Fri, 28 Jun 2019 19:11:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 0131228837; Fri, 28 Jun 2019 19:11:31 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 7D73D28833 for ; Fri, 28 Jun 2019 19:11:31 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 82F9F212ABA23; Fri, 28 Jun 2019 12:11:25 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id CC5F22194EB76 for ; Fri, 28 Jun 2019 12:11:21 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564756" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:21 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 10/13] Documentation: Add man pages for daxctl-{on, off}line-memory Date: Fri, 28 Jun 2019 13:11:07 -0600 Message-Id: <20190628191110.21428-11-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add man pages for the two new commands: daxctl-online-memory, and daxctl-offline-memory. Cc: Dan Williams Cc: Dave Hansen Signed-off-by: Vishal Verma --- Documentation/daxctl/Makefile.am | 4 +- .../daxctl/daxctl-offline-memory.txt | 72 +++++++++++++++++ Documentation/daxctl/daxctl-online-memory.txt | 80 +++++++++++++++++++ 3 files changed, 155 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-offline-memory.txt create mode 100644 Documentation/daxctl/daxctl-online-memory.txt diff --git a/Documentation/daxctl/Makefile.am b/Documentation/daxctl/Makefile.am index 715fbad..37c3bde 100644 --- a/Documentation/daxctl/Makefile.am +++ b/Documentation/daxctl/Makefile.am @@ -29,7 +29,9 @@ man1_MANS = \ daxctl.1 \ daxctl-list.1 \ daxctl-migrate-device-model.1 \ - daxctl-reconfigure-device.1 + daxctl-reconfigure-device.1 \ + daxctl-online-memory.1 \ + daxctl-offline-memory.1 CLEANFILES = $(man1_MANS) diff --git a/Documentation/daxctl/daxctl-offline-memory.txt b/Documentation/daxctl/daxctl-offline-memory.txt new file mode 100644 index 0000000..ba06287 --- /dev/null +++ b/Documentation/daxctl/daxctl-offline-memory.txt @@ -0,0 +1,72 @@ +// SPDX-License-Identifier: GPL-2.0 + +daxctl-offline-memory(1) +======================== + +NAME +---- +daxctl-offline-memory - Offline the memory for a device that is in system-ram mode + +SYNOPSIS +-------- +[verse] +'daxctl offline-memory' [...] [] + +EXAMPLES +-------- + +* Reconfigure dax0.0 to system-ram mode +---- +# daxctl reconfigure-device --mode=system-ram --human dax0.0 +{ + "chardev":"dax0.0", + "size":"7.87 GiB (8.45 GB)", + "target_node":2, + "mode":"system-ram" +} +---- + +* Offline the memory +---- +# daxctl offline-memory dax0.0 +dax0.0: 62 sections offlined +offlined memory for 1 device +---- + +DESCRIPTION +----------- + +Offline the memory sections associated with a device that has been converted +to the system-ram mode. If one or more blocks are already offline, attempt to +offline the remaining blocks. If all blocks were already offline, print a +message and return success without actually doing anything. + +This is complementary to the 'daxctl-online-memory' command, and may be used +when it is wished to offline the memory sections, but not convert the device +back to 'devdax' mode. + +OPTIONS +------- +-r:: +--region=:: + Restrict the operation to devices belonging to the specified region(s). + A device-dax region is a contiguous range of memory that hosts one or + more /dev/daxX.Y devices, where X is the region id and Y is the device + instance id. + +-u:: +--human:: + By default the command will output machine-friendly raw-integer + data. Instead, with this flag, numbers representing storage size + will be formatted as human readable strings with units, other + fields are converted to hexadecimal strings. + +-v:: +--verbose:: + Emit more debug messages + +include::../copyright.txt[] + +SEE ALSO +-------- +linkdaxctl:daxctl-reconfigure-device[1],daxctl-online-memory[1] diff --git a/Documentation/daxctl/daxctl-online-memory.txt b/Documentation/daxctl/daxctl-online-memory.txt new file mode 100644 index 0000000..5ac1cbf --- /dev/null +++ b/Documentation/daxctl/daxctl-online-memory.txt @@ -0,0 +1,80 @@ +// SPDX-License-Identifier: GPL-2.0 + +daxctl-online-memory(1) +======================= + +NAME +---- +daxctl-online-memory - Online the memory for a device that is in system-ram mode + +SYNOPSIS +-------- +[verse] +'daxctl online-memory' [...] [] + +EXAMPLES +-------- + +* Reconfigure dax0.0 to system-ram mode, don't online the memory +---- +# daxctl reconfigure-device --mode=system-ram --no-online --human dax0.0 +{ + "chardev":"dax0.0", + "size":"7.87 GiB (8.45 GB)", + "target_node":2, + "mode":"system-ram" +} +---- + +* Online the memory separately +---- +# daxctl online-memory dax0.0 +dax0.0: 62 new sections onlined +onlined memory for 1 device +---- + +* Onlining memory when some sections were already online +---- +# daxctl online-memory dax0.0 +dax0.0: 1 section already online +dax0.0: 61 new sections onlined +onlined memory for 1 device +---- + +DESCRIPTION +----------- + +Online the memory sections associated with a device that has been converted +to the system-ram mode. If one or more blocks are already online, print a +message about them, and attempt to online the remaining blocks. + +This is complementary to the 'daxctl-reconfigure-device' command, when used with +the '--no-online' option to skip onlining memory sections immediately after the +reconfigure. In these scenarios, the memory can be onlined at a later time using +'daxctl-online-memory'. + +OPTIONS +------- +-r:: +--region=:: + Restrict the operation to devices belonging to the specified region(s). + A device-dax region is a contiguous range of memory that hosts one or + more /dev/daxX.Y devices, where X is the region id and Y is the device + instance id. + +-u:: +--human:: + By default the command will output machine-friendly raw-integer + data. Instead, with this flag, numbers representing storage size + will be formatted as human readable strings with units, other + fields are converted to hexadecimal strings. + +-v:: +--verbose:: + Emit more debug messages + +include::../copyright.txt[] + +SEE ALSO +-------- +linkdaxctl:daxctl-reconfigure-device[1],daxctl-offline-memory[1] From patchwork Fri Jun 28 19:11:08 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023361 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 3D4F6138B for ; Fri, 28 Jun 2019 19:11:33 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 2E29728833 for ; Fri, 28 Jun 2019 19:11:33 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 22AA828837; Fri, 28 Jun 2019 19:11:33 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DB49F28833 for ; Fri, 28 Jun 2019 19:11:32 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id D19D7219493E6; Fri, 28 Jun 2019 12:11:25 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 4A6402129EB8E for ; Fri, 28 Jun 2019 12:11:22 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564759" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:21 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 11/13] contrib/ndctl: fix region-id completions for daxctl Date: Fri, 28 Jun 2019 13:11:08 -0600 Message-Id: <20190628191110.21428-12-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP The completion helpers for daxctl assumed the region arguments for specifying daxctl regions were the same as ndctl regions, i.e. "regionX". This is not true - daxctl region arguments are a simple numeric 'id'. Add a new helper __daxctl_get_regions() to complete daxctl region IDs properly. While at it, fix a useless use of 'echo' in __daxctl_get_devs() and quoting in __daxctl_comp_options() Fixes: d6790a32f32c ("daxctl: Add bash-completion") Signed-off-by: Vishal Verma --- contrib/ndctl | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/contrib/ndctl b/contrib/ndctl index 396a344..cacee2d 100755 --- a/contrib/ndctl +++ b/contrib/ndctl @@ -531,8 +531,14 @@ _ndctl() __daxctl_get_devs() { - local opts="--devices $*" - echo "$(daxctl list $opts | grep -E "^\s*\"chardev\":" | cut -d\" -f4)" + local opts=("--devices" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"chardev\":" | cut -d'"' -f4 +} + +__daxctl_get_regions() +{ + local opts=("--regions" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"id\":" | grep -Eo "[0-9]+" } __daxctlcomp() @@ -561,10 +567,10 @@ __daxctl_comp_options() local cur_arg=${cur##*=} case $cur_subopt in --region) - opts=$(__ndctl_get_regions -i) + opts="$(__daxctl_get_regions -i)" ;; --dev) - opts=$(__daxctl_get_devs -i) + opts="$(__daxctl_get_devs -i)" ;; *) return From patchwork Fri Jun 28 19:11:09 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023363 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9AB17138B for ; Fri, 28 Jun 2019 19:11:34 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 8BE6328833 for ; Fri, 28 Jun 2019 19:11:34 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 809D628837; Fri, 28 Jun 2019 19:11:34 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4788B28833 for ; Fri, 28 Jun 2019 19:11:34 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 13D6621945DE6; Fri, 28 Jun 2019 12:11:26 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id C7D712129EB9B for ; Fri, 28 Jun 2019 12:11:22 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564765" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:22 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 12/13] contrib/ndctl: add bash-completion for the new daxctl commands Date: Fri, 28 Jun 2019 13:11:09 -0600 Message-Id: <20190628191110.21428-13-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add bash completion helpers for the new daxctl-reconfigure-device, daxctl-online-memory, and daxctl-offline-memory commands. Cc: Dan Williams Signed-off-by: Vishal Verma --- contrib/ndctl | 24 +++++++++++++++++++++--- 1 file changed, 21 insertions(+), 3 deletions(-) diff --git a/contrib/ndctl b/contrib/ndctl index cacee2d..680fe6a 100755 --- a/contrib/ndctl +++ b/contrib/ndctl @@ -547,7 +547,7 @@ __daxctlcomp() COMPREPLY=( $( compgen -W "$1" -- "$2" ) ) for cword in "${COMPREPLY[@]}"; do - if [[ "$cword" == @(--region|--dev) ]]; then + if [[ "$cword" == @(--region|--dev|--mode) ]]; then COMPREPLY[$i]="${cword}=" else COMPREPLY[$i]="${cword} " @@ -572,6 +572,9 @@ __daxctl_comp_options() --dev) opts="$(__daxctl_get_devs -i)" ;; + --mode) + opts="system-ram devdax" + ;; *) return ;; @@ -582,8 +585,23 @@ __daxctl_comp_options() __daxctl_comp_non_option_args() { - # there aren't any commands that accept non option arguments yet - return + local subcmd=$1 + local cur=$2 + local opts + + case $subcmd in + reconfigure-device) + ;& + online-memory) + ;& + offline-memory) + opts="$(__daxctl_get_devs -i) all" + ;; + *) + return + ;; + esac + __daxctlcomp "$opts" "$cur" } __daxctl_main() From patchwork Fri Jun 28 19:11:10 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 11023365 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 51F2A138D for ; Fri, 28 Jun 2019 19:11:36 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 40AD128833 for ; Fri, 28 Jun 2019 19:11:36 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 3535328837; Fri, 28 Jun 2019 19:11:36 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C5F3928833 for ; Fri, 28 Jun 2019 19:11:35 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 4E20121BADAB2; Fri, 28 Jun 2019 12:11:26 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=192.55.52.43; helo=mga05.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 3A857212AB4FE for ; Fri, 28 Jun 2019 12:11:23 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Jun 2019 12:11:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.63,428,1557212400"; d="scan'208";a="173564769" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by orsmga002.jf.intel.com with ESMTP; 28 Jun 2019 12:11:22 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v5 13/13] test: Add a unit test for daxctl-reconfigure-device and friends Date: Fri, 28 Jun 2019 13:11:10 -0600 Message-Id: <20190628191110.21428-14-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190628191110.21428-1-vishal.l.verma@intel.com> References: <20190628191110.21428-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add a new unit test to test dax device reconfiguration and memory operations. This teaches test/common about daxctl, and adds an ACPI.NFIT bus variable. Since we have to operate on the ACPI.NFIT bus, the test is marked as destructive. Cc: Dan Williams Signed-off-by: Vishal Verma --- test/Makefile.am | 3 +- test/common | 19 ++++++++-- test/daxctl-devices.sh | 81 ++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 99 insertions(+), 4 deletions(-) create mode 100755 test/daxctl-devices.sh diff --git a/test/Makefile.am b/test/Makefile.am index 874c4bb..84474d0 100644 --- a/test/Makefile.am +++ b/test/Makefile.am @@ -49,7 +49,8 @@ TESTS +=\ dax.sh \ device-dax \ device-dax-fio.sh \ - mmap.sh + mmap.sh \ + daxctl-devices.sh if ENABLE_KEYUTILS TESTS += security.sh diff --git a/test/common b/test/common index 1b9d3da..1814a0c 100644 --- a/test/common +++ b/test/common @@ -15,12 +15,25 @@ else exit 1 fi -# NFIT_TEST_BUS[01] +# DAXCTL # -NFIT_TEST_BUS0=nfit_test.0 -NFIT_TEST_BUS1=nfit_test.1 +if [ -f "../daxctl/daxctl" ] && [ -x "../daxctl/daxctl" ]; then + export DAXCTL=../daxctl/daxctl +elif [ -f "./daxctl/daxctl" ] && [ -x "./daxctl/daxctl" ]; then + export DAXCTL=./daxctl/daxctl +else + echo "Couldn't find an daxctl binary" + exit 1 +fi +# NFIT_TEST_BUS[01] +# +NFIT_TEST_BUS0="nfit_test.0" +NFIT_TEST_BUS1="nfit_test.1" +ACPI_BUS="ACPI.NFIT" +E820_BUS="e820" + # Functions # err diff --git a/test/daxctl-devices.sh b/test/daxctl-devices.sh new file mode 100755 index 0000000..cfd9362 --- /dev/null +++ b/test/daxctl-devices.sh @@ -0,0 +1,81 @@ +#!/bin/bash -Ex +# SPDX-License-Identifier: GPL-2.0 +# Copyright(c) 2019 Intel Corporation. All rights reserved. + +rc=77 +. ./common + +trap 'cleanup $LINENO' ERR + +cleanup() +{ + printf "Error at line %d\n" "$1" + [[ $testdev ]] && reset_dev + exit $rc +} + +find_testdev() +{ + local rc=77 + + # find a victim device + testbus="$ACPI_BUS" + testdev=$("$NDCTL" list -b "$testbus" -Ni | jq -er '.[0].dev | .//""') + if [[ ! $testdev ]]; then + printf "Unable to find a victim device\n" + exit "$rc" + fi + printf "Found victim dev: %s on bus: %s\n" "$testdev" "$testbus" +} + +setup_dev() +{ + test -n "$testbus" + test -n "$testdev" + + "$NDCTL" destroy-namespace -f -b "$testbus" "$testdev" + testdev=$("$NDCTL" create-namespace -b "$testbus" -m devdax -fe "$testdev" -s 256M | \ + jq -er '.dev') + test -n "$testdev" +} + +reset_dev() +{ + "$NDCTL" destroy-namespace -f -b "$testbus" "$testdev" +} + +daxctl_get_dev() +{ + "$NDCTL" list -n "$1" -X | jq -er '.[].daxregion.devices[0].chardev' +} + +daxctl_get_mode() +{ + "$DAXCTL" list -d "$1" | jq -er '.[].mode' +} + +daxctl_test() +{ + local daxdev + + daxdev=$(daxctl_get_dev "$testdev") + test -n "$daxdev" + + "$DAXCTL" reconfigure-device -N -m system-ram "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "system-ram" ]] + "$DAXCTL" online-memory "$daxdev" + "$DAXCTL" offline-memory "$daxdev" + "$DAXCTL" reconfigure-device -m devdax "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "devdax" ]] + "$DAXCTL" reconfigure-device -m system-ram "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "system-ram" ]] + "$DAXCTL" reconfigure-device -O -m devdax "$daxdev" + [[ $(daxctl_get_mode "$daxdev") == "devdax" ]] +} + +find_testdev +setup_dev +rc=1 +daxctl_test +reset_dev +exit 0