From patchwork Wed May 8 00:38:42 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934025 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 993C61515 for ; Wed, 8 May 2019 00:39:21 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 86D1A286EC for ; Wed, 8 May 2019 00:39:21 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 79C652873B; Wed, 8 May 2019 00:39:21 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 27613286EC for ; Wed, 8 May 2019 00:39:21 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 07BF22124B939; Tue, 7 May 2019 17:39:21 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 10731211EDB5C for ; Tue, 7 May 2019 17:39:19 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427666" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:18 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 01/10] libdaxctl: add interfaces in support of device modes Date: Tue, 7 May 2019 18:38:42 -0600 Message-Id: <20190508003851.32416-2-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP In preparation for libdaxctl and daxctl to grow operational modes for DAX devices, add the following supporting APIs: daxctl_dev_get_ctx daxctl_dev_is_enabled Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl.c | 30 ++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 6 ++++++ daxctl/libdaxctl.h | 2 ++ 3 files changed, 38 insertions(+) diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index c2e3a52..70f896b 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -559,6 +559,36 @@ static void dax_regions_init(struct daxctl_ctx *ctx) } } +static int is_enabled(const char *drvpath) +{ + struct stat st; + + if (lstat(drvpath, &st) < 0 || !S_ISLNK(st.st_mode)) + return 0; + else + return 1; +} + +DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) +{ + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + char *path = dev->dev_buf; + int len = dev->buf_len; + + if (snprintf(path, len, "%s/driver", dev->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", + daxctl_dev_get_devname(dev)); + return 0; + } + + return is_enabled(path); +} + +DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) +{ + return dev->region->ctx; +} + DAXCTL_EXPORT struct daxctl_dev *daxctl_dev_get_first(struct daxctl_region *region) { dax_devices_init(region); diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 84d3a69..c4af9a7 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -50,3 +50,9 @@ LIBDAXCTL_5 { global: daxctl_region_get_path; } LIBDAXCTL_4; + +LIBDAXCTL_6 { +global: + daxctl_dev_get_ctx; + daxctl_dev_is_enabled; +} LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index 1d13ea2..e20ccb4 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -67,6 +67,8 @@ const char *daxctl_dev_get_devname(struct daxctl_dev *dev); int daxctl_dev_get_major(struct daxctl_dev *dev); int daxctl_dev_get_minor(struct daxctl_dev *dev); unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev); +struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev); +int daxctl_dev_is_enabled(struct daxctl_dev *dev); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ From patchwork Wed May 8 00:38:43 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934029 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 46E9516C1 for ; Wed, 8 May 2019 00:39:23 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 357CA286EC for ; Wed, 8 May 2019 00:39:23 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 299AA28715; Wed, 8 May 2019 00:39:23 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id C0FDC2873B for ; Wed, 8 May 2019 00:39:22 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 7FC362194D3AE; Tue, 7 May 2019 17:39:22 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 3DE88212449EF for ; Tue, 7 May 2019 17:39:20 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:19 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427670" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:19 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 02/10] libdaxctl: cache 'subsystem' in daxctl_ctx Date: Tue, 7 May 2019 18:38:43 -0600 Message-Id: <20190508003851.32416-3-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP The 'DAX subsystem' in effect is determined at region or device init time, and dictates the sysfs base paths for all device/region operations. In preparation for adding bind/unbind functionality, cache the subsystem as determined at init time in the library context. Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl.c | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index 70f896b..f8f5b8c 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -46,6 +46,7 @@ struct daxctl_ctx { void *userdata; int regions_init; struct list_head regions; + enum dax_subsystem subsys; }; /** @@ -96,6 +97,7 @@ DAXCTL_EXPORT int daxctl_new(struct daxctl_ctx **ctx) dbg(c, "log_priority=%d\n", c->ctx.log_priority); *ctx = c; list_head_init(&c->regions); + c->subsys = DAX_UNKNOWN; return 0; } @@ -454,14 +456,18 @@ static void dax_devices_init(struct daxctl_region *region) for (i = 0; i < ARRAY_SIZE(dax_subsystems); i++) { char *region_path; - if (i == DAX_BUS) + if (i == DAX_BUS) { region_path = region->region_path; - else if (i == DAX_CLASS) { + if (ctx->subsys == DAX_UNKNOWN) + ctx->subsys = DAX_BUS; + } else if (i == DAX_CLASS) { if (asprintf(®ion_path, "%s/dax", region->region_path) < 0) { dbg(ctx, "region path alloc fail\n"); continue; } + if (ctx->subsys == DAX_UNKNOWN) + ctx->subsys = DAX_CLASS; } else continue; sysfs_device_parse(ctx, region_path, daxdev_fmt, region, @@ -539,6 +545,8 @@ static void __dax_regions_init(struct daxctl_ctx *ctx, enum dax_subsystem subsys free(dev_path); if (!region) err(ctx, "add_dax_region() for %s failed\n", de->d_name); + if (ctx->subsys == DAX_UNKNOWN) + ctx->subsys = subsys; } closedir(dir); } From patchwork Wed May 8 00:38:44 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934031 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 8D7A31390 for ; Wed, 8 May 2019 00:39:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 7B847286EC for ; Wed, 8 May 2019 00:39:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 62EEE28751; Wed, 8 May 2019 00:39:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 9A6C0286EC for ; Wed, 8 May 2019 00:39:24 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B811421255843; Tue, 7 May 2019 17:39:22 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 5833A212449F0 for ; Tue, 7 May 2019 17:39:20 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427674" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:19 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 03/10] libdaxctl: add interfaces to enable/disable devices Date: Tue, 7 May 2019 18:38:44 -0600 Message-Id: <20190508003851.32416-4-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add new libdaxctl interfaces to disable a device_dax based devices, and to enable it into a given mode. The modes available are 'device_dax', and 'system-ram', where device_dax is the normal device DAX mode used via a character device, and 'system-ram' uses the kernel's 'kmem' facility to hotplug the device into the system's memory space, and can be used as normal system memory. This adds the following new interfaces: daxctl_dev_disable; daxctl_dev_enable_devdax; daxctl_dev_enable_ram; Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/Makefile.am | 3 +- daxctl/lib/libdaxctl-private.h | 15 ++ daxctl/lib/libdaxctl.c | 241 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 3 + daxctl/libdaxctl.h | 9 ++ 5 files changed, 270 insertions(+), 1 deletion(-) diff --git a/daxctl/lib/Makefile.am b/daxctl/lib/Makefile.am index d3d4852..9f0e444 100644 --- a/daxctl/lib/Makefile.am +++ b/daxctl/lib/Makefile.am @@ -16,7 +16,8 @@ libdaxctl_la_SOURCES =\ libdaxctl.c libdaxctl_la_LIBADD =\ - $(UUID_LIBS) + $(UUID_LIBS) \ + $(KMOD_LIBS) daxctl_modprobe_data_DATA = daxctl.conf diff --git a/daxctl/lib/libdaxctl-private.h b/daxctl/lib/libdaxctl-private.h index 4a462e7..e64d0a7 100644 --- a/daxctl/lib/libdaxctl-private.h +++ b/daxctl/lib/libdaxctl-private.h @@ -13,6 +13,8 @@ #ifndef _LIBDAXCTL_PRIVATE_H_ #define _LIBDAXCTL_PRIVATE_H_ +#include + #define DAXCTL_EXPORT __attribute__ ((visibility("default"))) enum dax_subsystem { @@ -26,6 +28,11 @@ static const char *dax_subsystems[] = { [DAX_BUS] = "/sys/bus/dax/devices", }; +static const char *dax_modules[] = { + [DAXCTL_DEV_MODE_DEVDAX] = "device_dax", + [DAXCTL_DEV_MODE_RAM] = "kmem", +}; + /** * struct daxctl_region - container for dax_devices */ @@ -53,6 +60,14 @@ struct daxctl_dev { char *dev_path; struct list_node list; unsigned long long size; + struct kmod_module *module; + struct kmod_list *kmod_list; struct daxctl_region *region; }; + +static inline int check_kmod(struct kmod_ctx *kmod_ctx) +{ + return kmod_ctx ? 0 : -ENXIO; +} + #endif /* _LIBDAXCTL_PRIVATE_H_ */ diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index f8f5b8c..d50b321 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -47,6 +47,7 @@ struct daxctl_ctx { int regions_init; struct list_head regions; enum dax_subsystem subsys; + struct kmod_ctx *kmod_ctx; }; /** @@ -85,12 +86,20 @@ DAXCTL_EXPORT void daxctl_set_userdata(struct daxctl_ctx *ctx, void *userdata) */ DAXCTL_EXPORT int daxctl_new(struct daxctl_ctx **ctx) { + struct kmod_ctx *kmod_ctx; struct daxctl_ctx *c; + int rc = 0; c = calloc(1, sizeof(struct daxctl_ctx)); if (!c) return -ENOMEM; + kmod_ctx = kmod_new(NULL, NULL); + if (check_kmod(kmod_ctx) != 0) { + rc = -ENXIO; + goto out; + } + c->refcount = 1; log_init(&c->ctx, "libdaxctl", "DAXCTL_LOG"); info(c, "ctx %p created\n", c); @@ -98,8 +107,12 @@ DAXCTL_EXPORT int daxctl_new(struct daxctl_ctx **ctx) *ctx = c; list_head_init(&c->regions); c->subsys = DAX_UNKNOWN; + c->kmod_ctx = kmod_ctx; return 0; +out: + free(c); + return rc; } /** @@ -134,6 +147,7 @@ DAXCTL_EXPORT void daxctl_unref(struct daxctl_ctx *ctx) list_for_each_safe(&ctx->regions, region, _r, list) free_region(region, &ctx->regions); + kmod_unref(ctx->kmod_ctx); info(ctx, "context %p released\n", ctx); free(ctx); } @@ -191,6 +205,7 @@ static void free_dev(struct daxctl_dev *dev, struct list_head *head) { if (head) list_del_from(head, &dev->list); + kmod_module_unref_list(dev->kmod_list); free(dev->dev_buf); free(dev->dev_path); free(dev); @@ -308,6 +323,27 @@ DAXCTL_EXPORT struct daxctl_region *daxctl_new_region(struct daxctl_ctx *ctx, return region; } +static struct kmod_list *to_module_list(struct daxctl_ctx *ctx, + const char *alias) +{ + struct kmod_list *list = NULL; + int rc; + + if (!ctx->kmod_ctx || !alias) + return NULL; + if (alias[0] == 0) + return NULL; + + rc = kmod_module_new_from_lookup(ctx->kmod_ctx, alias, &list); + if (rc < 0 || !list) { + dbg(ctx, "failed to find modules for alias: %s %d list: %s\n", + alias, rc, list ? "populated" : "empty"); + return NULL; + } + + return list; +} + static void *add_dax_dev(void *parent, int id, const char *daxdev_base) { const char *devname = devpath_to_devname(daxdev_base); @@ -317,6 +353,7 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) struct daxctl_dev *dev, *dev_dup; char buf[SYSFS_ATTR_SIZE]; struct stat st; + int rc; if (!path) return NULL; @@ -348,6 +385,18 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) goto err_read; dev->buf_len = strlen(daxdev_base) + 50; + sprintf(path, "%s/modalias", daxdev_base); + rc = sysfs_read_attr(ctx, path, buf); + /* older kernels may be lack the modalias attribute */ + if (rc < 0 && rc != -ENOENT) + goto err_read; + if (rc == 0) { + dev->kmod_list = to_module_list(ctx, buf); + if (dev->kmod_list == NULL) + goto err_read; + } else + dbg(ctx, "%s: modalias attribute missing\n", devname); + daxctl_dev_foreach(region, dev_dup) if (dev_dup->id == dev->id) { free_dev(dev, NULL); @@ -577,6 +626,83 @@ static int is_enabled(const char *drvpath) return 1; } +static int daxctl_bind(struct daxctl_ctx *ctx, const char *devname, + const char *mod_name) +{ + DIR *dir; + int rc = 0; + char path[200]; + struct dirent *de; + const int len = sizeof(path); + + if (!devname) { + err(ctx, "missing devname\n"); + return -EINVAL; + } + + /* this is only called by daxctl_dev_enable which gates on dax-bus */ + if (snprintf(path, len, "/sys/bus/dax/drivers") >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + dir = opendir(path); + if (!dir) { + err(ctx, "%s: opendir(\"%s\") failed\n", devname, path); + return -ENXIO; + } + + while ((de = readdir(dir)) != NULL) { + char *drv_path; + + if (de->d_ino == 0) + continue; + if (de->d_name[0] == '.') + continue; + if (strcmp(de->d_name, mod_name) != 0) + continue; + + if (asprintf(&drv_path, "%s/%s/new_id", path, de->d_name) < 0) { + err(ctx, "%s: path allocation failure\n", devname); + rc = -ENOMEM; + break; + } + rc = sysfs_write_attr_quiet(ctx, drv_path, devname); + free(drv_path); + + if (asprintf(&drv_path, "%s/%s/bind", path, de->d_name) < 0) { + err(ctx, "%s: path allocation failure\n", devname); + rc = -ENOMEM; + break; + } + rc = sysfs_write_attr_quiet(ctx, drv_path, devname); + free(drv_path); + break; + } + closedir(dir); + + if (rc) { + dbg(ctx, "%s: bind failed\n", devname); + return rc; + } + return 0; +} + +static int daxctl_unbind(struct daxctl_ctx *ctx, const char *devpath) +{ + const char *devname = devpath_to_devname(devpath); + char path[200]; + const int len = sizeof(path); + + /* this is only called by daxctl_dev_disable which gates on dax-bus */ + if (snprintf(path, len, "%s/driver/unbind", devpath) >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + return sysfs_write_attr(ctx, path, devname); +} + DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) { struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); @@ -592,6 +718,121 @@ DAXCTL_EXPORT int daxctl_dev_is_enabled(struct daxctl_dev *dev) return is_enabled(path); } +static int daxctl_insert_kmod_for_mode(struct daxctl_dev *dev, + const char *mod_name) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + struct kmod_list *iter; + int rc = -ENXIO; + + if (dev->kmod_list == NULL) { + err(ctx, "%s: a modalias lookup list was not created\n", + devname); + return rc; + } + + kmod_list_foreach(iter, dev->kmod_list) { + struct kmod_module *mod = kmod_module_get_module(iter); + const char *name = kmod_module_get_name(mod); + + if (strcmp(name, mod_name) != 0) { + kmod_module_unref(mod); + continue; + } + dbg(ctx, "%s inserting module: %s\n", devname, name); + rc = kmod_module_probe_insert_module(mod, + KMOD_PROBE_APPLY_BLACKLIST, NULL, NULL, NULL, + NULL); + if (rc < 0) { + err(ctx, "%s: insert failure: %d\n", devname, rc); + return rc; + } + dev->module = mod; + } + + if (rc == -ENXIO) + err(ctx, "%s: Unable to find module: %s in alias list\n", + devname, mod_name); + return rc; +} + +static int daxctl_dev_enable(struct daxctl_dev *dev, enum daxctl_dev_mode mode) +{ + struct daxctl_region *region = daxctl_dev_get_region(dev); + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *mod_name = dax_modules[mode]; + int rc; + + if (daxctl_dev_is_enabled(dev)) + return 0; + + if (ctx->subsys != DAX_BUS) { + err(ctx, "%s: invalid operation for dax subsystem\n", devname); + err(ctx, "%s: see daxctl-migrate-device-model(1)\n", devname); + return -ENXIO; + } + + if (mode == DAXCTL_DEV_MODE_UNKNOWN || mod_name == NULL) { + err(ctx, "%s: Invalid mode: %d\n", devname, mode); + return -EINVAL; + } + + rc = daxctl_insert_kmod_for_mode(dev, mod_name); + if (rc) + return rc; + + rc = daxctl_bind(ctx, devname, mod_name); + if (!daxctl_dev_is_enabled(dev)) { + err(ctx, "%s: failed to enable\n", devname); + return rc ? rc : -ENXIO; + } + + region->devices_init = 0; + dax_devices_init(region); + rc = 0; + dbg(ctx, "%s: enabled\n", devname); + return rc; +} + +DAXCTL_EXPORT int daxctl_dev_enable_devdax(struct daxctl_dev *dev) +{ + return daxctl_dev_enable(dev, DAXCTL_DEV_MODE_DEVDAX); +} + +DAXCTL_EXPORT int daxctl_dev_enable_ram(struct daxctl_dev *dev) +{ + return daxctl_dev_enable(dev, DAXCTL_DEV_MODE_RAM); +} + +DAXCTL_EXPORT int daxctl_dev_disable(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + + if (!daxctl_dev_is_enabled(dev)) + return 0; + + if (ctx->subsys != DAX_BUS) { + err(ctx, "%s: invalid operation for dax subsystem\n", devname); + err(ctx, "%s: see daxctl-migrate-device-model(1)\n", devname); + return -ENXIO; + } + + daxctl_unbind(ctx, dev->dev_path); + + if (daxctl_dev_is_enabled(dev)) { + err(ctx, "%s: failed to disable\n", devname); + return -EBUSY; + } + + kmod_module_unref(dev->module); + dbg(ctx, "%s: disabled\n", devname); + + return 0; +} + DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) { return dev->region->ctx; diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index c4af9a7..19904a2 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -55,4 +55,7 @@ LIBDAXCTL_6 { global: daxctl_dev_get_ctx; daxctl_dev_is_enabled; + daxctl_dev_disable; + daxctl_dev_enable_devdax; + daxctl_dev_enable_ram; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index e20ccb4..b80488e 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -69,6 +69,9 @@ int daxctl_dev_get_minor(struct daxctl_dev *dev); unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev); struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev); int daxctl_dev_is_enabled(struct daxctl_dev *dev); +int daxctl_dev_disable(struct daxctl_dev *dev); +int daxctl_dev_enable_devdax(struct daxctl_dev *dev); +int daxctl_dev_enable_ram(struct daxctl_dev *dev); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ @@ -81,6 +84,12 @@ int daxctl_dev_is_enabled(struct daxctl_dev *dev); region != NULL; \ region = daxctl_region_get_next(region)) +enum daxctl_dev_mode { + DAXCTL_DEV_MODE_UNKNOWN, + DAXCTL_DEV_MODE_RAM, + DAXCTL_DEV_MODE_DEVDAX, +}; + #ifdef __cplusplus } /* extern "C" */ #endif From patchwork Wed May 8 00:38:45 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934033 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AD5FB1515 for ; Wed, 8 May 2019 00:39:25 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99F23286EC for ; Wed, 8 May 2019 00:39:25 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 848CC28715; Wed, 8 May 2019 00:39:25 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id DCB4428715 for ; Wed, 8 May 2019 00:39:24 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 01C1521945DDF; Tue, 7 May 2019 17:39:23 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 8D937212449F1 for ; Tue, 7 May 2019 17:39:20 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427677" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:20 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 04/10] libdaxctl: add interfaces to get/set the online state for a node Date: Tue, 7 May 2019 18:38:45 -0600 Message-Id: <20190508003851.32416-5-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP In preparation for converting DAX devices for use as system-ram via the kernel's 'kmem' facility, add libndctl helpers to manipulate the sysfs interfaces to get the target_node of a DAX device, and to online, offline, and query the state of hotplugged memory sections associated with a given node. This adds the following new interfaces: daxctl_dev_get_target_node daxctl_dev_online_node daxctl_dev_offline_node daxctl_dev_node_is_online Cc: Pavel Tatashin Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl-private.h | 6 + daxctl/lib/libdaxctl.c | 228 +++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 4 + daxctl/libdaxctl.h | 4 + 4 files changed, 242 insertions(+) diff --git a/daxctl/lib/libdaxctl-private.h b/daxctl/lib/libdaxctl-private.h index e64d0a7..ef443aa 100644 --- a/daxctl/lib/libdaxctl-private.h +++ b/daxctl/lib/libdaxctl-private.h @@ -33,6 +33,11 @@ static const char *dax_modules[] = { [DAXCTL_DEV_MODE_RAM] = "kmem", }; +enum node_state { + NODE_OFFLINE, + NODE_ONLINE, +}; + /** * struct daxctl_region - container for dax_devices */ @@ -63,6 +68,7 @@ struct daxctl_dev { struct kmod_module *module; struct kmod_list *kmod_list; struct daxctl_region *region; + int target_node; }; static inline int check_kmod(struct kmod_ctx *kmod_ctx) diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index d50b321..aab2364 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -28,6 +28,7 @@ #include "libdaxctl-private.h" static const char *attrs = "dax_region"; +static const char *node_base = "/sys/devices/system/node"; static void free_region(struct daxctl_region *region, struct list_head *head); @@ -397,6 +398,12 @@ static void *add_dax_dev(void *parent, int id, const char *daxdev_base) } else dbg(ctx, "%s: modalias attribute missing\n", devname); + sprintf(path, "%s/target_node", daxdev_base); + if (sysfs_read_attr(ctx, path, buf) == 0) + dev->target_node = strtol(buf, NULL, 0); + else + dev->target_node = -1; + daxctl_dev_foreach(region, dev_dup) if (dev_dup->id == dev->id) { free_dev(dev, NULL); @@ -897,3 +904,224 @@ DAXCTL_EXPORT unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev) { return dev->size; } + +DAXCTL_EXPORT int daxctl_dev_get_target_node(struct daxctl_dev *dev) +{ + return dev->target_node; +} + +static int online_one_memblock(struct daxctl_dev *dev, char *path) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *mode = "online_movable"; + char buf[SYSFS_ATTR_SIZE]; + int rc; + + rc = sysfs_read_attr(ctx, path, buf); + if (rc) { + err(ctx, "%s: Failed to read %s: %s\n", + devname, path, strerror(-rc)); + return rc; + } + + /* + * if already online, possibly due to kernel config or a udev rule, + * there is nothing to do and we can skip over the memblock + */ + if (strncmp(buf, "online", 6) == 0) + return 0; + + rc = sysfs_write_attr_quiet(ctx, path, mode); + if (rc) + err(ctx, "%s: Failed to online %s: %s\n", + devname, path, strerror(-rc)); + return rc; +} + +static int offline_one_memblock(struct daxctl_dev *dev, char *path) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + const char *mode = "offline"; + char buf[SYSFS_ATTR_SIZE]; + int rc; + + rc = sysfs_read_attr(ctx, path, buf); + if (rc) { + err(ctx, "%s: Failed to read %s: %s\n", + devname, path, strerror(-rc)); + return rc; + } + + /* if already offline, there is nothing to do */ + if (strncmp(buf, "offline", 6) == 0) + return 0; + + rc = sysfs_write_attr_quiet(ctx, path, mode); + if (rc) + err(ctx, "%s: Failed to offline %s: %s\n", + devname, path, strerror(-rc)); + return rc; +} + +static int daxctl_dev_node_set_state(struct daxctl_dev *dev, + enum node_state state) +{ + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + int target_node, rc, changed = 0; + struct dirent *de; + char *node_path; + DIR *node_dir; + + target_node = daxctl_dev_get_target_node(dev); + if (target_node < 0) { + err(ctx, "%s: Unable to get target node\n", + daxctl_dev_get_devname(dev)); + return -ENXIO; + } + + rc = asprintf(&node_path, "%s/node%d", node_base, target_node); + if (rc < 0) + return -ENOMEM; + + node_dir = opendir(node_path); + if (!node_dir) { + rc = -errno; + goto out_path; + } + + errno = 0; + while ((de = readdir(node_dir)) != NULL) { + char *mem_path; + + if (strcmp(de->d_name, ".") == 0 || + strcmp(de->d_name, "..") == 0) + continue; + if (strncmp(de->d_name, "memory", 6) == 0) { + rc = asprintf(&mem_path, "%s/%s/state", + node_path, de->d_name); + if (rc < 0) { + rc = -ENOMEM; + goto out_dir; + } + if (state == NODE_ONLINE) + rc = online_one_memblock(dev, mem_path); + else if (state == NODE_OFFLINE) + rc = offline_one_memblock(dev, mem_path); + else + rc = -EINVAL; + free(mem_path); + if (rc) + goto out_dir; + changed++; + } + errno = 0; + } + if (errno) { + rc = -errno; + goto out_dir; + } + rc = changed; + +out_dir: + closedir(node_dir); +out_path: + free(node_path); + return rc; +} + +DAXCTL_EXPORT int daxctl_dev_online_node(struct daxctl_dev *dev) +{ + return daxctl_dev_node_set_state(dev, NODE_ONLINE); +} + +DAXCTL_EXPORT int daxctl_dev_offline_node(struct daxctl_dev *dev) +{ + return daxctl_dev_node_set_state(dev, NODE_OFFLINE); +} + +static int memblock_is_online(struct daxctl_dev *dev, char *path) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + char buf[SYSFS_ATTR_SIZE]; + int rc; + + rc = sysfs_read_attr(ctx, path, buf); + if (rc) { + err(ctx, "%s: Failed to read %s: %s\n", + devname, path, strerror(-rc)); + return rc; + } + + if (strncmp(buf, "online", 6) == 0) + return 1; + + return 0; +} + +DAXCTL_EXPORT int daxctl_dev_node_is_online(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + int target_node, rc, num_online = 0; + struct dirent *de; + char *node_path; + DIR *node_dir; + + target_node = daxctl_dev_get_target_node(dev); + if (target_node < 0) { + err(ctx, "%s: Unable to get target node\n", devname); + return -ENXIO; + } + + rc = asprintf(&node_path, "%s/node%d", node_base, target_node); + if (rc < 0) + return -ENOMEM; + + node_dir = opendir(node_path); + if (!node_dir) { + rc = -errno; + goto out_path; + } + + errno = 0; + while ((de = readdir(node_dir)) != NULL) { + char *mem_path; + + if (strcmp(de->d_name, ".") == 0 || + strcmp(de->d_name, "..") == 0) + continue; + if (strncmp(de->d_name, "memory", 6) == 0) { + rc = asprintf(&mem_path, "%s/%s/state", + node_path, de->d_name); + if (rc < 0) { + rc = -ENOMEM; + goto out_dir; + } + rc = memblock_is_online(dev, mem_path); + if (rc < 0) { + err(ctx, "%s: Unable to determine state: %s\n", + devname, mem_path); + goto out_dir; + } + if (rc > 0) + num_online++; + free(mem_path); + } + errno = 0; + } + if (errno) { + rc = -errno; + goto out_dir; + } + rc = num_online; + +out_dir: + closedir(node_dir); +out_path: + free(node_path); + return rc; + +} diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index 19904a2..cc47ed6 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -54,8 +54,12 @@ global: LIBDAXCTL_6 { global: daxctl_dev_get_ctx; + daxctl_dev_get_target_node; daxctl_dev_is_enabled; daxctl_dev_disable; daxctl_dev_enable_devdax; daxctl_dev_enable_ram; + daxctl_dev_online_node; + daxctl_dev_offline_node; + daxctl_dev_node_is_online; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index b80488e..db0d4ea 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -68,10 +68,14 @@ int daxctl_dev_get_major(struct daxctl_dev *dev); int daxctl_dev_get_minor(struct daxctl_dev *dev); unsigned long long daxctl_dev_get_size(struct daxctl_dev *dev); struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev); +int daxctl_dev_get_target_node(struct daxctl_dev *dev); int daxctl_dev_is_enabled(struct daxctl_dev *dev); int daxctl_dev_disable(struct daxctl_dev *dev); int daxctl_dev_enable_devdax(struct daxctl_dev *dev); int daxctl_dev_enable_ram(struct daxctl_dev *dev); +int daxctl_dev_online_node(struct daxctl_dev *dev); +int daxctl_dev_offline_node(struct daxctl_dev *dev); +int daxctl_dev_node_is_online(struct daxctl_dev *dev); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ From patchwork Wed May 8 00:38:46 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934035 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 5CFDA1515 for ; Wed, 8 May 2019 00:39:26 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 49BC3286EC for ; Wed, 8 May 2019 00:39:26 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 356FB2873B; Wed, 8 May 2019 00:39:26 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id EA0EC286EC for ; Wed, 8 May 2019 00:39:25 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 399E521945DF1; Tue, 7 May 2019 17:39:23 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id CF81F21237AD3 for ; Tue, 7 May 2019 17:39:20 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:20 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427680" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:20 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 05/10] daxctl/list: add numa_node for device listings Date: Tue, 7 May 2019 18:38:46 -0600 Message-Id: <20190508003851.32416-6-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP The kernel provides a 'target_node' attribute for dax devices. When converting a dax device to the system-ram mode, the memory is hotplugged into this numa node. It would be helpful to print this in device listings so that it is easy for applications to detect the numa node to which the new memory belongs. Cc: Dan Williams Cc: Dave Hansen Signed-off-by: Vishal Verma --- util/json.c | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/util/json.c b/util/json.c index babdc8c..b7ce719 100644 --- a/util/json.c +++ b/util/json.c @@ -271,6 +271,7 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, { const char *devname = daxctl_dev_get_devname(dev); struct json_object *jdev, *jobj; + int node; jdev = json_object_new_object(); if (!devname || !jdev) @@ -284,6 +285,13 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, if (jobj) json_object_object_add(jdev, "size", jobj); + node = daxctl_dev_get_target_node(dev); + if (node >= 0) { + jobj = json_object_new_int(node); + if (jobj) + json_object_object_add(jdev, "numa_node", jobj); + } + return jdev; } From patchwork Wed May 8 00:38:47 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934037 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id AF3521390 for ; Wed, 8 May 2019 00:39:27 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9DBD5286EC for ; Wed, 8 May 2019 00:39:27 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 91E7D28751; Wed, 8 May 2019 00:39:27 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 24094286EC for ; Wed, 8 May 2019 00:39:27 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 8183A21BADAB2; Tue, 7 May 2019 17:39:23 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 2B04D2194EB70 for ; Tue, 7 May 2019 17:39:21 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427683" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:20 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 06/10] libdaxctl: add an interface to get the mode for a dax device Date: Tue, 7 May 2019 18:38:47 -0600 Message-Id: <20190508003851.32416-7-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP In preparation for a reconfigure-device command, add an interface to retrieve the 'mode' of a dax device. This will allow the reconfigure-device command (and via daxctl_dev_to_json()), also daxctl-list) to print the mode on device listings via a list command or immediately after a mode change. Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/lib/libdaxctl.c | 43 ++++++++++++++++++++++++++++++++++++++++ daxctl/lib/libdaxctl.sym | 1 + daxctl/libdaxctl.h | 1 + util/json.c | 14 +++++++++++++ 4 files changed, 59 insertions(+) diff --git a/daxctl/lib/libdaxctl.c b/daxctl/lib/libdaxctl.c index aab2364..a4919e0 100644 --- a/daxctl/lib/libdaxctl.c +++ b/daxctl/lib/libdaxctl.c @@ -12,6 +12,8 @@ */ #include #include +#include +#include #include #include #include @@ -840,6 +842,47 @@ DAXCTL_EXPORT int daxctl_dev_disable(struct daxctl_dev *dev) return 0; } +DAXCTL_EXPORT enum daxctl_dev_mode daxctl_dev_get_mode(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + struct daxctl_ctx *ctx = daxctl_dev_get_ctx(dev); + int rc = DAXCTL_DEV_MODE_UNKNOWN; + char path[200]; + const int len = sizeof(path); + char *mod_path, *mod_base; + + if (!daxctl_dev_is_enabled(dev)) + return rc; + + if (ctx->subsys != DAX_BUS) { + err(ctx, "%s: invalid operation for dax subsystem\n", devname); + err(ctx, "%s: see daxctl-migrate-device-model(1)\n", devname); + return -ENXIO; + } + + if (snprintf(path, len, "%s/driver/module", dev->dev_path) >= len) { + err(ctx, "%s: buffer too small!\n", devname); + return -ENXIO; + } + + mod_path = realpath(path, NULL); + if (!mod_path) { + rc = -errno; + err(ctx, "%s: unable to determine module: %s\n", devname, + strerror(errno)); + return rc; + } + + mod_base = basename(mod_path); + if (strcmp(mod_base, dax_modules[DAXCTL_DEV_MODE_RAM]) == 0) + rc = DAXCTL_DEV_MODE_RAM; + else if (strcmp(mod_base, dax_modules[DAXCTL_DEV_MODE_DEVDAX]) == 0) + rc = DAXCTL_DEV_MODE_DEVDAX; + + free(mod_path); + return rc; +} + DAXCTL_EXPORT struct daxctl_ctx *daxctl_dev_get_ctx(struct daxctl_dev *dev) { return dev->region->ctx; diff --git a/daxctl/lib/libdaxctl.sym b/daxctl/lib/libdaxctl.sym index cc47ed6..d53976d 100644 --- a/daxctl/lib/libdaxctl.sym +++ b/daxctl/lib/libdaxctl.sym @@ -62,4 +62,5 @@ global: daxctl_dev_online_node; daxctl_dev_offline_node; daxctl_dev_node_is_online; + daxctl_dev_get_mode; } LIBDAXCTL_5; diff --git a/daxctl/libdaxctl.h b/daxctl/libdaxctl.h index db0d4ea..4f9088f 100644 --- a/daxctl/libdaxctl.h +++ b/daxctl/libdaxctl.h @@ -76,6 +76,7 @@ int daxctl_dev_enable_ram(struct daxctl_dev *dev); int daxctl_dev_online_node(struct daxctl_dev *dev); int daxctl_dev_offline_node(struct daxctl_dev *dev); int daxctl_dev_node_is_online(struct daxctl_dev *dev); +enum daxctl_dev_mode daxctl_dev_get_mode(struct daxctl_dev *dev); #define daxctl_dev_foreach(region, dev) \ for (dev = daxctl_dev_get_first(region); \ diff --git a/util/json.c b/util/json.c index b7ce719..4f13222 100644 --- a/util/json.c +++ b/util/json.c @@ -271,6 +271,7 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, { const char *devname = daxctl_dev_get_devname(dev); struct json_object *jdev, *jobj; + enum daxctl_dev_mode mode; int node; jdev = json_object_new_object(); @@ -292,6 +293,19 @@ struct json_object *util_daxctl_dev_to_json(struct daxctl_dev *dev, json_object_object_add(jdev, "numa_node", jobj); } + mode = daxctl_dev_get_mode(dev); + if (mode > 0) { + jobj = NULL; + if (mode == DAXCTL_DEV_MODE_RAM) + jobj = json_object_new_string("system-ram"); + else if (mode == DAXCTL_DEV_MODE_DEVDAX) + jobj = json_object_new_string("devdax"); + else + jobj = json_object_new_string("unknown"); + if (jobj) + json_object_object_add(jdev, "mode", jobj); + } + return jdev; } From patchwork Wed May 8 00:38:48 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934039 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C56AE1390 for ; Wed, 8 May 2019 00:39:29 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B2817286EC for ; Wed, 8 May 2019 00:39:29 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A6D1128751; Wed, 8 May 2019 00:39:29 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 18BFA286EC for ; Wed, 8 May 2019 00:39:29 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id B99B22125583F; Tue, 7 May 2019 17:39:23 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id BDF6F21255825 for ; Tue, 7 May 2019 17:39:21 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427688" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:21 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 07/10] daxctl: add a new reconfigure-device command Date: Tue, 7 May 2019 18:38:48 -0600 Message-Id: <20190508003851.32416-8-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add a new command 'daxctl-reconfigure-device'. This is used to switch the mode of a dax device between regular 'device_dax' and 'system-memory'. The command also uses the memory hotplug sysfs interfaces to online the newly available memory when converting to 'system-ram', and to attempt to offline the memory when converting back to a DAX device. Cc: Pavel Tatashin Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- daxctl/Makefile.am | 2 + daxctl/builtin.h | 1 + daxctl/daxctl.c | 1 + daxctl/device.c | 237 +++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 241 insertions(+) create mode 100644 daxctl/device.c diff --git a/daxctl/Makefile.am b/daxctl/Makefile.am index 94f73f9..66dcc7f 100644 --- a/daxctl/Makefile.am +++ b/daxctl/Makefile.am @@ -15,10 +15,12 @@ daxctl_SOURCES =\ daxctl.c \ list.c \ migrate.c \ + device.c \ ../util/json.c daxctl_LDADD =\ lib/libdaxctl.la \ ../libutil.a \ $(UUID_LIBS) \ + $(KMOD_LIBS) \ $(JSON_LIBS) diff --git a/daxctl/builtin.h b/daxctl/builtin.h index 00ef5e9..756ba2a 100644 --- a/daxctl/builtin.h +++ b/daxctl/builtin.h @@ -6,4 +6,5 @@ struct daxctl_ctx; int cmd_list(int argc, const char **argv, struct daxctl_ctx *ctx); int cmd_migrate(int argc, const char **argv, struct daxctl_ctx *ctx); +int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx); #endif /* _DAXCTL_BUILTIN_H_ */ diff --git a/daxctl/daxctl.c b/daxctl/daxctl.c index 2e41747..e1ba7b8 100644 --- a/daxctl/daxctl.c +++ b/daxctl/daxctl.c @@ -71,6 +71,7 @@ static struct cmd_struct commands[] = { { "list", .d_fn = cmd_list }, { "help", .d_fn = cmd_help }, { "migrate-device-model", .d_fn = cmd_migrate }, + { "reconfigure-device", .d_fn = cmd_reconfig_device }, }; int main(int argc, const char **argv) diff --git a/daxctl/device.c b/daxctl/device.c new file mode 100644 index 0000000..12644c5 --- /dev/null +++ b/daxctl/device.c @@ -0,0 +1,237 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright(c) 2019 Intel Corporation. All rights reserved. */ +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +static struct { + const char *dev; + const char *mode; + int region_id; + bool no_online; + bool do_offline; + bool human; + bool verbose; +} param = { + .region_id = -1, +}; + +static int dev_disable(struct daxctl_dev *dev) +{ + int rc; + + if (!daxctl_dev_is_enabled(dev)) + return 0; + + rc = daxctl_dev_disable(dev); + if (rc) + fprintf(stderr, "%s: disable failed: %s\n", + daxctl_dev_get_devname(dev), strerror(-rc)); + + return rc; +} + +static int reconfig_mode_ram(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + int rc; + + rc = dev_disable(dev); + if (rc) + return rc; + rc = daxctl_dev_enable_ram(dev); + if (rc) + return rc; + + if (param.no_online) + return 0; + + rc = daxctl_dev_online_node(dev); + if (rc < 0) { + fprintf(stderr, "%s: unable to online memory: %s\n", + devname, strerror(-rc)); + return rc; + } + if (param.verbose) + fprintf(stderr, "%s: onlined %d memory sections\n", + devname, rc); + + return 0; +} + +static int reconfig_mode_devdax(struct daxctl_dev *dev) +{ + const char *devname = daxctl_dev_get_devname(dev); + int rc; + + if (param.do_offline) { + rc = daxctl_dev_offline_node(dev); + if (rc < 0) { + fprintf(stderr, "%s: unable to offline memory: %s\n", + devname, strerror(-rc)); + return rc; + } + if (param.verbose) + fprintf(stderr, "%s: offlined %d memory sections\n", + devname, rc); + } + + rc = daxctl_dev_node_is_online(dev); + if (rc < 0) { + fprintf(stderr, "%s: unable to determine node state: %s\n", + devname, strerror(-rc)); + return rc; + } + if (rc > 0) { + if (param.verbose) { + fprintf(stderr, "%s: found %d memory sections online\n", + devname, rc); + fprintf(stderr, "%s: refusing to change modes\n", + devname); + } + return -EBUSY; + } + + rc = dev_disable(dev); + if (rc) + return rc; + + rc = daxctl_dev_enable_devdax(dev); + if (rc) + return rc; + + return 0; +} + +static int do_reconfig(struct daxctl_dev *dev, enum daxctl_dev_mode mode) +{ + int rc = 0; + + switch (mode) { + case DAXCTL_DEV_MODE_RAM: + rc = reconfig_mode_ram(dev); + break; + case DAXCTL_DEV_MODE_DEVDAX: + rc = reconfig_mode_devdax(dev); + break; + default: + fprintf(stderr, "%s: unknown mode: %d\n", + daxctl_dev_get_devname(dev), mode); + rc = -EINVAL; + } + + return rc; +} + +int cmd_reconfig_device(int argc, const char **argv, struct daxctl_ctx *ctx) +{ + const struct option options[] = { + OPT_INTEGER('r', "region", ¶m.region_id, + "restrict to the given region"), + OPT_STRING('m', "mode", ¶m.mode, "mode", + "mode to switch the device to"), + OPT_BOOLEAN('N', "no-online", ¶m.no_online, + "don't auto-online memory sections"), + OPT_BOOLEAN('O', "attempt-offline", ¶m.do_offline, + "attempt to offline memory sections"), + OPT_BOOLEAN('u', "human", ¶m.human, + "use human friendly number formats"), + OPT_BOOLEAN('v', "verbose", ¶m.verbose, + "emit more debug messages"), + OPT_END(), + }; + const char * const u[] = { + "daxctl reconfigure-device [] ...", + NULL + }; + enum daxctl_dev_mode mode = DAXCTL_DEV_MODE_UNKNOWN; + struct json_object *jdevs = json_object_new_array(); + struct daxctl_region *region; + struct json_object *jdev; + int i, rc = 0, done = 0; + unsigned long flags = 0; + struct daxctl_dev *dev; + + argc = parse_options(argc, argv, options, u, 0); + if (argc == 0) + usage_with_options(u, options); + for (i = 0; i < argc; i++) { + if (strcmp(argv[i], "all") == 0) { + argv[0] = "all"; + argc = 1; + break; + } + } + + if (param.human) + flags |= UTIL_JSON_HUMAN; + + if (!param.mode) { + fprintf(stderr, "error: a 'mode' option is required\n"); + usage_with_options(u, options); + } + if (strcmp(param.mode, "system-ram") == 0) { + mode = DAXCTL_DEV_MODE_RAM; + if (param.do_offline) { + fprintf(stderr, + "can't --attempt-offline for system-ram mode\n"); + return -EINVAL; + } + } else if (strcmp(param.mode, "devdax") == 0) { + mode = DAXCTL_DEV_MODE_DEVDAX; + if (param.no_online) { + fprintf(stderr, + "can't --no-online for devdax mode\n"); + return -EINVAL; + } + } + + daxctl_region_foreach(ctx, region) { + if (param.region_id >= 0 && param.region_id + != daxctl_region_get_id(region)) + continue; + + daxctl_dev_foreach(region, dev) { + bool dev_requested = false; + + for (i = 0; i < argc; i++) { + if ((strcmp(daxctl_dev_get_devname(dev), + argv[i]) == 0) || + (strcmp(argv[i], "all") == 0)) { + dev_requested = true; + break; + } + } + if (dev_requested) { + rc = do_reconfig(dev, mode); + if (rc < 0) + goto out_err; + done++; + if (!jdevs) + continue; + jdev = util_daxctl_dev_to_json(dev, flags); + if (jdev) + json_object_array_add(jdevs, jdev); + } + } + } + if (jdevs) + util_display_json_array(stdout, jdevs, flags); + + fprintf(stderr, "reconfigured %d device%s\n", done, + done == 1 ? "" : "s"); + return 0; + +out_err: + fprintf(stderr, "error reconfiguring %s: %s\n", + daxctl_dev_get_devname(dev), strerror(-rc)); + return rc; +} From patchwork Wed May 8 00:38:49 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934041 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A8C911390 for ; Wed, 8 May 2019 00:39:30 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 981A3286EC for ; Wed, 8 May 2019 00:39:30 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8C4E528751; Wed, 8 May 2019 00:39:30 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 2517F286EC for ; Wed, 8 May 2019 00:39:30 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id EBA9721945DCD; Tue, 7 May 2019 17:39:23 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 0CA0521255842 for ; Tue, 7 May 2019 17:39:22 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:21 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427691" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:21 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 08/10] Documentation/daxctl: add a man page for daxctl-reconfigure-device Date: Tue, 7 May 2019 18:38:49 -0600 Message-Id: <20190508003851.32416-9-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add a man page describing the new daxctl-reconfigure-device command. Cc: Pavel Tatashin Cc: Dave Hansen Cc: Dan Williams Signed-off-by: Vishal Verma --- Documentation/daxctl/Makefile.am | 3 +- .../daxctl/daxctl-reconfigure-device.txt | 118 ++++++++++++++++++ 2 files changed, 120 insertions(+), 1 deletion(-) create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt diff --git a/Documentation/daxctl/Makefile.am b/Documentation/daxctl/Makefile.am index 6aba035..715fbad 100644 --- a/Documentation/daxctl/Makefile.am +++ b/Documentation/daxctl/Makefile.am @@ -28,7 +28,8 @@ endif man1_MANS = \ daxctl.1 \ daxctl-list.1 \ - daxctl-migrate-device-model.1 + daxctl-migrate-device-model.1 \ + daxctl-reconfigure-device.1 CLEANFILES = $(man1_MANS) diff --git a/Documentation/daxctl/daxctl-reconfigure-device.txt b/Documentation/daxctl/daxctl-reconfigure-device.txt new file mode 100644 index 0000000..b575091 --- /dev/null +++ b/Documentation/daxctl/daxctl-reconfigure-device.txt @@ -0,0 +1,118 @@ +// SPDX-License-Identifier: GPL-2.0 + +daxctl-reconfigure-device(1) +============================ + +NAME +---- +daxctl-reconfigure-device - Reconfigure a dax device into a different mode + +SYNOPSIS +-------- +[verse] +'daxctl reconfigure-device' [...] [] + +EXAMPLES +-------- + +* Reconfigure dax0.0 to system-ram mode, don't online the memory +---- +# daxctl reconfigure-device --mode=system-ram --no-online dax0.0 +[ + { + "chardev":"dax0.0", + "size":16777216000, + "numa_node":2, + "mode":"system-ram" + } +] +---- + +* Reconfigure dax0.0 to devdax mode, attempt to offline the memory +---- +# daxctl reconfigure-device --human --mode=devdax --attempt-offline dax0.0 +{ + "chardev":"dax0.0", + "size":"15.63 GiB (16.78 GB)", + "numa_node":2, + "mode":"devdax" +} +---- + +* Reconfigure all dax devices on region0 to system-ram mode +---- +# daxctl reconfigure-device --mode=system-ram --region=0 all +[ + { + "chardev":"dax0.0", + "size":16777216000, + "numa_node":2, + "mode":"system-ram" + }, + { + "chardev":"dax0.1", + "size":16777216000, + "numa_node":3, + "mode":"system-ram" + } +] +---- + +DESCRIPTION +----------- + +Reconfigure the operational mode of a dax device. This can be used to convert +a regular 'devdax' mode device to the 'system-ram' mode which allows for the dax +range to be hot-plugged into the system as regular memory. + +NOTE: This is a destructive operation. Any data on the dax device *will* be +lost. + +OPTIONS +------- +-r:: +--region=:: + Restrict the reconfigure operation to devices belonging to the + specified region(s). A device-dax region is a contiguous range of + memory that hosts one or more /dev/daxX.Y devices, where X is the + region id and Y is the device instance id. + +-m:: +--mode=:: + Specify the mode to which the dax device(s) should be reconfigured. + - "system-ram": hotplug the device into system memory. + + - "devdax": switch to the normal "device dax" mode. This may not work + on kernels prior to v5.2. In such a case, a reboot is the only way to + switch back to 'devdax' mode. + +-N:: +--no-online:: + By default, memory sections provided by system-ram devices will be + brought online automatically and immediately. Use this option to + disable the automatic onlining behavior. + +-O:: +--attempt-offline:: + When converting from "system-ram" mode to "devdax", it is expected + that all the memory sections are first made offline. By default, + daxctl won't touch online memory. However with this option, attempt + to offline the memory on the NUMA node associated with the dax device + before converting it back to "devdax" mode. + +-u:: +--human:: + By default the command will output machine-friendly raw-integer + data. Instead, with this flag, numbers representing storage size + will be formatted as human readable strings with units, other + fields are converted to hexadecimal strings. + +-v:: +--verbose:: + Emit more debug messages for the reconfiguration process + +include::../copyright.txt[] + +SEE ALSO +-------- +linkdaxctl:daxctl-list[1] From patchwork Wed May 8 00:38:50 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934043 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 707BA16C1 for ; Wed, 8 May 2019 00:39:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 5FEFC286EC for ; Wed, 8 May 2019 00:39:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 5415A28751; Wed, 8 May 2019 00:39:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 5A9D1286EC for ; Wed, 8 May 2019 00:39:31 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 303E721BC6A80; Tue, 7 May 2019 17:39:24 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id 9E0B3212449EF for ; Tue, 7 May 2019 17:39:22 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427695" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:21 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 09/10] contrib/ndctl: fix region-id completions for daxctl Date: Tue, 7 May 2019 18:38:50 -0600 Message-Id: <20190508003851.32416-10-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP The completion helpers for daxctl assumed the region arguments for specifying daxctl regions were the same as ndctl regions, i.e. "regionX". This is not true - daxctl region arguments are a simple numeric 'id'. Add a new helper __daxctl_get_regions() to complete daxctl region IDs properly. While at it, fix a useless use of 'echo' in __daxctl_get_devs() and quoting in __daxctl_comp_options() Fixes: d6790a32f32c ("daxctl: Add bash-completion") Signed-off-by: Vishal Verma --- contrib/ndctl | 14 ++++++++++---- 1 file changed, 10 insertions(+), 4 deletions(-) diff --git a/contrib/ndctl b/contrib/ndctl index e17fb0b..d1f8bd6 100755 --- a/contrib/ndctl +++ b/contrib/ndctl @@ -528,8 +528,14 @@ _ndctl() __daxctl_get_devs() { - local opts="--devices $*" - echo "$(daxctl list $opts | grep -E "^\s*\"chardev\":" | cut -d\" -f4)" + local opts=("--devices" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"chardev\":" | cut -d'"' -f4 +} + +__daxctl_get_regions() +{ + local opts=("--regions" "$*") + daxctl list "${opts[@]}" | grep -E "^\s*\"id\":" | grep -Eo "[0-9]+" } __daxctlcomp() @@ -558,10 +564,10 @@ __daxctl_comp_options() local cur_arg=${cur##*=} case $cur_subopt in --region) - opts=$(__ndctl_get_regions -i) + opts="$(__daxctl_get_regions -i)" ;; --dev) - opts=$(__daxctl_get_devs -i) + opts="$(__daxctl_get_devs -i)" ;; *) return From patchwork Wed May 8 00:38:51 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Verma, Vishal L" X-Patchwork-Id: 10934045 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CFF511390 for ; Wed, 8 May 2019 00:39:32 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id BF31E286EC for ; Wed, 8 May 2019 00:39:32 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id B38F428751; Wed, 8 May 2019 00:39:32 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-2.9 required=2.0 tests=BAYES_00,MAILING_LIST_MULTI, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 Received: from ml01.01.org (ml01.01.org [198.145.21.10]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 78184286EC for ; Wed, 8 May 2019 00:39:32 +0000 (UTC) Received: from [127.0.0.1] (localhost [IPv6:::1]) by ml01.01.org (Postfix) with ESMTP id 8C715212449EF; Tue, 7 May 2019 17:39:24 -0700 (PDT) X-Original-To: linux-nvdimm@lists.01.org Delivered-To: linux-nvdimm@lists.01.org Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=134.134.136.24; helo=mga09.intel.com; envelope-from=vishal.l.verma@intel.com; receiver=linux-nvdimm@lists.01.org Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ml01.01.org (Postfix) with ESMTPS id E532E2194D387 for ; Tue, 7 May 2019 17:39:22 -0700 (PDT) X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga102.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 07 May 2019 17:39:22 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,443,1549958400"; d="scan'208";a="169427700" Received: from vverma7-desk1.lm.intel.com ([10.232.112.185]) by fmsmga002.fm.intel.com with ESMTP; 07 May 2019 17:39:22 -0700 From: Vishal Verma To: Subject: [ndctl PATCH v2 10/10] contrib/ndctl: add bash-completion for daxctl-reconfigure-device Date: Tue, 7 May 2019 18:38:51 -0600 Message-Id: <20190508003851.32416-11-vishal.l.verma@intel.com> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20190508003851.32416-1-vishal.l.verma@intel.com> References: <20190508003851.32416-1-vishal.l.verma@intel.com> MIME-Version: 1.0 X-BeenThere: linux-nvdimm@lists.01.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "Linux-nvdimm developer list." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Dave Hansen , Pavel Tatashin Errors-To: linux-nvdimm-bounces@lists.01.org Sender: "Linux-nvdimm" X-Virus-Scanned: ClamAV using ClamSMTP Add bash completion helpers for the new daxctl-reconfigure-device command. Cc: Dan Williams Signed-off-by: Vishal Verma --- contrib/ndctl | 20 +++++++++++++++++--- 1 file changed, 17 insertions(+), 3 deletions(-) diff --git a/contrib/ndctl b/contrib/ndctl index d1f8bd6..32c4731 100755 --- a/contrib/ndctl +++ b/contrib/ndctl @@ -544,7 +544,7 @@ __daxctlcomp() COMPREPLY=( $( compgen -W "$1" -- "$2" ) ) for cword in "${COMPREPLY[@]}"; do - if [[ "$cword" == @(--region|--dev) ]]; then + if [[ "$cword" == @(--region|--dev|--mode) ]]; then COMPREPLY[$i]="${cword}=" else COMPREPLY[$i]="${cword} " @@ -569,6 +569,9 @@ __daxctl_comp_options() --dev) opts="$(__daxctl_get_devs -i)" ;; + --mode) + opts="system-ram devdax" + ;; *) return ;; @@ -579,8 +582,19 @@ __daxctl_comp_options() __daxctl_comp_non_option_args() { - # there aren't any commands that accept non option arguments yet - return + local subcmd=$1 + local cur=$2 + local opts + + case $subcmd in + reconfigure-device) + opts="$(__daxctl_get_devs -i) all" + ;; + *) + return + ;; + esac + __daxctlcomp "$opts" "$cur" } __daxctl_main()