[ndctl,v4,00/10] daxctl: add a new reconfigure-device command

Message

Verma, Vishal L May 28, 2019, 10:24 p.m. UTC

Changes in v4:
 - Don't fail add_dax_dev for kmod failures. Instead fail only when the kmod
   list is actually used, i.e. during daxctl-reconfigure-device

Changes in v3:
 - In daxctl_dev_get_mode(), remove the subsystem warning, detect dax-class
   and simply make it return devdax

Changes in v2:
 - Add examples to the documentation page (Dave Hansen)
 - Clarify documentation regarding the conversion from system-ram to devdax
 - Remove any references to a persistent config from the documentation -
   those can be added when the feature is added.
 - device.c: validate option compatibility
 - daxctl-list: display numa_node for device listings
 - daxctl-list: display mode for device listings
 - make the options more consistent by adding a '-O' short option
   for --attempt-offline

Add a new daxctl-reconfigure-device command that lets us reconfigure DAX
devices back and forth between 'system-ram' and 'device-dax' modes. It
also includes facilities to online any newly hot-plugged memory
(default), and attempt to offline memory before converting away from the
system-ram mode (not default, requires a --attempt-offline option).

Currently missing from this series is a way to persistently store which
devices have been 'marked' for use as system-ram. This depends on a
config system overhaul in ndctl, and patches for those will follow
separately and are independent of this work.

Example invocations:

1. Reconfigure dax0.0 to system-ram mode, don’t online the memory
    # daxctl reconfigure-device --mode=system-ram --no-online dax0.0
    [
      {
        "chardev":"dax0.0",
        "size":16777216000,
        "numa_node":2,
        "mode":"system-ram"
      }
    ]

2. Reconfigure dax0.0 to devdax mode, attempt to offline the memory
    # daxctl reconfigure-device --human --mode=devdax --attempt-offline dax0.0
    {
      "chardev":"dax0.0",
      "size":"15.63 GiB (16.78 GB)",
      "numa_node":2,
      "mode":"devdax"
    }

3. Reconfigure all dax devices on region0 to system-ram mode
    # daxctl reconfigure-device --mode=system-ram --region=0 all
    [
      {
        "chardev":"dax0.0",
        "size":16777216000,
        "numa_node":2,
        "mode":"system-ram"
      },
      {
        "chardev":"dax0.1",
        "size":16777216000,
        "numa_node":3,
        "mode":"system-ram"
      }
    ]

These patches can also be found in the 'kmem-pending' branch on github:
https://github.com/pmem/ndctl/tree/kmem-pending

Cc: Dan Williams <[e-mail]>
Cc: Dave Hansen <[e-mail]>
Cc: Pavel Tatashin <[e-mail]>

Vishal Verma (10):
  libdaxctl: add interfaces in support of device modes
  libdaxctl: cache 'subsystem' in daxctl_ctx
  libdaxctl: add interfaces to enable/disable devices
  libdaxctl: add interfaces to get/set the online state for a node
  daxctl/list: add numa_node for device listings
  libdaxctl: add an interface to get the mode for a dax device
  daxctl: add a new reconfigure-device command
  Documentation/daxctl: add a man page for daxctl-reconfigure-device
  contrib/ndctl: fix region-id completions for daxctl
  contrib/ndctl: add bash-completion for daxctl-reconfigure-device

 Documentation/daxctl/Makefile.am              |   3 +-
 .../daxctl/daxctl-reconfigure-device.txt      | 118 ++++
 contrib/ndctl                                 |  34 +-
 daxctl/Makefile.am                            |   2 +
 daxctl/builtin.h                              |   1 +
 daxctl/daxctl.c                               |   1 +
 daxctl/device.c                               | 237 ++++++++
 daxctl/lib/Makefile.am                        |   3 +-
 daxctl/lib/libdaxctl-private.h                |  21 +
 daxctl/lib/libdaxctl.c                        | 548 +++++++++++++++++-
 daxctl/lib/libdaxctl.sym                      |  14 +
 daxctl/libdaxctl.h                            |  16 +
 util/json.c                                   |  22 +
 13 files changed, 1009 insertions(+), 11 deletions(-)
 create mode 100644 Documentation/daxctl/daxctl-reconfigure-device.txt
 create mode 100644 daxctl/device.c