From patchwork Mon Sep 11 04:38:17 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haozhong Zhang X-Patchwork-Id: 9946627 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id 1716E6035D for ; Mon, 11 Sep 2017 04:42:18 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 0C92527D0C for ; Mon, 11 Sep 2017 04:42:18 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 017F327F2B; Mon, 11 Sep 2017 04:42:17 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-4.2 required=2.0 tests=BAYES_00, RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 19C772837E for ; Mon, 11 Sep 2017 04:42:17 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drGVz-0003RD-Fe; Mon, 11 Sep 2017 04:40:07 +0000 Received: from mail6.bemta6.messagelabs.com ([193.109.254.103]) by lists.xenproject.org with esmtp (Exim 4.84_2) (envelope-from ) id 1drGVy-0003LH-HD for xen-devel@lists.xen.org; Mon, 11 Sep 2017 04:40:06 +0000 Received: from [193.109.254.147] by server-6.bemta-6.messagelabs.com id BF/5B-03423-5A316B95; Mon, 11 Sep 2017 04:40:05 +0000 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFmpjkeJIrShJLcpLzFFi42Jpa+sQ0V0qvC3 S4NUZNoslHxezODB6HN39mymAMYo1My8pvyKBNePGwp2sBY8iKzrWnWNvYJzu3MXIxSEkMJ1R YlXnItYuRk4OCQFeiSPLZkDZARJnpm0Ds4UEehkl1k1WA7HZBPQlVjw+CBYXEZCWuPb5MiOIz Szwh1Hi3idZEFtYIEli2eTZYHEWAVWJe/cvs4HYvAJ2EicnbmeHmC8vsavtItgcTqD4wZfvmC F22UosOL2AdQIj7wJGhlWMGsWpRWWpRbpG5npJRZnpGSW5iZk5uoYGZnq5qcXFiempOYlJxXr J+bmbGIHhwAAEOxgXrw08xCjJwaQkyvvu+JZIIb6k/JTKjMTijPii0pzU4kOMMhwcShK8KkLb IoUEi1LTUyvSMnOAgQmTluDgURLhjQJJ8xYXJOYWZ6ZDpE4xWnIc23T5DxPHgQlXgGTHzbt/m IRY8vLzUqXEeeVAGgRAGjJK8+DGwaLnEqOslDAvI9CBQjwFqUW5mSWo8q8YxTkYlYR5I0Cm8G TmlcBtfQV0EBPQQTyXtoAcVJKIkJJqYJT/d/tMqV2DwvPY809PO0yP7998geuPxC3Zqb1t0vO FY6yC3qWGxHI+aEvdulUme7fNK/85NuVMOvfmGySkRaabLd8pt3ZdY9fiqQryVuet+CObQ5+8 6ZTcuyvUKEkgcVdt/t/zGz/r5qxO3zcvdLeCVnbOywUVHuZau6orLV/vU5p2IKjzjBJLcUaio RZzUXEiACY83QWZAgAA X-Env-Sender: haozhong.zhang@intel.com X-Msg-Ref: server-2.tower-27.messagelabs.com!1505104735!56506342!31 X-Originating-IP: [134.134.136.20] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTM0LjEzNC4xMzYuMjAgPT4gMzU1MzU4\n X-StarScan-Received: X-StarScan-Version: 9.4.45; banners=-,-,- X-VirusChecked: Checked Received: (qmail 52159 invoked from network); 11 Sep 2017 04:40:04 -0000 Received: from mga02.intel.com (HELO mga02.intel.com) (134.134.136.20) by server-2.tower-27.messagelabs.com with DHE-RSA-AES256-GCM-SHA384 encrypted SMTP; 11 Sep 2017 04:40:04 -0000 Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga101.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Sep 2017 21:40:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos; i="5.42,376,1500966000"; d="scan'208"; a="1217078627" Received: from hz-desktop.sh.intel.com (HELO localhost) ([10.239.159.142]) by fmsmga002.fm.intel.com with ESMTP; 10 Sep 2017 21:40:02 -0700 From: Haozhong Zhang To: xen-devel@lists.xen.org Date: Mon, 11 Sep 2017 12:38:17 +0800 Message-Id: <20170911043820.14617-37-haozhong.zhang@intel.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20170911043820.14617-1-haozhong.zhang@intel.com> References: <20170911043820.14617-1-haozhong.zhang@intel.com> Cc: Haozhong Zhang , Wei Liu , Ian Jackson , Chao Peng , Dan Williams Subject: [Xen-devel] [RFC XEN PATCH v3 36/39] tools/xl: add xl domain configuration for virtual NVDIMM devices X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: xen-devel-bounces@lists.xen.org Sender: "Xen-devel" X-Virus-Scanned: ClamAV using ClamSMTP A new xl domain configuration vnvdimms = [ 'type=mfn, backend=START_PMEM_MFN, nr_pages=N', ... ] is added to specify the virtual NVDIMM devices backed by the specified host PMEM pages. As the kernel PMEM driver does not work in Dom0 now, we have to specify MFNs. Signed-off-by: Haozhong Zhang --- Cc: Ian Jackson Cc: Wei Liu --- docs/man/xl.cfg.pod.5.in | 33 +++++++++++++ tools/libxl/Makefile | 2 +- tools/libxl/libxl.h | 5 ++ tools/libxl/libxl_types.idl | 15 ++++++ tools/libxl/libxl_vnvdimm.c | 49 ++++++++++++++++++++ tools/xl/xl_parse.c | 110 +++++++++++++++++++++++++++++++++++++++++++- tools/xl/xl_vmcontrol.c | 15 +++++- 7 files changed, 226 insertions(+), 3 deletions(-) create mode 100644 tools/libxl/libxl_vnvdimm.c diff --git a/docs/man/xl.cfg.pod.5.in b/docs/man/xl.cfg.pod.5.in index 79cb2eaea7..092b051561 100644 --- a/docs/man/xl.cfg.pod.5.in +++ b/docs/man/xl.cfg.pod.5.in @@ -1116,6 +1116,39 @@ FIFO-based event channel ABI support up to 131,071 event channels. Other guests are limited to 4095 (64-bit x86 and ARM) or 1023 (32-bit x86). +=item B + +Specifies the virtual NVDIMM devices which are provided to the guest. + +Each B is a comma-separated list of C settings +from the following list: + +=over 4 + +=item B + +Specifies the type of host backend of the virtual NVDIMM device. Following +is a list of supported types: + +=over 4 + +=item B + +backs the virtual NVDIMM device by a contiguous host PMEM region. + +=back + +=item B + +Specifies the host backend of the virtual NVDIMM device. If C, +then B specifies the start MFN of the host PMEM region. + +=item B + +Specifies the number of pages of the host backend. + +=back + =back =head2 Paravirtualised (PV) Guest Specific Options diff --git a/tools/libxl/Makefile b/tools/libxl/Makefile index 791c9ad05e..b4c2ccb7ff 100644 --- a/tools/libxl/Makefile +++ b/tools/libxl/Makefile @@ -139,7 +139,7 @@ LIBXL_OBJS = flexarray.o libxl.o libxl_create.o libxl_dm.o libxl_pci.o \ libxl_dom_suspend.o libxl_dom_save.o libxl_usb.o \ libxl_vtpm.o libxl_nic.o libxl_disk.o libxl_console.o \ libxl_cpupool.o libxl_mem.o libxl_sched.o libxl_tmem.o \ - libxl_9pfs.o libxl_domain.o \ + libxl_9pfs.o libxl_domain.o libxl_vnvdimm.o \ $(LIBXL_OBJS-y) LIBXL_OBJS += libxl_genid.o LIBXL_OBJS += _libxl_types.o libxl_flask.o _libxl_types_internal.o diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 91408b47b5..8156c08ed3 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -1474,6 +1474,11 @@ int libxl_get_memory_target_0x040700(libxl_ctx *ctx, uint32_t domid, uint32_t *out_target) LIBXL_EXTERNAL_CALLERS_ONLY; +int libxl_vnvdimm_copy_config(libxl_ctx *ctx, + libxl_domain_config *dst, + const libxl_domain_config *src) + LIBXL_EXTERNAL_CALLERS_ONLY; + /* * WARNING * This memory management API is unstable even in Xen 4.2. diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index 4acc0457f4..ad236de34a 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -240,6 +240,10 @@ libxl_checkpointed_stream = Enumeration("checkpointed_stream", [ (2, "COLO"), ]) +libxl_vnvdimm_backend_type = Enumeration("vnvdimm_backend_type", [ + (0, "mfn"), + ]) + # # Complex libxl types # @@ -780,6 +784,16 @@ libxl_device_channel = Struct("device_channel", [ ])), ]) +libxl_device_vnvdimm = Struct("device_vnvdimm", [ + ("backend_domid", libxl_domid), + ("backend_domname", string), + ("devid", libxl_devid), + ("nr_pages", uint64), + ("u", KeyedUnion(None, libxl_vnvdimm_backend_type, "backend_type", + [("mfn", uint64), + ])), +]) + libxl_domain_config = Struct("domain_config", [ ("c_info", libxl_domain_create_info), ("b_info", libxl_domain_build_info), @@ -798,6 +812,7 @@ libxl_domain_config = Struct("domain_config", [ ("channels", Array(libxl_device_channel, "num_channels")), ("usbctrls", Array(libxl_device_usbctrl, "num_usbctrls")), ("usbdevs", Array(libxl_device_usbdev, "num_usbdevs")), + ("vnvdimms", Array(libxl_device_vnvdimm, "num_vnvdimms")), ("on_poweroff", libxl_action_on_shutdown), ("on_reboot", libxl_action_on_shutdown), diff --git a/tools/libxl/libxl_vnvdimm.c b/tools/libxl/libxl_vnvdimm.c new file mode 100644 index 0000000000..4de8f04303 --- /dev/null +++ b/tools/libxl/libxl_vnvdimm.c @@ -0,0 +1,49 @@ +/* + * tools/libxl/libxl_vnvdimm.c + * + * Copyright (C) 2017, Intel Corporation + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License, version 2.1, as published by the Free Software Foundation. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; If not, see . + */ + +#include + +#include "libxl_internal.h" + +int libxl_vnvdimm_copy_config(libxl_ctx *ctx, + libxl_domain_config *dst, + const libxl_domain_config *src) +{ + GC_INIT(ctx); + unsigned int nr = src->num_vnvdimms; + libxl_device_vnvdimm *vnvdimms; + int rc = 0; + + if (!nr) + goto out; + + vnvdimms = libxl__calloc(NOGC, nr, sizeof(*vnvdimms)); + if (!vnvdimms) { + rc = ERROR_NOMEM; + goto out; + } + + dst->num_vnvdimms = nr; + while (nr--) + libxl_device_vnvdimm_copy(ctx, &vnvdimms[nr], &src->vnvdimms[nr]); + dst->vnvdimms = vnvdimms; + + out: + GC_FREE; + return rc; +} diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index ed562a1956..388a135dbf 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -804,13 +804,111 @@ int parse_usbdev_config(libxl_device_usbdev *usbdev, char *token) return 0; } +static int parse_vnvdimm_config(libxl_device_vnvdimm *vnvdimm, char *token) +{ + char *oparg, *endptr; + unsigned long val; + + if (MATCH_OPTION("type", token, oparg)) { + if (libxl_vnvdimm_backend_type_from_string(oparg, + &vnvdimm->backend_type)) { + fprintf(stderr, + "ERROR: invalid vNVDIMM backend type '%s'\n", + oparg); + return 1; + } + } else if (MATCH_OPTION("nr_pages", token, oparg)) { + val = strtoul(oparg, &endptr, 0); + if (endptr == oparg || val == ULONG_MAX) + { + fprintf(stderr, + "ERROR: invalid number of vNVDIMM backend pages '%s'\n", + oparg); + return 1; + } + vnvdimm->nr_pages = val; + } else if (MATCH_OPTION("backend", token, oparg)) { + /* Skip: handled by parse_vnvdimms() */ + } else { + fprintf(stderr, "ERROR: unknown string '%s' in vnvdimm spec\n", token); + return 1; + } + + return 0; +} + +/* + * vnvdimms = [ 'type=, backend=, nr_pages=', ... ] + */ +static void parse_vnvdimms(XLU_Config *config, libxl_domain_config *d_config) +{ + XLU_ConfigList *vnvdimms; + const char *buf; + int rc; + + rc = xlu_cfg_get_list(config, "vnvdimms", &vnvdimms, 0, 0); + if ( rc ) + return; + +#if !defined(__linux__) + fprintf(stderr, "ERROR: 'vnvdimms' is only supported on Linux\n"); + exit(-ERROR_FAIL); +#endif + + d_config->num_vnvdimms = 0; + d_config->vnvdimms = NULL; + + while ((buf = xlu_cfg_get_listitem(vnvdimms, + d_config->num_vnvdimms)) != NULL) { + libxl_device_vnvdimm *vnvdimm = + ARRAY_EXTEND_INIT(d_config->vnvdimms, d_config->num_vnvdimms, + libxl_device_vnvdimm_init); + char *buf2 = strdup(buf), *backend = NULL, *p, *endptr; + unsigned long mfn; + + p = strtok(buf2, ","); + if (!p) + goto skip_nvdimm; + + do { + while (*p == ' ') + p++; + + rc = 0; + if (!MATCH_OPTION("backend", p, backend)) + rc = parse_vnvdimm_config(vnvdimm, p); + if (rc) + exit(-ERROR_FAIL); + } while ((p = strtok(NULL, ",")) != NULL); + + switch (vnvdimm->backend_type) + { + case LIBXL_VNVDIMM_BACKEND_TYPE_MFN: + mfn = strtoul(backend, &endptr, 0); + if (endptr == backend || mfn == ULONG_MAX) + { + fprintf(stderr, + "ERROR: invalid start MFN of host NVDIMM '%s'\n", + backend); + exit(-ERROR_FAIL); + } + vnvdimm->u.mfn = mfn; + + break; + } + + skip_nvdimm: + free(buf2); + } +} + void parse_config_data(const char *config_source, const char *config_data, int config_len, libxl_domain_config *d_config) { const char *buf; - long l, vcpus = 0, nr_dm_acpi_pages; + long l, vcpus = 0, nr_dm_acpi_pages = 0; XLU_Config *config; XLU_ConfigList *cpus, *vbds, *nics, *pcis, *cvfbs, *cpuids, *vtpms, *usbctrls, *usbdevs, *p9devs; @@ -1942,6 +2040,16 @@ skip_usbdev: exit(-ERROR_FAIL); } b_info->u.hvm.dm_acpi_pages = nr_dm_acpi_pages; + + /* parse 'vnvdimms' */ + parse_vnvdimms(config, d_config); + + /* + * If 'dm_acpi_pages' is not specified, reserve one DM ACPI + * page for vNVDIMM devices. + */ + if (d_config->vnvdimms && !nr_dm_acpi_pages) + b_info->u.hvm.dm_acpi_pages = 1; } /* If we've already got vfb=[] for PV guest then ignore top level diff --git a/tools/xl/xl_vmcontrol.c b/tools/xl/xl_vmcontrol.c index 89c2b25ded..1bdc173e04 100644 --- a/tools/xl/xl_vmcontrol.c +++ b/tools/xl/xl_vmcontrol.c @@ -381,12 +381,25 @@ static void reload_domain_config(uint32_t domid, if (rc) { LOG("failed to retrieve guest configuration (rc=%d). " "reusing old configuration", rc); - libxl_domain_config_dispose(&d_config_new); + goto error_out; } else { + rc = libxl_vnvdimm_copy_config(ctx, &d_config_new, d_config); + if (rc) { + LOG("failed to copy vnvdimm configuration (rc=%d). " + "reusing old configuration", rc); + libxl_domain_config_dispose(&d_config_new); + goto error_out; + } + libxl_domain_config_dispose(d_config); /* Steal allocations */ memcpy(d_config, &d_config_new, sizeof(libxl_domain_config)); } + + return; + + error_out: + libxl_domain_config_dispose(&d_config_new); } /* Can update r_domid if domain is destroyed */