From patchwork Tue Dec 29 11:28:46 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Haozhong Zhang X-Patchwork-Id: 7929751 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork1.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork1.web.kernel.org (Postfix) with ESMTP id CB2C69F350 for ; Tue, 29 Dec 2015 11:32:22 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id 6414B20265 for ; Tue, 29 Dec 2015 11:32:21 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0E49A20221 for ; Tue, 29 Dec 2015 11:32:20 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aDsT2-00010X-3J; Tue, 29 Dec 2015 11:29:28 +0000 Received: from mail6.bemta4.messagelabs.com ([85.158.143.247]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aDsT0-00010K-1C for xen-devel@lists.xensource.com; Tue, 29 Dec 2015 11:29:26 +0000 Received: from [85.158.143.35] by server-1.bemta-4.messagelabs.com id CE/77-21571-59E62865; Tue, 29 Dec 2015 11:29:25 +0000 X-Env-Sender: haozhong.zhang@intel.com X-Msg-Ref: server-11.tower-21.messagelabs.com!1451388563!7565211!1 X-Originating-IP: [134.134.136.65] X-SpamReason: No, hits=0.0 required=7.0 tests= X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 19917 invoked from network); 29 Dec 2015 11:29:24 -0000 Received: from mga03.intel.com (HELO mga03.intel.com) (134.134.136.65) by server-11.tower-21.messagelabs.com with SMTP; 29 Dec 2015 11:29:24 -0000 Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by orsmga103.jf.intel.com with ESMTP; 29 Dec 2015 03:29:23 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.20,495,1444719600"; d="scan'208";a="871146791" Received: from hz-desktop.sh.intel.com (HELO localhost) ([10.239.13.40]) by fmsmga001.fm.intel.com with ESMTP; 29 Dec 2015 03:29:21 -0800 From: Haozhong Zhang To: xen-devel@lists.xensource.com Date: Tue, 29 Dec 2015 19:28:46 +0800 Message-Id: <1451388527-18009-2-git-send-email-haozhong.zhang@intel.com> X-Mailer: git-send-email 2.4.8 In-Reply-To: <1451388527-18009-1-git-send-email-haozhong.zhang@intel.com> References: <1451388527-18009-1-git-send-email-haozhong.zhang@intel.com> Cc: Haozhong Zhang , Xiao Guangrong , Eduardo Habkost , "Michael S. Tsirkin" , Stefano Stabellini , Paolo Bonzini , Igor Mammedov , Richard Henderson Subject: [Xen-devel] [PATCH 1/2] pc-nvdimm: implement pc-nvdimm device abstract X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP The current implementation of vNVDIMM is based on pc-dimm and uses memory-backend device to allocate memory, which is not compatible with Xen ("-mem-path not supported with Xen" in qemu_ram_alloc_file()). This patch adds another pc-nvdimm device that does not rely on pc-dimm and allocates memory by itself. This patch combines several parts of Guangrong's v2 patch series "implement vNVDIMM" and Xen-specific adjustments. Signed-off-by: Haozhong Zhang --- hw/mem/Makefile.objs | 1 + hw/mem/pc-nvdimm.c | 239 +++++++++++++++++++++++++++++++++++++++++++++ include/hw/mem/pc-nvdimm.h | 49 ++++++++++ xen-hvm.c | 2 + 4 files changed, 291 insertions(+) create mode 100644 hw/mem/pc-nvdimm.c create mode 100644 include/hw/mem/pc-nvdimm.h diff --git a/hw/mem/Makefile.objs b/hw/mem/Makefile.objs index f12f8b9..4257da0 100644 --- a/hw/mem/Makefile.objs +++ b/hw/mem/Makefile.objs @@ -1,2 +1,3 @@ common-obj-$(CONFIG_MEM_HOTPLUG) += pc-dimm.o common-obj-$(CONFIG_NVDIMM) += nvdimm.o +common-obj-$(CONFIG_NVDIMM) += pc-nvdimm.o diff --git a/hw/mem/pc-nvdimm.c b/hw/mem/pc-nvdimm.c new file mode 100644 index 0000000..321f734 --- /dev/null +++ b/hw/mem/pc-nvdimm.c @@ -0,0 +1,239 @@ +/* + * NVDIMM (A Non-Volatile Dual In-line Memory Module) Virtualization Implement + * + * Copyright(C) 2015 Intel Corporation. + * + * Author: + * Xiao Guangrong + * Haozhong Zhang + * + * Currently, it only supports PMEM Virtualization. + * + * This library is free software; you can redistribute it and/or + * modify it under the terms of the GNU Lesser General Public + * License as published by the Free Software Foundation; either + * version 2 of the License, or (at your option) any later version. + * + * This library is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU + * Lesser General Public License for more details. + * + * You should have received a copy of the GNU Lesser General Public + * License along with this library; if not, see + */ + +#include + +#include "qom/object.h" +#include "qapi/visitor.h" +#include "qemu/mmap-alloc.h" +#include "exec/address-spaces.h" +#include "hw/mem/pc-nvdimm.h" +#include "hw/xen/xen.h" + +#define PC_NVDIMM_ADDR_ALIGN 0x40000000 + +struct NvdimmsInfo { + ram_addr_t current_addr; + int device_index; +}; + +static struct NvdimmsInfo nvdimms_info; + +static ram_addr_t pc_nvdimm_reserved_range_push(uint64_t size) +{ + uint64_t current; + + current = ROUND_UP(nvdimms_info.current_addr, PC_NVDIMM_ADDR_ALIGN); + + /* do not have enough space? */ + if (current + size < current) { + return 0; + } + + nvdimms_info.current_addr = current + size; + return current; +} + +void pc_nvdimm_reserve_range(ram_addr_t offset) +{ + nvdimms_info.current_addr = ROUND_UP(offset, PC_NVDIMM_ADDR_ALIGN); +} + +static int pc_nvdimm_new_device_index(void) +{ + return nvdimms_info.device_index++; +} + +static void pc_nvdimm_get_addr(Object *obj, Visitor *v, void *opaque, + const char *name, Error **errp) +{ + int64_t value; + PCNVDIMMDevice *nvdimm = PC_NVDIMM(obj); + value = nvdimm->addr; + visit_type_int(v, &value, name, errp); +} + +static void pc_nvdimm_get_slot(Object *obj, Visitor *v, void *opaque, + const char *name, Error **errp) +{ + int64_t value; + PCNVDIMMDevice *nvdimm = PC_NVDIMM(obj); + value = nvdimm->dev_idx + 1; + visit_type_int(v, &value, name, errp); +} + +static void pc_nvdimm_get_node(Object *obj, Visitor *v, void *opaque, + const char *name, Error **errp) +{ + int64_t value; + value = 0; + visit_type_int(v, &value, name, errp); +} + +static void pc_nvdimm_get_size(Object *obj, Visitor *v, void *opaque, + const char *name, Error **errp) +{ + PCNVDIMMDevice *nvdimm = PC_NVDIMM(obj); + uint64_t value = nvdimm->size; + visit_type_size(v, &value, name, errp); +} + +static void pc_nvdimm_set_size(Object *obj, Visitor *v, void *opaque, + const char *name, Error **errp) +{ + PCNVDIMMDevice *nvdimm = PC_NVDIMM(obj); + Error *local_err = NULL; + uint64_t value; + + if (memory_region_size(&nvdimm->mr)) { + error_setg(&local_err, "cannot change property value"); + goto out; + } + + visit_type_size(v, &value, name, &local_err); + if (local_err) { + goto out; + } + if (!value) { + error_setg(&local_err, "Property '%s.%s' doesn't take value '%" + PRIu64 "'", object_get_typename(obj), name, value); + goto out; + } + nvdimm->size = value << 20; + out: + error_propagate(errp, local_err); +} + +static char *pc_nvdimm_get_file(Object *obj, Error **errp) +{ + PCNVDIMMDevice *nvdimm = PC_NVDIMM(obj); + return g_strdup(nvdimm->file); +} + +static void pc_nvdimm_set_file(Object *obj, const char *str, Error **errp) +{ + PCNVDIMMDevice *nvdimm = PC_NVDIMM(obj); + if (nvdimm->file) { + g_free(nvdimm->file); + } + nvdimm->file = g_strdup(str); +} + +static void pc_nvdimm_init(Object *obj) +{ + object_property_add(obj, PC_NVDIMM_ADDR_PROP, "int", pc_nvdimm_get_addr, + NULL, NULL, NULL, &error_abort); + object_property_add(obj, PC_NVDIMM_SLOT_PROP, "int", pc_nvdimm_get_slot, + NULL, NULL, NULL, &error_abort); + object_property_add(obj, PC_NVDIMM_NODE_PROP, "int", pc_nvdimm_get_node, + NULL, NULL, NULL, &error_abort); + object_property_add(obj, PC_NVDIMM_SIZE_PROP, "int", pc_nvdimm_get_size, + pc_nvdimm_set_size, NULL, NULL, &error_abort); + object_property_add_str(obj, PC_NVDIMM_FILE_PROP, + pc_nvdimm_get_file, pc_nvdimm_set_file, NULL); +} + +static void pc_nvdimm_realize(DeviceState *dev, Error **errp) +{ + PCNVDIMMDevice *nvdimm = PC_NVDIMM(dev); + MemoryRegion *nvdimm_mr = &nvdimm->mr; + char name[13] = { 0 }; + void *buf; + ram_addr_t addr; + uint64_t size = nvdimm->size; + int fd; + + if (!xen_enabled()) { + error_setg(errp, "xen is not enabled"); + return; + } + + if (!nvdimm->file) { + error_setg(errp, "file property is not set"); + return; + } + if (!size) { + error_setg(errp, "size property is not set"); + return; + } + + fd = open(nvdimm->file, O_RDWR); + if (fd < 0) { + error_setg(errp, "can not open %s", nvdimm->file); + return; + } + + buf = qemu_ram_mmap(fd, size, PC_NVDIMM_ADDR_ALIGN, true); + if (buf == MAP_FAILED) { + error_setg(errp, "can not do mmap on %s", nvdimm->file); + goto do_close; + } + + addr = pc_nvdimm_reserved_range_push(size); + if (!addr) { + error_setg(errp, "do not have enough space for size %#lx.\n", size); + goto do_unmap; + } + nvdimm->addr = addr; + + nvdimm->dev_idx = pc_nvdimm_new_device_index(); + sprintf(name, "xen.nvdimm%02x", nvdimm->dev_idx); + memory_region_init_ram_ptr(nvdimm_mr, NULL, name, size, buf); + vmstate_register_ram(nvdimm_mr, DEVICE(dev)); + memory_region_add_subregion(get_system_memory(), addr, nvdimm_mr); + + return; + + do_unmap: + qemu_ram_munmap(buf, size); + do_close: + close(fd); +} + +static void pc_nvdimm_class_init(ObjectClass *oc, void *data) +{ + DeviceClass *dc = DEVICE_CLASS(oc); + + /* nvdimm hotplug has not been supported yet. */ + dc->hotpluggable = false; + + dc->realize = pc_nvdimm_realize; + dc->desc = "NVDIMM memory module"; +} + +static TypeInfo pc_nvdimm_info = { + .name = TYPE_PC_NVDIMM, + .parent = TYPE_DEVICE, + .instance_size = sizeof(PCNVDIMMDevice), + .instance_init = pc_nvdimm_init, + .class_init = pc_nvdimm_class_init, +}; + +static void pc_nvdimm_register_types(void) +{ + type_register_static(&pc_nvdimm_info); +} + +type_init(pc_nvdimm_register_types) diff --git a/include/hw/mem/pc-nvdimm.h b/include/hw/mem/pc-nvdimm.h new file mode 100644 index 0000000..797ea02 --- /dev/null +++ b/include/hw/mem/pc-nvdimm.h @@ -0,0 +1,49 @@ +/* + * NVDIMM (A Non-Volatile Dual In-line Memory Module) Virtualization Implement + * + * Copyright(C) 2015 Intel Corporation. + * + * Author: + * Xiao Guangrong + * Haozhong Zhang + * + * This work is licensed under the terms of the GNU GPL, version 2 or later. + * See the COPYING file in the top-level directory. + */ + +#ifndef QEMU_PC_NVDIMM_H +#define QEMU_PC_NVDIMM_H + +#include "hw/qdev.h" +#include "hw/mem/pc-dimm.h" +#include "exec/memory.h" + +/* Xen is incompatible with memory management of pc-dimm, + * so fallback to a standalone device type and manage memory + * by itself. + */ + +typedef struct PCNVDIMMDevice { + /* private */ + DeviceState parent_obj; + + char *file; + MemoryRegion mr; + uint64_t addr; + uint64_t size; + int dev_idx; +} PCNVDIMMDevice; + +#define TYPE_PC_NVDIMM "pc-nvdimm" +#define PC_NVDIMM(obj) \ + OBJECT_CHECK(PCNVDIMMDevice, (obj), TYPE_PC_NVDIMM) + +#define PC_NVDIMM_ADDR_PROP PC_DIMM_ADDR_PROP +#define PC_NVDIMM_SLOT_PROP PC_DIMM_SLOT_PROP +#define PC_NVDIMM_NODE_PROP PC_DIMM_NODE_PROP +#define PC_NVDIMM_SIZE_PROP PC_DIMM_SIZE_PROP +#define PC_NVDIMM_FILE_PROP "file" + +void pc_nvdimm_reserve_range(ram_addr_t offset); + +#endif diff --git a/xen-hvm.c b/xen-hvm.c index 3d78a0c..6ebf43f 100644 --- a/xen-hvm.c +++ b/xen-hvm.c @@ -236,6 +236,8 @@ static void xen_ram_init(PCMachineState *pcms, pcms->above_4g_mem_size); memory_region_add_subregion(sysmem, 0x100000000ULL, &ram_hi); } + + pc_nvdimm_reserve_range((1ULL << 32) + pcms->above_4g_mem_size); } void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr)