From patchwork Fri Jan 29 10:45:14 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yu Zhang X-Patchwork-Id: 8162191 Return-Path: X-Original-To: patchwork-xen-devel@patchwork.kernel.org Delivered-To: patchwork-parsemail@patchwork2.web.kernel.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.136]) by patchwork2.web.kernel.org (Postfix) with ESMTP id EECC3BEEE5 for ; Fri, 29 Jan 2016 10:56:21 +0000 (UTC) Received: from mail.kernel.org (localhost [127.0.0.1]) by mail.kernel.org (Postfix) with ESMTP id A5AE820389 for ; Fri, 29 Jan 2016 10:56:20 +0000 (UTC) Received: from lists.xen.org (lists.xenproject.org [50.57.142.19]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 646D42038A for ; Fri, 29 Jan 2016 10:56:19 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=lists.xen.org) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aP6gY-0004DC-8O; Fri, 29 Jan 2016 10:53:50 +0000 Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1aP6gW-0004CO-P4 for xen-devel@lists.xen.org; Fri, 29 Jan 2016 10:53:48 +0000 Received: from [85.158.139.211] by server-2.bemta-5.messagelabs.com id F8/A8-21594-CB44BA65; Fri, 29 Jan 2016 10:53:48 +0000 X-Env-Sender: yu.c.zhang@linux.intel.com X-Msg-Ref: server-12.tower-206.messagelabs.com!1454064826!19007703!1 X-Originating-IP: [192.55.52.88] X-SpamReason: No, hits=0.0 required=7.0 tests=sa_preprocessor: VHJ1c3RlZCBJUDogMTkyLjU1LjUyLjg4ID0+IDM3NDcyNQ==\n X-StarScan-Received: X-StarScan-Version: 7.35.1; banners=-,-,- X-VirusChecked: Checked Received: (qmail 14827 invoked from network); 29 Jan 2016 10:53:47 -0000 Received: from mga01.intel.com (HELO mga01.intel.com) (192.55.52.88) by server-12.tower-206.messagelabs.com with SMTP; 29 Jan 2016 10:53:47 -0000 Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP; 29 Jan 2016 02:53:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.22,363,1449561600"; d="scan'208";a="871601774" Received: from zhangyu-xengt.bj.intel.com ([10.238.157.63]) by orsmga001.jf.intel.com with ESMTP; 29 Jan 2016 02:53:43 -0800 From: Yu Zhang To: xen-devel@lists.xen.org Date: Fri, 29 Jan 2016 18:45:14 +0800 Message-Id: <1454064314-7799-4-git-send-email-yu.c.zhang@linux.intel.com> X-Mailer: git-send-email 1.9.1 In-Reply-To: <1454064314-7799-1-git-send-email-yu.c.zhang@linux.intel.com> References: <1454064314-7799-1-git-send-email-yu.c.zhang@linux.intel.com> Cc: kevin.tian@intel.com, keir@xen.org, ian.campbell@citrix.com, stefano.stabellini@eu.citrix.com, andrew.cooper3@citrix.com, ian.jackson@eu.citrix.com, Paul.Durrant@citrix.com, zhiyuan.lv@intel.com, jbeulich@suse.com, wei.liu2@citrix.com Subject: [Xen-devel] [PATCH v3 3/3] tools: introduce parameter max_wp_ram_ranges. X-BeenThere: xen-devel@lists.xen.org X-Mailman-Version: 2.1.13 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00, RCVD_IN_DNSWL_MED, UNPARSEABLE_RELAY autolearn=unavailable version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on mail.kernel.org X-Virus-Scanned: ClamAV using ClamSMTP A new parameter - max_wp_ram_ranges is added to set the upper limit of write-protected ram ranges to be tracked inside one ioreq server rangeset. Ioreq server uses a group of rangesets to track the I/O or memory resources to be emulated. Default limit of ranges that one rangeset can allocate is set to a small value, due to the fact that these ranges are allocated in xen heap. Yet for the write-protected ram ranges, there are circumstances under which the upper limit inside one rangeset should exceed the default one. E.g. in XenGT, when tracking the PPGTTs( per-process graphic translation tables) on Intel BDW platforms, number of page tables concerned will be of several thousand. For XenGT runing on Intel BDW platform, 8192 is a suggested value for this parameter in most cases. But users who set his item explicitly are also supposed to know the specific scenarios that necessitate this configuration. Especially when this parameter is used: 1> for virtual devices other than vGPUs in XenGT; 2> for XenGT, there also might be some extreme cases, e.g. too many graphic related applications in one VM, which create a great deal of per-process graphic translation tables; 3> for XenGT, future cpu platforms which provide even more per-process graphic translation tables. Signed-off-by: Yu Zhang --- docs/man/xl.cfg.pod.5 | 26 ++++++++++++++++++++++++++ tools/libxl/libxl.h | 5 +++++ tools/libxl/libxl_dom.c | 3 +++ tools/libxl/libxl_types.idl | 1 + tools/libxl/xl_cmdimpl.c | 17 +++++++++++++++++ xen/arch/x86/hvm/hvm.c | 7 ++++++- xen/include/public/hvm/params.h | 5 ++++- 7 files changed, 62 insertions(+), 2 deletions(-) diff --git a/docs/man/xl.cfg.pod.5 b/docs/man/xl.cfg.pod.5 index 8899f75..c294fd3 100644 --- a/docs/man/xl.cfg.pod.5 +++ b/docs/man/xl.cfg.pod.5 @@ -962,6 +962,32 @@ FIFO-based event channel ABI support up to 131,071 event channels. Other guests are limited to 4095 (64-bit x86 and ARM) or 1023 (32-bit x86). +=item B + +Limit the maximum write-protected ram ranges that can be tracked +inside one ioreq server rangeset. + +Ioreq server uses a group of rangesets to track the I/O or memory +resources to be emulated. Default limit of ranges that one rangeset +can allocate is set to a small value, due to the fact that these +ranges are allocated in xen heap. Yet for the write-protected ram +ranges, there are circumstances under which the upper limit inside +one rangeset should exceed the default one. E.g. in Intel GVT-g, +when tracking the PPGTT(per-process graphic translation tables) on +Intel broadwell platforms, the number of page tables concerned will +be of several thousand. + +For Intel GVT-g broadwell platform, 8192 is a suggested value for +this parameter in most cases. But users who set his item explicitly +are also supposed to know the specific scenarios that necessitate +this configuration. Especially when this parameter is used: +1> for virtual devices other than vGPU in GVT-g; +2> for GVT-g, there also might be some extreme cases, e.g. too many +graphic related applications in one VM, which create a great deal of +per-process graphic translation tables; +3> for GVT-g, future cpu platforms which provide even more per-process +graphic translation tables. + =back =head2 Paravirtualised (PV) Guest Specific Options diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index fa87f53..18828c5 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -136,6 +136,11 @@ #define LIBXL_HAVE_BUILDINFO_EVENT_CHANNELS 1 /* + * libxl_domain_build_info has the u.hvm.max_wp_ram_ranges field. + */ +#define LIBXL_HAVE_BUILDINFO_HVM_MAX_WP_RAM_RANGES 1 + +/* * libxl_domain_build_info has the u.hvm.ms_vm_genid field. */ #define LIBXL_HAVE_BUILDINFO_HVM_MS_VM_GENID 1 diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index 2269998..54173cb 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -288,6 +288,9 @@ static void hvm_set_conf_params(xc_interface *handle, uint32_t domid, libxl_defbool_val(info->u.hvm.nested_hvm)); xc_hvm_param_set(handle, domid, HVM_PARAM_ALTP2M, libxl_defbool_val(info->u.hvm.altp2m)); + if (info->u.hvm.max_wp_ram_ranges > 0) + xc_hvm_param_set(handle, domid, HVM_PARAM_MAX_WP_RAM_RANGES, + info->u.hvm.max_wp_ram_ranges); } int libxl__build_pre(libxl__gc *gc, uint32_t domid, diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index 9ad7eba..9185014 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -518,6 +518,7 @@ libxl_domain_build_info = Struct("domain_build_info",[ ("serial_list", libxl_string_list), ("rdm", libxl_rdm_reserve), ("rdm_mem_boundary_memkb", MemKB), + ("max_wp_ram_ranges", uint64), ])), ("pv", Struct(None, [("kernel", string), ("slack_memkb", MemKB), diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 25507c7..0c19dee 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -35,6 +35,7 @@ #include #include #include +#include #include "libxl.h" #include "libxl_utils.h" @@ -1626,6 +1627,22 @@ static void parse_config_data(const char *config_source, if (!xlu_cfg_get_long (config, "rdm_mem_boundary", &l, 0)) b_info->u.hvm.rdm_mem_boundary_memkb = l * 1024; + + if (!xlu_cfg_get_long (config, "max_wp_ram_ranges", &l, 0)) { + uint64_t nr_pages = (b_info->max_memkb << 10) >> XC_PAGE_SHIFT; + + /* Due to rangeset's ability to combine continuous ranges, this + * parameter shall not be configured with values greater than half + * of the number of VM's page frames. It also shall not exceed 4G, + * because of the limitation from the rangeset side. */ + if (l > (nr_pages / 2) || l > UINT32_MAX) { + fprintf(stderr, "ERROR: Invalid value for \"max_wp_ram_ranges\". " + "Shall not exceed %ld or 4G.\n", nr_pages / 2); + exit(1); + } + b_info->u.hvm.max_wp_ram_ranges = l; + } + break; case LIBXL_DOMAIN_TYPE_PV: { diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index e0d998f..7339833 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -940,6 +940,8 @@ static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s, { unsigned int i; int rc; + unsigned int max_wp_ram_ranges = + s->domain->arch.hvm_domain.params[HVM_PARAM_MAX_WP_RAM_RANGES]; if ( is_default ) goto done; @@ -962,7 +964,10 @@ static int hvm_ioreq_server_alloc_rangesets(struct hvm_ioreq_server *s, if ( !s->range[i] ) goto fail; - rangeset_limit(s->range[i], MAX_NR_IO_RANGES); + rangeset_limit(s->range[i], + ((i == HVMOP_IO_RANGE_WP_MEM) ? + max_wp_ram_ranges : + MAX_NR_IO_RANGES)); } done: diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h index 81f9451..ab3b11d 100644 --- a/xen/include/public/hvm/params.h +++ b/xen/include/public/hvm/params.h @@ -210,6 +210,9 @@ /* Boolean: Enable altp2m */ #define HVM_PARAM_ALTP2M 35 -#define HVM_NR_PARAMS 36 +/* Max write-protected ram ranges to be tracked in one ioreq server rangeset */ +#define HVM_PARAM_MAX_WP_RAM_RANGES 36 + +#define HVM_NR_PARAMS 37 #endif /* __XEN_PUBLIC_HVM_PARAMS_H__ */