From patchwork Fri Sep 13 09:47:38 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Paul Durrant X-Patchwork-Id: 11144293 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 787D2912 for ; Fri, 13 Sep 2019 09:49:24 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4740520717 for ; Fri, 13 Sep 2019 09:49:24 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=citrix.com header.i=@citrix.com header.b="B1pkQRD4" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4740520717 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i8iBK-0002GW-64; Fri, 13 Sep 2019 09:47:58 +0000 Received: from us1-rack-iad1.inumbo.com ([172.99.69.81]) by lists.xenproject.org with esmtp (Exim 4.89) (envelope-from ) id 1i8iBI-0002FB-Iz for xen-devel@lists.xenproject.org; Fri, 13 Sep 2019 09:47:56 +0000 X-Inumbo-ID: 8af63b62-d60b-11e9-a337-bc764e2007e4 Received: from esa3.hc3370-68.iphmx.com (unknown [216.71.145.155]) by us1-rack-iad1.inumbo.com (Halon) with ESMTPS id 8af63b62-d60b-11e9-a337-bc764e2007e4; Fri, 13 Sep 2019 09:47:47 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1568368068; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fgoZLPrXZJnKRh1u8ZJ6ZzZF9exZGd7aGvwLwgUkfZU=; b=B1pkQRD49U3c0geh4cQLIo5ihs7QoKJXGjcPHc2G6CuW4Cz4okS0KRfC 92nI2zUc33iHKm0xO7cYrYI1RaSOkyQRGMGuUzK5NQtcDmv2B70RuZq9k WQr0s0k89ZDh5gx7E38EFYJF6UNJ+NY1F1p6Mfy1JMn3F/dBZWdN0XKbD M=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none; spf=None smtp.pra=paul.durrant@citrix.com; spf=Pass smtp.mailfrom=Paul.Durrant@citrix.com; spf=None smtp.helo=postmaster@mail.citrix.com Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of paul.durrant@citrix.com) identity=pra; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Paul.Durrant@citrix.com"; x-sender="paul.durrant@citrix.com"; x-conformance=sidf_compatible Received-SPF: Pass (esa3.hc3370-68.iphmx.com: domain of Paul.Durrant@citrix.com designates 162.221.158.21 as permitted sender) identity=mailfrom; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Paul.Durrant@citrix.com"; x-sender="Paul.Durrant@citrix.com"; x-conformance=sidf_compatible; x-record-type="v=spf1"; x-record-text="v=spf1 ip4:209.167.231.154 ip4:178.63.86.133 ip4:195.66.111.40/30 ip4:85.115.9.32/28 ip4:199.102.83.4 ip4:192.28.146.160 ip4:192.28.146.107 ip4:216.52.6.88 ip4:216.52.6.188 ip4:162.221.158.21 ip4:162.221.156.83 ~all" Received-SPF: None (esa3.hc3370-68.iphmx.com: no sender authenticity information available from domain of postmaster@mail.citrix.com) identity=helo; client-ip=162.221.158.21; receiver=esa3.hc3370-68.iphmx.com; envelope-from="Paul.Durrant@citrix.com"; x-sender="postmaster@mail.citrix.com"; x-conformance=sidf_compatible IronPort-SDR: vxyJwyRp6Rc6K+Czwh7XatPkvyUCu+eT2ReLlltwdHb/wPQKWD99gJULGH8pQhFAWEVAD7WlQl nQnWLAPRZxlloJ5HIpeJ4kOyhM0iSsmG0u6NmJU0ZOg6zf/3+LAWfTx6/ngv12t0X8UcXNx04S wExtCyO+p4htiQ3M6vCgHHU6W/CjEYsu0q+IbS7DMDK5pTPrkwNwGNB5k/sbv4HBWnSTmBZatU qXlytDz+3c6SQehUc2rNK1B+/uVTdXaG86pWpe6UGMe+EP0FQ3TUwr3cotOejNAUHmpCzTkzu1 CKQ= X-SBRS: 2.7 X-MesageID: 5528290 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.64,500,1559534400"; d="scan'208";a="5528290" From: Paul Durrant To: Date: Fri, 13 Sep 2019 10:47:38 +0100 Message-ID: <20190913094741.31451-4-paul.durrant@citrix.com> X-Mailer: git-send-email 2.20.1.2.gb21ebb671 In-Reply-To: <20190913094741.31451-1-paul.durrant@citrix.com> References: <20190913094741.31451-1-paul.durrant@citrix.com> MIME-Version: 1.0 Subject: [Xen-devel] [PATCH v10 3/6] sysctl / libxl: report whether IOMMU/HAP page table sharing is supported X-BeenThere: xen-devel@lists.xenproject.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Cc: Kevin Tian , Stefano Stabellini , Suravee Suthikulpanit , Wei Liu , Konrad Rzeszutek Wilk , George Dunlap , Andrew Cooper , Ian Jackson , Tim Deegan , Julien Grall , Paul Durrant , Christian Lindig , Jan Beulich , David Scott , Anthony PERARD , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Errors-To: xen-devel-bounces@lists.xenproject.org Sender: "Xen-devel" This patch defines a new bit reported in the hw_cap field of struct xen_sysctl_physinfo to indicate whether the platform supports sharing of HAP page tables (i.e. the P2M) with the IOMMU. This informs the toolstack whether the domain needs extra memory to store discrete IOMMU page tables or not. NOTE: This patch makes sure iommu_hap_pt_shared is clear if HAP is not supported or the IOMMU is disabled, and defines it to false if !CONFIG_HVM. Signed-off-by: Paul Durrant Acked-by: Christian Lindig Acked-by: Wei Liu --- Cc: Ian Jackson Cc: Wei Liu Cc: Anthony PERARD Cc: Andrew Cooper Cc: George Dunlap Cc: Jan Beulich Cc: Julien Grall Cc: Konrad Rzeszutek Wilk Cc: Stefano Stabellini Cc: Tim Deegan Cc: Christian Lindig Cc: David Scott Cc: "Roger Pau Monné" Cc: Suravee Suthikulpanit Cc: Kevin Tian v10: - Set flag in common code (which means clearing iommu_hap_pt_share if HAP cannot be enabled or is configured out). v9: - New in v9 --- tools/libxl/libxl.c | 2 ++ tools/libxl/libxl.h | 7 +++++++ tools/libxl/libxl_types.idl | 1 + tools/ocaml/libs/xc/xenctrl.ml | 1 + tools/ocaml/libs/xc/xenctrl.mli | 3 ++- tools/xl/xl_info.c | 5 +++-- xen/arch/x86/hvm/hvm.c | 25 +++++++++++++++++------- xen/common/sysctl.c | 2 ++ xen/drivers/passthrough/amd/iommu_init.c | 5 ++++- xen/drivers/passthrough/iommu.c | 18 ++++++++++++++++- xen/drivers/passthrough/vtd/iommu.c | 6 +++++- xen/include/public/sysctl.h | 6 +++++- xen/include/xen/iommu.h | 8 +++++++- 13 files changed, 74 insertions(+), 15 deletions(-) diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 57073c06d5..a0d84281d0 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -402,6 +402,8 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *physinfo) physinfo->cap_hap = !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_hap); physinfo->cap_shadow = !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_shadow); + physinfo->cap_iommu_hap_pt_share = + !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_iommu_hap_pt_share); GC_FREE; return 0; diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index 466df2cdf5..8169d44bda 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -401,6 +401,13 @@ */ #define LIBXL_HAVE_PHYSINFO_CAP_HAP_SHADOW 1 +/* + * LIBXL_HAVE_PHYSINFO_CAP_IOMMU_HAP_PT_SHARE indicates that libxl_physinfo + * has a cap_iommu_hap_pt_share field that indicates whether the hardware + * supports sharing the IOMMU and HAP page tables. + */ +#define LIBXL_HAVE_PHYSINFO_CAP_IOMMU_HAP_PT_SHARE 1 + /* * libxl ABI compatibility * diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index 6f431baec2..7253d6e0fb 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -1027,6 +1027,7 @@ libxl_physinfo = Struct("physinfo", [ ("cap_hvm_directio", bool), # No longer HVM specific ("cap_hap", bool), ("cap_shadow", bool), + ("cap_iommu_hap_pt_share", bool), ], dir=DIR_OUT) libxl_connectorinfo = Struct("connectorinfo", [ diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml index 35dddbbd9c..de4bae6012 100644 --- a/tools/ocaml/libs/xc/xenctrl.ml +++ b/tools/ocaml/libs/xc/xenctrl.ml @@ -110,6 +110,7 @@ type physinfo_cap_flag = | CAP_DirectIO | CAP_HAP | CAP_Shadow + | CAP_IOMMU_HAP_PT_SHARE type physinfo = { diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli index 0dd55e9d8b..c885e75895 100644 --- a/tools/ocaml/libs/xc/xenctrl.mli +++ b/tools/ocaml/libs/xc/xenctrl.mli @@ -57,7 +57,6 @@ type domain_create_flag = | CDF_OOS_OFF | CDF_XS_DOMAIN | CDF_IOMMU - type domctl_create_config = { ssidref: int32; handle: string; @@ -95,6 +94,8 @@ type physinfo_cap_flag = | CAP_DirectIO | CAP_HAP | CAP_Shadow + | CAP_IOMMU_HAP_PT_SHARE + type physinfo = { threads_per_core : int; cores_per_socket : int; diff --git a/tools/xl/xl_info.c b/tools/xl/xl_info.c index 148c4740ae..bfbca93997 100644 --- a/tools/xl/xl_info.c +++ b/tools/xl/xl_info.c @@ -210,13 +210,14 @@ static void output_physinfo(void) info.hw_cap[4], info.hw_cap[5], info.hw_cap[6], info.hw_cap[7] ); - maybe_printf("virt_caps :%s%s%s%s%s%s\n", + maybe_printf("virt_caps :%s%s%s%s%s%s%s\n", info.cap_pv ? " pv" : "", info.cap_hvm ? " hvm" : "", info.cap_hvm && info.cap_hvm_directio ? " hvm_directio" : "", info.cap_pv && info.cap_hvm_directio ? " pv_directio" : "", info.cap_hap ? " hap" : "", - info.cap_shadow ? " shadow" : "" + info.cap_shadow ? " shadow" : "", + info.cap_iommu_hap_pt_share ? " iommu_hap_pt_share" : "" ); vinfo = libxl_get_version_info(ctx); diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 3831c6d4c1..ed8272cf93 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -142,6 +142,22 @@ static struct notifier_block cpu_nfb = { .notifier_call = cpu_callback }; +static bool __init hap_supported(const struct hvm_function_table *fns) +{ + if ( !fns->hap_supported ) + { + printk("HVM: Hardware Assisted Paging (HAP) not detected\n"); + return false; + } + else if ( !opt_hap_enabled ) + { + printk("HVM: Hardware Assisted Paging (HAP) detected but disabled\n"); + return false; + } + + return true; +} + static int __init hvm_enable(void) { const struct hvm_function_table *fns = NULL; @@ -158,13 +174,8 @@ static int __init hvm_enable(void) hvm_enabled = 1; printk("HVM: %s enabled\n", fns->name); - if ( !fns->hap_supported ) - printk("HVM: Hardware Assisted Paging (HAP) not detected\n"); - else if ( !opt_hap_enabled ) - { - hvm_funcs.hap_supported = 0; - printk("HVM: Hardware Assisted Paging (HAP) detected but disabled\n"); - } + if ( !hap_supported(fns) ) + iommu_hap_pt_share = false; else { printk("HVM: Hardware Assisted Paging (HAP) detected\n"); diff --git a/xen/common/sysctl.c b/xen/common/sysctl.c index 92b4ea0d21..ef840fcb76 100644 --- a/xen/common/sysctl.c +++ b/xen/common/sysctl.c @@ -269,6 +269,8 @@ long do_sysctl(XEN_GUEST_HANDLE_PARAM(xen_sysctl_t) u_sysctl) arch_do_physinfo(pi); if ( iommu_enabled ) pi->capabilities |= XEN_SYSCTL_PHYSCAP_directio; + if ( iommu_hap_pt_share ) + pi->capabilities |= XEN_SYSCTL_PHYSCAP_iommu_hap_pt_share; if ( copy_to_guest(u_sysctl, op, 1) ) ret = -EFAULT; diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c index bb5a3e57c9..2e011d4e43 100644 --- a/xen/drivers/passthrough/amd/iommu_init.c +++ b/xen/drivers/passthrough/amd/iommu_init.c @@ -1401,12 +1401,15 @@ int __init amd_iommu_init(bool xt) if ( rc ) goto error_out; +#ifndef iommu_hap_pt_share /* * Disable sharing HAP page tables with AMD IOMMU, * since it only supports p2m_ram_rw, and this would * prevent doing IO to/from mapped grant frames. */ - iommu_hap_pt_share = 0; + iommu_hap_pt_share = false; +#endif + printk(XENLOG_DEBUG "AMD-Vi: Disabled HAP memory map sharing with IOMMU\n"); /* per iommu initialization */ diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c index 09ce9d9294..bdf12f85e6 100644 --- a/xen/drivers/passthrough/iommu.c +++ b/xen/drivers/passthrough/iommu.c @@ -49,7 +49,11 @@ int8_t __hwdom_initdata iommu_hwdom_reserved = -1; * default until we find a good solution to resolve it. */ bool_t __read_mostly iommu_intpost; -bool_t __read_mostly iommu_hap_pt_share = 1; + +#ifndef iommu_hap_pt_share +bool __read_mostly iommu_hap_pt_share = true; +#endif + bool_t __read_mostly iommu_debug; bool_t __read_mostly amd_iommu_perdev_intremap = 1; @@ -103,7 +107,14 @@ static int __init parse_iommu_param(const char *s) else if ( (val = parse_boolean("dom0-strict", s, ss)) >= 0 ) iommu_hwdom_strict = val; else if ( (val = parse_boolean("sharept", s, ss)) >= 0 ) + { +#ifndef iommu_hap_pt_share iommu_hap_pt_share = val; +#else + if (val != iommu_hap_pt_share) + rc = -EINVAL; +#endif + } else rc = -EINVAL; @@ -511,7 +522,12 @@ int __init iommu_setup(void) iommu_enabled = (rc == 0); } if ( !iommu_enabled ) + { iommu_intremap = 0; +#ifndef iommu_hap_pt_share + iommu_hap_pt_share = false; +#endif + } if ( (force_iommu && !iommu_enabled) || (force_intremap && !iommu_intremap) ) diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c index 7ffafdc065..a7381a7449 100644 --- a/xen/drivers/passthrough/vtd/iommu.c +++ b/xen/drivers/passthrough/vtd/iommu.c @@ -1898,6 +1898,7 @@ int iommu_pte_flush(struct domain *d, uint64_t dfn, uint64_t *pte, return rc; } +#ifndef iommu_hap_pt_share static int __init vtd_ept_page_compatible(struct vtd_iommu *iommu) { u64 ept_cap, vtd_cap = iommu->cap; @@ -1910,6 +1911,7 @@ static int __init vtd_ept_page_compatible(struct vtd_iommu *iommu) return (ept_has_2mb(ept_cap) && opt_hap_2mb) == cap_sps_2mb(vtd_cap) && (ept_has_1gb(ept_cap) && opt_hap_1gb) == cap_sps_1gb(vtd_cap); } +#endif /* * set VT-d page table directory to EPT table if allowed @@ -2309,8 +2311,10 @@ static int __init vtd_setup(void) if ( !cap_intr_post(iommu->cap) || !iommu_intremap || !cpu_has_cx16 ) iommu_intpost = 0; +#ifndef iommu_hap_pt_share if ( !vtd_ept_page_compatible(iommu) ) - iommu_hap_pt_share = 0; + iommu_hap_pt_share = false; +#endif ret = iommu_set_interrupt(drhd); if ( ret ) diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h index e324442f92..2be013de5b 100644 --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -96,9 +96,13 @@ struct xen_sysctl_tbuf_op { /* The platform supports software paging. */ #define _XEN_SYSCTL_PHYSCAP_shadow 4 #define XEN_SYSCTL_PHYSCAP_shadow (1u<<_XEN_SYSCTL_PHYSCAP_shadow) +/* The platform supports sharing of HAP page tables with the IOMMU. */ +#define _XEN_SYSCTL_PHYSCAP_iommu_hap_pt_share 5 +#define XEN_SYSCTL_PHYSCAP_iommu_hap_pt_share \ + (1u << _XEN_SYSCTL_PHYSCAP_iommu_hap_pt_share) /* Max XEN_SYSCTL_PHYSCAP_* constant. Used for ABI checking. */ -#define XEN_SYSCTL_PHYSCAP_MAX XEN_SYSCTL_PHYSCAP_shadow +#define XEN_SYSCTL_PHYSCAP_MAX XEN_SYSCTL_PHYSCAP_hap_pt_share struct xen_sysctl_physinfo { uint32_t threads_per_core; diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h index ab258b848b..a519f4d87b 100644 --- a/xen/include/xen/iommu.h +++ b/xen/include/xen/iommu.h @@ -55,7 +55,13 @@ static inline bool_t dfn_eq(dfn_t x, dfn_t y) extern bool_t iommu_enable, iommu_enabled; extern bool_t force_iommu, iommu_verbose, iommu_igfx; extern bool_t iommu_snoop, iommu_qinval, iommu_intremap, iommu_intpost; -extern bool_t iommu_hap_pt_share; + +#ifdef CONFIG_HVM +extern bool iommu_hap_pt_share; +#else +#define iommu_hap_pt_share false +#endif + extern bool_t iommu_debug; extern bool_t amd_iommu_perdev_intremap;