From patchwork Mon Feb 7 18:21:00 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jane Malalane X-Patchwork-Id: 12737804 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D5C21C433EF for ; Mon, 7 Feb 2022 18:22:07 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.267364.461123 (Exim 4.92) (envelope-from ) id 1nH8eA-0004FS-7L; Mon, 07 Feb 2022 18:21:54 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 267364.461123; Mon, 07 Feb 2022 18:21:54 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nH8eA-0004FL-3k; Mon, 07 Feb 2022 18:21:54 +0000 Received: by outflank-mailman (input) for mailman id 267364; Mon, 07 Feb 2022 18:21:52 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nH8e8-0003rG-Ei for xen-devel@lists.xenproject.org; Mon, 07 Feb 2022 18:21:52 +0000 Received: from esa5.hc3370-68.iphmx.com (esa5.hc3370-68.iphmx.com [216.71.155.168]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id d0ddd19b-8842-11ec-8eb8-a37418f5ba1a; Mon, 07 Feb 2022 19:21:51 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: d0ddd19b-8842-11ec-8eb8-a37418f5ba1a DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1644258110; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=GkKTcTZiNh1+hb7yPpxCjXnkYET2lQ1dJBUO9p9Q0d4=; b=eK5QpiHEv8SLEqnE2qPl4KC0mSqfMYcAcqGMWMEPW0b1EbgsiqkXGLr+ 32njOdVzi0gKz129o3DqVlKrE0GCHCUrFALIDKkRLVhPNnk35LYtvTMLI 5op+xA+14346U7ceEW0nidYMjMvuD0xUZO3ku8GRsVZ+vsDoNT2skH/9X w=; Authentication-Results: esa5.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: SBh0kkoPNRTFTZT3XZMTM2U0+puMCbu/KfgIASf2AzexXFRgdVMJTaUGebOjzZVnZl1WtrlecT 54apB2MRI8OsYcFhyWCIms501NM3CxF/SkyWgpD0yicx3LPai334Rpyd/qpdVJSoZfylG0zeaM olz6RPAqku+Ifwxx6Y5m6eKUU3IHhJwWvDD7Z4gAbG2aFg9q82uipiwf+BdloJ+C7lJJar6Oyn gvZHNsjP9ZWzeVLVDZiCgVPZWGTsUmCl2TKrDBhD9IhSPhrlLT2nZzwxV/0HpCCSDqV9bBspHK g1ZgyWc8kQVwxbRLLB5ptsSr X-SBRS: 5.1 X-MesageID: 63127623 X-Ironport-Server: esa5.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.83 X-Policy: $RELAYED IronPort-Data: A9a23:eWMKL6hCas00JmqtA68UjXaTX161BxYKZh0ujC45NGQN5FlHY01je htvXWCAPv+KMDSgLYt1aNywo08F6MCAztI1HFRsrn1jRC4b9cadCdqndUqhZCn6wu8v7a5EA 2fyTvGacajYm1eF/k/F3oAMKRCQ7InQLlbGILes1htZGEk0GE/NtTo5w7Rj2tQw3oDja++wk YiaT/P3aQfNNwFcagr424rbwP+4lK2v0N+wlgVWicFj5DcypVFMZH4sDfjZw0/DaptVBoaHq 9Prl9lVyI97EyAFUbtJmp6jGqEDryW70QKm0hK6UID66vROS7BbPg/W+5PwZG8O4whlkeydx /1k65+MWysDGpTXu+4kX0h3L31SJa1JreqvzXiX6aR/zmXDenrohf5vEFs3LcsT/eMf7WNmr KJCbmpXN1ba2rzwkOnTpupE36zPKOHJNYUS/FRpyTjdBPAraZvCX7/L9ZlT2zJYasVmQ62HP ZVHM2EHgBLoT0EIBV5QUKgFwKSXnyLHShparn6euv9ii4TU5FMoi+W8WDbPQfS0QsFSkledt 3jx1W3zCREHN/SS0TOAtHmrg4fnhjjnUYgfELm58P9Cg1CJwGEXThoMWjOTq/SjllS3Xd4ZL kUO4zcvtoA77kntRd74NzW7rWCFuFgAWtNWO+w89AyJjKHT5m6xBHUCCDhIa9Uktco/bT0sy lKN2djuAFRHsrKPTmmG3qyJtj70Mi8QRUcSaClBQQYb7t3LpIAokgmJXttlCLSyjND+BXf32 T/ihCo0iqgXjMUL/76m5l2BiDWpzrDLUwo06wP/Tm+jqARja+aNeIiA+VXdq/FaI+6kokKp5 SZe3ZLEtaZXUM/LxHflrPgx8K+BzqiGDC/wq3pWBLorzjj81EL/UZhQ72QrTKt2CfosdTjsa U7VnApe4p5PIXenBZNKj5KN59cClva5S4m8PhzARp8XO8UqKlfblM17TRPIhwjQfF4QfbbT0 HtxWeKlFj4kBKtu11JarM9NgOZwlkjSKY4+LK0XLihLM5LDPhZ5qp9faTNii9zVC4ve8W3oH y53bZfi9vmmeLSWjtPr2YASN0sWCnMwGIr7rcdaHsbafFY6QTB4UqGAm+p6E2CAo0iyvr2Zl kxRp2cCkAav7ZE5AVniho9fhEPHAs8k8CNT0d0ENle0wXkzCbtDH49EH6bbiYIPrbQ5pdYtF qFtU5zZXpxnF2SWkxxAPMKVhNEzK3yD21nUVwL7O2dXQnKVb1GQkjMSVlC0r3dm4+venZZWn oBMISuBHcNTHFozUJmKAB9tpnvo1UUgdCtJdxOgCrFulI/EqdACx/XZgqBlLscSBw/EwzfGh Q+aDQ1B/bvGopMv8cmPjqeB9t/7H+x7F0tcPm/a8bfpanWKojv9mddNALSSYDTQdGLo46H+N +9b+O7xba8cl1FQvosiT7sylfAi58HirqNxxxh/GCmZdEyiD75teyHU3cRGuqBX6KVevA+6B hCG9tVAYO3bM8L5Cl8BYgEia73bh/0TnzDT69UzIVn7u3ArrObWDx0KMkDV2iJHLbZzPIc0+ ssbuZYbu16llx4nEtealSQIpW6CGWMNDvc8vZYADY611gdykgNeYYbRAzPd6Y2Ub4keKVEjJ zKZifaQh7lYwUaeIXM/GWKUgLhYjJUK/htL0EUDNxKCnd+c3q072xhY8DIWSAVJz0oYj7Iva zYzb0Ald7+T+zpIhdRYWzH+Eg5MMxSV50jtxgZbj2beVUSpCjTAIWBV1TxhJ6zFH7awpgRmw Yw= IronPort-HdrOrdr: A9a23:fY9mpa8cBxsAlhz13r5uk+DgI+orL9Y04lQ7vn2YSXRuHPBw8P re5cjztCWE7gr5N0tBpTntAsW9qDbnhPtICOoqTNCftWvdyQiVxehZhOOIqVDd8m/Fh4pgPM 9bAtBD4bbLbGSS4/yU3ODBKadD/OW6 X-IronPort-AV: E=Sophos;i="5.88,350,1635220800"; d="scan'208";a="63127623" From: Jane Malalane To: Xen-devel CC: Jane Malalane , Wei Liu , "Anthony PERARD" , Juergen Gross , "Andrew Cooper" , George Dunlap , Jan Beulich , Julien Grall , "Stefano Stabellini" , Volodymyr Babchuk , Bertrand Marquis , Jun Nakajima , Kevin Tian , =?utf-8?q?Roger_Pau_Monn=C3=A9?= Subject: [PATCH v2 1/2] xen+tools: Report Interrupt Controller Virtualization capabilities on x86 Date: Mon, 7 Feb 2022 18:21:00 +0000 Message-ID: <20220207182101.31941-2-jane.malalane@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20220207182101.31941-1-jane.malalane@citrix.com> References: <20220207182101.31941-1-jane.malalane@citrix.com> MIME-Version: 1.0 Add XEN_SYSCTL_PHYSCAP_ARCH_ASSISTED_xapic and XEN_SYSCTL_PHYSCAP_ARCH_ASSISTED_x2apic to report accelerated xapic and x2apic, on x86 hardware. No such features are currently implemented on AMD hardware. For that purpose, also add an arch-specific "capabilities" parameter to struct xen_sysctl_physinfo. Signed-off-by: Jane Malalane Suggested-by: Andrew Cooper --- CC: Wei Liu CC: Anthony PERARD CC: Juergen Gross CC: Andrew Cooper CC: George Dunlap CC: Jan Beulich CC: Julien Grall CC: Stefano Stabellini CC: Volodymyr Babchuk CC: Bertrand Marquis CC: Jun Nakajima CC: Kevin Tian CC: "Roger Pau Monné" v2: * Use one macro LIBXL_HAVE_PHYSINFO_ASSISTED_APIC instead of two * Pass xcpyshinfo as a pointer in libxl__arch_get_physinfo * Set assisted_x{2}apic_available to be conditional upon "bsp" and annotate it with __ro_after_init * Change XEN_SYSCTL_PHYSCAP_ARCH_ASSISTED_X{2}APIC to .._X86_ASSISTED_X{2}APIC * Keep XEN_SYSCTL_PHYSCAP_X86_ASSISTED_X{2}APIC contained within sysctl.h * Fix padding introduced in struct xen_sysctl_physinfo and bump XEN_SYSCTL_INTERFACE_VERSION --- tools/golang/xenlight/helpers.gen.go | 4 ++++ tools/golang/xenlight/types.gen.go | 6 ++++++ tools/include/libxl.h | 7 +++++++ tools/libs/light/libxl.c | 3 +++ tools/libs/light/libxl_arch.h | 4 ++++ tools/libs/light/libxl_arm.c | 5 +++++ tools/libs/light/libxl_types.idl | 2 ++ tools/libs/light/libxl_x86.c | 11 +++++++++++ tools/ocaml/libs/xc/xenctrl.ml | 5 +++++ tools/ocaml/libs/xc/xenctrl.mli | 5 +++++ tools/xl/xl_info.c | 6 ++++-- xen/arch/x86/hvm/vmx/vmcs.c | 9 +++++++++ xen/arch/x86/include/asm/domain.h | 3 +++ xen/arch/x86/sysctl.c | 7 +++++++ xen/include/public/sysctl.h | 8 +++++++- 15 files changed, 82 insertions(+), 3 deletions(-) diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go index b746ff1081..dd4e6c9f14 100644 --- a/tools/golang/xenlight/helpers.gen.go +++ b/tools/golang/xenlight/helpers.gen.go @@ -3373,6 +3373,8 @@ x.CapVmtrace = bool(xc.cap_vmtrace) x.CapVpmu = bool(xc.cap_vpmu) x.CapGnttabV1 = bool(xc.cap_gnttab_v1) x.CapGnttabV2 = bool(xc.cap_gnttab_v2) +x.CapAssistedXapic = bool(xc.cap_assisted_xapic) +x.CapAssistedX2Apic = bool(xc.cap_assisted_x2apic) return nil} @@ -3407,6 +3409,8 @@ xc.cap_vmtrace = C.bool(x.CapVmtrace) xc.cap_vpmu = C.bool(x.CapVpmu) xc.cap_gnttab_v1 = C.bool(x.CapGnttabV1) xc.cap_gnttab_v2 = C.bool(x.CapGnttabV2) +xc.cap_assisted_xapic = C.bool(x.CapAssistedXapic) +xc.cap_assisted_x2apic = C.bool(x.CapAssistedX2Apic) return nil } diff --git a/tools/golang/xenlight/types.gen.go b/tools/golang/xenlight/types.gen.go index b1e84d5258..5f384b767c 100644 --- a/tools/golang/xenlight/types.gen.go +++ b/tools/golang/xenlight/types.gen.go @@ -389,6 +389,10 @@ RunHotplugScripts Defbool DriverDomain Defbool Passthrough Passthrough XendSuspendEvtchnCompat Defbool +ArchX86 struct { +AssistedXapic Defbool +AssistedX2Apic Defbool +} } type DomainRestoreParams struct { @@ -1014,6 +1018,8 @@ CapVmtrace bool CapVpmu bool CapGnttabV1 bool CapGnttabV2 bool +CapAssistedXApic bool +CapAssistedX2apic bool } type Connectorinfo struct { diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 2bbbd21f0b..924e142628 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -528,6 +528,13 @@ #define LIBXL_HAVE_MAX_GRANT_VERSION 1 /* + * LIBXL_HAVE_PHYSINFO_ASSISTED_APIC indicates that libxl_physinfo has + * cap_assisted_x{2}apic fields, which indicates the availability of x{2}APIC + * hardware assisted virtualization. + */ +#define LIBXL_HAVE_PHYSINFO_ASSISTED_APIC 1 + +/* * libxl ABI compatibility * * The only guarantee which libxl makes regarding ABI compatibility diff --git a/tools/libs/light/libxl.c b/tools/libs/light/libxl.c index 667ae6409b..fabb474221 100644 --- a/tools/libs/light/libxl.c +++ b/tools/libs/light/libxl.c @@ -15,6 +15,7 @@ #include "libxl_osdeps.h" #include "libxl_internal.h" +#include "libxl_arch.h" int libxl_ctx_alloc(libxl_ctx **pctx, int version, unsigned flags, xentoollog_logger * lg) @@ -410,6 +411,8 @@ int libxl_get_physinfo(libxl_ctx *ctx, libxl_physinfo *physinfo) physinfo->cap_gnttab_v2 = !!(xcphysinfo.capabilities & XEN_SYSCTL_PHYSCAP_gnttab_v2); + libxl__arch_get_physinfo(physinfo, &xcphysinfo); + GC_FREE; return 0; } diff --git a/tools/libs/light/libxl_arch.h b/tools/libs/light/libxl_arch.h index 1522ecb97f..207ceac6a1 100644 --- a/tools/libs/light/libxl_arch.h +++ b/tools/libs/light/libxl_arch.h @@ -86,6 +86,10 @@ int libxl__arch_extra_memory(libxl__gc *gc, uint64_t *out); _hidden +void libxl__arch_get_physinfo(libxl_physinfo *physinfo, + const xc_physinfo_t *xcphysinfo); + +_hidden void libxl__arch_update_domain_config(libxl__gc *gc, libxl_domain_config *dst, const libxl_domain_config *src); diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index eef1de0939..39fdca1b49 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -1431,6 +1431,11 @@ int libxl__arch_passthrough_mode_setdefault(libxl__gc *gc, return rc; } +void libxl__arch_get_physinfo(libxl_physinfo *physinfo, + const xc_physinfo_t *xcphysinfo) +{ +} + void libxl__arch_update_domain_config(libxl__gc *gc, libxl_domain_config *dst, const libxl_domain_config *src) diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl index 2a42da2f7d..42ac6c357b 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -1068,6 +1068,8 @@ libxl_physinfo = Struct("physinfo", [ ("cap_vpmu", bool), ("cap_gnttab_v1", bool), ("cap_gnttab_v2", bool), + ("cap_assisted_xapic", bool), + ("cap_assisted_x2apic", bool), ], dir=DIR_OUT) libxl_connectorinfo = Struct("connectorinfo", [ diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c index 1feadebb18..e0a06ecfe3 100644 --- a/tools/libs/light/libxl_x86.c +++ b/tools/libs/light/libxl_x86.c @@ -866,6 +866,17 @@ int libxl__arch_passthrough_mode_setdefault(libxl__gc *gc, return rc; } +void libxl__arch_get_physinfo(libxl_physinfo *physinfo, + const xc_physinfo_t *xcphysinfo) +{ + physinfo->cap_assisted_xapic = + !!(xcphysinfo->arch_capabilities & + XEN_SYSCTL_PHYSCAP_X86_ASSISTED_XAPIC); + physinfo->cap_assisted_x2apic = + !!(xcphysinfo->arch_capabilities & + XEN_SYSCTL_PHYSCAP_X86_ASSISTED_X2APIC); +} + void libxl__arch_update_domain_config(libxl__gc *gc, libxl_domain_config *dst, const libxl_domain_config *src) diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml index 7503031d8f..7ce832d605 100644 --- a/tools/ocaml/libs/xc/xenctrl.ml +++ b/tools/ocaml/libs/xc/xenctrl.ml @@ -127,6 +127,10 @@ type physinfo_cap_flag = | CAP_Gnttab_v1 | CAP_Gnttab_v2 +type physinfo_cap_arch_flag = + | CAP_ARCH_ASSISTED_XAPIC + | CAP_ARCH_ASSISTED_X2APIC + type physinfo = { threads_per_core : int; @@ -139,6 +143,7 @@ type physinfo = scrub_pages : nativeint; (* XXX hw_cap *) capabilities : physinfo_cap_flag list; + arch_capabilities : physinfo_cap_arch_flag list; max_nr_cpus : int; } diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli index d1d9c9247a..a2b15130ee 100644 --- a/tools/ocaml/libs/xc/xenctrl.mli +++ b/tools/ocaml/libs/xc/xenctrl.mli @@ -112,6 +112,10 @@ type physinfo_cap_flag = | CAP_Gnttab_v1 | CAP_Gnttab_v2 +type physinfo_cap_arch_flag = + | CAP_ARCH_ASSISTED_XAPIC + | CAP_ARCH_ASSISTED_X2APIC + type physinfo = { threads_per_core : int; cores_per_socket : int; @@ -122,6 +126,7 @@ type physinfo = { free_pages : nativeint; scrub_pages : nativeint; capabilities : physinfo_cap_flag list; + arch_capabilities : physinfo_cap_arch_flag list; max_nr_cpus : int; (** compile-time max possible number of nr_cpus *) } type version = { major : int; minor : int; extra : string; } diff --git a/tools/xl/xl_info.c b/tools/xl/xl_info.c index 712b7638b0..3205270754 100644 --- a/tools/xl/xl_info.c +++ b/tools/xl/xl_info.c @@ -210,7 +210,7 @@ static void output_physinfo(void) info.hw_cap[4], info.hw_cap[5], info.hw_cap[6], info.hw_cap[7] ); - maybe_printf("virt_caps :%s%s%s%s%s%s%s%s%s%s%s\n", + maybe_printf("virt_caps :%s%s%s%s%s%s%s%s%s%s%s%s%s\n", info.cap_pv ? " pv" : "", info.cap_hvm ? " hvm" : "", info.cap_hvm && info.cap_hvm_directio ? " hvm_directio" : "", @@ -221,7 +221,9 @@ static void output_physinfo(void) info.cap_vmtrace ? " vmtrace" : "", info.cap_vpmu ? " vpmu" : "", info.cap_gnttab_v1 ? " gnttab-v1" : "", - info.cap_gnttab_v2 ? " gnttab-v2" : "" + info.cap_gnttab_v2 ? " gnttab-v2" : "", + info.cap_assisted_xapic ? " assisted_xapic" : "", + info.cap_assisted_x2apic ? " assisted_x2apic" : "" ); vinfo = libxl_get_version_info(ctx); diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index 7ab15e07a0..4060aef1bd 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -343,6 +343,15 @@ static int vmx_init_vmcs_config(bool bsp) MSR_IA32_VMX_PROCBASED_CTLS2, &mismatch); } + /* Check whether hardware supports accelerated xapic and x2apic. */ + if ( bsp ) + { + assisted_xapic_available = cpu_has_vmx_virtualize_apic_accesses; + assisted_x2apic_available = (cpu_has_vmx_apic_reg_virt || + cpu_has_vmx_virtual_intr_delivery) && + cpu_has_vmx_virtualize_x2apic_mode; + } + /* The IA32_VMX_EPT_VPID_CAP MSR exists only when EPT or VPID available */ if ( _vmx_secondary_exec_control & (SECONDARY_EXEC_ENABLE_EPT | SECONDARY_EXEC_ENABLE_VPID) ) diff --git a/xen/arch/x86/include/asm/domain.h b/xen/arch/x86/include/asm/domain.h index e62e109598..72431df26d 100644 --- a/xen/arch/x86/include/asm/domain.h +++ b/xen/arch/x86/include/asm/domain.h @@ -756,6 +756,9 @@ static inline void pv_inject_sw_interrupt(unsigned int vector) : is_pv_32bit_domain(d) ? PV32_VM_ASSIST_MASK \ : PV64_VM_ASSIST_MASK) +extern bool assisted_xapic_available; +extern bool assisted_x2apic_available; + #endif /* __ASM_DOMAIN_H__ */ /* diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c index aff52a13f3..642cc96985 100644 --- a/xen/arch/x86/sysctl.c +++ b/xen/arch/x86/sysctl.c @@ -69,6 +69,9 @@ struct l3_cache_info { unsigned long size; }; +bool __ro_after_init assisted_xapic_available; +bool __ro_after_init assisted_x2apic_available; + static void l3_cache_get(void *arg) { struct cpuid4_info info; @@ -135,6 +138,10 @@ void arch_do_physinfo(struct xen_sysctl_physinfo *pi) pi->capabilities |= XEN_SYSCTL_PHYSCAP_hap; if ( IS_ENABLED(CONFIG_SHADOW_PAGING) ) pi->capabilities |= XEN_SYSCTL_PHYSCAP_shadow; + if ( assisted_xapic_available ) + pi->arch_capabilities |= XEN_SYSCTL_PHYSCAP_X86_ASSISTED_XAPIC; + if ( assisted_x2apic_available ) + pi->arch_capabilities |= XEN_SYSCTL_PHYSCAP_X86_ASSISTED_X2APIC; } long arch_do_sysctl( diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h index 55252e97f2..11328bbf78 100644 --- a/xen/include/public/sysctl.h +++ b/xen/include/public/sysctl.h @@ -35,7 +35,7 @@ #include "domctl.h" #include "physdev.h" -#define XEN_SYSCTL_INTERFACE_VERSION 0x00000014 +#define XEN_SYSCTL_INTERFACE_VERSION 0x00000015 /* * Read console content from Xen buffer ring. @@ -111,6 +111,10 @@ struct xen_sysctl_tbuf_op { /* Max XEN_SYSCTL_PHYSCAP_* constant. Used for ABI checking. */ #define XEN_SYSCTL_PHYSCAP_MAX XEN_SYSCTL_PHYSCAP_gnttab_v2 +/* The platform supports x{2}apic hardware assisted emulation. */ +#define XEN_SYSCTL_PHYSCAP_X86_ASSISTED_XAPIC (1u << 0) +#define XEN_SYSCTL_PHYSCAP_X86_ASSISTED_X2APIC (1u << 1) + struct xen_sysctl_physinfo { uint32_t threads_per_core; uint32_t cores_per_socket; @@ -120,6 +124,8 @@ struct xen_sysctl_physinfo { uint32_t max_node_id; /* Largest possible node ID on this host */ uint32_t cpu_khz; uint32_t capabilities;/* XEN_SYSCTL_PHYSCAP_??? */ + uint32_t arch_capabilities;/* XEN_SYSCTL_PHYSCAP_X86{ARM}_??? */ + uint32_t pad; /* Must be zero. */ uint64_aligned_t total_pages; uint64_aligned_t free_pages; uint64_aligned_t scrub_pages; From patchwork Mon Feb 7 18:21:01 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Jane Malalane X-Patchwork-Id: 12737805 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 95B1CC43217 for ; Mon, 7 Feb 2022 18:22:09 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.267363.461111 (Exim 4.92) (envelope-from ) id 1nH8e3-0003sY-PM; Mon, 07 Feb 2022 18:21:47 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 267363.461111; Mon, 07 Feb 2022 18:21:47 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nH8e3-0003sR-MS; Mon, 07 Feb 2022 18:21:47 +0000 Received: by outflank-mailman (input) for mailman id 267363; Mon, 07 Feb 2022 18:21:46 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1nH8e2-0003rG-3k for xen-devel@lists.xenproject.org; Mon, 07 Feb 2022 18:21:46 +0000 Received: from esa3.hc3370-68.iphmx.com (esa3.hc3370-68.iphmx.com [216.71.145.155]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id cc31fc32-8842-11ec-8eb8-a37418f5ba1a; Mon, 07 Feb 2022 19:21:43 +0100 (CET) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: cc31fc32-8842-11ec-8eb8-a37418f5ba1a DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1644258103; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=0XTOLAnBMrMDqngU7nmiL5ddS04Uw/To/WSMXa29m/A=; b=EGPMrYg2dNxVMCQroLXBcG1k317lv6cKNZjV8ighzAU9Ks/pWEPLoHNM brV+JuUDGh/h8+k13mm9JvKMOAfG/DvLFbdlyFYIxIDT2iL1dJgMWMlrq MPFLIUz/02M4WVR0RZbu7bXzQ9HpmFkjyCsxP9gGJmXgSYDi0GnDQ7z6H 8=; Authentication-Results: esa3.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: T3R5fMvCYDm98IDc3cMvpC5ZfIvf7PIFIXPH0chGPizKGJJ4ajJ/OpK5q6uHvYhgo1VA7cx+U4 bDYau0XgWst7vgnhMahtLTErk8twuBQognLxxi2mMaFhr3l9EfOlbPjENq7orMF1p1WRJ5riCg WKmK4TwcNDfifFKWpu3bcrXz2HWiCOTmStfSyz4Pg1InWrd1mOCtgF7mqQ1/FlwIYbtVhD8A2M wIUtm/NKP2tnbnRVRZAri1pdjR61NL9T8C6lGLntEdsl14nJzNQlA0gGBpjMLEGuRCPdCBbhDJ nKSyl4PmmKvdTWYANwyQv29l X-SBRS: 5.1 X-MesageID: 63670008 X-Ironport-Server: esa3.hc3370-68.iphmx.com X-Remote-IP: 162.221.156.83 X-Policy: $RELAYED IronPort-Data: A9a23:wpgam60FJ1AGs5YsgPbD5Xx2kn2cJEfYwER7XKvMYLTBsI5bpzEHx 2RMXz+HaK2PMWT9edlyaIq38BgDsZbTm9BlGQc5pC1hF35El5HIVI+TRqvS04J+DSFhoGZPt Zh2hgzodZhsJpPkS5PE3oHJ9RGQ74nRLlbHILOCanAZqTNMEn9700o5w7Fh2OaEvPDia++zk YKqyyHgEAfNNw5cagr4PIra9XuDFNyr0N8plgRWicJj5TcypFFMZH4rHomjLmOQf2VhNrXSq 9Avbl2O1jixEx8FUrtJm1tgG6EAaua60QOm0hK6V0U+6/TrS+NbPqsTbZIhhUlrZzqhmMBv1 9Nhl4CLeF0jAKvDwM9CAysDHHQrVUFG0OevzXmXtMWSywvNcmf2wuUoB0YzVWEa0r8pWycUr 6VecW1TKEDY7w616OvTpu1EpM0lIY/ONYcWvnhmwBnSDOo8QICFSKLPjTNd9Glr2J8WRa6OD yYfQT5GbgrkTCRjAQdNNc49lvm3l3rBSCIN/Tp5ooJoujOOnWSdyoPFK8HJc9aHQcFUmEewp W/c+Wn9RBYAO7S3yyeB83+qrv/Cm2X8Qo16PIO/8vlmkViC3Fs5ARcdVUa4ifShg0v4UNVaQ 2QY8zQjhbI//0uqSp/6RRLQiHyLpBkHQPJLDvY3rgqKz8L88wufQ2QJUDNFQNgnr9MtAywn0 EeTmNHkDiApt6eaIVqC8p+EoDX0PjIaRUcAbyIZSQoO4/H4vZo+yBnIS75e/LWd14OvX2uqm nbT8XZ41+57YdM3O7uT+kDZnDSAhITyf1Q/2APYBkn6vjEia9vwD2C30mTz4fFFJYefa1COu nkYhsSThNwz4YGxeD+lG7tUQuzwjxqRGHiF2AM0QcF9n9i40yP7JehtDCdCyFCF2yruURvge wfttAxY//e/11P6PPYsM+pd5ynHpJUM9OgJtNiJNLKigbArLWdrGR2CgmbKjgjQfLAEy/1XB HtiWZ/E4YwmIapm1iGqYOwWzKUmwCszrUuKG8ynlkX+iOTPPifOIVvgDLdpRrpohJ5oXS2Pq 4oPXyd040k3vBLCjtn/rtdIcAFiwYkTDpHqsc1HHtNv0SI9cFzN/8T5mOt7E6Q8xvw9vr6Ro hmVBx8JoHKi1CavAVjbNRhLNui1Nb4h9i1TAMDZFQvxs5TVSd30t/l3mlpeVeRPydGPOtYuE 6BVIJvRXqsTItkFkhxEBaTAQEVZXEzDrWqz0+CNOlDTprZsGF7E/MHKZAzq+HVcBya7r5Jm8 bahyhnaUdwIQAE7VJTab/emzlWQu3kBmb0tAxuUc4cLIEi8opJ3LyHRj+MsJ51eIxv02TbHh R2dBg0VpLeRrtZtosXJn62Ns6ygD/B6QhhBB2De4LvvbXva82OvzJVuSuGNeTyBBmr49L/7P bdezu3mMe1Bl1FP6tIuH7FuxKM4xt3uu74FkVg0QCSVNwymU+oyLGOH0M9Dsrx26oVY4QbmC FiS/tR6OKmSPJ+3GlAmOwd4PP+I0usZm2eO4K1tcln6/iJ+4JGOTV5WY0uXkCVYIbZ4bNElz OMmtJJE4gCzkEN3YNOPjyQS/GWQNH0QFa4gs8hCUoPsjwMqzHBEYIDdVXCqsM3eNY0UPxl4O CKQiYrDm69YlxjLfHcEHHTQ2fZQ2MYVsxdQwV5ef1mEl7IpXBPsMMG9Jdjvcjlo8w== IronPort-HdrOrdr: A9a23:6tttQa+0D8s+Au1OT7xuk+DgI+orL9Y04lQ7vn2YSXRuHPBw8P re5cjztCWE7gr5N0tBpTntAsW9qDbnhPtICOoqTNCftWvdyQiVxehZhOOIqVDd8m/Fh4pgPM 9bAtBD4bbLbGSS4/yU3ODBKadD/OW6 X-IronPort-AV: E=Sophos;i="5.88,350,1635220800"; d="scan'208";a="63670008" From: Jane Malalane To: Xen-devel CC: Jane Malalane , Wei Liu , "Anthony PERARD" , Juergen Gross , "Andrew Cooper" , George Dunlap , Jan Beulich , Julien Grall , "Stefano Stabellini" , Christian Lindig , David Scott , "Volodymyr Babchuk" , =?utf-8?q?Roger_Pau_Monn?= =?utf-8?q?=C3=A9?= Subject: [PATCH v2 2/2] x86/xen: Allow per-domain usage of hardware virtualized APIC Date: Mon, 7 Feb 2022 18:21:01 +0000 Message-ID: <20220207182101.31941-3-jane.malalane@citrix.com> X-Mailer: git-send-email 2.11.0 In-Reply-To: <20220207182101.31941-1-jane.malalane@citrix.com> References: <20220207182101.31941-1-jane.malalane@citrix.com> MIME-Version: 1.0 Introduce a new per-domain creation x86 specific flag to select whether hardware assisted virtualization should be used for x{2}APIC. A per-domain option is added to xl in order to select the usage of x{2}APIC hardware assisted vitualization, as well as a global configuration option. Having all APIC interaction exit to Xen for emulation is slow and can induce much overhead. Hardware can speed up x{2}APIC by running APIC read/write accesses without taking a VM exit. Being able to disable x{2}APIC hardware assisted vitualization can be useful for testing and debugging purposes. Signed-off-by: Jane Malalane Suggested-by: Andrew Cooper Acked-by: Christian Lindig --- CC: Wei Liu CC: Anthony PERARD CC: Juergen Gross CC: Andrew Cooper CC: George Dunlap CC: Jan Beulich CC: Julien Grall CC: Stefano Stabellini CC: Christian Lindig CC: David Scott CC: Volodymyr Babchuk CC: "Roger Pau Monné" v2: * Add a LIBXL_HAVE_ASSISTED_APIC macro * Pass xcpyshinfo as a pointer in libxl__arch_get_physinfo * Add a return statement in now "int" libxl__arch_domain_build_info_setdefault * Preserve libxl__arch_domain_build_info_setdefault 's location in libxl_create.c * Correct x{2}apic default setting logic in libxl__arch_domain_prepare_config * Correct logic for parsing assisted_x{2}apic host/guest options in xl_parse.c and initialize them to -1 in xl.c * Use guest options directly in vmx_vlapic_msr_changed * Fix indentation of bool assisted_x{2}apic in struct hvm_domain * Add a change in xenctrl_stubs.c to pass xenctrl ABI checks --- docs/man/xl.cfg.5.pod.in | 10 ++++++++++ docs/man/xl.conf.5.pod.in | 12 ++++++++++++ tools/golang/xenlight/helpers.gen.go | 12 ++++++++++++ tools/include/libxl.h | 7 +++++++ tools/libs/light/libxl_arch.h | 5 +++-- tools/libs/light/libxl_arm.c | 7 +++++-- tools/libs/light/libxl_create.c | 23 ++++++++++++++--------- tools/libs/light/libxl_types.idl | 2 ++ tools/libs/light/libxl_x86.c | 31 +++++++++++++++++++++++++++++-- tools/ocaml/libs/xc/xenctrl.ml | 2 ++ tools/ocaml/libs/xc/xenctrl.mli | 2 ++ tools/ocaml/libs/xc/xenctrl_stubs.c | 2 +- tools/xl/xl.c | 8 ++++++++ tools/xl/xl.h | 2 ++ tools/xl/xl_parse.c | 16 ++++++++++++++++ xen/arch/x86/domain.c | 28 +++++++++++++++++++++++++++- xen/arch/x86/hvm/vmx/vmcs.c | 4 ++++ xen/arch/x86/hvm/vmx/vmx.c | 14 +++++--------- xen/arch/x86/include/asm/hvm/domain.h | 6 ++++++ xen/arch/x86/traps.c | 8 ++++---- xen/include/public/arch-x86/xen.h | 2 ++ 21 files changed, 173 insertions(+), 30 deletions(-) diff --git a/docs/man/xl.cfg.5.pod.in b/docs/man/xl.cfg.5.pod.in index b98d161398..1d98bbd182 100644 --- a/docs/man/xl.cfg.5.pod.in +++ b/docs/man/xl.cfg.5.pod.in @@ -1862,6 +1862,16 @@ firmware tables when using certain older guest Operating Systems. These tables have been superseded by newer constructs within the ACPI tables. +=item B +B<(x86 only)> Enables or disables hardware assisted virtualization for xapic. +This allows accessing APIC registers without a VM-exit. +The default is settable via L. + +=item B +B<(x86 only)> Enables or disables hardware assisted virtualization for x2apic. +This allows accessing APIC registers without a VM-exit. +The default is settable via L. + =item B B<(x86 only)> Hides or exposes the No-eXecute capability. This allows a guest diff --git a/docs/man/xl.conf.5.pod.in b/docs/man/xl.conf.5.pod.in index df20c08137..30993827e5 100644 --- a/docs/man/xl.conf.5.pod.in +++ b/docs/man/xl.conf.5.pod.in @@ -107,6 +107,18 @@ Sets the default value for the C domain config value. Default: maximum grant version supported by the hypervisor. +=item B + +If enabled, domains will use xAPIC hardware assisted virtualization by default. + +Default: enabled if supported. + +=item B + +If enabled, domains will use x2APIC hardware assisted virtualization by default. + +Default: enabled if supported. + =item B Configures the default hotplug script used by virtual network devices. diff --git a/tools/golang/xenlight/helpers.gen.go b/tools/golang/xenlight/helpers.gen.go index dd4e6c9f14..90e7b9b205 100644 --- a/tools/golang/xenlight/helpers.gen.go +++ b/tools/golang/xenlight/helpers.gen.go @@ -636,6 +636,12 @@ x.Passthrough = Passthrough(xc.passthrough) if err := x.XendSuspendEvtchnCompat.fromC(&xc.xend_suspend_evtchn_compat);err != nil { return fmt.Errorf("converting field XendSuspendEvtchnCompat: %v", err) } +if err := x.ArchX86.AssistedXapic.fromC(&xc.arch_x86.assisted_xapic);err != nil { +return fmt.Errorf("converting field ArchX86.AssistedXapic: %v", err) +} +if err := x.ArchX86.AssistedX2Apic.fromC(&xc.arch_x86.assisted_x2apic);err != nil { +return fmt.Errorf("converting field ArchX86.AssistedX2Apic: %v", err) +} return nil} @@ -679,6 +685,12 @@ xc.passthrough = C.libxl_passthrough(x.Passthrough) if err := x.XendSuspendEvtchnCompat.toC(&xc.xend_suspend_evtchn_compat); err != nil { return fmt.Errorf("converting field XendSuspendEvtchnCompat: %v", err) } +if err := x.ArchX86.AssistedXapic.toC(&xc.arch_x86.assisted_xapic); err != nil { +return fmt.Errorf("converting field ArchX86.AssistedXapic: %v", err) +} +if err := x.ArchX86.AssistedX2Apic.toC(&xc.arch_x86.assisted_x2apic); err != nil { +return fmt.Errorf("converting field ArchX86.AssistedX2Apic: %v", err) +} return nil } diff --git a/tools/include/libxl.h b/tools/include/libxl.h index 924e142628..83944c17ae 100644 --- a/tools/include/libxl.h +++ b/tools/include/libxl.h @@ -535,6 +535,13 @@ #define LIBXL_HAVE_PHYSINFO_ASSISTED_APIC 1 /* + * LIBXL_HAVE_ASSISTED_APIC indicates that libxl_domain_build_info has + * assisted_x{2}apic fields, for enabling hardware assisted virtualization for + * x{2}apic per domain. + */ +#define LIBXL_HAVE_ASSISTED_APIC 1 + +/* * libxl ABI compatibility * * The only guarantee which libxl makes regarding ABI compatibility diff --git a/tools/libs/light/libxl_arch.h b/tools/libs/light/libxl_arch.h index 207ceac6a1..03b89929e6 100644 --- a/tools/libs/light/libxl_arch.h +++ b/tools/libs/light/libxl_arch.h @@ -71,8 +71,9 @@ void libxl__arch_domain_create_info_setdefault(libxl__gc *gc, libxl_domain_create_info *c_info); _hidden -void libxl__arch_domain_build_info_setdefault(libxl__gc *gc, - libxl_domain_build_info *b_info); +int libxl__arch_domain_build_info_setdefault(libxl__gc *gc, + libxl_domain_build_info *b_info, + const libxl_physinfo *physinfo); _hidden int libxl__arch_passthrough_mode_setdefault(libxl__gc *gc, diff --git a/tools/libs/light/libxl_arm.c b/tools/libs/light/libxl_arm.c index 39fdca1b49..ba5b8f433f 100644 --- a/tools/libs/light/libxl_arm.c +++ b/tools/libs/light/libxl_arm.c @@ -1384,8 +1384,9 @@ void libxl__arch_domain_create_info_setdefault(libxl__gc *gc, } } -void libxl__arch_domain_build_info_setdefault(libxl__gc *gc, - libxl_domain_build_info *b_info) +int libxl__arch_domain_build_info_setdefault(libxl__gc *gc, + libxl_domain_build_info *b_info, + const libxl_physinfo *physinfo) { /* ACPI is disabled by default */ libxl_defbool_setdefault(&b_info->acpi, false); @@ -1399,6 +1400,8 @@ void libxl__arch_domain_build_info_setdefault(libxl__gc *gc, memset(&b_info->u, '\0', sizeof(b_info->u)); b_info->type = LIBXL_DOMAIN_TYPE_INVALID; libxl_domain_build_info_init_type(b_info, LIBXL_DOMAIN_TYPE_PVH); + + return 0; } int libxl__arch_passthrough_mode_setdefault(libxl__gc *gc, diff --git a/tools/libs/light/libxl_create.c b/tools/libs/light/libxl_create.c index d7a40d7550..843e523df9 100644 --- a/tools/libs/light/libxl_create.c +++ b/tools/libs/light/libxl_create.c @@ -264,7 +264,20 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc, if (!b_info->event_channels) b_info->event_channels = 1023; - libxl__arch_domain_build_info_setdefault(gc, b_info); + libxl_physinfo info; + + rc = libxl_get_physinfo(CTX, &info); + if (rc) { + LOG(ERROR, "failed to get hypervisor info"); + return rc; + } + + rc = libxl__arch_domain_build_info_setdefault(gc, b_info, &info); + if (rc) { + LOG(ERROR, "unable to set domain arch build info defaults"); + return rc; + } + libxl_defbool_setdefault(&b_info->dm_restrict, false); if (b_info->iommu_memkb == LIBXL_MEMKB_DEFAULT) @@ -457,14 +470,6 @@ int libxl__domain_build_info_setdefault(libxl__gc *gc, } if (b_info->max_grant_version == LIBXL_MAX_GRANT_DEFAULT) { - libxl_physinfo info; - - rc = libxl_get_physinfo(CTX, &info); - if (rc) { - LOG(ERROR, "failed to get hypervisor info"); - return rc; - } - if (info.cap_gnttab_v2) b_info->max_grant_version = 2; else if (info.cap_gnttab_v1) diff --git a/tools/libs/light/libxl_types.idl b/tools/libs/light/libxl_types.idl index 42ac6c357b..db5eb0a0b3 100644 --- a/tools/libs/light/libxl_types.idl +++ b/tools/libs/light/libxl_types.idl @@ -648,6 +648,8 @@ libxl_domain_build_info = Struct("domain_build_info",[ ("vuart", libxl_vuart_type), ])), ("arch_x86", Struct(None, [("msr_relaxed", libxl_defbool), + ("assisted_xapic", libxl_defbool), + ("assisted_x2apic", libxl_defbool), ])), # Alternate p2m is not bound to any architecture or guest type, as it is # supported by x86 HVM and ARM support is planned. diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c index e0a06ecfe3..f0fa0ceea2 100644 --- a/tools/libs/light/libxl_x86.c +++ b/tools/libs/light/libxl_x86.c @@ -23,6 +23,14 @@ int libxl__arch_domain_prepare_config(libxl__gc *gc, if (libxl_defbool_val(d_config->b_info.arch_x86.msr_relaxed)) config->arch.misc_flags |= XEN_X86_MSR_RELAXED; + if (d_config->c_info.type != LIBXL_DOMAIN_TYPE_PV) + { + if (libxl_defbool_val(d_config->b_info.arch_x86.assisted_xapic)) + config->arch.misc_flags |= XEN_X86_ASSISTED_XAPIC; + + if (libxl_defbool_val(d_config->b_info.arch_x86.assisted_x2apic)) + config->arch.misc_flags |= XEN_X86_ASSISTED_X2APIC; + } return 0; } @@ -819,11 +827,30 @@ void libxl__arch_domain_create_info_setdefault(libxl__gc *gc, { } -void libxl__arch_domain_build_info_setdefault(libxl__gc *gc, - libxl_domain_build_info *b_info) +int libxl__arch_domain_build_info_setdefault(libxl__gc *gc, + libxl_domain_build_info *b_info, + const libxl_physinfo *physinfo) { libxl_defbool_setdefault(&b_info->acpi, true); libxl_defbool_setdefault(&b_info->arch_x86.msr_relaxed, false); + + if (b_info->type != LIBXL_DOMAIN_TYPE_PV) + { + libxl_defbool_setdefault(&b_info->arch_x86.assisted_xapic, + physinfo->cap_assisted_xapic); + libxl_defbool_setdefault(&b_info->arch_x86.assisted_x2apic, + physinfo->cap_assisted_x2apic); + } + + if (b_info->type == LIBXL_DOMAIN_TYPE_PV && + (!libxl_defbool_is_default(b_info->arch_x86.assisted_xapic) || + !libxl_defbool_is_default(b_info->arch_x86.assisted_x2apic))) + { + LOG(ERROR, "Interrupt Controller Virtualization not supported for PV"); + return ERROR_INVAL; + } + + return 0; } int libxl__arch_passthrough_mode_setdefault(libxl__gc *gc, diff --git a/tools/ocaml/libs/xc/xenctrl.ml b/tools/ocaml/libs/xc/xenctrl.ml index 7ce832d605..cce30d8731 100644 --- a/tools/ocaml/libs/xc/xenctrl.ml +++ b/tools/ocaml/libs/xc/xenctrl.ml @@ -50,6 +50,8 @@ type x86_arch_emulation_flags = type x86_arch_misc_flags = | X86_MSR_RELAXED + | X86_ASSISTED_XAPIC + | X86_ASSISTED_X2APIC type xen_x86_arch_domainconfig = { diff --git a/tools/ocaml/libs/xc/xenctrl.mli b/tools/ocaml/libs/xc/xenctrl.mli index a2b15130ee..67a22ec15c 100644 --- a/tools/ocaml/libs/xc/xenctrl.mli +++ b/tools/ocaml/libs/xc/xenctrl.mli @@ -44,6 +44,8 @@ type x86_arch_emulation_flags = type x86_arch_misc_flags = | X86_MSR_RELAXED + | X86_ASSISTED_XAPIC + | X86_ASSISTED_X2APIC type xen_x86_arch_domainconfig = { emulation_flags: x86_arch_emulation_flags list; diff --git a/tools/ocaml/libs/xc/xenctrl_stubs.c b/tools/ocaml/libs/xc/xenctrl_stubs.c index 5b4fe72c8d..0aa957d379 100644 --- a/tools/ocaml/libs/xc/xenctrl_stubs.c +++ b/tools/ocaml/libs/xc/xenctrl_stubs.c @@ -239,7 +239,7 @@ CAMLprim value stub_xc_domain_create(value xch, value wanted_domid, value config cfg.arch.misc_flags = ocaml_list_to_c_bitmap /* ! x86_arch_misc_flags X86_ none */ - /* ! XEN_X86_ XEN_X86_MSR_RELAXED all */ + /* ! XEN_X86_ XEN_X86_ASSISTED_X2APIC max */ (VAL_MISC_FLAGS); #undef VAL_MISC_FLAGS diff --git a/tools/xl/xl.c b/tools/xl/xl.c index 2d1ec18ea3..31eb223309 100644 --- a/tools/xl/xl.c +++ b/tools/xl/xl.c @@ -57,6 +57,8 @@ int max_grant_frames = -1; int max_maptrack_frames = -1; int max_grant_version = LIBXL_MAX_GRANT_DEFAULT; libxl_domid domid_policy = INVALID_DOMID; +int assisted_xapic = -1; +int assisted_x2apic = -1; xentoollog_level minmsglevel = minmsglevel_default; @@ -201,6 +203,12 @@ static void parse_global_config(const char *configfile, if (!xlu_cfg_get_long (config, "claim_mode", &l, 0)) claim_mode = l; + if (!xlu_cfg_get_long (config, "assisted_xapic", &l, 0)) + assisted_xapic = l; + + if (!xlu_cfg_get_long (config, "assisted_x2apic", &l, 0)) + assisted_x2apic = l; + xlu_cfg_replace_string (config, "remus.default.netbufscript", &default_remus_netbufscript, 0); xlu_cfg_replace_string (config, "colo.default.proxyscript", diff --git a/tools/xl/xl.h b/tools/xl/xl.h index c5c4bedbdd..528deb3feb 100644 --- a/tools/xl/xl.h +++ b/tools/xl/xl.h @@ -286,6 +286,8 @@ extern libxl_bitmap global_vm_affinity_mask; extern libxl_bitmap global_hvm_affinity_mask; extern libxl_bitmap global_pv_affinity_mask; extern libxl_domid domid_policy; +extern int assisted_xapic; +extern int assisted_x2apic; enum output_format { OUTPUT_FORMAT_JSON, diff --git a/tools/xl/xl_parse.c b/tools/xl/xl_parse.c index 117fcdcb2b..0ab9b145fe 100644 --- a/tools/xl/xl_parse.c +++ b/tools/xl/xl_parse.c @@ -1681,6 +1681,22 @@ void parse_config_data(const char *config_source, xlu_cfg_get_defbool(config, "vpt_align", &b_info->u.hvm.vpt_align, 0); xlu_cfg_get_defbool(config, "apic", &b_info->apic, 0); + e = xlu_cfg_get_long(config, "assisted_xapic", &l , 0); + if ((e == ESRCH && assisted_xapic != -1)) /* use global default if present */ + libxl_defbool_set(&b_info->arch_x86.assisted_xapic, assisted_xapic); + else if (!e) + libxl_defbool_set(&b_info->arch_x86.assisted_xapic, l); + else + exit(1); + + e = xlu_cfg_get_long(config, "assisted_x2apic", &l, 0); + if ((e == ESRCH && assisted_x2apic != -1)) /* use global default if present */ + libxl_defbool_set(&b_info->arch_x86.assisted_x2apic, assisted_x2apic); + else if (!e) + libxl_defbool_set(&b_info->arch_x86.assisted_x2apic, l); + else + exit(1); + switch (xlu_cfg_get_list(config, "viridian", &viridian, &num_viridian, 1)) { diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c index ef1812dc14..9033a0e181 100644 --- a/xen/arch/x86/domain.c +++ b/xen/arch/x86/domain.c @@ -619,6 +619,8 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config) bool hvm = config->flags & XEN_DOMCTL_CDF_hvm; bool hap = config->flags & XEN_DOMCTL_CDF_hap; bool nested_virt = config->flags & XEN_DOMCTL_CDF_nested_virt; + bool assisted_xapic = config->arch.misc_flags & XEN_X86_ASSISTED_XAPIC; + bool assisted_x2apic = config->arch.misc_flags & XEN_X86_ASSISTED_X2APIC; unsigned int max_vcpus; if ( hvm ? !hvm_enabled : !IS_ENABLED(CONFIG_PV) ) @@ -685,13 +687,31 @@ int arch_sanitise_domain_config(struct xen_domctl_createdomain *config) } } - if ( config->arch.misc_flags & ~XEN_X86_MSR_RELAXED ) + if ( config->arch.misc_flags & ~(XEN_X86_MSR_RELAXED | + XEN_X86_ASSISTED_XAPIC | + XEN_X86_ASSISTED_X2APIC) ) { dprintk(XENLOG_INFO, "Invalid arch misc flags %#x\n", config->arch.misc_flags); return -EINVAL; } + if ( (assisted_xapic || assisted_x2apic) && !hvm ) + { + dprintk(XENLOG_INFO, + "Interrupt Controller Virtualization not supported for PV\n"); + return -EINVAL; + } + + if ( (assisted_xapic && !assisted_xapic_available) || + (assisted_x2apic && !assisted_x2apic_available) ) + { + dprintk(XENLOG_INFO, + "Hardware assisted x%sAPIC requested but not available\n", + assisted_xapic && !assisted_xapic_available ? "" : "2"); + return -EINVAL; + } + return 0; } @@ -863,6 +883,12 @@ int arch_domain_create(struct domain *d, d->arch.msr_relaxed = config->arch.misc_flags & XEN_X86_MSR_RELAXED; + d->arch.hvm.assisted_xapic = + config->arch.misc_flags & XEN_X86_ASSISTED_XAPIC; + + d->arch.hvm.assisted_x2apic = + config->arch.misc_flags & XEN_X86_ASSISTED_X2APIC; + return 0; fail: diff --git a/xen/arch/x86/hvm/vmx/vmcs.c b/xen/arch/x86/hvm/vmx/vmcs.c index 4060aef1bd..614db5c4a4 100644 --- a/xen/arch/x86/hvm/vmx/vmcs.c +++ b/xen/arch/x86/hvm/vmx/vmcs.c @@ -1157,6 +1157,10 @@ static int construct_vmcs(struct vcpu *v) __vmwrite(PLE_WINDOW, ple_window); } + if ( !v->domain->arch.hvm.assisted_xapic ) + v->arch.hvm.vmx.secondary_exec_control &= + ~SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES; + if ( cpu_has_vmx_secondary_exec_control ) __vmwrite(SECONDARY_VM_EXEC_CONTROL, v->arch.hvm.vmx.secondary_exec_control); diff --git a/xen/arch/x86/hvm/vmx/vmx.c b/xen/arch/x86/hvm/vmx/vmx.c index 36c8a12cfe..3c9ff60154 100644 --- a/xen/arch/x86/hvm/vmx/vmx.c +++ b/xen/arch/x86/hvm/vmx/vmx.c @@ -3333,16 +3333,11 @@ static void vmx_install_vlapic_mapping(struct vcpu *v) void vmx_vlapic_msr_changed(struct vcpu *v) { - int virtualize_x2apic_mode; struct vlapic *vlapic = vcpu_vlapic(v); unsigned int msr; - virtualize_x2apic_mode = ( (cpu_has_vmx_apic_reg_virt || - cpu_has_vmx_virtual_intr_delivery) && - cpu_has_vmx_virtualize_x2apic_mode ); - - if ( !cpu_has_vmx_virtualize_apic_accesses && - !virtualize_x2apic_mode ) + if ( ! v->domain->arch.hvm.assisted_xapic && + ! v->domain->arch.hvm.assisted_x2apic ) return; vmx_vmcs_enter(v); @@ -3352,7 +3347,8 @@ void vmx_vlapic_msr_changed(struct vcpu *v) if ( !vlapic_hw_disabled(vlapic) && (vlapic_base_address(vlapic) == APIC_DEFAULT_PHYS_BASE) ) { - if ( virtualize_x2apic_mode && vlapic_x2apic_mode(vlapic) ) + if ( v->domain->arch.hvm.assisted_x2apic + && vlapic_x2apic_mode(vlapic) ) { v->arch.hvm.vmx.secondary_exec_control |= SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE; @@ -3373,7 +3369,7 @@ void vmx_vlapic_msr_changed(struct vcpu *v) vmx_clear_msr_intercept(v, MSR_X2APIC_SELF, VMX_MSR_W); } } - else + else if ( v->domain->arch.hvm.assisted_xapic ) v->arch.hvm.vmx.secondary_exec_control |= SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES; } diff --git a/xen/arch/x86/include/asm/hvm/domain.h b/xen/arch/x86/include/asm/hvm/domain.h index 698455444e..92bf53483c 100644 --- a/xen/arch/x86/include/asm/hvm/domain.h +++ b/xen/arch/x86/include/asm/hvm/domain.h @@ -117,6 +117,12 @@ struct hvm_domain { bool is_s3_suspended; + /* xAPIC hardware assisted virtualization. */ + bool assisted_xapic; + + /* x2APIC hardware assisted virtualization. */ + bool assisted_x2apic; + /* hypervisor intercepted msix table */ struct list_head msixtbl_list; diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c index 485bd66971..33694acc99 100644 --- a/xen/arch/x86/traps.c +++ b/xen/arch/x86/traps.c @@ -1115,7 +1115,8 @@ void cpuid_hypervisor_leaves(const struct vcpu *v, uint32_t leaf, if ( !is_hvm_domain(d) || subleaf != 0 ) break; - if ( cpu_has_vmx_apic_reg_virt ) + if ( cpu_has_vmx_apic_reg_virt && + v->domain->arch.hvm.assisted_xapic ) res->a |= XEN_HVM_CPUID_APIC_ACCESS_VIRT; /* @@ -1124,9 +1125,8 @@ void cpuid_hypervisor_leaves(const struct vcpu *v, uint32_t leaf, * and wrmsr in the guest will run without VMEXITs (see * vmx_vlapic_msr_changed()). */ - if ( cpu_has_vmx_virtualize_x2apic_mode && - cpu_has_vmx_apic_reg_virt && - cpu_has_vmx_virtual_intr_delivery ) + if ( (cpu_has_vmx_apic_reg_virt && cpu_has_vmx_virtual_intr_delivery) && + v->domain->arch.hvm.assisted_x2apic ) res->a |= XEN_HVM_CPUID_X2APIC_VIRT; /* diff --git a/xen/include/public/arch-x86/xen.h b/xen/include/public/arch-x86/xen.h index 7acd94c8eb..9da32c6239 100644 --- a/xen/include/public/arch-x86/xen.h +++ b/xen/include/public/arch-x86/xen.h @@ -317,6 +317,8 @@ struct xen_arch_domainconfig { * doesn't allow the guest to read or write to the underlying MSR. */ #define XEN_X86_MSR_RELAXED (1u << 0) +#define XEN_X86_ASSISTED_XAPIC (1u << 1) +#define XEN_X86_ASSISTED_X2APIC (1u << 2) uint32_t misc_flags; };