From patchwork Fri Jan 8 00:46:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Igor Druzhinin X-Patchwork-Id: 12005499 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19D92C433E0 for ; Fri, 8 Jan 2021 00:47:53 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id BBA9623447 for ; Fri, 8 Jan 2021 00:47:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org BBA9623447 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.63148.112125 (Exim 4.92) (envelope-from ) id 1kxfvy-0003B3-6k; Fri, 08 Jan 2021 00:47:18 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 63148.112125; Fri, 08 Jan 2021 00:47:18 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kxfvy-0003Aw-2j; Fri, 08 Jan 2021 00:47:18 +0000 Received: by outflank-mailman (input) for mailman id 63148; Fri, 08 Jan 2021 00:47:16 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kxfvw-0003Ar-Ib for xen-devel@lists.xenproject.org; Fri, 08 Jan 2021 00:47:16 +0000 Received: from esa6.hc3370-68.iphmx.com (unknown [216.71.155.175]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 7869df95-db25-470f-bfbf-4ea62f9ea9f1; Fri, 08 Jan 2021 00:47:14 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 7869df95-db25-470f-bfbf-4ea62f9ea9f1 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1610066834; h=from:to:cc:subject:date:message-id:mime-version; bh=DhagZe3FUKNmQkCfV2aS126g9ne70TITatIXvo0P88Q=; b=OXmX9Dd46ZIlM3RjcQg5bo7XTTOsTD1ZXetH+o0D5H53ZuxfDazqqhzh Dz3BzutLGuCQSUC9zPGAwokLBtjwKQE2pOguFJYgcXzpW0i+TOKy1a3MC Yw0U+9WzjxRIBjcCi+jq8LTfx0OKL7fUikNOdD5TviTFyrYDUtG0dxjGe I=; Authentication-Results: esa6.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: xG38NNKQICbni7WDPveOQyTvt2m4i0KjekVxjjo+7aLZho7gFCO212LBVBi1/aRezh2+iO3TsQ C7jh6CRNIeuwel25cS+t+AJ8a9DRCwo0X0x9gx5HZpX78WhGLcYo+M0FkWzLBoPaCLvKBJWVKn GFxeBTRck5cThbBejpu8baoO28u1Yik6E6SCyWPmkdovsLkW6G3A7FtKdqjdMxTkD+d5z0tPP6 kQTyBzoo0RZUHxuhZnQC33Tc0crM9VWPIEdLRnTn//41wDORUWjJ8yzAFlushuZZ1gGm2dkI4q Lvw= X-SBRS: 5.2 X-MesageID: 34863485 X-Ironport-Server: esa6.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.79,330,1602561600"; d="scan'208";a="34863485" From: Igor Druzhinin To: CC: , , , , , , , , , , Igor Druzhinin Subject: [PATCH 1/2] viridian: remove implicit limit of 64 VPs per partition Date: Fri, 8 Jan 2021 00:46:35 +0000 Message-ID: <1610066796-17266-1-git-send-email-igor.druzhinin@citrix.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 TLFS 7.8.1 stipulates that "a virtual processor index must be less than the maximum number of virtual processors per partition" that "can be obtained through CPUID leaf 0x40000005". Furthermore, "Requirements for Implementing the Microsoft Hypervisor Interface" defines that starting from Windows Server 2012, which allowed more than 64 CPUs to be brought up, this leaf can now contain a value -1 basically assuming the hypervisor has no restriction while 0 (that we currently expose) means the default restriction is still present. Along with previous changes exposing ExProcessorMasks this allows a recent Windows VM with Viridian extension enabled to have more than 64 vCPUs without going into immediate BSOD. Since we didn't expose the leaf before and to keep CPUID data consistent for incoming streams from previous Xen versions - let's keep it behind an option. Signed-off-by: Igor Druzhinin --- tools/libs/light/libxl_x86.c | 2 +- xen/arch/x86/hvm/viridian/viridian.c | 23 +++++++++++++++++++++++ xen/include/public/hvm/params.h | 7 ++++++- 3 files changed, 30 insertions(+), 2 deletions(-) diff --git a/tools/libs/light/libxl_x86.c b/tools/libs/light/libxl_x86.c index 86d2729..96c8bf1 100644 --- a/tools/libs/light/libxl_x86.c +++ b/tools/libs/light/libxl_x86.c @@ -336,7 +336,7 @@ static int hvm_set_viridian_features(libxl__gc *gc, uint32_t domid, LOG(DETAIL, "%s group enabled", libxl_viridian_enlightenment_to_string(v)); if (libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_BASE)) { - mask |= HVMPV_base_freq; + mask |= HVMPV_base_freq | HVMPV_no_vp_limit; if (!libxl_bitmap_test(&enlightenments, LIBXL_VIRIDIAN_ENLIGHTENMENT_FREQ)) mask |= HVMPV_no_freq; diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viridian/viridian.c index ed97804..ae1ea86 100644 --- a/xen/arch/x86/hvm/viridian/viridian.c +++ b/xen/arch/x86/hvm/viridian/viridian.c @@ -209,6 +209,29 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf, res->b = viridian_spinlock_retry_count; break; + case 5: + /* + * From "Requirements for Implementing the Microsoft Hypervisor + * Interface": + * + * "On Windows operating systems versions through Windows Server + * 2008 R2, reporting the HV#1 hypervisor interface limits + * the Windows virtual machine to a maximum of 64 VPs, regardless of + * what is reported via CPUID.40000005.EAX. + * + * Starting with Windows Server 2012 and Windows 8, if + * CPUID.40000005.EAX containsa value of -1, Windows assumes that + * the hypervisor imposes no specific limit to the number of VPs. + * In this case, Windows Server 2012 guest VMs may use more than 64 + * VPs, up to the maximum supported number of processors applicable + * to the specific Windows version being used." + * + * For compatibility we hide it behind an option. + */ + if ( viridian_feature_mask(d) & HVMPV_no_vp_limit ) + res->a = -1; + break; + case 6: /* Detected and in use hardware features. */ if ( cpu_has_vmx_virtualize_apic_accesses ) diff --git a/xen/include/public/hvm/params.h b/xen/include/public/hvm/params.h index 3b0a0f4..805f4ca 100644 --- a/xen/include/public/hvm/params.h +++ b/xen/include/public/hvm/params.h @@ -168,6 +168,10 @@ #define _HVMPV_ex_processor_masks 10 #define HVMPV_ex_processor_masks (1 << _HVMPV_ex_processor_masks) +/* Allow more than 64 VPs */ +#define _HVMPV_no_vp_limit 11 +#define HVMPV_no_vp_limit (1 << _HVMPV_no_vp_limit) + #define HVMPV_feature_mask \ (HVMPV_base_freq | \ HVMPV_no_freq | \ @@ -179,7 +183,8 @@ HVMPV_synic | \ HVMPV_stimer | \ HVMPV_hcall_ipi | \ - HVMPV_ex_processor_masks) + HVMPV_ex_processor_masks | \ + HVMPV_no_vp_limit) #endif From patchwork Fri Jan 8 00:46:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Igor Druzhinin X-Patchwork-Id: 12005497 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-19.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 977FFC433DB for ; Fri, 8 Jan 2021 00:47:49 +0000 (UTC) Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4DE9823447 for ; Fri, 8 Jan 2021 00:47:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4DE9823447 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=citrix.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=xen-devel-bounces@lists.xenproject.org Received: from list by lists.xenproject.org with outflank-mailman.63149.112137 (Exim 4.92) (envelope-from ) id 1kxfw1-0003Bz-Eg; Fri, 08 Jan 2021 00:47:21 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 63149.112137; Fri, 08 Jan 2021 00:47:21 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kxfw1-0003Bs-Ay; Fri, 08 Jan 2021 00:47:21 +0000 Received: by outflank-mailman (input) for mailman id 63149; Fri, 08 Jan 2021 00:47:20 +0000 Received: from all-amaz-eas1.inumbo.com ([34.197.232.57] helo=us1-amaz-eas2.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1kxfw0-0003Ar-NK for xen-devel@lists.xenproject.org; Fri, 08 Jan 2021 00:47:20 +0000 Received: from esa4.hc3370-68.iphmx.com (unknown [216.71.155.144]) by us1-amaz-eas2.inumbo.com (Halon) with ESMTPS id 0592e1db-2daf-4fb4-8481-ddf20c38ec64; Fri, 08 Jan 2021 00:47:15 +0000 (UTC) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 0592e1db-2daf-4fb4-8481-ddf20c38ec64 DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=citrix.com; s=securemail; t=1610066835; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=AjOVgnBRHW6mA8jfsxCNVWvF+SDZiBno1KSGGGCBrKU=; b=JXXrwAS/COCFLDm/sP69KHauBmwOrV66GiMt64Pl+4oPZM8cGINp8gDG 17uNWv0wncmPnBjfzB6yRGN+B1wCU5TZe2HK4214jwcBSL9On3jjyMwzF RySxUlhwzsHuzNQcIb9ztryu6VUKhTKpjUbp+JHSRCjxbWnzQELhhTWgR Q=; Authentication-Results: esa4.hc3370-68.iphmx.com; dkim=none (message not signed) header.i=none IronPort-SDR: RsX61pEuroged8+NrQDxKIntpoiZrZT/j9/fr2jqRh1K/HbYS1xQByzvkBYsYTm+GOp6YfYNdm VOrOl10qV//W8fLwN0i6L6G4UzqsvqDWRgkC3nLWQLbf8/Eapgtuy6ujdUyA79COa9BNjEZsTf liVtfCFemuNqiBze2Gz/NLrmVi3LnFsKaLZf5pcpWSUdmztsApdV6EP4JYwc65FddhIgeI2eAf Iy9qHqG+tkPQHRRaVrxcmNASttPl65K/BQHcEll2eZJbDghGKY4678bAtT45stJyXuMQSl6Rz+ pG8= X-SBRS: 5.2 X-MesageID: 35912825 X-Ironport-Server: esa4.hc3370-68.iphmx.com X-Remote-IP: 162.221.158.21 X-Policy: $RELAYED X-IronPort-AV: E=Sophos;i="5.79,330,1602561600"; d="scan'208";a="35912825" From: Igor Druzhinin To: CC: , , , , , , , , , , Igor Druzhinin Subject: [PATCH 2/2] viridian: allow vCPU hotplug for Windows VMs Date: Fri, 8 Jan 2021 00:46:36 +0000 Message-ID: <1610066796-17266-2-git-send-email-igor.druzhinin@citrix.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1610066796-17266-1-git-send-email-igor.druzhinin@citrix.com> References: <1610066796-17266-1-git-send-email-igor.druzhinin@citrix.com> MIME-Version: 1.0 If Viridian extensions are enabled, Windows wouldn't currently allow a hotplugged vCPU to be brought up dynamically. We need to expose a special bit to let the guest know we allow it. It appears we can just start exposing it without worrying too much about compatibility - see relevant QEMU discussion here: https://patchwork.kernel.org/project/qemu-devel/patch/1455364815-19586-1-git-send-email-den@openvz.org/ Signed-off-by: Igor Druzhinin --- xen/arch/x86/hvm/viridian/viridian.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/hvm/viridian/viridian.c b/xen/arch/x86/hvm/viridian/viridian.c index ae1ea86..76e8291 100644 --- a/xen/arch/x86/hvm/viridian/viridian.c +++ b/xen/arch/x86/hvm/viridian/viridian.c @@ -76,6 +76,7 @@ typedef union _HV_CRASH_CTL_REG_CONTENTS } HV_CRASH_CTL_REG_CONTENTS; /* Viridian CPUID leaf 3, Hypervisor Feature Indication */ +#define CPUID3D_CPU_DYNAMIC_PARTITIONING (1 << 3) #define CPUID3D_CRASH_MSRS (1 << 10) #define CPUID3D_SINT_POLLING (1 << 17) @@ -179,8 +180,11 @@ void cpuid_viridian_leaves(const struct vcpu *v, uint32_t leaf, res->a = u.lo; res->b = u.hi; + /* Expose ability to bring up VPs dynamically - allows vCPU hotplug */ + res->d = CPUID3D_CPU_DYNAMIC_PARTITIONING; + if ( viridian_feature_mask(d) & HVMPV_crash_ctl ) - res->d = CPUID3D_CRASH_MSRS; + res->d |= CPUID3D_CRASH_MSRS; if ( viridian_feature_mask(d) & HVMPV_synic ) res->d |= CPUID3D_SINT_POLLING;