From patchwork Mon Jun 17 17:39:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13701118 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6F15DC41513 for ; Mon, 17 Jun 2024 17:39:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.742538.1149361 (Exim 4.92) (envelope-from ) id 1sJGKI-0003EG-8o; Mon, 17 Jun 2024 17:39:30 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 742538.1149361; Mon, 17 Jun 2024 17:39:30 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKH-0003Bx-W6; Mon, 17 Jun 2024 17:39:29 +0000 Received: by outflank-mailman (input) for mailman id 742538; Mon, 17 Jun 2024 17:39:29 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKH-00036g-3H for xen-devel@lists.xenproject.org; Mon, 17 Jun 2024 17:39:29 +0000 Received: from mail-ej1-x62e.google.com (mail-ej1-x62e.google.com [2a00:1450:4864:20::62e]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8a6e9f41-2cd0-11ef-b4bb-af5377834399; Mon, 17 Jun 2024 19:39:26 +0200 (CEST) Received: by mail-ej1-x62e.google.com with SMTP id a640c23a62f3a-a6f958a3a69so7951566b.1 for ; Mon, 17 Jun 2024 10:39:26 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6f56da4496sm532010666b.8.2024.06.17.10.39.24 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 10:39:24 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8a6e9f41-2cd0-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718645965; x=1719250765; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Z5V14Q8Gcx1y36spQ2JEJ7cbqPfjGOYxmj1ldZEYhTQ=; b=M9Owe0bbw1/pBNPLQBAKvhPY1TDE7qQhmLlkFBcM3ye+wsgSP9ER2N6e/BA1bYoSaA 77ZHH2Xg4lZrr++2TtMiHQufGaoDoCVJLMFRNjOhbxTZSNjSVf6guwdqU/ye6m+DI0ns GI8U4OBygkj+2Ly4h2ylZjN7cPN21SIdiuhJ0= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718645965; x=1719250765; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Z5V14Q8Gcx1y36spQ2JEJ7cbqPfjGOYxmj1ldZEYhTQ=; b=OXWTiFkKQMP2wqZOOSq85H/fpHqz/NxLAqsoovNAXGzP/DuExvCvZLCISc2U6j176G Yk5zXa8wuJOEnv5w8KDaPwYwg/IIul4VLxdrR4b9+/7p1nUwtCVIwtyMaXoVRuXme1of wbmKoVxLxOShrB3xyXL1TRBWf5hcGlMGIJMihj+631S/f1fsme8qywICrFecOc3TpQ6L AU79K+Fxe1ccfRRLBEvrFErbpwKgtCyTW+bIe5HzZr7f28hRQ6dQvbNpGUihYB9HxnjY nc1btGf6mQE9WQRnlnBVLAKJOiCLD6XbL1N28086IvkesrOIiIQyUQilNz/JT7u+5JTn t7NA== X-Gm-Message-State: AOJu0YzCW5RGUf2CzdSYTrV/2PFiJrzcEWT7gUFpJ8iAxT9ll/5+uk+G OZeaIyCE+zNv2FSeW/giXiodzlk+Y/5howRDl/UwEsorIkirrRyq2bDsx5aKlQqBcrYc9iuDNRg 5PeM= X-Google-Smtp-Source: AGHT+IHDPj2YJF/xoswS4Hxe1qyfmkAv45GDhCV3GJGVWSh9zGuNHazvSlL1zN/PexzvEbPHklSOoQ== X-Received: by 2002:a17:906:b756:b0:a6e:fad9:6dbb with SMTP id a640c23a62f3a-a6f60bcbb01mr710081566b.0.1718645965045; Mon, 17 Jun 2024 10:39:25 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Oleksii Kurochko Subject: [PATCH v4 1/7] x86/xstate: Fix initialisation of XSS cache Date: Mon, 17 Jun 2024 18:39:15 +0100 Message-Id: <20240617173921.1755439-2-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240617173921.1755439-1-andrew.cooper3@citrix.com> References: <20240617173921.1755439-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 The clobbering of this_cpu(xcr0) and this_cpu(xss) to architecturally invalid values is to force the subsequent set_xcr0() and set_msr_xss() to reload the hardware register. While XCR0 is reloaded in xstate_init(), MSR_XSS isn't. This causes get_msr_xss() to return the invalid value, and logic of the form: old = get_msr_xss(); set_msr_xss(new); ... set_msr_xss(old); to try and restore said invalid value. The architecturally invalid value must be purged from the cache, meaning the hardware register must be written at least once. This in turn highlights that the invalid value must only be used in the case that the hardware register is available. Fixes: f7f4a523927f ("x86/xstate: reset cached register values on resume") Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Oleksii Kurochko v3: * Split out of later patch --- xen/arch/x86/xstate.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 99cedb4f5e24..75788147966a 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -641,13 +641,6 @@ void xstate_init(struct cpuinfo_x86 *c) return; } - /* - * Zap the cached values to make set_xcr0() and set_msr_xss() really - * write it. - */ - this_cpu(xcr0) = 0; - this_cpu(xss) = ~0; - cpuid_count(XSTATE_CPUID, 0, &eax, &ebx, &ecx, &edx); feature_mask = (((u64)edx << 32) | eax) & XCNTXT_MASK; BUG_ON(!valid_xcr0(feature_mask)); @@ -657,8 +650,19 @@ void xstate_init(struct cpuinfo_x86 *c) * Set CR4_OSXSAVE and run "cpuid" to get xsave_cntxt_size. */ set_in_cr4(X86_CR4_OSXSAVE); + + /* + * Zap the cached values to make set_xcr0() and set_msr_xss() really write + * the hardware register. + */ + this_cpu(xcr0) = 0; if ( !set_xcr0(feature_mask) ) BUG(); + if ( cpu_has_xsaves ) + { + this_cpu(xss) = ~0; + set_msr_xss(0); + } if ( bsp ) { From patchwork Mon Jun 17 17:39:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13701123 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 08BABC2BBCA for ; Mon, 17 Jun 2024 17:39:43 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.742540.1149385 (Exim 4.92) (envelope-from ) id 1sJGKJ-0003qq-PA; Mon, 17 Jun 2024 17:39:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 742540.1149385; Mon, 17 Jun 2024 17:39:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKJ-0003pT-HZ; Mon, 17 Jun 2024 17:39:31 +0000 Received: by outflank-mailman (input) for mailman id 742540; Mon, 17 Jun 2024 17:39:29 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKH-00036g-L8 for xen-devel@lists.xenproject.org; Mon, 17 Jun 2024 17:39:29 +0000 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [2a00:1450:4864:20::634]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8b0748c8-2cd0-11ef-b4bb-af5377834399; Mon, 17 Jun 2024 19:39:27 +0200 (CEST) Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-a6f936d2ba1so36321466b.0 for ; Mon, 17 Jun 2024 10:39:27 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6f56da4496sm532010666b.8.2024.06.17.10.39.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 10:39:25 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8b0748c8-2cd0-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718645966; x=1719250766; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Zhl3ukCaOtB54z1wi1NLzU/jmU9SjtnheNpVm9jxFJQ=; b=QDCbE3XBxwBmCDbPdgqBaGh0tGF2vut0nRt3/l35Xh9dsR5M3sS+YoFU3THwsTYaOY ghyI6xKoEBvUtoiJqvBV+1Z7HpKSxxz4u4vTcuMH5ZU3ttlXWBMnvY6lNP7W8e6RIY6/ CIQmBAhF+9LiuYxMkHzTRVrftEMN58dHQ6NEs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718645966; x=1719250766; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zhl3ukCaOtB54z1wi1NLzU/jmU9SjtnheNpVm9jxFJQ=; b=Lgltc1MlhFgFx8zdRfCbuD3HJLUB/H8yYE8kE5aQ4MeaozTmSWysgFmAM2qufPycDp Cgk4H5Blk2lI14LY1zr7/B16rqsE81WSMlb13sR86msZLEVKUJCk9QyBZ9dcy5cxizVw aF+gbfgXuDVEVqSuX32nGIX76MS6+tbViCttXzRigxrKdXjA2u+uIP8AxTlKERwdeSJm OHBUT8pvhAprCuFDKUkN8xFJfaZ6UVGnFDzEPJZoLVxAOA4Li+elCSf8R+dukYF1LTvF c4BoKAJlw1L1b0SZ4Ovqgo9iMI9OUISHZfKRKULxaoxPm32Oqmn2k3Y8qHgQuxFtt8xe 4c0Q== X-Gm-Message-State: AOJu0YypyJkAFOB2xzd+w/fcVEO2whxF8Y9pILQjjGT43fGGwubBhuld 0V8ozmzAiOftw65WAaBwkMS4rqtxGhlQ3s6ZLLkB8SjoChyUcma6pITaUm135IVZsoAv+yTNGgy XnvE= X-Google-Smtp-Source: AGHT+IE2ePwbkm/Jzh5+1PD5kPs6RXA3hTe5nwJ2ydBXKARDAIfPc8oUxJZ52aObkYo78bSbB9bKjw== X-Received: by 2002:a17:906:a847:b0:a6f:46f1:5434 with SMTP id a640c23a62f3a-a6f94c047famr24028366b.6.1718645966154; Mon, 17 Jun 2024 10:39:26 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Oleksii Kurochko Subject: [PATCH v4 2/7] x86/xstate: Cross-check dynamic XSTATE sizes at boot Date: Mon, 17 Jun 2024 18:39:16 +0100 Message-Id: <20240617173921.1755439-3-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240617173921.1755439-1-andrew.cooper3@citrix.com> References: <20240617173921.1755439-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Right now, xstate_ctxt_size() performs a cross-check of size with CPUID in for every call. This is expensive, being used for domain create/migrate, as well as to service certain guest CPUID instructions. Instead, arrange to check the sizes once at boot. See the code comments for details. Right now, it just checks hardware against the algorithm expectations. Later patches will add further cross-checking. Introduce more X86_XCR0_* and X86_XSS_* constants CPUID bits. This is to maximise coverage in the sanity check, even if we don't expect to use/virtualise some of these features any time soon. Leave HDC and HWP alone for now; we don't have CPUID bits from them stored nicely. Only perform the cross-checks when SELF_TESTS are active. It's only developers or new hardware liable to trip these checks, and Xen at least tracks "maximum value ever seen in xcr0" for the lifetime of the VM, which we don't want to be tickling in the general case. Signed-off-by: Andrew Cooper --- CC: Jan Beulich CC: Roger Pau Monné CC: Oleksii Kurochko v3: * New v4: * Rebase over CONFIG_SELF_TESTS * Swap one BUG_ON() for a WARN() On Sapphire Rapids with the whole series inc diagnostics, we get this pattern: (XEN) *** check_new_xstate(, 0x00000003) (XEN) *** check_new_xstate(, 0x00000004) (XEN) *** check_new_xstate(, 0x000000e0) (XEN) *** check_new_xstate(, 0x00000200) (XEN) *** check_new_xstate(, 0x00060000) (XEN) *** check_new_xstate(, 0x00000100) (XEN) *** check_new_xstate(, 0x00000400) (XEN) *** check_new_xstate(, 0x00000800) (XEN) *** check_new_xstate(, 0x00001000) (XEN) *** check_new_xstate(, 0x00004000) (XEN) *** check_new_xstate(, 0x00008000) and on Genoa, this pattern: (XEN) *** check_new_xstate(, 0x00000003) (XEN) *** check_new_xstate(, 0x00000004) (XEN) *** check_new_xstate(, 0x000000e0) (XEN) *** check_new_xstate(, 0x00000200) (XEN) *** check_new_xstate(, 0x00000800) (XEN) *** check_new_xstate(, 0x00001000) --- xen/arch/x86/include/asm/x86-defns.h | 25 +++- xen/arch/x86/xstate.c | 158 ++++++++++++++++++++ xen/include/public/arch-x86/cpufeatureset.h | 3 + 3 files changed, 185 insertions(+), 1 deletion(-) diff --git a/xen/arch/x86/include/asm/x86-defns.h b/xen/arch/x86/include/asm/x86-defns.h index 48d7a3b7af45..d7602ab225c4 100644 --- a/xen/arch/x86/include/asm/x86-defns.h +++ b/xen/arch/x86/include/asm/x86-defns.h @@ -77,7 +77,7 @@ #define X86_CR4_PKS 0x01000000 /* Protection Key Supervisor */ /* - * XSTATE component flags in XCR0 + * XSTATE component flags in XCR0 | MSR_XSS */ #define X86_XCR0_FP_POS 0 #define X86_XCR0_FP (1ULL << X86_XCR0_FP_POS) @@ -95,11 +95,34 @@ #define X86_XCR0_ZMM (1ULL << X86_XCR0_ZMM_POS) #define X86_XCR0_HI_ZMM_POS 7 #define X86_XCR0_HI_ZMM (1ULL << X86_XCR0_HI_ZMM_POS) +#define X86_XSS_PROC_TRACE (_AC(1, ULL) << 8) #define X86_XCR0_PKRU_POS 9 #define X86_XCR0_PKRU (1ULL << X86_XCR0_PKRU_POS) +#define X86_XSS_PASID (_AC(1, ULL) << 10) +#define X86_XSS_CET_U (_AC(1, ULL) << 11) +#define X86_XSS_CET_S (_AC(1, ULL) << 12) +#define X86_XSS_HDC (_AC(1, ULL) << 13) +#define X86_XSS_UINTR (_AC(1, ULL) << 14) +#define X86_XSS_LBR (_AC(1, ULL) << 15) +#define X86_XSS_HWP (_AC(1, ULL) << 16) +#define X86_XCR0_TILE_CFG (_AC(1, ULL) << 17) +#define X86_XCR0_TILE_DATA (_AC(1, ULL) << 18) #define X86_XCR0_LWP_POS 62 #define X86_XCR0_LWP (1ULL << X86_XCR0_LWP_POS) +#define X86_XCR0_STATES \ + (X86_XCR0_FP | X86_XCR0_SSE | X86_XCR0_YMM | X86_XCR0_BNDREGS | \ + X86_XCR0_BNDCSR | X86_XCR0_OPMASK | X86_XCR0_ZMM | \ + X86_XCR0_HI_ZMM | X86_XCR0_PKRU | X86_XCR0_TILE_CFG | \ + X86_XCR0_TILE_DATA | \ + X86_XCR0_LWP) + +#define X86_XSS_STATES \ + (X86_XSS_PROC_TRACE | X86_XSS_PASID | X86_XSS_CET_U | \ + X86_XSS_CET_S | X86_XSS_HDC | X86_XSS_UINTR | X86_XSS_LBR | \ + X86_XSS_HWP | \ + 0) + /* * Debug status flags in DR6. * diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 75788147966a..650206d9d2b6 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -604,9 +604,164 @@ static bool valid_xcr0(uint64_t xcr0) if ( !(xcr0 & X86_XCR0_BNDREGS) != !(xcr0 & X86_XCR0_BNDCSR) ) return false; + /* TILECFG and TILEDATA must be the same. */ + if ( !(xcr0 & X86_XCR0_TILE_CFG) != !(xcr0 & X86_XCR0_TILE_DATA) ) + return false; + return true; } +struct xcheck_state { + uint64_t states; + uint32_t uncomp_size; + uint32_t comp_size; +}; + +static void __init check_new_xstate(struct xcheck_state *s, uint64_t new) +{ + uint32_t hw_size; + + BUILD_BUG_ON(X86_XCR0_STATES & X86_XSS_STATES); + + BUG_ON(s->states & new); /* States only increase. */ + BUG_ON(!valid_xcr0(s->states | new)); /* Xen thinks it's a good value. */ + BUG_ON(new & ~(X86_XCR0_STATES | X86_XSS_STATES)); /* Known state. */ + BUG_ON((new & X86_XCR0_STATES) && + (new & X86_XSS_STATES)); /* User or supervisor, not both. */ + + s->states |= new; + if ( new & X86_XCR0_STATES ) + { + if ( !set_xcr0(s->states & X86_XCR0_STATES) ) + BUG(); + } + else + set_msr_xss(s->states & X86_XSS_STATES); + + /* + * Check the uncompressed size. Some XSTATEs are out-of-order and fill in + * prior holes in the state area, so we check that the size doesn't + * decrease. + */ + hw_size = cpuid_count_ebx(0xd, 0); + + if ( hw_size < s->uncomp_size ) + panic("XSTATE 0x%016"PRIx64", new bits {%63pbl}, uncompressed hw size %#x < prev size %#x\n", + s->states, &new, hw_size, s->uncomp_size); + + s->uncomp_size = hw_size; + + /* + * Check the compressed size, if available. All components strictly + * appear in index order. In principle there are no holes, but some + * components have their base address 64-byte aligned for efficiency + * reasons (e.g. AMX-TILE) and there are other components small enough to + * fit in the gap (e.g. PKRU) without increasing the overall length. + */ + hw_size = cpuid_count_ebx(0xd, 1); + + if ( cpu_has_xsavec ) + { + if ( hw_size < s->comp_size ) + panic("XSTATE 0x%016"PRIx64", new bits {%63pbl}, compressed hw size %#x < prev size %#x\n", + s->states, &new, hw_size, s->comp_size); + + s->comp_size = hw_size; + } + else if ( hw_size ) /* Compressed size reported, but no XSAVEC ? */ + { + static bool once; + + if ( !once ) + { + WARN(); + once = true; + } + } +} + +/* + * The {un,}compressed XSTATE sizes are reported by dynamic CPUID value, based + * on the current %XCR0 and MSR_XSS values. The exact layout is also feature + * and vendor specific. Cross-check Xen's understanding against real hardware + * on boot. + * + * Testing every combination is prohibitive, so we use a partial approach. + * Starting with nothing active, we add new XSTATEs and check that the CPUID + * dynamic values never decreases. + */ +static void __init noinline xstate_check_sizes(void) +{ + uint64_t old_xcr0 = get_xcr0(); + uint64_t old_xss = get_msr_xss(); + struct xcheck_state s = {}; + + /* + * User XSTATEs, increasing by index. + * + * Chronologically, Intel and AMD had identical layouts for AVX (YMM). + * AMD introduced LWP in Fam15h, following immediately on from YMM. Intel + * left an LWP-shaped hole when adding MPX (BND{CSR,REGS}) in Skylake. + * AMD removed LWP in Fam17h, putting PKRU in the same space, breaking + * layout compatibility with Intel and having a knock-on effect on all + * subsequent states. + */ + check_new_xstate(&s, X86_XCR0_SSE | X86_XCR0_FP); + + if ( cpu_has_avx ) + check_new_xstate(&s, X86_XCR0_YMM); + + if ( cpu_has_mpx ) + check_new_xstate(&s, X86_XCR0_BNDCSR | X86_XCR0_BNDREGS); + + if ( cpu_has_avx512f ) + check_new_xstate(&s, X86_XCR0_HI_ZMM | X86_XCR0_ZMM | X86_XCR0_OPMASK); + + if ( cpu_has_pku ) + check_new_xstate(&s, X86_XCR0_PKRU); + + if ( boot_cpu_has(X86_FEATURE_AMX_TILE) ) + check_new_xstate(&s, X86_XCR0_TILE_DATA | X86_XCR0_TILE_CFG); + + if ( boot_cpu_has(X86_FEATURE_LWP) ) + check_new_xstate(&s, X86_XCR0_LWP); + + /* + * Supervisor XSTATEs, increasing by index. + * + * Intel Broadwell has Processor Trace but no XSAVES. There doesn't + * appear to have been a new enumeration when X86_XSS_PROC_TRACE was + * introduced in Skylake. + */ + if ( cpu_has_xsaves ) + { + if ( cpu_has_proc_trace ) + check_new_xstate(&s, X86_XSS_PROC_TRACE); + + if ( boot_cpu_has(X86_FEATURE_ENQCMD) ) + check_new_xstate(&s, X86_XSS_PASID); + + if ( boot_cpu_has(X86_FEATURE_CET_SS) || + boot_cpu_has(X86_FEATURE_CET_IBT) ) + { + check_new_xstate(&s, X86_XSS_CET_U); + check_new_xstate(&s, X86_XSS_CET_S); + } + + if ( boot_cpu_has(X86_FEATURE_UINTR) ) + check_new_xstate(&s, X86_XSS_UINTR); + + if ( boot_cpu_has(X86_FEATURE_ARCH_LBR) ) + check_new_xstate(&s, X86_XSS_LBR); + } + + /* Restore old state now the test is done. */ + if ( !set_xcr0(old_xcr0) ) + BUG(); + if ( cpu_has_xsaves ) + set_msr_xss(old_xss); +} + /* Collect the information of processor's extended state */ void xstate_init(struct cpuinfo_x86 *c) { @@ -683,6 +838,9 @@ void xstate_init(struct cpuinfo_x86 *c) if ( setup_xstate_features(bsp) && bsp ) BUG(); + + if ( IS_ENABLED(CONFIG_SELF_TESTS) && bsp ) + xstate_check_sizes(); } int validate_xstate(const struct domain *d, uint64_t xcr0, uint64_t xcr0_accum, diff --git a/xen/include/public/arch-x86/cpufeatureset.h b/xen/include/public/arch-x86/cpufeatureset.h index 6627453e3985..d9eba5e9a714 100644 --- a/xen/include/public/arch-x86/cpufeatureset.h +++ b/xen/include/public/arch-x86/cpufeatureset.h @@ -266,6 +266,7 @@ XEN_CPUFEATURE(IBPB_RET, 8*32+30) /*A IBPB clears RSB/RAS too. */ XEN_CPUFEATURE(AVX512_4VNNIW, 9*32+ 2) /*A AVX512 Neural Network Instructions */ XEN_CPUFEATURE(AVX512_4FMAPS, 9*32+ 3) /*A AVX512 Multiply Accumulation Single Precision */ XEN_CPUFEATURE(FSRM, 9*32+ 4) /*A Fast Short REP MOVS */ +XEN_CPUFEATURE(UINTR, 9*32+ 5) /* User-mode Interrupts */ XEN_CPUFEATURE(AVX512_VP2INTERSECT, 9*32+8) /*a VP2INTERSECT{D,Q} insns */ XEN_CPUFEATURE(SRBDS_CTRL, 9*32+ 9) /* MSR_MCU_OPT_CTRL and RNGDS_MITG_DIS. */ XEN_CPUFEATURE(MD_CLEAR, 9*32+10) /*!A| VERW clears microarchitectural buffers */ @@ -274,8 +275,10 @@ XEN_CPUFEATURE(TSX_FORCE_ABORT, 9*32+13) /* MSR_TSX_FORCE_ABORT.RTM_ABORT */ XEN_CPUFEATURE(SERIALIZE, 9*32+14) /*A SERIALIZE insn */ XEN_CPUFEATURE(HYBRID, 9*32+15) /* Heterogeneous platform */ XEN_CPUFEATURE(TSXLDTRK, 9*32+16) /*a TSX load tracking suspend/resume insns */ +XEN_CPUFEATURE(ARCH_LBR, 9*32+19) /* Architectural Last Branch Record */ XEN_CPUFEATURE(CET_IBT, 9*32+20) /* CET - Indirect Branch Tracking */ XEN_CPUFEATURE(AVX512_FP16, 9*32+23) /*A AVX512 FP16 instructions */ +XEN_CPUFEATURE(AMX_TILE, 9*32+24) /* AMX Tile architecture */ XEN_CPUFEATURE(IBRSB, 9*32+26) /*A IBRS and IBPB support (used by Intel) */ XEN_CPUFEATURE(STIBP, 9*32+27) /*A STIBP */ XEN_CPUFEATURE(L1D_FLUSH, 9*32+28) /*S MSR_FLUSH_CMD and L1D flush. */ From patchwork Mon Jun 17 17:39:17 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13701124 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 550C0C2BD09 for ; Mon, 17 Jun 2024 17:39:45 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.742541.1149391 (Exim 4.92) (envelope-from ) id 1sJGKK-000421-9A; Mon, 17 Jun 2024 17:39:32 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 742541.1149391; Mon, 17 Jun 2024 17:39:32 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKK-0003zi-3w; Mon, 17 Jun 2024 17:39:32 +0000 Received: by outflank-mailman (input) for mailman id 742541; Mon, 17 Jun 2024 17:39:30 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKI-00036g-LB for xen-devel@lists.xenproject.org; Mon, 17 Jun 2024 17:39:30 +0000 Received: from mail-lf1-x135.google.com (mail-lf1-x135.google.com [2a00:1450:4864:20::135]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8b6c3158-2cd0-11ef-b4bb-af5377834399; Mon, 17 Jun 2024 19:39:27 +0200 (CEST) Received: by mail-lf1-x135.google.com with SMTP id 2adb3069b0e04-52b7ffd9f6eso4553420e87.3 for ; Mon, 17 Jun 2024 10:39:27 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6f56da4496sm532010666b.8.2024.06.17.10.39.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 10:39:26 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8b6c3158-2cd0-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718645967; x=1719250767; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=7hZrd7J+i5bTlpEezybGSWjb1th3JpIpYnEjFoW/+BI=; b=eqqLgikruXq+2H7oZBZ19ee9T8xWoxjR2ZAEXu6DtEhhp0r2+vODmYF3fhqluhZrqU zHaLmG5GqG/sQXd2xI9xffLyBN4CIC8erBCoaruu+yn2BCImWBjRjKrtFVWFdEM5PdlB Okt1Me80+Qbx9IBbGIt64pmq0FQ1To8XozU/o= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718645967; x=1719250767; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=7hZrd7J+i5bTlpEezybGSWjb1th3JpIpYnEjFoW/+BI=; b=Ue8JcaPB3WyX61ob3Qd01lRLEFOKiqmCBjg9s2hBVd+UQa46aJADLCBOutYUIP1sNk BarxM2SZ3QP76/gW3Zf7ryhXH7oeBE6bHyJ4AtsDk6lrCf/d37gDQ0QDdMF42onAreT6 yAvggoUb9PN5IwzBNevXq1hEHl8py3Qd/2Q7REHz30K/iJJC7dcx/+J+m5IQOK33xRdm XgADBRfHd0AqbIk+KSj4lcXu3D8kYlAEWMlL7BuQAtwrAdGZjMZnnoo+mC8eWRwuNljN VUqhHZEuSv6JB+4LbwkavEjILIZpVmKi6G4zjwzOcxeENRIs2jviBf9I9lxEggyVf9GD 2Cww== X-Gm-Message-State: AOJu0YzYbSth5amJlnPaqj4tHsj6Dis4HnXGsQtxE8p5pcNfA0oDVeGt AmSvx/sdiYFQybO+d2JItSYERS33BHirhs57d+qrI7ZArz9E39sxXPqJvI1kK15hlmzdXmBuww6 N8tw= X-Google-Smtp-Source: AGHT+IGqswVteiKgyhCwzghy5F3j/dZoVVGu9AAXTgMs18mPwS1twjrtvvS5gzvDBXXDpOpKJ67J7Q== X-Received: by 2002:a05:6512:3113:b0:52b:c089:4579 with SMTP id 2adb3069b0e04-52ca6e91501mr7271082e87.49.1718645966685; Mon, 17 Jun 2024 10:39:26 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Oleksii Kurochko Subject: [PATCH v4 3/7] x86/boot: Collect the Raw CPU Policy earlier on boot Date: Mon, 17 Jun 2024 18:39:17 +0100 Message-Id: <20240617173921.1755439-4-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240617173921.1755439-1-andrew.cooper3@citrix.com> References: <20240617173921.1755439-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 This is a tangle, but it's a small step in the right direction. In the following change, xstate_init() is going to start using the Raw policy. calculate_raw_cpu_policy() is sufficiently separate from the other policies to safely move like this. No functional change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Oleksii Kurochko This is necessary for the forthcoming xstate_{un,}compressed_size() to perform boot-time sanity checks on state components which aren't fully enabled yet. I decided that doing this was better than extending the xstate_{offsets,sizes}[] logic that we're intending to retire in due course. v3: * New. v4: * Adjust commit message a little. --- xen/arch/x86/cpu-policy.c | 1 - xen/arch/x86/setup.c | 4 +++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c index b96f4ee55cc4..5b66f002df05 100644 --- a/xen/arch/x86/cpu-policy.c +++ b/xen/arch/x86/cpu-policy.c @@ -845,7 +845,6 @@ static void __init calculate_hvm_def_policy(void) void __init init_guest_cpu_policies(void) { - calculate_raw_cpu_policy(); calculate_host_policy(); if ( IS_ENABLED(CONFIG_PV) ) diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c index dd51e68dbe5b..eee20bb1753c 100644 --- a/xen/arch/x86/setup.c +++ b/xen/arch/x86/setup.c @@ -1888,7 +1888,9 @@ void asmlinkage __init noreturn __start_xen(unsigned long mbi_p) tsx_init(); /* Needs microcode. May change HLE/RTM feature bits. */ - identify_cpu(&boot_cpu_data); + calculate_raw_cpu_policy(); /* Needs microcode. No other dependenices. */ + + identify_cpu(&boot_cpu_data); /* Needs microcode and raw policy. */ set_in_cr4(X86_CR4_OSFXSR | X86_CR4_OSXMMEXCPT); From patchwork Mon Jun 17 17:39:18 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13701122 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 96A36C2BA1A for ; Mon, 17 Jun 2024 17:39:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.742537.1149354 (Exim 4.92) (envelope-from ) id 1sJGKH-00039T-T8; Mon, 17 Jun 2024 17:39:29 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 742537.1149354; Mon, 17 Jun 2024 17:39:29 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKH-00039D-Nm; Mon, 17 Jun 2024 17:39:29 +0000 Received: by outflank-mailman (input) for mailman id 742537; Mon, 17 Jun 2024 17:39:29 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKG-00036h-Vr for xen-devel@lists.xenproject.org; Mon, 17 Jun 2024 17:39:28 +0000 Received: from mail-ej1-x62a.google.com (mail-ej1-x62a.google.com [2a00:1450:4864:20::62a]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 8b93d0cf-2cd0-11ef-90a3-e314d9c70b13; Mon, 17 Jun 2024 19:39:27 +0200 (CEST) Received: by mail-ej1-x62a.google.com with SMTP id a640c23a62f3a-a689ad8d1f6so564426666b.2 for ; Mon, 17 Jun 2024 10:39:27 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6f56da4496sm532010666b.8.2024.06.17.10.39.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 10:39:27 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8b93d0cf-2cd0-11ef-90a3-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718645967; x=1719250767; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dJvTgVXIVw103tYxIPd995a8BWqGdmptrn91B2t2Nw8=; b=vXETgQC29QAKPiVQWNRD9KJ+OuWoQrXbOHBHKn0j+9HFiDMnmYso40/nXYqfXdloaV rQOD9bo0zPPAIxjiy+sO7e3X3cnL7HfnEYc2j/xvTOU/ttwFvq1w5c/L2ynZbqI26EYH hI2gg8l24qNpQsN9Ymj3DDkzP7b3jkmGS32qs= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718645967; x=1719250767; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dJvTgVXIVw103tYxIPd995a8BWqGdmptrn91B2t2Nw8=; b=vJhmgK+/uqQUBZtJ5xHP0IxN4k97yZ25Os7tbuk/yq2k6tLO6hst0I8IO4xWTP3JVU Yi/eZX0sU806jbdLyvywLrYCnBv5FwocEXmUKi9uMX8SN4aGaGW7pEjVjeyT74UrgFVk m8xKqYMgNIER/DJPfJGk7zVZ3cFAXWsfPpeYxLbwC4zrItkoUWCmCobxK73ZC5AYz9+g pEskoYok3M29Q8Ow/StVMbMSdX/dTdrv13973J5pMgh8pV1vJGwi840dnJVLciQ0dE3p 8JKf8fpQZOvMq/NJcVKPhKkxwIncbnbA7kj7Ij8YP6/R0eSWLByH6IsDTto+E//yOYUq 3DGA== X-Gm-Message-State: AOJu0YykImBmENl4hy6HS87EiCWzFawXQCmem+js082zKTbvTtRJQof4 EmuRKwKd1OOsY6gjvEZRl50x7KK6A2DKsU8/ewxIOSrzRmuFdcrxNjnK9KGoowpy/i7ANXEf6Jp 04wE= X-Google-Smtp-Source: AGHT+IHFvU0Yf0KF7Z3VV7Z6VimGbUT9n427p2ZJ0GGUaMHTo0dABR06CfcAJSIZ2fNhWii6+O+jNg== X-Received: by 2002:a17:906:d111:b0:a6f:10d0:fb85 with SMTP id a640c23a62f3a-a6f60d203e1mr638499366b.19.1718645967214; Mon, 17 Jun 2024 10:39:27 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Oleksii Kurochko Subject: [PATCH v4 4/7] x86/xstate: Rework xstate_ctxt_size() as xstate_uncompressed_size() Date: Mon, 17 Jun 2024 18:39:18 +0100 Message-Id: <20240617173921.1755439-5-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240617173921.1755439-1-andrew.cooper3@citrix.com> References: <20240617173921.1755439-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 We're soon going to need a compressed helper of the same form. The size of the uncompressed image depends on the single element with the largest offset + size. Sadly this isn't always the element with the largest index. Name the per-xstate-component cpu_policy struture, for legibility of the logic in xstate_uncompressed_size(). Cross-check with hardware during boot, and remove hw_uncompressed_size(). This means that the migration paths don't need to mess with XCR0 just to sanity check the buffer size. It also means we can drop the "fastpath" check against xfeature_mask (there to skip some XCR0 writes); this path is going to be dead logic the moment Xen starts using supervisor states itself. The users of hw_uncompressed_size() in xstate_init() can (and indeed need) to be replaced with CPUID instructions. They run with feature_mask in XCR0, and prior to setup_xstate_features() on the BSP. No practical change. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Oleksii Kurochko v2: * Scan all features. LWP/APX_F are out-of-order. v3: * Rebase over boot time check. * Use the raw CPU policy. v4: * Explain the ASSERT() checking X86_XCR0_STATES. * Drop the xfeature_mask check. * Drop the comment about 0, because we'll probably never be able to clean up the use from the CPUID path. --- xen/arch/x86/domctl.c | 2 +- xen/arch/x86/hvm/hvm.c | 2 +- xen/arch/x86/include/asm/xstate.h | 2 +- xen/arch/x86/xstate.c | 76 ++++++++++++++++------------ xen/include/xen/lib/x86/cpu-policy.h | 2 +- 5 files changed, 49 insertions(+), 35 deletions(-) diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c index 335aedf46d03..9190e11faaa3 100644 --- a/xen/arch/x86/domctl.c +++ b/xen/arch/x86/domctl.c @@ -833,7 +833,7 @@ long arch_do_domctl( uint32_t offset = 0; #define PV_XSAVE_HDR_SIZE (2 * sizeof(uint64_t)) -#define PV_XSAVE_SIZE(xcr0) (PV_XSAVE_HDR_SIZE + xstate_ctxt_size(xcr0)) +#define PV_XSAVE_SIZE(xcr0) (PV_XSAVE_HDR_SIZE + xstate_uncompressed_size(xcr0)) ret = -ESRCH; if ( (evc->vcpu >= d->max_vcpus) || diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c index 8334ab171110..7f4b627b1f5f 100644 --- a/xen/arch/x86/hvm/hvm.c +++ b/xen/arch/x86/hvm/hvm.c @@ -1206,7 +1206,7 @@ HVM_REGISTER_SAVE_RESTORE(CPU, hvm_save_cpu_ctxt, NULL, hvm_load_cpu_ctxt, 1, #define HVM_CPU_XSAVE_SIZE(xcr0) (offsetof(struct hvm_hw_cpu_xsave, \ save_area) + \ - xstate_ctxt_size(xcr0)) + xstate_uncompressed_size(xcr0)) static int cf_check hvm_save_cpu_xsave_states( struct vcpu *v, hvm_domain_context_t *h) diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h index c08c267884f0..f5115199d4f9 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -107,7 +107,7 @@ void compress_xsave_states(struct vcpu *v, const void *src, unsigned int size); void xstate_free_save_area(struct vcpu *v); int xstate_alloc_save_area(struct vcpu *v); void xstate_init(struct cpuinfo_x86 *c); -unsigned int xstate_ctxt_size(u64 xcr0); +unsigned int xstate_uncompressed_size(uint64_t xcr0); static inline uint64_t xgetbv(unsigned int index) { diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 650206d9d2b6..8edc4792a8fd 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -8,6 +8,8 @@ #include #include #include + +#include #include #include #include @@ -183,7 +185,7 @@ void expand_xsave_states(const struct vcpu *v, void *dest, unsigned int size) /* Check there is state to serialise (i.e. at least an XSAVE_HDR) */ BUG_ON(!v->arch.xcr0_accum); /* Check there is the correct room to decompress into. */ - BUG_ON(size != xstate_ctxt_size(v->arch.xcr0_accum)); + BUG_ON(size != xstate_uncompressed_size(v->arch.xcr0_accum)); if ( !(xstate->xsave_hdr.xcomp_bv & XSTATE_COMPACTION_ENABLED) ) { @@ -245,7 +247,7 @@ void compress_xsave_states(struct vcpu *v, const void *src, unsigned int size) u64 xstate_bv, valid; BUG_ON(!v->arch.xcr0_accum); - BUG_ON(size != xstate_ctxt_size(v->arch.xcr0_accum)); + BUG_ON(size != xstate_uncompressed_size(v->arch.xcr0_accum)); ASSERT(!xsave_area_compressed(src)); xstate_bv = ((const struct xsave_struct *)src)->xsave_hdr.xstate_bv; @@ -553,32 +555,6 @@ void xstate_free_save_area(struct vcpu *v) v->arch.xsave_area = NULL; } -static unsigned int hw_uncompressed_size(uint64_t xcr0) -{ - u64 act_xcr0 = get_xcr0(); - unsigned int size; - bool ok = set_xcr0(xcr0); - - ASSERT(ok); - size = cpuid_count_ebx(XSTATE_CPUID, 0); - ok = set_xcr0(act_xcr0); - ASSERT(ok); - - return size; -} - -/* Fastpath for common xstate size requests, avoiding reloads of xcr0. */ -unsigned int xstate_ctxt_size(u64 xcr0) -{ - if ( xcr0 == xfeature_mask ) - return xsave_cntxt_size; - - if ( xcr0 == 0 ) /* TODO: clean up paths passing 0 in here. */ - return 0; - - return hw_uncompressed_size(xcr0); -} - static bool valid_xcr0(uint64_t xcr0) { /* FP must be unconditionally set. */ @@ -611,6 +587,38 @@ static bool valid_xcr0(uint64_t xcr0) return true; } +unsigned int xstate_uncompressed_size(uint64_t xcr0) +{ + unsigned int size = XSTATE_AREA_MIN_SIZE, i; + + /* Non-XCR0 states don't exist in an uncompressed image. */ + ASSERT((xcr0 & ~X86_XCR0_STATES) == 0); + + if ( xcr0 == 0 ) + return 0; + + if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_FP) ) + return size; + + /* + * For the non-legacy states, search all activate states and find the + * maximum offset+size. Some states (e.g. LWP, APX_F) are out-of-order + * with respect their index. + */ + xcr0 &= ~(X86_XCR0_SSE | X86_XCR0_FP); + for_each_set_bit ( i, &xcr0, 63 ) + { + const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i]; + unsigned int s = c->offset + c->size; + + ASSERT(c->offset && c->size); + + size = max(size, s); + } + + return size; +} + struct xcheck_state { uint64_t states; uint32_t uncomp_size; @@ -619,7 +627,7 @@ struct xcheck_state { static void __init check_new_xstate(struct xcheck_state *s, uint64_t new) { - uint32_t hw_size; + uint32_t hw_size, xen_size; BUILD_BUG_ON(X86_XCR0_STATES & X86_XSS_STATES); @@ -651,6 +659,12 @@ static void __init check_new_xstate(struct xcheck_state *s, uint64_t new) s->uncomp_size = hw_size; + xen_size = xstate_uncompressed_size(s->states & X86_XCR0_STATES); + + if ( xen_size != hw_size ) + panic("XSTATE 0x%016"PRIx64", uncompressed hw size %#x != xen size %#x\n", + s->states, hw_size, xen_size); + /* * Check the compressed size, if available. All components strictly * appear in index order. In principle there are no holes, but some @@ -826,14 +840,14 @@ void xstate_init(struct cpuinfo_x86 *c) * xsave_cntxt_size is the max size required by enabled features. * We know FP/SSE and YMM about eax, and nothing about edx at present. */ - xsave_cntxt_size = hw_uncompressed_size(feature_mask); + xsave_cntxt_size = cpuid_count_ebx(0xd, 0); printk("xstate: size: %#x and states: %#"PRIx64"\n", xsave_cntxt_size, xfeature_mask); } else { BUG_ON(xfeature_mask != feature_mask); - BUG_ON(xsave_cntxt_size != hw_uncompressed_size(feature_mask)); + BUG_ON(xsave_cntxt_size != cpuid_count_ebx(0xd, 0)); } if ( setup_xstate_features(bsp) && bsp ) diff --git a/xen/include/xen/lib/x86/cpu-policy.h b/xen/include/xen/lib/x86/cpu-policy.h index d5e447e9dc06..d26012c6da78 100644 --- a/xen/include/xen/lib/x86/cpu-policy.h +++ b/xen/include/xen/lib/x86/cpu-policy.h @@ -248,7 +248,7 @@ struct cpu_policy }; /* Per-component common state. Valid for i >= 2. */ - struct { + struct xstate_component { uint32_t size, offset; bool xss:1, align:1; uint32_t _res_d; From patchwork Mon Jun 17 17:39:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13701120 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AC514C2BB9A for ; Mon, 17 Jun 2024 17:39:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.742539.1149380 (Exim 4.92) (envelope-from ) id 1sJGKJ-0003nz-F3; Mon, 17 Jun 2024 17:39:31 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 742539.1149380; Mon, 17 Jun 2024 17:39:31 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKJ-0003nq-9G; Mon, 17 Jun 2024 17:39:31 +0000 Received: by outflank-mailman (input) for mailman id 742539; Mon, 17 Jun 2024 17:39:29 +0000 Received: from se1-gles-sth1-in.inumbo.com ([159.253.27.254] helo=se1-gles-sth1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKH-00036h-L3 for xen-devel@lists.xenproject.org; Mon, 17 Jun 2024 17:39:29 +0000 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [2a00:1450:4864:20::634]) by se1-gles-sth1.inumbo.com (Halon) with ESMTPS id 8c2e2180-2cd0-11ef-90a3-e314d9c70b13; Mon, 17 Jun 2024 19:39:29 +0200 (CEST) Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-a6f0c3d0792so531942266b.3 for ; Mon, 17 Jun 2024 10:39:28 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6f56da4496sm532010666b.8.2024.06.17.10.39.27 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 10:39:27 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8c2e2180-2cd0-11ef-90a3-e314d9c70b13 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718645968; x=1719250768; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=3uvI0uPWUeQgT6Y03c0cvjouO28il7Tm0yQNEqwNcQQ=; b=cUcmywmQ9KtmYsGeHwwNG/Z0718OLni1WCnSOh66t1FlsDC2wcTrFBKi7vBr8Bhsyy oXkfPYYgXSmkjswirH2DKu6ThShxOHuhpgmxzIQft+1M0MHpWetYM9SK1Xy+FMvblP5t MqBKcbkLOL+2WeCzSzk6lDmsjm68csfRs39WM= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718645968; x=1719250768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=3uvI0uPWUeQgT6Y03c0cvjouO28il7Tm0yQNEqwNcQQ=; b=FG1egbY4DGvXGAawn6ASbeQFFiRTr2bwq0gAM6gmrAhrvNSR3kJsCBp12wQdhOWvQ/ WiQTS7IpOYu/5SeO7PF2pv8OerUmY/fIdu/lTldNxCfkZffCsY9Gf3ghdoPvZrLWyfzJ IK+CCcfzROgAfWiEfLMaeTCqVrPBDy3w4skTf7wgMDySjvWyUXiSbAiJAo9kRRV6pX5V TxRPJLeXlBTVcbUNEGX+be2sQ5iFzTiA6VRM0VcnDQJaSaSrEJUuMZWg0chHltviPavF E0IeJPdeOtL7tHHRCEQv+TPJ9zLWl6yYLVcnb0CXnWv4tctwDSTymyLn/gySs0Qg9CHw 9iAA== X-Gm-Message-State: AOJu0YxzpQmf+Ze9aUwPw4szhi/PnTs6qBNhFdC/3dkrFFGhLBXXgboF XtUMHfi8uG+fqN2z65QePd2aH1ed5wwzi4TSAN2WRRSHBTx9uKzlu+ZdCWdo1wneZAxlaaTUeih 3+CY= X-Google-Smtp-Source: AGHT+IHfbAFMyFk+KRS69Rs46acGyugZCHtGPpEtfuM/I3FtBCHjLXabA2YjEFtlhhgRngQufGCmTw== X-Received: by 2002:a17:906:1854:b0:a6f:1dbb:d38b with SMTP id a640c23a62f3a-a6f60d2bd6dmr685432466b.28.1718645968045; Mon, 17 Jun 2024 10:39:28 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Oleksii Kurochko Subject: [PATCH v4 5/7] x86/cpu-policy: Simplify recalculate_xstate() Date: Mon, 17 Jun 2024 18:39:19 +0100 Message-Id: <20240617173921.1755439-6-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240617173921.1755439-1-andrew.cooper3@citrix.com> References: <20240617173921.1755439-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 Make use of xstate_uncompressed_size() helper rather than maintaining the running calculation while accumulating feature components. The rest of the CPUID data can come direct from the raw cpu policy. All per-component data form an ABI through the behaviour of the X{SAVE,RSTOR}* instructions. Use for_each_set_bit() rather than opencoding a slightly awkward version of it. Mask the attributes in ecx down based on the visible features. This isn't actually necessary for any components or attributes defined at the time of writing (up to AMX), but is added out of an abundance of caution. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Oleksii Kurochko v2: * Tie ALIGN64 to xsavec rather than xsaves. v3: * Tweak commit message. --- xen/arch/x86/cpu-policy.c | 55 +++++++++++-------------------- xen/arch/x86/include/asm/xstate.h | 1 + 2 files changed, 21 insertions(+), 35 deletions(-) diff --git a/xen/arch/x86/cpu-policy.c b/xen/arch/x86/cpu-policy.c index 5b66f002df05..304dc20cfab8 100644 --- a/xen/arch/x86/cpu-policy.c +++ b/xen/arch/x86/cpu-policy.c @@ -193,8 +193,7 @@ static void sanitise_featureset(uint32_t *fs) static void recalculate_xstate(struct cpu_policy *p) { uint64_t xstates = XSTATE_FP_SSE; - uint32_t xstate_size = XSTATE_AREA_MIN_SIZE; - unsigned int i, Da1 = p->xstate.Da1; + unsigned int i, ecx_mask = 0, Da1 = p->xstate.Da1; /* * The Da1 leaf is the only piece of information preserved in the common @@ -206,61 +205,47 @@ static void recalculate_xstate(struct cpu_policy *p) return; if ( p->basic.avx ) - { xstates |= X86_XCR0_YMM; - xstate_size = max(xstate_size, - xstate_offsets[X86_XCR0_YMM_POS] + - xstate_sizes[X86_XCR0_YMM_POS]); - } if ( p->feat.mpx ) - { xstates |= X86_XCR0_BNDREGS | X86_XCR0_BNDCSR; - xstate_size = max(xstate_size, - xstate_offsets[X86_XCR0_BNDCSR_POS] + - xstate_sizes[X86_XCR0_BNDCSR_POS]); - } if ( p->feat.avx512f ) - { xstates |= X86_XCR0_OPMASK | X86_XCR0_ZMM | X86_XCR0_HI_ZMM; - xstate_size = max(xstate_size, - xstate_offsets[X86_XCR0_HI_ZMM_POS] + - xstate_sizes[X86_XCR0_HI_ZMM_POS]); - } if ( p->feat.pku ) - { xstates |= X86_XCR0_PKRU; - xstate_size = max(xstate_size, - xstate_offsets[X86_XCR0_PKRU_POS] + - xstate_sizes[X86_XCR0_PKRU_POS]); - } - p->xstate.max_size = xstate_size; + /* Subleaf 0 */ + p->xstate.max_size = + xstate_uncompressed_size(xstates & ~XSTATE_XSAVES_ONLY); p->xstate.xcr0_low = xstates & ~XSTATE_XSAVES_ONLY; p->xstate.xcr0_high = (xstates & ~XSTATE_XSAVES_ONLY) >> 32; + /* Subleaf 1 */ p->xstate.Da1 = Da1; + if ( p->xstate.xsavec ) + ecx_mask |= XSTATE_ALIGN64; + if ( p->xstate.xsaves ) { + ecx_mask |= XSTATE_XSS; p->xstate.xss_low = xstates & XSTATE_XSAVES_ONLY; p->xstate.xss_high = (xstates & XSTATE_XSAVES_ONLY) >> 32; } - else - xstates &= ~XSTATE_XSAVES_ONLY; - for ( i = 2; i < min(63UL, ARRAY_SIZE(p->xstate.comp)); ++i ) + /* Subleafs 2+ */ + xstates &= ~XSTATE_FP_SSE; + BUILD_BUG_ON(ARRAY_SIZE(p->xstate.comp) < 63); + for_each_set_bit ( i, &xstates, 63 ) { - uint64_t curr_xstate = 1UL << i; - - if ( !(xstates & curr_xstate) ) - continue; - - p->xstate.comp[i].size = xstate_sizes[i]; - p->xstate.comp[i].offset = xstate_offsets[i]; - p->xstate.comp[i].xss = curr_xstate & XSTATE_XSAVES_ONLY; - p->xstate.comp[i].align = curr_xstate & xstate_align; + /* + * Pass through size (eax) and offset (ebx) directly. Visbility of + * attributes in ecx limited by visible features in Da1. + */ + p->xstate.raw[i].a = raw_cpu_policy.xstate.raw[i].a; + p->xstate.raw[i].b = raw_cpu_policy.xstate.raw[i].b; + p->xstate.raw[i].c = raw_cpu_policy.xstate.raw[i].c & ecx_mask; } } diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h index f5115199d4f9..bfb66dd766b6 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -40,6 +40,7 @@ extern uint32_t mxcsr_mask; #define XSTATE_XSAVES_ONLY 0 #define XSTATE_COMPACTION_ENABLED (1ULL << 63) +#define XSTATE_XSS (1U << 0) #define XSTATE_ALIGN64 (1U << 1) extern u64 xfeature_mask; From patchwork Mon Jun 17 17:39:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13701119 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 66A21C27C79 for ; Mon, 17 Jun 2024 17:39:42 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.742542.1149408 (Exim 4.92) (envelope-from ) id 1sJGKL-0004V6-Ij; Mon, 17 Jun 2024 17:39:33 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 742542.1149408; Mon, 17 Jun 2024 17:39:33 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKL-0004UF-Dw; Mon, 17 Jun 2024 17:39:33 +0000 Received: by outflank-mailman (input) for mailman id 742542; Mon, 17 Jun 2024 17:39:31 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKJ-00036g-LD for xen-devel@lists.xenproject.org; Mon, 17 Jun 2024 17:39:31 +0000 Received: from mail-ej1-x634.google.com (mail-ej1-x634.google.com [2a00:1450:4864:20::634]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8c77ddd1-2cd0-11ef-b4bb-af5377834399; Mon, 17 Jun 2024 19:39:29 +0200 (CEST) Received: by mail-ej1-x634.google.com with SMTP id a640c23a62f3a-a6f176c5c10so549351266b.2 for ; Mon, 17 Jun 2024 10:39:29 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6f56da4496sm532010666b.8.2024.06.17.10.39.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 10:39:28 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8c77ddd1-2cd0-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718645968; x=1719250768; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iaQNeWXVPJOBqi4Jz3ws3Uw0UDtY4Ak6EMg/xM7DVZo=; b=VrFGtfmSSLW8DVJ5vvTBz2Ob+XtmWvZSEd+RK64X477iC0lyGg6e8MOS/E/x1tFjL8 vBZ+VV4H2fmzcDJ3m0IJJic2VpzyGLF4v87hqB8V6gqv7Ke7bWUm9LfLX6H2QBR9sY9K qz0VkGNmcAZox1QtsBWdCcajvFyb5OTTaosdE= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718645968; x=1719250768; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iaQNeWXVPJOBqi4Jz3ws3Uw0UDtY4Ak6EMg/xM7DVZo=; b=lWTsItn18jIuVvLUXWJdvK8FVMNg+abRB24p3G5pyfdFRMXW5Q06JAnF4bZVZ8EdHH UQUf2aNgsjTKorAwnScTsIbLbsxtNugB4aFEjGJJi00LbL7x1Rjd7YWU4imRSug7zDfq k2YkXiPhjvJvYEqIQtkuupXHa1lTTkjViCvmGhM7tRowjuCGzIdhfGPDhFSgL142ue1F A1HX96teIC9qCk83ylPKj4aaJIDpk54dJO6rd4pOH5Qt9WgkeKnG5fsjfR5CIaOE1XmY ZHmMBJTeR8qIo71HnuGMGvYZgoOTXjc1+0TPrSq7DodVVO/nVwEVFhAJZQvXnPaumqio OPMQ== X-Gm-Message-State: AOJu0YyVFnU7tWum9jhfb4wyW9VSasNcuyw+y3nRAY9pLwahK+IRMuHV bC8Xn1Knl8SIoluP/2PLel9UzA2SdoaKT8SdJL//QnZpggxlAzyUEQjRYCkYMgl/LIIbwFmWcyW KO1U= X-Google-Smtp-Source: AGHT+IGKrdBfI9Nm+aHZ6UkUA5/JNpbfFSVehqAWMLC2Ni5M753x4xlQAZiw4qCHY7guT4Ih8WAYvg== X-Received: by 2002:a17:906:2c0d:b0:a64:a091:91f2 with SMTP id a640c23a62f3a-a6f60d40c99mr732397266b.37.1718645968592; Mon, 17 Jun 2024 10:39:28 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Oleksii Kurochko Subject: [PATCH v4 6/7] x86/cpuid: Fix handling of XSAVE dynamic leaves Date: Mon, 17 Jun 2024 18:39:20 +0100 Message-Id: <20240617173921.1755439-7-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240617173921.1755439-1-andrew.cooper3@citrix.com> References: <20240617173921.1755439-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 First, if XSAVE is available in hardware but not visible to the guest, the dynamic leaves shouldn't be filled in. Second, the comment concerning XSS state is wrong. VT-x doesn't manage host/guest state automatically, but there is provision for "host only" bits to be set, so the implications are still accurate. Introduce xstate_compressed_size() to mirror the uncompressed one. Cross check it at boot. Signed-off-by: Andrew Cooper Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Oleksii Kurochko v3: * Adjust commit message about !XSAVE guests * Rebase over boot time cross check * Use raw policy v4: * Drop the TODO comment. The CPUID path is always liable to pass 0 here. * ASSERT() a nonzero c->size like we do in the uncompressed helper. --- xen/arch/x86/cpuid.c | 24 +++++++-------------- xen/arch/x86/include/asm/xstate.h | 1 + xen/arch/x86/xstate.c | 36 +++++++++++++++++++++++++++++++ 3 files changed, 45 insertions(+), 16 deletions(-) diff --git a/xen/arch/x86/cpuid.c b/xen/arch/x86/cpuid.c index 7a38e032146a..a822e80c7ea7 100644 --- a/xen/arch/x86/cpuid.c +++ b/xen/arch/x86/cpuid.c @@ -330,23 +330,15 @@ void guest_cpuid(const struct vcpu *v, uint32_t leaf, case XSTATE_CPUID: switch ( subleaf ) { - case 1: - if ( !p->xstate.xsavec && !p->xstate.xsaves ) - break; - - /* - * TODO: Figure out what to do for XSS state. VT-x manages host - * vs guest MSR_XSS automatically, so as soon as we start - * supporting any XSS states, the wrong XSS will be in context. - */ - BUILD_BUG_ON(XSTATE_XSAVES_ONLY != 0); - fallthrough; case 0: - /* - * Read CPUID[0xD,0/1].EBX from hardware. They vary with enabled - * XSTATE, and appropriate XCR0|XSS are in context. - */ - res->b = cpuid_count_ebx(leaf, subleaf); + if ( p->basic.xsave ) + res->b = xstate_uncompressed_size(v->arch.xcr0); + break; + + case 1: + if ( p->xstate.xsavec ) + res->b = xstate_compressed_size(v->arch.xcr0 | + v->arch.msrs->xss.raw); break; } break; diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h index bfb66dd766b6..da1d89d2f416 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -109,6 +109,7 @@ void xstate_free_save_area(struct vcpu *v); int xstate_alloc_save_area(struct vcpu *v); void xstate_init(struct cpuinfo_x86 *c); unsigned int xstate_uncompressed_size(uint64_t xcr0); +unsigned int xstate_compressed_size(uint64_t xstates); static inline uint64_t xgetbv(unsigned int index) { diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 8edc4792a8fd..31bf2dc95f57 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -619,6 +619,36 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0) return size; } +unsigned int xstate_compressed_size(uint64_t xstates) +{ + unsigned int i, size = XSTATE_AREA_MIN_SIZE; + + if ( xstates == 0 ) + return 0; + + if ( xstates <= (X86_XCR0_SSE | X86_XCR0_FP) ) + return size; + + /* + * For the compressed size, every non-legacy component matters. Some + * componenets require aligning to 64 first. + */ + xstates &= ~(X86_XCR0_SSE | X86_XCR0_FP); + for_each_set_bit ( i, &xstates, 63 ) + { + const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i]; + + ASSERT(c->size); + + if ( c->align ) + size = ROUNDUP(size, 64); + + size += c->size; + } + + return size; +} + struct xcheck_state { uint64_t states; uint32_t uncomp_size; @@ -681,6 +711,12 @@ static void __init check_new_xstate(struct xcheck_state *s, uint64_t new) s->states, &new, hw_size, s->comp_size); s->comp_size = hw_size; + + xen_size = xstate_compressed_size(s->states); + + if ( xen_size != hw_size ) + panic("XSTATE 0x%016"PRIx64", compressed hw size %#x != xen size %#x\n", + s->states, hw_size, xen_size); } else if ( hw_size ) /* Compressed size reported, but no XSAVEC ? */ { From patchwork Mon Jun 17 17:39:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrew Cooper X-Patchwork-Id: 13701125 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.xenproject.org (lists.xenproject.org [192.237.175.120]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6DE53C2BD08 for ; Mon, 17 Jun 2024 17:39:43 +0000 (UTC) Received: from list by lists.xenproject.org with outflank-mailman.742543.1149414 (Exim 4.92) (envelope-from ) id 1sJGKM-0004cy-6Q; Mon, 17 Jun 2024 17:39:34 +0000 X-Outflank-Mailman: Message body and most headers restored to incoming version Received: by outflank-mailman (output) from mailman id 742543.1149414; Mon, 17 Jun 2024 17:39:34 +0000 Received: from localhost ([127.0.0.1] helo=lists.xenproject.org) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKM-0004az-0p; Mon, 17 Jun 2024 17:39:34 +0000 Received: by outflank-mailman (input) for mailman id 742543; Mon, 17 Jun 2024 17:39:32 +0000 Received: from se1-gles-flk1-in.inumbo.com ([94.247.172.50] helo=se1-gles-flk1.inumbo.com) by lists.xenproject.org with esmtp (Exim 4.92) (envelope-from ) id 1sJGKK-00036g-LI for xen-devel@lists.xenproject.org; Mon, 17 Jun 2024 17:39:32 +0000 Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [2a00:1450:4864:20::629]) by se1-gles-flk1.inumbo.com (Halon) with ESMTPS id 8cdfb16e-2cd0-11ef-b4bb-af5377834399; Mon, 17 Jun 2024 19:39:30 +0200 (CEST) Received: by mail-ej1-x629.google.com with SMTP id a640c23a62f3a-a6f0dc80ab9so704391266b.2 for ; Mon, 17 Jun 2024 10:39:30 -0700 (PDT) Received: from andrewcoop.eng.citrite.net ([160.101.139.1]) by smtp.gmail.com with ESMTPSA id a640c23a62f3a-a6f56da4496sm532010666b.8.2024.06.17.10.39.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 17 Jun 2024 10:39:28 -0700 (PDT) X-BeenThere: xen-devel@lists.xenproject.org List-Id: Xen developer discussion List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Errors-To: xen-devel-bounces@lists.xenproject.org Precedence: list Sender: "Xen-devel" X-Inumbo-ID: 8cdfb16e-2cd0-11ef-b4bb-af5377834399 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=citrix.com; s=google; t=1718645969; x=1719250769; darn=lists.xenproject.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=91rqmVIJvtgwrFeMK2tOHbISsSTuJ9u05BoupIBlW0Q=; b=dlPLAIWROcYXZC/TEGChqyb2bI0PIbZX5zYiAJg2wLV5M1qOWO+0+Dtzs17Bz4ceVJ GAH7ORlU8qPN0DF3trGnF0CQBHdrVg92sIKoH0OkEmvDT4YUWMij3BKa3t9F+yjQlW+n Kae2NElpdL9uOvbHI3Cla4uvQl9ce0uzYZOlc= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718645969; x=1719250769; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=91rqmVIJvtgwrFeMK2tOHbISsSTuJ9u05BoupIBlW0Q=; b=dBs8nasJTnogFK/JdKecSy5zreTkaJPU2WVFTEV2PobpYsbzOkzO0a7A5JEIV+iooR 56sfkGvRJrXswo6OZM0HzdnVUOKAfxPXh1sykjUXOhPC1aFnJSWQFs0ZrhsjtgcEMIKl VWhxCKUO0mU8owFwQba+i3DwDCj14gquxyf4phXFKj0mZPnPmofw6J2spi3JYeP1gH6n A34wYfmAONa9nWUaUX/JDYKZS3jsvQibhr9MGARqknprSz7/aV9b7wzDdKBV0PB1cjMR AQLn/VLBupyft3M2ID4ooWpFi5p1uzaOH8nqQcO7EebjVffJja/ijV/SXtZ6jq6H+TlI /ZEg== X-Gm-Message-State: AOJu0Yxl1hj+PztDiRAdiFY40bnT9tDNKYMNfaSZtdAzdBGMQqvQLApm 6loSr//tXt4dPCJEw0wq8ZdAy0iU6smW9l7lL5AwqVCqXMw9PvFhNXWgy+d+rSps/fk93GuAqCc 76Kk= X-Google-Smtp-Source: AGHT+IGzn5ZkK1YZ3Abyw+STSNcbhHiblepXrKhd7oNDvDgw69jiQUs3j43NO0ZVmfS9SARRXyaAxQ== X-Received: by 2002:a17:907:d312:b0:a6f:668b:3434 with SMTP id a640c23a62f3a-a6f668b35f7mr806562166b.31.1718645969387; Mon, 17 Jun 2024 10:39:29 -0700 (PDT) From: Andrew Cooper To: Xen-devel Cc: Andrew Cooper , Jan Beulich , Jan Beulich , =?utf-8?q?Roger_Pau_Monn=C3=A9?= , Oleksii Kurochko Subject: [PATCH v4 7/7] x86/defns: Clean up X86_{XCR0,XSS}_* constants Date: Mon, 17 Jun 2024 18:39:21 +0100 Message-Id: <20240617173921.1755439-8-andrew.cooper3@citrix.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240617173921.1755439-1-andrew.cooper3@citrix.com> References: <20240617173921.1755439-1-andrew.cooper3@citrix.com> MIME-Version: 1.0 With the exception of one case in read_bndcfgu() which can use ilog2(), the *_POS defines are unused. Drop them. X86_XCR0_X87 is the name used by both the SDM and APM, rather than X86_XCR0_FP. No functional change. Signed-off-by: Andrew Cooper Acked-by: Jan Beulich --- CC: Jan Beulich CC: Roger Pau Monné CC: Oleksii Kurochko v3: * New --- xen/arch/x86/i387.c | 2 +- xen/arch/x86/include/asm/x86-defns.h | 32 ++++++++++------------------ xen/arch/x86/include/asm/xstate.h | 4 ++-- xen/arch/x86/xstate.c | 18 ++++++++-------- 4 files changed, 23 insertions(+), 33 deletions(-) diff --git a/xen/arch/x86/i387.c b/xen/arch/x86/i387.c index 7a4297cc921e..fcdee10a6e69 100644 --- a/xen/arch/x86/i387.c +++ b/xen/arch/x86/i387.c @@ -369,7 +369,7 @@ void vcpu_setup_fpu(struct vcpu *v, struct xsave_struct *xsave_area, { v->arch.xsave_area->xsave_hdr.xstate_bv &= ~XSTATE_FP_SSE; if ( fcw_default != FCW_DEFAULT ) - v->arch.xsave_area->xsave_hdr.xstate_bv |= X86_XCR0_FP; + v->arch.xsave_area->xsave_hdr.xstate_bv |= X86_XCR0_X87; } } diff --git a/xen/arch/x86/include/asm/x86-defns.h b/xen/arch/x86/include/asm/x86-defns.h index d7602ab225c4..3bcdbaccd3aa 100644 --- a/xen/arch/x86/include/asm/x86-defns.h +++ b/xen/arch/x86/include/asm/x86-defns.h @@ -79,25 +79,16 @@ /* * XSTATE component flags in XCR0 | MSR_XSS */ -#define X86_XCR0_FP_POS 0 -#define X86_XCR0_FP (1ULL << X86_XCR0_FP_POS) -#define X86_XCR0_SSE_POS 1 -#define X86_XCR0_SSE (1ULL << X86_XCR0_SSE_POS) -#define X86_XCR0_YMM_POS 2 -#define X86_XCR0_YMM (1ULL << X86_XCR0_YMM_POS) -#define X86_XCR0_BNDREGS_POS 3 -#define X86_XCR0_BNDREGS (1ULL << X86_XCR0_BNDREGS_POS) -#define X86_XCR0_BNDCSR_POS 4 -#define X86_XCR0_BNDCSR (1ULL << X86_XCR0_BNDCSR_POS) -#define X86_XCR0_OPMASK_POS 5 -#define X86_XCR0_OPMASK (1ULL << X86_XCR0_OPMASK_POS) -#define X86_XCR0_ZMM_POS 6 -#define X86_XCR0_ZMM (1ULL << X86_XCR0_ZMM_POS) -#define X86_XCR0_HI_ZMM_POS 7 -#define X86_XCR0_HI_ZMM (1ULL << X86_XCR0_HI_ZMM_POS) +#define X86_XCR0_X87 (_AC(1, ULL) << 0) +#define X86_XCR0_SSE (_AC(1, ULL) << 1) +#define X86_XCR0_YMM (_AC(1, ULL) << 2) +#define X86_XCR0_BNDREGS (_AC(1, ULL) << 3) +#define X86_XCR0_BNDCSR (_AC(1, ULL) << 4) +#define X86_XCR0_OPMASK (_AC(1, ULL) << 5) +#define X86_XCR0_ZMM (_AC(1, ULL) << 6) +#define X86_XCR0_HI_ZMM (_AC(1, ULL) << 7) #define X86_XSS_PROC_TRACE (_AC(1, ULL) << 8) -#define X86_XCR0_PKRU_POS 9 -#define X86_XCR0_PKRU (1ULL << X86_XCR0_PKRU_POS) +#define X86_XCR0_PKRU (_AC(1, ULL) << 9) #define X86_XSS_PASID (_AC(1, ULL) << 10) #define X86_XSS_CET_U (_AC(1, ULL) << 11) #define X86_XSS_CET_S (_AC(1, ULL) << 12) @@ -107,11 +98,10 @@ #define X86_XSS_HWP (_AC(1, ULL) << 16) #define X86_XCR0_TILE_CFG (_AC(1, ULL) << 17) #define X86_XCR0_TILE_DATA (_AC(1, ULL) << 18) -#define X86_XCR0_LWP_POS 62 -#define X86_XCR0_LWP (1ULL << X86_XCR0_LWP_POS) +#define X86_XCR0_LWP (_AC(1, ULL) << 62) #define X86_XCR0_STATES \ - (X86_XCR0_FP | X86_XCR0_SSE | X86_XCR0_YMM | X86_XCR0_BNDREGS | \ + (X86_XCR0_X87 | X86_XCR0_SSE | X86_XCR0_YMM | X86_XCR0_BNDREGS | \ X86_XCR0_BNDCSR | X86_XCR0_OPMASK | X86_XCR0_ZMM | \ X86_XCR0_HI_ZMM | X86_XCR0_PKRU | X86_XCR0_TILE_CFG | \ X86_XCR0_TILE_DATA | \ diff --git a/xen/arch/x86/include/asm/xstate.h b/xen/arch/x86/include/asm/xstate.h index da1d89d2f416..f4a8e5f814a0 100644 --- a/xen/arch/x86/include/asm/xstate.h +++ b/xen/arch/x86/include/asm/xstate.h @@ -29,8 +29,8 @@ extern uint32_t mxcsr_mask; #define XSAVE_HDR_OFFSET FXSAVE_SIZE #define XSTATE_AREA_MIN_SIZE (FXSAVE_SIZE + XSAVE_HDR_SIZE) -#define XSTATE_FP_SSE (X86_XCR0_FP | X86_XCR0_SSE) -#define XCNTXT_MASK (X86_XCR0_FP | X86_XCR0_SSE | X86_XCR0_YMM | \ +#define XSTATE_FP_SSE (X86_XCR0_X87 | X86_XCR0_SSE) +#define XCNTXT_MASK (X86_XCR0_X87 | X86_XCR0_SSE | X86_XCR0_YMM | \ X86_XCR0_OPMASK | X86_XCR0_ZMM | X86_XCR0_HI_ZMM | \ XSTATE_NONLAZY) diff --git a/xen/arch/x86/xstate.c b/xen/arch/x86/xstate.c index 31bf2dc95f57..2acd02449dba 100644 --- a/xen/arch/x86/xstate.c +++ b/xen/arch/x86/xstate.c @@ -313,7 +313,7 @@ void xsave(struct vcpu *v, uint64_t mask) "=m" (*ptr), \ "a" (lmask), "d" (hmask), "D" (ptr)) - if ( fip_width == 8 || !(mask & X86_XCR0_FP) ) + if ( fip_width == 8 || !(mask & X86_XCR0_X87) ) { XSAVE("0x48,"); } @@ -366,7 +366,7 @@ void xsave(struct vcpu *v, uint64_t mask) fip_width = 8; } #undef XSAVE - if ( mask & X86_XCR0_FP ) + if ( mask & X86_XCR0_X87 ) ptr->fpu_sse.x[FPU_WORD_SIZE_OFFSET] = fip_width; } @@ -558,7 +558,7 @@ void xstate_free_save_area(struct vcpu *v) static bool valid_xcr0(uint64_t xcr0) { /* FP must be unconditionally set. */ - if ( !(xcr0 & X86_XCR0_FP) ) + if ( !(xcr0 & X86_XCR0_X87) ) return false; /* YMM depends on SSE. */ @@ -597,7 +597,7 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0) if ( xcr0 == 0 ) return 0; - if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_FP) ) + if ( xcr0 <= (X86_XCR0_SSE | X86_XCR0_X87) ) return size; /* @@ -605,7 +605,7 @@ unsigned int xstate_uncompressed_size(uint64_t xcr0) * maximum offset+size. Some states (e.g. LWP, APX_F) are out-of-order * with respect their index. */ - xcr0 &= ~(X86_XCR0_SSE | X86_XCR0_FP); + xcr0 &= ~(X86_XCR0_SSE | X86_XCR0_X87); for_each_set_bit ( i, &xcr0, 63 ) { const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i]; @@ -626,14 +626,14 @@ unsigned int xstate_compressed_size(uint64_t xstates) if ( xstates == 0 ) return 0; - if ( xstates <= (X86_XCR0_SSE | X86_XCR0_FP) ) + if ( xstates <= (X86_XCR0_SSE | X86_XCR0_X87) ) return size; /* * For the compressed size, every non-legacy component matters. Some * componenets require aligning to 64 first. */ - xstates &= ~(X86_XCR0_SSE | X86_XCR0_FP); + xstates &= ~(X86_XCR0_SSE | X86_XCR0_X87); for_each_set_bit ( i, &xstates, 63 ) { const struct xstate_component *c = &raw_cpu_policy.xstate.comp[i]; @@ -756,7 +756,7 @@ static void __init noinline xstate_check_sizes(void) * layout compatibility with Intel and having a knock-on effect on all * subsequent states. */ - check_new_xstate(&s, X86_XCR0_SSE | X86_XCR0_FP); + check_new_xstate(&s, X86_XCR0_SSE | X86_XCR0_X87); if ( cpu_has_avx ) check_new_xstate(&s, X86_XCR0_YMM); @@ -1008,7 +1008,7 @@ uint64_t read_bndcfgu(void) : "=m" (*xstate) : "a" (X86_XCR0_BNDCSR), "d" (0), "D" (xstate) ); - bndcsr = (void *)xstate + xstate_offsets[X86_XCR0_BNDCSR_POS]; + bndcsr = (void *)xstate + xstate_offsets[ilog2(X86_XCR0_BNDCSR)]; } if ( cr0 & X86_CR0_TS )