From patchwork Fri May 31 09:03:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13681343 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2514E156674; Fri, 31 May 2024 09:04:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146242; cv=none; b=WY4+MmvceVFtbegas/cgarvK0BD4rRD4xPVGM1g42ZP+OaFWAm37vH96bWu3KXpyk0vl3Nep+ibLf4cV5xjEffY0iiXoW3Pwryx4h3n0OCug6UXPfkBbOvfHGvvVTbO+Gt6QNTrqaoJO6ACOVtLyRKCcrMDciLP+pa+ELFtGFc4= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146242; c=relaxed/simple; bh=6z2KeHN5Uj3V2GX6S/+U9oo8mWQvlMrTx1zKOKi30K0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=OgcxvkQLoFIP7t0djuzUa3Dpbbry664i5IOY1AC1bTSk/tHspqnpH0b1qNvBmgiY1kLU7NOc4cW0RWPQ73K13hLUB94vK0jhRY5FrsJ2yQ5dWZR/ugS1E/7Z59hLbaVa3+QsEXNW/SOIJaXhD58XjjoDwUoeDz8jOdC+guhHe+8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=US9HditZ; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="US9HditZ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717146241; x=1748682241; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=6z2KeHN5Uj3V2GX6S/+U9oo8mWQvlMrTx1zKOKi30K0=; b=US9HditZeZhB5iKvjI4/s4gTWObqyVx252pojpeA08gbBgAFtT83L+fS aFd7r7ovlfYMCuub6qqRTzsD3dFFTSTv3QjnUEt0J1/Cju2cDWTdDSD7X bCxnbQmBdVAGOshpzTwLp+P5Hq8cXn4dzBBJ01gQGfFUu+HnUP6E0CsO7 lBbn3deDfqwQSgvzFGRkD5N43wb/54s0cfuafvnAAcTy/c1CCJSz76jUe o6k10j78Uei/EFFdjLxoKuvi/l3Dks+ZW5sXzyK3oQ/sS/jvkpGVsg/oD fF7AKPK4PFbvuEi9rcbs/sV3Txx3CMtrkRA1CaMjaZayf677aDrbnyAlg w==; X-CSE-ConnectionGUID: KbfTv4neTZGOcFlVQ4CduQ== X-CSE-MsgGUID: EtNFWPIaSmyaHDHBYhRx0g== X-IronPort-AV: E=McAfee;i="6600,9927,11088"; a="17480583" X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="17480583" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:03:59 -0700 X-CSE-ConnectionGUID: w2604bKRTNOluHRULva0Pg== X-CSE-MsgGUID: 8wCcf+i/TtyE8BwGGqWXFA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="36102735" Received: from jf.jf.intel.com ([10.165.9.183]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:03:59 -0700 From: Yang Weijiang To: tglx@linutronix.de, dave.hansen@intel.com, x86@kernel.org, seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, weijiang.yang@intel.com, john.allen@amd.com Subject: [PATCH 1/6] x86/fpu/xstate: Always preserve non-user xfeatures/flags in __state_perm Date: Fri, 31 May 2024 02:03:26 -0700 Message-ID: <20240531090331.13713-2-weijiang.yang@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240531090331.13713-1-weijiang.yang@intel.com> References: <20240531090331.13713-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 From: Sean Christopherson When granting userspace or a KVM guest access to an xfeature, preserve the entity's existing supervisor and software-defined permissions as tracked by __state_perm, i.e. use __state_perm to track *all* permissions even though all supported supervisor xfeatures are granted to all FPUs and FPU_GUEST_PERM_LOCKED disallows changing permissions. Effectively clobbering supervisor permissions results in inconsistent behavior, as xstate_get_group_perm() will report supervisor features for process that do NOT request access to dynamic user xfeatures, whereas any and all supervisor features will be absent from the set of permissions for any process that is granted access to one or more dynamic xfeatures (which right now means AMX). The inconsistency isn't problematic because fpu_xstate_prctl() already strips out everything except user xfeatures: case ARCH_GET_XCOMP_PERM: /* * Lockless snapshot as it can also change right after the * dropping the lock. */ permitted = xstate_get_host_group_perm(); permitted &= XFEATURE_MASK_USER_SUPPORTED; return put_user(permitted, uptr); case ARCH_GET_XCOMP_GUEST_PERM: permitted = xstate_get_guest_group_perm(); permitted &= XFEATURE_MASK_USER_SUPPORTED; return put_user(permitted, uptr); and similarly KVM doesn't apply the __state_perm to supervisor states (kvm_get_filtered_xcr0() incorporates xstate_get_guest_group_perm()): case 0xd: { u64 permitted_xcr0 = kvm_get_filtered_xcr0(); u64 permitted_xss = kvm_caps.supported_xss; But if KVM in particular were to ever change, dropping supervisor permissions would result in subtle bugs in KVM's reporting of supported CPUID settings. And the above behavior also means that having supervisor xfeatures in __state_perm is correctly handled by all users. Dropping supervisor permissions also creates another landmine for KVM. If more dynamic user xfeatures are ever added, requesting access to multiple xfeatures in separate ARCH_REQ_XCOMP_GUEST_PERM calls will result in the second invocation of __xstate_request_perm() computing the wrong ksize, as as the mask passed to xstate_calculate_size() would not contain *any* supervisor features. Commit 781c64bfcb73 ("x86/fpu/xstate: Handle supervisor states in XSTATE permissions") fudged around the size issue for userspace FPUs, but for reasons unknown skipped guest FPUs. Lack of a fix for KVM "works" only because KVM doesn't yet support virtualizing features that have supervisor xfeatures, i.e. as of today, KVM guest FPUs will never need the relevant xfeatures. Simply extending the hack-a-fix for guests would temporarily solve the ksize issue, but wouldn't address the inconsistency issue and would leave another lurking pitfall for KVM. KVM support for virtualizing CET will likely add CET_KERNEL as a guest-only xfeature, i.e. CET_KERNEL will not be set in xfeatures_mask_supervisor() and would again be dropped when granting access to dynamic xfeatures. Note, the existing clobbering behavior is rather subtle. The @permitted parameter to __xstate_request_perm() comes from: permitted = xstate_get_group_perm(guest); which is either fpu->guest_perm.__state_perm or fpu->perm.__state_perm, where __state_perm is initialized to: fpu->perm.__state_perm = fpu_kernel_cfg.default_features; and copied to the guest side of things: /* Same defaults for guests */ fpu->guest_perm = fpu->perm; fpu_kernel_cfg.default_features contains everything except the dynamic xfeatures, i.e. everything except XFEATURE_MASK_XTILE_DATA: fpu_kernel_cfg.default_features = fpu_kernel_cfg.max_features; fpu_kernel_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; When __xstate_request_perm() restricts the local "mask" variable to compute the user state size: mask &= XFEATURE_MASK_USER_SUPPORTED; usize = xstate_calculate_size(mask, false); it subtly overwrites the target __state_perm with "mask" containing only user xfeatures: perm = guest ? &fpu->guest_perm : &fpu->perm; /* Pairs with the READ_ONCE() in xstate_get_group_perm() */ WRITE_ONCE(perm->__state_perm, mask); Cc: Maxim Levitsky Cc: Weijiang Yang Cc: Dave Hansen Cc: Paolo Bonzini Cc: Peter Zijlstra Cc: Chao Gao Cc: Rick Edgecombe Cc: John Allen Cc: kvm@vger.kernel.org Link: https://lore.kernel.org/all/ZTqgzZl-reO1m01I@google.com Signed-off-by: Sean Christopherson Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Rick Edgecombe --- arch/x86/kernel/fpu/xstate.c | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index c5a026fee5e0..bc66183c7df2 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -1603,16 +1603,20 @@ static int __xstate_request_perm(u64 permitted, u64 requested, bool guest) if ((permitted & requested) == requested) return 0; - /* Calculate the resulting kernel state size */ + /* + * Calculate the resulting kernel state size. Note, @permitted also + * contains supervisor xfeatures even though supervisor are always + * permitted for kernel and guest FPUs, and never permitted for user + * FPUs. + */ mask = permitted | requested; - /* Take supervisor states into account on the host */ - if (!guest) - mask |= xfeatures_mask_supervisor(); ksize = xstate_calculate_size(mask, compacted); - /* Calculate the resulting user state size */ - mask &= XFEATURE_MASK_USER_SUPPORTED; - usize = xstate_calculate_size(mask, false); + /* + * Calculate the resulting user state size. Take care not to clobber + * the supervisor xfeatures in the new mask! + */ + usize = xstate_calculate_size(mask & XFEATURE_MASK_USER_SUPPORTED, false); if (!guest) { ret = validate_sigaltstack(usize); From patchwork Fri May 31 09:03:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13681342 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 8716815665D; Fri, 31 May 2024 09:04:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146242; cv=none; b=YmSShSvnmciwK+7YBBx8+7p+t4HxuGCN/ZlCUWV0Jf0/Bk1fuIX0NZ1cjXvryLGVZWLj+K8cVTspErCpUiCH491UJICv09tBkTOSbhq+XBd+L6YXhGtQ2vS3E6gpuqZ0PvqpPiSfYU9A5YRRp6NVHL1LbVBOttMqAN1DMfWR2fo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146242; c=relaxed/simple; bh=HgGje5OjSiCYYJ7sGjRMBDMbj0piQLEBRn57zq5aDns=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=hhLcIQiyRm2Py42DOW0pUoh7YKwy4fw0E/UV2H3pGb1wIgU0YoWHspEIkRGNjuz2WwnJ9ks8H2wZRcU+jH0EC9JKH32ulq7BKREwNUB+25CbeHpUeXaPjODpqryVxewi037ijrCR/7QidTQmV8nXGUpeXoDj2KdTBWJLXF9kiRI= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=jtAhz4LQ; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="jtAhz4LQ" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717146240; x=1748682240; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=HgGje5OjSiCYYJ7sGjRMBDMbj0piQLEBRn57zq5aDns=; b=jtAhz4LQbTkzoAuLwKaxwfg2YRuSCjBchExZKOHhPiihjqZYMt0yaMGx BZ9iQYQrMAFF9uaSFrS3A4ZBHGsuhPE/42JvsU70+eQcV+nDk8x0QfcwF 6cKgky7T84mhEzLW735llKX+SZOpOSesGavlnW3ll1S/8B9THvf0x+SkW 6va/RIMXJICCQKO/Q1LnAzblzSXL4GedmlNW/e1Ztc2FUwSlwG4ebp4bW /TZOfciYOiYhTjRzIoMd4V5ChRh10BsE85meQ9pIOvJav7d4lO9lu2Mmy Xy0ftOtvhncIrnXILKgifH78ZOJc23c+D5ImJww26XjvNFA9BGGVbY56c g==; X-CSE-ConnectionGUID: ktvBhpZ6TPmeORvBEOgwnQ== X-CSE-MsgGUID: SGiHP/qOTXanl/XdNh9laQ== X-IronPort-AV: E=McAfee;i="6600,9927,11088"; a="17480590" X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="17480590" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:03:59 -0700 X-CSE-ConnectionGUID: KBH2UrLaSPWV/ysJh/TIcw== X-CSE-MsgGUID: VWFcxs/GTsea62xQKwey9g== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="36102742" Received: from jf.jf.intel.com ([10.165.9.183]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:03:59 -0700 From: Yang Weijiang To: tglx@linutronix.de, dave.hansen@intel.com, x86@kernel.org, seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, weijiang.yang@intel.com, john.allen@amd.com Subject: [PATCH 2/6] x86/fpu/xstate: Add CET supervisor mode state support Date: Fri, 31 May 2024 02:03:27 -0700 Message-ID: <20240531090331.13713-3-weijiang.yang@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240531090331.13713-1-weijiang.yang@intel.com> References: <20240531090331.13713-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Add supervisor mode state support within FPU xstate management framework. Although supervisor shadow stack is not enabled/used today in kernel,KVM requires the support because when KVM advertises shadow stack feature to guest, architecturally it claims the support for both user and supervisor modes for guest OSes(Linux or non-Linux). CET supervisor states not only includes PL{0,1,2}_SSP but also IA32_S_CET MSR, but the latter is not xsave-managed. In virtualization world, guest IA32_S_CET is saved/stored into/from VM control structure. With supervisor xstate support, guest supervisor mode shadow stack state can be properly saved/restored when 1) guest/host FPU context is swapped 2) vCPU thread is sched out/in. The alternative is to enable it in KVM domain, but KVM maintainers NAKed the solution. The external discussion can be found at [*], it ended up with adding the support in kernel instead of KVM domain. Note, in KVM case, guest CET supervisor state i.e., IA32_PL{0,1,2}_MSRs, are preserved after VM-Exit until host/guest fpstates are swapped, but since host supervisor shadow stack is disabled, the preserved MSRs won't hurt host. [*]: https://lore.kernel.org/all/806e26c2-8d21-9cc9-a0b7-7787dd231729@intel.com/ Signed-off-by: Yang Weijiang Reviewed-by: Rick Edgecombe Reviewed-by: Maxim Levitsky --- arch/x86/include/asm/fpu/types.h | 14 ++++++++++++-- arch/x86/include/asm/fpu/xstate.h | 6 +++--- arch/x86/kernel/fpu/xstate.c | 6 +++++- 3 files changed, 20 insertions(+), 6 deletions(-) diff --git a/arch/x86/include/asm/fpu/types.h b/arch/x86/include/asm/fpu/types.h index eb17f31b06d2..d633cf833411 100644 --- a/arch/x86/include/asm/fpu/types.h +++ b/arch/x86/include/asm/fpu/types.h @@ -118,7 +118,7 @@ enum xfeature { XFEATURE_PKRU, XFEATURE_PASID, XFEATURE_CET_USER, - XFEATURE_CET_KERNEL_UNUSED, + XFEATURE_CET_KERNEL, XFEATURE_RSRVD_COMP_13, XFEATURE_RSRVD_COMP_14, XFEATURE_LBR, @@ -141,7 +141,7 @@ enum xfeature { #define XFEATURE_MASK_PKRU (1 << XFEATURE_PKRU) #define XFEATURE_MASK_PASID (1 << XFEATURE_PASID) #define XFEATURE_MASK_CET_USER (1 << XFEATURE_CET_USER) -#define XFEATURE_MASK_CET_KERNEL (1 << XFEATURE_CET_KERNEL_UNUSED) +#define XFEATURE_MASK_CET_KERNEL (1 << XFEATURE_CET_KERNEL) #define XFEATURE_MASK_LBR (1 << XFEATURE_LBR) #define XFEATURE_MASK_XTILE_CFG (1 << XFEATURE_XTILE_CFG) #define XFEATURE_MASK_XTILE_DATA (1 << XFEATURE_XTILE_DATA) @@ -266,6 +266,16 @@ struct cet_user_state { u64 user_ssp; }; +/* + * State component 12 is Control-flow Enforcement supervisor states + */ +struct cet_supervisor_state { + /* supervisor ssp pointers */ + u64 pl0_ssp; + u64 pl1_ssp; + u64 pl2_ssp; +}; + /* * State component 15: Architectural LBR configuration state. * The size of Arch LBR state depends on the number of LBRs (lbr_depth). diff --git a/arch/x86/include/asm/fpu/xstate.h b/arch/x86/include/asm/fpu/xstate.h index d4427b88ee12..3b4a038d3c57 100644 --- a/arch/x86/include/asm/fpu/xstate.h +++ b/arch/x86/include/asm/fpu/xstate.h @@ -51,7 +51,8 @@ /* All currently supported supervisor features */ #define XFEATURE_MASK_SUPERVISOR_SUPPORTED (XFEATURE_MASK_PASID | \ - XFEATURE_MASK_CET_USER) + XFEATURE_MASK_CET_USER | \ + XFEATURE_MASK_CET_KERNEL) /* * A supervisor state component may not always contain valuable information, @@ -78,8 +79,7 @@ * Unsupported supervisor features. When a supervisor feature in this mask is * supported in the future, move it to the supported supervisor feature mask. */ -#define XFEATURE_MASK_SUPERVISOR_UNSUPPORTED (XFEATURE_MASK_PT | \ - XFEATURE_MASK_CET_KERNEL) +#define XFEATURE_MASK_SUPERVISOR_UNSUPPORTED (XFEATURE_MASK_PT) /* All supervisor states including supported and unsupported states. */ #define XFEATURE_MASK_SUPERVISOR_ALL (XFEATURE_MASK_SUPERVISOR_SUPPORTED | \ diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index bc66183c7df2..84d4fcaeff35 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -51,7 +51,7 @@ static const char *xfeature_names[] = "Protection Keys User registers", "PASID state", "Control-flow User registers", - "Control-flow Kernel registers (unused)", + "Control-flow Kernel registers", "unknown xstate feature", "unknown xstate feature", "unknown xstate feature", @@ -74,6 +74,7 @@ static unsigned short xsave_cpuid_features[] __initdata = { [XFEATURE_PKRU] = X86_FEATURE_OSPKE, [XFEATURE_PASID] = X86_FEATURE_ENQCMD, [XFEATURE_CET_USER] = X86_FEATURE_SHSTK, + [XFEATURE_CET_KERNEL] = X86_FEATURE_SHSTK, [XFEATURE_XTILE_CFG] = X86_FEATURE_AMX_TILE, [XFEATURE_XTILE_DATA] = X86_FEATURE_AMX_TILE, }; @@ -279,6 +280,7 @@ static void __init print_xstate_features(void) print_xstate_feature(XFEATURE_MASK_PKRU); print_xstate_feature(XFEATURE_MASK_PASID); print_xstate_feature(XFEATURE_MASK_CET_USER); + print_xstate_feature(XFEATURE_MASK_CET_KERNEL); print_xstate_feature(XFEATURE_MASK_XTILE_CFG); print_xstate_feature(XFEATURE_MASK_XTILE_DATA); } @@ -348,6 +350,7 @@ static __init void os_xrstor_booting(struct xregs_state *xstate) XFEATURE_MASK_BNDCSR | \ XFEATURE_MASK_PASID | \ XFEATURE_MASK_CET_USER | \ + XFEATURE_MASK_CET_KERNEL | \ XFEATURE_MASK_XTILE) /* @@ -548,6 +551,7 @@ static bool __init check_xstate_against_struct(int nr) case XFEATURE_PASID: return XCHECK_SZ(sz, nr, struct ia32_pasid_state); case XFEATURE_XTILE_CFG: return XCHECK_SZ(sz, nr, struct xtile_cfg); case XFEATURE_CET_USER: return XCHECK_SZ(sz, nr, struct cet_user_state); + case XFEATURE_CET_KERNEL: return XCHECK_SZ(sz, nr, struct cet_supervisor_state); case XFEATURE_XTILE_DATA: check_xtile_data_against_struct(sz); return true; default: XSTATE_WARN_ON(1, "No structure for xstate: %d\n", nr); From patchwork Fri May 31 09:03:28 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13681344 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 6DB7B156679; Fri, 31 May 2024 09:04:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146243; cv=none; b=YV+9QECgjKYOGDt0GQjVdzIseaKdgWet9jksqxu30Bp8kp5LGaIdRXYWcRFRHlfCB5H1LwLQ4PLuP7HSgsuFlqNQcuc0sp6rrfmcbJitJE3qjdrs9koetFtv3I0gYk7P8TmRczKBAMrNL/glZqKNbS+uV0oYOTY7XA3FLuaheEk= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146243; c=relaxed/simple; bh=4QuEaaIyM77rU5Gvud0H6S7EuBirb3AmIOw/kWt/SiE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=nmeSECa2B0DjTxZFxYbmWBXmKgvgvG4a5WYyrnaPdQ/SkkZCNiisuqMfiJ9MQoQsIhUGc9bi5oiTRoWKDTEh1P8keEyMPEk39kbcx7ct+uY1/5oe7M3YujtjkAyqrMD/h6nLR9qvqQDQivOw/nub9Fz2Zz3C+nnYbkrzgkKX4hg= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=C1wTa+xd; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="C1wTa+xd" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717146241; x=1748682241; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=4QuEaaIyM77rU5Gvud0H6S7EuBirb3AmIOw/kWt/SiE=; b=C1wTa+xd4+GNa62TlGmBslutN9HyRXPJ5g1jez2udCVo0+0WOwUnxNSI NBi4yjturbjdcfySBBJ3Xl4oDVh7oL6/5zK1vDA+K+EkISaNZcwUgq5p+ epVkoqtR606nvmU+qZibea1bNWRyz2gXw4cNhjjc1MaubcOlH6meVXq6G Z9i0Q/uWuhaibzu+dOFwI/mWtShcVFM/DhKpOlqSG+6BpxReXjFD9i73G wSMa9nfeZTAkBR+wCgguy1SqRx5IxUk6c9kTRxb5TYkHs7QnxEQHI7APW Yh+6/In9T25zZQWtz614uEFPudZXz3gzhxegUPAvm1tgR3E/F49L8d/+/ g==; X-CSE-ConnectionGUID: tBMG5Go5Qs24wecizm3YAg== X-CSE-MsgGUID: vVSQ62fDTnCtfVruQYNmGA== X-IronPort-AV: E=McAfee;i="6600,9927,11088"; a="17480597" X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="17480597" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:03:59 -0700 X-CSE-ConnectionGUID: ZpHNPchjRxe4DhDu7pcT2g== X-CSE-MsgGUID: XROkOIGgSrGONW3qMFycEQ== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="36102746" Received: from jf.jf.intel.com ([10.165.9.183]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:03:59 -0700 From: Yang Weijiang To: tglx@linutronix.de, dave.hansen@intel.com, x86@kernel.org, seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, weijiang.yang@intel.com, john.allen@amd.com Subject: [PATCH 3/6] x86/fpu/xstate: Introduce XFEATURE_MASK_KERNEL_DYNAMIC xfeature set Date: Fri, 31 May 2024 02:03:28 -0700 Message-ID: <20240531090331.13713-4-weijiang.yang@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240531090331.13713-1-weijiang.yang@intel.com> References: <20240531090331.13713-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Define a new XFEATURE_MASK_KERNEL_DYNAMIC mask to specify the features that can be optionally enabled by kernel components. This is similar to XFEATURE_MASK_USER_DYNAMIC in that it contains optional xfeatures that can allows the FPU buffer to be dynamically sized. The difference is that the KERNEL variant contains supervisor features and will be enabled by kernel components that need them, and not directly by the user. Currently it's used by KVM to configure guest dedicated fpstate for calculating the xfeature and fpstate storage size etc. The kernel dynamic xfeatures now only contain XFEATURE_CET_KERNEL, which is supported by host as they're enabled in kernel XSS MSR setting but relevant CPU feature, i.e., supervisor shadow stack, is not enabled in host kernel therefore it can be omitted for normal fpstate by default. Remove the kernel dynamic feature from fpu_kernel_cfg.default_features so that the bits in xstate_bv and xcomp_bv are cleared and xsaves/xrstors can be optimized by HW for normal fpstate. Suggested-by: Dave Hansen Signed-off-by: Yang Weijiang Reviewed-by: Rick Edgecombe --- arch/x86/include/asm/fpu/xstate.h | 5 ++++- arch/x86/kernel/fpu/xstate.c | 1 + 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/arch/x86/include/asm/fpu/xstate.h b/arch/x86/include/asm/fpu/xstate.h index 3b4a038d3c57..a212d3851429 100644 --- a/arch/x86/include/asm/fpu/xstate.h +++ b/arch/x86/include/asm/fpu/xstate.h @@ -46,9 +46,12 @@ #define XFEATURE_MASK_USER_RESTORE \ (XFEATURE_MASK_USER_SUPPORTED & ~XFEATURE_MASK_PKRU) -/* Features which are dynamically enabled for a process on request */ +/* Features which are dynamically enabled per userspace request */ #define XFEATURE_MASK_USER_DYNAMIC XFEATURE_MASK_XTILE_DATA +/* Features which are dynamically enabled per kernel side request */ +#define XFEATURE_MASK_KERNEL_DYNAMIC XFEATURE_MASK_CET_KERNEL + /* All currently supported supervisor features */ #define XFEATURE_MASK_SUPERVISOR_SUPPORTED (XFEATURE_MASK_PASID | \ XFEATURE_MASK_CET_USER | \ diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 84d4fcaeff35..1a8a187351d3 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -818,6 +818,7 @@ void __init fpu__init_system_xstate(unsigned int legacy_size) /* Clean out dynamic features from default */ fpu_kernel_cfg.default_features = fpu_kernel_cfg.max_features; fpu_kernel_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; + fpu_kernel_cfg.default_features &= ~XFEATURE_MASK_KERNEL_DYNAMIC; fpu_user_cfg.default_features = fpu_user_cfg.max_features; fpu_user_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; From patchwork Fri May 31 09:03:29 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13681345 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BC1A0156228; Fri, 31 May 2024 09:04:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146244; cv=none; b=Zo8cxXjOZg5YnjKRQEM+Q4OOuCvj7Ugl1FZIoDgd24dy7/IvjT/WoEwVhNKxqj3nHcl73QVzWMbeVUWslWMfDXUnXQBrYpP+wyhlzPJFYjSsx3FAtwJSyr1PAn/5FrCssxkXsp3XRt4VAalqaeMLRQ+ejSdjCF+Rhthjbb4owDw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146244; c=relaxed/simple; bh=OHVq25vZLsqyLTtyp75ZE0G7ZNYjK4i/SyW0ERPfjvA=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=S5vlRZc0eWOVluLxfkh2P/0i4bOCDvy8E85a4t4dL77/4JXgg35sSsxFJWaJ5Fo4ehIxR2sSyT3F73N3MaI8NtBMoQVEf3lFR4hbv/887drVt9DkXCsTYGg0Nd0XPuc4goc07XMwfrCPoE4CPwDRRrW5al8MDZJ32tdfEJoqSho= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=JNQsHtvS; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="JNQsHtvS" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717146242; x=1748682242; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=OHVq25vZLsqyLTtyp75ZE0G7ZNYjK4i/SyW0ERPfjvA=; b=JNQsHtvSK3n8FkSccktvt3IICxcZLmkrQqXNqGiptSj+QF5p3hlQFgd3 NAOsyMymi6EN4pDuKaqJo4W26eP0GFBHzJPb6yD/vNN7V80v+5Vycmm6P oXiiPeI7ysP6YgqRJTu7CDQfreoXqcS0lg2n01hd6958dDmxCinEcAHdq GkHTZkavB46SaB+GnIsW+xqLhg1/4W8MrRANbO9macI9hzXI2bd0pK5nW Njlt1fveBe1CGa7AW/3f0wkPFMgyyaNUFZnOEBgSOZ+h4gJx6/IX7fDkn sz3R3sD5O9Z5dvTpSg8t7pfCpM8reDCy70WJw3aME4OUkquARsV3d4nLs Q==; X-CSE-ConnectionGUID: YFiEU1KkQme+JfK8rNcI0Q== X-CSE-MsgGUID: lELhkIdgQeKppo4SWVy8OQ== X-IronPort-AV: E=McAfee;i="6600,9927,11088"; a="17480605" X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="17480605" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:03:59 -0700 X-CSE-ConnectionGUID: FkvDbV+FTbOWVqeGkMwdiQ== X-CSE-MsgGUID: 0qNeSv2bSImIGUYhFKgxFg== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="36102750" Received: from jf.jf.intel.com ([10.165.9.183]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:04:00 -0700 From: Yang Weijiang To: tglx@linutronix.de, dave.hansen@intel.com, x86@kernel.org, seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, weijiang.yang@intel.com, john.allen@amd.com Subject: [PATCH 4/6] x86/fpu/xstate: Introduce fpu_guest_cfg for guest FPU configuration Date: Fri, 31 May 2024 02:03:29 -0700 Message-ID: <20240531090331.13713-5-weijiang.yang@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240531090331.13713-1-weijiang.yang@intel.com> References: <20240531090331.13713-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Define new fpu_guest_cfg to hold all guest FPU settings so that it can differ from generic kernel FPU settings, e.g., enabling CET supervisor xstate by default for guest fpstate while it's remained disabled in kernel FPU config. The kernel dynamic xfeatures are specifically used by guest fpstate now, add the mask for guest fpstate so that guest_perm.__state_perm == (fpu_kernel_cfg.default_xfeature | XFEATURE_MASK_KERNEL_DYNAMIC). And if guest fpstate is re-allocated to hold user dynamic xfeatures, the resulting permissions are consumed before calculate new guest fpstate. With new guest FPU config added, there're 3 categories of FPU configs in kernel, the usages and key fields are recapped as below. kernel FPU config: @fpu_kernel_cfg.max_features - all known and CPU supported user and supervisor features except independent kernel features @fpu_kernel_cfg.default_features - all known and CPU supported user and supervisor features except dynamic kernel features, independent kernel features and dynamic userspace features. @fpu_kernel_cfg.max_size - size of compacted buffer with 'fpu_kernel_cfg.max_features' @fpu_kernel_cfg.default_size - size of compacted buffer with 'fpu_kernel_cfg.default_features' user FPU config: @fpu_user_cfg.max_features - all known and CPU supported user features @fpu_user_cfg.default_features - all known and CPU supported user features except dynamic userspace features. @fpu_user_cfg.max_size - size of non-compacted buffer with 'fpu_user_cfg.max_features' @fpu_user_cfg.default_size - size of non-compacted buffer with 'fpu_user_cfg.default_features' guest FPU config: @fpu_guest_cfg.max_features - all known and CPU supported user and supervisor features except independent kernel features. @fpu_guest_cfg.default_features - all known and CPU supported user and supervisor features except independent kernel features and dynamic userspace features. @fpu_guest_cfg.max_size - size of compacted buffer with 'fpu_guest_cfg.max_features' @fpu_guest_cfg.default_size - size of compacted buffer with 'fpu_guest_cfg.default_features' Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Rick Edgecombe --- arch/x86/include/asm/fpu/types.h | 2 +- arch/x86/kernel/fpu/core.c | 14 +++++++++++--- arch/x86/kernel/fpu/xstate.c | 10 ++++++++++ 3 files changed, 22 insertions(+), 4 deletions(-) diff --git a/arch/x86/include/asm/fpu/types.h b/arch/x86/include/asm/fpu/types.h index d633cf833411..0ad17b0ffbe8 100644 --- a/arch/x86/include/asm/fpu/types.h +++ b/arch/x86/include/asm/fpu/types.h @@ -604,6 +604,6 @@ struct fpu_state_config { }; /* FPU state configuration information */ -extern struct fpu_state_config fpu_kernel_cfg, fpu_user_cfg; +extern struct fpu_state_config fpu_kernel_cfg, fpu_user_cfg, fpu_guest_cfg; #endif /* _ASM_X86_FPU_TYPES_H */ diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 1209c7aebb21..9e2e5c46cf28 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -33,9 +33,10 @@ DEFINE_STATIC_KEY_FALSE(__fpu_state_size_dynamic); DEFINE_PER_CPU(u64, xfd_state); #endif -/* The FPU state configuration data for kernel and user space */ +/* The FPU state configuration data for kernel, user space and guest. */ struct fpu_state_config fpu_kernel_cfg __ro_after_init; struct fpu_state_config fpu_user_cfg __ro_after_init; +struct fpu_state_config fpu_guest_cfg __ro_after_init; /* * Represents the initial FPU state. It's mostly (but not completely) zeroes, @@ -536,8 +537,15 @@ void fpstate_reset(struct fpu *fpu) fpu->perm.__state_perm = fpu_kernel_cfg.default_features; fpu->perm.__state_size = fpu_kernel_cfg.default_size; fpu->perm.__user_state_size = fpu_user_cfg.default_size; - /* Same defaults for guests */ - fpu->guest_perm = fpu->perm; + + /* Guest permission settings */ + fpu->guest_perm.__state_perm = fpu_guest_cfg.default_features; + fpu->guest_perm.__state_size = fpu_guest_cfg.default_size; + /* + * Set guest's __user_state_size to fpu_user_cfg.default_size so that + * existing uAPIs can still work. + */ + fpu->guest_perm.__user_state_size = fpu_user_cfg.default_size; } static inline void fpu_inherit_perms(struct fpu *dst_fpu) diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c index 1a8a187351d3..392f3ba3aa27 100644 --- a/arch/x86/kernel/fpu/xstate.c +++ b/arch/x86/kernel/fpu/xstate.c @@ -683,6 +683,7 @@ static int __init init_xstate_size(void) { /* Recompute the context size for enabled features: */ unsigned int user_size, kernel_size, kernel_default_size; + unsigned int guest_default_size; bool compacted = cpu_feature_enabled(X86_FEATURE_XCOMPACTED); /* Uncompacted user space size */ @@ -704,13 +705,18 @@ static int __init init_xstate_size(void) kernel_default_size = xstate_calculate_size(fpu_kernel_cfg.default_features, compacted); + guest_default_size = + xstate_calculate_size(fpu_guest_cfg.default_features, compacted); + if (!paranoid_xstate_size_valid(kernel_size)) return -EINVAL; fpu_kernel_cfg.max_size = kernel_size; fpu_user_cfg.max_size = user_size; + fpu_guest_cfg.max_size = kernel_size; fpu_kernel_cfg.default_size = kernel_default_size; + fpu_guest_cfg.default_size = guest_default_size; fpu_user_cfg.default_size = xstate_calculate_size(fpu_user_cfg.default_features, false); @@ -823,6 +829,10 @@ void __init fpu__init_system_xstate(unsigned int legacy_size) fpu_user_cfg.default_features = fpu_user_cfg.max_features; fpu_user_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; + fpu_guest_cfg.max_features = fpu_kernel_cfg.max_features; + fpu_guest_cfg.default_features = fpu_guest_cfg.max_features; + fpu_guest_cfg.default_features &= ~XFEATURE_MASK_USER_DYNAMIC; + /* Store it for paranoia check at the end */ xfeatures = fpu_kernel_cfg.max_features; From patchwork Fri May 31 09:03:30 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13681346 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id DD5AA15689B; Fri, 31 May 2024 09:04:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146244; cv=none; b=BosxhM7c3VeteZYaM2g6aMn1OEnpV68MnGVFJpjq7pSeSvIV+OQ+kmMRraidwuJsldtp3G2ZjxmpgxwItJwvu+nT9IF6sBeYPIp1WupdrcEYLIRjEnNk0skvdnE1pYTJrTyILAEsdi+Srb7AVs9b9Mre9f5WdBl00FkmgE38wfo= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146244; c=relaxed/simple; bh=XHJjqKGhN585X7aLvVu2CbdGCurM4gt6+uoq5Ic29mQ=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=I4NQQg+4XzXwD8y85KxsRkLFU7/HXOiSf5Goffg3EDgClIjbzym5ZWky87YB8wfMXws2jGv5rXLnteVNZNV3ECwzkgGBEslpOxT4JGfrN3pk/67ouBoF63GKLX8Jr5fKJYKD5lojhBA2xXL2cV3/KNEGHkzDizC+2SqivHwJHh0= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=EL7e6Pya; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="EL7e6Pya" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717146243; x=1748682243; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=XHJjqKGhN585X7aLvVu2CbdGCurM4gt6+uoq5Ic29mQ=; b=EL7e6PyayPUmhQ0x1F9zgxKiCFWxCfRA7OvVTI+FtO7fZEvxVv7zpoga TCK46WYVuOrteoZD3YJXOzAN5IzLqZGWJoQRxBKr/PYWWuPqrIQ+ugtQ1 dL7VcKXSNm3/bh8oE7LkrGzNX1AfqRA8c4IxP+55l4gh+TBDyxIIUzQpg RN5Og18ATgwnuhV0Lb57rLHx9nF/Pm/ASGKRCXecYSNVYQp55wkFmwrph bXYyvJLT590tv4a3SCssQyF3WcFgVKmeLp0q1mI4FQ7ca+V7jcILxOBxZ ArA2zq+hhIMubCWPp0hpXig2USswRFv1A712NsLk0iSKLkXZCtLT2mtXz g==; X-CSE-ConnectionGUID: KD1FglHsTEOdTYBfD/ROPQ== X-CSE-MsgGUID: KX72JwJ9QVmo0GUF5NbTFQ== X-IronPort-AV: E=McAfee;i="6600,9927,11088"; a="17480614" X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="17480614" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:04:00 -0700 X-CSE-ConnectionGUID: pHRfIuaaTOisDoUHZY+0XA== X-CSE-MsgGUID: kUyLSYosQE+CVKNchuXdgw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="36102754" Received: from jf.jf.intel.com ([10.165.9.183]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:04:00 -0700 From: Yang Weijiang To: tglx@linutronix.de, dave.hansen@intel.com, x86@kernel.org, seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, weijiang.yang@intel.com, john.allen@amd.com Subject: [PATCH 5/6] x86/fpu/xstate: Create guest fpstate with guest specific config Date: Fri, 31 May 2024 02:03:30 -0700 Message-ID: <20240531090331.13713-6-weijiang.yang@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240531090331.13713-1-weijiang.yang@intel.com> References: <20240531090331.13713-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Use fpu_guest_cfg to calculate guest fpstate settings, open code for __fpstate_reset() to avoid using kernel FPU config. Below configuration steps are currently enforced to get guest fpstate: 1) Kernel sets up guest FPU settings in fpu__init_system_xstate(). 2) User space sets vCPU thread group xstate permits via arch_prctl(). 3) User space creates guest fpstate via __fpu_alloc_init_guest_fpstate() for vcpu thread. 4) User space enables guest dynamic xfeatures and re-allocate guest fpstate. By adding kernel dynamic xfeatures in above #1 and #2, guest xstate area size is expanded to hold (fpu_kernel_cfg.default_features | kernel dynamic xfeatures | user dynamic xfeatures), then host xsaves/xrstors can operate for all guest xfeatures. The user_* fields remain unchanged for compatibility with KVM uAPIs. Signed-off-by: Yang Weijiang Reviewed-by: Maxim Levitsky Reviewed-by: Rick Edgecombe --- arch/x86/kernel/fpu/core.c | 39 +++++++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c index 9e2e5c46cf28..00d7dcf45b34 100644 --- a/arch/x86/kernel/fpu/core.c +++ b/arch/x86/kernel/fpu/core.c @@ -194,8 +194,6 @@ void fpu_reset_from_exception_fixup(void) } #if IS_ENABLED(CONFIG_KVM) -static void __fpstate_reset(struct fpstate *fpstate, u64 xfd); - static void fpu_init_guest_permissions(struct fpu_guest *gfpu) { struct fpu_state_perm *fpuperm; @@ -216,25 +214,48 @@ static void fpu_init_guest_permissions(struct fpu_guest *gfpu) gfpu->perm = perm & ~FPU_GUEST_PERM_LOCKED; } -bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) +static struct fpstate *__fpu_alloc_init_guest_fpstate(struct fpu_guest *gfpu) { struct fpstate *fpstate; unsigned int size; - size = fpu_user_cfg.default_size + ALIGN(offsetof(struct fpstate, regs), 64); + /* + * fpu_guest_cfg.default_size is initialized to hold all enabled + * xfeatures except the user dynamic xfeatures. If the user dynamic + * xfeatures are enabled, the guest fpstate will be re-allocated to + * hold all guest enabled xfeatures, so omit user dynamic xfeatures + * here. + */ + size = fpu_guest_cfg.default_size + ALIGN(offsetof(struct fpstate, regs), 64); + fpstate = vzalloc(size); if (!fpstate) - return false; + return NULL; + /* + * Initialize sizes and feature masks, use fpu_user_cfg.* + * for user_* settings for compatibility of exiting uAPIs. + */ + fpstate->size = fpu_guest_cfg.default_size; + fpstate->xfeatures = fpu_guest_cfg.default_features; + fpstate->user_size = fpu_user_cfg.default_size; + fpstate->user_xfeatures = fpu_user_cfg.default_features; + fpstate->xfd = 0; - /* Leave xfd to 0 (the reset value defined by spec) */ - __fpstate_reset(fpstate, 0); fpstate_init_user(fpstate); fpstate->is_valloc = true; fpstate->is_guest = true; gfpu->fpstate = fpstate; - gfpu->xfeatures = fpu_user_cfg.default_features; - gfpu->perm = fpu_user_cfg.default_features; + gfpu->xfeatures = fpu_guest_cfg.default_features; + gfpu->perm = fpu_guest_cfg.default_features; + + return fpstate; +} + +bool fpu_alloc_guest_fpstate(struct fpu_guest *gfpu) +{ + if (!__fpu_alloc_init_guest_fpstate(gfpu)) + return false; /* * KVM sets the FP+SSE bits in the XSAVE header when copying FPU state From patchwork Fri May 31 09:03:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yang, Weijiang" X-Patchwork-Id: 13681347 Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.15]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 16D0815697A; Fri, 31 May 2024 09:04:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=198.175.65.15 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146244; cv=none; b=QW4eRSriMd8nQNO2x1ATo707Y9Dqr4tVt2IhWVbDbGAVuiodyYaMxzLmBx8PWLRPVF8Izpj/C7/cbRliWOkr0ClBOV1FFbMXVRannka7M7wKfGVfQA38YKsOKU2jb8FAUfBvuT5XhnOkOnTzHfPG2bm1EFpO2T5QGGLgsXDz+Is= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1717146244; c=relaxed/simple; bh=RNQ631wq+tFcd6BAOu9FMkT9bNMeAzeuDXhiZg+LHe0=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=r6LtVTyjMmHBxOGkZ1uikJT0G5jih+rwv0kigJz08nF1ZFwgJd199OXTOaYTBFvAiS8co7SboOttjwTrwl77EpvhP9+gcexuJRSvWA3QNRtbRgfeZRA329G8+t/c1Q1fSFiS7zgEv1RTafSSUjBH0iG8Dt5ieVdhGhzQ0MASpwk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=heGfTpQ+; arc=none smtp.client-ip=198.175.65.15 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="heGfTpQ+" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1717146243; x=1748682243; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=RNQ631wq+tFcd6BAOu9FMkT9bNMeAzeuDXhiZg+LHe0=; b=heGfTpQ+INrfmthY0WwIuv+0Ec/SA24VMvQS5RFt2bY5Eb3dg29lFLig UTT8Nop9odIWtr3iK8KAwehmMHg9RcL3BvuOLPGADTipQLCrVp+0S6DkJ RTW4fbWH5qJeTXRsz3GkLxBbbFb+jckLstcOosFasPGhNZYAr/jYvxvne 4rHderARvcB5DRpq0jXscIqRZlhLO+0QqQdjAK5nRcwvws790f/H6/8bP hrJK9mtP5yw/N8KtGKsJiYyxAFzQI8QB6ih6huXMlBn2QyKuZDL3JUXI9 Zia1nivq2fK8RcidfEQvvk0R2AFj4cDSlYa5iOCUaXeYBl34RuSQxrzzO A==; X-CSE-ConnectionGUID: dPq+XxmoT4a15+Dow+Mb/A== X-CSE-MsgGUID: BkbUaT63ToOaiwPO/KcWHA== X-IronPort-AV: E=McAfee;i="6600,9927,11088"; a="17480622" X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="17480622" Received: from fmviesa008.fm.intel.com ([10.60.135.148]) by orvoesa107.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:04:00 -0700 X-CSE-ConnectionGUID: ZCghcb8kQuWOI6toIDkwiA== X-CSE-MsgGUID: Bm2es+QJR1acdezFCvu0Qw== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.08,203,1712646000"; d="scan'208";a="36102759" Received: from jf.jf.intel.com ([10.165.9.183]) by fmviesa008-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 May 2024 02:04:00 -0700 From: Yang Weijiang To: tglx@linutronix.de, dave.hansen@intel.com, x86@kernel.org, seanjc@google.com, pbonzini@redhat.com, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: peterz@infradead.org, chao.gao@intel.com, rick.p.edgecombe@intel.com, mlevitsk@redhat.com, weijiang.yang@intel.com, john.allen@amd.com Subject: [PATCH 6/6] x86/fpu/xstate: Warn if CET supervisor state is detected in normal fpstate Date: Fri, 31 May 2024 02:03:31 -0700 Message-ID: <20240531090331.13713-7-weijiang.yang@intel.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240531090331.13713-1-weijiang.yang@intel.com> References: <20240531090331.13713-1-weijiang.yang@intel.com> Precedence: bulk X-Mailing-List: kvm@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 CET supervisor state bit is __ONLY__ enabled for guest fpstate, i.e., never for normal kernel fpstate. The bit is set when guest FPU config is initialized. For normal fpstate, the bit should have been removed when initializes kernel FPU config settings, WARN_ONCE() if kernel detects normal fpstate xfeatures contains CET supervisor state bit before xsaves operation. Signed-off-by: Yang Weijiang --- arch/x86/kernel/fpu/xstate.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/x86/kernel/fpu/xstate.h b/arch/x86/kernel/fpu/xstate.h index 05df04f39628..b1b3e0fe02c6 100644 --- a/arch/x86/kernel/fpu/xstate.h +++ b/arch/x86/kernel/fpu/xstate.h @@ -189,6 +189,8 @@ static inline void os_xsave(struct fpstate *fpstate) WARN_ON_FPU(!alternatives_patched); xfd_validate_state(fpstate, mask, false); + WARN_ON_FPU(!fpstate->is_guest && (mask & XFEATURE_MASK_CET_KERNEL)); + XSTATE_XSAVE(&fpstate->regs.xsave, lmask, hmask, err); /* We should never fault when copying to a kernel buffer: */