From patchwork Tue Jul 30 13:29:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13747403 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C4D28C3DA49 for ; Tue, 30 Jul 2024 13:32:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=pCRgFiJo/O0okelwHTYLQZDYwnG4zJJ+jptWAIF7yCE=; b=sIx+zbYzPbLZjbiVjKfelVqNdO 2t8R4RfAUBk/IQxG8vef2JRKKvLXSmfjThwObaLpq1yM9NQ7VF6ELX5aVy4JseqYxk8Dew/85F4Ot QvxQ8jSD6vnkjCV3OOc8US6gscT4vdaTQy0i11OSTXezXyw+5oUr1bS+kW20LZfGhWXZpJDRVM0IT GlWDkh/dnEUaUuIGqo/2h/OpnH31m9Lb0bQGYISs+5bKZzv6d4BFNnjXuxGwIsU3cnR3GAspJDeP+ 8QDHpKG87t6rIo7okTNFBWV9XjRiH/khzg0H/rgsT5mpenKzvELs16wwxsuJDgWUiaxFmHBNoY/bB 6Miek8vA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sYmxJ-0000000FHXP-3G4c; Tue, 30 Jul 2024 13:31:57 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sYmwR-0000000FHEo-1rpq for linux-arm-kernel@lists.infradead.org; Tue, 30 Jul 2024 13:31:05 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id D630A61EF5; Tue, 30 Jul 2024 13:31:02 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8F795C4AF0A; Tue, 30 Jul 2024 13:31:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722346262; bh=NbB0wtTd9/6wN6rW9W6ApESo3bbFx8yYxHqilIZ4buw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=rcXyM5EruvY1lEGPpUyu1Ns7OH/+cYB/9zc2CqD92dK+skajJAm/OS5Lez9Gj/QJm fauN3jDgR1LyjBZ7ZQQPb7jUvGdGmPaWMmg6DFrGuhc0DnyGbAPqEfWj/Ndc0a+CEl ZHxgiYW5rrPzoXXD2mo5c4EA1lNwekDd+/iM3lMduWZUHRSOI5fxgZITcnKl9a5tDQ LpMd8CAGnP8eoFFANHoGd+NHLZGPe2rP9yN/mAB4wuO0qa6OgOiDzImlCjoBOKVn/a 1K74TE5ahUIXlqtMXmWGx4osfqq5yiAxJDOtqr0efW1npyPr+X/HQqnukVfRvy15Ai wusQjN+cPKMbA== From: Mark Brown Date: Tue, 30 Jul 2024 14:29:13 +0100 Subject: [PATCH v6 1/4] arm64/fpsimd: Introduce __bit_to_vl() helper MIME-Version: 1.0 Message-Id: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-1-cae8a2e0bd66@kernel.org> References: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-0-cae8a2e0bd66@kernel.org> In-Reply-To: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-0-cae8a2e0bd66@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=3064; i=broonie@kernel.org; h=from:subject:message-id; bh=NbB0wtTd9/6wN6rW9W6ApESo3bbFx8yYxHqilIZ4buw=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmqOsOI5kTkWURoz5mMs9dngUzBcq7zZEGfZxLhdSL 3pASJIiJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZqjrDgAKCRAk1otyXVSH0EZxB/ wPz2QT0RwfeWLLwSrqIuqwD0KCtOK2+QxCkhjtuSSTwgYHH9N6b4PZeJ/JFQYQOCW8/7O8oUebrIe5 jZCoO/aKjMDwsyfXTC7WqNBOT8ZoegqH9drOJUB22FHzGbTBgwGtXsWFHy3JM/4I7trcG0k3PprB+B dym8i7nSYB8zMWEvVR9Jz8fsfABsqSjLmz/ktXSHcIHeZZ9cKSBr5E7HMxPLn2QLxswPoBgk4/1+ru DjM6YnrCjw57o3bYXyWOvkj9wrknByigsv1Vuyf47QHRe/SosdDLn1otiSoAz68Svn30VAbsxCvvIE ZjPQCTyjIFXe+PCChpl132oXYPV39B X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240730_063103_623487_F2022FD6 X-CRM114-Status: GOOD ( 13.68 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In all cases where we use the existing __bit_to_vq() helper we immediately convert the result into a VL. Provide and use __bit_to_vl() doing this directly. Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 4 ++++ arch/arm64/kernel/fpsimd.c | 12 ++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index bc69ac368d73..51c21265b4fa 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -172,6 +172,10 @@ static inline unsigned int __bit_to_vq(unsigned int bit) return SVE_VQ_MAX - bit; } +static inline unsigned int __bit_to_vl(unsigned int bit) +{ + return sve_vl_from_vq(__bit_to_vq(bit)); +} struct vl_info { enum vec_type type; diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 77006df20a75..8a080dbd8988 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -530,7 +530,7 @@ static unsigned int find_supported_vector_length(enum vec_type type, bit = find_next_bit(info->vq_map, SVE_VQ_MAX, __vq_to_bit(sve_vq_from_vl(vl))); - return sve_vl_from_vq(__bit_to_vq(bit)); + return __bit_to_vl(bit); } #if defined(CONFIG_ARM64_SVE) && defined(CONFIG_SYSCTL) @@ -1103,7 +1103,7 @@ int vec_verify_vq_map(enum vec_type type) * Mismatches above sve_max_virtualisable_vl are fine, since * no guest is allowed to configure ZCR_EL2.LEN to exceed this: */ - if (sve_vl_from_vq(__bit_to_vq(b)) <= info->max_virtualisable_vl) { + if (__bit_to_vl(b) <= info->max_virtualisable_vl) { pr_warn("%s: cpu%d: Unsupported vector length(s) present\n", info->name, smp_processor_id()); return -EINVAL; @@ -1169,7 +1169,7 @@ void __init sve_setup(void) set_bit(__vq_to_bit(SVE_VQ_MIN), info->vq_map); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); /* * For the default VL, pick the maximum supported value <= 64. @@ -1188,7 +1188,7 @@ void __init sve_setup(void) /* No virtualisable VLs? This is architecturally forbidden. */ info->max_virtualisable_vl = SVE_VQ_MIN; else /* b + 1 < SVE_VQ_MAX */ - info->max_virtualisable_vl = sve_vl_from_vq(__bit_to_vq(b + 1)); + info->max_virtualisable_vl = __bit_to_vl(b + 1); if (info->max_virtualisable_vl > info->max_vl) info->max_virtualisable_vl = info->max_vl; @@ -1305,10 +1305,10 @@ void __init sme_setup(void) WARN_ON(bitmap_empty(info->vq_map, SVE_VQ_MAX)); min_bit = find_last_bit(info->vq_map, SVE_VQ_MAX); - info->min_vl = sve_vl_from_vq(__bit_to_vq(min_bit)); + info->min_vl = __bit_to_vl(min_bit); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); WARN_ON(info->min_vl > info->max_vl); From patchwork Tue Jul 30 13:29:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13747404 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D9C1DC3DA70 for ; Tue, 30 Jul 2024 13:32:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=zsSDAUS2YoeQAUt/tVUpr3FQBKWXuk/WGCBQ6XC6P8Q=; b=jK8+YByzjjWtiRUWAIcAymsM00 o0v6RIiO/ZofkIESe40HVgHeKCmZYTSR1bpwx54lKxqBICxX3XWrBZbvtCAXEcIDnkGnA+ZfkvPNl e6o/UCRJby1WO1Yz6ozScg45eSxfJ2qCpWDQUmfd6+x+OCOMezMDpiIsSh2zAggVhqQcWgjJ1iAik rtmG8kr87l8rlr5TuTQ7AUr68G8+zee5NFDSEFWNu9co+HW7eTGF0LA8hyNmHOwOISHaEyKUwwojV 3p7/g9kFJBT1V98RBfi4GlPrtikZ0nvf8X9kLZfWrb0jIRUIPHr1oVBw8m6g5D7H/SREluXdV2+31 ZfkPmtgw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sYmxj-0000000FHcT-16cZ; Tue, 30 Jul 2024 13:32:23 +0000 Received: from sin.source.kernel.org ([2604:1380:40e1:4800::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sYmwV-0000000FHFo-3DMm for linux-arm-kernel@lists.infradead.org; Tue, 30 Jul 2024 13:31:09 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 07AC2CE104A; Tue, 30 Jul 2024 13:31:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1F6F3C32782; Tue, 30 Jul 2024 13:31:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722346265; bh=gxPQeR4JEm4ED3xLchT5zAbPzANcTdMNMhIHx/82Uz0=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=PZIMfhpVnD5TXNCVpu81Vu9Zf20macrzd8iD0N/ztCOh8mQ+wH1a8HssbwzdfSW/H zXyM/0eCYq5PxhKaI/RqtSsMbYJHRc3xS1Pl5Jd0vtQAV/x1maRyTPUH+sU3mV3AVx 0D904Wsh1SKqHNfQ4/Vlb0M+Loou/3dPMRga6sPSR1uDOPD/Hos6oySclyXi43U6Tg oXocISUvUERZyQmHAxJH1HieMH/Ze4393mMF0NTjGKtE4KFwIc9uOaY5FBAtj5Wt8W CboShfMbh01gBE12JGXABXZ3TzjW6QWS5eR3VJJdD8P8kRaIW6Z9KB4QHV1/oN9ky7 c+swJMORJRVsw== From: Mark Brown Date: Tue, 30 Jul 2024 14:29:14 +0100 Subject: [PATCH v6 2/4] arm64/fpsimd: Discover maximum vector length implemented by any CPU MIME-Version: 1.0 Message-Id: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-2-cae8a2e0bd66@kernel.org> References: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-0-cae8a2e0bd66@kernel.org> In-Reply-To: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-0-cae8a2e0bd66@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=4205; i=broonie@kernel.org; h=from:subject:message-id; bh=gxPQeR4JEm4ED3xLchT5zAbPzANcTdMNMhIHx/82Uz0=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmqOsPLRabMhVM/5P25hmNsYB5kqWoe6yX2/VVbRsu OVa/KZKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZqjrDwAKCRAk1otyXVSH0FnEB/ 9XXxxphdLHXNVf9+3cixke8oXrd98bWG26W7HuY/f3O0KoiTl2TgieB+5f1E3J1p2BnguzoIqmVMbP Z9DCo+SKme/Jv9NrTEKS51ONvzghY31bZYC+VnCjiv+ndJqsSYxwIE29aM8jOvVdqNnoXxGUrJY2qt 25yfFgX5JIt95X1IM/Ysm6LeBT5QLgA6zP6DI55rjrpk8jVD6ptzb8bizvCw58oxG7dq3UQvX5WYF2 +OQMn8DhnvAVRgeqmNEBQiTv5PBD5sGwORFjmBEwTjHkWGfYxZKNhwNFFgoLcwy4DzC9SJBX6wKRTP gQ+ErMwYIGhXoE4FXxhhGamgXTAHdN X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240730_063108_193042_11A57C40 X-CRM114-Status: GOOD ( 17.31 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When discovering the vector lengths for SVE and SME we do not currently record the maximum VL supported on any individual CPU. This is expected to be the same for all CPUs but the architecture allows asymmetry, if we do encounter an asymmetric system then some CPUs may support VLs higher than the maximum Linux will use. Since the pKVM hypervisor needs to support saving and restoring anything the host can physically set it needs to know the maximum value any CPU could have, add support for enumerating it and validation for late CPUs. Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 13 +++++++++++++ arch/arm64/kernel/fpsimd.c | 26 +++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 51c21265b4fa..cd19713c9deb 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -188,6 +188,9 @@ struct vl_info { int max_vl; int max_virtualisable_vl; + /* Maximum vector length observed on any CPU */ + int max_cpu_vl; + /* * Set of available vector lengths, * where length vq encoded as bit __vq_to_bit(vq): @@ -278,6 +281,11 @@ static inline int vec_max_virtualisable_vl(enum vec_type type) return vl_info[type].max_virtualisable_vl; } +static inline int vec_max_cpu_vl(enum vec_type type) +{ + return vl_info[type].max_cpu_vl; +} + static inline int sve_max_vl(void) { return vec_max_vl(ARM64_VEC_SVE); @@ -288,6 +296,11 @@ static inline int sve_max_virtualisable_vl(void) return vec_max_virtualisable_vl(ARM64_VEC_SVE); } +static inline int sve_max_cpu_vl(void) +{ + return vec_max_cpu_vl(ARM64_VEC_SVE); +} + /* Ensure vq >= SVE_VQ_MIN && vq <= SVE_VQ_MAX before calling this function */ static inline bool vq_available(enum vec_type type, unsigned int vq) { diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 8a080dbd8988..0bf0837d4adb 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -129,6 +129,7 @@ __ro_after_init struct vl_info vl_info[ARM64_VEC_MAX] = { .min_vl = SVE_VL_MIN, .max_vl = SVE_VL_MIN, .max_virtualisable_vl = SVE_VL_MIN, + .max_cpu_vl = SVE_VL_MIN, }, #endif #ifdef CONFIG_ARM64_SME @@ -1041,8 +1042,13 @@ static void vec_probe_vqs(struct vl_info *info, void __init vec_init_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; + unsigned long b; + vec_probe_vqs(info, info->vq_map); bitmap_copy(info->vq_partial_map, info->vq_map, SVE_VQ_MAX); + + b = find_first_bit(info->vq_map, SVE_VQ_MAX); + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1054,11 +1060,16 @@ void vec_update_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); + unsigned long b; vec_probe_vqs(info, tmp_map); bitmap_and(info->vq_map, info->vq_map, tmp_map, SVE_VQ_MAX); bitmap_or(info->vq_partial_map, info->vq_partial_map, tmp_map, SVE_VQ_MAX); + + b = find_first_bit(tmp_map, SVE_VQ_MAX); + if (__bit_to_vl(b) > info->max_cpu_vl) + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1069,10 +1080,23 @@ int vec_verify_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); - unsigned long b; + unsigned long b, max_vl; vec_probe_vqs(info, tmp_map); + /* + * Currently the maximum VL is only used for pKVM which + * doesn't allow late CPUs but we don't expect asymmetry and + * if we encounter any then future users will need handling so + * warn if we see anything. + */ + max_vl = __bit_to_vl(find_first_bit(tmp_map, SVE_VQ_MAX)); + if (max_vl > info->max_cpu_vl) { + pr_warn("%s: cpu%d: increases maximum VL to %lu\n", + info->name, smp_processor_id(), max_vl); + info->max_cpu_vl = max_vl; + } + bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX); if (bitmap_intersects(tmp_map, info->vq_map, SVE_VQ_MAX)) { pr_warn("%s: cpu%d: Required vector length(s) missing\n", From patchwork Tue Jul 30 13:29:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13747405 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8BA24C3DA49 for ; Tue, 30 Jul 2024 13:33:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=UeUIfmVd2L1bFv87y4nlazJJh6F1riiSutBaaqv8tEw=; b=M/fjLCplcHNn197w+afJdPCD9R hxu4qgoCYwJ6feMNXAymya2jm1lHi6+FKMoIwCbHtUNeQj3/RaWd4hXz74XdKqG94fKit1vvpnp9C xO9hnHke+3f2Kak9NSaNYjlRFyEtH8NAhhHHHAApuUYkp0VyusnvRKPAPcOWXkmR/Zs66npmoMMA5 Sk7blSpv5s5Gh6SDBWLJYcDoqbIW1baAJW9sOEj9qVhkjY6+mah1dHf6U9Y1/goSx9P4CPzB9uz2U I2ATKlsMcLhJp0EiIZgWCFUi0a32nKHvG/6YhABh+DJjaw6pF/ckSZer/cKxwT+CADNI5gOHQSYWA 5O4LoVNw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sYmy8-0000000FHiz-2Sk8; Tue, 30 Jul 2024 13:32:48 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sYmwX-0000000FHGk-44cj for linux-arm-kernel@lists.infradead.org; Tue, 30 Jul 2024 13:31:11 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6BFBD61EF9; Tue, 30 Jul 2024 13:31:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CC638C4AF0F; Tue, 30 Jul 2024 13:31:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722346269; bh=Hh0bBv0Uinp19NOdsxRBjh5fZ/0nyJqTw+fsAoXtvjo=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=ADIJ1hB9RSjFfyojHeR87Wgjg0+BLjd5ZuV3bILAa+ZOn+rfu1Pn/YPmetVPNVQ7I opWl3+lwQ1AtAZOx/vyuUL+kKesPLWg66TqlVKD4MX/by5lD0AujDWfXJm0JUaSGlQ AyX6VDjKp4W49NPS3dRwM8mD7/zHG/ERpFreAYVGl1vZjgHRMmPZXWL4NtUMuot3nc xpfUAZSVcyV6oaHDKhd3SDneBXJlEM4FdH47sdfeuafmOTCPcUEgZz6Ol2psbzCDHR GNsMb85Hy+eWAb3Q7KGpnqAGMQ9Seng+CHmUkI4DEQQIuOtvfSRxXCuERf/mh0PoeA 40tQbfl5GOXBQ== From: Mark Brown Date: Tue, 30 Jul 2024 14:29:15 +0100 Subject: [PATCH v6 3/4] KVM: arm64: Fix FFR offset calculation for pKVM host state save and restore MIME-Version: 1.0 Message-Id: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-3-cae8a2e0bd66@kernel.org> References: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-0-cae8a2e0bd66@kernel.org> In-Reply-To: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-0-cae8a2e0bd66@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=3206; i=broonie@kernel.org; h=from:subject:message-id; bh=Hh0bBv0Uinp19NOdsxRBjh5fZ/0nyJqTw+fsAoXtvjo=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmqOsQJWqCq6fmsC0JdY+DmVUd1I3fd1kQDilFaVnx a07qBhSJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZqjrEAAKCRAk1otyXVSH0BODB/ 9Cm+AOQiqOzGljQgfVO2X+nhRfYllxzG+ma4VpNwilRfbKXBxbtsYZbNxLooa4VUcBTf0Je4aqCaik SotpKbJ7qNBVX1OcNs/4hEX2+SI0QLP8sRyvh43YIJmPdUY20nkbI9YTr4S4mkvWXAeYk89Oxi/4Sd 5Sg8T0Q0UJFkL5HQSlLAInrZ1d54JGChgdFAYMfzOgeY+q3FuYoAIKkR/T5YyHWj6rswVi/whsfxjB Q/LzDc+ZJ+zec9L1d5z1EH6qKBrpsnOVKb8Ll3C3tq/MrL7FQRNRwkoNZ7tVgydrSNEczlMTWoPA3+ nOlLuiVT0WC4YGJbDdYnDS7LslGjLB X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240730_063110_149744_2D367F86 X-CRM114-Status: GOOD ( 17.52 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When saving and restoring the SVE state for the host we configure the hardware for the maximum VL it supports, but when calculating offsets in memory we use the maximum usable VL for the host. Since these two values may not be the same this may result in data corruption in the case where the PE supports a VL larger than the maximum usable VL for the host. We can just read the current VL from the hardware with an instruction so use that instead of a saved value, we need to correct the value used to lay out the stored data and this makes it clear that the layout is consistent with the hardware configuration. Fixes: b5b9955617bc ("KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_hyp.h | 1 + arch/arm64/kvm/hyp/fpsimd.S | 5 +++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 2 +- 4 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index c838309e4ec4..6b074f4d48b2 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -113,6 +113,7 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); +int __sve_get_vl(void); u64 __guest_enter(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index e950875e31ce..d272dbf36da8 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -31,3 +31,8 @@ SYM_FUNC_START(__sve_save_state) sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) + +SYM_FUNC_START(__sve_get_vl) + _sve_rdvl 0, 1 + ret +SYM_FUNC_END(__sve_get_vl) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index f59ccfe11ab9..526e49b6f09e 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -340,7 +340,7 @@ static inline void __hyp_sve_save_host(void) sve_state->zcr_el1 = read_sysreg_el1(SYS_ZCR); write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); } diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index f43d845f3c4e..bd8f671e848c 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -49,7 +49,7 @@ static void __hyp_sve_restore_host(void) * supported by the system (or limited at EL3). */ write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); From patchwork Tue Jul 30 13:29:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13747406 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 50657C3DA70 for ; Tue, 30 Jul 2024 13:33:25 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=NhndWHFDrq6nyiHQenyYPYtflAuQqdyLmV+zGQchIjU=; b=k7E+puPEjvhh2EqBfrzjdP/t95 DDc3UkD6uVePe+0ptkRnSnL0xmYSWfwHZhtSHUltFrcZVzLawOI2KoFfIltAZPWYszvMfidDY1Kfy ZE2AwFEhnDjLnLb0RxCLnt8MBjCJJjRCOKGDkX6wI+pC031HmcPUE6ETh97yZV5JiTTCbGBybkDzt b9dbf+AQ0IoLMvsDnfRydewXlSercpqFhqKSqpYyqXaFS2XYEjrztAJXiO5t1nVdnpSWZNxzd9EIH HciAhm0h2pLaB6YV51/3UnZiPkI64G2RM2wcpZWpcC5rYJwJBVg07Q7v9yiq+Lq6WKNbSy3MQ8js9 TFQ2fqmw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sYmyZ-0000000FHrU-1SYf; Tue, 30 Jul 2024 13:33:15 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sYmwc-0000000FHIP-1Oee for linux-arm-kernel@lists.infradead.org; Tue, 30 Jul 2024 13:31:16 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 727F1CE1049; Tue, 30 Jul 2024 13:31:12 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id AAAF9C32782; Tue, 30 Jul 2024 13:31:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1722346271; bh=azeTNSAQsJHmB+1qtA+vN+h7AFTh1KTVQoOhua+u3Cw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=FpKi+w1bzbmSael71ujmkqqanGBv9Vkv42KavF/xlXfy11drjoKXAeD7UG10lqs01 uJElHjRU6bDcZyxOAibEvusLriezau0DkBU3brXaQ3fqcc6nh3Pczu41HAxJXNQ+GB 19oPPZl+xuBF6Og519YAi2eXgVn2z45bSiuUrTMQ90JVjEq87ij6DABVpFu5segubF PBynNvyjdLl/rPyhsQlYydy5KV7Em480KZrAGPIc/yRhAKlNKcrRxlgoQroj/nfSJ5 CBjgsiXwZYSNUBo1sh0T9n05hGlby90MB5w6Tq7tNy3Gv1uB5c+l0xgR9gh1oLtogB itYya3mJPVl2w== From: Mark Brown Date: Tue, 30 Jul 2024 14:29:16 +0100 Subject: [PATCH v6 4/4] KVM: arm64: Avoid underallocating storage for host SVE state MIME-Version: 1.0 Message-Id: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-4-cae8a2e0bd66@kernel.org> References: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-0-cae8a2e0bd66@kernel.org> In-Reply-To: <20240730-kvm-arm64-fix-pkvm-sve-vl-v6-0-cae8a2e0bd66@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.15-dev-37811 X-Developer-Signature: v=1; a=openpgp-sha256; l=4775; i=broonie@kernel.org; h=from:subject:message-id; bh=azeTNSAQsJHmB+1qtA+vN+h7AFTh1KTVQoOhua+u3Cw=; b=owGbwMvMwMWocq27KDak/QLjabUkhrQVrwUZLyS8/usqfqR2yh1u3x2rJZUbfihxnM+UUMmJ5M58 uYupk9GYhYGRi0FWTJFl7bOMVenhElvnP5r/CmYQKxPIFAYuTgGYiGQa+z/z96rZTiHr+VJeqR5fsG PduR217pvyPTifTmQtVtDvv7C97xP/VjWFE7M1qhOnufdvDu1SOrFMN0CY77u9lb3cdvkFU+xKdx4x N1G//nWy9sy9PsHhnPWGyZzqU/zNWH4tFJ5i82a6ju20ghjrGwIliRs0D2wMiVXaI/vNVeCPb8Qz7i nmB//tK/UIKZ57d2dCVPH/9N43TZ+yH92YEPJFVuhly9H3gsHr9/ROtv12jDvRzmzptbRd2tudmUr/ Z6d8S+AqO2RX2GlhHxf7NOhlZN0y02Pveos2cWx4GZidXuHGbvju/mwN1uWcD8J5VhuJx7MIvgyy/r Xwd726x7oQF4apT7ozEp3LOaUTAA== X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240730_063114_810107_CBC84A13 X-CRM114-Status: GOOD ( 16.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We size the allocation for the host SVE state using the maximum VL shared by all CPUs in the host. As observed during review on an asymmetric system this may be less than the maximum VL supported on some of the CPUs. Since the pKVM hypervisor saves and restores the host state using the maximum VL for the current CPU this may lead to buffer overflows, fix this by changing pKVM to use the maximum VL for any CPU to size allocations and limit host configurations. Fixes: 66d5b53e20a6 ("KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 4 ++-- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 6 +++--- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index a33f5996ca9f..c0ea0b7841d6 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -76,7 +76,7 @@ static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; }; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); extern unsigned int __ro_after_init kvm_sve_max_vl; -extern unsigned int __ro_after_init kvm_host_sve_max_vl; +extern unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; int __init kvm_arm_init_sve(void); u32 __attribute_const__ kvm_target_cpu(void); diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 6b074f4d48b2..19f3ae9f05a9 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -144,6 +144,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); -extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_cpu_vl); #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h index cd56acd9a842..6fc0cf42fca3 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -134,7 +134,7 @@ static inline size_t pkvm_host_sve_state_size(void) return 0; return size_add(sizeof(struct cpu_sve_state), - SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_cpu_vl))); } #endif /* __ARM64_KVM_PKVM_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index bd8f671e848c..d232775b72c9 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -90,8 +90,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_vcpu->vcpu.arch.ctxt = host_vcpu->arch.ctxt; hyp_vcpu->vcpu.arch.sve_state = kern_hyp_va(host_vcpu->arch.sve_state); - /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_vl); + /* Limit guest vector length to the maximum supported by any CPU. */ + hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_cpu_vl); hyp_vcpu->vcpu.arch.hw_mmu = host_vcpu->arch.hw_mmu; diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 187a5f4d56c0..770d66491b76 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -18,7 +18,7 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; -unsigned int kvm_host_sve_max_vl; +unsigned int kvm_host_sve_max_cpu_vl; /* * Set trap register values based on features in ID_AA64PFR0. diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 0b0ae5ae7bc2..6c87d01514ff 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,7 +32,7 @@ /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; -unsigned int __ro_after_init kvm_host_sve_max_vl; +unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; /* * ARMv8 Reset Values @@ -52,8 +52,8 @@ int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { kvm_sve_max_vl = sve_max_virtualisable_vl(); - kvm_host_sve_max_vl = sve_max_vl(); - kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl; + kvm_host_sve_max_cpu_vl = sve_max_cpu_vl(); + kvm_nvhe_sym(kvm_host_sve_max_cpu_vl) = kvm_host_sve_max_cpu_vl; /* * The get_sve_reg()/set_sve_reg() ioctl interface will need