From patchwork Mon Jul 8 14:19:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13726660 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D864FC3DA42 for ; Mon, 8 Jul 2024 14:21:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=ZReY5oQp5q5JDnQkU4OgYtMQicBtBNzGvNRsdVnSk/U=; b=G9qCFuB6/+zmddAS5ZskyRMScg FiJNxb//oL2hjsRS/FzaROlgBu0rPbegHRtJkY5I1bNyUda/Lgn3MBhy7yIvRPfxDGJQuZJTydvyw baRH65OGMEv2Ppo6T4ILC3+FmEmz4cJtTYvBAnGlGHnGpN5PsBcXq7eaK5iFbooommYEz5adYO5OB O2CoKnLQ8q00uooAVDwc0Zqghe81fAHwFxrSdQaDatYk9OxM18aDbhffy4ejN/jG+xnjayUFNcRuS uRVlEtNim0LPIAb65B87aTRcMWKIzVwIJnxYZJryZ5sFoxfTGn0CAT+l7TWQFvj4huG5dhfbEOSUE Mg3HdsVQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sQpFK-000000042yf-1ttE; Mon, 08 Jul 2024 14:21:38 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sQpEw-000000042r0-1bGW for linux-arm-kernel@lists.infradead.org; Mon, 08 Jul 2024 14:21:15 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 9F7ED60D39; Mon, 8 Jul 2024 14:21:13 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C5CE2C32786; Mon, 8 Jul 2024 14:21:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720448473; bh=hZLMZ8MyT6kMtoPFEmNPK6rAznf1lF+TnbSpYE4U9Gw=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=igG2dyhbNOmzvNvCMHEukqBdePfK9G/NYPDS2EO7nic3MZZh8iNe8dGQbeuIr1GDG /NDtmIHVZFsz3jzVxElOZhTorpMTlkbL/3u0tJ2c4IxfFSKxxfilnR8cmOmecf9kpp xIHbyV4NI4iyvHJja+b+pvE7vF8vtpU1/SIfP+0CcYI68WLkady/Sb+XiQl1iuUWPE fjTMedVifM0hnFBIlY8kP/YHGiOV9/zgh2vixqzRuEuAAnwxGg5BnhuMJgLl7YkzYH vA3Oy7REIpn5tJETofvIXhZzUOK3qfCROzzip5O+R0iSvoXOThUNj732pEfm/ISGqe RtXjR7AfDj8Yg== From: Mark Brown Date: Mon, 08 Jul 2024 15:19:13 +0100 Subject: [PATCH v5 1/4] arm64/fpsimd: Introduce __bit_to_vl() helper MIME-Version: 1.0 Message-Id: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-1-d2175738456b@kernel.org> References: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-0-d2175738456b@kernel.org> In-Reply-To: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-0-d2175738456b@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=3064; i=broonie@kernel.org; h=from:subject:message-id; bh=hZLMZ8MyT6kMtoPFEmNPK6rAznf1lF+TnbSpYE4U9Gw=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmi/XKD2+TUI1Cyn7ePjXDspx0VpCug0kIGJ3W17j+ 9vXMgyOJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZov1ygAKCRAk1otyXVSH0GrqB/ 9NYKedb06mp3A8L+/yLmh/C+DS10cFI2D9vDHdMbYe2lm1NaaFK1UeX3fKV6oOa0V2K0ginMD8xQmZ 9By4lViccEEJXgGM+Bs51VZ0KwFl51N34wiC3fLMOhYZCGRGPp1YdjNrSmbKHQGvevm3og64/1Oq+r qXl+Zf3CA6vL3hWf4vlV666MHRQtMQjHhyB3kR69RkgWG3apvWSorwWjsIQdIFBNC//FnpR72bjR/N e1RNkqY/lIDaXMccswaDgANHVFuJMRN7tCFk4d0YEVT5n+4xEeT5VQxtg7a/mf9az+l845mutVc1l+ gjd4cUB8a5FUeU7ux5Ysfua/xbC+lP X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240708_072114_547924_9DB47E5D X-CRM114-Status: GOOD ( 13.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In all cases where we use the existing __bit_to_vq() helper we immediately convert the result into a VL. Provide and use __bit_to_vl() doing this directly. Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 4 ++++ arch/arm64/kernel/fpsimd.c | 12 ++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index bc69ac368d73..51c21265b4fa 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -172,6 +172,10 @@ static inline unsigned int __bit_to_vq(unsigned int bit) return SVE_VQ_MAX - bit; } +static inline unsigned int __bit_to_vl(unsigned int bit) +{ + return sve_vl_from_vq(__bit_to_vq(bit)); +} struct vl_info { enum vec_type type; diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 82e8a6017382..22542fb81812 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -530,7 +530,7 @@ static unsigned int find_supported_vector_length(enum vec_type type, bit = find_next_bit(info->vq_map, SVE_VQ_MAX, __vq_to_bit(sve_vq_from_vl(vl))); - return sve_vl_from_vq(__bit_to_vq(bit)); + return __bit_to_vl(bit); } #if defined(CONFIG_ARM64_SVE) && defined(CONFIG_SYSCTL) @@ -1103,7 +1103,7 @@ int vec_verify_vq_map(enum vec_type type) * Mismatches above sve_max_virtualisable_vl are fine, since * no guest is allowed to configure ZCR_EL2.LEN to exceed this: */ - if (sve_vl_from_vq(__bit_to_vq(b)) <= info->max_virtualisable_vl) { + if (__bit_to_vl(b) <= info->max_virtualisable_vl) { pr_warn("%s: cpu%d: Unsupported vector length(s) present\n", info->name, smp_processor_id()); return -EINVAL; @@ -1169,7 +1169,7 @@ void __init sve_setup(void) set_bit(__vq_to_bit(SVE_VQ_MIN), info->vq_map); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); /* * For the default VL, pick the maximum supported value <= 64. @@ -1188,7 +1188,7 @@ void __init sve_setup(void) /* No virtualisable VLs? This is architecturally forbidden. */ info->max_virtualisable_vl = SVE_VQ_MIN; else /* b + 1 < SVE_VQ_MAX */ - info->max_virtualisable_vl = sve_vl_from_vq(__bit_to_vq(b + 1)); + info->max_virtualisable_vl = __bit_to_vl(b + 1); if (info->max_virtualisable_vl > info->max_vl) info->max_virtualisable_vl = info->max_vl; @@ -1305,10 +1305,10 @@ void __init sme_setup(void) WARN_ON(bitmap_empty(info->vq_map, SVE_VQ_MAX)); min_bit = find_last_bit(info->vq_map, SVE_VQ_MAX); - info->min_vl = sve_vl_from_vq(__bit_to_vq(min_bit)); + info->min_vl = __bit_to_vl(min_bit); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); WARN_ON(info->min_vl > info->max_vl); From patchwork Mon Jul 8 14:19:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13726661 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 77BA9C3DA42 for ; Mon, 8 Jul 2024 14:22:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=1t0C0IrVVScI2l434dpW/oJqYwsOOgHr+pEfLpV/lmM=; b=xhpajLQTJYJKHX13RHPssLPf5z CwGWMtRUEeYEL139pTVOeKJ0grXgSPshxsz4UsiHrdlcP3Nj51bHZY8qDr8LETDLnl4QKyXpW74WO RlXU59qm2BH3oLyluKkd+o1RB97Kj2BMTBUT4kk+T3U2vZjT1ulXK8BXH4wbOvTF9jKiGw6Q5may6 7hBV3jFKBrurKOD3JXuCAc/VnCRrNiBjlkLPzDfyeaom4d0iqqnlTBmXnu41lHlahHHYISqS4UWgC dEBd2DP0DLwCTsF5GR17ACWGk9rsbJEx90Sik6owKz+tYQMjcoIk5CdqsM58mNZTvq8G66TDokMnf WbNHB6kQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sQpFW-00000004349-3TVm; Mon, 08 Jul 2024 14:21:50 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sQpF2-000000042se-0nyo for linux-arm-kernel@lists.infradead.org; Mon, 08 Jul 2024 14:21:21 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 6F44560DCF; Mon, 8 Jul 2024 14:21:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0C5FFC4AF0C; Mon, 8 Jul 2024 14:21:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720448479; bh=qYUCz/GnFjAVns0VEq9hJ3jJLihZ2D7gvH2hjVtPvmE=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=itHV/iQJwX8AF0mI9TaSqAD973EohWsJWScNxYGLJWJTGYEIC6IKqWebdTzlgP9tb lrY9lQw1dBCemPH2nR2Zl+LJ0Ng69HF7xguBjtKXYvjlKPhZPEkKUOQZZ/xvk66um1 Tk9bpL2U3K9ZfHYq7lQ7MUm9DpVJLZGmc19AjRMYKn/AQ96pesrEzlTLSRidQ4x+xu WCxElCKnuT5EkUQ30/DM+gOqffsPWf0buki3itzo3tBr+dHz84evoVgYgucyI9x846 NrVyueuGOb9CqNN5iqKpYd9x2Tm+BzLWNd54TRn2XzWcuN1ZpUgrL1B+jr3WWQN+Fd Lwzh7V8N5Y/oA== From: Mark Brown Date: Mon, 08 Jul 2024 15:19:14 +0100 Subject: [PATCH v5 2/4] arm64/fpsimd: Discover maximum vector length implemented by any CPU MIME-Version: 1.0 Message-Id: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-2-d2175738456b@kernel.org> References: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-0-d2175738456b@kernel.org> In-Reply-To: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-0-d2175738456b@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=4205; i=broonie@kernel.org; h=from:subject:message-id; bh=qYUCz/GnFjAVns0VEq9hJ3jJLihZ2D7gvH2hjVtPvmE=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmi/XKqIXw3fqv/owoP6gSsQUyUHt0UKjGj4ewoE4Y aUiUetuJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZov1ygAKCRAk1otyXVSH0LALB/ 9loGOqhZh6mzKtDzQY7Gt3WP8tKnsrr8g3ThOs4gC1wusZ15hNO9LnmgRrdGew8JfJbMNnXsIDJcju WBy3IKnes8EvHng3lyvt8kFK7tVd/qnpmjoa+B/a4F/sds3n75V2QcWeDZu4Ss3ohtocFnTa/OR3CV PjgaMuVr6yHq+QzPKIoG2+ct49pPmJMIH1npDHAy5PKT80Sk4dUQHtyofOtjmyuTsvIVQVLVJ2/sId R8pnUNFShoC+4ChVRAEW0I+a2OS3vCgyEFg3JfZte9sMgpeJtiPdPzYq95EEal/SSosY2u7KCs8LJW ksqORQ+4LddvoKt7XqD1YAICWUsqqX X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240708_072120_428705_1479F61B X-CRM114-Status: GOOD ( 17.48 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When discovering the vector lengths for SVE and SME we do not currently record the maximum VL supported on any individual CPU. This is expected to be the same for all CPUs but the architecture allows asymmetry, if we do encounter an asymmetric system then some CPUs may support VLs higher than the maximum Linux will use. Since the pKVM hypervisor needs to support saving and restoring anything the host can physically set it needs to know the maximum value any CPU could have, add support for enumerating it and validation for late CPUs. Acked-by: Catalin Marinas Signed-off-by: Mark Brown --- arch/arm64/include/asm/fpsimd.h | 13 +++++++++++++ arch/arm64/kernel/fpsimd.c | 26 +++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 51c21265b4fa..cd19713c9deb 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -188,6 +188,9 @@ struct vl_info { int max_vl; int max_virtualisable_vl; + /* Maximum vector length observed on any CPU */ + int max_cpu_vl; + /* * Set of available vector lengths, * where length vq encoded as bit __vq_to_bit(vq): @@ -278,6 +281,11 @@ static inline int vec_max_virtualisable_vl(enum vec_type type) return vl_info[type].max_virtualisable_vl; } +static inline int vec_max_cpu_vl(enum vec_type type) +{ + return vl_info[type].max_cpu_vl; +} + static inline int sve_max_vl(void) { return vec_max_vl(ARM64_VEC_SVE); @@ -288,6 +296,11 @@ static inline int sve_max_virtualisable_vl(void) return vec_max_virtualisable_vl(ARM64_VEC_SVE); } +static inline int sve_max_cpu_vl(void) +{ + return vec_max_cpu_vl(ARM64_VEC_SVE); +} + /* Ensure vq >= SVE_VQ_MIN && vq <= SVE_VQ_MAX before calling this function */ static inline bool vq_available(enum vec_type type, unsigned int vq) { diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 22542fb81812..83984cb3f21a 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -129,6 +129,7 @@ __ro_after_init struct vl_info vl_info[ARM64_VEC_MAX] = { .min_vl = SVE_VL_MIN, .max_vl = SVE_VL_MIN, .max_virtualisable_vl = SVE_VL_MIN, + .max_cpu_vl = SVE_VL_MIN, }, #endif #ifdef CONFIG_ARM64_SME @@ -1041,8 +1042,13 @@ static void vec_probe_vqs(struct vl_info *info, void __init vec_init_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; + unsigned long b; + vec_probe_vqs(info, info->vq_map); bitmap_copy(info->vq_partial_map, info->vq_map, SVE_VQ_MAX); + + b = find_first_bit(info->vq_map, SVE_VQ_MAX); + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1054,11 +1060,16 @@ void vec_update_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); + unsigned long b; vec_probe_vqs(info, tmp_map); bitmap_and(info->vq_map, info->vq_map, tmp_map, SVE_VQ_MAX); bitmap_or(info->vq_partial_map, info->vq_partial_map, tmp_map, SVE_VQ_MAX); + + b = find_first_bit(tmp_map, SVE_VQ_MAX); + if (__bit_to_vl(b) > info->max_cpu_vl) + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1069,10 +1080,23 @@ int vec_verify_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); - unsigned long b; + unsigned long b, max_vl; vec_probe_vqs(info, tmp_map); + /* + * Currently the maximum VL is only used for pKVM which + * doesn't allow late CPUs but we don't expect asymmetry and + * if we encounter any then future users will need handling so + * warn if we see anything. + */ + max_vl = __bit_to_vl(find_first_bit(tmp_map, SVE_VQ_MAX)); + if (max_vl > info->max_cpu_vl) { + pr_warn("%s: cpu%d: increases maximum VL to %lu\n", + info->name, smp_processor_id(), max_vl); + info->max_cpu_vl = max_vl; + } + bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX); if (bitmap_intersects(tmp_map, info->vq_map, SVE_VQ_MAX)) { pr_warn("%s: cpu%d: Required vector length(s) missing\n", From patchwork Mon Jul 8 14:19:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13726667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 86C23C3271E for ; Mon, 8 Jul 2024 14:22:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=BBngTtrENLiuwKGJgnVKMpB6+6gfpmn2p5MKjDnT+a0=; b=lK/jpcIC7dDgAVMbb7aUanD2yQ o5yLROSffkqb9z/NkcFm6Ows4hyk3PqrSvLmb534uhGlTlDILD9tfd5ai32/hRWRt3Ji/y++nmWBi 0iGq/mlXqSuMELULM2HaIPTTPbMk68nTBtlahKkWE+QoD8DYtdJlg7hVzCJ6h66hWwwG+lGURXJJw kT56B6jiw9lgwqz3CNJNM86nYsaY4JbI3urPXp+VBeC1z+blbecbXdmT5CcLUFFOv5aoI+X1WKcuC e4XTAaa5S5Nh0v8o5AelAowjXDfYraHe2GfCP05adq8KJPZte7BtLGRyMOHBVT/ppSx5nKJVQ6JI+ ahg/oGfw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sQpFk-000000043Ah-3LlL; Mon, 08 Jul 2024 14:22:04 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sQpF9-000000042uK-0GLO for linux-arm-kernel@lists.infradead.org; Mon, 08 Jul 2024 14:21:28 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 3ED97CE0CEB; Mon, 8 Jul 2024 14:21:25 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 24F07C32786; Mon, 8 Jul 2024 14:21:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720448484; bh=lND+9Wrq6h84wg/7dcgVXzhKhNM8RJROdnPLpTYN0xk=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=GaWkGbdRkfWJTCn44B+jyvf7DEOkcAdjP+Q7GAUwynSRjyx9/oK2thfv35LHDzCvm LYTBJ8gvAyWR5q7Oy7t8KOS0eWJcadvlbfybA7R3n5UiBMbyGppxz14sAYPcAgNjeE 9vv2yWC38y6FDr/UNDQ7jH0iHtrQ/lCvWeK6A+GUChkp0knVOJr56y4MOPJDIrAd3c cCjqoZC6nIELBzT2AVVpx/VONrU/AxJaHZgItiDtm/tZovejftMgsIzKrhip76HKQV hMfjf4vGpHXO7cdGBfeLKZOX/MyZGv3nnjUFVfR762Eor+pX0VOB3lvwXtQl8MjZyF hBuOX6nsaSXMA== From: Mark Brown Date: Mon, 08 Jul 2024 15:19:15 +0100 Subject: [PATCH v5 3/4] KVM: arm64: Fix FFR offset calculation for pKVM host state save and restore MIME-Version: 1.0 Message-Id: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-3-d2175738456b@kernel.org> References: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-0-d2175738456b@kernel.org> In-Reply-To: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-0-d2175738456b@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=3206; i=broonie@kernel.org; h=from:subject:message-id; bh=lND+9Wrq6h84wg/7dcgVXzhKhNM8RJROdnPLpTYN0xk=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmi/XLR+gOeEj+oJx6GbLGY6GRKjPkxOMypVwTg0mg cp5Wv+KJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZov1ywAKCRAk1otyXVSH0N2cB/ 0XusVxwjo05ZSzJHcrIrNFF/cFKAlf/3cx4Slc8IPYSDz/JSPi5w5FT/2F5CzMoliN9owVuPNDYWxF IQcpTCybn0vX+fzS4PB+odCHagG475eG2J/t76uh4P4DCoHN+v65XJsUgi795PrcYfc+RMUx75YNgX 1vNZzZOT6lXrCqybWKKMqzl/pj9TJdPPqj/+6ILmBGmqLrVi0+iWKgLu9IvGcpBx9vnzwHVrqYJVKj rge+WgNfvRWuAg+iA7A92qsjLJKXFwHo8WJgBgyQBvFKSKBDLg5pF8BIgO1vP6eQj6/bCDAd6SGdx6 NNXfx6NdzOZolCACd146CtIzfRGCDi X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240708_072127_498921_E699F565 X-CRM114-Status: GOOD ( 17.70 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When saving and restoring the SVE state for the host we configure the hardware for the maximum VL it supports, but when calculating offsets in memory we use the maximum usable VL for the host. Since these two values may not be the same this may result in data corruption in the case where the PE supports a VL larger than the maximum usable VL for the host. We can just read the current VL from the hardware with an instruction so use that instead of a saved value, we need to correct the value used to lay out the stored data and this makes it clear that the layout is consistent with the hardware configuration. Fixes: b5b9955617bc ("KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_hyp.h | 1 + arch/arm64/kvm/hyp/fpsimd.S | 5 +++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 2 +- 4 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index b05bceca3385..7510383d78a6 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -113,6 +113,7 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); +int __sve_get_vl(void); u64 __guest_enter(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index e950875e31ce..d272dbf36da8 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -31,3 +31,8 @@ SYM_FUNC_START(__sve_save_state) sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) + +SYM_FUNC_START(__sve_get_vl) + _sve_rdvl 0, 1 + ret +SYM_FUNC_END(__sve_get_vl) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 0c4de44534b7..06efcca765cc 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -327,7 +327,7 @@ static inline void __hyp_sve_save_host(void) sve_state->zcr_el1 = read_sysreg_el1(SYS_ZCR); write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); } diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index f43d845f3c4e..bd8f671e848c 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -49,7 +49,7 @@ static void __hyp_sve_restore_host(void) * supported by the system (or limited at EL3). */ write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); From patchwork Mon Jul 8 14:19:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13726668 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B38D2C3DA42 for ; Mon, 8 Jul 2024 14:22:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References :Message-Id:Content-Transfer-Encoding:Content-Type:MIME-Version:Subject:Date: From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=Opd3A7s84qlzwKyOZlIUZ/pQ0b5O9yKQ/e+oIiMrphg=; b=DG6U+w8kkrKNx3Ujm/MVYXmOUB WNIEYTelALG2m7NBE2d38Pih9GknscarqT9K3mELhSWc+0QcC6qaxzUocK4wkaX8wrzbOu2/52YxC ObVWMgIrWbaoceZYC8OZj01tW/LSz0x3LMBw5+rX+aUNL6rd72K5/enKZ/2J3hT12/RIYsceCSmSE Oi+8Rxm/O9xDO5yHLu1UjixEtgABE9j08U5U2xoMlpw9GaOOo3CLy4SMIxNQwj8TE+P/6DUs0PxX0 R66HB/cY+iJatmVv/chzXsBDye3wOkWZ42ybkf0d63PFKpMF9267AZcggbOWxTWt6MIbwZ8Gob44U NZftKF2Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sQpFy-000000043G3-2D3I; Mon, 08 Jul 2024 14:22:18 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sQpFA-000000042vK-3xR8 for linux-arm-kernel@lists.infradead.org; Mon, 08 Jul 2024 14:21:31 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 5E27F60DCF; Mon, 8 Jul 2024 14:21:28 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 2C71DC4AF0D; Mon, 8 Jul 2024 14:21:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1720448488; bh=I+qiD9f7AIjkeTzE7i3sCyRtTk9/3BCUmNFuB2K/uis=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Vp4kzNNvKbXWf3zhVGfHBzdKRJNBgFz9qjeST/6+M7zzz4CKGIEOYxgbYiWqLq7j9 tyXrIi6LN0AWYZzwFYTBX8XEG7L4ZbwTTIMm9yi5jncKFmTTG4wa3dJrDD5MNQpo91 huzVH0pnR8YlMn9U0dDA/TFB+Sj4QwHPu8UuM9hEYGB+mIlWRQRCV97pDBHCtZyMab Ux9Rs3igd0F0KCO7ONFFihg7Q4fs+4bAj6Ofa3HXBWNDrT8NgHcHvXvWJJ8iXgGGgc 7xVLurNYMzrjF+3n4TQgUktBPZuRBKTrtMIfdztmO1hY8yB/ol5QSKKRhwF1KE+I2f 79RuN76zSPyWQ== From: Mark Brown Date: Mon, 08 Jul 2024 15:19:16 +0100 Subject: [PATCH v5 4/4] KVM: arm64: Avoid underallocating storage for host SVE state MIME-Version: 1.0 Message-Id: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-4-d2175738456b@kernel.org> References: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-0-d2175738456b@kernel.org> In-Reply-To: <20240708-kvm-arm64-fix-pkvm-sve-vl-v5-0-d2175738456b@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=4775; i=broonie@kernel.org; h=from:subject:message-id; bh=I+qiD9f7AIjkeTzE7i3sCyRtTk9/3BCUmNFuB2K/uis=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmi/XMu4cRYgtgns0eJFSPFSyBZvnxLfPxFSYzaMa4 EA7BnIKJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZov1zAAKCRAk1otyXVSH0O64B/ 9a+Gsw3n7tlK39MwlUbKY1Rjc7CO6k6+AtOnCCBHECbaoBSUZvZb65Zq9Aa+5mtmveLDrZeUzHyQaF 9J5V4C2Z11KYVVp2WNE9d6YyNXxcuyXr2ZtyrL2iYU2W+9IVzP8Hw96IQW5BaMV/xdXofqVvITih8M y3Pcp5bO6or2u/xf08e5V0osihM3cICK4cYA1fJmV9bjMJPltr01POuTRYgss02et7Y0klfgWHOk+y 6mcqjH3GTWRWbBQmdwAQjf3FBRnAqPSWIngoJ7fKus5SpLzO0jOhe5GR3l7fhx+gChAqIUBaVnt3YX SJEKuV1IiADf4UFB32XgJq2q6rE0v7 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240708_072129_146448_B4898D1A X-CRM114-Status: GOOD ( 16.43 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We size the allocation for the host SVE state using the maximum VL shared by all CPUs in the host. As observed during review on an asymmetric system this may be less than the maximum VL supported on some of the CPUs. Since the pKVM hypervisor saves and restores the host state using the maximum VL for the current CPU this may lead to buffer overflows, fix this by changing pKVM to use the maximum VL for any CPU to size allocations and limit host configurations. Fixes: 66d5b53e20a6 ("KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 4 ++-- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 6 +++--- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 36b8e97bf49e..a28fae10596f 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -76,7 +76,7 @@ static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; }; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); extern unsigned int __ro_after_init kvm_sve_max_vl; -extern unsigned int __ro_after_init kvm_host_sve_max_vl; +extern unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; int __init kvm_arm_init_sve(void); u32 __attribute_const__ kvm_target_cpu(void); diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 7510383d78a6..47426df69875 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -144,6 +144,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); -extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_cpu_vl); #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h index cd56acd9a842..6fc0cf42fca3 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -134,7 +134,7 @@ static inline size_t pkvm_host_sve_state_size(void) return 0; return size_add(sizeof(struct cpu_sve_state), - SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_cpu_vl))); } #endif /* __ARM64_KVM_PKVM_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index bd8f671e848c..d232775b72c9 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -90,8 +90,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_vcpu->vcpu.arch.ctxt = host_vcpu->arch.ctxt; hyp_vcpu->vcpu.arch.sve_state = kern_hyp_va(host_vcpu->arch.sve_state); - /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_vl); + /* Limit guest vector length to the maximum supported by any CPU. */ + hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_cpu_vl); hyp_vcpu->vcpu.arch.hw_mmu = host_vcpu->arch.hw_mmu; diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 95cf18574251..08e825de09d1 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -18,7 +18,7 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; -unsigned int kvm_host_sve_max_vl; +unsigned int kvm_host_sve_max_cpu_vl; /* * Set trap register values based on features in ID_AA64PFR0. diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 3fc8ca164dbe..59cccb477cf3 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,7 +32,7 @@ /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; -unsigned int __ro_after_init kvm_host_sve_max_vl; +unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; /* * ARMv8 Reset Values @@ -52,8 +52,8 @@ int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { kvm_sve_max_vl = sve_max_virtualisable_vl(); - kvm_host_sve_max_vl = sve_max_vl(); - kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl; + kvm_host_sve_max_cpu_vl = sve_max_cpu_vl(); + kvm_nvhe_sym(kvm_host_sve_max_cpu_vl) = kvm_host_sve_max_cpu_vl; /* * The get_sve_reg()/set_sve_reg() ioctl interface will need