From patchwork Thu Jun 6 15:21:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13688676 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4CE32C25B75 for ; Thu, 6 Jun 2024 15:32:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wseIEfTyuev/m0ML5A5GwwEwvwK7NfsmdD83ezd9KuU=; b=ffUhiEMKlmxFb+ CemZnjvLHMokcDAbDE4dralSUAHZ2654zeBM8IGG867fcVD3V2tkcBBtj1l3VprB/DEBj+d3vh2PM W6kZ3yuE1pcGqjqxPiUs3PzndUvZOjx2OGfsMGjDTaVJvH9WsIl6NrcyfcslXLlQ/phujRVzve7PN gj8lTuQOKwSTODGVDyRx5+Gw/URgHFUDKikJXRgMJGeb8w/uuTW3vlPMXD35ZLa72coLVrH/KKmNk n4kfTR0B1DXWi3GFZgekLjkCwDb7PL6rDfqnd7k8kA5a6mUWm74N3rCGjdWv3JacBVUBQDGg4mmy1 5qn3EY3oU8UoFBuz7jnA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFF62-0000000AJYJ-1bRd; Thu, 06 Jun 2024 15:32:10 +0000 Received: from sin.source.kernel.org ([145.40.73.55]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFF5x-0000000AJVh-26T2 for linux-arm-kernel@lists.infradead.org; Thu, 06 Jun 2024 15:32:07 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id B3B34CE1A62; Thu, 6 Jun 2024 15:32:03 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 29E65C4AF07; Thu, 6 Jun 2024 15:32:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717687923; bh=Aa17kOoXpX/HoXHXzofiE99MuhIDY6p0rI+xW1GTtrQ=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=mTQrRX1dO1n+8cgLj61SGwqQly5aDKw24AUm7MUQgBslD5lkSJYXmBDZGG9a+UAv/ 5H5AFUbHgj9iDcGuMtzZb8IgPhVitB1FBeDcsZ0hLc+RZBDmg27EnIku+LiXi7yV1T nZCnBq9jpVB5BweNWUHxeGbQuaMHzE3BCbVa8cwgafdDuHZSCMfLtw8TNR8+np+3Fb Zt8ODaNFL2whdpI1s1R+S1JTqE43YsBdKVSjTZhvJh5Slh7e2pnHIkkBghLHKGo3Ba pG4nhziKIYhgdT5SFjhYuo7CUO8jFcbIEomwzu+1Bn3oTJZh/W9Vv6Gfe5qdyMLaIg UPIwurMsIPvQA== From: Mark Brown Date: Thu, 06 Jun 2024 16:21:43 +0100 Subject: [PATCH v2 1/4] arm64/fpsimd: Introduce __bit_to_vl() helper MIME-Version: 1.0 Message-Id: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-1-c88f4eb4b14b@kernel.org> References: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-0-c88f4eb4b14b@kernel.org> In-Reply-To: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-0-c88f4eb4b14b@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=3011; i=broonie@kernel.org; h=from:subject:message-id; bh=Aa17kOoXpX/HoXHXzofiE99MuhIDY6p0rI+xW1GTtrQ=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmYdZq5XeihIh4fcv6onzasAqm1cre+hwvJKRCbbeQ C7IZOfSJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZmHWagAKCRAk1otyXVSH0Ly5B/ 4u1wKyoe85K4X+FUsIco+bd3WhkFtaFxx7izIviLRWJQIkq/EGCq2oJ4HWCvyJzVi7oS9+5T0ARm5H 6dHLTbKAN+aFnuujqOlzMb7uJdakO7HiMPhOHW2jhssMwQnuHC2I43GU+nwwWvPlAgSDTpjk1RuWK3 MEvLMNoAAUy1C6NDXwDJqV6TcQd8O4vEBxE9WkZSKlczYdHOh+Nmq0yfD3dH798mXDWd6Ezk33xdK6 rbqgveKpI+1vuQUV5Y74eYzpTK5JZpl7LD/sioaDBZDhJivBVd4pJw8SJZh+DL5Rh+y125JZh4CqPb xhHS8RuI5NLIyrmR1XLFn7UakgBWzA X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240606_083205_923297_8C98113B X-CRM114-Status: GOOD ( 13.51 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org In all cases where we use the existing __bit_to_vq() helper we immediately convert the result into a VL. Provide and use __bit_to_vl() doing this directly. Signed-off-by: Mark Brown Acked-by: Catalin Marinas --- arch/arm64/include/asm/fpsimd.h | 4 ++++ arch/arm64/kernel/fpsimd.c | 12 ++++++------ 2 files changed, 10 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index bc69ac368d73..51c21265b4fa 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -172,6 +172,10 @@ static inline unsigned int __bit_to_vq(unsigned int bit) return SVE_VQ_MAX - bit; } +static inline unsigned int __bit_to_vl(unsigned int bit) +{ + return sve_vl_from_vq(__bit_to_vq(bit)); +} struct vl_info { enum vec_type type; diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 82e8a6017382..22542fb81812 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -530,7 +530,7 @@ static unsigned int find_supported_vector_length(enum vec_type type, bit = find_next_bit(info->vq_map, SVE_VQ_MAX, __vq_to_bit(sve_vq_from_vl(vl))); - return sve_vl_from_vq(__bit_to_vq(bit)); + return __bit_to_vl(bit); } #if defined(CONFIG_ARM64_SVE) && defined(CONFIG_SYSCTL) @@ -1103,7 +1103,7 @@ int vec_verify_vq_map(enum vec_type type) * Mismatches above sve_max_virtualisable_vl are fine, since * no guest is allowed to configure ZCR_EL2.LEN to exceed this: */ - if (sve_vl_from_vq(__bit_to_vq(b)) <= info->max_virtualisable_vl) { + if (__bit_to_vl(b) <= info->max_virtualisable_vl) { pr_warn("%s: cpu%d: Unsupported vector length(s) present\n", info->name, smp_processor_id()); return -EINVAL; @@ -1169,7 +1169,7 @@ void __init sve_setup(void) set_bit(__vq_to_bit(SVE_VQ_MIN), info->vq_map); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); /* * For the default VL, pick the maximum supported value <= 64. @@ -1188,7 +1188,7 @@ void __init sve_setup(void) /* No virtualisable VLs? This is architecturally forbidden. */ info->max_virtualisable_vl = SVE_VQ_MIN; else /* b + 1 < SVE_VQ_MAX */ - info->max_virtualisable_vl = sve_vl_from_vq(__bit_to_vq(b + 1)); + info->max_virtualisable_vl = __bit_to_vl(b + 1); if (info->max_virtualisable_vl > info->max_vl) info->max_virtualisable_vl = info->max_vl; @@ -1305,10 +1305,10 @@ void __init sme_setup(void) WARN_ON(bitmap_empty(info->vq_map, SVE_VQ_MAX)); min_bit = find_last_bit(info->vq_map, SVE_VQ_MAX); - info->min_vl = sve_vl_from_vq(__bit_to_vq(min_bit)); + info->min_vl = __bit_to_vl(min_bit); max_bit = find_first_bit(info->vq_map, SVE_VQ_MAX); - info->max_vl = sve_vl_from_vq(__bit_to_vq(max_bit)); + info->max_vl = __bit_to_vl(max_bit); WARN_ON(info->min_vl > info->max_vl); From patchwork Thu Jun 6 15:21:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13688678 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 697E6C27C65 for ; Thu, 6 Jun 2024 15:32:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=QqL7gRvCIXEAPmq6rSjJ1k1GtUkzDQm0dta3mY5+D4s=; b=gVH7CDsoGig+t0 xg+53zUefbSG7S0TuqOjiBckIqcSFt3UzNEdrJXuANaaPU9V/MEkJHPgrgkf992oP5PA06XlaqwP/ yidXHCAkikO6pN3MRt5iOtx3YoB9A5yA2t/cSPNl370z77JuQIud+pGzUgP0xrhrb4bx3WsV9IMCD W8cJYKr5r7yYgwTeM5SQ4+BPIrVYF5iRO2arXFS70FCNR1GmGvsctonWY7e8xAna99zCK51FbSrjI Yz2g1FuO+9cow/zBXjmhVMK4q29R051YVfFddaNasS2tkRoRlbw4DZ4lfmUYFARh3p44CTsLKBbsp gLtKAoigrLAmuDlU4c1Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFF63-0000000AJYu-0qjl; Thu, 06 Jun 2024 15:32:11 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFF5y-0000000AJWH-46e7 for linux-arm-kernel@lists.infradead.org; Thu, 06 Jun 2024 15:32:08 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 2FCA161DFF; Thu, 6 Jun 2024 15:32:06 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id B976DC2BD10; Thu, 6 Jun 2024 15:32:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717687926; bh=H3jcxvAHXptT4L841OkSY/wbD68Q058k6Wy2910HAmg=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=Y1ABfoKraIr62CY8JwyBPb7z6fKdniAElsElHbF1eC/TLRRG+mRO1IhFeDSy0cuCv JMyt1NSw6J2OK2NmVX0wUfOO3grajN6DD4G0MshYmeZuOHJcfpaLenC9SmZH/O4LcS rWAHBJLdoOHMTFDD23gmILoMj3u782w447FCYumDYslRhYqgJ3qPbR7xVoqI2bg5Mn jUl8/BnlDtUnpUI+NqWwghBel1DQeqWg8OcAQGaTM5i+H+YzLrObwXG3D5q6xPrYuG 1mmIWeEt2n1GP1gXWRAorkRbR6gaQ0mhftA5rbU5Xp+NbC+31Gj3/NaR+hN7aE5JSG 6kfuXb+cdNLkA== From: Mark Brown Date: Thu, 06 Jun 2024 16:21:44 +0100 Subject: [PATCH v2 2/4] arm64/fpsimd: Discover maximum vector length implemented by any CPU MIME-Version: 1.0 Message-Id: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-2-c88f4eb4b14b@kernel.org> References: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-0-c88f4eb4b14b@kernel.org> In-Reply-To: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-0-c88f4eb4b14b@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=4151; i=broonie@kernel.org; h=from:subject:message-id; bh=H3jcxvAHXptT4L841OkSY/wbD68Q058k6Wy2910HAmg=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmYdZrXwyvssj4DZUsq5Oq0pfKbm9b2RzxxDY7A4mX rG4EM4SJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZmHWawAKCRAk1otyXVSH0MW7B/ 4zRzbdQVt988qBR2E0HSNmsQ3ScD6/w8A55zELsnqftkLXWoPyjvYA4oWQdIluke0jF5N7T/6kJgWb B7itNqcV5SEgVK/IvQ86Sq/NkJ/bedLgWyqyELlEwHs6z0Fxu062KcZAyrczajDmuMgKeYg/q3n3nB MCmA32OMYiWDZ5Nh0xdrrO1pQWacYtiN9mQsRqhFHi8TjzV4n3DBIw8f1j1X3Og3BcwlOalXdF54X/ Kd68sOpBD89fYBkh0ROlMJ2Yc6Hp3rZfPdasagZo8MY9/RjUrQ3XqcRovWFVFf+tbfQzUI2E3PL4eV +LPMypMoA0HM6PfdjDcVXPFMT30j31 X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240606_083207_309787_54C9C98E X-CRM114-Status: GOOD ( 17.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When discovering the vector lengths for SVE and SME we do not currently record the maximum VL supported on any individual CPU. This is expected to be the same for all CPUs but the architecture allows asymmetry, if we do encounter an asymmetric system then some CPUs may support VLs higher than the maximum Linux will use. Since the pKVM hypervisor needs to support saving and restoring anything the host can physically set it needs to know the maximum value any CPU could have, add support for enumerating it and validation for late CPUs. Signed-off-by: Mark Brown Acked-by: Catalin Marinas --- arch/arm64/include/asm/fpsimd.h | 13 +++++++++++++ arch/arm64/kernel/fpsimd.c | 26 +++++++++++++++++++++++++- 2 files changed, 38 insertions(+), 1 deletion(-) diff --git a/arch/arm64/include/asm/fpsimd.h b/arch/arm64/include/asm/fpsimd.h index 51c21265b4fa..cd19713c9deb 100644 --- a/arch/arm64/include/asm/fpsimd.h +++ b/arch/arm64/include/asm/fpsimd.h @@ -188,6 +188,9 @@ struct vl_info { int max_vl; int max_virtualisable_vl; + /* Maximum vector length observed on any CPU */ + int max_cpu_vl; + /* * Set of available vector lengths, * where length vq encoded as bit __vq_to_bit(vq): @@ -278,6 +281,11 @@ static inline int vec_max_virtualisable_vl(enum vec_type type) return vl_info[type].max_virtualisable_vl; } +static inline int vec_max_cpu_vl(enum vec_type type) +{ + return vl_info[type].max_cpu_vl; +} + static inline int sve_max_vl(void) { return vec_max_vl(ARM64_VEC_SVE); @@ -288,6 +296,11 @@ static inline int sve_max_virtualisable_vl(void) return vec_max_virtualisable_vl(ARM64_VEC_SVE); } +static inline int sve_max_cpu_vl(void) +{ + return vec_max_cpu_vl(ARM64_VEC_SVE); +} + /* Ensure vq >= SVE_VQ_MIN && vq <= SVE_VQ_MAX before calling this function */ static inline bool vq_available(enum vec_type type, unsigned int vq) { diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 22542fb81812..ee6fb8c4b16d 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -129,6 +129,7 @@ __ro_after_init struct vl_info vl_info[ARM64_VEC_MAX] = { .min_vl = SVE_VL_MIN, .max_vl = SVE_VL_MIN, .max_virtualisable_vl = SVE_VL_MIN, + .max_cpu_vl = SVE_VL_MIN, }, #endif #ifdef CONFIG_ARM64_SME @@ -1041,8 +1042,13 @@ static void vec_probe_vqs(struct vl_info *info, void __init vec_init_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; + unsigned long b; + vec_probe_vqs(info, info->vq_map); bitmap_copy(info->vq_partial_map, info->vq_map, SVE_VQ_MAX); + + b = find_first_bit(info->vq_map, SVE_VQ_MAX); + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1054,11 +1060,16 @@ void vec_update_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); + unsigned long b; vec_probe_vqs(info, tmp_map); bitmap_and(info->vq_map, info->vq_map, tmp_map, SVE_VQ_MAX); bitmap_or(info->vq_partial_map, info->vq_partial_map, tmp_map, SVE_VQ_MAX); + + b = find_first_bit(tmp_map, SVE_VQ_MAX); + if (__bit_to_vl(b) > info->max_cpu_vl) + info->max_cpu_vl = __bit_to_vl(b); } /* @@ -1069,10 +1080,23 @@ int vec_verify_vq_map(enum vec_type type) { struct vl_info *info = &vl_info[type]; DECLARE_BITMAP(tmp_map, SVE_VQ_MAX); - unsigned long b; + unsigned long b, max_vl; vec_probe_vqs(info, tmp_map); + /* + * Currently the maximum VL is only used for pKVM which + * doesn't allow late CPUs but we don't expect asymmetry and + * if we encounter any then future users will need handling so + * warn if we see anything. + */ + max_vl = __bit_to_vl(find_first_bit(tmp_map, SVE_VQ_MAX)); + if (max_vl > info->max_cpu_vl) { + pr_warn("%s: cpu%d: increases maximum VL to %u\n", + info->name, smp_processor_id(), max_vl); + info->max_cpu_vl = max_vl; + } + bitmap_complement(tmp_map, tmp_map, SVE_VQ_MAX); if (bitmap_intersects(tmp_map, info->vq_map, SVE_VQ_MAX)) { pr_warn("%s: cpu%d: Required vector length(s) missing\n", From patchwork Thu Jun 6 15:21:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13688680 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9B8E6C27C54 for ; Thu, 6 Jun 2024 15:32:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=99wSPIB9zNrNDHwfxnORc12e9bc3qe9aev2f3C4a/ts=; b=NlXrftMYqp+OqP jY+5lTLSFVRGS8viY81yeHmUU5BXVT/NLT8cjAHzA0kKQxvfM/f7tubmQjKdbr+LewQuUpvOgEHOw 0GupsXf/PxMNgRuNMiA7/RORfPrxBa37H2zfUIYMpEBOOZCXurr2GFLUlyw+Z0ncEssXotYII4yyq JU4pt8NhNyHWQvz8I9F4vNxjeGS9+ReZIBlxIwypLXz6KF3AI4tVTY6vuoZv5vXdYkUHpj3APDzz8 umOljnmca7ytunYPc0TuR8WshV9Pu7O1J1DgrocAWqJ2WzwCEKXPA8K2HNX3E09qXR22iSOWkA3oW FsMgRQLxRDd2I8flAfHg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFF68-0000000AJci-37md; Thu, 06 Jun 2024 15:32:16 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFF61-0000000AJXW-2ISi for linux-arm-kernel@lists.infradead.org; Thu, 06 Jun 2024 15:32:12 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id D503761D62; Thu, 6 Jun 2024 15:32:08 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8701EC32782; Thu, 6 Jun 2024 15:32:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717687928; bh=uej+Tf8xf9z3hwiLLHmVJLafNhINst5vjRm2ck0g7NI=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=g+LjCG5Dckh2ymqVHPJbXOurf4EebD7bdxByAad7JMaf34VU3t5ZktiW32JOaqfg/ 0yYaH5smz8iUiMs+iWkQtMMtXvshNLHpFGVEPS6LIVRhQ3nnjS/sW/wTiiqyEi0lld 7gnvSTO/E9YgV+f0unr5p3jFPkXvp1amS2UJLU3co53afuX5JVoHHCzhNADl5+nP/g H43FGZ2r3FPKkwLA0VqDXQT4CaNMQXzhFa/zvxwzAEmINSHTEapKXrZXTAN2Vr2gS4 pIHxaax2aREy4s1hZ979ZYTLSMNGgvEEDhrLpQPaJM1l+Al9yXhcd6KqxrNaRBVWaS yACtJB7XGpe7w== From: Mark Brown Date: Thu, 06 Jun 2024 16:21:45 +0100 Subject: [PATCH v2 3/4] KVM: arm64: Fix FFR offset calculation for pKVM host state save and restore MIME-Version: 1.0 Message-Id: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-3-c88f4eb4b14b@kernel.org> References: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-0-c88f4eb4b14b@kernel.org> In-Reply-To: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-0-c88f4eb4b14b@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=3038; i=broonie@kernel.org; h=from:subject:message-id; bh=uej+Tf8xf9z3hwiLLHmVJLafNhINst5vjRm2ck0g7NI=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBmYdZsz4r7UnVjNwEMykuqo5GUe570WUMT0pqnJdWE gToCCkWJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCZmHWbAAKCRAk1otyXVSH0Mw4B/ 943uDUyYIEx5pZu40qDu0DfW1U3V+HvNFfCFkiV2wuM9gthMVvxU3mdEKV8ycpgkIFfheiOOxXrdlN L45cC3KEK4JltkArIAf0igt7SRKdcQfhfq30AJwXcRsw+di5RlXoqQL07wQ5oVJAqnb4EbXYcpWfL/ WOiBpO7bSj6Jqxy+eWzqiz2gzzUrVCecNYkasdJhvxtHG1UyHfcKBTT9OAL97pVa9KdOzhVIhah4YX 4ccR2jQi1A40+WoQEQO8Uq7Z3HsKcFEQVWg7hYdxQ2GZtQHf1Y17yxigN/3JvEs7lWu9ndaxlp9j53 KXRUj+78kTnQLVr7UzqV33uUNcwqVR X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240606_083209_775867_3880FFBA X-CRM114-Status: GOOD ( 15.81 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org When saving and restoring the SVE state for the host we configure the hardware for the maximum VL it supports, but when calculating offsets in memory we use the maximum usable VL for the host. Since these two values may not be the same this may result in data corruption. We can just read the current VL from the hardware with an instruction so do that instead of a saved value, we need to correct the value and this makes the consistency obvious. Fixes: b5b9955617bc ("KVM: arm64: Eagerly restore host fpsimd/sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_hyp.h | 1 + arch/arm64/kvm/hyp/fpsimd.S | 5 +++++ arch/arm64/kvm/hyp/include/hyp/switch.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 2 +- 4 files changed, 8 insertions(+), 2 deletions(-) diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index b05bceca3385..7510383d78a6 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -113,6 +113,7 @@ void __fpsimd_save_state(struct user_fpsimd_state *fp_regs); void __fpsimd_restore_state(struct user_fpsimd_state *fp_regs); void __sve_save_state(void *sve_pffr, u32 *fpsr, int save_ffr); void __sve_restore_state(void *sve_pffr, u32 *fpsr, int restore_ffr); +int __sve_get_vl(void); u64 __guest_enter(struct kvm_vcpu *vcpu); diff --git a/arch/arm64/kvm/hyp/fpsimd.S b/arch/arm64/kvm/hyp/fpsimd.S index e950875e31ce..d272dbf36da8 100644 --- a/arch/arm64/kvm/hyp/fpsimd.S +++ b/arch/arm64/kvm/hyp/fpsimd.S @@ -31,3 +31,8 @@ SYM_FUNC_START(__sve_save_state) sve_save 0, x1, x2, 3 ret SYM_FUNC_END(__sve_save_state) + +SYM_FUNC_START(__sve_get_vl) + _sve_rdvl 0, 1 + ret +SYM_FUNC_END(__sve_get_vl) diff --git a/arch/arm64/kvm/hyp/include/hyp/switch.h b/arch/arm64/kvm/hyp/include/hyp/switch.h index 0c4de44534b7..06efcca765cc 100644 --- a/arch/arm64/kvm/hyp/include/hyp/switch.h +++ b/arch/arm64/kvm/hyp/include/hyp/switch.h @@ -327,7 +327,7 @@ static inline void __hyp_sve_save_host(void) sve_state->zcr_el1 = read_sysreg_el1(SYS_ZCR); write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_save_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_save_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); } diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index f43d845f3c4e..bd8f671e848c 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -49,7 +49,7 @@ static void __hyp_sve_restore_host(void) * supported by the system (or limited at EL3). */ write_sysreg_s(ZCR_ELx_LEN_MASK, SYS_ZCR_EL2); - __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(kvm_host_sve_max_vl), + __sve_restore_state(sve_state->sve_regs + sve_ffr_offset(__sve_get_vl()), &sve_state->fpsr, true); write_sysreg_el1(sve_state->zcr_el1, SYS_ZCR); From patchwork Thu Jun 6 15:21:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 13688679 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0F4CBC27C5F for ; Thu, 6 Jun 2024 15:32:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:In-Reply-To:References:Message-Id :MIME-Version:Subject:Date:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=fjsfPTGCTboU4WKfkV0z6cmcdSOmRMSn0niO3LkAPXM=; b=CI3SqEWidNAx1P NUevfn4hYXsFJkF6oCPrK/Fkj0d2nGHrBvEj97MK0+UzkcLoEa6aSlH/4cpxX77aL+In14FWVL1Cv PquY+OenmjQ9AJjVlK1o8Aih2XjvXKTQETCTXNU/byovvkXOdLOLhbz6hrVIiGOM/7qSrBqen30kg YCxkoG/B16IPixOtcTwRU1VLH0YhdUKGEU0+QFOJRCJXJq09ZKVx9aDswpDxtfb3re/SGdo3VKdyj u0b6JyIdRj9Tq3Z7VydC/uXf4gVKDZQzpvdHX4aFGO7YqaHfTKzTlmLAPGGdsK2GMq92GVttYVKhS X0XEz3kKEQYDKhcT6szw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFF69-0000000AJdc-20TY; Thu, 06 Jun 2024 15:32:17 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sFF64-0000000AJZY-0MXW for linux-arm-kernel@lists.infradead.org; Thu, 06 Jun 2024 15:32:13 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 739FE61B48; Thu, 6 Jun 2024 15:32:11 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1FAD8C32786; Thu, 6 Jun 2024 15:32:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1717687931; bh=I+qiD9f7AIjkeTzE7i3sCyRtTk9/3BCUmNFuB2K/uis=; h=From:Date:Subject:References:In-Reply-To:To:Cc:From; b=FzVOJ3lr0INEoHiotznW8SDe8HJ3POVh31xzThqGfrbaYnu8dnMoWcphOGtUdd8j3 fGEVK2Uv1kZIjzDdwPAex6g+Ql9BRD+LwJshrYKl3i3jZXUuxsHg5U/XhVB3osR7iQ RX7ifnqYXVtIYg9+u536CHTaf5j6ZKSMnuZG4S4PN+A+FHQCGvQ6kAiuZWuTbaGy6S k+Xt9SCm2S10ktQWQtJZ9IqBtXSZqUVHWLmYGQscqG0ikcZ7phbkQFbqVj1mlHv6Mf CIwrVVw47NMqcj/uqwqnsGjtSFmieEx66sD0HZJ9Ah6h0zdf+aRYcYuZ/HLlS+EbM9 vqD6jUEe3Jvkg== From: Mark Brown Date: Thu, 06 Jun 2024 16:21:46 +0100 Subject: [PATCH v2 4/4] KVM: arm64: Avoid underallocating storage for host SVE state MIME-Version: 1.0 Message-Id: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-4-c88f4eb4b14b@kernel.org> References: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-0-c88f4eb4b14b@kernel.org> In-Reply-To: <20240606-kvm-arm64-fix-pkvm-sve-vl-v2-0-c88f4eb4b14b@kernel.org> To: Catalin Marinas , Will Deacon , Marc Zyngier , Oliver Upton , James Morse , Suzuki K Poulose , Fuad Tabba Cc: linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, Mark Brown X-Mailer: b4 0.14-dev-d4707 X-Developer-Signature: v=1; a=openpgp-sha256; l=4775; i=broonie@kernel.org; h=from:subject:message-id; bh=I+qiD9f7AIjkeTzE7i3sCyRtTk9/3BCUmNFuB2K/uis=; b=owGbwMvMwMWocq27KDak/QLjabUkhrTEa7mvI+ar1U3OnsL6Nva70cesxpJ3GkoX2WMlhBQubvJj qjXrZDRmYWDkYpAVU2RZ+yxjVXq4xNb5j+a/ghnEygQyhYGLUwAmYv6d/RdTYQPTVbuJk9KDhWZwhp z/yPjEt+unpW90p8eq6RMcNn5O5p9m6qr2313lRf/bbzuvls0t/tQovtx/i5tsvPn/DVt1Wep1Opbm ZUubM8hrGe14vVOg6dW8qVpJCQ//fpz1Vq5LZ+sm3rsVgs7PHoh6aoReimdZxxpoNLX8W9OMkpyYpS YqlaZTbitwLd6UmaJ/36hutctiB69pEx/xTO0SsVZ9en3O0yxNsXvG2xldyu6Ilsu+uxie4FX+qDa8 8tqCY2niga5qwSUbm56waUeaXpC0/8UltC6I+bqu94/lVuk/hEMl13032qnSULPw6t5gkYJbM9SarK yLLv3ubG9/YMy6n3f2l08H2Hp1AQ== X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240606_083212_386252_7FC64E16 X-CRM114-Status: GOOD ( 16.00 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org We size the allocation for the host SVE state using the maximum VL shared by all CPUs in the host. As observed during review on an asymmetric system this may be less than the maximum VL supported on some of the CPUs. Since the pKVM hypervisor saves and restores the host state using the maximum VL for the current CPU this may lead to buffer overflows, fix this by changing pKVM to use the maximum VL for any CPU to size allocations and limit host configurations. Fixes: 66d5b53e20a6 ("KVM: arm64: Allocate memory mapped at hyp for host sve state in pKVM") Signed-off-by: Mark Brown --- arch/arm64/include/asm/kvm_host.h | 2 +- arch/arm64/include/asm/kvm_hyp.h | 2 +- arch/arm64/include/asm/kvm_pkvm.h | 2 +- arch/arm64/kvm/hyp/nvhe/hyp-main.c | 4 ++-- arch/arm64/kvm/hyp/nvhe/pkvm.c | 2 +- arch/arm64/kvm/reset.c | 6 +++--- 6 files changed, 9 insertions(+), 9 deletions(-) diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h index 36b8e97bf49e..a28fae10596f 100644 --- a/arch/arm64/include/asm/kvm_host.h +++ b/arch/arm64/include/asm/kvm_host.h @@ -76,7 +76,7 @@ static inline enum kvm_mode kvm_get_mode(void) { return KVM_MODE_NONE; }; DECLARE_STATIC_KEY_FALSE(userspace_irqchip_in_use); extern unsigned int __ro_after_init kvm_sve_max_vl; -extern unsigned int __ro_after_init kvm_host_sve_max_vl; +extern unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; int __init kvm_arm_init_sve(void); u32 __attribute_const__ kvm_target_cpu(void); diff --git a/arch/arm64/include/asm/kvm_hyp.h b/arch/arm64/include/asm/kvm_hyp.h index 7510383d78a6..47426df69875 100644 --- a/arch/arm64/include/asm/kvm_hyp.h +++ b/arch/arm64/include/asm/kvm_hyp.h @@ -144,6 +144,6 @@ extern u64 kvm_nvhe_sym(id_aa64smfr0_el1_sys_val); extern unsigned long kvm_nvhe_sym(__icache_flags); extern unsigned int kvm_nvhe_sym(kvm_arm_vmid_bits); -extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_vl); +extern unsigned int kvm_nvhe_sym(kvm_host_sve_max_cpu_vl); #endif /* __ARM64_KVM_HYP_H__ */ diff --git a/arch/arm64/include/asm/kvm_pkvm.h b/arch/arm64/include/asm/kvm_pkvm.h index cd56acd9a842..6fc0cf42fca3 100644 --- a/arch/arm64/include/asm/kvm_pkvm.h +++ b/arch/arm64/include/asm/kvm_pkvm.h @@ -134,7 +134,7 @@ static inline size_t pkvm_host_sve_state_size(void) return 0; return size_add(sizeof(struct cpu_sve_state), - SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_vl))); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(kvm_host_sve_max_cpu_vl))); } #endif /* __ARM64_KVM_PKVM_H__ */ diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-main.c b/arch/arm64/kvm/hyp/nvhe/hyp-main.c index bd8f671e848c..d232775b72c9 100644 --- a/arch/arm64/kvm/hyp/nvhe/hyp-main.c +++ b/arch/arm64/kvm/hyp/nvhe/hyp-main.c @@ -90,8 +90,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu) hyp_vcpu->vcpu.arch.ctxt = host_vcpu->arch.ctxt; hyp_vcpu->vcpu.arch.sve_state = kern_hyp_va(host_vcpu->arch.sve_state); - /* Limit guest vector length to the maximum supported by the host. */ - hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_vl); + /* Limit guest vector length to the maximum supported by any CPU. */ + hyp_vcpu->vcpu.arch.sve_max_vl = min(host_vcpu->arch.sve_max_vl, kvm_host_sve_max_cpu_vl); hyp_vcpu->vcpu.arch.hw_mmu = host_vcpu->arch.hw_mmu; diff --git a/arch/arm64/kvm/hyp/nvhe/pkvm.c b/arch/arm64/kvm/hyp/nvhe/pkvm.c index 95cf18574251..08e825de09d1 100644 --- a/arch/arm64/kvm/hyp/nvhe/pkvm.c +++ b/arch/arm64/kvm/hyp/nvhe/pkvm.c @@ -18,7 +18,7 @@ unsigned long __icache_flags; /* Used by kvm_get_vttbr(). */ unsigned int kvm_arm_vmid_bits; -unsigned int kvm_host_sve_max_vl; +unsigned int kvm_host_sve_max_cpu_vl; /* * Set trap register values based on features in ID_AA64PFR0. diff --git a/arch/arm64/kvm/reset.c b/arch/arm64/kvm/reset.c index 3fc8ca164dbe..59cccb477cf3 100644 --- a/arch/arm64/kvm/reset.c +++ b/arch/arm64/kvm/reset.c @@ -32,7 +32,7 @@ /* Maximum phys_shift supported for any VM on this host */ static u32 __ro_after_init kvm_ipa_limit; -unsigned int __ro_after_init kvm_host_sve_max_vl; +unsigned int __ro_after_init kvm_host_sve_max_cpu_vl; /* * ARMv8 Reset Values @@ -52,8 +52,8 @@ int __init kvm_arm_init_sve(void) { if (system_supports_sve()) { kvm_sve_max_vl = sve_max_virtualisable_vl(); - kvm_host_sve_max_vl = sve_max_vl(); - kvm_nvhe_sym(kvm_host_sve_max_vl) = kvm_host_sve_max_vl; + kvm_host_sve_max_cpu_vl = sve_max_cpu_vl(); + kvm_nvhe_sym(kvm_host_sve_max_cpu_vl) = kvm_host_sve_max_cpu_vl; /* * The get_sve_reg()/set_sve_reg() ioctl interface will need