From patchwork Tue Oct 19 17:22:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Brown X-Patchwork-Id: 12570737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 0A639C433F5 for ; Tue, 19 Oct 2021 17:25:08 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id E7BEF61212 for ; Tue, 19 Oct 2021 17:25:07 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231226AbhJSR1U (ORCPT ); Tue, 19 Oct 2021 13:27:20 -0400 Received: from mail.kernel.org ([198.145.29.99]:55370 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230158AbhJSR1T (ORCPT ); Tue, 19 Oct 2021 13:27:19 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 75227610E5; Tue, 19 Oct 2021 17:25:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1634664307; bh=zwCbiXqY1k08D4UkqONMPZ6kZewacYOMJKHBbTc2W0g=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=O+bPt1g9gFodUJT+lWGSoMxiPJG92r4SZAK5SNNShQWeWTs1xnFx0GBdchAtkDYAV 9WAC3Haj+BYdjJYyADU0eIQM/lqmWLbep1Nh72DomglEkE+3Xee7cJWr8WbC2VscaC rQlyOXbtVTAgtWjm7AnRVH4uG+v9XcfLDhXOwvT/W4bA2+KatzsxMbWNKJj2wtJVX6 qRT8N0CXGBH11XzjtJxmONk+iAMJYrWq4O+U9wJ0HmLsqVmp5sbvv1TwPO5aXpez4E oq3gclq/cpDQ2R2zpt6euaxDcbSfsSTtll0lD4exg+8ZK5zo/0rE28QZMGR3detETj Enw7U3WL5ZPUA== From: Mark Brown To: Catalin Marinas , Will Deacon , Shuah Khan , Shuah Khan Cc: Alan Hayward , Luis Machado , Salil Akerkar , Basant Kumar Dwivedi , Szabolcs Nagy , linux-arm-kernel@lists.infradead.org, linux-kselftest@vger.kernel.org, Mark Brown Subject: [PATCH v3 33/42] arm64/sme: Save and restore streaming mode over EFI runtime calls Date: Tue, 19 Oct 2021 18:22:38 +0100 Message-Id: <20211019172247.3045838-34-broonie@kernel.org> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211019172247.3045838-1-broonie@kernel.org> References: <20211019172247.3045838-1-broonie@kernel.org> MIME-Version: 1.0 X-Developer-Signature: v=1; a=openpgp-sha256; l=3623; h=from:subject; bh=zwCbiXqY1k08D4UkqONMPZ6kZewacYOMJKHBbTc2W0g=; b=owEBbQGS/pANAwAKASTWi3JdVIfQAcsmYgBhbv7dhrJ4LJIl/VpLF/DeSTxw6/9nApnLsBv1rRCS Y5r1OOGJATMEAAEKAB0WIQSt5miqZ1cYtZ/in+ok1otyXVSH0AUCYW7+3QAKCRAk1otyXVSH0BjKB/ 9x/+Gs0dNZe6owtgQHj8ygGDYHadus3h+D5+Poo2X/i1dVIm17D33H6Cp0agANtyXSf72X1n7Z+h+/ +fgxeysbfLhoWbx3mZ1AistYJ3PDk34PhwoCH67XrUEYd2h8/jZ1QBS9G5H+ljAwte+WH3yUZa2Uub 2djJgRhWGQ9P3vruIuyx609ht5R88D5iFuUIFT0cxzjZHi9yHT44YNRaqmft+TvuBxvKTpjINTkTWj iReA+F3bHLehJnEkUXpEE1RQotyEnrfRN2PInMuvrCHEwGTuLaBZxk4KJPm1D498r2Op5NYz5ifq/q Wwz6c10UPP7K+RKyb0w87vjkPpoUSR X-Developer-Key: i=broonie@kernel.org; a=openpgp; fpr=3F2568AAC26998F9E813A1C5C3F436CA30F5D8EB Precedence: bulk List-ID: X-Mailing-List: linux-kselftest@vger.kernel.org When saving and restoring the floating point state over an EFI runtime call ensure that we handle streaming mode, only handling FFR if we are not in streaming mode and ensuring that we are in normal mode over the call into runtime services. We currently assume that ZA will not be modified by runtime services, the specification is not yet finalised so this may need updating if that changes. Signed-off-by: Mark Brown --- arch/arm64/kernel/fpsimd.c | 47 +++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kernel/fpsimd.c b/arch/arm64/kernel/fpsimd.c index 725343c1bc65..bbb09efb0a2a 100644 --- a/arch/arm64/kernel/fpsimd.c +++ b/arch/arm64/kernel/fpsimd.c @@ -1049,21 +1049,25 @@ int vec_verify_vq_map(enum vec_type type) static void __init sve_efi_setup(void) { - struct vl_info *info = &vl_info[ARM64_VEC_SVE]; + int max_vl = 0; + int i; if (!IS_ENABLED(CONFIG_EFI)) return; + for (i = 0; i < ARRAY_SIZE(vl_info); i++) + max_vl = max(vl_info[i].max_vl, max_vl); + /* * alloc_percpu() warns and prints a backtrace if this goes wrong. * This is evidence of a crippled system and we are returning void, * so no attempt is made to handle this situation here. */ - if (!sve_vl_valid(info->max_vl)) + if (!sve_vl_valid(max_vl)) goto fail; efi_sve_state = __alloc_percpu( - SVE_SIG_REGS_SIZE(sve_vq_from_vl(info->max_vl)), SVE_VQ_BYTES); + SVE_SIG_REGS_SIZE(sve_vq_from_vl(max_vl)), SVE_VQ_BYTES); if (!efi_sve_state) goto fail; @@ -1833,6 +1837,7 @@ EXPORT_SYMBOL(kernel_neon_end); static DEFINE_PER_CPU(struct user_fpsimd_state, efi_fpsimd_state); static DEFINE_PER_CPU(bool, efi_fpsimd_state_used); static DEFINE_PER_CPU(bool, efi_sve_state_used); +static DEFINE_PER_CPU(bool, efi_sm_state); /* * EFI runtime services support functions @@ -1867,12 +1872,28 @@ void __efi_fpsimd_begin(void) */ if (system_supports_sve() && likely(efi_sve_state)) { char *sve_state = this_cpu_ptr(efi_sve_state); + bool ffr = true; + u64 svcr; __this_cpu_write(efi_sve_state_used, true); + /* If we are in streaming mode don't touch FFR */ + if (system_supports_sme()) { + svcr = read_sysreg_s(SYS_SVCR_EL0); + + ffr = svcr & SYS_SVCR_EL0_SM_MASK; + + __this_cpu_write(efi_sm_state, ffr); + } + sve_save_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true); + ffr); + + if (system_supports_sme()) + sysreg_clear_set_s(SYS_SVCR_EL0, + SYS_SVCR_EL0_SM_MASK, 0); + } else { fpsimd_save_state(this_cpu_ptr(&efi_fpsimd_state)); } @@ -1895,11 +1916,25 @@ void __efi_fpsimd_end(void) if (system_supports_sve() && likely(__this_cpu_read(efi_sve_state_used))) { char const *sve_state = this_cpu_ptr(efi_sve_state); + bool ffr = true; + + /* + * Restore streaming mode; EFI calls are + * normal function calls so should not return in + * streaming mode. + */ + if (system_supports_sme()) { + if (__this_cpu_read(efi_sm_state)) { + sysreg_clear_set_s(SYS_SVCR_EL0, + 0, + SYS_SVCR_EL0_SM_MASK); + ffr = false; + } + } - sve_set_vq(sve_vq_from_vl(sve_get_vl()) - 1); sve_load_state(sve_state + sve_ffr_offset(sve_max_vl()), &this_cpu_ptr(&efi_fpsimd_state)->fpsr, - true); + ffr); __this_cpu_write(efi_sve_state_used, false); } else {