From patchwork Sat Oct 19 09:55:17 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 11200057 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 9183317EE for ; Sat, 19 Oct 2019 09:55:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 70BB6222BD for ; Sat, 19 Oct 2019 09:55:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478948; bh=0ydo3zWocyZ2R1TjMk6U6Pm/YQU4lvknC5Dj+8pFO8I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=lO/LnEXjiVyMUJChhnGPOMzlFwjaav3/JypmDV3pEjS2OHbXX1okAWwB6kndv42dQ IDZGZ51S4Jp9qXXC9uGk0n70TX+DSttvR0aKehrgHD493q1XlZy5M+NDWZEiY+h/wA K12m12h8XUg5UNRVz5XJSPYytQMG3ymlERvJkGGc= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725802AbfJSJza (ORCPT ); Sat, 19 Oct 2019 05:55:30 -0400 Received: from mail.kernel.org ([198.145.29.99]:53428 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725283AbfJSJza (ORCPT ); Sat, 19 Oct 2019 05:55:30 -0400 Received: from big-swifty.lan (78.163-31-62.static.virginmediabusiness.co.uk [62.31.163.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id C2EA4222CC; Sat, 19 Oct 2019 09:55:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478930; bh=0ydo3zWocyZ2R1TjMk6U6Pm/YQU4lvknC5Dj+8pFO8I=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Gywd3mq6ho0RcqzjTr5TjpZFv6g7TQInCq20wYBkqiI+t4T9pwUgtzUGKXpApuqFY 4Y59074Z+gaoRT8b2na8QOPVVUmCPkaiRHwgXHTjppUePfaJK6Zjdc2Ny9ft6iaQRb q4TLIjgk+46OcepRBR4l7Uh57+53Ad4lQJQMg6ao= From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org Cc: Mark Rutland , Suzuki K Poulose , Catalin Marinas , James Morse , Will Deacon , Julien Thierry Subject: [PATCH v2 1/5] arm64: Add ARM64_WORKAROUND_1319367 for all A57 and A72 versions Date: Sat, 19 Oct 2019 10:55:17 +0100 Message-Id: <20191019095521.31722-2-maz@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191019095521.31722-1-maz@kernel.org> References: <20191019095521.31722-1-maz@kernel.org> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Rework the EL2 vector hardening that is only selected for A57 and A72 so that the table can also be used for ARM64_WORKAROUND_1319367. Acked-by: Catalin Marinas Reviewed-by: Suzuki K Poulose Signed-off-by: Marc Zyngier --- arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/kernel/cpu_errata.c | 13 ++++++++++--- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index f19fe4b9acc4..277e37b2a513 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -52,7 +52,8 @@ #define ARM64_HAS_IRQ_PRIO_MASKING 42 #define ARM64_HAS_DCPODP 43 #define ARM64_WORKAROUND_1463225 44 +#define ARM64_WORKAROUND_1319367 45 -#define ARM64_NCAPS 45 +#define ARM64_NCAPS 46 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index f593f4cffc0d..a7afa6d4a58f 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -623,9 +623,9 @@ check_branch_predictor(const struct arm64_cpu_capabilities *entry, int scope) return (need_wa > 0); } -#ifdef CONFIG_HARDEN_EL2_VECTORS +#if defined(CONFIG_HARDEN_EL2_VECTORS) || defined(CONFIG_ARM64_ERRATUM_1319367) -static const struct midr_range arm64_harden_el2_vectors[] = { +static const struct midr_range ca57_a72[] = { MIDR_ALL_VERSIONS(MIDR_CORTEX_A57), MIDR_ALL_VERSIONS(MIDR_CORTEX_A72), {}, @@ -819,7 +819,7 @@ const struct arm64_cpu_capabilities arm64_errata[] = { { .desc = "EL2 vector hardening", .capability = ARM64_HARDEN_EL2_VECTORS, - ERRATA_MIDR_RANGE_LIST(arm64_harden_el2_vectors), + ERRATA_MIDR_RANGE_LIST(ca57_a72), }, #endif { @@ -851,6 +851,13 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .type = ARM64_CPUCAP_LOCAL_CPU_ERRATUM, .matches = has_cortex_a76_erratum_1463225, }, +#endif +#ifdef CONFIG_ARM64_ERRATUM_1319367 + { + .desc = "ARM erratum 1319367", + .capability = ARM64_WORKAROUND_1319367, + ERRATA_MIDR_RANGE_LIST(ca57_a72), + }, #endif { } From patchwork Sat Oct 19 09:55:18 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 11200059 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C54B51390 for ; Sat, 19 Oct 2019 09:55:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 9B1D6222BD for ; Sat, 19 Oct 2019 09:55:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478948; bh=C+wxLu/WnnwZ3cqp2GHFnfaQZgrXjj31+JCX0m8MeoM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=bbrTaK/Mc65TWPMQn5CW7vbI6WTu5cSbeSC3zOhTjLubFV+K8RY8GylC/KsZTlB2w Orpj5RlufaHfPf2oIxGbDi3N4jifqF5LSPT9XL+qz00qiwnXnErx8a1OZFfHyUcl5q ZQDLpWVES1KusacGeHPZEFpxSm41tVLk6vvPgdBI= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725933AbfJSJzd (ORCPT ); Sat, 19 Oct 2019 05:55:33 -0400 Received: from mail.kernel.org ([198.145.29.99]:53472 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725823AbfJSJzd (ORCPT ); Sat, 19 Oct 2019 05:55:33 -0400 Received: from big-swifty.lan (78.163-31-62.static.virginmediabusiness.co.uk [62.31.163.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id A153E222C2; Sat, 19 Oct 2019 09:55:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478932; bh=C+wxLu/WnnwZ3cqp2GHFnfaQZgrXjj31+JCX0m8MeoM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=mYqx+7UMtpLMaE/KOY3jsL1gUUeMf3s+YgsbRmwJeHNiJhsZhiQnsl7bbhvmNKWrg s3seIIPbTqbICqVFLO61SkWTYsh7XposMSxyYa3dW0GZ6V0kOrF7u+0yAl2g0BqYPA nrqcyHiWEkARmUq3m6gv36fmahQT1YvI0NELSY5Y= From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org Cc: Mark Rutland , Suzuki K Poulose , Catalin Marinas , James Morse , Will Deacon , Julien Thierry Subject: [PATCH v2 2/5] arm64: KVM: Reorder system register restoration and stage-2 activation Date: Sat, 19 Oct 2019 10:55:18 +0100 Message-Id: <20191019095521.31722-3-maz@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191019095521.31722-1-maz@kernel.org> References: <20191019095521.31722-1-maz@kernel.org> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org In order to prepare for handling erratum 1319367, we need to make sure that all system registers (and most importantly the registers configuring the virtual memory) are set before we enable stage-2 translation. This results in a minor reorganisation of the load sequence, without any functional change. Signed-off-by: Marc Zyngier Reviewed-by: James Morse --- arch/arm64/kvm/hyp/switch.c | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-) diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 3d3815020e36..69e10b29cbd0 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -605,18 +605,23 @@ int __hyp_text __kvm_vcpu_run_nvhe(struct kvm_vcpu *vcpu) __sysreg_save_state_nvhe(host_ctxt); - __activate_vm(kern_hyp_va(vcpu->kvm)); - __activate_traps(vcpu); - - __hyp_vgic_restore_state(vcpu); - __timer_enable_traps(vcpu); - /* * We must restore the 32-bit state before the sysregs, thanks * to erratum #852523 (Cortex-A57) or #853709 (Cortex-A72). + * + * Also, and in order to be able to deal with erratum #1319537 (A57) + * and #1319367 (A72), we must ensure that all VM-related sysreg are + * restored before we enable S2 translation. */ __sysreg32_restore_state(vcpu); __sysreg_restore_state_nvhe(guest_ctxt); + + __activate_vm(kern_hyp_va(vcpu->kvm)); + __activate_traps(vcpu); + + __hyp_vgic_restore_state(vcpu); + __timer_enable_traps(vcpu); + __debug_switch_to_guest(vcpu); __set_guest_arch_workaround_state(vcpu); From patchwork Sat Oct 19 09:55:19 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 11200061 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id EF51E6A61 for ; Sat, 19 Oct 2019 09:55:48 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id CEEFA222CC for ; Sat, 19 Oct 2019 09:55:48 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478948; bh=cBYrKAV4O8BFISadDqGvhB8cGTVx7az9VNh8naTENqA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=lZ+Gs2ORQyE7TTgiRBtdQ8Hb5L4H4Wzp6qmmQKso2Nk8/DnAZocR6Gbit9DIehSHn 4xuI1zHVr+iDlq+K0lcny0ER44D4JoIeWZpqvA29lqKZG9MLJ1xOernY5BDNg4UsW9 6S6vyYCX5opSMdMAdryCm03VqBvaedpKPt937J+Y= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725895AbfJSJzg (ORCPT ); Sat, 19 Oct 2019 05:55:36 -0400 Received: from mail.kernel.org ([198.145.29.99]:53516 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725856AbfJSJzg (ORCPT ); Sat, 19 Oct 2019 05:55:36 -0400 Received: from big-swifty.lan (78.163-31-62.static.virginmediabusiness.co.uk [62.31.163.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 81BBB222CD; Sat, 19 Oct 2019 09:55:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478935; bh=cBYrKAV4O8BFISadDqGvhB8cGTVx7az9VNh8naTENqA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=lB6du6o6np7KyZJ0ZdiVBJc9kjZJUsnjLjB4gnszQ5jtgdvg4uPUNibAa435nV8C6 KDnpGhbrgtbAEJK+OctIrIqOFV4ZYqWzVEsy/Ajq711lB2n0rwkpZXYhVwP80gz3F+ XzLTPv/DtGRO6bZVOEfDeofgYow+VVY8bnoDAHjA= From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org Cc: Mark Rutland , Suzuki K Poulose , Catalin Marinas , James Morse , Will Deacon , Julien Thierry Subject: [PATCH v2 3/5] arm64: KVM: Disable EL1 PTW when invalidating S2 TLBs Date: Sat, 19 Oct 2019 10:55:19 +0100 Message-Id: <20191019095521.31722-4-maz@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191019095521.31722-1-maz@kernel.org> References: <20191019095521.31722-1-maz@kernel.org> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When erratum 1319367 is being worked around, special care must be taken not to allow the page table walker to populate TLBs while we have the stage-2 translation enabled (which would otherwise result in a bizare mix of the host S1 and the guest S2). We enforce this by setting TCR_EL1.EPD{0,1} before restoring the S2 configuration, and clear the same bits after having disabled S2. Signed-off-by: Marc Zyngier Reviewed-by: James Morse --- arch/arm64/kvm/hyp/tlb.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+) diff --git a/arch/arm64/kvm/hyp/tlb.c b/arch/arm64/kvm/hyp/tlb.c index eb0efc5557f3..c2bc17ca6430 100644 --- a/arch/arm64/kvm/hyp/tlb.c +++ b/arch/arm64/kvm/hyp/tlb.c @@ -63,6 +63,22 @@ static void __hyp_text __tlb_switch_to_guest_vhe(struct kvm *kvm, static void __hyp_text __tlb_switch_to_guest_nvhe(struct kvm *kvm, struct tlb_inv_context *cxt) { + if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + u64 val; + + /* + * For CPUs that are affected by ARM 1319367, we need to + * avoid a host Stage-1 walk while we have the guest's + * VMID set in the VTTBR in order to invalidate TLBs. + * We're guaranteed that the S1 MMU is enabled, so we can + * simply set the EPD bits to avoid any further TLB fill. + */ + val = cxt->tcr = read_sysreg_el1(SYS_TCR); + val |= TCR_EPD1_MASK | TCR_EPD0_MASK; + write_sysreg_el1(val, SYS_TCR); + isb(); + } + __load_guest_stage2(kvm); isb(); } @@ -100,6 +116,13 @@ static void __hyp_text __tlb_switch_to_host_nvhe(struct kvm *kvm, struct tlb_inv_context *cxt) { write_sysreg(0, vttbr_el2); + + if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + /* Ensure write of the host VMID */ + isb(); + /* Restore the host's TCR_EL1 */ + write_sysreg_el1(cxt->tcr, SYS_TCR); + } } static void __hyp_text __tlb_switch_to_host(struct kvm *kvm, From patchwork Sat Oct 19 09:55:20 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 11200063 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 251A2112B for ; Sat, 19 Oct 2019 09:55:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 049FE222C2 for ; Sat, 19 Oct 2019 09:55:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478949; bh=/BHwWAb8kea1kN6CGDZVKU3zOASIcTMHmEllourGtwM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=oqc4NnpF9xmCV4ktWOIyIqrQ7BR1PRv34bqWufdBwBowQEqZG+xP1Vr9T/NFvs7at z75pARcayODMa2FSXAHxkW4HDQ6U8vDy3WGyC1QrYETm4TX46mT5gP8tu4Gv1uPPw7 i+l1EJ+9kkRNMgbwtFwS5y2ykMR6L2/b18MxzZ68= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725948AbfJSJzk (ORCPT ); Sat, 19 Oct 2019 05:55:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:53586 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725856AbfJSJzj (ORCPT ); Sat, 19 Oct 2019 05:55:39 -0400 Received: from big-swifty.lan (78.163-31-62.static.virginmediabusiness.co.uk [62.31.163.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 6253F222D2; Sat, 19 Oct 2019 09:55:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478938; bh=/BHwWAb8kea1kN6CGDZVKU3zOASIcTMHmEllourGtwM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=Nt2VeCabzxc0pbTpxeH9OCZsa/doEDq5b0S3kx1dh0zgew5lOfs0BZSS10XrMJGxF ATKHsPCMXuH64Z+pzkieG6fJbLo0A/4ZqJ2mp2HtoB4jQL2UqhS5kgpzDnWKhh0a7t E3o9P/0JxOICBWnkSwABYA1U25SPUUnH7DRQ+jno= From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org Cc: Mark Rutland , Suzuki K Poulose , Catalin Marinas , James Morse , Will Deacon , Julien Thierry Subject: [PATCH v2 4/5] arm64: KVM: Prevent speculative S1 PTW when restoring vcpu context Date: Sat, 19 Oct 2019 10:55:20 +0100 Message-Id: <20191019095521.31722-5-maz@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191019095521.31722-1-maz@kernel.org> References: <20191019095521.31722-1-maz@kernel.org> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org When handling erratum 1319367, we must ensure that the page table walker cannot parse the S1 page tables while the guest is in an inconsistent state. This is done as follows: On guest entry: - TCR_EL1.EPD{0,1} are set, ensuring that no PTW can occur - all system registers are restored, except for TCR_EL1 and SCTLR_EL1 - stage-2 is restored - SCTLR_EL1 and TCR_EL1 are restored On guest exit: - SCTLR_EL1.M and TCR_EL1.EPD{0,1} are set, ensuring that no PTW can occur - stage-2 is disabled - All host system registers are restored Signed-off-by: Marc Zyngier Reviewed-by: James Morse --- arch/arm64/kvm/hyp/switch.c | 31 ++++++++++++++++++++++++++++++ arch/arm64/kvm/hyp/sysreg-sr.c | 35 ++++++++++++++++++++++++++++++++-- 2 files changed, 64 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kvm/hyp/switch.c b/arch/arm64/kvm/hyp/switch.c index 69e10b29cbd0..5765b17c38c7 100644 --- a/arch/arm64/kvm/hyp/switch.c +++ b/arch/arm64/kvm/hyp/switch.c @@ -118,6 +118,20 @@ static void __hyp_text __activate_traps_nvhe(struct kvm_vcpu *vcpu) } write_sysreg(val, cptr_el2); + + if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + struct kvm_cpu_context *ctxt = &vcpu->arch.ctxt; + + isb(); + /* + * At this stage, and thanks to the above isb(), S2 is + * configured and enabled. We can now restore the guest's S1 + * configuration: SCTLR, and only then TCR. + */ + write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR); + isb(); + write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR); + } } static void __hyp_text __activate_traps(struct kvm_vcpu *vcpu) @@ -156,6 +170,23 @@ static void __hyp_text __deactivate_traps_nvhe(void) { u64 mdcr_el2 = read_sysreg(mdcr_el2); + if (cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + u64 val; + + /* + * Set the TCR and SCTLR registers in the exact opposite + * sequence as __activate_traps_nvhe (first prevent walks, + * then force the MMU on). A generous sprinkling of isb() + * ensure that things happen in this exact order. + */ + val = read_sysreg_el1(SYS_TCR); + write_sysreg_el1(val | TCR_EPD1_MASK | TCR_EPD0_MASK, SYS_TCR); + isb(); + val = read_sysreg_el1(SYS_SCTLR); + write_sysreg_el1(val | SCTLR_ELx_M, SYS_SCTLR); + isb(); + } + __deactivate_traps_common(); mdcr_el2 &= MDCR_EL2_HPMN_MASK; diff --git a/arch/arm64/kvm/hyp/sysreg-sr.c b/arch/arm64/kvm/hyp/sysreg-sr.c index 7ddbc849b580..fb97547bfa79 100644 --- a/arch/arm64/kvm/hyp/sysreg-sr.c +++ b/arch/arm64/kvm/hyp/sysreg-sr.c @@ -117,12 +117,26 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) { write_sysreg(ctxt->sys_regs[MPIDR_EL1], vmpidr_el2); write_sysreg(ctxt->sys_regs[CSSELR_EL1], csselr_el1); - write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR); + + if (!cpus_have_const_cap(ARM64_WORKAROUND_1319367)) { + write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR); + write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR); + } else if (!ctxt->__hyp_running_vcpu) { + /* + * Must only be done for guest registers, hence the context + * test. We'recoming from the host, so SCTLR.M is already + * set. Pairs with __activate_traps_nvhe(). + */ + write_sysreg_el1((ctxt->sys_regs[TCR_EL1] | + TCR_EPD1_MASK | TCR_EPD0_MASK), + SYS_TCR); + isb(); + } + write_sysreg(ctxt->sys_regs[ACTLR_EL1], actlr_el1); write_sysreg_el1(ctxt->sys_regs[CPACR_EL1], SYS_CPACR); write_sysreg_el1(ctxt->sys_regs[TTBR0_EL1], SYS_TTBR0); write_sysreg_el1(ctxt->sys_regs[TTBR1_EL1], SYS_TTBR1); - write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR); write_sysreg_el1(ctxt->sys_regs[ESR_EL1], SYS_ESR); write_sysreg_el1(ctxt->sys_regs[AFSR0_EL1], SYS_AFSR0); write_sysreg_el1(ctxt->sys_regs[AFSR1_EL1], SYS_AFSR1); @@ -135,6 +149,23 @@ static void __hyp_text __sysreg_restore_el1_state(struct kvm_cpu_context *ctxt) write_sysreg(ctxt->sys_regs[PAR_EL1], par_el1); write_sysreg(ctxt->sys_regs[TPIDR_EL1], tpidr_el1); + if (cpus_have_const_cap(ARM64_WORKAROUND_1319367) && + ctxt->__hyp_running_vcpu) { + /* + * Must only be done for host registers, hence the context + * test. Pairs with __deactivate_traps_nvhe(). + */ + isb(); + /* + * At this stage, and thanks to the above isb(), S2 is + * deconfigured and disabled. We can now restore the host's + * S1 configuration: SCTLR, and only then TCR. + */ + write_sysreg_el1(ctxt->sys_regs[SCTLR_EL1], SYS_SCTLR); + isb(); + write_sysreg_el1(ctxt->sys_regs[TCR_EL1], SYS_TCR); + } + write_sysreg(ctxt->gp_regs.sp_el1, sp_el1); write_sysreg_el1(ctxt->gp_regs.elr_el1, SYS_ELR); write_sysreg_el1(ctxt->gp_regs.spsr[KVM_SPSR_EL1],SYS_SPSR); From patchwork Sat Oct 19 09:55:21 2019 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Marc Zyngier X-Patchwork-Id: 11200065 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 4EFA292DD for ; Sat, 19 Oct 2019 09:55:49 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 2EB0F222BD for ; Sat, 19 Oct 2019 09:55:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478949; bh=NIhQTmaSfZdOFxlsYacZmWRGnEWsqvAAqOXU5n1fBjc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:List-ID:From; b=iPmeKBHgtYLZcoHotdaflTqKyQCGd1CWM+O/RiFTaOp1OZKeY6WC9RW+W5UpLVmKp iIdP+BNvpuH1DalFEfxIMy7xUq5UiI5Ya/tt5KuT5KuCFNq+J9ZLlUrXL+/beXsRgI /HFb7GbwWBP/yIj0Vurfjqe+q78YMK/NtQcLFcxo= Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725992AbfJSJzm (ORCPT ); Sat, 19 Oct 2019 05:55:42 -0400 Received: from mail.kernel.org ([198.145.29.99]:53632 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725937AbfJSJzm (ORCPT ); Sat, 19 Oct 2019 05:55:42 -0400 Received: from big-swifty.lan (78.163-31-62.static.virginmediabusiness.co.uk [62.31.163.78]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 4010F222C2; Sat, 19 Oct 2019 09:55:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1571478941; bh=NIhQTmaSfZdOFxlsYacZmWRGnEWsqvAAqOXU5n1fBjc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=2Ke1oSPhBBMSE/PKuHl0F96K39z/y/L2tu7iwYUo/hIR0eOS9mV/NP7807EKmNR7j hcCy70w0ARPX++VLoxQv+SMqs5Gk/oECK0KwHPRMSI3SZ2C+hhJGJZXimwyPPR86tS yGoynaIh12/EeSYLwt8Nzb6h2ZD/8fku07bANjmA= From: Marc Zyngier To: linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org Cc: Mark Rutland , Suzuki K Poulose , Catalin Marinas , James Morse , Will Deacon , Julien Thierry Subject: [PATCH v2 5/5] arm64: Enable and document ARM errata 1319367 and 1319537 Date: Sat, 19 Oct 2019 10:55:21 +0100 Message-Id: <20191019095521.31722-6-maz@kernel.org> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20191019095521.31722-1-maz@kernel.org> References: <20191019095521.31722-1-maz@kernel.org> MIME-Version: 1.0 Sender: kvm-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org Now that everything is in place, let's get the ball rolling by allowing the corresponding config option to be selected. Also add the required information to silicon_errata.rst. Acked-by: Catalin Marinas Signed-off-by: Marc Zyngier --- Documentation/arm64/silicon-errata.rst | 4 ++++ arch/arm64/Kconfig | 10 ++++++++++ 2 files changed, 14 insertions(+) diff --git a/Documentation/arm64/silicon-errata.rst b/Documentation/arm64/silicon-errata.rst index 17ea3fecddaa..0808be134fce 100644 --- a/Documentation/arm64/silicon-errata.rst +++ b/Documentation/arm64/silicon-errata.rst @@ -70,8 +70,12 @@ stable kernels. +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A57 | #834220 | ARM64_ERRATUM_834220 | +----------------+-----------------+-----------------+-----------------------------+ +| ARM | Cortex-A57 | #1319537 | ARM64_ERRATUM_1319367 | ++----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A72 | #853709 | N/A | +----------------+-----------------+-----------------+-----------------------------+ +| ARM | Cortex-A72 | #1319367 | ARM64_ERRATUM_1319367 | ++----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A73 | #858921 | ARM64_ERRATUM_858921 | +----------------+-----------------+-----------------+-----------------------------+ | ARM | Cortex-A55 | #1024718 | ARM64_ERRATUM_1024718 | diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 950a56b71ff0..b2877ed09307 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -538,6 +538,16 @@ config ARM64_ERRATUM_1286807 invalidated has been observed by other observers. The workaround repeats the TLBI+DSB operation. +config ARM64_ERRATUM_1319367 + bool "Cortex-A57/A72: Speculative AT instruction using out-of-context translation regime could cause subsequent request to generate an incorrect translation" + default y + help + This option adds work arounds for ARM Cortex-A57 erratum 1319537 + and A72 erratum 1319367 + + Cortex-A57 and A72 cores could end-up with corrupted TLBs by + speculating an AT instruction during a guest context switch. + If unsure, say Y. config ARM64_ERRATUM_1463225