From patchwork Wed Feb 8 00:35:16 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Christopher Covington X-Patchwork-Id: 9561553 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork.web.codeaurora.org (Postfix) with ESMTP id A7CF26047A for ; Wed, 8 Feb 2017 00:35:56 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 99A7328434 for ; Wed, 8 Feb 2017 00:35:56 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 8DA9228487; Wed, 8 Feb 2017 00:35:56 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-1.8 required=2.0 tests=BAYES_00,DKIM_SIGNED, T_DKIM_INVALID autolearn=no version=3.3.1 Received: from bombadil.infradead.org (bombadil.infradead.org [65.50.211.133]) (using TLSv1.2 with cipher AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id E187528434 for ; Wed, 8 Feb 2017 00:35:55 +0000 (UTC) Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.87 #1 (Red Hat Linux)) id 1cbGEf-0003rU-BG; Wed, 08 Feb 2017 00:35:49 +0000 Received: from smtp.codeaurora.org ([198.145.29.96]) by bombadil.infradead.org with esmtps (Exim 4.87 #1 (Red Hat Linux)) id 1cbGEa-0003q4-E6 for linux-arm-kernel@lists.infradead.org; Wed, 08 Feb 2017 00:35:47 +0000 Received: by smtp.codeaurora.org (Postfix, from userid 1000) id B103460A1E; Wed, 8 Feb 2017 00:35:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1486514123; bh=JrTbsz1ZGRv2tSQtTvhvQ3uD4OWdvzHSrvVBY1HnZnY=; h=From:To:Cc:Subject:Date:From; b=S1otT5kC+q6+VAwWbFY0jUyDpjmZW1ZBCk4NHPDK3DkoX0yOiier5qo5zJRi6ciAk f9159B6ozaNjUxJwFtyLfMZ4PXx1/xjrU1Rd6Bpkc5yW6pOX8E5DZqlTpHfTgLgWf7 3zTx39A+gFcxK7wyGGqQ9Tc2z0wD6S5E9bdvSONE= Received: from illium.qualcomm.com (global_nat1_iad_fw.qualcomm.com [129.46.232.65]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: cov@smtp.codeaurora.org) by smtp.codeaurora.org (Postfix) with ESMTPSA id 6904360884; Wed, 8 Feb 2017 00:35:21 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=codeaurora.org; s=default; t=1486514122; bh=JrTbsz1ZGRv2tSQtTvhvQ3uD4OWdvzHSrvVBY1HnZnY=; h=From:To:Cc:Subject:Date:From; b=drZHgttD8TnCdyWVn4x7XdYcLGUjkFWHOtbFh3yvbIrUHEwFGHFAKHom56gy2HKv1 45CHCjlWUNGwQd0WCqs8ufR3ayzTvTE/FJ0dcGfPC1t/e1Uv9zS7n1oYuwhpv5/yqb 5cL9Ap6oYEYBIc09TJBGsGVUVm6ECGhCnjbv1p50= DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 6904360884 Authentication-Results: pdx-caf-mail.web.codeaurora.org; dmarc=none (p=none dis=none) header.from=codeaurora.org Authentication-Results: pdx-caf-mail.web.codeaurora.org; spf=none smtp.mailfrom=cov@codeaurora.org From: Christopher Covington To: Jonathan Corbet , Marc Zyngier , Catalin Marinas , Will Deacon , linux-doc@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Mark Rutland , linux-kernel@vger.kernel.org, shankerd@codeaurora.org, timur@codeaurora.org Subject: [PATCH v6] arm64: Work around Falkor erratum 1003 Date: Tue, 7 Feb 2017 19:35:16 -0500 Message-Id: <20170208003516.9519-1-cov@codeaurora.org> X-Mailer: git-send-email 2.9.3 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20170207_163545_016812_30A981F4 X-CRM114-Status: GOOD ( 21.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Jon Masters , Neil Leeder , Mark Langsdorf , Christopher Covington , Mark Salter MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+patchwork-linux-arm=patchwork.kernel.org@lists.infradead.org X-Virus-Scanned: ClamAV using ClamSMTP The Qualcomm Datacenter Technologies Falkor v1 CPU may allocate TLB entries using an incorrect ASID when TTBRx_EL1 is being updated. When the erratum is triggered, page table entries using the new translation table base address (BADDR) will be allocated into the TLB using the old ASID. All circumstances leading to the incorrect ASID being cached in the TLB arise when software writes TTBRx_EL1[ASID] and TTBRx_EL1[BADDR], a memory operation is in the process of performing a translation using the specific TTBRx_EL1 being written, and the memory operation uses a translation table descriptor designated as non-global. EL2 and EL3 code changing the EL1&0 ASID is not subject to this erratum because hardware is prohibited from performing translations from an out-of-context translation regime. Consider the following pseudo code. write new BADDR and ASID values to TTBRx_EL1 Replacing the above sequence with the one below will ensure that no TLB entries with an incorrect ASID are used by software. write reserved value to TTBRx_EL1[ASID] ISB write new value to TTBRx_EL1[BADDR] ISB write new value to TTBRx_EL1[ASID] ISB When the above sequence is used, page table entries using the new BADDR value may still be incorrectly allocated into the TLB using the reserved ASID. Yet this will not reduce functionality, since TLB entries incorrectly tagged with the reserved ASID will never be hit by a later instruction. Based on work by Shanker Donthineni Signed-off-by: Christopher Covington --- Based on https://git.kernel.org/cgit/linux/kernel/git/arm64/linux.git/log/?h=for-next/core --- Documentation/arm64/silicon-errata.txt | 1 + arch/arm64/Kconfig | 12 ++++++++++++ arch/arm64/include/asm/assembler.h | 23 +++++++++++++++++++++++ arch/arm64/include/asm/cpucaps.h | 3 ++- arch/arm64/include/asm/mmu_context.h | 8 +++++++- arch/arm64/kernel/cpu_errata.c | 9 +++++++++ arch/arm64/mm/context.c | 11 +++++++++++ arch/arm64/mm/proc.S | 1 + 8 files changed, 66 insertions(+), 2 deletions(-) diff --git a/Documentation/arm64/silicon-errata.txt b/Documentation/arm64/silicon-errata.txt index 0006a94c2321..50da8391e9dd 100644 --- a/Documentation/arm64/silicon-errata.txt +++ b/Documentation/arm64/silicon-errata.txt @@ -63,4 +63,5 @@ stable kernels. | Cavium | ThunderX SMMUv2 | #27704 | N/A | | | | | | | Freescale/NXP | LS2080A/LS1043A | A-008585 | FSL_ERRATUM_A008585 | +| Qualcomm Tech. | Falkor v1 | E1003 | QCOM_FALKOR_ERRATUM_1003| | Qualcomm Tech. | Falkor v1 | E1009 | QCOM_FALKOR_ERRATUM_1009| diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index f45405664558..c97d7fe78c7a 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -480,6 +480,18 @@ config CAVIUM_ERRATUM_27456 If unsure, say Y. +config QCOM_FALKOR_ERRATUM_1003 + bool "Falkor E1003: Incorrect translation due to ASID change" + default y + select ARM64_PAN if ARM64_SW_TTBR0_PAN + help + On Falkor v1, an incorrect ASID may be cached in the TLB when ASID + and BADDR are changed together in TTBRx_EL1. The workaround for this + issue is to use a reserved ASID in cpu_do_switch_mm() before + switching to the new ASID. + + If unsure, say Y. + config QCOM_FALKOR_ERRATUM_1009 bool "Falkor E1009: Prematurely complete a DSB after a TLBI" default y diff --git a/arch/arm64/include/asm/assembler.h b/arch/arm64/include/asm/assembler.h index 446f6c46d4b1..33b20c075fb3 100644 --- a/arch/arm64/include/asm/assembler.h +++ b/arch/arm64/include/asm/assembler.h @@ -25,6 +25,7 @@ #include #include +#include #include #include #include @@ -423,6 +424,28 @@ alternative_endif .endm /* + * Errata workaround prior to TTBR0_EL1 update + * + * val: TTBR value with new BADDR, preserved + * tmp0: temporary register, clobbered + * tmp1: other temporary register, clobbered + */ + .macro pre_ttbr0_update_workaround, val, tmp0, tmp1 +#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003 +alternative_if ARM64_WORKAROUND_QCOM_FALKOR_E1003 + mrs \tmp0, ttbr0_el1 + mov \tmp1, #FALKOR_RESERVED_ASID + bfi \tmp0, \tmp1, #48, #16 // reserved ASID + old BADDR + msr ttbr0_el1, \tmp0 + isb + bfi \tmp0, \val, #0, #48 // reserved ASID + new BADDR + msr ttbr0_el1, \tmp0 + isb +alternative_else_nop_endif +#endif + .endm + +/* * Errata workaround post TTBR0_EL1 update. */ .macro post_ttbr0_update_workaround diff --git a/arch/arm64/include/asm/cpucaps.h b/arch/arm64/include/asm/cpucaps.h index d1207ac696ac..fb78a5d3b60b 100644 --- a/arch/arm64/include/asm/cpucaps.h +++ b/arch/arm64/include/asm/cpucaps.h @@ -36,7 +36,8 @@ #define ARM64_MISMATCHED_CACHE_LINE_SIZE 15 #define ARM64_HAS_NO_FPSIMD 16 #define ARM64_WORKAROUND_REPEAT_TLBI 17 +#define ARM64_WORKAROUND_QCOM_FALKOR_E1003 18 -#define ARM64_NCAPS 18 +#define ARM64_NCAPS 19 #endif /* __ASM_CPUCAPS_H */ diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 63e9982daca1..1ef40d82cfd3 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -19,6 +19,10 @@ #ifndef __ASM_MMU_CONTEXT_H #define __ASM_MMU_CONTEXT_H +#define FALKOR_RESERVED_ASID 1 + +#ifndef __ASSEMBLY__ + #include #include @@ -220,4 +224,6 @@ switch_mm(struct mm_struct *prev, struct mm_struct *next, void verify_cpu_asid_bits(void); -#endif +#endif /* !__ASSEMBLY__ */ + +#endif /* !__ASM_MMU_CONTEXT_H */ diff --git a/arch/arm64/kernel/cpu_errata.c b/arch/arm64/kernel/cpu_errata.c index 32b9beda2ac8..f6cc67e7626e 100644 --- a/arch/arm64/kernel/cpu_errata.c +++ b/arch/arm64/kernel/cpu_errata.c @@ -133,6 +133,15 @@ const struct arm64_cpu_capabilities arm64_errata[] = { .def_scope = SCOPE_LOCAL_CPU, .enable = cpu_enable_trap_ctr_access, }, +#ifdef CONFIG_QCOM_FALKOR_ERRATUM_1003 + { + .desc = "Qualcomm Technologies Falkor erratum 1003", + .capability = ARM64_WORKAROUND_QCOM_FALKOR_E1003, + MIDR_RANGE(MIDR_QCOM_FALKOR_V1, + MIDR_CPU_VAR_REV(0, 0), + MIDR_CPU_VAR_REV(0, 0)), + }, +#endif #ifdef CONFIG_QCOM_FALKOR_ERRATUM_1009 { .desc = "Qualcomm Technologies Falkor erratum 1009", diff --git a/arch/arm64/mm/context.c b/arch/arm64/mm/context.c index 4c63cb154859..67df38184903 100644 --- a/arch/arm64/mm/context.c +++ b/arch/arm64/mm/context.c @@ -79,6 +79,13 @@ void verify_cpu_asid_bits(void) } } +static void set_reserved_asid_bits(void) +{ + if (IS_ENABLED(CONFIG_QCOM_FALKOR_ERRATUM_1003) && + cpus_have_cap(ARM64_WORKAROUND_QCOM_FALKOR_E1003)) + __set_bit(FALKOR_RESERVED_ASID, asid_map); +} + static void flush_context(unsigned int cpu) { int i; @@ -87,6 +94,8 @@ static void flush_context(unsigned int cpu) /* Update the list of reserved ASIDs and the ASID bitmap. */ bitmap_clear(asid_map, 0, NUM_USER_ASIDS); + set_reserved_asid_bits(); + /* * Ensure the generation bump is observed before we xchg the * active_asids. @@ -244,6 +253,8 @@ static int asids_init(void) panic("Failed to allocate bitmap for %lu ASIDs\n", NUM_USER_ASIDS); + set_reserved_asid_bits(); + pr_info("ASID allocator initialised with %lu entries\n", NUM_USER_ASIDS); return 0; } diff --git a/arch/arm64/mm/proc.S b/arch/arm64/mm/proc.S index 32682be978e0..cd4d53d7e458 100644 --- a/arch/arm64/mm/proc.S +++ b/arch/arm64/mm/proc.S @@ -138,6 +138,7 @@ ENDPROC(cpu_do_resume) * - pgd_phys - physical address of new TTB */ ENTRY(cpu_do_switch_mm) + pre_ttbr0_update_workaround x0, x1, x2 mmid x1, x1 // get mm->context.id bfi x0, x1, #48, #16 // set the ASID msr ttbr0_el1, x0 // set TTBR0