From patchwork Tue Jan 23 14:53:34 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ard Biesheuvel X-Patchwork-Id: 13527554 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EF80BC47258 for ; Tue, 23 Jan 2024 14:57:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:Cc:To:From:Subject:Message-ID: References:Mime-Version:In-Reply-To:Date:Reply-To:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:List-Owner; bh=p7wFpU8MRkku6ldiW3Kc/R24Tk2Us67u1LFUeMDYgZs=; b=von5g4XIRAS+WyLgFMAVgpMrg4 7fVnznkB3IOOrMQEYIWIcE3wW4bB+4/6riSZANsLU4pnCWCHusAHU4k8kMlv5Hn7+tmHbVQFydaV7 G6VzvE0gQMCp3DsmIDDb+7UqMnIP9zc2FAFh6mQ3aWoDMbdYLhAc8fAxHNrODZxFCIp0E48AettZR f1V2jF1n30DlneGODqoU3TGN59p/mzXxd0b8fiI7WDAGbGSWHsfTy3rbj1+5/yPFwyvNDDqvsKh5O rHN9kSzGQefGwTj3dJHDSuUDbZ+y4aef7zv8gNUHhDTVLT3/evqHZbt83cZ/7T1TgCtzz6kgPXjHL SwUtiKxw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rSICe-00Gulc-0X; Tue, 23 Jan 2024 14:56:40 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rSIBo-00GuLQ-2B for linux-arm-kernel@bombadil.infradead.org; Tue, 23 Jan 2024 14:55:48 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:Cc:To:From:Subject: Message-ID:References:Mime-Version:In-Reply-To:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=F3a2hc8n1NEqmp15ROD9lmeVnHS2pOeqn2GhIHm4mdM=; b=ocmv6TNsr7CbRpftIaoLGkvWmM udk6/PY7ndmnBS6QX5ukenVnqlZSX1wllyL8JqKOiQk+pEvMXxHITaAF5Qrb44EByDGFZPnKGqft5 4GRJkixb+UzL6E96grl0AJPN2uzAgmnAdBzdAV3q6bgSLTjna2tcS+y/lPxwiMfl6qt9MqBUvOueL Jm0mYhJt6OeknAYHIfw1bNqzI+Vk2h0HOr6+6PZH6y9/Swsg1pZ/Misk65l02F3gDeTxI9utW6oeW OYn0nk4210hsic4TLeMlu2oQpf1fJSw9Gi1dzY5+Vv5WsvPO95YhVskp6k8H42BGU4Fj7ghNj+csl gBddUhig==; Received: from mail-yw1-x114a.google.com ([2607:f8b0:4864:20::114a]) by desiato.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rSIBl-00000003vW6-265x for linux-arm-kernel@lists.infradead.org; Tue, 23 Jan 2024 14:55:47 +0000 Received: by mail-yw1-x114a.google.com with SMTP id 00721157ae682-5f874219ff9so59508647b3.0 for ; Tue, 23 Jan 2024 06:55:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1706021743; x=1706626543; darn=lists.infradead.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=F3a2hc8n1NEqmp15ROD9lmeVnHS2pOeqn2GhIHm4mdM=; b=TjchzgTkS2tlEuYc3JWnYvMGT+2civJtMt+2FuXmmiJEnH0d/6IJPgTH3CWaUK6BmP 9ul2nBETn0VCfVqy5UgSj2B/2t4Vc+sCvmnecv8PSX9o5cqOTrEG/T72+aVO/n6z4jZm 9eL6eZrlJNX/xv3z/z/T45aVL08Ms5LYZbfEYxOwtDYHwK+FZ8/ppxdhiTD9MF3Q9W5A gOSuJLPg72/pmnGRLMlUXSLlo5fQv/cZb2/CxsnM7XqiO+zqzBK0GCBj+QVXhDRKoxH1 FgcD/QYCA0OlSq6IrHriLXw9imxXSCOLG162IGRFrOk4RKGVEmlRMuJaGB2+xrHhVqvB igYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1706021743; x=1706626543; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=F3a2hc8n1NEqmp15ROD9lmeVnHS2pOeqn2GhIHm4mdM=; b=JUSPWJtOwEJ7Pss5G8R5LXmE2lgZdP9ySa7G5X/ONRlDSo3l9t2B+k0RV6lcZd067v Sz0SRvexKRj3jMcAqjGdYqB5w4LgrLMWIiy28wrBJ8W6ejSt+YheX3fTRt17BBKCs6cz Q0bwlCrz3J8p/BsNiFOTTWOX3GgMgInKmrt8aNyqikSbsZT7+RQ32ARnoJBj1IHESCW1 zitOXbf8G1JZ5nL/YBQxbVsNJdUE9j53S7t1xHNS6SI064s5m0jk7kRn6jyd58d9tLW6 f/hz53CZ+UX207PvNcsVh9ILgSj8OaDS2yyPDetowk3qvJs5x8pu/gzZuPj3qwaMU7An 9iYA== X-Gm-Message-State: AOJu0YyX+Nh60qORiBo0F8u3H2BUEntdT+QsV63GnmZOZbidOWBAlnMB AykolWeTcsX/I4oxMQZHQ3K230OQ8IhkSgZvRWugeJQuxe3pZcKdUa1D8AqY4mNPWbBpYN67zBS LqB01jY9h8ud5NM+7HV39Mpm5SXHpIPwoGTDP7ilPCfvH90r37GBEx6xqlBTmx2sw8mvCI13o19 Gw73EO90jPCX2xtC5MKZJB/LBfhHNm5JUP0jt6TB1S X-Google-Smtp-Source: AGHT+IHYKvKWDX7s05YamCtVyxJyTIOWUJzX9xOAqIuqKd42b+llgpmBuQjcXmEBDVvTGI+ifhP6wU5D X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:690c:300c:b0:5ff:83c7:b5a0 with SMTP id ey12-20020a05690c300c00b005ff83c7b5a0mr3135678ywb.5.1706021743058; Tue, 23 Jan 2024 06:55:43 -0800 (PST) Date: Tue, 23 Jan 2024 15:53:34 +0100 In-Reply-To: <20240123145258.1462979-52-ardb+git@google.com> Mime-Version: 1.0 References: <20240123145258.1462979-52-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=8056; i=ardb@kernel.org; h=from:subject; bh=TzOGEPJlOyzNvktAFd7lIPH+fx980ZM7MqoATlpDSOQ=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIXX9pQ/ZUUkBNvv7D104teGLUOT5xxk8VtPMs1z+/Hiyn GWflte/jlIWBjEOBlkxRRaB2X/f7Tw9UarWeZYszBxWJpAhDFycAjCRNTaMDMtePG77s2VxRPA7 hb6vfxYu/tCnHOb9SW/7HpHsrK9+jycxMhyeGPdljWyz6nP1B48Due4m3O6S7ZETzfqr8tJqKrN 3PRMA X-Mailer: git-send-email 2.43.0.429.g432eaa2c6b-goog Message-ID: <20240123145258.1462979-87-ardb+git@google.com> Subject: [PATCH v7 35/50] arm64: Add ESR decoding for exceptions involving translation level -1 From: Ard Biesheuvel To: linux-arm-kernel@lists.infradead.org Cc: Ard Biesheuvel , Catalin Marinas , Will Deacon , Marc Zyngier , Mark Rutland , Ryan Roberts , Anshuman Khandual , Kees Cook X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240123_145545_800887_A4EF27A2 X-CRM114-Status: GOOD ( 17.60 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Ard Biesheuvel The LPA2 feature introduces new FSC values to report abort exceptions related to translation level -1. Define these and wire them up. Reuse the new ESR FSC classification helpers that arrived via the KVM arm64 tree, and update the one for translation faults to check specifically for a translation fault at level -1. (Access flag or permission faults cannot occur at level -1 because they alway involve a descriptor at the superior level so changing those helpers is not needed). Signed-off-by: Ard Biesheuvel --- arch/arm64/include/asm/esr.h | 13 ++++----- arch/arm64/include/asm/kvm_emulate.h | 10 ++----- arch/arm64/mm/fault.c | 30 +++++++------------- 3 files changed, 18 insertions(+), 35 deletions(-) diff --git a/arch/arm64/include/asm/esr.h b/arch/arm64/include/asm/esr.h index 353fe08546cf..81606bf7d5ac 100644 --- a/arch/arm64/include/asm/esr.h +++ b/arch/arm64/include/asm/esr.h @@ -117,15 +117,9 @@ #define ESR_ELx_FSC_ACCESS (0x08) #define ESR_ELx_FSC_FAULT (0x04) #define ESR_ELx_FSC_PERM (0x0C) -#define ESR_ELx_FSC_SEA_TTW0 (0x14) -#define ESR_ELx_FSC_SEA_TTW1 (0x15) -#define ESR_ELx_FSC_SEA_TTW2 (0x16) -#define ESR_ELx_FSC_SEA_TTW3 (0x17) +#define ESR_ELx_FSC_SEA_TTW(n) (0x14 + (n)) #define ESR_ELx_FSC_SECC (0x18) -#define ESR_ELx_FSC_SECC_TTW0 (0x1c) -#define ESR_ELx_FSC_SECC_TTW1 (0x1d) -#define ESR_ELx_FSC_SECC_TTW2 (0x1e) -#define ESR_ELx_FSC_SECC_TTW3 (0x1f) +#define ESR_ELx_FSC_SECC_TTW(n) (0x1c + (n)) /* ISS field definitions for Data Aborts */ #define ESR_ELx_ISV_SHIFT (24) @@ -394,6 +388,9 @@ static inline bool esr_is_data_abort(unsigned long esr) static inline bool esr_fsc_is_translation_fault(unsigned long esr) { + /* Translation fault, level -1 */ + if ((esr & ESR_ELx_FSC) == 0b101011) + return true; return (esr & ESR_ELx_FSC_TYPE) == ESR_ELx_FSC_FAULT; } diff --git a/arch/arm64/include/asm/kvm_emulate.h b/arch/arm64/include/asm/kvm_emulate.h index b804fe832184..6f5b41c70103 100644 --- a/arch/arm64/include/asm/kvm_emulate.h +++ b/arch/arm64/include/asm/kvm_emulate.h @@ -425,15 +425,9 @@ static __always_inline bool kvm_vcpu_abt_issea(const struct kvm_vcpu *vcpu) { switch (kvm_vcpu_trap_get_fault(vcpu)) { case ESR_ELx_FSC_EXTABT: - case ESR_ELx_FSC_SEA_TTW0: - case ESR_ELx_FSC_SEA_TTW1: - case ESR_ELx_FSC_SEA_TTW2: - case ESR_ELx_FSC_SEA_TTW3: + case ESR_ELx_FSC_SEA_TTW(-1) ... ESR_ELx_FSC_SEA_TTW(3): case ESR_ELx_FSC_SECC: - case ESR_ELx_FSC_SECC_TTW0: - case ESR_ELx_FSC_SECC_TTW1: - case ESR_ELx_FSC_SECC_TTW2: - case ESR_ELx_FSC_SECC_TTW3: + case ESR_ELx_FSC_SECC_TTW(-1) ... ESR_ELx_FSC_SECC_TTW(3): return true; default: return false; diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index 55f6455a8284..60265ede48fe 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -257,16 +257,14 @@ static bool is_el1_data_abort(unsigned long esr) static inline bool is_el1_permission_fault(unsigned long addr, unsigned long esr, struct pt_regs *regs) { - unsigned long fsc_type = esr & ESR_ELx_FSC_TYPE; - if (!is_el1_data_abort(esr) && !is_el1_instruction_abort(esr)) return false; - if (fsc_type == ESR_ELx_FSC_PERM) + if (esr_fsc_is_permission_fault(esr)) return true; if (is_ttbr0_addr(addr) && system_uses_ttbr0_pan()) - return fsc_type == ESR_ELx_FSC_FAULT && + return esr_fsc_is_translation_fault(esr) && (regs->pstate & PSR_PAN_BIT); return false; @@ -279,8 +277,7 @@ static bool __kprobes is_spurious_el1_translation_fault(unsigned long addr, unsigned long flags; u64 par, dfsc; - if (!is_el1_data_abort(esr) || - (esr & ESR_ELx_FSC_TYPE) != ESR_ELx_FSC_FAULT) + if (!is_el1_data_abort(esr) || !esr_fsc_is_translation_fault(esr)) return false; local_irq_save(flags); @@ -301,7 +298,7 @@ static bool __kprobes is_spurious_el1_translation_fault(unsigned long addr, * treat the translation fault as spurious. */ dfsc = FIELD_GET(SYS_PAR_EL1_FST, par); - return (dfsc & ESR_ELx_FSC_TYPE) != ESR_ELx_FSC_FAULT; + return !esr_fsc_is_translation_fault(dfsc); } static void die_kernel_fault(const char *msg, unsigned long addr, @@ -368,11 +365,6 @@ static bool is_el1_mte_sync_tag_check_fault(unsigned long esr) return false; } -static bool is_translation_fault(unsigned long esr) -{ - return (esr & ESR_ELx_FSC_TYPE) == ESR_ELx_FSC_FAULT; -} - static void __do_kernel_fault(unsigned long addr, unsigned long esr, struct pt_regs *regs) { @@ -405,7 +397,7 @@ static void __do_kernel_fault(unsigned long addr, unsigned long esr, } else if (addr < PAGE_SIZE) { msg = "NULL pointer dereference"; } else { - if (is_translation_fault(esr) && + if (esr_fsc_is_translation_fault(esr) && kfence_handle_page_fault(addr, esr & ESR_ELx_WNR, regs)) return; @@ -782,18 +774,18 @@ static const struct fault_info fault_info[] = { { do_translation_fault, SIGSEGV, SEGV_MAPERR, "level 1 translation fault" }, { do_translation_fault, SIGSEGV, SEGV_MAPERR, "level 2 translation fault" }, { do_translation_fault, SIGSEGV, SEGV_MAPERR, "level 3 translation fault" }, - { do_bad, SIGKILL, SI_KERNEL, "unknown 8" }, + { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 0 access flag fault" }, { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 1 access flag fault" }, { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 2 access flag fault" }, { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 access flag fault" }, - { do_bad, SIGKILL, SI_KERNEL, "unknown 12" }, + { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 0 permission fault" }, { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 1 permission fault" }, { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 2 permission fault" }, { do_page_fault, SIGSEGV, SEGV_ACCERR, "level 3 permission fault" }, { do_sea, SIGBUS, BUS_OBJERR, "synchronous external abort" }, { do_tag_check_fault, SIGSEGV, SEGV_MTESERR, "synchronous tag check fault" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 18" }, - { do_bad, SIGKILL, SI_KERNEL, "unknown 19" }, + { do_sea, SIGKILL, SI_KERNEL, "level -1 (translation table walk)" }, { do_sea, SIGKILL, SI_KERNEL, "level 0 (translation table walk)" }, { do_sea, SIGKILL, SI_KERNEL, "level 1 (translation table walk)" }, { do_sea, SIGKILL, SI_KERNEL, "level 2 (translation table walk)" }, @@ -801,7 +793,7 @@ static const struct fault_info fault_info[] = { { do_sea, SIGBUS, BUS_OBJERR, "synchronous parity or ECC error" }, // Reserved when RAS is implemented { do_bad, SIGKILL, SI_KERNEL, "unknown 25" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 26" }, - { do_bad, SIGKILL, SI_KERNEL, "unknown 27" }, + { do_sea, SIGKILL, SI_KERNEL, "level -1 synchronous parity error (translation table walk)" }, // Reserved when RAS is implemented { do_sea, SIGKILL, SI_KERNEL, "level 0 synchronous parity error (translation table walk)" }, // Reserved when RAS is implemented { do_sea, SIGKILL, SI_KERNEL, "level 1 synchronous parity error (translation table walk)" }, // Reserved when RAS is implemented { do_sea, SIGKILL, SI_KERNEL, "level 2 synchronous parity error (translation table walk)" }, // Reserved when RAS is implemented @@ -815,9 +807,9 @@ static const struct fault_info fault_info[] = { { do_bad, SIGKILL, SI_KERNEL, "unknown 38" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 39" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 40" }, - { do_bad, SIGKILL, SI_KERNEL, "unknown 41" }, + { do_bad, SIGKILL, SI_KERNEL, "level -1 address size fault" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 42" }, - { do_bad, SIGKILL, SI_KERNEL, "unknown 43" }, + { do_translation_fault, SIGSEGV, SEGV_MAPERR, "level -1 translation fault" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 44" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 45" }, { do_bad, SIGKILL, SI_KERNEL, "unknown 46" },