From patchwork Mon Dec 9 02:42:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Tong Tiangen X-Patchwork-Id: 13898749 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ACC1E77180 for ; Mon, 9 Dec 2024 02:43:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 585186B0088; Sun, 8 Dec 2024 21:43:29 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 50F546B03A2; Sun, 8 Dec 2024 21:43:29 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 339866B03A3; Sun, 8 Dec 2024 21:43:29 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 17E3A6B0088 for ; Sun, 8 Dec 2024 21:43:29 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id BE9E8160B5A for ; Mon, 9 Dec 2024 02:43:28 +0000 (UTC) X-FDA: 82873874286.02.969A6C1 Received: from szxga05-in.huawei.com (szxga05-in.huawei.com [45.249.212.191]) by imf25.hostedemail.com (Postfix) with ESMTP id CE845A0009 for ; Mon, 9 Dec 2024 02:43:11 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of tongtiangen@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=tongtiangen@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1733712188; a=rsa-sha256; cv=none; b=3lFIfX6xz375iLGmxoM1/5WdzrTzuENXXSArXbShZF15FnNxwGd+UCTYCkqEXj5JWqSwWh f8HoChswv3rTNqbAqkLTwrAj0P9/L1C4gHwJT9fNzEnbO1Zjk1GMGHZE+DvulpJpokeQNA 17ow65q1nlx0eQEdL34r9zf2GgJnys4= ARC-Authentication-Results: i=1; imf25.hostedemail.com; dkim=none; spf=pass (imf25.hostedemail.com: domain of tongtiangen@huawei.com designates 45.249.212.191 as permitted sender) smtp.mailfrom=tongtiangen@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1733712188; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=iDrZfdrMnIOWDOEvnU3lBEDaJYl49vcnJr2daN4zEaU=; b=QA684p4aj2Uj7WAfIX6NqzBQ6/cBI5lRxm4qbl0F1XdL6Mwkt7yFYlYfkD93BwI74foiwL 0F/rXWVB3kBttTr5jpy2+Bt/Gcda/mmpes2uY8TIV56omJFRru2+a/DB/NoeRoKXarliv8 /tSNNu8znUA3OGKCyvJ+KlRWIex+gV4= Received: from mail.maildlp.com (unknown [172.19.163.17]) by szxga05-in.huawei.com (SkyGuard) with ESMTP id 4Y65l23pZpz1JDw3; Mon, 9 Dec 2024 10:43:10 +0800 (CST) Received: from kwepemk500005.china.huawei.com (unknown [7.202.194.90]) by mail.maildlp.com (Postfix) with ESMTPS id 23F251A0188; Mon, 9 Dec 2024 10:43:23 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by kwepemk500005.china.huawei.com (7.202.194.90) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1544.11; Mon, 9 Dec 2024 10:43:21 +0800 From: Tong Tiangen To: Mark Rutland , Jonathan Cameron , Mauro Carvalho Chehab , Catalin Marinas , Will Deacon , Andrew Morton , James Morse , Robin Murphy , Andrey Konovalov , Dmitry Vyukov , Vincenzo Frascino , Michael Ellerman , Nicholas Piggin , Andrey Ryabinin , Alexander Potapenko , Christophe Leroy , Aneesh Kumar K.V , "Naveen N. Rao" , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , , "H. Peter Anvin" , Madhavan Srinivasan CC: , , , , , Tong Tiangen , , Guohanjun Subject: [PATCH v13 2/5] arm64: add support for ARCH_HAS_COPY_MC Date: Mon, 9 Dec 2024 10:42:54 +0800 Message-ID: <20241209024257.3618492-3-tongtiangen@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20241209024257.3618492-1-tongtiangen@huawei.com> References: <20241209024257.3618492-1-tongtiangen@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To kwepemk500005.china.huawei.com (7.202.194.90) X-Rspamd-Queue-Id: CE845A0009 X-Stat-Signature: dpde4ynptxwfygb14j1hfchhz63g8jrb X-Rspam-User: X-Rspamd-Server: rspam09 X-HE-Tag: 1733712191-930424 X-HE-Meta: U2FsdGVkX1/kht6zNA29wmVoBpuJbWy53BmAI1JLhLwFJX8Rl1znCHY4ghfkz8dTY0ls7Ctbe0jQ6HDg9GLB0lsrPlzeNh3qRMz1dIKFTrsWQwjJJxFjuwWH599oszYztJio3FFRN+u5WaUmXQYZ3ttCdcIoML65LWDQ3IIPrQ5daqT0RPxE4Ymv54yGjS2gwnzywJ1F2FpCqz4QUTrHSvJEpiRfgrh5GBySZmqcKtPP/reFTWZd68JJ589jHdYU6yC+32qPHuuKMBH6woj4cCHl/bmpFsUUkoSEDC3+3eaMG5avRzc8vVW8zSYawDUedeTGF1FgUv8l69wkBj1c6mvSQL7wWt36sAkn4y8u+QzgS/pzl1FiyMdKZ1Ts05RQ7CNnyzUlawB2iXUIfO+ePPpbKynRITSypEtxAo4EV0EahfxKwbDCGYdR4TB7QHrI2jc1oZeDfcKege5FY6LCv7MM73M8IHT+Z0iMvpm+Me3f6Xj0hTWmbxy5q4vKjdQ7eO0HtSdge9bXFKKZcDW+/2R0yG7F1G5+2y/dHg7u+tUSDHwmfVPqopzoiJRrzpGcWo1/+PojBmlXgPXIFkdIUgVIhhpMTJWEZtO1S6CgeHKUxXPofF4pWCTVLlGshFHZ8qjdra6WI2Kn5oUoB1EijH8gk62jnbCgRwHvC+fi8LYW/+/UT2L+t9HgJVjfVlb0uIb4MNpdbvBc0UzT8GzXI1pj60hcLUIfiBsQhrVHo0u4Y1/cklBIxJKOHQK+68ac5kJS/YlNuxnU7Sxbtq4T3vbZKvFkss/a2pAGQ4prfWkcYEv1di2/09OhlmEaC82cvhMKhWb0pGko2DsEVqwNCxDsmuFkiHnQeBwc0DpK6SoIVzoicFJp+WffsS1LU+rMDQH+33i8hWfQA840lt2AjLgnK+pT6EdSFAfTsvdYcTi1C7FRBX7kbOeoAFXFhxGeRFp92Q5uLc/0tzR670k f9vLz0X8 CB6QaPQdGhdgnwBiUTqfSryQNfLJOMfZsGXklkPZiHmeUO3lhqE8W961y0FW7o4sn5J3/Qi/ej5DRK+4/lPaN/KafCiv81UC5supSRFsiIf8GVeDIGSNeh84GLu1m5epQbsN7Fvw4JOkdf2ArtEQHTN9FEDxRNpme9cKcoXFwVmlIatvirEWyXqlBjBYVWoY5P03YEC2yo4Q1TKSJ4/yB1d7tBfC51rLCPJKBxw8IDJDHv8Yx+7HxFt7F6MJedhPOk0YJ6/IQuZu0nKWB1bN9ziwtaQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: For the arm64 kernel, when it processes hardware memory errors for synchronize notifications(do_sea()), if the errors is consumed within the kernel, the current processing is panic. However, it is not optimal. Take copy_from/to_user for example, If ld* triggers a memory error, even in kernel mode, only the associated process is affected. Killing the user process and isolating the corrupt page is a better choice. Add new fixup type EX_TYPE_KACCESS_ERR_ZERO_MEM_ERR to identify insn that can recover from memory errors triggered by access to kernel memory, and this fixup type is used in __arch_copy_to_user(), This make the regular copy_to_user() will handle kernel memory errors. Signed-off-by: Tong Tiangen --- arch/arm64/Kconfig | 1 + arch/arm64/include/asm/asm-extable.h | 31 +++++++++++++++++++++++----- arch/arm64/include/asm/asm-uaccess.h | 4 ++++ arch/arm64/include/asm/extable.h | 1 + arch/arm64/lib/copy_to_user.S | 10 ++++----- arch/arm64/mm/extable.c | 19 +++++++++++++++++ arch/arm64/mm/fault.c | 30 ++++++++++++++++++++------- 7 files changed, 78 insertions(+), 18 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 100570a048c5..5fa54d31162c 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -21,6 +21,7 @@ config ARM64 select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE select ARCH_HAS_CACHE_LINE_SIZE select ARCH_HAS_CC_PLATFORM + select ARCH_HAS_COPY_MC if ACPI_APEI_GHES select ARCH_HAS_CURRENT_STACK_POINTER select ARCH_HAS_DEBUG_VIRTUAL select ARCH_HAS_DEBUG_VM_PGTABLE diff --git a/arch/arm64/include/asm/asm-extable.h b/arch/arm64/include/asm/asm-extable.h index b8a5861dc7b7..0f9123efca0a 100644 --- a/arch/arm64/include/asm/asm-extable.h +++ b/arch/arm64/include/asm/asm-extable.h @@ -5,11 +5,13 @@ #include #include -#define EX_TYPE_NONE 0 -#define EX_TYPE_BPF 1 -#define EX_TYPE_UACCESS_ERR_ZERO 2 -#define EX_TYPE_KACCESS_ERR_ZERO 3 -#define EX_TYPE_LOAD_UNALIGNED_ZEROPAD 4 +#define EX_TYPE_NONE 0 +#define EX_TYPE_BPF 1 +#define EX_TYPE_UACCESS_ERR_ZERO 2 +#define EX_TYPE_KACCESS_ERR_ZERO 3 +#define EX_TYPE_LOAD_UNALIGNED_ZEROPAD 4 +/* kernel access memory error safe */ +#define EX_TYPE_KACCESS_ERR_ZERO_MEM_ERR 5 /* Data fields for EX_TYPE_UACCESS_ERR_ZERO */ #define EX_DATA_REG_ERR_SHIFT 0 @@ -51,6 +53,17 @@ #define _ASM_EXTABLE_UACCESS(insn, fixup) \ _ASM_EXTABLE_UACCESS_ERR_ZERO(insn, fixup, wzr, wzr) +#define _ASM_EXTABLE_KACCESS_ERR_ZERO_MEM_ERR(insn, fixup, err, zero) \ + __ASM_EXTABLE_RAW(insn, fixup, \ + EX_TYPE_KACCESS_ERR_ZERO_MEM_ERR, \ + ( \ + EX_DATA_REG(ERR, err) | \ + EX_DATA_REG(ZERO, zero) \ + )) + +#define _ASM_EXTABLE_KACCESS_MEM_ERR(insn, fixup) \ + _ASM_EXTABLE_KACCESS_ERR_ZERO_MEM_ERR(insn, fixup, wzr, wzr) + /* * Create an exception table entry for uaccess `insn`, which will branch to `fixup` * when an unhandled fault is taken. @@ -69,6 +82,14 @@ .endif .endm +/* + * Create an exception table entry for kaccess `insn`, which will branch to + * `fixup` when an unhandled fault is taken. + */ + .macro _asm_extable_kaccess_mem_err, insn, fixup + _ASM_EXTABLE_KACCESS_MEM_ERR(\insn, \fixup) + .endm + #else /* __ASSEMBLY__ */ #include diff --git a/arch/arm64/include/asm/asm-uaccess.h b/arch/arm64/include/asm/asm-uaccess.h index 5b6efe8abeeb..19aa0180f645 100644 --- a/arch/arm64/include/asm/asm-uaccess.h +++ b/arch/arm64/include/asm/asm-uaccess.h @@ -57,6 +57,10 @@ alternative_else_nop_endif .endm #endif +#define KERNEL_MEM_ERR(l, x...) \ +9999: x; \ + _asm_extable_kaccess_mem_err 9999b, l + #define USER(l, x...) \ 9999: x; \ _asm_extable_uaccess 9999b, l diff --git a/arch/arm64/include/asm/extable.h b/arch/arm64/include/asm/extable.h index 72b0e71cc3de..bc49443bc502 100644 --- a/arch/arm64/include/asm/extable.h +++ b/arch/arm64/include/asm/extable.h @@ -46,4 +46,5 @@ bool ex_handler_bpf(const struct exception_table_entry *ex, #endif /* !CONFIG_BPF_JIT */ bool fixup_exception(struct pt_regs *regs); +bool fixup_exception_me(struct pt_regs *regs); #endif diff --git a/arch/arm64/lib/copy_to_user.S b/arch/arm64/lib/copy_to_user.S index 802231772608..bedab1678431 100644 --- a/arch/arm64/lib/copy_to_user.S +++ b/arch/arm64/lib/copy_to_user.S @@ -20,7 +20,7 @@ * x0 - bytes not copied */ .macro ldrb1 reg, ptr, val - ldrb \reg, [\ptr], \val + KERNEL_MEM_ERR(9998f, ldrb \reg, [\ptr], \val) .endm .macro strb1 reg, ptr, val @@ -28,7 +28,7 @@ .endm .macro ldrh1 reg, ptr, val - ldrh \reg, [\ptr], \val + KERNEL_MEM_ERR(9998f, ldrh \reg, [\ptr], \val) .endm .macro strh1 reg, ptr, val @@ -36,7 +36,7 @@ .endm .macro ldr1 reg, ptr, val - ldr \reg, [\ptr], \val + KERNEL_MEM_ERR(9998f, ldr \reg, [\ptr], \val) .endm .macro str1 reg, ptr, val @@ -44,7 +44,7 @@ .endm .macro ldp1 reg1, reg2, ptr, val - ldp \reg1, \reg2, [\ptr], \val + KERNEL_MEM_ERR(9998f, ldp \reg1, \reg2, [\ptr], \val) .endm .macro stp1 reg1, reg2, ptr, val @@ -64,7 +64,7 @@ SYM_FUNC_START(__arch_copy_to_user) 9997: cmp dst, dstin b.ne 9998f // Before being absolutely sure we couldn't copy anything, try harder - ldrb tmp1w, [srcin] +KERNEL_MEM_ERR(9998f, ldrb tmp1w, [srcin]) USER(9998f, sttrb tmp1w, [dst]) add dst, dst, #1 9998: sub x0, end, dst // bytes not copied diff --git a/arch/arm64/mm/extable.c b/arch/arm64/mm/extable.c index 228d681a8715..9ad2b6473b60 100644 --- a/arch/arm64/mm/extable.c +++ b/arch/arm64/mm/extable.c @@ -72,7 +72,26 @@ bool fixup_exception(struct pt_regs *regs) return ex_handler_uaccess_err_zero(ex, regs); case EX_TYPE_LOAD_UNALIGNED_ZEROPAD: return ex_handler_load_unaligned_zeropad(ex, regs); + case EX_TYPE_KACCESS_ERR_ZERO_MEM_ERR: + return false; } BUG(); } + +bool fixup_exception_me(struct pt_regs *regs) +{ + const struct exception_table_entry *ex; + + ex = search_exception_tables(instruction_pointer(regs)); + if (!ex) + return false; + + switch (ex->type) { + case EX_TYPE_UACCESS_ERR_ZERO: + case EX_TYPE_KACCESS_ERR_ZERO_MEM_ERR: + return ex_handler_uaccess_err_zero(ex, regs); + } + + return false; +} diff --git a/arch/arm64/mm/fault.c b/arch/arm64/mm/fault.c index ef63651099a9..278e67357f49 100644 --- a/arch/arm64/mm/fault.c +++ b/arch/arm64/mm/fault.c @@ -801,21 +801,35 @@ static int do_bad(unsigned long far, unsigned long esr, struct pt_regs *regs) return 1; /* "fault" */ } +/* + * APEI claimed this as a firmware-first notification. + * Some processing deferred to task_work before ret_to_user(). + */ +static int do_apei_claim_sea(struct pt_regs *regs) +{ + int ret; + + ret = apei_claim_sea(regs); + if (ret) + return ret; + + if (!user_mode(regs) && IS_ENABLED(CONFIG_ARCH_HAS_COPY_MC)) { + if (!fixup_exception_me(regs)) + return -ENOENT; + } + + return ret; +} + static int do_sea(unsigned long far, unsigned long esr, struct pt_regs *regs) { const struct fault_info *inf; unsigned long siaddr; - inf = esr_to_fault_info(esr); - - if (user_mode(regs) && apei_claim_sea(regs) == 0) { - /* - * APEI claimed this as a firmware-first notification. - * Some processing deferred to task_work before ret_to_user(). - */ + if (do_apei_claim_sea(regs) == 0) return 0; - } + inf = esr_to_fault_info(esr); if (esr & ESR_ELx_FnV) { siaddr = 0; } else {