From patchwork Tue Mar 23 20:39:30 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159347 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2FEA2C433DB for ; Tue, 23 Mar 2021 20:42:38 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 451BF61574 for ; Tue, 23 Mar 2021 20:42:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 451BF61574 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wFaVOpmb4E/n9e4oIMH4S0ZDx1oeY1wuY2Hra3jkr6I=; b=ROnmX/S68IJkfQ 2vLe206sypqNGLYAeldVc/jl75r2ceM5B8tzWVMqfhMYXA9wqGVD4yLERO/it0ob4eTOPOJH4bUPz G6164iKGFSkWqP+XESWcAQAa932PHXl629F32yoaXe7C3xR7O3Qk0TyA5XFA5l1ElUPReYKmNDm0F cO9fS5vDdvTfikcmuNzKe8ygHHZONfEyp4FMgWHzP4P/2HKwROoKWGYG7L1ZnV+ylmYM39+tN2P43 Zf9L228HEyzrgtYuWWXF/TNc7SVnVQQ+JLrXdvebckSm4Y+eUpH+d8jfiQI6gFNP5Y6JLNLBuwBzm rb30HTBSSfnaEVD3CJ/A==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnoq-00Fg6j-U9; Tue, 23 Mar 2021 20:40:05 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnoe-00Fg3q-Dx for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:39:55 +0000 Received: by mail-qk1-x749.google.com with SMTP id j14so134791qka.7 for ; Tue, 23 Mar 2021 13:39:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc:content-transfer-encoding; bh=oRH0oPqc1YkPpMAYQYFZl9QAi5/yaxV7NoOeL8QzXY4=; b=GjBgkHJGOeUrIBhop7aWNfhrqfm1aGxDtV3jtuOZIQDlOColM38iVA4ChTKOGs4Vn7 N5IFA5VTu6lclmpBPE9BdZbHEyBaMp8briSjKqsuUmOVK1wpG2wcpsCP9VXyZYyVIoam DHTzjCZ4RDeu+ljC3DvSxDKyaKnBoK+rI5S1K95pDpB6ARMOrZsyjJ5Wd+dT6a8ezDnZ DBoHStC+B5jrj4U2u+dFApkUlFZRW4e48m6eMII6ItNn+1FP3mcuay/8SuuPmC/fnIDZ sKwOsG+GCY4mLxFS+ojNJLKvGu+RTvlb4oH1ee+XCmCiFdJuVke/9wV+bZ+l9fx/HDTd BCfw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc:content-transfer-encoding; bh=oRH0oPqc1YkPpMAYQYFZl9QAi5/yaxV7NoOeL8QzXY4=; b=YTsF0mcyFclrfiKBwuz4S7zdsAbQaQmKwxosUHA6xeDwiRGRAsmNr1dyyHZgKSrwMX f5DpGjyWdnpN30fBXoqk2CUB1TSdvS9uBJOkY4UiC5kHHXKclnDMom9PsQ1Sf4ERdaom Ol/+ATujwc67GjEXukCcjSD+cW/1CFjY3LM90xBTBuGNi7NSzWjkpGAoXuRSB9EdiT/B cqwL21NBAGhf8KECQ605ZWoPbPBfeCtoQyuTBfSuOby0XZUvW20DaeWi8xx3BRm7e5kZ zZ3aQIkpW8mbKhCQieWUhBCMbjP4gZZV3wHby5DFaFBgiwknp88zp5ADQ1ZwQa5uY2Xr MVSQ== X-Gm-Message-State: AOAM533O7fOD9INDM+pKTKeP39ybNkFzJovH9E/88cdyUYIUG8sQEhs1 HaLdXlVTe4wYPzkdfX0xpH9iBWfAHft7UiEKT7E= X-Google-Smtp-Source: ABdhPJxDF0lHo0+rGsyYGKZTGaYYdqb/VVtyXHwYyK1sQr6dy4J7IMGg0tunH3tqmmIpzV7AcPx23lj4ssrrCTE+rJc= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a05:6214:90b:: with SMTP id dj11mr6674797qvb.52.1616531990609; Tue, 23 Mar 2021 13:39:50 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:30 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-2-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 01/17] add support for Clang CFI From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_203952_845208_C9E0D015 X-CRM114-Status: GOOD ( 36.75 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org This change adds support for Clang’s forward-edge Control Flow Integrity (CFI) checking. With CONFIG_CFI_CLANG, the compiler injects a runtime check before each indirect function call to ensure the target is a valid function with the correct static type. This restricts possible call targets and makes it more difficult for an attacker to exploit bugs that allow the modification of stored function pointers. For more details, see: https://clang.llvm.org/docs/ControlFlowIntegrity.html Clang requires CONFIG_LTO_CLANG to be enabled with CFI to gain visibility to possible call targets. Kernel modules are supported with Clang’s cross-DSO CFI mode, which allows checking between independently compiled components. With CFI enabled, the compiler injects a __cfi_check() function into the kernel and each module for validating local call targets. For cross-module calls that cannot be validated locally, the compiler calls the global __cfi_slowpath_diag() function, which determines the target module and calls the correct __cfi_check() function. This patch includes a slowpath implementation that uses __module_address() to resolve call targets, and with CONFIG_CFI_CLANG_SHADOW enabled, a shadow map that speeds up module look-ups by ~3x. Clang implements indirect call checking using jump tables and offers two methods of generating them. With canonical jump tables, the compiler renames each address-taken function to .cfi and points the original symbol to a jump table entry, which passes __cfi_check() validation. This isn’t compatible with stand-alone assembly code, which the compiler doesn’t instrument, and would result in indirect calls to assembly code to fail. Therefore, we default to using non-canonical jump tables instead, where the compiler generates a local jump table entry .cfi_jt for each address-taken function, and replaces all references to the function with the address of the jump table entry. Note that because non-canonical jump table addresses are local to each component, they break cross-module function address equality. Specifically, the address of a global function will be different in each module, as it's replaced with the address of a local jump table entry. If this address is passed to a different module, it won’t match the address of the same function taken there. This may break code that relies on comparing addresses passed from other components. CFI checking can be disabled in a function with the __nocfi attribute. Additionally, CFI can be disabled for an entire compilation unit by filtering out CC_FLAGS_CFI. By default, CFI failures result in a kernel panic to stop a potential exploit. CONFIG_CFI_PERMISSIVE enables a permissive mode, where the kernel prints out a rate-limited warning instead, and allows execution to continue. This option is helpful for locating type mismatches, but should only be enabled during development. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- Makefile | 17 ++ arch/Kconfig | 45 ++++ include/asm-generic/bug.h | 16 ++ include/asm-generic/vmlinux.lds.h | 20 +- include/linux/cfi.h | 41 ++++ include/linux/compiler-clang.h | 2 + include/linux/compiler_types.h | 4 + include/linux/init.h | 2 +- include/linux/module.h | 13 +- init/Kconfig | 2 +- kernel/Makefile | 4 + kernel/cfi.c | 329 ++++++++++++++++++++++++++++++ kernel/module.c | 43 ++++ scripts/Makefile.modfinal | 2 +- 14 files changed, 534 insertions(+), 6 deletions(-) create mode 100644 include/linux/cfi.h create mode 100644 kernel/cfi.c diff --git a/Makefile b/Makefile index d4784d181123..2b90a5cc08c8 100644 --- a/Makefile +++ b/Makefile @@ -920,6 +920,23 @@ KBUILD_AFLAGS += -fno-lto export CC_FLAGS_LTO endif +ifdef CONFIG_CFI_CLANG +CC_FLAGS_CFI := -fsanitize=cfi \ + -fsanitize-cfi-cross-dso \ + -fno-sanitize-cfi-canonical-jump-tables \ + -fno-sanitize-trap=cfi \ + -fno-sanitize-blacklist + +ifdef CONFIG_CFI_PERMISSIVE +CC_FLAGS_CFI += -fsanitize-recover=cfi +endif + +# If LTO flags are filtered out, we must also filter out CFI. +CC_FLAGS_LTO += $(CC_FLAGS_CFI) +KBUILD_CFLAGS += $(CC_FLAGS_CFI) +export CC_FLAGS_CFI +endif + ifdef CONFIG_DEBUG_FORCE_FUNCTION_ALIGN_32B KBUILD_CFLAGS += -falign-functions=32 endif diff --git a/arch/Kconfig b/arch/Kconfig index ecfd3520b676..f6a85ba6cba2 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -692,6 +692,51 @@ config LTO_CLANG_THIN If unsure, say Y. endchoice +config ARCH_SUPPORTS_CFI_CLANG + bool + help + An architecture should select this option if it can support Clang's + Control-Flow Integrity (CFI) checking. + +config CFI_CLANG + bool "Use Clang's Control Flow Integrity (CFI)" + depends on LTO_CLANG && ARCH_SUPPORTS_CFI_CLANG + # Clang >= 12: + # - https://bugs.llvm.org/show_bug.cgi?id=46258 + # - https://bugs.llvm.org/show_bug.cgi?id=47479 + depends on CLANG_VERSION >= 120000 + select KALLSYMS + help + This option enables Clang’s forward-edge Control Flow Integrity + (CFI) checking, where the compiler injects a runtime check to each + indirect function call to ensure the target is a valid function with + the correct static type. This restricts possible call targets and + makes it more difficult for an attacker to exploit bugs that allow + the modification of stored function pointers. More information can be + found from Clang's documentation: + + https://clang.llvm.org/docs/ControlFlowIntegrity.html + +config CFI_CLANG_SHADOW + bool "Use CFI shadow to speed up cross-module checks" + default y + depends on CFI_CLANG && MODULES + help + If you select this option, the kernel builds a fast look-up table of + CFI check functions in loaded modules to reduce performance overhead. + + If unsure, say Y. + +config CFI_PERMISSIVE + bool "Use CFI in permissive mode" + depends on CFI_CLANG + help + When selected, Control Flow Integrity (CFI) violations result in a + warning instead of a kernel panic. This option should only be used + for finding indirect call type mismatches during development. + + If unsure, say N. + config HAVE_ARCH_WITHIN_STACK_FRAMES bool help diff --git a/include/asm-generic/bug.h b/include/asm-generic/bug.h index 76a10e0dca9f..b402494883b6 100644 --- a/include/asm-generic/bug.h +++ b/include/asm-generic/bug.h @@ -241,6 +241,22 @@ void __warn(const char *file, int line, void *caller, unsigned taint, # define WARN_ON_SMP(x) ({0;}) #endif +/* + * WARN_ON_FUNCTION_MISMATCH() warns if a value doesn't match a + * function address, and can be useful for catching issues with + * callback functions, for example. + * + * With CONFIG_CFI_CLANG, the warning is disabled because the + * compiler replaces function addresses taken in C code with + * local jump table addresses, which breaks cross-module function + * address equality. + */ +#if defined(CONFIG_CFI_CLANG) && defined(CONFIG_MODULES) +# define WARN_ON_FUNCTION_MISMATCH(x, fn) ({ 0; }) +#else +# define WARN_ON_FUNCTION_MISMATCH(x, fn) WARN_ON_ONCE((x) != (fn)) +#endif + #endif /* __ASSEMBLY__ */ #endif diff --git a/include/asm-generic/vmlinux.lds.h b/include/asm-generic/vmlinux.lds.h index 0331d5d49551..40a9c101565e 100644 --- a/include/asm-generic/vmlinux.lds.h +++ b/include/asm-generic/vmlinux.lds.h @@ -544,6 +544,22 @@ . = ALIGN((align)); \ __end_rodata = .; + +/* + * .text..L.cfi.jumptable.* contain Control-Flow Integrity (CFI) + * jump table entries. + */ +#ifdef CONFIG_CFI_CLANG +#define TEXT_CFI_JT \ + . = ALIGN(PMD_SIZE); \ + __cfi_jt_start = .; \ + *(.text..L.cfi.jumptable .text..L.cfi.jumptable.*) \ + . = ALIGN(PMD_SIZE); \ + __cfi_jt_end = .; +#else +#define TEXT_CFI_JT +#endif + /* * Non-instrumentable text section */ @@ -570,6 +586,7 @@ NOINSTR_TEXT \ *(.text..refcount) \ *(.ref.text) \ + TEXT_CFI_JT \ MEM_KEEP(init.text*) \ MEM_KEEP(exit.text*) \ @@ -974,7 +991,8 @@ * keep any .init_array.* sections. * https://bugs.llvm.org/show_bug.cgi?id=46478 */ -#if defined(CONFIG_GCOV_KERNEL) || defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KCSAN) +#if defined(CONFIG_GCOV_KERNEL) || defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KCSAN) || \ + defined(CONFIG_CFI_CLANG) # ifdef CONFIG_CONSTRUCTORS # define SANITIZER_DISCARDS \ *(.eh_frame) diff --git a/include/linux/cfi.h b/include/linux/cfi.h new file mode 100644 index 000000000000..879744aaa6e0 --- /dev/null +++ b/include/linux/cfi.h @@ -0,0 +1,41 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Clang Control Flow Integrity (CFI) support. + * + * Copyright (C) 2021 Google LLC + */ +#ifndef _LINUX_CFI_H +#define _LINUX_CFI_H + +#ifdef CONFIG_CFI_CLANG +typedef void (*cfi_check_fn)(uint64_t id, void *ptr, void *diag); + +/* Compiler-generated function in each module, and the kernel */ +extern void __cfi_check(uint64_t id, void *ptr, void *diag); + +/* + * Force the compiler to generate a CFI jump table entry for a function + * and store the jump table address to __cfi_jt_. + */ +#define __CFI_ADDRESSABLE(fn, __attr) \ + const void *__cfi_jt_ ## fn __visible __attr = (void *)&fn + +#ifdef CONFIG_CFI_CLANG_SHADOW + +extern void cfi_module_add(struct module *mod, unsigned long base_addr); +extern void cfi_module_remove(struct module *mod, unsigned long base_addr); + +#else + +static inline void cfi_module_add(struct module *mod, unsigned long base_addr) {} +static inline void cfi_module_remove(struct module *mod, unsigned long base_addr) {} + +#endif /* CONFIG_CFI_CLANG_SHADOW */ + +#else /* !CONFIG_CFI_CLANG */ + +#define __CFI_ADDRESSABLE(fn, __attr) + +#endif /* CONFIG_CFI_CLANG */ + +#endif /* _LINUX_CFI_H */ diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index d217c382b02d..6de9d0c9377e 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -61,3 +61,5 @@ #if __has_feature(shadow_call_stack) # define __noscs __attribute__((__no_sanitize__("shadow-call-stack"))) #endif + +#define __nocfi __attribute__((__no_sanitize__("cfi"))) diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index e5dd5a4ae946..796935a37e37 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -242,6 +242,10 @@ struct ftrace_likely_data { # define __noscs #endif +#ifndef __nocfi +# define __nocfi +#endif + #ifndef asm_volatile_goto #define asm_volatile_goto(x...) asm goto(x) #endif diff --git a/include/linux/init.h b/include/linux/init.h index 31f54de58429..b3ea15348fbd 100644 --- a/include/linux/init.h +++ b/include/linux/init.h @@ -47,7 +47,7 @@ /* These are for everybody (although not all archs will actually discard it in modules) */ -#define __init __section(".init.text") __cold __latent_entropy __noinitretpoline +#define __init __section(".init.text") __cold __latent_entropy __noinitretpoline __nocfi #define __initdata __section(".init.data") #define __initconst __section(".init.rodata") #define __exitdata __section(".exit.data") diff --git a/include/linux/module.h b/include/linux/module.h index da4b6fbe8ebe..8100bb477d86 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -26,6 +26,7 @@ #include #include #include +#include #include #include @@ -128,13 +129,17 @@ extern void cleanup_module(void); #define module_init(initfn) \ static inline initcall_t __maybe_unused __inittest(void) \ { return initfn; } \ - int init_module(void) __copy(initfn) __attribute__((alias(#initfn))); + int init_module(void) __copy(initfn) \ + __attribute__((alias(#initfn))); \ + __CFI_ADDRESSABLE(init_module, __initdata); /* This is only required if you want to be unloadable. */ #define module_exit(exitfn) \ static inline exitcall_t __maybe_unused __exittest(void) \ { return exitfn; } \ - void cleanup_module(void) __copy(exitfn) __attribute__((alias(#exitfn))); + void cleanup_module(void) __copy(exitfn) \ + __attribute__((alias(#exitfn))); \ + __CFI_ADDRESSABLE(cleanup_module, __exitdata); #endif @@ -376,6 +381,10 @@ struct module { const s32 *crcs; unsigned int num_syms; +#ifdef CONFIG_CFI_CLANG + cfi_check_fn cfi_check; +#endif + /* Kernel parameters. */ #ifdef CONFIG_SYSFS struct mutex param_lock; diff --git a/init/Kconfig b/init/Kconfig index 5f5c776ef192..5babea38e346 100644 --- a/init/Kconfig +++ b/init/Kconfig @@ -2296,7 +2296,7 @@ endif # MODULES config MODULES_TREE_LOOKUP def_bool y - depends on PERF_EVENTS || TRACING + depends on PERF_EVENTS || TRACING || CFI_CLANG config INIT_ALL_POSSIBLE bool diff --git a/kernel/Makefile b/kernel/Makefile index 320f1f3941b7..e8a6715f38dc 100644 --- a/kernel/Makefile +++ b/kernel/Makefile @@ -41,6 +41,9 @@ KCSAN_SANITIZE_kcov.o := n UBSAN_SANITIZE_kcov.o := n CFLAGS_kcov.o := $(call cc-option, -fno-conserve-stack) -fno-stack-protector +# Don't instrument error handlers +CFLAGS_REMOVE_cfi.o := $(CC_FLAGS_CFI) + obj-y += sched/ obj-y += locking/ obj-y += power/ @@ -111,6 +114,7 @@ obj-$(CONFIG_BPF) += bpf/ obj-$(CONFIG_KCSAN) += kcsan/ obj-$(CONFIG_SHADOW_CALL_STACK) += scs.o obj-$(CONFIG_HAVE_STATIC_CALL_INLINE) += static_call.o +obj-$(CONFIG_CFI_CLANG) += cfi.o obj-$(CONFIG_PERF_EVENTS) += events/ diff --git a/kernel/cfi.c b/kernel/cfi.c new file mode 100644 index 000000000000..e17a56639766 --- /dev/null +++ b/kernel/cfi.c @@ -0,0 +1,329 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Clang Control Flow Integrity (CFI) error and slowpath handling. + * + * Copyright (C) 2021 Google LLC + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +/* Compiler-defined handler names */ +#ifdef CONFIG_CFI_PERMISSIVE +#define cfi_failure_handler __ubsan_handle_cfi_check_fail +#else +#define cfi_failure_handler __ubsan_handle_cfi_check_fail_abort +#endif + +static inline void handle_cfi_failure(void *ptr) +{ + if (IS_ENABLED(CONFIG_CFI_PERMISSIVE)) + WARN_RATELIMIT(1, "CFI failure (target: %pS):\n", ptr); + else + panic("CFI failure (target: %pS)\n", ptr); +} + +#ifdef CONFIG_MODULES +#ifdef CONFIG_CFI_CLANG_SHADOW +/* + * Index type. A 16-bit index can address at most (2^16)-2 pages (taking + * into account SHADOW_INVALID), i.e. ~256M with 4k pages. + */ +typedef u16 shadow_t; +#define SHADOW_INVALID ((shadow_t)~0UL) + +struct cfi_shadow { + /* Page index for the beginning of the shadow */ + unsigned long base; + /* An array of __cfi_check locations (as indices to the shadow) */ + shadow_t shadow[1]; +} __packed; + +/* + * The shadow covers ~128M from the beginning of the module region. If + * the region is larger, we fall back to __module_address for the rest. + */ +#define __SHADOW_RANGE (_UL(SZ_128M) >> PAGE_SHIFT) + +/* The in-memory size of struct cfi_shadow, always at least one page */ +#define __SHADOW_PAGES ((__SHADOW_RANGE * sizeof(shadow_t)) >> PAGE_SHIFT) +#define SHADOW_PAGES max(1UL, __SHADOW_PAGES) +#define SHADOW_SIZE (SHADOW_PAGES << PAGE_SHIFT) + +/* The actual size of the shadow array, minus metadata */ +#define SHADOW_ARR_SIZE (SHADOW_SIZE - offsetof(struct cfi_shadow, shadow)) +#define SHADOW_ARR_SLOTS (SHADOW_ARR_SIZE / sizeof(shadow_t)) + +static DEFINE_MUTEX(shadow_update_lock); +static struct cfi_shadow __rcu *cfi_shadow __read_mostly; + +/* Returns the index in the shadow for the given address */ +static inline int ptr_to_shadow(const struct cfi_shadow *s, unsigned long ptr) +{ + unsigned long index; + unsigned long page = ptr >> PAGE_SHIFT; + + if (unlikely(page < s->base)) + return -1; /* Outside of module area */ + + index = page - s->base; + + if (index >= SHADOW_ARR_SLOTS) + return -1; /* Cannot be addressed with shadow */ + + return (int)index; +} + +/* Returns the page address for an index in the shadow */ +static inline unsigned long shadow_to_ptr(const struct cfi_shadow *s, + int index) +{ + if (unlikely(index < 0 || index >= SHADOW_ARR_SLOTS)) + return 0; + + return (s->base + index) << PAGE_SHIFT; +} + +/* Returns the __cfi_check function address for the given shadow location */ +static inline unsigned long shadow_to_check_fn(const struct cfi_shadow *s, + int index) +{ + if (unlikely(index < 0 || index >= SHADOW_ARR_SLOTS)) + return 0; + + if (unlikely(s->shadow[index] == SHADOW_INVALID)) + return 0; + + /* __cfi_check is always page aligned */ + return (s->base + s->shadow[index]) << PAGE_SHIFT; +} + +static void prepare_next_shadow(const struct cfi_shadow __rcu *prev, + struct cfi_shadow *next) +{ + int i, index, check; + + /* Mark everything invalid */ + memset(next->shadow, 0xFF, SHADOW_ARR_SIZE); + + if (!prev) + return; /* No previous shadow */ + + /* If the base address didn't change, an update is not needed */ + if (prev->base == next->base) { + memcpy(next->shadow, prev->shadow, SHADOW_ARR_SIZE); + return; + } + + /* Convert the previous shadow to the new address range */ + for (i = 0; i < SHADOW_ARR_SLOTS; ++i) { + if (prev->shadow[i] == SHADOW_INVALID) + continue; + + index = ptr_to_shadow(next, shadow_to_ptr(prev, i)); + if (index < 0) + continue; + + check = ptr_to_shadow(next, + shadow_to_check_fn(prev, prev->shadow[i])); + if (check < 0) + continue; + + next->shadow[index] = (shadow_t)check; + } +} + +static void add_module_to_shadow(struct cfi_shadow *s, struct module *mod, + unsigned long min_addr, unsigned long max_addr) +{ + int check_index; + unsigned long check = (unsigned long)mod->cfi_check; + unsigned long ptr; + + if (unlikely(!PAGE_ALIGNED(check))) { + pr_warn("cfi: not using shadow for module %s\n", mod->name); + return; + } + + check_index = ptr_to_shadow(s, check); + if (check_index < 0) + return; /* Module not addressable with shadow */ + + /* For each page, store the check function index in the shadow */ + for (ptr = min_addr; ptr <= max_addr; ptr += PAGE_SIZE) { + int index = ptr_to_shadow(s, ptr); + + if (index >= 0) { + /* Each page must only contain one module */ + WARN_ON_ONCE(s->shadow[index] != SHADOW_INVALID); + s->shadow[index] = (shadow_t)check_index; + } + } +} + +static void remove_module_from_shadow(struct cfi_shadow *s, struct module *mod, + unsigned long min_addr, unsigned long max_addr) +{ + unsigned long ptr; + + for (ptr = min_addr; ptr <= max_addr; ptr += PAGE_SIZE) { + int index = ptr_to_shadow(s, ptr); + + if (index >= 0) + s->shadow[index] = SHADOW_INVALID; + } +} + +typedef void (*update_shadow_fn)(struct cfi_shadow *, struct module *, + unsigned long min_addr, unsigned long max_addr); + +static void update_shadow(struct module *mod, unsigned long base_addr, + update_shadow_fn fn) +{ + struct cfi_shadow *prev; + struct cfi_shadow *next; + unsigned long min_addr, max_addr; + + next = vmalloc(SHADOW_SIZE); + + mutex_lock(&shadow_update_lock); + prev = rcu_dereference_protected(cfi_shadow, + mutex_is_locked(&shadow_update_lock)); + + if (next) { + next->base = base_addr >> PAGE_SHIFT; + prepare_next_shadow(prev, next); + + min_addr = (unsigned long)mod->core_layout.base; + max_addr = min_addr + mod->core_layout.text_size; + fn(next, mod, min_addr & PAGE_MASK, max_addr & PAGE_MASK); + + set_memory_ro((unsigned long)next, SHADOW_PAGES); + } + + rcu_assign_pointer(cfi_shadow, next); + mutex_unlock(&shadow_update_lock); + synchronize_rcu(); + + if (prev) { + set_memory_rw((unsigned long)prev, SHADOW_PAGES); + vfree(prev); + } +} + +void cfi_module_add(struct module *mod, unsigned long base_addr) +{ + update_shadow(mod, base_addr, add_module_to_shadow); +} + +void cfi_module_remove(struct module *mod, unsigned long base_addr) +{ + update_shadow(mod, base_addr, remove_module_from_shadow); +} + +static inline cfi_check_fn ptr_to_check_fn(const struct cfi_shadow __rcu *s, + unsigned long ptr) +{ + int index; + + if (unlikely(!s)) + return NULL; /* No shadow available */ + + index = ptr_to_shadow(s, ptr); + if (index < 0) + return NULL; /* Cannot be addressed with shadow */ + + return (cfi_check_fn)shadow_to_check_fn(s, index); +} + +static inline cfi_check_fn find_shadow_check_fn(unsigned long ptr) +{ + cfi_check_fn fn; + + rcu_read_lock_sched(); + fn = ptr_to_check_fn(rcu_dereference_sched(cfi_shadow), ptr); + rcu_read_unlock_sched(); + + return fn; +} + +#else /* !CONFIG_CFI_CLANG_SHADOW */ + +static inline cfi_check_fn find_shadow_check_fn(unsigned long ptr) +{ + return NULL; +} + +#endif /* CONFIG_CFI_CLANG_SHADOW */ + +static inline cfi_check_fn find_module_check_fn(unsigned long ptr) +{ + cfi_check_fn fn = NULL; + struct module *mod; + + rcu_read_lock_sched(); + mod = __module_address(ptr); + if (mod) + fn = mod->cfi_check; + rcu_read_unlock_sched(); + + return fn; +} + +static inline cfi_check_fn find_check_fn(unsigned long ptr) +{ + cfi_check_fn fn = NULL; + + if (is_kernel_text(ptr)) + return __cfi_check; + + /* + * Indirect call checks can happen when RCU is not watching. Both + * the shadow and __module_address use RCU, so we need to wake it + * up if necessary. + */ + RCU_NONIDLE({ + if (IS_ENABLED(CONFIG_CFI_CLANG_SHADOW)) + fn = find_shadow_check_fn(ptr); + + if (!fn) + fn = find_module_check_fn(ptr); + }); + + return fn; +} + +void __cfi_slowpath_diag(uint64_t id, void *ptr, void *diag) +{ + cfi_check_fn fn = find_check_fn((unsigned long)ptr); + + if (likely(fn)) + fn(id, ptr, diag); + else /* Don't allow unchecked modules */ + handle_cfi_failure(ptr); +} +EXPORT_SYMBOL(__cfi_slowpath_diag); + +#else /* !CONFIG_MODULES */ + +void __cfi_slowpath_diag(uint64_t id, void *ptr, void *diag) +{ + handle_cfi_failure(ptr); /* No modules */ +} +EXPORT_SYMBOL(__cfi_slowpath_diag); + +#endif /* CONFIG_MODULES */ + +void cfi_failure_handler(void *data, void *ptr, void *vtable) +{ + handle_cfi_failure(ptr); +} +EXPORT_SYMBOL(cfi_failure_handler); diff --git a/kernel/module.c b/kernel/module.c index 30479355ab85..20fb004e7d8d 100644 --- a/kernel/module.c +++ b/kernel/module.c @@ -2146,6 +2146,8 @@ void __weak module_arch_freeing_init(struct module *mod) { } +static void cfi_cleanup(struct module *mod); + /* Free a module, remove from lists, etc. */ static void free_module(struct module *mod) { @@ -2187,6 +2189,9 @@ static void free_module(struct module *mod) synchronize_rcu(); mutex_unlock(&module_mutex); + /* Clean up CFI for the module. */ + cfi_cleanup(mod); + /* This may be empty, but that's OK */ module_arch_freeing_init(mod); module_memfree(mod->init_layout.base); @@ -3866,6 +3871,8 @@ static int unknown_module_param_cb(char *param, char *val, const char *modname, return 0; } +static void cfi_init(struct module *mod); + /* * Allocate and load the module: note that size of section 0 is always * zero, and we rely on this for optional sections. @@ -3997,6 +4004,9 @@ static int load_module(struct load_info *info, const char __user *uargs, flush_module_icache(mod); + /* Setup CFI for the module. */ + cfi_init(mod); + /* Now copy in args */ mod->args = strndup_user(uargs, ~0UL >> 1); if (IS_ERR(mod->args)) { @@ -4070,6 +4080,7 @@ static int load_module(struct load_info *info, const char __user *uargs, synchronize_rcu(); kfree(mod->args); free_arch_cleanup: + cfi_cleanup(mod); module_arch_cleanup(mod); free_modinfo: free_modinfo(mod); @@ -4415,6 +4426,38 @@ int module_kallsyms_on_each_symbol(int (*fn)(void *, const char *, #endif /* CONFIG_LIVEPATCH */ #endif /* CONFIG_KALLSYMS */ +static void cfi_init(struct module *mod) +{ +#ifdef CONFIG_CFI_CLANG + initcall_t *init; + exitcall_t *exit; + + rcu_read_lock_sched(); + mod->cfi_check = (cfi_check_fn) + find_kallsyms_symbol_value(mod, "__cfi_check"); + init = (initcall_t *) + find_kallsyms_symbol_value(mod, "__cfi_jt_init_module"); + exit = (exitcall_t *) + find_kallsyms_symbol_value(mod, "__cfi_jt_cleanup_module"); + rcu_read_unlock_sched(); + + /* Fix init/exit functions to point to the CFI jump table */ + if (init) + mod->init = *init; + if (exit) + mod->exit = *exit; + + cfi_module_add(mod, module_addr_min); +#endif +} + +static void cfi_cleanup(struct module *mod) +{ +#ifdef CONFIG_CFI_CLANG + cfi_module_remove(mod, module_addr_min); +#endif +} + /* Maximum number of characters written by module_flags() */ #define MODULE_FLAGS_BUF_SIZE (TAINT_FLAGS_COUNT + 4) diff --git a/scripts/Makefile.modfinal b/scripts/Makefile.modfinal index 735e11e9041b..dd87cea9fba7 100644 --- a/scripts/Makefile.modfinal +++ b/scripts/Makefile.modfinal @@ -23,7 +23,7 @@ modname = $(notdir $(@:.mod.o=)) part-of-module = y quiet_cmd_cc_o_c = CC [M] $@ - cmd_cc_o_c = $(CC) $(c_flags) -c -o $@ $< + cmd_cc_o_c = $(CC) $(filter-out $(CC_FLAGS_CFI), $(c_flags)) -c -o $@ $< %.mod.o: %.mod.c FORCE $(call if_changed_dep,cc_o_c) From patchwork Tue Mar 23 20:39:31 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159337 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6C927C433DB for ; Tue, 23 Mar 2021 20:41:54 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B69EF619BA for ; Tue, 23 Mar 2021 20:41:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B69EF619BA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1adTffdfLMDyHhngyP8HVWeI8H8uSxm0uZqAbyGjakk=; b=UXxms/w04t4Lc+ BIUhj5n8MR7QnrAM+cezd7UhDioHxVL5ztvpg/0aVPpFolJGsoUsAma+DKgHfivRZyxO8NAnQVtNo 2Jk73TzkPGmvSvsvgT9imSvSdZC1n5j8ANJM5fGOvV0+JooDRh4YebFWrgfe15ufjVvMwQtK4U8t3 +SxucTXHhZSSZfJ8PPfGptqDWWraZO6HEY2mUVhx/fCN85Eqwzhm27aYKiQZwqMlFyFjsYj9+EcHY 8ZAkQ4vw2vPoUeV2g4HjF+nRgxl7R9bZBw29tvv3LgZk29IjmMRrvzjMWaFCtMZquMY5L7tLMwIge BaGhz7+5D0T8pzif5d9Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnp0-00Fg8F-PJ; Tue, 23 Mar 2021 20:40:15 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnog-00Fg47-Am for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:39:56 +0000 Received: by mail-yb1-xb49.google.com with SMTP id x8so3860697ybo.6 for ; Tue, 23 Mar 2021 13:39:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=bTCEWvQMjkTW/eVBx3Fi28gIb6vhFXkJKuQZd7egWL4=; b=Mvn7lVq7e94Xs9OllSd9NAojDT1Q4suOxQi12d+B51KtOKIdMx0eF7tdNTA4iGEpNc ZT4DHBgg761FglHw7pwMEYDTT1Ecj7P/xF38j+p9TKtZbkyqcsGDtFE/RHzgKWZl+VHO uCsdIPhwwFaLxjgvQ6z41fx3DrWVbQUxBcWiGwtvTfBRpGu+zwZz1oFLdNIu/A2tGpWz P0ivEwPNEDS/3rKoR7F5Z2X623Tsi3/kfzJf/dJCiIqTQEt8dnId9DYrNflKjRBmYsEY 3yjnIOKkQ8kHE6WAEa51Dbz5jOPibBaaQiBVe2nGs+TAM55eM5uQVd3zyU422J7qChbA V5HQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=bTCEWvQMjkTW/eVBx3Fi28gIb6vhFXkJKuQZd7egWL4=; b=McwjEdbGPPKFvkQZVrhrmuQrVHhHWy63pKMzDAItcyYqrcPO6xd3uv0148lFkGlIbz 51fK96dnF3gnzjEQeBMl4o1r4fqX28x2ReNuHrLfuhrVfXAPFSktOpgD69/+lZpuWnoC rowlQIo9MfMkA4UJnBH7PfSadhBY4We82cpxU9MpJ7dak2FcoUJWtmXPb1xTRNLjQC/2 dx4t/YIrgaB/hIk4eG2KKmXnvuqOnhK0rSpHcuhEgleAd7QHRWtALkku9nDZzUrSy5+q 653rnJERTpNJ4qh/l8/c9assVDcfxt2Pnr6FPzLaHiV4/NEdOPxmcdcRXl71kK1De0VE SACw== X-Gm-Message-State: AOAM530xlbf4Hv2q1AkpK1ADgG7uV64YsOsh9/zVIcc/NQRLGr4lhBXb IbI+xBLHNS+jzkQKgmN7jt1L6yqlihQMjX896Gg= X-Google-Smtp-Source: ABdhPJxwOPrsV/lppntljv3ZRk7U60aDtP04A2wkg90aE1+shnozpf82bcMb0fGuvfEtjKLsVJLLsrwjtM8144LkmFo= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a25:3ac1:: with SMTP id h184mr34203yba.503.1616531992519; Tue, 23 Mar 2021 13:39:52 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:31 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-3-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 02/17] cfi: add __cficanonical From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_203954_560851_E305F185 X-CRM114-Status: GOOD ( 18.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With CONFIG_CFI_CLANG, the compiler replaces a function address taken in C code with the address of a local jump table entry, which passes runtime indirect call checks. However, the compiler won't replace addresses taken in assembly code, which will result in a CFI failure if we later jump to such an address in instrumented C code. The code generated for the non-canonical jump table looks this: : /* In C, &noncanonical points here */ jmp noncanonical ... : /* function body */ ... This change adds the __cficanonical attribute, which tells the compiler to use a canonical jump table for the function instead. This means the compiler will rename the actual function to .cfi and points the original symbol to the jump table entry instead: : /* jump table entry */ jmp canonical.cfi ... : /* function body */ ... As a result, the address taken in assembly, or other non-instrumented code always points to the jump table and therefore, can be used for indirect calls in instrumented code without tripping CFI checks. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook Acked-by: Bjorn Helgaas # pci.h --- include/linux/compiler-clang.h | 1 + include/linux/compiler_types.h | 4 ++++ include/linux/init.h | 4 ++-- include/linux/pci.h | 4 ++-- 4 files changed, 9 insertions(+), 4 deletions(-) diff --git a/include/linux/compiler-clang.h b/include/linux/compiler-clang.h index 6de9d0c9377e..adbe76b203e2 100644 --- a/include/linux/compiler-clang.h +++ b/include/linux/compiler-clang.h @@ -63,3 +63,4 @@ #endif #define __nocfi __attribute__((__no_sanitize__("cfi"))) +#define __cficanonical __attribute__((__cfi_canonical_jump_table__)) diff --git a/include/linux/compiler_types.h b/include/linux/compiler_types.h index 796935a37e37..d29bda7f6ebd 100644 --- a/include/linux/compiler_types.h +++ b/include/linux/compiler_types.h @@ -246,6 +246,10 @@ struct ftrace_likely_data { # define __nocfi #endif +#ifndef __cficanonical +# define __cficanonical +#endif + #ifndef asm_volatile_goto #define asm_volatile_goto(x...) asm goto(x) #endif diff --git a/include/linux/init.h b/include/linux/init.h index b3ea15348fbd..045ad1650ed1 100644 --- a/include/linux/init.h +++ b/include/linux/init.h @@ -220,8 +220,8 @@ extern bool initcall_debug; __initcall_name(initstub, __iid, id) #define __define_initcall_stub(__stub, fn) \ - int __init __stub(void); \ - int __init __stub(void) \ + int __init __cficanonical __stub(void); \ + int __init __cficanonical __stub(void) \ { \ return fn(); \ } \ diff --git a/include/linux/pci.h b/include/linux/pci.h index 86c799c97b77..39684b72db91 100644 --- a/include/linux/pci.h +++ b/include/linux/pci.h @@ -1944,8 +1944,8 @@ enum pci_fixup_pass { #ifdef CONFIG_LTO_CLANG #define __DECLARE_PCI_FIXUP_SECTION(sec, name, vendor, device, class, \ class_shift, hook, stub) \ - void stub(struct pci_dev *dev); \ - void stub(struct pci_dev *dev) \ + void __cficanonical stub(struct pci_dev *dev); \ + void __cficanonical stub(struct pci_dev *dev) \ { \ hook(dev); \ } \ From patchwork Tue Mar 23 20:39:32 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159353 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 405D5C433DB for ; Tue, 23 Mar 2021 20:43:23 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8868861574 for ; Tue, 23 Mar 2021 20:43:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8868861574 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=tn/cx0ZeyJk0WIvHqUNedYp9YqL1wBDyAcy+52luO0I=; b=YW6Cu1/K35iRGq cyml+D8u/xigH60tyFMBkKmbBTixqC0xei1NJc6fyP5dxvGixcbqfCnN5FFv/XdX9FRqvlEnyWd72 Q5RCXTIGc3M4X7nS5akBEdTNgWqoImfJoTZaEuFtsHkGIkHH6KpSTgKRXkUaZ+QM6GjYsICAI3+V7 nLoNPNrGamowcOaDmsvs+WR9OZ/zXJU8kJJ2WoLhvxPolTCTEIvdO5mnSsrbTJb6MRy3lQ01s61k2 QBuC0dWs1q/8EB2ppL1LEjKZKtkjLE/QnXl9O865/nFakkyPhJXn+p3svGySQphaW7U2Et0EdLtf/ os3K9LEnby3w0Wqc8tQQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnpH-00FgC9-2e; Tue, 23 Mar 2021 20:40:31 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnoi-00Fg4S-9C for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:39:57 +0000 Received: by mail-qk1-x74a.google.com with SMTP id a1so125818qkn.11 for ; Tue, 23 Mar 2021 13:39:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=PeAEi4QdQeJS0R2WYWNNntX7QF/OnwnEUYEO8FB/zZI=; b=ME4u5UjKX6YIE3uv5oyrvvXZZ3jDHKxvg0e+6aqMZKUztdYp7UH3vHRfgdQzxCN4zy AMR5XMoxvqeXVqyIlJn+hPFAxZL4TQMv8yXuE2A03sfzLj0rDMM9qpT6OZbwGvww9Mv8 grV2XE0qf126qqR9VAHX70l9XILxGlcga68lUIyRIcjYojuoLcnDWDSs1xGhji/tepMG PBfp9Zvp0mqwZ8XFmTGrrGrlAaTnxT7M5AW6ANe2fAcqA7CEXDKc4DyDrxO6kIuRFYR5 fLVqRiEV3Wwjl4e92Jo79o5RUw9447jw9S1TrpuirFvX+vPBTTSo2yyfDiUXeMVwmetm m2dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=PeAEi4QdQeJS0R2WYWNNntX7QF/OnwnEUYEO8FB/zZI=; b=K2BwtvlryBGCH/P1l2pYTgcWvJwzN+r5xVe/1Me/RHcP5eMPmb+nqmwRK0p01/rW4g 13XH90+5QfagRjQMY7HK7xkJsiJ9/zT8Q/Ob1yZsqf1Jy+Zie7Bv9hjpM3LzjPOKKvqf MuvKY3aEE0fStOkIK1r+JkWMnHUfvf09uDDNZSicl216s3OGiK0Dx1207hLyYUe7wVRK /G7L0tjXY9h1Qj5Wd6gOfdaj9U7FjqnzlyQCUNwyX1OByAKg4J/6mY639ZpexCdOdt/V 5E9OheZvrwI4xIGpKTCvSAXG10wTAh950DjGZaftISS4Z2vYUTLF1Db4flHPy/zmzHCT sVHA== X-Gm-Message-State: AOAM5314hj3nqj53qMWat2ZAm+ylgdCuS8x0eojZ1hX9hbrodB0wwpmT RZVHAVQaLH323zJrFZ9HTXztU8SdSJWmmJJjxuI= X-Google-Smtp-Source: ABdhPJwlRHGVBhUTH+DIN3gqF/+urMKSoa77/A4cj7xtmd6a4b4sdc/0r2jdYr0i9s2c6CcWzw1XB1vTRyoJLpGvhpA= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a05:6214:f27:: with SMTP id iw7mr6736714qvb.50.1616531994399; Tue, 23 Mar 2021 13:39:54 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:32 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-4-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 03/17] mm: add generic __va_function and __pa_function macros From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_203956_356428_3CC44B6D X-CRM114-Status: GOOD ( 13.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With CONFIG_CFI_CLANG, the compiler replaces function addresses in instrumented C code with jump table addresses. This means that __pa_symbol(function) returns the physical address of the jump table entry instead of the actual function, which may not work as the jump table code will immediately jump to a virtual address that may not be mapped. To avoid this address space confusion, this change adds generic definitions for __va_function and __pa_function, which architectures that support CFI can override. The typical implementation of the __va_function macro would use inline assembly to take the function address, which avoids compiler instrumentation. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- include/linux/mm.h | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 64a71bf20536..a0d285cd59ce 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -116,6 +116,14 @@ extern int mmap_rnd_compat_bits __read_mostly; #define __pa_symbol(x) __pa(RELOC_HIDE((unsigned long)(x), 0)) #endif +#ifndef __va_function +#define __va_function(x) (x) +#endif + +#ifndef __pa_function +#define __pa_function(x) __pa_symbol(__va_function(x)) +#endif + #ifndef page_to_virt #define page_to_virt(x) __va(PFN_PHYS(page_to_pfn(x))) #endif From patchwork Tue Mar 23 20:39:33 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159341 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8AC77C433C1 for ; Tue, 23 Mar 2021 20:42:19 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D0DB061574 for ; Tue, 23 Mar 2021 20:42:18 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D0DB061574 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=820Bz3PdjKX+3P2dDUpcRAyIKNxhhwIpwo3S6Ex0Am4=; b=FvSbLUdhIr91dX BYsrKucnwIQZu4nbVaAHiI9xvdtN4ekoYRX0rBGh/bFDMd10N3fR+vuP5yTWxi4cMVADUo1GVWcW3 +NcwS9rjJwguGMbvCxF5mfGPTrrB9Eroir7T8eS3/IqEcTEWqNRYrZ4RXMr07pPHEmAR6uu0Yu+GS RW8X+xJNKybCrslkvyCI33SJsgxN6n73tVdNMhkVso/CZ0t/ofHjdBC4N4CRRHDCPkGNYK8Q0mJqf M8Ep/sEdEpIXyN5xUITKl++ZQZidEn22WhjIPMEs5q3SS03rAJrj4g6vM68ZFsnthGQwpxJH6IaqQ R7Q+CTe54SjB0UL1rlgg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnpN-00FgDS-Vq; Tue, 23 Mar 2021 20:40:40 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnok-00Fg4s-0T for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:00 +0000 Received: by mail-qt1-x849.google.com with SMTP id v18so1985955qtx.0 for ; Tue, 23 Mar 2021 13:39:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=2eJKl2P+mphx8MXuY5eZ3j0r297zj+fSj+T5WKlAy0E=; b=F7DyYoTINGuplDxZdhovljaDSjsbsbsHDaccNJNpVFG1uHl0pH2Jb8ZGxI9pRViGzn P9bXGOaY4DL6wqzXTQkCjWG3FbAPFgW9yZwRM/QptFOt0nz84XtEbqR1NLBkEGidZJa3 CrJkw9qpHdhwZrMFRZHgVsRqmb2+bUfA3erzvVzdp42rL0vgo1/PfyIuO8eBVtII3M96 zF220AVy6U0HEFNIfOCZZan85P2t79rpIjJGqHqbbixIsFbkx3V9i1GVLeV4Nyw/Kyqe 1GNwZ/VIvbro/fYpjQY24QiS/HC8Qnb62x6xeycl0QxRVHSbtfzTj1X67bnIdj4QhdeH nwSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=2eJKl2P+mphx8MXuY5eZ3j0r297zj+fSj+T5WKlAy0E=; b=cNHA1vRUpDnxAg79hLhdApR/rb3ZSUpH7TBHOrg1wAZ5v6w8gi8vqs97veJwGqmZEo iQAXw7p3vPbxoIoIzdbM2wrBXQ9vSMsWnJ8ORimtBV2ogMUiZYNk3Ded+9ulmqGBwbmk l3h5tBqNOe7gubNNwGaNXHRmwk35tAjHOVFC/KidpjT+h119TL8bbCZQ4Dlup4ItixTS tWnWh4FEMGuNw+UsF/KpkJzQMlvoAkJJfRUqIAdkI1pUikMFv1c4rAzdikC3ZdnmdoBx fL/mp6XtOZ0Nrc72c5efQx1KhxmbR61HXVTR0BI1CabwUYb7YHW7ynLFukHN4ZllLWbT i+WA== X-Gm-Message-State: AOAM532GUW1tE41rmdQiQqwBa47H++ktQBKnb+9idwEQK9Sbgelo5RTg AeaY7wznF53usQ1uhe9uFIFwaUYphWiyItmTweY= X-Google-Smtp-Source: ABdhPJxkrzfmdWRBTsfApsBgU0/vtdyh9mzsMw1+qzCVhW8+YL7dn0iZkK/RRt+cwJ1LZa5E4YW9O+Nm9AJq6lZMkTQ= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a0c:ea4b:: with SMTP id u11mr129682qvp.43.1616531996172; Tue, 23 Mar 2021 13:39:56 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:33 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-5-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 04/17] module: ensure __cfi_check alignment From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_203958_410863_B2C9646B X-CRM114-Status: GOOD ( 13.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org CONFIG_CFI_CLANG_SHADOW assumes the __cfi_check() function is page aligned and at the beginning of the .text section. While Clang would normally align the function correctly, it fails to do so for modules with no executable code. This change ensures the correct __cfi_check() location and alignment. It also discards the .eh_frame section, which Clang can generate with certain sanitizers, such as CFI. Link: https://bugs.llvm.org/show_bug.cgi?id=46293 Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook Acked-by: Jessica Yu --- scripts/module.lds.S | 20 +++++++++++++++++++- 1 file changed, 19 insertions(+), 1 deletion(-) diff --git a/scripts/module.lds.S b/scripts/module.lds.S index 168cd27e6122..2ba9e5ce71df 100644 --- a/scripts/module.lds.S +++ b/scripts/module.lds.S @@ -3,10 +3,21 @@ * Archs are free to supply their own linker scripts. ld will * combine them automatically. */ +#include + +#ifdef CONFIG_CFI_CLANG +# define ALIGN_CFI ALIGN(PAGE_SIZE) +# define SANITIZER_DISCARDS *(.eh_frame) +#else +# define ALIGN_CFI +# define SANITIZER_DISCARDS +#endif + SECTIONS { /DISCARD/ : { *(.discard) *(.discard.*) + SANITIZER_DISCARDS } __ksymtab 0 : { *(SORT(___ksymtab+*)) } @@ -40,7 +51,14 @@ SECTIONS { *(.rodata..L*) } - .text : { *(.text .text.[0-9a-zA-Z_]*) } + /* + * With CONFIG_CFI_CLANG, we assume __cfi_check is at the beginning + * of the .text section, and is aligned to PAGE_SIZE. + */ + .text : ALIGN_CFI { + *(.text.__cfi_check) + *(.text .text.[0-9a-zA-Z_]* .text..L.cfi*) + } } /* bring in arch-specific sections */ From patchwork Tue Mar 23 20:39:34 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159345 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6BAE0C433E0 for ; Tue, 23 Mar 2021 20:42:39 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E2B5D61574 for ; Tue, 23 Mar 2021 20:42:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E2B5D61574 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xAJ6voTohL+zYkDZUkLJsdRZIhPlcrUtO1EVPjFa8SE=; b=LIgodAEhw78A7p TZpYrl2R9omFf9LT/AfrXFuFsOF23gSU7VM+72hJVVcgWaYt6adpkes8VQGo4yEcIOkNxaxa36WCz JKo5HY0AG5eLyjanZUL2C+ExTvD45yo1sT6V9B1g7nHjndcDxwPzmtrcY/4up7+LWOk1SeWjm8NHP m93d6CdkAc7UBa5a+38EdccA8o10M5CH1yAposphSMKYBje2PJ2VtpypLIMRtf/2GYUFSn8l1XRBD c54+mytkj7RtScWTI1NktYIOW3ld+Bw3VzEsFN4FvtWUiRU66T6SLC/juhvnjxDyEyyVLalt+HKHo APz18zF96EyCl5u7hQsg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnpd-00FgHx-6Q; Tue, 23 Mar 2021 20:40:54 +0000 Received: from mail-qk1-x74a.google.com ([2607:f8b0:4864:20::74a]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnom-00Fg5G-Si for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:02 +0000 Received: by mail-qk1-x74a.google.com with SMTP id b78so123491qkg.13 for ; Tue, 23 Mar 2021 13:39:59 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=EKy6eSpRty5bSmZbwR30NkKAGzsTESRLoXrAS0NLKzs=; b=SHiuDwK39cZpUPhp4hdoHOuGGDuRvdgVruquA5OhzT20BoTxNAsr9v4IssAUSm++Wu Gdigd5fbbNiaLy/ry4ApwbLMVFLE/K8FS9T8hHmeF+HMjwCwC1pX53Mm0BNeat42pW6A vudqk7VvICGdH7ZzkH/77tQzcGfMs8BokGif9s1iaTr4/OXPToVBkScJWmDX17YliXvx qqE/rODtG05P0dv7CE4/9UcCl0G27AKyydISoJgw84bjphFaaOWXGCFldHhaDhA0vXjv 6uLVLqt6EaTZd1uB/NSBl+SDtBiFuBudKFH8Sf5XHxGzWkbtxux1it75UPI4B3F4MoBR bzIw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=EKy6eSpRty5bSmZbwR30NkKAGzsTESRLoXrAS0NLKzs=; b=hYDCCmjD+47wcncjNBAyGdWshr6Fwr5Ml2rG32vbmMpEHgX1TR07b7TZcABdrtVTmQ YKYspUiFM3msygIOysOnU0D9EMxwcIx5wiwfiXqtqppTI3YWVfpte0v7Fj8vEaMlv99q aZw7YKd0aIDptPbhlDRSUm7XaVmf4u0XBQbErZ4K7ul71w1YpqKrCzJ+29HNemmQPjRc VR13+vtYk0FJBHUFBFG8tXoisTTWyyAXQozjWtftxXERfFFpTVwm1VPuDk//ijbuLGvd QhTAlnh4RB/mJXbCCzSYaqWbk37zQ4y3DpuHI79TeWWwIpPf70IpNV8J4GkmeUPZFXEH PSqA== X-Gm-Message-State: AOAM531oeYTMHaDuW1Oy8MeWjQivAhQyCBldbvsOPvoLHWPiay3EZtC/ s/hbwx//XsMqO4uao1ziwtBsJjhbXASRAqV1JSs= X-Google-Smtp-Source: ABdhPJzHeOXxkC8OlUHUZt0DqSgHN9t2rhpi+pE22x1dKho2TmMlfck6McMlcAxQskeb4L29PcbGG9gfenN7WPTbeOU= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:ad4:5887:: with SMTP id dz7mr198051qvb.12.1616531998043; Tue, 23 Mar 2021 13:39:58 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:34 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-6-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 05/17] workqueue: use WARN_ON_FUNCTION_MISMATCH From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204001_039175_6CC33064 X-CRM114-Status: GOOD ( 15.35 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With CONFIG_CFI_CLANG, a callback function passed to __queue_delayed_work from a module points to a jump table entry defined in the module instead of the one used in the core kernel, which breaks function address equality in this check: WARN_ON_ONCE(timer->function != delayed_work_timer_fn); Use WARN_ON_FUNCTION_MISMATCH() instead to disable the warning when CFI and modules are both enabled. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- kernel/workqueue.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/kernel/workqueue.c b/kernel/workqueue.c index 0d150da252e8..03fe07d2f39f 100644 --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -1630,7 +1630,7 @@ static void __queue_delayed_work(int cpu, struct workqueue_struct *wq, struct work_struct *work = &dwork->work; WARN_ON_ONCE(!wq); - WARN_ON_ONCE(timer->function != delayed_work_timer_fn); + WARN_ON_FUNCTION_MISMATCH(timer->function, delayed_work_timer_fn); WARN_ON_ONCE(timer_pending(timer)); WARN_ON_ONCE(!list_empty(&work->entry)); From patchwork Tue Mar 23 20:39:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id DDD4BC433DB for ; Tue, 23 Mar 2021 20:42:34 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 50A5D619BA for ; Tue, 23 Mar 2021 20:42:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 50A5D619BA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=98c/tmAaU1wF6NJzBzoWBLROEvYbS9Pwn+gE1NAGrU8=; b=rWpIBejZA9X2NC 0DVWHryXod4yR2/GcqNupRH97zifBUNyO+w10xudgBpvmnE0ALp7igZ/jWGLcrEWqOXwcF1jAAIZz kg1ch7vEFGyAtDfnv08HA8Ty06xQ8Vl5/PzZ1M0cHH9zk2fJpvAPBwtxHDkIntWgmoMCL+kwmb0Bg iGTM6G9teM6QEWzOnjCN6KtxzP5/9zXqRmU0Dkke9iElAybc1DR2oIEIR/TXiUWH7T13/CzJ3cP8t D0gyvAk+i1OXE8cxnVDk+oOqDNh2fTPJxLD8NnlqCPGO0XTpTxFZh65FdUeBl3A+XYnf4GFK97fQa 6W5Mqj/0OLTGRMkE3IDA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnpo-00FgL1-WE; Tue, 23 Mar 2021 20:41:05 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnoo-00Fg5K-Ik for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:04 +0000 Received: by mail-yb1-xb49.google.com with SMTP id v124so3797739ybc.15 for ; Tue, 23 Mar 2021 13:40:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=kAxWBGQQ6gb1U0MwyH2aGrnOUTzFGjVGvgXGkxVH/cc=; b=obb/St0ECVpiVYPjl0tFmSeAu+nwHW6a+FyVf5Y93kL0zciJ7OpXo5pz5/5qK+sMFf HVCB7lPNLLCcgrreAMPMIq89DUCTawBOwNPUPSIBya5VRwan8cbfXQx100+5xN/Qkd+t DrRy/pbSThpGxZ1QzI4eyYQAJqKt5nt14oxKeAqxjwjAb66eROjyJZpq+iXVKnIt1UeO qE3Het9/jgbk33l/79gSOA0M2UKmfHun8/qcQ5XwaA/WAB2nFQGlLlfTzyIcIhj7o/5V IAm35WHoPPYpvmYzDW1CTc/YgvwTLIfoZ+x6jtjFf4U8j7x1w0Jt0fXDFRfY/dN/7g6Y zgEA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=kAxWBGQQ6gb1U0MwyH2aGrnOUTzFGjVGvgXGkxVH/cc=; b=TgTKgPR2l/302aqHfmKfvGDPSbi5IjfYjwV+qVXtvsIf0fgKHw8GxPcYtqoO0V1hQA 375l3VxoNeCF7yaAaUhCaOU8f4RQfeI5aA+8JQw6njvTe0/y0HdncBOzXYJHC4L9+/k4 TFBTdB8Jw2aJDl7DQ8thA4sntfDVePLgQZSkjchDmM8rAQfIazDySGTyNJzgXnYQqBO5 hWl1+PbTcoQEVBSrLEaWrOvBI8h7TsbqSDNU1j+bW02WLYNJmA2bgjUItBMk3mj8gd29 iw4AM2n9fvLnwz0J7mnARUJNUdrkKkEBNbVX/gJoqpTWAXDjOt/sHDQYacJUgZRNbmMT 5d/g== X-Gm-Message-State: AOAM530Hx5VsBMoN9Ip9+AqBfprt5zb8/SiLkDHCBDj8fpKlk7jwG/tt hlKg+02eEGzHX8Gm+H0O1BquW2qNBtgzsN2Vbr8= X-Google-Smtp-Source: ABdhPJy06lzastyLa7rjZrqrKe8Kw6vypE8VYVF9BzE0zOiJRLohmqAfcSnsVJk7XWpS4yGsr/w/V3cRXxwJCIzEtGs= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a25:e008:: with SMTP id x8mr147121ybg.90.1616531999835; Tue, 23 Mar 2021 13:39:59 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:35 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-7-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 06/17] kthread: use WARN_ON_FUNCTION_MISMATCH From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204002_844529_EFAAD3C6 X-CRM114-Status: GOOD ( 14.77 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With CONFIG_CFI_CLANG, a callback function passed to __kthread_queue_delayed_work from a module points to a jump table entry defined in the module instead of the one used in the core kernel, which breaks function address equality in this check: WARN_ON_ONCE(timer->function != ktead_delayed_work_timer_fn); Use WARN_ON_FUNCTION_MISMATCH() instead to disable the warning when CFI and modules are both enabled. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- kernel/kthread.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/kernel/kthread.c b/kernel/kthread.c index 1578973c5740..a1972eba2917 100644 --- a/kernel/kthread.c +++ b/kernel/kthread.c @@ -963,7 +963,8 @@ static void __kthread_queue_delayed_work(struct kthread_worker *worker, struct timer_list *timer = &dwork->timer; struct kthread_work *work = &dwork->work; - WARN_ON_ONCE(timer->function != kthread_delayed_work_timer_fn); + WARN_ON_FUNCTION_MISMATCH(timer->function, + kthread_delayed_work_timer_fn); /* * If @delay is 0, queue @dwork->work immediately. This is for From patchwork Tue Mar 23 20:39:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159349 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8ED2CC433DB for ; Tue, 23 Mar 2021 20:42:57 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 1066C619BA for ; Tue, 23 Mar 2021 20:42:57 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 1066C619BA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=aBol3Y7NIhcZGxz+FQVROpC4xDg4Itax6spFNCK/s5c=; b=GGzK+A7CrSU3We APm42PdZGZ1Z7VLVEYWA/v4ypQAhtFxBy6duquiGJ6XaBcWuedqD1ywFJ2KCahf3H8YDtb2bIygzM gqHYp0Xu9pupFVauHrMgQvZsehy2RdJE61/zjp905TC0p9WAwV7GCxpmCqUN3VX1wRSjHc/z32rEN QWysJYyqygPNODLBmPbL6mG9TbT3meNqOEKxlFJVUKQ5cHRgEHkHyozWInSTuvJUZ8LxmuRf8tb91 glTYFd1twcVzIpYoDgZEFppPLV701W/4rRqPehV0u87ndJKmPmCpEW6eErFp3VmpDCPAsUaX2nG9n Yp15quFgn/LPMAGP8gjw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnq4-00FgPu-Jv; Tue, 23 Mar 2021 20:41:20 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnop-00Fg5u-8B for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:04 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 131so3833197ybp.16 for ; Tue, 23 Mar 2021 13:40:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=i1PxfqvHNAZaUSY0Wt2J9hLbKLkB/NragVXsh/CJXkk=; b=r3h1x+fU5lE/E/LEWwengkgZi5PRsX186ijNQ327TMBLawIZY6LBYeXoo+w52nP2gv rF5I2VWq76FFpaCoMspUWvw8qDIKJm1AsH0+oLVk3E9T+doX05a07ZFN00OhJPnaxkmu jxmLG5HZc+yt4wtJhjo3sZUXCMUwF3pZoczzJSo1cNByG2VVWNmY/KAxOvwH7o3TUBYH bGowaPWnFDFXQgdWtgWmUwuCqSOx5f/zwc5Gwk2eWSiXszhRE499K2081dnK0xsNlJj1 /F+vI9niI/hzqmXXXuXFw6fn+8OOPf3ryNNVi941QfjddB4grHq+9+iY1IfzHYtwau73 lqqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=i1PxfqvHNAZaUSY0Wt2J9hLbKLkB/NragVXsh/CJXkk=; b=JMl+qbcULV/jKN70A4ssNBujABZV7HIRWQGjxaG5spQC6yNVN+77otnFhMvym9fM2Z 2zxvXTRAdDnDD3RNQ72d7vYTdLScPJ3kvXGB+7GIur6RhLLNC49WsUjglXYwGxoSG4Gp R8VLiuVyj5VcbPR/DxLvKoLO+4svP8G/+Nad9wWqkW9Egs3lg9aMty/y+KrPDMpCOot2 f6ivKQqitPW6RZiXUTC+AwIjp6bqDTz2ZC1rVbgsTn4HQl7abayR3X7gRYN8jbvNAaH4 fgWMGrqH/11YRH2AaWyK27UqK2W351odgvNCCAtN6MQRY3whF5+EGT+JcIjMdf8ZlJJs 5P8g== X-Gm-Message-State: AOAM5304R/g6n3kkGmH+EEn9z2XaDGNXerdTI5mkXYWc7ampCbAk5Ul4 ywo3nqx46SiIdAYGqB0MirZTx8KqmzhhJaZn3Pk= X-Google-Smtp-Source: ABdhPJxSGEWBzQB/5ovI2vIjo7a1D7A+5mPzIG6DfuCWCyGPkoV9bpBY6tsZB7tVHHd6ltUsB02iXZ3BUgMvr1I18/M= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a25:a166:: with SMTP id z93mr111661ybh.354.1616532001793; Tue, 23 Mar 2021 13:40:01 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:36 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-8-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 07/17] kallsyms: strip ThinLTO hashes from static functions From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204003_385957_85454C51 X-CRM114-Status: GOOD ( 17.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With CONFIG_CFI_CLANG and ThinLTO, Clang appends a hash to the names of all static functions not marked __used. This can break userspace tools that don't expect the function name to change, so strip out the hash from the output. Suggested-by: Jack Pham Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- kernel/kallsyms.c | 55 ++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 50 insertions(+), 5 deletions(-) diff --git a/kernel/kallsyms.c b/kernel/kallsyms.c index 8043a90aa50e..c851ca0ed357 100644 --- a/kernel/kallsyms.c +++ b/kernel/kallsyms.c @@ -161,6 +161,27 @@ static unsigned long kallsyms_sym_address(int idx) return kallsyms_relative_base - 1 - kallsyms_offsets[idx]; } +#if defined(CONFIG_CFI_CLANG) && defined(CONFIG_LTO_CLANG_THIN) +/* + * LLVM appends a hash to static function names when ThinLTO and CFI are + * both enabled, i.e. foo() becomes foo$707af9a22804d33c81801f27dcfe489b. + * This causes confusion and potentially breaks user space tools, so we + * strip the suffix from expanded symbol names. + */ +static inline bool cleanup_symbol_name(char *s) +{ + char *res; + + res = strrchr(s, '$'); + if (res) + *res = '\0'; + + return res != NULL; +} +#else +static inline bool cleanup_symbol_name(char *s) { return false; } +#endif + /* Lookup the address for this symbol. Returns 0 if not found. */ unsigned long kallsyms_lookup_name(const char *name) { @@ -173,6 +194,9 @@ unsigned long kallsyms_lookup_name(const char *name) if (strcmp(namebuf, name) == 0) return kallsyms_sym_address(i); + + if (cleanup_symbol_name(namebuf) && strcmp(namebuf, name) == 0) + return kallsyms_sym_address(i); } return module_kallsyms_lookup_name(name); } @@ -303,7 +327,9 @@ const char *kallsyms_lookup(unsigned long addr, namebuf, KSYM_NAME_LEN); if (modname) *modname = NULL; - return namebuf; + + ret = namebuf; + goto found; } /* See if it's in a module or a BPF JITed image. */ @@ -316,11 +342,16 @@ const char *kallsyms_lookup(unsigned long addr, if (!ret) ret = ftrace_mod_address_lookup(addr, symbolsize, offset, modname, namebuf); + +found: + cleanup_symbol_name(namebuf); return ret; } int lookup_symbol_name(unsigned long addr, char *symname) { + int res; + symname[0] = '\0'; symname[KSYM_NAME_LEN - 1] = '\0'; @@ -331,15 +362,23 @@ int lookup_symbol_name(unsigned long addr, char *symname) /* Grab name */ kallsyms_expand_symbol(get_symbol_offset(pos), symname, KSYM_NAME_LEN); - return 0; + goto found; } /* See if it's in a module. */ - return lookup_module_symbol_name(addr, symname); + res = lookup_module_symbol_name(addr, symname); + if (res) + return res; + +found: + cleanup_symbol_name(symname); + return 0; } int lookup_symbol_attrs(unsigned long addr, unsigned long *size, unsigned long *offset, char *modname, char *name) { + int res; + name[0] = '\0'; name[KSYM_NAME_LEN - 1] = '\0'; @@ -351,10 +390,16 @@ int lookup_symbol_attrs(unsigned long addr, unsigned long *size, kallsyms_expand_symbol(get_symbol_offset(pos), name, KSYM_NAME_LEN); modname[0] = '\0'; - return 0; + goto found; } /* See if it's in a module. */ - return lookup_module_symbol_attrs(addr, size, offset, modname, name); + res = lookup_module_symbol_attrs(addr, size, offset, modname, name); + if (res) + return res; + +found: + cleanup_symbol_name(name); + return 0; } /* Look up a kernel symbol and return it in a text buffer. */ From patchwork Tue Mar 23 20:39:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159351 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B0DF0C433C1 for ; Tue, 23 Mar 2021 20:43:15 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DCFE061574 for ; Tue, 23 Mar 2021 20:43:14 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DCFE061574 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=3uZizOqdcvm3JfAtHbe+lPniyqdpVDw6FlcrqGCkb8k=; b=ZPMIW1woMibqb6 NfzWi6rA2EBMJODP7J6UDQHw+2dgtDSiCpgFiY7g+UL5lWyFmvg93r9uwoih01w8Hl559rUorpP0t B5GE8eq+o34RSk9WcyPuPqgu6UBuTf0APNFh1GKPNa3OyWrNP1DFzUai5fwm1qtCnIP/XjRW+XmbE 8Ysm0hiRa1ygzl/jxySUHUpavWr8zGkZFsn0rJLfAwLhSnYehUtaSyaklmzM8Ms5fHbgYRq2jLHf2 gvQseX8wZY1s7bSukxFZeKhsKhaR+B7W0f3MC5SMt9P0fElxlLnhSoV/pyyJ1/KZ/1iwUzxBmDBWB zkggLUBlKRkBAF2Cib/Q==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnqH-00FgUV-NX; Tue, 23 Mar 2021 20:41:34 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnor-00Fg6h-5g for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:07 +0000 Received: by mail-qt1-x84a.google.com with SMTP id f8so1946374qtv.22 for ; Tue, 23 Mar 2021 13:40:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=hHyY7qre9OXEm5PJ4BWXLYwsH5KdHQkHk/skTedwvAs=; b=XrXpk80QpW18uEoFzgTskAcjmvDwp4KkI6czCZLCI2LllpcLlxdrxTo1ItDmNHN8O3 bTKrEPYdAX8nqjUwEIjMGeZsRGCVSfDL1H2a3iHP2ngOiAKHMzDoZoN0zJI6U4uE7k4F VmweQXHEF/+JfAZFlAnZ8tr59hnNtlg0Zu2hks4+H3cinnrhxWtfOBc/gs/UrtgKxOLm BhVL1pP08EBbqj7+0X2iUTjwgv413Vz8hEc3BGPp0zOsqaAbAJtmiERhqTuyjM44Skll xAU/PM/Gg5Aoxn2Inc/k5V8U3wmSxVTzMEVnRHN2lA+v/IcNPy7/8t7fb9X4R/xp1aHQ xZhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=hHyY7qre9OXEm5PJ4BWXLYwsH5KdHQkHk/skTedwvAs=; b=YSmmiumbuTQg5rd3lGOEkMNX+JYfG1kmGl/6THKzMPQ+eHYg0xbFaW6MM+/yhQAFRA smsxiARN7CZuJ+Ni8auA37IA0Z2awWeYe6LfWdFn76tAw6K4szdB9VbeJJCdwUYeIWEE xobULxgCaWypPH0NTfMuHgC2T4+xEdm5TJq7hl85DrqRH6o6ZT4WMzLz/Q/1EfulTlCw QmOUe4lXk/aLLZNEOVj9NAsXZHAPQVosu0hF+hfqiDUjc19YbOaPA2P8O3a27Mz2wCf8 2/oruJPePx2dDSvWXHH5Ci9kz30ZwxLG//DMxvLLXOVNSn14dVWZIR5sP5hjiX1fnK7i DS0w== X-Gm-Message-State: AOAM530yym4LhiW+d8jZZ3rW5a5L9SSuh/ZuRHMPla+/gdW+hHVMPfU7 WO+n0tJus7Zm88TAtihBJtOuf+wO39x5cQaEE3Y= X-Google-Smtp-Source: ABdhPJzK5aKSbLwoPjOa/fZ+4wQdl/i/PHvrBqs1DnRZn9DH8KD/9yfcQQbrZv/qBxwpr5GT4qC2fq5WQsc1rIzUeMY= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:ad4:542b:: with SMTP id g11mr7093888qvt.47.1616532003425; Tue, 23 Mar 2021 13:40:03 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:37 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-9-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 08/17] bpf: disable CFI in dispatcher functions From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204005_395199_98A03FB6 X-CRM114-Status: GOOD ( 12.62 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org BPF dispatcher functions are patched at runtime to perform direct instead of indirect calls. Disable CFI for the dispatcher functions to avoid conflicts. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- include/linux/bpf.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/include/linux/bpf.h b/include/linux/bpf.h index cccaef1088ea..9acdca574527 100644 --- a/include/linux/bpf.h +++ b/include/linux/bpf.h @@ -635,7 +635,7 @@ struct bpf_dispatcher { struct bpf_ksym ksym; }; -static __always_inline unsigned int bpf_dispatcher_nop_func( +static __always_inline __nocfi unsigned int bpf_dispatcher_nop_func( const void *ctx, const struct bpf_insn *insnsi, unsigned int (*bpf_func)(const void *, @@ -663,7 +663,7 @@ void bpf_trampoline_put(struct bpf_trampoline *tr); } #define DEFINE_BPF_DISPATCHER(name) \ - noinline unsigned int bpf_dispatcher_##name##_func( \ + noinline __nocfi unsigned int bpf_dispatcher_##name##_func( \ const void *ctx, \ const struct bpf_insn *insnsi, \ unsigned int (*bpf_func)(const void *, \ From patchwork Tue Mar 23 20:39:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159355 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07037C433DB for ; Tue, 23 Mar 2021 20:43:35 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 486F8619CB for ; Tue, 23 Mar 2021 20:43:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 486F8619CB Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=KlBDGwtEoRw4ZOqE0XJe0dfdnu1X9OQSdnv48U2//AE=; b=bMNkkTzg+ridMb mp8BofQzSPYRpwQE/YHNIMM7W/a96doBb/tHpmgRvfTQ79OWVZYoIXzJXDDCQtUCNyaJwM0QhIg0F T8EMQXmigSbJ1WYBuM1InpmEpzqTAK76XbIxDKWo5PX052sNzC2NKDzY5XuvxZLnWEwIfD8mp3p9V nIc/xAdJ7CxlKFMqTClUhSYuI5bArh0nwIxidBS6oshUjSd+HDNUr8ddrsMNLUg+ysflYGcWGTd1t jE0Ga53PP5qHgsBhuV8JbZfONWoQA+nctDw178DhX07BNPdIk9MtM6jcGln/x66CA76nnc6XZyKgX ad1CdyEW6bygIRyMFVhQ==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnqV-00Fgb8-7x; Tue, 23 Mar 2021 20:41:47 +0000 Received: from mail-qk1-x749.google.com ([2607:f8b0:4864:20::749]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnot-00Fg79-Eb for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:12 +0000 Received: by mail-qk1-x749.google.com with SMTP id u5so128694qkj.10 for ; Tue, 23 Mar 2021 13:40:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=oSLEMg7GGIKORNSEYDg4zTDHCSSUdp4C0SVXhBUbQec=; b=s2BZM/N0HH8/9PNWfRW2gXaj2noR5IPAJ1bWzjd/e6puEVp+5uZ3ohIzdRrd4ZSx0t cnCkC7ZU2YVtSCzObWzRumz/qkrZjBDqJw6ATjmcWMgS4DN6+lXlYZHh66XH+E1I/rfr zl9RbRtMHZsad1PzLHRSucTJs9UCdI6WA5SLRT3L5GAXafHZTaNQXEy7toIRMKDwh3un NkMt++Y7539PeP36Gy9z9up8PahxBKAwv9pxwDyzxbaU22mVIHdcS/+71rLnF8Bo8Cs/ 6WgB8m4s9SKHLGjo8N3QifK1aq7unUCRd6vTWtTocfteX7kk2hBCwGCtqdYH4NiAtkPw K2DA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=oSLEMg7GGIKORNSEYDg4zTDHCSSUdp4C0SVXhBUbQec=; b=EwENCtUWxYlAzoqA9tQRIgvDS0mvdrVA1wvbs72nnRAtI11RZzC4stEN7EVp5FTbiY 9X0SqR0Aj4kTI14AFFEbeV5g1z2BU/ZDT4+EPOHPk9ePzfH912BPOCqvshLKQF+cF2rS YyGPPv6ykeHK2F2MT7eSMBl2qUfgvFZMNnotnOMBJsoqie6y2xgYHwYU4Yw/pYc08RYR wBqW9unL1hVRPyb4maP31GIEuFvy8ZizwUJXmwpuZ5/PSYfwcskfE11Ff05KBEtCHY1z TmfSRB9g9wKR/0FJDZWPIo1gZTdxB/yd3h55wdLQH+fxJRmWVL9/QzHUNtdAG3vue4ki P5QA== X-Gm-Message-State: AOAM533/MxCTvpcm2knOGPsWp8S3bqrvAffI0FeY7id3EzopIKDAZGE/ Ku+YaaiQDO4JU/dTEtbOCS7dveOBBlnkIqpu2uo= X-Google-Smtp-Source: ABdhPJy/fQ9IFtsc+EMGqD4fqCuFugXICVklwPSsYgpjYGLO9CHKcCnMLS+c6QRb3Q7ejXnxGX5MJ2bLG61ykRJNdD4= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a05:6214:f27:: with SMTP id iw7mr6737418qvb.50.1616532005204; Tue, 23 Mar 2021 13:40:05 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:38 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-10-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 09/17] treewide: Change list_sort to use const pointers From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204009_671704_568D8855 X-CRM114-Status: GOOD ( 18.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org list_sort() internally casts the comparison function passed to it to a different type with constant struct list_head pointers, and uses this pointer to call the functions, which trips indirect call Control-Flow Integrity (CFI) checking. Instead of removing the consts, this change defines the list_cmp_func_t type and changes the comparison function types of all list_sort() callers to use const pointers, thus avoiding type mismatches. Suggested-by: Nick Desaulniers Signed-off-by: Sami Tolvanen Reviewed-by: Nick Desaulniers Tested-by: Nick Desaulniers Reviewed-by: Christoph Hellwig Reviewed-by: Kees Cook --- arch/arm64/kvm/vgic/vgic-its.c | 8 ++++---- arch/arm64/kvm/vgic/vgic.c | 3 ++- block/blk-mq-sched.c | 3 ++- block/blk-mq.c | 3 ++- drivers/acpi/nfit/core.c | 3 ++- drivers/acpi/numa/hmat.c | 3 ++- drivers/clk/keystone/sci-clk.c | 4 ++-- drivers/gpu/drm/drm_modes.c | 3 ++- drivers/gpu/drm/i915/gt/intel_engine_user.c | 3 ++- drivers/gpu/drm/i915/gvt/debugfs.c | 2 +- drivers/gpu/drm/i915/selftests/i915_gem_gtt.c | 3 ++- drivers/gpu/drm/radeon/radeon_cs.c | 4 ++-- .../hw/usnic/usnic_uiom_interval_tree.c | 3 ++- drivers/interconnect/qcom/bcm-voter.c | 2 +- drivers/md/raid5.c | 3 ++- drivers/misc/sram.c | 4 ++-- drivers/nvme/host/core.c | 3 ++- .../pci/controller/cadence/pcie-cadence-host.c | 3 ++- drivers/spi/spi-loopback-test.c | 3 ++- fs/btrfs/raid56.c | 3 ++- fs/btrfs/tree-log.c | 3 ++- fs/btrfs/volumes.c | 3 ++- fs/ext4/fsmap.c | 4 ++-- fs/gfs2/glock.c | 3 ++- fs/gfs2/log.c | 2 +- fs/gfs2/lops.c | 3 ++- fs/iomap/buffered-io.c | 3 ++- fs/ubifs/gc.c | 7 ++++--- fs/ubifs/replay.c | 4 ++-- fs/xfs/scrub/bitmap.c | 4 ++-- fs/xfs/xfs_bmap_item.c | 4 ++-- fs/xfs/xfs_buf.c | 6 +++--- fs/xfs/xfs_extent_busy.c | 4 ++-- fs/xfs/xfs_extent_busy.h | 3 ++- fs/xfs/xfs_extfree_item.c | 4 ++-- fs/xfs/xfs_refcount_item.c | 4 ++-- fs/xfs/xfs_rmap_item.c | 4 ++-- include/linux/list_sort.h | 7 ++++--- lib/list_sort.c | 17 ++++++----------- lib/test_list_sort.c | 3 ++- net/tipc/name_table.c | 4 ++-- 41 files changed, 90 insertions(+), 72 deletions(-) diff --git a/arch/arm64/kvm/vgic/vgic-its.c b/arch/arm64/kvm/vgic/vgic-its.c index 40cbaca81333..b9518f94bd43 100644 --- a/arch/arm64/kvm/vgic/vgic-its.c +++ b/arch/arm64/kvm/vgic/vgic-its.c @@ -2190,8 +2190,8 @@ static int vgic_its_restore_ite(struct vgic_its *its, u32 event_id, return offset; } -static int vgic_its_ite_cmp(void *priv, struct list_head *a, - struct list_head *b) +static int vgic_its_ite_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct its_ite *itea = container_of(a, struct its_ite, ite_list); struct its_ite *iteb = container_of(b, struct its_ite, ite_list); @@ -2329,8 +2329,8 @@ static int vgic_its_restore_dte(struct vgic_its *its, u32 id, return offset; } -static int vgic_its_device_cmp(void *priv, struct list_head *a, - struct list_head *b) +static int vgic_its_device_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct its_device *deva = container_of(a, struct its_device, dev_list); struct its_device *devb = container_of(b, struct its_device, dev_list); diff --git a/arch/arm64/kvm/vgic/vgic.c b/arch/arm64/kvm/vgic/vgic.c index 1c597c9885fa..15b666200f0b 100644 --- a/arch/arm64/kvm/vgic/vgic.c +++ b/arch/arm64/kvm/vgic/vgic.c @@ -255,7 +255,8 @@ static struct kvm_vcpu *vgic_target_oracle(struct vgic_irq *irq) * Return negative if "a" sorts before "b", 0 to preserve order, and positive * to sort "b" before "a". */ -static int vgic_irq_cmp(void *priv, struct list_head *a, struct list_head *b) +static int vgic_irq_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct vgic_irq *irqa = container_of(a, struct vgic_irq, ap_list); struct vgic_irq *irqb = container_of(b, struct vgic_irq, ap_list); diff --git a/block/blk-mq-sched.c b/block/blk-mq-sched.c index e1e997af89a0..3ebd6f10f728 100644 --- a/block/blk-mq-sched.c +++ b/block/blk-mq-sched.c @@ -75,7 +75,8 @@ void blk_mq_sched_restart(struct blk_mq_hw_ctx *hctx) blk_mq_run_hw_queue(hctx, true); } -static int sched_rq_cmp(void *priv, struct list_head *a, struct list_head *b) +static int sched_rq_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct request *rqa = container_of(a, struct request, queuelist); struct request *rqb = container_of(b, struct request, queuelist); diff --git a/block/blk-mq.c b/block/blk-mq.c index d4d7c1caa439..4e3a70ab5be1 100644 --- a/block/blk-mq.c +++ b/block/blk-mq.c @@ -1895,7 +1895,8 @@ void blk_mq_insert_requests(struct blk_mq_hw_ctx *hctx, struct blk_mq_ctx *ctx, spin_unlock(&ctx->lock); } -static int plug_rq_cmp(void *priv, struct list_head *a, struct list_head *b) +static int plug_rq_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct request *rqa = container_of(a, struct request, queuelist); struct request *rqb = container_of(b, struct request, queuelist); diff --git a/drivers/acpi/nfit/core.c b/drivers/acpi/nfit/core.c index 8c5dde628405..d15e3ee93b5b 100644 --- a/drivers/acpi/nfit/core.c +++ b/drivers/acpi/nfit/core.c @@ -1195,7 +1195,8 @@ static int __nfit_mem_init(struct acpi_nfit_desc *acpi_desc, return 0; } -static int nfit_mem_cmp(void *priv, struct list_head *_a, struct list_head *_b) +static int nfit_mem_cmp(void *priv, const struct list_head *_a, + const struct list_head *_b) { struct nfit_mem *a = container_of(_a, typeof(*a), list); struct nfit_mem *b = container_of(_b, typeof(*b), list); diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c index cb73a5d6ea76..137a5dd880c2 100644 --- a/drivers/acpi/numa/hmat.c +++ b/drivers/acpi/numa/hmat.c @@ -558,7 +558,8 @@ static bool hmat_update_best(u8 type, u32 value, u32 *best) return updated; } -static int initiator_cmp(void *priv, struct list_head *a, struct list_head *b) +static int initiator_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct memory_initiator *ia; struct memory_initiator *ib; diff --git a/drivers/clk/keystone/sci-clk.c b/drivers/clk/keystone/sci-clk.c index aaf31abe1c8f..7e1b136e71ae 100644 --- a/drivers/clk/keystone/sci-clk.c +++ b/drivers/clk/keystone/sci-clk.c @@ -503,8 +503,8 @@ static int ti_sci_scan_clocks_from_fw(struct sci_clk_provider *provider) #else -static int _cmp_sci_clk_list(void *priv, struct list_head *a, - struct list_head *b) +static int _cmp_sci_clk_list(void *priv, const struct list_head *a, + const struct list_head *b) { struct sci_clk *ca = container_of(a, struct sci_clk, node); struct sci_clk *cb = container_of(b, struct sci_clk, node); diff --git a/drivers/gpu/drm/drm_modes.c b/drivers/gpu/drm/drm_modes.c index 1ac67d4505e0..6662d0457ad6 100644 --- a/drivers/gpu/drm/drm_modes.c +++ b/drivers/gpu/drm/drm_modes.c @@ -1290,7 +1290,8 @@ EXPORT_SYMBOL(drm_mode_prune_invalid); * Negative if @lh_a is better than @lh_b, zero if they're equivalent, or * positive if @lh_b is better than @lh_a. */ -static int drm_mode_compare(void *priv, struct list_head *lh_a, struct list_head *lh_b) +static int drm_mode_compare(void *priv, const struct list_head *lh_a, + const struct list_head *lh_b) { struct drm_display_mode *a = list_entry(lh_a, struct drm_display_mode, head); struct drm_display_mode *b = list_entry(lh_b, struct drm_display_mode, head); diff --git a/drivers/gpu/drm/i915/gt/intel_engine_user.c b/drivers/gpu/drm/i915/gt/intel_engine_user.c index 34e6096f196e..da21d2a10cc9 100644 --- a/drivers/gpu/drm/i915/gt/intel_engine_user.c +++ b/drivers/gpu/drm/i915/gt/intel_engine_user.c @@ -49,7 +49,8 @@ static const u8 uabi_classes[] = { [VIDEO_ENHANCEMENT_CLASS] = I915_ENGINE_CLASS_VIDEO_ENHANCE, }; -static int engine_cmp(void *priv, struct list_head *A, struct list_head *B) +static int engine_cmp(void *priv, const struct list_head *A, + const struct list_head *B) { const struct intel_engine_cs *a = container_of((struct rb_node *)A, typeof(*a), uabi_node); diff --git a/drivers/gpu/drm/i915/gvt/debugfs.c b/drivers/gpu/drm/i915/gvt/debugfs.c index 62e6a14ad58e..9f1c209d9251 100644 --- a/drivers/gpu/drm/i915/gvt/debugfs.c +++ b/drivers/gpu/drm/i915/gvt/debugfs.c @@ -41,7 +41,7 @@ struct diff_mmio { /* Compare two diff_mmio items. */ static int mmio_offset_compare(void *priv, - struct list_head *a, struct list_head *b) + const struct list_head *a, const struct list_head *b) { struct diff_mmio *ma; struct diff_mmio *mb; diff --git a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c index c1adea8765a9..52b9c39e0155 100644 --- a/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c +++ b/drivers/gpu/drm/i915/selftests/i915_gem_gtt.c @@ -1076,7 +1076,8 @@ static int igt_ppgtt_shrink_boom(void *arg) return exercise_ppgtt(arg, shrink_boom); } -static int sort_holes(void *priv, struct list_head *A, struct list_head *B) +static int sort_holes(void *priv, const struct list_head *A, + const struct list_head *B) { struct drm_mm_node *a = list_entry(A, typeof(*a), hole_stack); struct drm_mm_node *b = list_entry(B, typeof(*b), hole_stack); diff --git a/drivers/gpu/drm/radeon/radeon_cs.c b/drivers/gpu/drm/radeon/radeon_cs.c index 35e937d39b51..1a5c3db1d53b 100644 --- a/drivers/gpu/drm/radeon/radeon_cs.c +++ b/drivers/gpu/drm/radeon/radeon_cs.c @@ -393,8 +393,8 @@ int radeon_cs_parser_init(struct radeon_cs_parser *p, void *data) return 0; } -static int cmp_size_smaller_first(void *priv, struct list_head *a, - struct list_head *b) +static int cmp_size_smaller_first(void *priv, const struct list_head *a, + const struct list_head *b) { struct radeon_bo_list *la = list_entry(a, struct radeon_bo_list, tv.head); struct radeon_bo_list *lb = list_entry(b, struct radeon_bo_list, tv.head); diff --git a/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.c b/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.c index d399523206c7..29d71267af78 100644 --- a/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.c +++ b/drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.c @@ -83,7 +83,8 @@ usnic_uiom_interval_node_alloc(long int start, long int last, int ref_cnt, return interval; } -static int interval_cmp(void *priv, struct list_head *a, struct list_head *b) +static int interval_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct usnic_uiom_interval_node *node_a, *node_b; diff --git a/drivers/interconnect/qcom/bcm-voter.c b/drivers/interconnect/qcom/bcm-voter.c index 1cc565bce2f4..d1591a28b743 100644 --- a/drivers/interconnect/qcom/bcm-voter.c +++ b/drivers/interconnect/qcom/bcm-voter.c @@ -39,7 +39,7 @@ struct bcm_voter { u32 tcs_wait; }; -static int cmp_vcd(void *priv, struct list_head *a, struct list_head *b) +static int cmp_vcd(void *priv, const struct list_head *a, const struct list_head *b) { const struct qcom_icc_bcm *bcm_a = list_entry(a, struct qcom_icc_bcm, list); const struct qcom_icc_bcm *bcm_b = list_entry(b, struct qcom_icc_bcm, list); diff --git a/drivers/md/raid5.c b/drivers/md/raid5.c index 5d57a5bd171f..841e1c1aa5e6 100644 --- a/drivers/md/raid5.c +++ b/drivers/md/raid5.c @@ -953,7 +953,8 @@ static void dispatch_bio_list(struct bio_list *tmp) submit_bio_noacct(bio); } -static int cmp_stripe(void *priv, struct list_head *a, struct list_head *b) +static int cmp_stripe(void *priv, const struct list_head *a, + const struct list_head *b) { const struct r5pending_data *da = list_entry(a, struct r5pending_data, sibling); diff --git a/drivers/misc/sram.c b/drivers/misc/sram.c index 6c1a23cb3e8c..202bf951e909 100644 --- a/drivers/misc/sram.c +++ b/drivers/misc/sram.c @@ -144,8 +144,8 @@ static void sram_free_partitions(struct sram_dev *sram) } } -static int sram_reserve_cmp(void *priv, struct list_head *a, - struct list_head *b) +static int sram_reserve_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct sram_reserve *ra = list_entry(a, struct sram_reserve, list); struct sram_reserve *rb = list_entry(b, struct sram_reserve, list); diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c index 0896e21642be..5eaaa51a5e30 100644 --- a/drivers/nvme/host/core.c +++ b/drivers/nvme/host/core.c @@ -3855,7 +3855,8 @@ static int nvme_init_ns_head(struct nvme_ns *ns, unsigned nsid, return ret; } -static int ns_cmp(void *priv, struct list_head *a, struct list_head *b) +static int ns_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct nvme_ns *nsa = container_of(a, struct nvme_ns, list); struct nvme_ns *nsb = container_of(b, struct nvme_ns, list); diff --git a/drivers/pci/controller/cadence/pcie-cadence-host.c b/drivers/pci/controller/cadence/pcie-cadence-host.c index 73dcf8cf98fb..ae1c55503513 100644 --- a/drivers/pci/controller/cadence/pcie-cadence-host.c +++ b/drivers/pci/controller/cadence/pcie-cadence-host.c @@ -345,7 +345,8 @@ static int cdns_pcie_host_bar_config(struct cdns_pcie_rc *rc, return 0; } -static int cdns_pcie_host_dma_ranges_cmp(void *priv, struct list_head *a, struct list_head *b) +static int cdns_pcie_host_dma_ranges_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct resource_entry *entry1, *entry2; diff --git a/drivers/spi/spi-loopback-test.c b/drivers/spi/spi-loopback-test.c index df981e55c24c..f1cf2232f0b5 100644 --- a/drivers/spi/spi-loopback-test.c +++ b/drivers/spi/spi-loopback-test.c @@ -454,7 +454,8 @@ struct rx_ranges { u8 *end; }; -static int rx_ranges_cmp(void *priv, struct list_head *a, struct list_head *b) +static int rx_ranges_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct rx_ranges *rx_a = list_entry(a, struct rx_ranges, list); struct rx_ranges *rx_b = list_entry(b, struct rx_ranges, list); diff --git a/fs/btrfs/raid56.c b/fs/btrfs/raid56.c index 8c31357f08ed..f4139de63b2e 100644 --- a/fs/btrfs/raid56.c +++ b/fs/btrfs/raid56.c @@ -1634,7 +1634,8 @@ struct btrfs_plug_cb { /* * rbios on the plug list are sorted for easier merging. */ -static int plug_cmp(void *priv, struct list_head *a, struct list_head *b) +static int plug_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct btrfs_raid_bio *ra = container_of(a, struct btrfs_raid_bio, plug_list); diff --git a/fs/btrfs/tree-log.c b/fs/btrfs/tree-log.c index 92a368627791..00a88bd8105e 100644 --- a/fs/btrfs/tree-log.c +++ b/fs/btrfs/tree-log.c @@ -4136,7 +4136,8 @@ static noinline int copy_items(struct btrfs_trans_handle *trans, return ret; } -static int extent_cmp(void *priv, struct list_head *a, struct list_head *b) +static int extent_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct extent_map *em1, *em2; diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index bc3b33efddc5..70dc25c8198e 100644 --- a/fs/btrfs/volumes.c +++ b/fs/btrfs/volumes.c @@ -1224,7 +1224,8 @@ static int open_fs_devices(struct btrfs_fs_devices *fs_devices, return 0; } -static int devid_cmp(void *priv, struct list_head *a, struct list_head *b) +static int devid_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct btrfs_device *dev1, *dev2; diff --git a/fs/ext4/fsmap.c b/fs/ext4/fsmap.c index 4c2a9fe30067..4493ef0c715e 100644 --- a/fs/ext4/fsmap.c +++ b/fs/ext4/fsmap.c @@ -354,8 +354,8 @@ static unsigned int ext4_getfsmap_find_sb(struct super_block *sb, /* Compare two fsmap items. */ static int ext4_getfsmap_compare(void *priv, - struct list_head *a, - struct list_head *b) + const struct list_head *a, + const struct list_head *b) { struct ext4_fsmap *fa; struct ext4_fsmap *fb; diff --git a/fs/gfs2/glock.c b/fs/gfs2/glock.c index 9567520d79f7..c06a6cdf05de 100644 --- a/fs/gfs2/glock.c +++ b/fs/gfs2/glock.c @@ -1732,7 +1732,8 @@ void gfs2_glock_complete(struct gfs2_glock *gl, int ret) spin_unlock(&gl->gl_lockref.lock); } -static int glock_cmp(void *priv, struct list_head *a, struct list_head *b) +static int glock_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct gfs2_glock *gla, *glb; diff --git a/fs/gfs2/log.c b/fs/gfs2/log.c index 6410281546f9..88649b43fcff 100644 --- a/fs/gfs2/log.c +++ b/fs/gfs2/log.c @@ -695,7 +695,7 @@ void log_flush_wait(struct gfs2_sbd *sdp) } } -static int ip_cmp(void *priv, struct list_head *a, struct list_head *b) +static int ip_cmp(void *priv, const struct list_head *a, const struct list_head *b) { struct gfs2_inode *ipa, *ipb; diff --git a/fs/gfs2/lops.c b/fs/gfs2/lops.c index a82f4747aa8d..b4809967efc6 100644 --- a/fs/gfs2/lops.c +++ b/fs/gfs2/lops.c @@ -634,7 +634,8 @@ static void gfs2_check_magic(struct buffer_head *bh) kunmap_atomic(kaddr); } -static int blocknr_cmp(void *priv, struct list_head *a, struct list_head *b) +static int blocknr_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct gfs2_bufdata *bda, *bdb; diff --git a/fs/iomap/buffered-io.c b/fs/iomap/buffered-io.c index 414769a6ad11..0129e6bab985 100644 --- a/fs/iomap/buffered-io.c +++ b/fs/iomap/buffered-io.c @@ -1155,7 +1155,8 @@ iomap_ioend_try_merge(struct iomap_ioend *ioend, struct list_head *more_ioends, EXPORT_SYMBOL_GPL(iomap_ioend_try_merge); static int -iomap_ioend_compare(void *priv, struct list_head *a, struct list_head *b) +iomap_ioend_compare(void *priv, const struct list_head *a, + const struct list_head *b) { struct iomap_ioend *ia = container_of(a, struct iomap_ioend, io_list); struct iomap_ioend *ib = container_of(b, struct iomap_ioend, io_list); diff --git a/fs/ubifs/gc.c b/fs/ubifs/gc.c index a4aaeea63893..dc3e26e9ed7b 100644 --- a/fs/ubifs/gc.c +++ b/fs/ubifs/gc.c @@ -102,7 +102,8 @@ static int switch_gc_head(struct ubifs_info *c) * This function compares data nodes @a and @b. Returns %1 if @a has greater * inode or block number, and %-1 otherwise. */ -static int data_nodes_cmp(void *priv, struct list_head *a, struct list_head *b) +static int data_nodes_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { ino_t inuma, inumb; struct ubifs_info *c = priv; @@ -145,8 +146,8 @@ static int data_nodes_cmp(void *priv, struct list_head *a, struct list_head *b) * first and sorted by length in descending order. Directory entry nodes go * after inode nodes and are sorted in ascending hash valuer order. */ -static int nondata_nodes_cmp(void *priv, struct list_head *a, - struct list_head *b) +static int nondata_nodes_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { ino_t inuma, inumb; struct ubifs_info *c = priv; diff --git a/fs/ubifs/replay.c b/fs/ubifs/replay.c index 0f8a6a16421b..4d17e5382b74 100644 --- a/fs/ubifs/replay.c +++ b/fs/ubifs/replay.c @@ -298,8 +298,8 @@ static int apply_replay_entry(struct ubifs_info *c, struct replay_entry *r) * entries @a and @b by comparing their sequence numer. Returns %1 if @a has * greater sequence number and %-1 otherwise. */ -static int replay_entries_cmp(void *priv, struct list_head *a, - struct list_head *b) +static int replay_entries_cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct ubifs_info *c = priv; struct replay_entry *ra, *rb; diff --git a/fs/xfs/scrub/bitmap.c b/fs/xfs/scrub/bitmap.c index f88694f22d05..813b5f219113 100644 --- a/fs/xfs/scrub/bitmap.c +++ b/fs/xfs/scrub/bitmap.c @@ -63,8 +63,8 @@ xbitmap_init( static int xbitmap_range_cmp( void *priv, - struct list_head *a, - struct list_head *b) + const struct list_head *a, + const struct list_head *b) { struct xbitmap_range *ap; struct xbitmap_range *bp; diff --git a/fs/xfs/xfs_bmap_item.c b/fs/xfs/xfs_bmap_item.c index 2344757ede63..e3a691937e92 100644 --- a/fs/xfs/xfs_bmap_item.c +++ b/fs/xfs/xfs_bmap_item.c @@ -265,8 +265,8 @@ xfs_trans_log_finish_bmap_update( static int xfs_bmap_update_diff_items( void *priv, - struct list_head *a, - struct list_head *b) + const struct list_head *a, + const struct list_head *b) { struct xfs_bmap_intent *ba; struct xfs_bmap_intent *bb; diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c index 37a1d12762d8..592800c8852f 100644 --- a/fs/xfs/xfs_buf.c +++ b/fs/xfs/xfs_buf.c @@ -2124,9 +2124,9 @@ xfs_buf_delwri_queue( */ static int xfs_buf_cmp( - void *priv, - struct list_head *a, - struct list_head *b) + void *priv, + const struct list_head *a, + const struct list_head *b) { struct xfs_buf *ap = container_of(a, struct xfs_buf, b_list); struct xfs_buf *bp = container_of(b, struct xfs_buf, b_list); diff --git a/fs/xfs/xfs_extent_busy.c b/fs/xfs/xfs_extent_busy.c index ef17c1f6db32..a4075685d9eb 100644 --- a/fs/xfs/xfs_extent_busy.c +++ b/fs/xfs/xfs_extent_busy.c @@ -629,8 +629,8 @@ xfs_extent_busy_wait_all( int xfs_extent_busy_ag_cmp( void *priv, - struct list_head *l1, - struct list_head *l2) + const struct list_head *l1, + const struct list_head *l2) { struct xfs_extent_busy *b1 = container_of(l1, struct xfs_extent_busy, list); diff --git a/fs/xfs/xfs_extent_busy.h b/fs/xfs/xfs_extent_busy.h index 990ab3891971..8aea07100092 100644 --- a/fs/xfs/xfs_extent_busy.h +++ b/fs/xfs/xfs_extent_busy.h @@ -58,7 +58,8 @@ void xfs_extent_busy_wait_all(struct xfs_mount *mp); int -xfs_extent_busy_ag_cmp(void *priv, struct list_head *a, struct list_head *b); +xfs_extent_busy_ag_cmp(void *priv, const struct list_head *a, + const struct list_head *b); static inline void xfs_extent_busy_sort(struct list_head *list) { diff --git a/fs/xfs/xfs_extfree_item.c b/fs/xfs/xfs_extfree_item.c index 93223ebb3372..2424230ca2c3 100644 --- a/fs/xfs/xfs_extfree_item.c +++ b/fs/xfs/xfs_extfree_item.c @@ -397,8 +397,8 @@ xfs_trans_free_extent( static int xfs_extent_free_diff_items( void *priv, - struct list_head *a, - struct list_head *b) + const struct list_head *a, + const struct list_head *b) { struct xfs_mount *mp = priv; struct xfs_extent_free_item *ra; diff --git a/fs/xfs/xfs_refcount_item.c b/fs/xfs/xfs_refcount_item.c index 07ebccbbf4df..746f4eda724c 100644 --- a/fs/xfs/xfs_refcount_item.c +++ b/fs/xfs/xfs_refcount_item.c @@ -269,8 +269,8 @@ xfs_trans_log_finish_refcount_update( static int xfs_refcount_update_diff_items( void *priv, - struct list_head *a, - struct list_head *b) + const struct list_head *a, + const struct list_head *b) { struct xfs_mount *mp = priv; struct xfs_refcount_intent *ra; diff --git a/fs/xfs/xfs_rmap_item.c b/fs/xfs/xfs_rmap_item.c index 49cebd68b672..dc4f0c9f0897 100644 --- a/fs/xfs/xfs_rmap_item.c +++ b/fs/xfs/xfs_rmap_item.c @@ -337,8 +337,8 @@ xfs_trans_log_finish_rmap_update( static int xfs_rmap_update_diff_items( void *priv, - struct list_head *a, - struct list_head *b) + const struct list_head *a, + const struct list_head *b) { struct xfs_mount *mp = priv; struct xfs_rmap_intent *ra; diff --git a/include/linux/list_sort.h b/include/linux/list_sort.h index 20f178c24e9d..453105f74e05 100644 --- a/include/linux/list_sort.h +++ b/include/linux/list_sort.h @@ -6,8 +6,9 @@ struct list_head; +typedef int __attribute__((nonnull(2,3))) (*list_cmp_func_t)(void *, + const struct list_head *, const struct list_head *); + __attribute__((nonnull(2,3))) -void list_sort(void *priv, struct list_head *head, - int (*cmp)(void *priv, struct list_head *a, - struct list_head *b)); +void list_sort(void *priv, struct list_head *head, list_cmp_func_t cmp); #endif diff --git a/lib/list_sort.c b/lib/list_sort.c index 52f0c258c895..a926d96ffd44 100644 --- a/lib/list_sort.c +++ b/lib/list_sort.c @@ -7,16 +7,13 @@ #include #include -typedef int __attribute__((nonnull(2,3))) (*cmp_func)(void *, - struct list_head const *, struct list_head const *); - /* * Returns a list organized in an intermediate format suited * to chaining of merge() calls: null-terminated, no reserved or * sentinel head node, "prev" links not maintained. */ __attribute__((nonnull(2,3,4))) -static struct list_head *merge(void *priv, cmp_func cmp, +static struct list_head *merge(void *priv, list_cmp_func_t cmp, struct list_head *a, struct list_head *b) { struct list_head *head, **tail = &head; @@ -52,7 +49,7 @@ static struct list_head *merge(void *priv, cmp_func cmp, * throughout. */ __attribute__((nonnull(2,3,4,5))) -static void merge_final(void *priv, cmp_func cmp, struct list_head *head, +static void merge_final(void *priv, list_cmp_func_t cmp, struct list_head *head, struct list_head *a, struct list_head *b) { struct list_head *tail = head; @@ -185,9 +182,7 @@ static void merge_final(void *priv, cmp_func cmp, struct list_head *head, * 2^(k+1) - 1 (second merge of case 5 when x == 2^(k-1) - 1). */ __attribute__((nonnull(2,3))) -void list_sort(void *priv, struct list_head *head, - int (*cmp)(void *priv, struct list_head *a, - struct list_head *b)) +void list_sort(void *priv, struct list_head *head, list_cmp_func_t cmp) { struct list_head *list = head->next, *pending = NULL; size_t count = 0; /* Count of pending */ @@ -227,7 +222,7 @@ void list_sort(void *priv, struct list_head *head, if (likely(bits)) { struct list_head *a = *tail, *b = a->prev; - a = merge(priv, (cmp_func)cmp, b, a); + a = merge(priv, cmp, b, a); /* Install the merged result in place of the inputs */ a->prev = b->prev; *tail = a; @@ -249,10 +244,10 @@ void list_sort(void *priv, struct list_head *head, if (!next) break; - list = merge(priv, (cmp_func)cmp, pending, list); + list = merge(priv, cmp, pending, list); pending = next; } /* The final merge, rebuilding prev links */ - merge_final(priv, (cmp_func)cmp, head, pending, list); + merge_final(priv, cmp, head, pending, list); } EXPORT_SYMBOL(list_sort); diff --git a/lib/test_list_sort.c b/lib/test_list_sort.c index 1f017d3b610e..00daaf23316f 100644 --- a/lib/test_list_sort.c +++ b/lib/test_list_sort.c @@ -56,7 +56,8 @@ static int __init check(struct debug_el *ela, struct debug_el *elb) return 0; } -static int __init cmp(void *priv, struct list_head *a, struct list_head *b) +static int __init cmp(void *priv, const struct list_head *a, + const struct list_head *b) { struct debug_el *ela, *elb; diff --git a/net/tipc/name_table.c b/net/tipc/name_table.c index ee5ac40ea2b6..f8141443f2e2 100644 --- a/net/tipc/name_table.c +++ b/net/tipc/name_table.c @@ -397,8 +397,8 @@ static struct publication *tipc_service_remove_publ(struct service_range *sr, * Code reused: time_after32() for the same purpose */ #define publication_after(pa, pb) time_after32((pa)->id, (pb)->id) -static int tipc_publ_sort(void *priv, struct list_head *a, - struct list_head *b) +static int tipc_publ_sort(void *priv, const struct list_head *a, + const struct list_head *b) { struct publication *pa, *pb; From patchwork Tue Mar 23 20:39:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159357 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A5FFC433DB for ; Tue, 23 Mar 2021 20:43:48 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A5561619BA for ; Tue, 23 Mar 2021 20:43:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A5561619BA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=SeytQPjYgWcYF12L2G6b7i73R4n6NTv+so0db6OMASM=; b=daY45fCn3wv3Te M8xaQzlfEBxomLW//k6lnqgX7ofLyZRlR3dlDIL7n4zGclG6kBb9LMxeqX0wdK9zGd86VsuD33XSK Jbqmfql8BCWBUd4NVcCAKZZ1mfCb+w8grVbt4VUlV5+QjPtTaBNM4zCcTS4VfiudJErvQW91WA6Vq VjeUYWQnQQmA60PhxOosXZFWnmcmZ6EOA01n96fEdDkAw3zhvJZeh42rVwnVDwqXsSmx4F4b3KqIM 4644HSyMOhJam+g09vfKOG5Lsbwk0r01+eR0EX2ca46SWUJcqjkow1XZZLWC6qrsD3663kMSKHr09 dYzGiUJ5bklhzdrcBftw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnqf-00FggH-64; Tue, 23 Mar 2021 20:41:57 +0000 Received: from mail-qv1-xf49.google.com ([2607:f8b0:4864:20::f49]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnov-00Fg7E-Py for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:13 +0000 Received: by mail-qv1-xf49.google.com with SMTP id e10so3468qvr.17 for ; Tue, 23 Mar 2021 13:40:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=xIKsEDWi8EL1j5NSYc2uWQA3l5Y2NiXXcLT9TovNhRQ=; b=CgO/iSD/xRAdG2p+rJSIXqPZiKxGiKV2KoCvVGLUH2ZMIqkf574rj//FtCji3y0utT 6lKsLLQNF9ZBDCdsLOxtHp++EYzY5pP2mxldfH597xr3rQ4xUq9h8H5ZqqqHCXkA5Woj AW4sxR0WuNwk6mjflnE1PSnqER0YBsuwjwXu3Ld7LoONyiCQrMHMa5vxLqAxT+oS7dew 573+hZHKrFPSsuuYfRH4A915CMBtXOIjkDVY3SgKsQFOEMK30MSps4h+NJvmkF2xbtOc I9PZSHpWCV2Y3qKuZASiPEN0Gbhj4D7LoWjpCn9ubP4M+BIswdDkzinaS2z4pXhbM2te iBHg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=xIKsEDWi8EL1j5NSYc2uWQA3l5Y2NiXXcLT9TovNhRQ=; b=X5+zOt3QK3Hde4NYC4mXfoy8Yc36W48A2t+2sN305yqD+ySRr7l+fuOEYqGTSGrFUF sFud9smOSG5Y3VBZq5b5xK2mbSep6YjJPohb6WrsCQGN6d9tICUMGEUPniiR1sl5qX+l gWOemapAudAq2rQoewvUHEeM8q2eZyO92780xh7fmshkBmx9c4xdjmORF21meJA/fztL 2Mq1rhyUFZ/rlfO6KgESQaV/O+zPqSry0rWHOe8WZvLnNr3kNNONH71cJlaxewtJ2AIG VNz2JaxjQhDitbeW2jRja9KJPGOHgmEQNhztdh9l8K+8bZyPNXjb5rg96Bgw0IRa4566 joig== X-Gm-Message-State: AOAM5317X0je5opnphrDODWRyHFON6si5G6G6/CAsO4oRwcT/y7QBcXe 9unQJSGyRHIM0DwqVaWM8eStXxxfgqbSyLLodhE= X-Google-Smtp-Source: ABdhPJxb7WUXqQY1HKNj9KDKWZ2mX+BL8U6d8oevhZ712uyBxXvQItdJYqzQ+dKFDvxGu4W+35Xr/uYgxKxus3pnNFc= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a0c:e9c7:: with SMTP id q7mr159305qvo.7.1616532007142; Tue, 23 Mar 2021 13:40:07 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:39 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-11-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 10/17] lkdtm: use __va_function From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204010_424555_63055043 X-CRM114-Status: GOOD ( 14.12 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org To ensure we take the actual address of a function in kernel text, use __va_function. Otherwise, with CONFIG_CFI_CLANG, the compiler replaces the address with a pointer to the CFI jump table, which is actually in the module when compiled with CONFIG_LKDTM=m. Signed-off-by: Sami Tolvanen Acked-by: Kees Cook --- drivers/misc/lkdtm/usercopy.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/misc/lkdtm/usercopy.c b/drivers/misc/lkdtm/usercopy.c index 109e8d4302c1..d173d6175c87 100644 --- a/drivers/misc/lkdtm/usercopy.c +++ b/drivers/misc/lkdtm/usercopy.c @@ -314,7 +314,7 @@ void lkdtm_USERCOPY_KERNEL(void) pr_info("attempting bad copy_to_user from kernel text: %px\n", vm_mmap); - if (copy_to_user((void __user *)user_addr, vm_mmap, + if (copy_to_user((void __user *)user_addr, __va_function(vm_mmap), unconst + PAGE_SIZE)) { pr_warn("copy_to_user failed, but lacked Oops\n"); goto free_user; From patchwork Tue Mar 23 20:39:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159361 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B884AC433C1 for ; Tue, 23 Mar 2021 20:44:01 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F394961554 for ; Tue, 23 Mar 2021 20:44:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F394961554 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=/bV771e6LeM77URAN8n0w7XYbbaougbXQdMz7qrZtlo=; b=i6oOnt/pgWtPJw EwD8RCxlDz8rN9AWCe9s3fVK6eut/wSZlmQX/BxxSRKu/48kDdB0PSQqEVUyFviw/tvXE/WF82j+R SHzCjgkr6rsNe2eMqfvwMMcamYgUBBwkIb3SQ5PkBVMd86Ybvx1ItDOhHjNrRJJVWqespAmSuI9r3 36GO3m8q5ENIKgoSrvb1+K2VCE6qR/kbzjeE9R5243YMnOjpMEJZxfgMpuvPkjhm1Ydl4/n4bCMsS nd0f3lPHxzIE6XV973ptS2rpgQEBNokJ+pu3PmchVWwq6iu4yhnRftlxgSlwFWYW8ECivgH1xYQjJ Cr0rh5SwUdaxIGUE4l7A==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnqu-00FgoW-WA; Tue, 23 Mar 2021 20:42:13 +0000 Received: from mail-qv1-xf4a.google.com ([2607:f8b0:4864:20::f4a]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnoy-00Fg7H-Ko for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:15 +0000 Received: by mail-qv1-xf4a.google.com with SMTP id n1so24527qvi.4 for ; Tue, 23 Mar 2021 13:40:09 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=bxVrKYuxs5XU9fnnKC3lGs6PXwAVYnFmVHHh5prD+bw=; b=tT+mhGxcn72LL+S/wxj6p6pgongKWrRuo4s35bHXpLA2dicqXg5urUYbPt3d9T/ByR IPHjRKTiXQnv2rbt1aRKlxrdR2rPLJ7tORytyy1drMdPlsn4xgMEb2VaXi9KnmRXw0b+ 1McGQM2Thg4csg/H4j9okV5jZZhUrLRSHuFW27uwZsHK3LpCc5+cy7/TMp0EDcRjnuha JF7+v1vVLu+/0lrJCTHtkCwIx8QFpif22GcIKNGwCbR4j/Je8b9blw7GwduI3m2NOHlM YfgJo/D20uuFbhQvjr6gzbOj6Nv05LUXHIHdnyfSlGGvEfwFsp60pVuwWb09eTm3iH6Q VSag== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=bxVrKYuxs5XU9fnnKC3lGs6PXwAVYnFmVHHh5prD+bw=; b=REjsciIHfJLxdSgpDJv6zRLeZgwv26zO1zpuIMSGtx1wmQ1CSK2TDLX3lunY0c/0o+ A7vBOaSq2y4UoMiqhAsxJ9C/fmizuTQTRvZkTBP4Qe6nAcGNzibz9LQB2MXdlLPlOoNM up+vd48jWXVNPh02OhLUBk1e9v/wz+epbXeJsoFU9EgwUkIIgA1HYY9CrbCfeRBpS7hm Jp0EZMZBzZcQ/7AJzAft72fca0eZ1fJpx36QaQ7BETTpHc09RXTBcXipu1iwXOa6wFXs h1sHMVpbMeVrKJmUGbjFtL1QTWRlZqtbiN/XRSLIRLT8xdZT5rU7LoUNic5UqllcPRQT 30cw== X-Gm-Message-State: AOAM5320ECLAUcVtX8lcRBzQ/ZYkKxmkJJOSgHrkrD6qayw8U1KCNQcV wCASXcV4ouxerbD0gKQrvNsTYEKNUtIKz00O8tE= X-Google-Smtp-Source: ABdhPJx9XxcfFaPwjV0eKcB1u5Is4MdpwwvMvMA5rF3C3ca6pKmdXd3MtbY+z05x6lmX/nx6Kd3vneHRgp5+amu1pQk= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a05:6214:aae:: with SMTP id ew14mr188842qvb.24.1616532008942; Tue, 23 Mar 2021 13:40:08 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:40 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-12-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 11/17] psci: use __pa_function for cpu_resume From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204013_635004_21146EE4 X-CRM114-Status: GOOD ( 12.83 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With CONFIG_CFI_CLANG, the compiler replaces function pointers with jump table addresses, which results in __pa_symbol returning the physical address of the jump table entry. As the jump table contains an immediate jump to an EL1 virtual address, this typically won't work as intended. Use __pa_function instead to get the address to cpu_resume. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- drivers/firmware/psci/psci.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c index f5fc429cae3f..facd3cce3244 100644 --- a/drivers/firmware/psci/psci.c +++ b/drivers/firmware/psci/psci.c @@ -326,7 +326,7 @@ static int psci_suspend_finisher(unsigned long state) { u32 power_state = state; - return psci_ops.cpu_suspend(power_state, __pa_symbol(cpu_resume)); + return psci_ops.cpu_suspend(power_state, __pa_function(cpu_resume)); } int psci_cpu_suspend_enter(u32 state) @@ -345,7 +345,7 @@ int psci_cpu_suspend_enter(u32 state) static int psci_system_suspend(unsigned long unused) { return invoke_psci_fn(PSCI_FN_NATIVE(1_0, SYSTEM_SUSPEND), - __pa_symbol(cpu_resume), 0, 0); + __pa_function(cpu_resume), 0, 0); } static int psci_system_suspend_enter(suspend_state_t state) From patchwork Tue Mar 23 20:39:41 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159359 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B4D98C433C1 for ; Tue, 23 Mar 2021 20:43:53 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B5E7F619BF for ; Tue, 23 Mar 2021 20:43:52 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B5E7F619BF Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=aoIZRaClm9Jl9//9wgpJRaYQ0GE7yVRb/lwUN9CEgDs=; b=B3HeNbe+FnR3Qv OXj52AbyiCTTqJbV3zzkrHFx7/3QpxnTJmyb04I6gGfS3/URl5EzGQikyWxKTQNgZ0/3uE8U55bnA So4k5vvntSRFn+WrSkdYlBHnb0KkVXaymEkYOZw4JFkD0hon6r3WKeWP0jMCP5kI0QP4KTPushAnr syjla48l5/gc8UrexP1a+65+0txqlPaIfhLr4ZB6cfnQNMlyR/8H6FjTKPFVyAFt0SLNkLI/ylawA hD3opDfuyWQjJtvWkfPc4PiI/A8up8OYYSW/TqQmG1F7U5gwOg17h0t0cV7FcozurheHkT2A7czNe sl/r1dZwo3yN6dR8nPZA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnr9-00Fgw8-QH; Tue, 23 Mar 2021 20:42:27 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnoz-00Fg87-EU for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:15 +0000 Received: by mail-yb1-xb49.google.com with SMTP id g64so3848138ybf.0 for ; Tue, 23 Mar 2021 13:40:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=pXHCPErazdX/nDg2/nb6QksjLfEFTtrkU2uTHlahGRs=; b=jFt+TBSDKrmbmcTGUS3Zh3LQfQ9h6g78j1z40AyKg7NaLPpZHn37Ldl7vtccRAf8UV GppokLpY6wTAB+Sle9DrxlzNvmD7RLVB1LrLtLP+DoPSdOHThcUhWPgZsZMHr1AypqJp 2ausF2iw7I06q65bPHkkK5uKHVdZZ+ckMEmm7IBf/pcXpGFOQXgKf2NyXu+UF4y2zTd2 ycqljJ06Wbbus0bG5F1C0bgOjXenQJWv7VPFmvmSNViXRdLRYpO43wanLsM7EnyPPUa4 y3y0OHiuMRejUijcBoHlT618Iy/tBMRleGTHHjCIOalK/zQVBqz//LPDgG2CS2s+wtjo u2Nw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=pXHCPErazdX/nDg2/nb6QksjLfEFTtrkU2uTHlahGRs=; b=AQfVF+7i1jeyihDo+eoR/IcJEVYm5DttoQDgjnKpK6tKJtTJ57SE/soeOzCeDeNvn5 I4xStIolT8w77Ao+EdkKPOqrBYQ2h9hZ/J9n7sl0D1XQMyYY4ttU25xG6cKD7WuQj41t qGh5WEgLMeN4mzekqWERghK0+en9jnsmiYVK6cgOe/bf2EZkF4kxIxE4+wJROc0L7tsB QbH5h7HcZx2j7U9asSKZpVF2F9yu6bMOA85MHg/WJOI3UH9dRbgeReIbp+NWio/qyly3 t9k7EIpGwYodCZEe3vMUjandl+gY8WZ3S45aZvpYsHEeSYKek3/ssqygSEqfjkOAyjkt NFoQ== X-Gm-Message-State: AOAM5333ix0+lHhU6d2ILsk377Q+1v+JIHDsTPp87QbNGu/ahMdtqqTh QL+z2kZB/2IywSJL3iU01/1EzvyYi3cMUxC4XUo= X-Google-Smtp-Source: ABdhPJy23IA9Pmn2B2QQVPkwdw9dKjsA0O0LWinFUMBV4532ZUThiTX267nWH6v33xCXfAmR7YR5nPY/6Ir/NDlbC70= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a25:d3d3:: with SMTP id e202mr58822ybf.283.1616532011040; Tue, 23 Mar 2021 13:40:11 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:41 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-13-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 12/17] arm64: implement __va_function From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204013_981273_F2760255 X-CRM114-Status: GOOD ( 11.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With CONFIG_CFI_CLANG, the compiler replaces function addresses in instrumented C code with jump table addresses. This change implements the __va_function() macro, which returns the actual function address instead. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/arm64/include/asm/memory.h | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/arch/arm64/include/asm/memory.h b/arch/arm64/include/asm/memory.h index 0aabc3be9a75..9a4887808681 100644 --- a/arch/arm64/include/asm/memory.h +++ b/arch/arm64/include/asm/memory.h @@ -321,6 +321,21 @@ static inline void *phys_to_virt(phys_addr_t x) #define virt_to_pfn(x) __phys_to_pfn(__virt_to_phys((unsigned long)(x))) #define sym_to_pfn(x) __phys_to_pfn(__pa_symbol(x)) +#ifdef CONFIG_CFI_CLANG +/* + * With CONFIG_CFI_CLANG, the compiler replaces function address + * references with the address of the function's CFI jump table + * entry. The __va_function macro always returns the address of the + * actual function instead. + */ +#define __va_function(x) ({ \ + void *addr; \ + asm("adrp %0, " __stringify(x) "\n\t" \ + "add %0, %0, :lo12:" __stringify(x) : "=r" (addr)); \ + addr; \ +}) +#endif + /* * virt_to_page(x) convert a _valid_ virtual address to struct page * * virt_addr_valid(x) indicates whether a virtual address is valid From patchwork Tue Mar 23 20:39:42 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159363 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 386EDC433C1 for ; Tue, 23 Mar 2021 20:44:25 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E808E619BA for ; Tue, 23 Mar 2021 20:44:23 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E808E619BA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=w3TAtaPF67P5CtaUCWuknAYnLLirTUZf6lAmJzbJEx8=; b=n+HoyXG2o3vpZ4 V2JdWg2uWEP5gOeZbcULFyzZNIp35YIVe9zGdeUr7RBWkGPX2G/sYdtx8wlfb1oahnWh2jM+fHCYn AX527qnq04E0PCEYulCFEKutbsO6+tlqqh7v0/Z+3/idoSYqVnAFiCvKAWhhlqoHDA541gxa/ZNbn kDjw7MMYQpkLulumVxXmYsPUUotS8t8QOztegkYu5q8XQe0feqth3Url3YUHVcWGwyrTQsGTt0+th iJH6zWfvF5mzsQARUG5/aH+w2fwxtzz9sbOxb2xfEz5jaA7dDl1G2CRaEVg+YU+oL1ye0fVMDWeTz oDl0y94lx13OtTF6GoNg==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnrU-00Fh6j-Va; Tue, 23 Mar 2021 20:42:49 +0000 Received: from mail-qt1-x84a.google.com ([2607:f8b0:4864:20::84a]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnp0-00Fg8C-7c for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:16 +0000 Received: by mail-qt1-x84a.google.com with SMTP id a11so1974870qtd.4 for ; Tue, 23 Mar 2021 13:40:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=TXt+gLVEDlmfHDoaitlym6lqHbHvj5KHcb6MKkix6ec=; b=a/yUJ0QXm/MOLiUOhSWc+MJHJGUrLH4d7XtxofXeHJURcZDnAcdDXmYDaJXv007JPz xAPUgq+GcXSMjRNECEOcb5ceYGk5hhoMrgh3corCxHVFDnt3oxiQpzo/WEufjeNi4uYk kDUvaRMYPbMJQYNM55tQ3P2kEh0EhTtHjTGI0CC8Zg/irBGrycsgiRLSeENrKcmhiR+I bJsyyonlBCpY+NDkalPH+ykewNK1Zj1R4goiy7twtXajFwl1AWjQ/zA45qNIc2aHNXe7 lyqrguVctk1AYoZ1oBuog71+Cw29qnmliEdTR+vA7O9FpgmlA3VhaVjPa0oDORYY38xM rrCg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=TXt+gLVEDlmfHDoaitlym6lqHbHvj5KHcb6MKkix6ec=; b=AUZjro4076zW1U3HNC+6rTpcShsDeSk0hRcIRE5qCq5wlDvaIeaRIbL0AyzVM8J85d SWo1IRDE19uppxiBtHRFJY1MyVO1SBbWCXk3ZcHiPwL4qhLkK9AWSWc6NqKcOjqDjawA Dk0XdXuKxAjsqo73pU6gnFhklVXCYOh/xomZwe0CbnqQAyMlkJboLpsAeEuZKNiydCAo fmZ7hdGrzUeBHrE/Y1HBWpmJucuIHQ+08ouyVoYzvUmWB1+dkRJ8nUtdA7Bll1nJagxf Hp+BJH8KADBCzJm1UcFGcVu0BoyFfICt27vHfybB5uWyob2UKUpFetUc0zC9Ocyn4na2 hlSQ== X-Gm-Message-State: AOAM53397k5GXLyJ6bdodi2zH/Rp61I5n56O7ODI5Z2hzzT4XsSpI9a2 D7XQtq+j4uXbDl+inL+21XrkCSIgIi+PP5ao2ME= X-Google-Smtp-Source: ABdhPJy6OMzEOHy6aXtVaxpn8SRHl25FYohWdXh4rj7GqknCA5DMEHWc/hOFNCg6er7t1Uz+avTocAMaMN2Bc434rVY= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:ad4:50d0:: with SMTP id e16mr136306qvq.37.1616532012794; Tue, 23 Mar 2021 13:40:12 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:42 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-14-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 13/17] arm64: use __pa_function From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204014_488273_E1799447 X-CRM114-Status: GOOD ( 16.87 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With CONFIG_CFI_CLANG, the compiler replaces function address references with the address of the function's CFI jump table entry. This means that __pa_symbol(function) returns the physical address of the jump table entry, which can lead to address space confusion as the jump table points to the function's virtual address. Therefore, use the __pa_function() macro to ensure we are always taking the address of the actual function instead. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/arm64/include/asm/mmu_context.h | 2 +- arch/arm64/kernel/acpi_parking_protocol.c | 2 +- arch/arm64/kernel/cpu-reset.h | 2 +- arch/arm64/kernel/cpufeature.c | 2 +- arch/arm64/kernel/psci.c | 3 ++- arch/arm64/kernel/smp_spin_table.c | 2 +- 6 files changed, 7 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index bd02e99b1a4c..16cc9a694bb2 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -140,7 +140,7 @@ static inline void cpu_replace_ttbr1(pgd_t *pgdp) ttbr1 |= TTBR_CNP_BIT; } - replace_phys = (void *)__pa_symbol(idmap_cpu_replace_ttbr1); + replace_phys = (void *)__pa_function(idmap_cpu_replace_ttbr1); cpu_install_idmap(); replace_phys(ttbr1); diff --git a/arch/arm64/kernel/acpi_parking_protocol.c b/arch/arm64/kernel/acpi_parking_protocol.c index e7c941d8340d..e7f3af6043c5 100644 --- a/arch/arm64/kernel/acpi_parking_protocol.c +++ b/arch/arm64/kernel/acpi_parking_protocol.c @@ -99,7 +99,7 @@ static int acpi_parking_protocol_cpu_boot(unsigned int cpu) * that read this address need to convert this address to the * Boot-Loader's endianness before jumping. */ - writeq_relaxed(__pa_symbol(secondary_entry), &mailbox->entry_point); + writeq_relaxed(__pa_function(secondary_entry), &mailbox->entry_point); writel_relaxed(cpu_entry->gic_cpu_id, &mailbox->cpu_id); arch_send_wakeup_ipi_mask(cpumask_of(cpu)); diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index ed50e9587ad8..dfba8cf921e5 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -22,7 +22,7 @@ static inline void __noreturn cpu_soft_restart(unsigned long entry, unsigned long el2_switch = !is_kernel_in_hyp_mode() && is_hyp_mode_available(); - restart = (void *)__pa_symbol(__cpu_soft_restart); + restart = (void *)__pa_function(__cpu_soft_restart); cpu_install_idmap(); restart(el2_switch, entry, arg0, arg1, arg2); diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 066030717a4c..7ec1c2ccdc0b 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1460,7 +1460,7 @@ kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused) if (arm64_use_ng_mappings) return; - remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings); + remap_fn = (void *)__pa_function(idmap_kpti_install_ng_mappings); cpu_install_idmap(); remap_fn(cpu, num_online_cpus(), __pa_symbol(swapper_pg_dir)); diff --git a/arch/arm64/kernel/psci.c b/arch/arm64/kernel/psci.c index 62d2bda7adb8..bfb1a6f8282d 100644 --- a/arch/arm64/kernel/psci.c +++ b/arch/arm64/kernel/psci.c @@ -38,7 +38,8 @@ static int __init cpu_psci_cpu_prepare(unsigned int cpu) static int cpu_psci_cpu_boot(unsigned int cpu) { - int err = psci_ops.cpu_on(cpu_logical_map(cpu), __pa_symbol(secondary_entry)); + int err = psci_ops.cpu_on(cpu_logical_map(cpu), + __pa_function(secondary_entry)); if (err) pr_err("failed to boot CPU%d (%d)\n", cpu, err); diff --git a/arch/arm64/kernel/smp_spin_table.c b/arch/arm64/kernel/smp_spin_table.c index 056772c26098..a80ff9092e86 100644 --- a/arch/arm64/kernel/smp_spin_table.c +++ b/arch/arm64/kernel/smp_spin_table.c @@ -88,7 +88,7 @@ static int smp_spin_table_cpu_prepare(unsigned int cpu) * boot-loader's endianness before jumping. This is mandated by * the boot protocol. */ - writeq_relaxed(__pa_symbol(secondary_holding_pen), release_addr); + writeq_relaxed(__pa_function(secondary_holding_pen), release_addr); __flush_dcache_area((__force void *)release_addr, sizeof(*release_addr)); From patchwork Tue Mar 23 20:39:43 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159365 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5A654C433DB for ; Tue, 23 Mar 2021 20:45:13 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C26AC619B2 for ; Tue, 23 Mar 2021 20:45:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C26AC619B2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=8stqagL0fCKq5i6GcqM0wwBvtmvNksMFWzUDWN3xMPw=; b=RxPzbxw3E/HOk7 EPmk+d81ZInJj5h1ZTzLyWbMnoCLq2Me2NPqK+3c4osqN59LysKiprPkvGR9/Q/w2HF+Z2TJ+hnWS pj31+zmWFzPPC/pOp4Ux00TihZ4YxJds2YtuMQNJ+YqMk/sZOqVW4mqxvn1VdYuTYR/zUlb6gockv RvX09qVBxgcOwcW9BdsswuIuC2ZiJtfwZhQVk1G7v0eSMOpx17rJH11c1IniuAoaydCwx6lZXEmbL Oe7qc9y77q9DnmlK19KgXUR3iae07S/WPU7s/TArATNMjSodKjskr5BuCkY83Kc8H9YJi9YJQsELg AqSyAzNF/nGTdbPtGy+w==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnrk-00FhEU-Jm; Tue, 23 Mar 2021 20:43:04 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnp2-00Fg9G-9f for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:18 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id x8so3861551ybo.6 for ; Tue, 23 Mar 2021 13:40:15 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=qy/qpaQcjeJGAGSX/SJfdvrV6+xztRS9WV/l31kV3fo=; b=p9ojVOb7QdO7JLZj9S/1SqPGP8MeifFmkf1e6TNJokU+4X+mDo0s3TIOhU1rmJD0gQ C3q3EV251sFO4e//IgvNjn6P29FqOur3szQsT80WVJVAgcgLC/L4gp8LF+fgzMkzOMdQ qK0XZckG24gPmdHwXF6V6z62Qj29O8AjeBevFbC/8BXBEoj5BXdSL0u6CL6w5E92BGE7 +vtDECL4yMlxuZ35+QWL2+CnrRi7FlCo5jHCQ87YudtBBhNKSZWNcK4Kl6XHbFJXUSPL qDxp8oIjW07ybSAxf6LQQ6rDchJmoirsp7IIZbdM6z6gMba2BAhKKUVuDpzy54UcfqSt 3+kA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=qy/qpaQcjeJGAGSX/SJfdvrV6+xztRS9WV/l31kV3fo=; b=tyObi+m6fq0B3u+1by0+mLP7mHxkOx3r2sYR2SwniD/xZHASCs8MGX8OGx5b46hRY/ AR3b7b1RoCx5hUWZCcrhUCOlYXSQPGLDvvsDx+SEip8e0JqJWeFRybhDQplScCEww5O1 0HLDORyluaUkm9c8+u5Y1txAeT15O1PO/Vh/7IkyF+GyOnLvg1j2KvGLpGGahieyimXR sbaCcpH48xuAyH/wQ5D6MSuEazrdFawfSbJBDQ7FN29relXCWx+C8czt68gMINh9k411 DotC+ADB0sav9tjn7r1Zm1PzFQzycJCht9ilmgEVSBIuAArT3rgjZsUaJXZOjyNDo90Q p3/Q== X-Gm-Message-State: AOAM5317oiTCsmg6kzCCQX+6FE/DW++OVpyjqbc12p7QO2Q4UQOee/fl fjiyw3tk6oc+TFSTlIWlgTlajJIWm4ZJxaoOteE= X-Google-Smtp-Source: ABdhPJynHOWSO11EeejwuL7BmZY2Zs5wB2dBb0RO3vU6HGkp4hd57n5miL7y+94vuamNMAUwpzA/HtJp7Bz7P8/50B8= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a25:da48:: with SMTP id n69mr74886ybf.47.1616532014644; Tue, 23 Mar 2021 13:40:14 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:43 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-15-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 14/17] arm64: add __nocfi to functions that jump to a physical address From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204016_657885_3DE35803 X-CRM114-Status: GOOD ( 14.84 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Disable CFI checking for functions that switch to linear mapping and make an indirect call to a physical address, since the compiler only understands virtual addresses and the CFI check for such indirect calls would always fail. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/arm64/include/asm/mmu_context.h | 2 +- arch/arm64/kernel/cpu-reset.h | 8 ++++---- arch/arm64/kernel/cpufeature.c | 2 +- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/arch/arm64/include/asm/mmu_context.h b/arch/arm64/include/asm/mmu_context.h index 16cc9a694bb2..270ba8761a23 100644 --- a/arch/arm64/include/asm/mmu_context.h +++ b/arch/arm64/include/asm/mmu_context.h @@ -119,7 +119,7 @@ static inline void cpu_install_idmap(void) * Atomically replaces the active TTBR1_EL1 PGD with a new VA-compatible PGD, * avoiding the possibility of conflicting TLB entries being allocated. */ -static inline void cpu_replace_ttbr1(pgd_t *pgdp) +static inline void __nocfi cpu_replace_ttbr1(pgd_t *pgdp) { typedef void (ttbr_replace_func)(phys_addr_t); extern ttbr_replace_func idmap_cpu_replace_ttbr1; diff --git a/arch/arm64/kernel/cpu-reset.h b/arch/arm64/kernel/cpu-reset.h index dfba8cf921e5..a05bda363272 100644 --- a/arch/arm64/kernel/cpu-reset.h +++ b/arch/arm64/kernel/cpu-reset.h @@ -13,10 +13,10 @@ void __cpu_soft_restart(unsigned long el2_switch, unsigned long entry, unsigned long arg0, unsigned long arg1, unsigned long arg2); -static inline void __noreturn cpu_soft_restart(unsigned long entry, - unsigned long arg0, - unsigned long arg1, - unsigned long arg2) +static inline void __noreturn __nocfi cpu_soft_restart(unsigned long entry, + unsigned long arg0, + unsigned long arg1, + unsigned long arg2) { typeof(__cpu_soft_restart) *restart; diff --git a/arch/arm64/kernel/cpufeature.c b/arch/arm64/kernel/cpufeature.c index 7ec1c2ccdc0b..473212ff4d70 100644 --- a/arch/arm64/kernel/cpufeature.c +++ b/arch/arm64/kernel/cpufeature.c @@ -1443,7 +1443,7 @@ static bool unmap_kernel_at_el0(const struct arm64_cpu_capabilities *entry, } #ifdef CONFIG_UNMAP_KERNEL_AT_EL0 -static void +static void __nocfi kpti_install_ng_mappings(const struct arm64_cpu_capabilities *__unused) { typedef void (kpti_remap_fn)(int, int, phys_addr_t); From patchwork Tue Mar 23 20:39:44 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159367 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65DB6C433DB for ; Tue, 23 Mar 2021 20:45:23 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CADBF619B2 for ; Tue, 23 Mar 2021 20:45:22 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CADBF619B2 Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=qfyhfYjeqFnD8ixQi4FbCsJcTVB2qS6wVS6KGVDYyyQ=; b=pesrHdIWlAVmnC /AHmDn9cQ0LBcalGk/6HMMm0QCtcULnhii/WjNvi1wBIs7kOuEDo+tEvhErwqOvliy6FxxPu+s7Cl bAJ3a1KTIwik3ppK6U9q1upSXepGGB4jf+KrGJer2e+jo1Xzc4mFhpO3oUkfo2mzqe63FhlgFPGJm UEVcOPSfO9XWV8xrhQmhGZanwH5tztVOgLVIjUxSzKtSveY5HUTtumPyV/V+jywzEdytAeFTZAsuK Ss+EPnEj+IyfNL2uGmcaQZorqbuxWVQzW29Krq31vlGhqfTyI/naIe+FheUrOJYAPbxwZ3Ls3NM9J UWaFu7L07bEUO3s8LYjw==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOns3-00FhNw-NG; Tue, 23 Mar 2021 20:43:23 +0000 Received: from mail-yb1-xb49.google.com ([2607:f8b0:4864:20::b49]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnp4-00Fg9w-5m for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:19 +0000 Received: by mail-yb1-xb49.google.com with SMTP id x8so3861620ybo.6 for ; Tue, 23 Mar 2021 13:40:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=qAbADIQAzXUoq/ElYpa9rjjQHxphEP8wPywcql//gEU=; b=T9lYeG1vNkvikIi777MRSbbTtj3IaPeu7RN7uXeSPXvS4b9+WLGfOdasWEAHo+i17u 3080AmXMs2xVaLte8P7ahk8JJEdHsGELqvOLIoahy4V4PMsoqaf2aZV65W6Qbg5iWS+h b6ti/m6+OZ84hMZziyoN5GFocxb13Za6OwHGFrBaUy1iO+H9JEiP0tyCwkF3p/KzGz5a KI2Lj51XAl8PJy62wWStBc6KSEOp8id1hQ564UGK3S0iIjj4kUsIubPJjw5De4OUhc9s huAxEcZhyUcc7CzcBmMkKd6YmZw4qc0E+2UxOVGGUsXyQLc3FPDMC5xWzijn66SgJ0Xs vHJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=qAbADIQAzXUoq/ElYpa9rjjQHxphEP8wPywcql//gEU=; b=K56keVRDyjkH04gbb1zJs/4FkpR7yRqeUag+MJHJPfsWXV9La1JLcUe9u/PSbUdnEG k60G7+Va6LnGdTi0Wjjv1SKm86LJ5rJ3kC2ztD3BqfImQNhEO20xl7Fc2MfgZyBOxiok f37djrkJE/KHFzw7nAyopezT0Elc0YLKXdDAHECrfj4gWCsyI2ZwOT4T90W7nA81y1zC tsv1zRh47zRESnMdw7kglYkCagIlEkeVRihUDmh/dUYBX+bAWAT+9AbLkpowBzmc9uiu 7k0j8CzBvqsgk6MXu3d1tlcKK7Gnrey9JiwgKQzDI3QdoCJfLFcVDYLxBKTx1UdnJcte KchA== X-Gm-Message-State: AOAM5310Y5Yj9oGljUGCfSOysLZM+zCweh6sXYb3tNKWd7UnxqCQLZS2 uHSxXWkCGCLLFOBhmojLhRVPwMyyX424R/kQsuE= X-Google-Smtp-Source: ABdhPJwmStks0YRxHTnj25YBiyDAjN/ixvNa9lapKnlHKARYrFssO9o0nOXAMkPoRpi0g6QTRn8BhWyhoUBcjcDwMF8= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a25:3493:: with SMTP id b141mr100244yba.342.1616532016542; Tue, 23 Mar 2021 13:40:16 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:44 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-16-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 15/17] arm64: add __nocfi to __apply_alternatives From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204018_293242_2FB6D8DE X-CRM114-Status: GOOD ( 14.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org __apply_alternatives makes indirect calls to functions whose address is taken in assembly code using the alternative_cb macro. With non-canonical CFI, the compiler won't replace these function references with the jump table addresses, which trips CFI. Disable CFI checking in the function to work around the issue. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/arm64/kernel/alternative.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/arm64/kernel/alternative.c b/arch/arm64/kernel/alternative.c index 1184c44ea2c7..abc84636af07 100644 --- a/arch/arm64/kernel/alternative.c +++ b/arch/arm64/kernel/alternative.c @@ -133,8 +133,8 @@ static void clean_dcache_range_nopatch(u64 start, u64 end) } while (cur += d_size, cur < end); } -static void __apply_alternatives(void *alt_region, bool is_module, - unsigned long *feature_mask) +static void __nocfi __apply_alternatives(void *alt_region, bool is_module, + unsigned long *feature_mask) { struct alt_instr *alt; struct alt_region *region = alt_region; From patchwork Tue Mar 23 20:39:45 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159369 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C8D02C433DB for ; Tue, 23 Mar 2021 20:45:39 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 18662619BA for ; Tue, 23 Mar 2021 20:45:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 18662619BA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=MCfNTU7OuFQmpgtFIv2zGFycGBgobtpaOvOilQ9vFTE=; b=E2LOu5PN1egjGF iEhgTYojJyAtjM131b4vbOEnSBKsUXWp2TdO6EYbF2YH2G10uVdItJQrgj52VOl7vO6RWmQG5JDwr ehC3QHNJbDH+wq0Gph83M9IkVMOZtuafkt5vmJMLYZ5JP2c6O6ogav0xFFmP9RdtoqAhWOOKzPFNS X9DtykBi7ipL8Zvr4Ba+pq6B4zQbbTY+ZQewsE6iEeoHk0mjjs3tWRBgMWpwX7FAQlzyEz8j98ZCO UfzUEYVgE8Qo6cUxvXLDwBlHrQNZAqTpaYDmO4a0Av/pPfiNHDYVh8VbvcBHMuHv5lYFaMvg73P5i /DioAnBJjEgE2L37g70g==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnsQ-00FhY8-Jq; Tue, 23 Mar 2021 20:43:46 +0000 Received: from mail-yb1-xb4a.google.com ([2607:f8b0:4864:20::b4a]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnp6-00FgAU-6T for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:22 +0000 Received: by mail-yb1-xb4a.google.com with SMTP id 6so3849098ybq.7 for ; Tue, 23 Mar 2021 13:40:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=tNWRsiiEAfetheYG6qtDYdd6Zu2YM8DJGmENfE26Aac=; b=QwtFzkNZ0ArdJ22uiNb1llxuSobP5aVQNzwPksMPm+yXD4iJErHZlbeYgNVneIsCaV eWzSIifLKq9OeR/urLV+kBUy5t9Lj6BMjZqFICzYH6wVYLJimy4yOQuD7NgKMnmibW9S zG50zFIIhTS3dotn12D1Zy6qsSjg8WUhwZTAAq2J7W9gn7luxZShIQYCTr+CZA3deHb8 U3jjpaLWT8ZiETFcnxTTPxy7Im1JnVTxQxGpi3yBHzfY2Dfw3S61ouZ35+FlKNnKAFPT AeZpguEmKCpHpboafyAslh+3UWQD/2MhlgpX4aQM7wK4y4d9OGGiM9NT5wh5PIZuoCtX 4HRQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=tNWRsiiEAfetheYG6qtDYdd6Zu2YM8DJGmENfE26Aac=; b=WuTJcfwxR2z5exIsu61AHktsxRS6ZxOVXDjgI7nv95FV+mjphbdtpsqhypjJgUzqBD A3T1caP2B5wTs3gIY7IVEtgcTsNI33S5SJ8GVZi/TFFmLnef4zn0FuP5QjkqnD/KZ3vi DuvzT9+o/qJggVXsV8lZgrnFhS/AjvbmjydFNjXHCHhLLk7kWf6wDSjAYkD/i3gvt0Y5 Lxpv/Lg9Yca0m7+OSzIcpDiSdHHLZ33IQSqaFN9o9HHdksdCjzPGIf8DIwaIFrjdH5SI VwuDqvLn+fetub6b+sPdcCJOLfWiNlM4okcEZZnEtNI4KH6Igh4Ou53pSLThZzg1EXAp JbKw== X-Gm-Message-State: AOAM533vnW5etHLfGwW3kETC+AMSVT5tPWPM97uzHn3cC47NFSGlBg6Y hX2pQfP7Lmsz/AsUkHhXN/DHEk6pJoY8hxfKw9c= X-Google-Smtp-Source: ABdhPJxEi3pQM02XMd23fjMyS1SetL4sDZHHMo2+iWEzXbz9tuF1eS1XAan8eMOnDTZL5Cre89JVC9ou2IlB88zz+ZI= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a25:686:: with SMTP id 128mr110596ybg.258.1616532018575; Tue, 23 Mar 2021 13:40:18 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:45 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-17-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 16/17] KVM: arm64: Disable CFI for nVHE From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204020_400260_E4500077 X-CRM114-Status: GOOD ( 11.91 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Disable CFI for the nVHE code to avoid address space confusion. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/arm64/kvm/hyp/nvhe/Makefile | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile b/arch/arm64/kvm/hyp/nvhe/Makefile index a6707df4f6c0..fb24a0f022ad 100644 --- a/arch/arm64/kvm/hyp/nvhe/Makefile +++ b/arch/arm64/kvm/hyp/nvhe/Makefile @@ -75,9 +75,9 @@ quiet_cmd_hyprel = HYPREL $@ quiet_cmd_hypcopy = HYPCOPY $@ cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@ -# Remove ftrace and Shadow Call Stack CFLAGS. -# This is equivalent to the 'notrace' and '__noscs' annotations. -KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS), $(KBUILD_CFLAGS)) +# Remove ftrace, Shadow Call Stack, and CFI CFLAGS. +# This is equivalent to the 'notrace', '__noscs', and '__nocfi' annotations. +KBUILD_CFLAGS := $(filter-out $(CC_FLAGS_FTRACE) $(CC_FLAGS_SCS) $(CC_FLAGS_CFI), $(KBUILD_CFLAGS)) # KVM nVHE code is run at a different exception code with a different map, so # compiler instrumentation that inserts callbacks or checks into the code may From patchwork Tue Mar 23 20:39:46 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Sami Tolvanen X-Patchwork-Id: 12159371 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-17.0 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_ADSP_CUSTOM_MED,DKIM_SIGNED,DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 847ECC433C1 for ; Tue, 23 Mar 2021 20:46:01 +0000 (UTC) Received: from desiato.infradead.org (desiato.infradead.org [90.155.92.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 67970619BA for ; Tue, 23 Mar 2021 20:46:00 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 67970619BA Authentication-Results: mail.kernel.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=desiato.20200630; h=Sender:Content-Transfer-Encoding :Content-Type:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:Cc:To:From:Subject:References:Mime-Version: Message-Id:In-Reply-To:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=xnlE0mTmgmo1Z3h9O7AOlQA9FKTbxTXJ7mwjGOeyaa8=; b=Fxn7+ILu/unllw UlBqat8idHYnXH5136ZVVxkWWnLW8Wj2HSqWhubi3n5ywUd/0hDMJj6P8Qo4dGcu2WxiWRE1xXMja JIO4iqhIjfu1AUmyriJjIGUxuW0Qtt2iFD396w9Wjfj20A/wyZ75ShV4OcU9VyBlXrCcnXZoCyOTD wLEbfIXktjZ+VQxJaJz7C2iUlRqHJ9XHdSZDSEPcL7UhxAbPZQfL4U9Hma6jZai8kWnxIzV+3qXNA NQDrTJlRUl3lE1YCzVFOQpkX539RlnEUagN3OEsYcNZ2MvN0AkYUXX67rLjD6vKVpH7UOY8PxWncr bqxROPfsNA6HOlxP3ebA==; Received: from localhost ([::1] helo=desiato.infradead.org) by desiato.infradead.org with esmtp (Exim 4.94 #2 (Red Hat Linux)) id 1lOnsm-00Fhgh-3T; Tue, 23 Mar 2021 20:44:10 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by desiato.infradead.org with esmtps (Exim 4.94 #2 (Red Hat Linux)) id 1lOnp9-00FgAv-HS for linux-arm-kernel@lists.infradead.org; Tue, 23 Mar 2021 20:40:28 +0000 Received: by mail-qt1-x849.google.com with SMTP id w2so1962704qts.18 for ; Tue, 23 Mar 2021 13:40:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:in-reply-to:message-id:mime-version:references:subject:from:to :cc; bh=WZ2fsTX5FLJhX01BulIdj9aVU+u+1SaaqVDcWrlk7vo=; b=ec/02RNqMxC3NtjfEqCKWLv4qwmQTmHrkXsCeNO2KBn8qSqKzTE24FKGqfuKH+TVkK X/VKFkbL4vsRYCrPeFIrZP8DeiwqKS96n7WGnqpK5r2/VAz0Iab+2cnLr9ixgbVx3i2+ bJO7ki2MVhZqoAWHQEzmE5B7oTVS3IIOSVMH6lZh48SF/0j7bgW6jHNwPIk8gKuBLJqR 4qQ5N4v6mKVofR2cSUqiTSphpJlczKzRN+vOPaGMsYJ3GFsS6r9eXFPtqA3WSVhoq05F zPIbSYNFmagSwu4VxL+cE8EfslHOGZyTYBOn5X3fzMMM4mw38aQ1r1FrLS9fBfhCQ2tj 1law== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:in-reply-to:message-id:mime-version :references:subject:from:to:cc; bh=WZ2fsTX5FLJhX01BulIdj9aVU+u+1SaaqVDcWrlk7vo=; b=DooDXUHitLM3T2p4kHcQsBlK3jPoQfFjCTGzeEh2r/0/7mC0/uZ0F7dt7X8CprvPFN 7w41KCVDEonhQW81c4Gmk1N+bZF/QeKBPEC8ykkyrfhrlRvN0fCRowT15M/oho1h0uAF /XlitJnzTY0zNezS36lz68r3IZjpCL3GkWz1+pQ6FPoXIUgNNMgWaLbKTHhwvh35b1m0 3zOvyZL4v6I41Ua5IDnXqKc0hF/Gb4pGH9Ikx+irun6NZS/W6fpRtN09jvqbEHrb8Zdq 0Fuqjk0PCXlux1JIphu0ix0AA4dmHdxbowuUFNy8La9nP5eM/ff0DXzH7DxwJJfY85nQ 3Z2g== X-Gm-Message-State: AOAM533yGycDtZWPMvEV29ofHg/6v2JayyJltpTang6/YahiLmz1DjIN VSSy9PjxHEFr1zUzBNO7mHTjKcjmpHNqBR5nJPk= X-Google-Smtp-Source: ABdhPJyxpPvs3kGL5X0rhxaceHJDwoHelX5/WOAlmnNB1fMlzzx1j3iwVr8PFhB1zHKJQ3nkMRVGiRyWH5xdyb6dFU8= X-Received: from samitolvanen1.mtv.corp.google.com ([2620:15c:201:2:e9a3:260d:763b:67dc]) (user=samitolvanen job=sendgmr) by 2002:a0c:bec3:: with SMTP id f3mr6577376qvj.49.1616532020445; Tue, 23 Mar 2021 13:40:20 -0700 (PDT) Date: Tue, 23 Mar 2021 13:39:46 -0700 In-Reply-To: <20210323203946.2159693-1-samitolvanen@google.com> Message-Id: <20210323203946.2159693-18-samitolvanen@google.com> Mime-Version: 1.0 References: <20210323203946.2159693-1-samitolvanen@google.com> X-Mailer: git-send-email 2.31.0.291.g576ba9dcdaf-goog Subject: [PATCH v3 17/17] arm64: allow CONFIG_CFI_CLANG to be selected From: Sami Tolvanen To: Kees Cook Cc: Nathan Chancellor , Nick Desaulniers , Masahiro Yamada , Will Deacon , Jessica Yu , Arnd Bergmann , Tejun Heo , "Paul E. McKenney" , Christoph Hellwig , Peter Zijlstra , bpf@vger.kernel.org, linux-hardening@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kbuild@vger.kernel.org, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Sami Tolvanen X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20210323_204026_706837_CDA9A65B X-CRM114-Status: GOOD ( 10.16 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Select ARCH_SUPPORTS_CFI_CLANG to allow CFI to be enabled. Signed-off-by: Sami Tolvanen Reviewed-by: Kees Cook --- arch/arm64/Kconfig | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 5656e7aacd69..2eefdbc3e3c9 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -75,6 +75,7 @@ config ARM64 select ARCH_SUPPORTS_SHADOW_CALL_STACK if CC_HAVE_SHADOW_CALL_STACK select ARCH_SUPPORTS_LTO_CLANG if CPU_LITTLE_ENDIAN select ARCH_SUPPORTS_LTO_CLANG_THIN + select ARCH_SUPPORTS_CFI_CLANG select ARCH_SUPPORTS_ATOMIC_RMW select ARCH_SUPPORTS_INT128 if CC_HAS_INT128 && (GCC_VERSION >= 50000 || CC_IS_CLANG) select ARCH_SUPPORTS_NUMA_BALANCING