From patchwork Wed Apr 3 15:01:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13616297 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 02D75CD1292 for ; Wed, 3 Apr 2024 15:02:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=gHxuF1aIQ68tQ3inoEZoyyygkPOv3PFRFbmrN+7LzbM=; b=24wUaEpn5rx3JA sAOEDMYrS5AIaNZMz1Mi3JCPZTATO4MY2BSzPobrhO3YMgrszPtI1iCTVAz9vuTOPYC1JEwAzXfBF avHnl1sHpTEEw3S+1Sv+oGGG926VaeZxdtmX1qzR3oNZnjSgAJv9jPbzLpbipu17/EX/yRmY/2/Ix a6gC+5EBJvdMFVBncEspXnbsMYTum5uPvTSNTSd1dmvrU1CceY5MC8Dt+o9Y15zY6qFugWYMalAsb Dgodmxez79/aj5SlRU80lcykiEMtOzWyZZkhM0FgJtxKAOZhpf7C9zTdMWlrVlJT2Pc4Y5I/PHxrV PMH7i1+nWXlUZ2n/rmPQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs27u-0000000GbCR-4Bbc; Wed, 03 Apr 2024 15:02:11 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs27p-0000000Gb9H-05Ij for linux-arm-kernel@lists.infradead.org; Wed, 03 Apr 2024 15:02:08 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 725511595; Wed, 3 Apr 2024 08:02:35 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id C2F883F7B4; Wed, 3 Apr 2024 08:02:02 -0700 (PDT) From: Mark Rutland To: linux-kernel@vger.kernel.org, Catalin Marinas , Will Deacon Cc: anil.s.keshavamurthy@intel.com, aou@eecs.berkeley.edu, davem@davemloft.net, jarkko@kernel.org, linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, mhiramat@kernel.org, naveen.n.rao@linux.ibm.com, palmer@dabbelt.com, paul.walmsley@sifive.com Subject: [PATCH v2 1/4] arm64: patching: always use fixmap Date: Wed, 3 Apr 2024 16:01:51 +0100 Message-Id: <20240403150154.667649-2-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240403150154.667649-1-mark.rutland@arm.com> References: <20240403150154.667649-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240403_080205_458552_BA8F8AAF X-CRM114-Status: GOOD ( 13.59 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org For historical reasons, patch_map() won't bother to fixmap non-image addresses when CONFIG_STRICT_MODULE_RWX=n, matching the behaviour prior to the introduction of CONFIG_STRICT_MODULE_RWX. However, as arm64 doesn't select CONFIG_ARCH_OPTIONAL_KERNEL_RWX, CONFIG_MODULES implies CONFIG_STRICT_MODULE_RWX, so any kernel built with module support will use the fixmap for any non-image address. Historically we only used patch_map() for the kernel image and modules, but these days its also used by BPF and KPROBES to write to read-only pages of executable text. Currently these both depend on CONFIG_MODULES, but we'd like to change that in subsequent patches, which will require using the fixmap regardless of CONFIG_STRICT_MODULE_RWX. This patch changes patch_map() to always use the fixmap, and simplifies the logic: * Use is_image_text() directly in the if-else, rather than using a temporary boolean variable. * Use offset_in_page() to get the offset within the mapping. * Remove uintaddr and cast the address directly when using is_image_text(). For kernels built with CONFIG_MODULES=y, there should be no functional change as a result of this patch. For kernels built with CONFIG_MODULES=n, patch_map() will use the fixmap for non-image addresses, but there are no extant users with non-image addresses when CONFIG_MODULES=n, and hence there should be no functional change as a result of this patch alone. Signed-off-by: Mark Rutland Cc: Catalin Marinas Cc: Will Deacon Reviewed-by: Jarkko Sakkinen Acked-by: Catalin Marinas --- arch/arm64/kernel/patching.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-) Catalin, Will, this is a prerequisite for the final two patches in the series. Are you happy for this go via the tracing tree? Mark. diff --git a/arch/arm64/kernel/patching.c b/arch/arm64/kernel/patching.c index 2555349303684..f0f3a2a82ca5a 100644 --- a/arch/arm64/kernel/patching.c +++ b/arch/arm64/kernel/patching.c @@ -30,20 +30,16 @@ static bool is_image_text(unsigned long addr) static void __kprobes *patch_map(void *addr, int fixmap) { - unsigned long uintaddr = (uintptr_t) addr; - bool image = is_image_text(uintaddr); struct page *page; - if (image) + if (is_image_text((unsigned long)addr)) page = phys_to_page(__pa_symbol(addr)); - else if (IS_ENABLED(CONFIG_STRICT_MODULE_RWX)) - page = vmalloc_to_page(addr); else - return addr; + page = vmalloc_to_page(addr); BUG_ON(!page); return (void *)set_fixmap_offset(fixmap, page_to_phys(page) + - (uintaddr & ~PAGE_MASK)); + offset_in_page(addr)); } static void __kprobes patch_unmap(int fixmap) From patchwork Wed Apr 3 15:01:52 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13616298 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 78159CD1288 for ; Wed, 3 Apr 2024 15:02:31 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=LYKP5lHUSYgHg5+LuUbG/KfZ3m5gSNsf4rMfAF6I2uM=; b=T3sSQJl18ElN9/ 2sUjyRvKLnordr61VHgBmUPB5S6OIQHGL+pA7B4j4YJsVj1mCHQCrBSGP1wj+G7WGCffRKKf+RwbC NDJyFEhtsyh6dLq9Yh0Nupy1Ke9z5nfJbS7tSqmbfjqN9oRspulB7hc5WIEGWxn9822cxwau7ALc6 9qqQX3xXh18Vnguq6pMBbBKXs9sTyZAn9Nzm6Y4Iru3BYABsDPFpvSvLzKUA5a+gqRK1mxIewJcFb 4d+zpIVv5x5yoD4/XQRZR8oBu2DMSoSyxxmQVmPgbwJIOJruxVho1UAV5rnlRVn3C7QYySdmCown+ NAXBwcnHRNAfTmJ1JN+g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs27w-0000000GbDL-2wjl; Wed, 03 Apr 2024 15:02:12 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs27s-0000000GbAv-2c7o for linux-arm-kernel@lists.infradead.org; Wed, 03 Apr 2024 15:02:09 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id D3E361650; Wed, 3 Apr 2024 08:02:37 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 318DB3F7B4; Wed, 3 Apr 2024 08:02:05 -0700 (PDT) From: Mark Rutland To: linux-kernel@vger.kernel.org Cc: anil.s.keshavamurthy@intel.com, aou@eecs.berkeley.edu, catalin.marinas@arm.com, davem@davemloft.net, jarkko@kernel.org, linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, mhiramat@kernel.org, naveen.n.rao@linux.ibm.com, palmer@dabbelt.com, paul.walmsley@sifive.com, will@kernel.org Subject: [PATCH v2 2/4] kprobes/treewide: Add kprobes_ prefix to insn alloc/free functions Date: Wed, 3 Apr 2024 16:01:52 +0100 Message-Id: <20240403150154.667649-3-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240403150154.667649-1-mark.rutland@arm.com> References: <20240403150154.667649-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240403_080208_819148_231E6FE9 X-CRM114-Status: GOOD ( 14.71 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The alloc_(opt)insn_page() and free_(opt)insn_page() functions are specific to KPROBES, but their name makes them sound more generic than they are. Given them a 'kprobes_' prefix to make it clear that they're part of kprobes. This was generated automatically with: sed -i 's/alloc_insn_page/kprobes_alloc_insn_page/' $(git grep -l 'alloc_insn_page') sed -i 's/free_insn_page/kprobes_free_insn_page/' $(git grep -l 'free_insn_page') sed -i 's/alloc_optinsn_page/kprobes_alloc_optinsn_page/' $(git grep -l 'alloc_optinsn_page') sed -i 's/free_optinsn_page/kprobes_free_optinsn_page/' $(git grep -l 'free_optinsn_page') There should be no functional change as a result of this patch. Signed-off-by: Mark Rutland Cc: Anil S Keshavamurthy Cc: David S. Miller Cc: Jarkko Sakkinen Cc: Masami Hiramatsu Cc: Naveen N. Rao Reviewed-by: Jarkko Sakkinen --- arch/arm64/kernel/probes/kprobes.c | 2 +- arch/powerpc/kernel/kprobes.c | 2 +- arch/powerpc/kernel/optprobes.c | 4 ++-- arch/riscv/kernel/probes/kprobes.c | 2 +- arch/s390/kernel/kprobes.c | 2 +- arch/x86/kernel/kprobes/core.c | 2 +- include/linux/kprobes.h | 6 +++--- kernel/kprobes.c | 20 ++++++++++---------- 8 files changed, 20 insertions(+), 20 deletions(-) diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index 327855a11df2f..4b6ab7b1fa211 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -129,7 +129,7 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p) return 0; } -void *alloc_insn_page(void) +void *kprobes_alloc_insn_page(void) { return __vmalloc_node_range(PAGE_SIZE, 1, VMALLOC_START, VMALLOC_END, GFP_KERNEL, PAGE_KERNEL_ROX, VM_FLUSH_RESET_PERMS, diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c index bbca90a5e2ec0..0b297718d5de6 100644 --- a/arch/powerpc/kernel/kprobes.c +++ b/arch/powerpc/kernel/kprobes.c @@ -126,7 +126,7 @@ kprobe_opcode_t *arch_adjust_kprobe_addr(unsigned long addr, unsigned long offse return (kprobe_opcode_t *)(addr + offset); } -void *alloc_insn_page(void) +void *kprobes_alloc_insn_page(void) { void *page; diff --git a/arch/powerpc/kernel/optprobes.c b/arch/powerpc/kernel/optprobes.c index 004fae2044a3e..0ddbda217073f 100644 --- a/arch/powerpc/kernel/optprobes.c +++ b/arch/powerpc/kernel/optprobes.c @@ -27,7 +27,7 @@ static bool insn_page_in_use; -void *alloc_optinsn_page(void) +void *kprobes_alloc_optinsn_page(void) { if (insn_page_in_use) return NULL; @@ -35,7 +35,7 @@ void *alloc_optinsn_page(void) return &optinsn_slot; } -void free_optinsn_page(void *page) +void kprobes_free_optinsn_page(void *page) { insn_page_in_use = false; } diff --git a/arch/riscv/kernel/probes/kprobes.c b/arch/riscv/kernel/probes/kprobes.c index 2f08c14a933d0..75201ce721057 100644 --- a/arch/riscv/kernel/probes/kprobes.c +++ b/arch/riscv/kernel/probes/kprobes.c @@ -105,7 +105,7 @@ int __kprobes arch_prepare_kprobe(struct kprobe *p) } #ifdef CONFIG_MMU -void *alloc_insn_page(void) +void *kprobes_alloc_insn_page(void) { return __vmalloc_node_range(PAGE_SIZE, 1, VMALLOC_START, VMALLOC_END, GFP_KERNEL, PAGE_KERNEL_READ_EXEC, diff --git a/arch/s390/kernel/kprobes.c b/arch/s390/kernel/kprobes.c index f0cf20d4b3c58..91ca4d501d4ef 100644 --- a/arch/s390/kernel/kprobes.c +++ b/arch/s390/kernel/kprobes.c @@ -34,7 +34,7 @@ struct kretprobe_blackpoint kretprobe_blacklist[] = { }; static int insn_page_in_use; -void *alloc_insn_page(void) +void *kprobes_alloc_insn_page(void) { void *page; diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index d0e49bd7c6f3f..7f01bbbfa9e2a 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -491,7 +491,7 @@ static int prepare_singlestep(kprobe_opcode_t *buf, struct kprobe *p, } /* Make page to RO mode when allocate it */ -void *alloc_insn_page(void) +void *kprobes_alloc_insn_page(void) { void *page; diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h index 0ff44d6633e33..ad4b561100f9e 100644 --- a/include/linux/kprobes.h +++ b/include/linux/kprobes.h @@ -430,10 +430,10 @@ int enable_kprobe(struct kprobe *kp); void dump_kprobe(struct kprobe *kp); -void *alloc_insn_page(void); +void *kprobes_alloc_insn_page(void); -void *alloc_optinsn_page(void); -void free_optinsn_page(void *page); +void *kprobes_alloc_optinsn_page(void); +void kprobes_free_optinsn_page(void *page); int kprobe_get_kallsym(unsigned int symnum, unsigned long *value, char *type, char *sym); diff --git a/kernel/kprobes.c b/kernel/kprobes.c index 9d9095e817928..35adf56430c9b 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -110,7 +110,7 @@ enum kprobe_slot_state { SLOT_USED = 2, }; -void __weak *alloc_insn_page(void) +void __weak *kprobes_alloc_insn_page(void) { /* * Use module_alloc() so this page is within +/- 2GB of where the @@ -121,15 +121,15 @@ void __weak *alloc_insn_page(void) return module_alloc(PAGE_SIZE); } -static void free_insn_page(void *page) +static void kprobes_free_insn_page(void *page) { module_memfree(page); } struct kprobe_insn_cache kprobe_insn_slots = { .mutex = __MUTEX_INITIALIZER(kprobe_insn_slots.mutex), - .alloc = alloc_insn_page, - .free = free_insn_page, + .alloc = kprobes_alloc_insn_page, + .free = kprobes_free_insn_page, .sym = KPROBE_INSN_PAGE_SYM, .pages = LIST_HEAD_INIT(kprobe_insn_slots.pages), .insn_size = MAX_INSN_SIZE, @@ -333,21 +333,21 @@ int kprobe_cache_get_kallsym(struct kprobe_insn_cache *c, unsigned int *symnum, } #ifdef CONFIG_OPTPROBES -void __weak *alloc_optinsn_page(void) +void __weak *kprobes_alloc_optinsn_page(void) { - return alloc_insn_page(); + return kprobes_alloc_insn_page(); } -void __weak free_optinsn_page(void *page) +void __weak kprobes_free_optinsn_page(void *page) { - free_insn_page(page); + kprobes_free_insn_page(page); } /* For optimized_kprobe buffer */ struct kprobe_insn_cache kprobe_optinsn_slots = { .mutex = __MUTEX_INITIALIZER(kprobe_optinsn_slots.mutex), - .alloc = alloc_optinsn_page, - .free = free_optinsn_page, + .alloc = kprobes_alloc_optinsn_page, + .free = kprobes_free_optinsn_page, .sym = KPROBE_OPTINSN_PAGE_SYM, .pages = LIST_HEAD_INIT(kprobe_optinsn_slots.pages), /* .insn_size is initialized later */ From patchwork Wed Apr 3 15:01:53 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13616299 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4C778CD128A for ; Wed, 3 Apr 2024 15:02:32 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=wuSU/QT0MJOA+OWWknWs4r1Fa6gnN6CigTAagor27NM=; b=nQaX3pTjph/7cl S7GaacvqVm/mILdhR7w/Dab3v5HVAOPulFCAHr+kXtCmz+0JB3OM5I+Ois5DR0dS1hwubElFXmV1n yD+J2e0w7gHXDTx0x/qPGlo3vWaVMoJFPyFYurtPr1cLRm040VWBK1zwvJ5Ae2Tx18FHtnRGYyYrm TltOxTWCuJJugxzVsG0pnGWApE1KQnFUDtNBrlrNIWkqyt2fnLhhRU6AOx2PP639Tr0v4EIqQ0Q3e YCIyYwsTxJox8I+OM5V79AkvY9ez59sxGDTQaiT/13FBkXjyOajcpr0YO+IAnB/6D1pLe2x3fMrSu cT/jJOiuuX9OLNuZYppw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs27x-0000000GbDj-1lBC; Wed, 03 Apr 2024 15:02:13 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs27t-0000000GbBl-2MgZ for linux-arm-kernel@lists.infradead.org; Wed, 03 Apr 2024 15:02:11 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 2C7C21007; Wed, 3 Apr 2024 08:02:40 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 7DE443F7B4; Wed, 3 Apr 2024 08:02:07 -0700 (PDT) From: Mark Rutland To: linux-kernel@vger.kernel.org Cc: anil.s.keshavamurthy@intel.com, aou@eecs.berkeley.edu, catalin.marinas@arm.com, davem@davemloft.net, jarkko@kernel.org, linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, mhiramat@kernel.org, naveen.n.rao@linux.ibm.com, palmer@dabbelt.com, paul.walmsley@sifive.com, will@kernel.org Subject: [PATCH v2 3/4] kprobes/treewide: Explicitly override alloc/free functions Date: Wed, 3 Apr 2024 16:01:53 +0100 Message-Id: <20240403150154.667649-4-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240403150154.667649-1-mark.rutland@arm.com> References: <20240403150154.667649-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240403_080209_747551_1F67939E X-CRM114-Status: GOOD ( 21.03 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Currently architectures can override kprobes_alloc_insn_page(), but kprobes_free_insn_page() is always implemented using module_memfree(), which might not be what an architecture needs, especially as we'd like to make it possible to use kprobes without requiring MODULES. It would be nicer if architectures either: (a) Used only the generic kprobes_alloc_insn_page() and kprobes_free_insn_page(), implicitly depending on MODULES. (b) Provided their own implementation of both kprobes_alloc_insn_page() and kprobes_free_insn_page(), handling the relevant dependencies themselves. This patch applies that split treewide: (a) Architectures using the generic kprobes_free_insn_page() and kprobes_free_insn_page() are left as-is. The __weak annotation is removed from the generic implementations so that accidental overrides/misuse can be detected easily. (b) Architectures which provide their own kprobes_free_insn_page() are given a matching implementation of kprobes_free_insn_page(), and select HAVE_KPROBES_ALLOC. This new Kconfig symbol will allow subsequent patches to relax the dependency on MODULES to (MODULES || HAVE_KPROBES_ALLOC) once other module dependencies in the core kprobes code are cleaned up. Architectures which use module_alloc() are given an implementation using module_memfree() along with an explicit dependency on MODULES. Architectures using __vmalloc_node_range() are given an implementation using vfree(). This loses the warning for in_interrupt(), but vfree() can handle this via vfree_atomic(), so the warning isn't necessary. On riscv, the allocator depends on !XIP_KERNEL, which is already a dependency for HAVE_KPROBES in arch/riscv/Kconfig. As of this patch arm64 and riscv have kprobe allocation functions which do not explicitly depend on MODULES. The core kprobes code still depends on MODULES. Signed-off-by: Mark Rutland Cc: Albert Ou Cc: Anil S Keshavamurthy Cc: Catalin Marinas Cc: David S. Miller Cc: Jarkko Sakkinen Cc: Masami Hiramatsu Cc: Naveen N. Rao Cc: Palmer Dabbelt Cc: Paul Walmsley Cc: Will Deacon Reviewed-by: Jarkko Sakkinen --- arch/Kconfig | 3 +++ arch/arm64/Kconfig | 1 + arch/arm64/kernel/probes/kprobes.c | 5 +++++ arch/powerpc/Kconfig | 3 ++- arch/powerpc/kernel/kprobes.c | 5 +++++ arch/riscv/Kconfig | 1 + arch/riscv/kernel/probes/kprobes.c | 5 +++++ arch/s390/Kconfig | 3 ++- arch/s390/kernel/kprobes.c | 5 +++++ arch/x86/Kconfig | 3 ++- arch/x86/kernel/kprobes/core.c | 5 +++++ include/linux/kprobes.h | 1 + kernel/kprobes.c | 6 ++++-- 13 files changed, 41 insertions(+), 5 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 9f066785bb71d..85bb59f7b8c07 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -206,6 +206,9 @@ config HAVE_IOREMAP_PROT config HAVE_KPROBES bool +config HAVE_KPROBES_ALLOC + bool + config HAVE_KRETPROBES bool diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 7b11c98b3e84b..bda7913d6c9b8 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -233,6 +233,7 @@ config ARM64 select HAVE_STACKPROTECTOR select HAVE_SYSCALL_TRACEPOINTS select HAVE_KPROBES + select HAVE_KPROBES_ALLOC select HAVE_KRETPROBES select HAVE_GENERIC_VDSO select HOTPLUG_CORE_SYNC_DEAD if HOTPLUG_CPU diff --git a/arch/arm64/kernel/probes/kprobes.c b/arch/arm64/kernel/probes/kprobes.c index 4b6ab7b1fa211..69d19a390cd48 100644 --- a/arch/arm64/kernel/probes/kprobes.c +++ b/arch/arm64/kernel/probes/kprobes.c @@ -136,6 +136,11 @@ void *kprobes_alloc_insn_page(void) NUMA_NO_NODE, __builtin_return_address(0)); } +void kprobes_free_insn_page(void *page) +{ + vfree(page); +} + /* arm kprobe: install breakpoint in text */ void __kprobes arch_arm_kprobe(struct kprobe *p) { diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig index 1c4be33736860..13e0fc51dcdcf 100644 --- a/arch/powerpc/Kconfig +++ b/arch/powerpc/Kconfig @@ -254,7 +254,8 @@ config PPC select HAVE_KERNEL_LZMA if DEFAULT_UIMAGE select HAVE_KERNEL_LZO if DEFAULT_UIMAGE select HAVE_KERNEL_XZ if PPC_BOOK3S || 44x - select HAVE_KPROBES + select HAVE_KPROBES if MODULES + select HAVE_KPROBES_ALLOC select HAVE_KPROBES_ON_FTRACE select HAVE_KRETPROBES select HAVE_LD_DEAD_CODE_DATA_ELIMINATION if HAVE_OBJTOOL_MCOUNT && (!ARCH_USING_PATCHABLE_FUNCTION_ENTRY || (!CC_IS_GCC || GCC_VERSION >= 110100)) diff --git a/arch/powerpc/kernel/kprobes.c b/arch/powerpc/kernel/kprobes.c index 0b297718d5de6..d0332aaebab09 100644 --- a/arch/powerpc/kernel/kprobes.c +++ b/arch/powerpc/kernel/kprobes.c @@ -146,6 +146,11 @@ void *kprobes_alloc_insn_page(void) return NULL; } +void kprobes_free_insn_page(void *page) +{ + module_memfree(page); +} + int arch_prepare_kprobe(struct kprobe *p) { int ret = 0; diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index be09c8836d56b..4e22549a522a5 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -139,6 +139,7 @@ config RISCV select HAVE_GENERIC_VDSO if MMU && 64BIT select HAVE_IRQ_TIME_ACCOUNTING select HAVE_KPROBES if !XIP_KERNEL + select HAVE_KPROBES_ALLOC select HAVE_KPROBES_ON_FTRACE if !XIP_KERNEL select HAVE_KRETPROBES if !XIP_KERNEL # https://github.com/ClangBuiltLinux/linux/issues/1881 diff --git a/arch/riscv/kernel/probes/kprobes.c b/arch/riscv/kernel/probes/kprobes.c index 75201ce721057..37fdfa952d999 100644 --- a/arch/riscv/kernel/probes/kprobes.c +++ b/arch/riscv/kernel/probes/kprobes.c @@ -112,6 +112,11 @@ void *kprobes_alloc_insn_page(void) VM_FLUSH_RESET_PERMS, NUMA_NO_NODE, __builtin_return_address(0)); } + +void kprobes_free_insn_page(void *page) +{ + vfree(page); +} #endif /* install breakpoint in text */ diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig index 8f01ada6845e3..635eddc3fce80 100644 --- a/arch/s390/Kconfig +++ b/arch/s390/Kconfig @@ -193,7 +193,8 @@ config S390 select HAVE_KERNEL_UNCOMPRESSED select HAVE_KERNEL_XZ select HAVE_KERNEL_ZSTD - select HAVE_KPROBES + select HAVE_KPROBES if MODULES + select HAVE_KPROBES_ALLOC select HAVE_KPROBES_ON_FTRACE select HAVE_KRETPROBES select HAVE_LIVEPATCH diff --git a/arch/s390/kernel/kprobes.c b/arch/s390/kernel/kprobes.c index 91ca4d501d4ef..a5b142b8eb0f7 100644 --- a/arch/s390/kernel/kprobes.c +++ b/arch/s390/kernel/kprobes.c @@ -45,6 +45,11 @@ void *kprobes_alloc_insn_page(void) return page; } +void kprobes_free_insn_page(void *page) +{ + module_memfree(page); +} + static void *alloc_s390_insn_page(void) { if (xchg(&insn_page_in_use, 1) == 1) diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig index 4fff6ed46e902..0810cd0bdeca9 100644 --- a/arch/x86/Kconfig +++ b/arch/x86/Kconfig @@ -240,7 +240,8 @@ config X86 select HAVE_KERNEL_LZO select HAVE_KERNEL_XZ select HAVE_KERNEL_ZSTD - select HAVE_KPROBES + select HAVE_KPROBES if MODULES + select HAVE_KPROBES_ALLOC select HAVE_KPROBES_ON_FTRACE select HAVE_FUNCTION_ERROR_INJECTION select HAVE_KRETPROBES diff --git a/arch/x86/kernel/kprobes/core.c b/arch/x86/kernel/kprobes/core.c index 7f01bbbfa9e2a..5f093b94d9b40 100644 --- a/arch/x86/kernel/kprobes/core.c +++ b/arch/x86/kernel/kprobes/core.c @@ -508,6 +508,11 @@ void *kprobes_alloc_insn_page(void) return page; } +void kprobes_free_insn_page(void *page) +{ + module_memfree(page); +} + /* Kprobe x86 instruction emulation - only regs->ip or IF flag modifiers */ static void kprobe_emulate_ifmodifiers(struct kprobe *p, struct pt_regs *regs) diff --git a/include/linux/kprobes.h b/include/linux/kprobes.h index ad4b561100f9e..651c807727bea 100644 --- a/include/linux/kprobes.h +++ b/include/linux/kprobes.h @@ -431,6 +431,7 @@ int enable_kprobe(struct kprobe *kp); void dump_kprobe(struct kprobe *kp); void *kprobes_alloc_insn_page(void); +void kprobes_free_insn_page(void *page); void *kprobes_alloc_optinsn_page(void); void kprobes_free_optinsn_page(void *page); diff --git a/kernel/kprobes.c b/kernel/kprobes.c index 35adf56430c9b..fa2ee4e59eca2 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -110,7 +110,8 @@ enum kprobe_slot_state { SLOT_USED = 2, }; -void __weak *kprobes_alloc_insn_page(void) +#ifndef CONFIG_HAVE_KPROBES_ALLOC +void *kprobes_alloc_insn_page(void) { /* * Use module_alloc() so this page is within +/- 2GB of where the @@ -121,10 +122,11 @@ void __weak *kprobes_alloc_insn_page(void) return module_alloc(PAGE_SIZE); } -static void kprobes_free_insn_page(void *page) +void kprobes_free_insn_page(void *page) { module_memfree(page); } +#endif struct kprobe_insn_cache kprobe_insn_slots = { .mutex = __MUTEX_INITIALIZER(kprobe_insn_slots.mutex), From patchwork Wed Apr 3 15:01:54 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mark Rutland X-Patchwork-Id: 13616300 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 16C81CD1292 for ; Wed, 3 Apr 2024 15:02:33 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RdvyjPAD4jKE2CqfTOyBjd9GZVDe8ykkrqLDiCz+ZMM=; b=SLqL0KT4zGTEQH nUg4WI0WMpfHnIxAhfdZwJ5I1MMWHMmgxB3jATSS86OiP9MtN908uSAQSlXRssEcn7drDQWwDK8Ep Nk1C5T07P+g4B45U+07aiERTHQwYZD17cBSRstGBLcvgsIuIvizLYnRkLAn1eoFmo/BavXycMKbFJ ZUT5DdjuX9KEWYybOBCgwlLPEdCwcIM4d8BfhQiFpAIiC/tSxwMaCa8GhIKreJqOenUVKZNTmXvO/ t5RkQSKnSM9gzu0KLVpEBzaiRMFFlbY+3r495k/USXMy57bPqpkX3JOZQEVN1BRoaaBf2JGlsHgyM V1T2ds4m8W0k4MlsXsFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs281-0000000GbHS-3cKG; Wed, 03 Apr 2024 15:02:17 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rs27v-0000000GbCw-3cFq for linux-arm-kernel@lists.infradead.org; Wed, 03 Apr 2024 15:02:13 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9E98A1595; Wed, 3 Apr 2024 08:02:42 -0700 (PDT) Received: from lakrids.cambridge.arm.com (usa-sjc-imap-foss1.foss.arm.com [10.121.207.14]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id F00113F7B4; Wed, 3 Apr 2024 08:02:09 -0700 (PDT) From: Mark Rutland To: linux-kernel@vger.kernel.org Cc: anil.s.keshavamurthy@intel.com, aou@eecs.berkeley.edu, catalin.marinas@arm.com, davem@davemloft.net, jarkko@kernel.org, linux-arm-kernel@lists.infradead.org, mark.rutland@arm.com, mhiramat@kernel.org, naveen.n.rao@linux.ibm.com, palmer@dabbelt.com, paul.walmsley@sifive.com, will@kernel.org Subject: [PATCH v2 4/4] kprobes: Remove core dependency on modules Date: Wed, 3 Apr 2024 16:01:54 +0100 Message-Id: <20240403150154.667649-5-mark.rutland@arm.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20240403150154.667649-1-mark.rutland@arm.com> References: <20240403150154.667649-1-mark.rutland@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240403_080212_014853_0397E1F8 X-CRM114-Status: GOOD ( 21.98 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Jarkko Sakkinen Tracing with kprobes while running a monolithic kernel is currently impossible because KPROBES depends on MODULES. While this dependency is necessary when HAVE_KPROBES_ALLOC=n and the core kprobes code allocates memory using module_alloc(), all the other module-specific code only exist to handle the case when MODULES=y, and can be hidden behind ifdeffery. Add the necessary ifdeffery, and remove the dependency on MODULES=y when HAVE_KPROBES_ALLOC=y. As of this patch kprobes can be used when MODULES=n on arm64 and riscv. All other architectures still depend on MODULES, either by virtue of the core dependency on MODULES when HAVE_KPROBES_ALLOC=n, or by virtue of an explciit dependency on MODULES in arch code. Other architectures can enable support by implementing their own kprobes_alloc_insn_page() and kprobes_free_insn_page() which do not depend on MODULES. Signed-off-by: Jarkko Sakkinen Link: https://lore.kernel.org/lkml/20240326134616.7691-1-jarkko@kernel.org/ [Mark: Remove execmem changes, depend on HAVE_KPROBES_ALLOC] Signed-off-by: Mark Rutland Cc: Albert Ou Cc: Anil S Keshavamurthy Cc: Catalin Marinas Cc: David S. Miller Cc: Jarkko Sakkinen Cc: Masami Hiramatsu Cc: Naveen N. Rao Cc: Palmer Dabbelt Cc: Paul Walmsley Cc: Will Deacon Tested-by: Jarkko Sakkinen # arch/riscv --- arch/Kconfig | 2 +- kernel/kprobes.c | 46 ++++++++++++++++++++++--------------- kernel/trace/trace_kprobe.c | 15 ++++++++++-- 3 files changed, 41 insertions(+), 22 deletions(-) diff --git a/arch/Kconfig b/arch/Kconfig index 85bb59f7b8c07..0df2c88547b3c 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -52,7 +52,7 @@ config GENERIC_ENTRY config KPROBES bool "Kprobes" - depends on MODULES + depends on MODULES || HAVE_KPROBES_ALLOC depends on HAVE_KPROBES select KALLSYMS select TASKS_RCU if PREEMPTION diff --git a/kernel/kprobes.c b/kernel/kprobes.c index fa2ee4e59eca2..ec4493a41b505 100644 --- a/kernel/kprobes.c +++ b/kernel/kprobes.c @@ -1594,6 +1594,7 @@ static int check_kprobe_address_safe(struct kprobe *p, goto out; } +#ifdef CONFIG_MODULES /* * If the module freed '.init.text', we couldn't insert * kprobes in there. @@ -1604,7 +1605,9 @@ static int check_kprobe_address_safe(struct kprobe *p, *probed_mod = NULL; ret = -ENOENT; } +#endif /* CONFIG_MODULES */ } + out: preempt_enable(); jump_label_unlock(); @@ -2484,24 +2487,6 @@ int kprobe_add_area_blacklist(unsigned long start, unsigned long end) return 0; } -/* Remove all symbols in given area from kprobe blacklist */ -static void kprobe_remove_area_blacklist(unsigned long start, unsigned long end) -{ - struct kprobe_blacklist_entry *ent, *n; - - list_for_each_entry_safe(ent, n, &kprobe_blacklist, list) { - if (ent->start_addr < start || ent->start_addr >= end) - continue; - list_del(&ent->list); - kfree(ent); - } -} - -static void kprobe_remove_ksym_blacklist(unsigned long entry) -{ - kprobe_remove_area_blacklist(entry, entry + 1); -} - int __weak arch_kprobe_get_kallsym(unsigned int *symnum, unsigned long *value, char *type, char *sym) { @@ -2566,6 +2551,25 @@ static int __init populate_kprobe_blacklist(unsigned long *start, return ret ? : arch_populate_kprobe_blacklist(); } +#ifdef CONFIG_MODULES +/* Remove all symbols in given area from kprobe blacklist */ +static void kprobe_remove_area_blacklist(unsigned long start, unsigned long end) +{ + struct kprobe_blacklist_entry *ent, *n; + + list_for_each_entry_safe(ent, n, &kprobe_blacklist, list) { + if (ent->start_addr < start || ent->start_addr >= end) + continue; + list_del(&ent->list); + kfree(ent); + } +} + +static void kprobe_remove_ksym_blacklist(unsigned long entry) +{ + kprobe_remove_area_blacklist(entry, entry + 1); +} + static void add_module_kprobe_blacklist(struct module *mod) { unsigned long start, end; @@ -2662,6 +2666,9 @@ static int kprobes_module_callback(struct notifier_block *nb, mutex_unlock(&kprobe_mutex); return NOTIFY_DONE; } +#else +#define kprobes_module_callback (NULL) +#endif /* CONFIG_MODULES */ static struct notifier_block kprobe_module_nb = { .notifier_call = kprobes_module_callback, @@ -2726,7 +2733,8 @@ static int __init init_kprobes(void) err = arch_init_kprobes(); if (!err) err = register_die_notifier(&kprobe_exceptions_nb); - if (!err) + + if (!err && IS_ENABLED(CONFIG_MODULES)) err = register_module_notifier(&kprobe_module_nb); kprobes_initialized = (err == 0); diff --git a/kernel/trace/trace_kprobe.c b/kernel/trace/trace_kprobe.c index 14099cc17fc9e..c509ba776e679 100644 --- a/kernel/trace/trace_kprobe.c +++ b/kernel/trace/trace_kprobe.c @@ -111,6 +111,7 @@ static nokprobe_inline bool trace_kprobe_within_module(struct trace_kprobe *tk, return strncmp(module_name(mod), name, len) == 0 && name[len] == ':'; } +#ifdef CONFIG_MODULES static nokprobe_inline bool trace_kprobe_module_exist(struct trace_kprobe *tk) { char *p; @@ -129,6 +130,9 @@ static nokprobe_inline bool trace_kprobe_module_exist(struct trace_kprobe *tk) return ret; } +#else +#define trace_kprobe_module_exist(tk) false /* aka a module never exists */ +#endif /* CONFIG_MODULES */ static bool trace_kprobe_is_busy(struct dyn_event *ev) { @@ -670,6 +674,7 @@ static int register_trace_kprobe(struct trace_kprobe *tk) return ret; } +#ifdef CONFIG_MODULES /* Module notifier call back, checking event on the module */ static int trace_kprobe_module_callback(struct notifier_block *nb, unsigned long val, void *data) @@ -699,6 +704,9 @@ static int trace_kprobe_module_callback(struct notifier_block *nb, return NOTIFY_DONE; } +#else +#define trace_kprobe_module_callback (NULL) +#endif /* CONFIG_MODULES */ static struct notifier_block trace_kprobe_module_nb = { .notifier_call = trace_kprobe_module_callback, @@ -1933,8 +1941,11 @@ static __init int init_kprobe_trace_early(void) if (ret) return ret; - if (register_module_notifier(&trace_kprobe_module_nb)) - return -EINVAL; + if (IS_ENABLED(CONFIG_MODULES)) { + ret = register_module_notifier(&trace_kprobe_module_nb); + if (ret) + return -EINVAL; + } return 0; }