From patchwork Tue Feb 18 19:59:19 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 13980824 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8ACDBC021B0 for ; Tue, 18 Feb 2025 20:28:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:Subject:Cc:To:From:Date:Message-ID:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: List-Owner; bh=QcGvTVwv0SX6bUdFmpY58vbQdWKKiBSIgUrxWyySk20=; b=uAmGOGk9Fe57W4 HSnN2fzID2q6RSj+Fp7E5HKH4saOyKupSZkWLZaiPRhTCdhyKminGcZsw8Y92ZiGTGvZM/lueXHk2 UdhvhYSHQfBidlT4oTxOcWzNKTRC9XxVeFv5QAlmYSuWxG6fVUR6D1MweA/ZQhGV69pNl04woWjJO tFfSxKXODGhyVJosaQFMv47FYcnHgp99a89zlmXsDRmbAmxUEhWJPkdr1SvLzGGZlU3M1rP1IUM3n lOy30wH8nNAdONkU/we8gDdU7vhzE8NyJDVVRxS/bSAqVAuB71MbvxyJSVRwYGZPrdpY/Gmh7AzSi 7MuicztAoIpZldhGXJoQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tkUCS-00000009ler-2OmE; Tue, 18 Feb 2025 20:28:12 +0000 Received: from nyc.source.kernel.org ([147.75.193.91]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tkTlB-00000009flf-1XTO for linux-arm-kernel@lists.infradead.org; Tue, 18 Feb 2025 20:00:03 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 9399EA41571; Tue, 18 Feb 2025 19:58:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id CB20AC4CEEB; Tue, 18 Feb 2025 19:59:59 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.98) (envelope-from ) id 1tkTlW-00000004ApJ-2Ann; Tue, 18 Feb 2025 15:00:22 -0500 Message-ID: <20250218200022.373319428@goodmis.org> User-Agent: quilt/0.68 Date: Tue, 18 Feb 2025 14:59:19 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, bpf , linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Peter Zijlstra , Linus Torvalds , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Zheng Yejian , Martin Kelly , Christophe Leroy , Josh Poimboeuf , Heiko Carstens , Catalin Marinas , Will Deacon , Vasily Gorbik , Alexander Gordeev Subject: [PATCH v5 1/6] arm64: scripts/sorttable: Implement sorting mcount_loc at boot for arm64 References: <20250218195918.255228630@goodmis.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250218_120001_557651_FA4B269A X-CRM114-Status: GOOD ( 27.10 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Steven Rostedt The mcount_loc section holds the addresses of the functions that get patched by ftrace when enabling function callbacks. It can contain tens of thousands of entries. These addresses must be sorted. If they are not sorted at compile time, they are sorted at boot. Sorting at boot does take some time and does have a small impact on boot performance. x86 and arm32 have the addresses in the mcount_loc section of the ELF file. But for arm64, the section just contains zeros. The .rela.dyn Elf_Rela section holds the addresses and they get patched at boot during the relocation phase. In order to sort these addresses, the Elf_Rela needs to be updated instead of the location in the binary that holds the mcount_loc section. Have the sorttable code, allocate an array to hold the functions, load the addresses from the Elf_Rela entries, sort them, then put them back in order into the Elf_rela entries so that they will be sorted at boot up without having to sort them during boot up. Acked-by: Catalin Marinas Signed-off-by: Steven Rostedt (Google) --- arch/arm64/Kconfig | 1 + scripts/sorttable.c | 185 +++++++++++++++++++++++++++++++++++++++++++- 2 files changed, 183 insertions(+), 3 deletions(-) diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig index 940343beb3d4..4521ecefc031 100644 --- a/arch/arm64/Kconfig +++ b/arch/arm64/Kconfig @@ -217,6 +217,7 @@ config ARM64 if DYNAMIC_FTRACE_WITH_ARGS select HAVE_SAMPLE_FTRACE_DIRECT select HAVE_SAMPLE_FTRACE_DIRECT_MULTI + select HAVE_BUILDTIME_MCOUNT_SORT select HAVE_EFFICIENT_UNALIGNED_ACCESS select HAVE_GUP_FAST select HAVE_FTRACE_GRAPH_FUNC diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 9f41575afd7a..4a34c275123e 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -28,6 +28,7 @@ #include #include #include +#include #include #include #include @@ -79,10 +80,16 @@ typedef union { Elf64_Sym e64; } Elf_Sym; +typedef union { + Elf32_Rela e32; + Elf64_Rela e64; +} Elf_Rela; + static uint32_t (*r)(const uint32_t *); static uint16_t (*r2)(const uint16_t *); static uint64_t (*r8)(const uint64_t *); static void (*w)(uint32_t, uint32_t *); +static void (*w8)(uint64_t, uint64_t *); typedef void (*table_sort_t)(char *, int); static struct elf_funcs { @@ -102,6 +109,10 @@ static struct elf_funcs { uint32_t (*sym_name)(Elf_Sym *sym); uint64_t (*sym_value)(Elf_Sym *sym); uint16_t (*sym_shndx)(Elf_Sym *sym); + uint64_t (*rela_offset)(Elf_Rela *rela); + uint64_t (*rela_info)(Elf_Rela *rela); + uint64_t (*rela_addend)(Elf_Rela *rela); + void (*rela_write_addend)(Elf_Rela *rela, uint64_t val); } e; static uint64_t ehdr64_shoff(Elf_Ehdr *ehdr) @@ -262,6 +273,38 @@ SYM_ADDR(value) SYM_WORD(name) SYM_HALF(shndx) +#define __maybe_unused __attribute__((__unused__)) + +#define RELA_ADDR(fn_name) \ +static uint64_t rela64_##fn_name(Elf_Rela *rela) \ +{ \ + return r8((uint64_t *)&rela->e64.r_##fn_name); \ +} \ + \ +static uint64_t rela32_##fn_name(Elf_Rela *rela) \ +{ \ + return r((uint32_t *)&rela->e32.r_##fn_name); \ +} \ + \ +static uint64_t __maybe_unused rela_##fn_name(Elf_Rela *rela) \ +{ \ + return e.rela_##fn_name(rela); \ +} + +RELA_ADDR(offset) +RELA_ADDR(info) +RELA_ADDR(addend) + +static void rela64_write_addend(Elf_Rela *rela, uint64_t val) +{ + w8(val, (uint64_t *)&rela->e64.r_addend); +} + +static void rela32_write_addend(Elf_Rela *rela, uint64_t val) +{ + w(val, (uint32_t *)&rela->e32.r_addend); +} + /* * Get the whole file as a programming convenience in order to avoid * malloc+lseek+read+free of many pieces. If successful, then mmap @@ -341,6 +384,16 @@ static void wle(uint32_t val, uint32_t *x) put_unaligned_le32(val, x); } +static void w8be(uint64_t val, uint64_t *x) +{ + put_unaligned_be64(val, x); +} + +static void w8le(uint64_t val, uint64_t *x) +{ + put_unaligned_le64(val, x); +} + /* * Move reserved section indices SHN_LORESERVE..SHN_HIRESERVE out of * the way to -256..-1, to avoid conflicting with real section @@ -398,13 +451,12 @@ static inline void *get_index(void *start, int entsize, int index) static int extable_ent_size; static int long_size; +#define ERRSTR_MAXSZ 256 #ifdef UNWINDER_ORC_ENABLED /* ORC unwinder only support X86_64 */ #include -#define ERRSTR_MAXSZ 256 - static char g_err[ERRSTR_MAXSZ]; static int *g_orc_ip_table; static struct orc_entry *g_orc_table; @@ -499,7 +551,19 @@ static void *sort_orctable(void *arg) #endif #ifdef MCOUNT_SORT_ENABLED + +/* Only used for sorting mcount table */ +static void rela_write_addend(Elf_Rela *rela, uint64_t val) +{ + e.rela_write_addend(rela, val); +} + static pthread_t mcount_sort_thread; +static bool sort_reloc; + +static long rela_type; + +static char m_err[ERRSTR_MAXSZ]; struct elf_mcount_loc { Elf_Ehdr *ehdr; @@ -508,6 +572,103 @@ struct elf_mcount_loc { uint64_t stop_mcount_loc; }; +/* Sort the relocations not the address itself */ +static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) +{ + Elf_Shdr *shdr_start; + Elf_Rela *rel; + unsigned int shnum; + unsigned int count; + int shentsize; + void *vals; + void *ptr; + + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); + shentsize = ehdr_shentsize(ehdr); + + vals = malloc(long_size * size); + if (!vals) { + snprintf(m_err, ERRSTR_MAXSZ, "Failed to allocate sort array"); + pthread_exit(m_err); + return NULL; + } + + ptr = vals; + + shnum = ehdr_shnum(ehdr); + if (shnum == SHN_UNDEF) + shnum = shdr_size(shdr_start); + + for (int i = 0; i < shnum; i++) { + Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); + void *end; + + if (shdr_type(shdr) != SHT_RELA) + continue; + + rel = (void *)ehdr + shdr_offset(shdr); + end = (void *)rel + shdr_size(shdr); + + for (; (void *)rel < end; rel = (void *)rel + shdr_entsize(shdr)) { + uint64_t offset = rela_offset(rel); + + if (offset >= start_loc && offset < start_loc + size) { + if (ptr + long_size > vals + size) { + free(vals); + snprintf(m_err, ERRSTR_MAXSZ, + "Too many relocations"); + pthread_exit(m_err); + return NULL; + } + + /* Make sure this has the correct type */ + if (rela_info(rel) != rela_type) { + free(vals); + snprintf(m_err, ERRSTR_MAXSZ, + "rela has type %lx but expected %lx\n", + (long)rela_info(rel), rela_type); + pthread_exit(m_err); + return NULL; + } + + if (long_size == 4) + *(uint32_t *)ptr = rela_addend(rel); + else + *(uint64_t *)ptr = rela_addend(rel); + ptr += long_size; + } + } + } + count = ptr - vals; + qsort(vals, count / long_size, long_size, compare_extable); + + ptr = vals; + for (int i = 0; i < shnum; i++) { + Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); + void *end; + + if (shdr_type(shdr) != SHT_RELA) + continue; + + rel = (void *)ehdr + shdr_offset(shdr); + end = (void *)rel + shdr_size(shdr); + + for (; (void *)rel < end; rel = (void *)rel + shdr_entsize(shdr)) { + uint64_t offset = rela_offset(rel); + + if (offset >= start_loc && offset < start_loc + size) { + if (long_size == 4) + rela_write_addend(rel, *(uint32_t *)ptr); + else + rela_write_addend(rel, *(uint64_t *)ptr); + ptr += long_size; + } + } + } + free(vals); + return NULL; +} + /* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */ static void *sort_mcount_loc(void *arg) { @@ -517,6 +678,9 @@ static void *sort_mcount_loc(void *arg) uint64_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; unsigned char *start_loc = (void *)emloc->ehdr + offset; + if (sort_reloc) + return sort_relocs(emloc->ehdr, emloc->start_mcount_loc, count); + qsort(start_loc, count/long_size, long_size, compare_extable); return NULL; } @@ -866,12 +1030,14 @@ static int do_file(char const *const fname, void *addr) r2 = r2le; r8 = r8le; w = wle; + w8 = w8le; break; case ELFDATA2MSB: r = rbe; r2 = r2be; r8 = r8be; w = wbe; + w8 = w8be; break; default: fprintf(stderr, "unrecognized ELF data encoding %d: %s\n", @@ -887,8 +1053,13 @@ static int do_file(char const *const fname, void *addr) } switch (r2(&ehdr->e32.e_machine)) { - case EM_386: case EM_AARCH64: +#ifdef MCOUNT_SORT_ENABLED + sort_reloc = true; + rela_type = 0x403; +#endif + /* fallthrough */ + case EM_386: case EM_LOONGARCH: case EM_RISCV: case EM_S390: @@ -932,6 +1103,10 @@ static int do_file(char const *const fname, void *addr) .sym_name = sym32_name, .sym_value = sym32_value, .sym_shndx = sym32_shndx, + .rela_offset = rela32_offset, + .rela_info = rela32_info, + .rela_addend = rela32_addend, + .rela_write_addend = rela32_write_addend, }; e = efuncs; @@ -965,6 +1140,10 @@ static int do_file(char const *const fname, void *addr) .sym_name = sym64_name, .sym_value = sym64_value, .sym_shndx = sym64_shndx, + .rela_offset = rela64_offset, + .rela_info = rela64_info, + .rela_addend = rela64_addend, + .rela_write_addend = rela64_write_addend, }; e = efuncs; From patchwork Tue Feb 18 19:59:20 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 13980826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id ECAD7C021AA for ; Tue, 18 Feb 2025 20:31:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:Subject:Cc:To:From:Date:Message-ID:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: List-Owner; bh=/UbQxd2gHuwSazOzBCkfogNyge30ll4F/uTuo1uhry4=; b=GLhqmLCvc6QkDH 39pAgeL/XmT8U7kd6f//ZHCyPUqDILIiHtZsbZcuV+EryLuh1UF9HvCllyvJxCf6V3/f6mJluQgfQ GKpmklzGd6ZFO2Estk6nO3TJwpiV5V0EVpvk7tK2tFYDQGem5vEDE3BQKFLv/IzMHeMk4CslRgtEJ j8s9X0z8ipd2mc0w61MOVCOXw+ONaT+uFeqEHUOvdQVuHGYjhzkqgGnwuhtk970QghGNR7AHRJbDS yuyCCd3YUTOT6q5h9OAdnho2jd3323EWNq96xP2McZCSUyv2xKjLqs8VXezdl/zz3H9ekz5zRETvO fxfPAsORigi66XYz/apg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tkUFM-00000009mO0-0UoH; Tue, 18 Feb 2025 20:31:12 +0000 Received: from nyc.source.kernel.org ([2604:1380:45d1:ec00::3]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tkTlB-00000009fmD-3Qmq for linux-arm-kernel@lists.infradead.org; Tue, 18 Feb 2025 20:00:05 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id A767FA41A8D; Tue, 18 Feb 2025 19:58:15 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 04986C4CEEE; Tue, 18 Feb 2025 20:00:00 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.98) (envelope-from ) id 1tkTlW-00000004App-2tWh; Tue, 18 Feb 2025 15:00:22 -0500 Message-ID: <20250218200022.538888594@goodmis.org> User-Agent: quilt/0.68 Date: Tue, 18 Feb 2025 14:59:20 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, bpf , linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Peter Zijlstra , Linus Torvalds , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Zheng Yejian , Martin Kelly , Christophe Leroy , Josh Poimboeuf , Heiko Carstens , Catalin Marinas , Will Deacon , Vasily Gorbik , Alexander Gordeev Subject: [PATCH v5 2/6] scripts/sorttable: Have mcount rela sort use direct values References: <20250218195918.255228630@goodmis.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250218_120002_000303_4E9B2B85 X-CRM114-Status: GOOD ( 16.66 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Steven Rostedt The mcount_loc sorting for when the values are stored in the Elf_Rela entries uses the compare_extable() function to do the compares in the qsort(). That function does handle byte swapping if the machine being compiled for is a different endian than the host machine. But the sort_relocs() function sorts an array that pulled in the values from the Elf_Rela section and has already done the swapping. Create two new compare functions that will sort the direct values. One will sort 32 bit values and the other will sort the 64 bit value. One of these will be assigned to a compare_values function pointer and that will be used for sorting the Elf_Rela mcount values. Signed-off-by: Steven Rostedt (Google) --- scripts/sorttable.c | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-) diff --git a/scripts/sorttable.c b/scripts/sorttable.c index 4a34c275123e..f62a91d8af0a 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -552,6 +552,28 @@ static void *sort_orctable(void *arg) #ifdef MCOUNT_SORT_ENABLED +static int compare_values_64(const void *a, const void *b) +{ + uint64_t av = *(uint64_t *)a; + uint64_t bv = *(uint64_t *)b; + + if (av < bv) + return -1; + return av > bv; +} + +static int compare_values_32(const void *a, const void *b) +{ + uint32_t av = *(uint32_t *)a; + uint32_t bv = *(uint32_t *)b; + + if (av < bv) + return -1; + return av > bv; +} + +static int (*compare_values)(const void *a, const void *b); + /* Only used for sorting mcount table */ static void rela_write_addend(Elf_Rela *rela, uint64_t val) { @@ -583,6 +605,8 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) void *vals; void *ptr; + compare_values = long_size == 4 ? compare_values_32 : compare_values_64; + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); shentsize = ehdr_shentsize(ehdr); @@ -640,7 +664,7 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) } } count = ptr - vals; - qsort(vals, count / long_size, long_size, compare_extable); + qsort(vals, count / long_size, long_size, compare_values); ptr = vals; for (int i = 0; i < shnum; i++) { From patchwork Tue Feb 18 19:59:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 13980825 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 17F54C021AD for ; Tue, 18 Feb 2025 20:29:52 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:Subject:Cc:To:From:Date:Message-ID:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: List-Owner; bh=5RbhS+zMfnJaqNmZ2hrpoGWgBgTc8EWmgM+o7CODprw=; b=K3UOrFSL3Y+5do 4iYQMp0K0hn2rb+cbQ6l9HdqVmocNq/9ELwZs67F8kWnilbfWUlCTN5YudyDjGfABloguqwqHx2Ch oLC0zLzDI9KkaXwSEc94jjiA5IXDYRuRl+CjYWPeCFbq3PgyiKblss+ATmDL8dYiWzDOvsm1LWOzi I+4HhCJE796GkT0D8W62R6LNL9LWXBTAdohyPV6ZsTt7baIyPUF4btLgEuJ3L284ViYZHZaPLqo4+ nME/9xppVCwH+Lr/oHOW21WNn98UEGjP8KybFJDwjzNSY2eNp5gGzzoYd4vLyC0zfIcsqknALbiRD BGjzRH5y9PtgkfmFCnFg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tkUDu-00000009m3Z-1pgC; Tue, 18 Feb 2025 20:29:42 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tkTlB-00000009flh-22p9 for linux-arm-kernel@lists.infradead.org; Tue, 18 Feb 2025 20:00:04 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 7B71B5C6250; Tue, 18 Feb 2025 19:59:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3757FC4CEE2; Tue, 18 Feb 2025 20:00:00 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.98) (envelope-from ) id 1tkTlW-00000004AqJ-3cQ4; Tue, 18 Feb 2025 15:00:22 -0500 Message-ID: <20250218200022.710676551@goodmis.org> User-Agent: quilt/0.68 Date: Tue, 18 Feb 2025 14:59:21 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, bpf , linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Peter Zijlstra , Linus Torvalds , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Zheng Yejian , Martin Kelly , Christophe Leroy , Josh Poimboeuf , Heiko Carstens , Catalin Marinas , Will Deacon , Vasily Gorbik , Alexander Gordeev Subject: [PATCH v5 3/6] scripts/sorttable: Always use an array for the mcount_loc sorting References: <20250218195918.255228630@goodmis.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250218_120001_644923_CA6C1284 X-CRM114-Status: GOOD ( 21.27 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Steven Rostedt The sorting of the mcount_loc section is done directly to the section for x86 and arm32 but it uses a separate array for arm64 as arm64 has the values for the mcount_loc stored in the rela sections of the vmlinux ELF file. In order to use the same code to remove weak functions, always use a separate array to do the sorting. This requires splitting up the filling of the array into one function and the placing the contents of the array back into the rela sections or into the mcount_loc section into a separate file. Signed-off-by: Steven Rostedt (Google) --- scripts/sorttable.c | 122 ++++++++++++++++++++++++++++++++------------ 1 file changed, 90 insertions(+), 32 deletions(-) diff --git a/scripts/sorttable.c b/scripts/sorttable.c index f62a91d8af0a..ec02a2852efb 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -594,31 +594,19 @@ struct elf_mcount_loc { uint64_t stop_mcount_loc; }; -/* Sort the relocations not the address itself */ -static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) +/* Fill the array with the content of the relocs */ +static int fill_relocs(void *ptr, uint64_t size, Elf_Ehdr *ehdr, uint64_t start_loc) { Elf_Shdr *shdr_start; Elf_Rela *rel; unsigned int shnum; - unsigned int count; + unsigned int count = 0; int shentsize; - void *vals; - void *ptr; - - compare_values = long_size == 4 ? compare_values_32 : compare_values_64; + void *array_end = ptr + size; shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); shentsize = ehdr_shentsize(ehdr); - vals = malloc(long_size * size); - if (!vals) { - snprintf(m_err, ERRSTR_MAXSZ, "Failed to allocate sort array"); - pthread_exit(m_err); - return NULL; - } - - ptr = vals; - shnum = ehdr_shnum(ehdr); if (shnum == SHN_UNDEF) shnum = shdr_size(shdr_start); @@ -637,22 +625,18 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) uint64_t offset = rela_offset(rel); if (offset >= start_loc && offset < start_loc + size) { - if (ptr + long_size > vals + size) { - free(vals); + if (ptr + long_size > array_end) { snprintf(m_err, ERRSTR_MAXSZ, "Too many relocations"); - pthread_exit(m_err); - return NULL; + return -1; } /* Make sure this has the correct type */ if (rela_info(rel) != rela_type) { - free(vals); snprintf(m_err, ERRSTR_MAXSZ, "rela has type %lx but expected %lx\n", (long)rela_info(rel), rela_type); - pthread_exit(m_err); - return NULL; + return -1; } if (long_size == 4) @@ -660,13 +644,28 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) else *(uint64_t *)ptr = rela_addend(rel); ptr += long_size; + count++; } } } - count = ptr - vals; - qsort(vals, count / long_size, long_size, compare_values); + return count; +} + +/* Put the sorted vals back into the relocation elements */ +static void replace_relocs(void *ptr, uint64_t size, Elf_Ehdr *ehdr, uint64_t start_loc) +{ + Elf_Shdr *shdr_start; + Elf_Rela *rel; + unsigned int shnum; + int shentsize; + + shdr_start = (Elf_Shdr *)((char *)ehdr + ehdr_shoff(ehdr)); + shentsize = ehdr_shentsize(ehdr); + + shnum = ehdr_shnum(ehdr); + if (shnum == SHN_UNDEF) + shnum = shdr_size(shdr_start); - ptr = vals; for (int i = 0; i < shnum; i++) { Elf_Shdr *shdr = get_index(shdr_start, shentsize, i); void *end; @@ -689,8 +688,32 @@ static void *sort_relocs(Elf_Ehdr *ehdr, uint64_t start_loc, uint64_t size) } } } - free(vals); - return NULL; +} + +static int fill_addrs(void *ptr, uint64_t size, void *addrs) +{ + void *end = ptr + size; + int count = 0; + + for (; ptr < end; ptr += long_size, addrs += long_size, count++) { + if (long_size == 4) + *(uint32_t *)ptr = r(addrs); + else + *(uint64_t *)ptr = r8(addrs); + } + return count; +} + +static void replace_addrs(void *ptr, uint64_t size, void *addrs) +{ + void *end = ptr + size; + + for (; ptr < end; ptr += long_size, addrs += long_size) { + if (long_size == 4) + w(*(uint32_t *)ptr, addrs); + else + w8(*(uint64_t *)ptr, addrs); + } } /* Sort the addresses stored between __start_mcount_loc to __stop_mcount_loc in vmlinux */ @@ -699,14 +722,49 @@ static void *sort_mcount_loc(void *arg) struct elf_mcount_loc *emloc = (struct elf_mcount_loc *)arg; uint64_t offset = emloc->start_mcount_loc - shdr_addr(emloc->init_data_sec) + shdr_offset(emloc->init_data_sec); - uint64_t count = emloc->stop_mcount_loc - emloc->start_mcount_loc; + uint64_t size = emloc->stop_mcount_loc - emloc->start_mcount_loc; unsigned char *start_loc = (void *)emloc->ehdr + offset; + Elf_Ehdr *ehdr = emloc->ehdr; + void *e_msg = NULL; + void *vals; + int count; + + vals = malloc(long_size * size); + if (!vals) { + snprintf(m_err, ERRSTR_MAXSZ, "Failed to allocate sort array"); + pthread_exit(m_err); + } if (sort_reloc) - return sort_relocs(emloc->ehdr, emloc->start_mcount_loc, count); + count = fill_relocs(vals, size, ehdr, emloc->start_mcount_loc); + else + count = fill_addrs(vals, size, start_loc); + + if (count < 0) { + e_msg = m_err; + goto out; + } + + if (count != size / long_size) { + snprintf(m_err, ERRSTR_MAXSZ, "Expected %u mcount elements but found %u\n", + (int)(size / long_size), count); + e_msg = m_err; + goto out; + } + + compare_values = long_size == 4 ? compare_values_32 : compare_values_64; + + qsort(vals, count, long_size, compare_values); + + if (sort_reloc) + replace_relocs(vals, size, ehdr, emloc->start_mcount_loc); + else + replace_addrs(vals, size, start_loc); + +out: + free(vals); - qsort(start_loc, count/long_size, long_size, compare_extable); - return NULL; + pthread_exit(e_msg); } /* Get the address of __start_mcount_loc and __stop_mcount_loc in System.map */ From patchwork Tue Feb 18 19:59:22 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 13980833 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7D3D0C021AD for ; Tue, 18 Feb 2025 20:35:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:Subject:Cc:To:From:Date:Message-ID:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: List-Owner; bh=Bi8fC0jCdsz21AqvSdU0kXPYT2U+dUBsVQ7lQCLLBH8=; b=WX2IA/hkHZL3dx aAdaFrLMRZ28K+d/zpIuFNmbbdcR4Q+JsFOO2BaCmpMehQCigBOVHDe4BCGczTSIoeEXZ1YEp+CcP zROfv+9Ma70MAup6d3hMk86X4gHsxl5R37vHNCE/SvjP11Ff1yqnmLpPvzruHk7LO/HvlA47IfGa7 PqkxRHiEfJLc+p5G2zkIuL2kM49VGcg14x7ztwDN6EPFynG9nD3EICHXBdLqKmEHh9vSAbbRXkR5Z x+vLKefJEOFas8LY13qpxBsl31RBKhqfBTZfY2oU+jtVWPR8ShSxx3BkB+RRA8uFQNtBzQ58wt27u q6XJL/pbdSQo8ymPFrQw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tkUJh-00000009nTH-0o7P; Tue, 18 Feb 2025 20:35:41 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tkTlN-00000009fuY-21Pq for linux-arm-kernel@bombadil.infradead.org; Tue, 18 Feb 2025 20:00:13 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Type:MIME-Version:References: Subject:Cc:To:From:Date:Message-ID:Sender:Reply-To:Content-Transfer-Encoding: Content-ID:Content-Description:In-Reply-To; bh=Bi8fC0jCdsz21AqvSdU0kXPYT2U+dUBsVQ7lQCLLBH8=; b=fUsRDojwFqZkqyMorG2/3ogeDK kDrWE1j+8PQE6rfS+Ua3R+G23goYEY5vREF5cCel2d4y+CNKjJ4/Iq+2WtcDQ16uR2AT1YtmhIS62 NtKmPq1tYZgXYK4dE0Eie9KhyhEeA7YXPLFZFERt0X2t/Um90z5sOWRC4gGG0mI3saH84GE5KQ8QS 3fXwitsTDT6IaTDiND9LFT8vel9s5mTo3s2Zrig+JwkrTt4wDUx5z8PRKSARYlOVQmXyKYFrJJM/2 wzF2VQLCLFnFBPnyOi6nppzvYGpoQYLcrqlnpMFsh3h+rjLScCiccwDUhZQ0Ce6X09nu4LSF+KM78 IWnKnO8g==; Received: from nyc.source.kernel.org ([147.75.193.91]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tkTlH-000000022VF-09tU for linux-arm-kernel@lists.infradead.org; Tue, 18 Feb 2025 20:00:10 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 34212A41B0B; Tue, 18 Feb 2025 19:58:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5E7CAC4CEFA; Tue, 18 Feb 2025 20:00:00 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.98) (envelope-from ) id 1tkTlX-00000004Aqo-0918; Tue, 18 Feb 2025 15:00:23 -0500 Message-ID: <20250218200022.883095980@goodmis.org> User-Agent: quilt/0.68 Date: Tue, 18 Feb 2025 14:59:22 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, bpf , linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Peter Zijlstra , Linus Torvalds , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Zheng Yejian , Martin Kelly , Christophe Leroy , Josh Poimboeuf , Heiko Carstens , Catalin Marinas , Will Deacon , Vasily Gorbik , Alexander Gordeev Subject: [PATCH v5 4/6] scripts/sorttable: Zero out weak functions in mcount_loc table References: <20250218195918.255228630@goodmis.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250218_200007_526188_47680B4D X-CRM114-Status: GOOD ( 36.72 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Steven Rostedt When a function is annotated as "weak" and is overridden, the code is not removed. If it is traced, the fentry/mcount location in the weak function will be referenced by the "__mcount_loc" section. This will then be added to the available_filter_functions list. Since only the address of the functions are listed, to find the name to show, a search of kallsyms is used. Since kallsyms will return the function by simply finding the function that the address is after but before the next function, an address of a weak function will show up as the function before it. This is because kallsyms does not save names of weak functions. This has caused issues in the past, as now the traced weak function will be listed in available_filter_functions with the name of the function before it. At best, this will cause the previous function's name to be listed twice. At worse, if the previous function was marked notrace, it will now show up as a function that can be traced. Note that it only shows up that it can be traced but will not be if enabled, which causes confusion. https://lore.kernel.org/all/20220412094923.0abe90955e5db486b7bca279@kernel.org/ The commit b39181f7c6907 ("ftrace: Add FTRACE_MCOUNT_MAX_OFFSET to avoid adding weak function") was a workaround to this by checking the function address before printing its name. If the address was too far from the function given by the name then instead of printing the name it would print: __ftrace_invalid_address___ The real issue is that these invalid addresses are listed in the ftrace table look up which available_filter_functions is derived from. A place holder must be listed in that file because set_ftrace_filter may take a series of indexes into that file instead of names to be able to do O(1) lookups to enable filtering (many tools use this method). Even if kallsyms saved the size of the function, it does not remove the need of having these place holders. The real solution is to not add a weak function into the ftrace table in the first place. To solve this, the sorttable.c code that sorts the mcount regions during the build is modified to take a "nm -S vmlinux" input, sort it, and any function listed in the mcount_loc section that is not within a boundary of the function list given by nm is considered a weak function and is zeroed out. Note, this does not mean they will remain zero when booting as KASLR will still shift those addresses. To handle this, the entries in the mcount_loc section will be ignored if they are zero or match the kaslr_offset() value. Before: ~# grep __ftrace_invalid_address___ /sys/kernel/tracing/available_filter_functions | wc -l 551 After: ~# grep __ftrace_invalid_address___ /sys/kernel/tracing/available_filter_functions | wc -l 0 Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ftrace.c | 6 +- scripts/link-vmlinux.sh | 4 +- scripts/sorttable.c | 128 +++++++++++++++++++++++++++++++++++++++- 3 files changed, 134 insertions(+), 4 deletions(-) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 728ecda6e8d4..e3f89924f603 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -7004,6 +7004,7 @@ static int ftrace_process_locs(struct module *mod, unsigned long count; unsigned long *p; unsigned long addr; + unsigned long kaslr; unsigned long flags = 0; /* Shut up gcc */ int ret = -ENOMEM; @@ -7052,6 +7053,9 @@ static int ftrace_process_locs(struct module *mod, ftrace_pages->next = start_pg; } + /* For zeroed locations that were shifted for core kernel */ + kaslr = !mod ? kaslr_offset() : 0; + p = start; pg = start_pg; while (p < end) { @@ -7063,7 +7067,7 @@ static int ftrace_process_locs(struct module *mod, * object files to satisfy alignments. * Skip any NULL pointers. */ - if (!addr) { + if (!addr || addr == kaslr) { skipped++; continue; } diff --git a/scripts/link-vmlinux.sh b/scripts/link-vmlinux.sh index 56a077d204cf..59b07fe6fd00 100755 --- a/scripts/link-vmlinux.sh +++ b/scripts/link-vmlinux.sh @@ -177,12 +177,14 @@ mksysmap() sorttable() { - ${objtree}/scripts/sorttable ${1} + ${NM} -S ${1} > .tmp_vmlinux.nm-sort + ${objtree}/scripts/sorttable -s .tmp_vmlinux.nm-sort ${1} } cleanup() { rm -f .btf.* + rm -f .tmp_vmlinux.nm-sort rm -f System.map rm -f vmlinux rm -f vmlinux.map diff --git a/scripts/sorttable.c b/scripts/sorttable.c index ec02a2852efb..23c7e0e6c024 100644 --- a/scripts/sorttable.c +++ b/scripts/sorttable.c @@ -580,6 +580,98 @@ static void rela_write_addend(Elf_Rela *rela, uint64_t val) e.rela_write_addend(rela, val); } +struct func_info { + uint64_t addr; + uint64_t size; +}; + +/* List of functions created by: nm -S vmlinux */ +static struct func_info *function_list; +static int function_list_size; + +/* Allocate functions in 1k blocks */ +#define FUNC_BLK_SIZE 1024 +#define FUNC_BLK_MASK (FUNC_BLK_SIZE - 1) + +static int add_field(uint64_t addr, uint64_t size) +{ + struct func_info *fi; + int fsize = function_list_size; + + if (!(fsize & FUNC_BLK_MASK)) { + fsize += FUNC_BLK_SIZE; + fi = realloc(function_list, fsize * sizeof(struct func_info)); + if (!fi) + return -1; + function_list = fi; + } + fi = &function_list[function_list_size++]; + fi->addr = addr; + fi->size = size; + return 0; +} + +/* Only return match if the address lies inside the function size */ +static int cmp_func_addr(const void *K, const void *A) +{ + uint64_t key = *(const uint64_t *)K; + const struct func_info *a = A; + + if (key < a->addr) + return -1; + return key >= a->addr + a->size; +} + +/* Find the function in function list that is bounded by the function size */ +static int find_func(uint64_t key) +{ + return bsearch(&key, function_list, function_list_size, + sizeof(struct func_info), cmp_func_addr) != NULL; +} + +static int cmp_funcs(const void *A, const void *B) +{ + const struct func_info *a = A; + const struct func_info *b = B; + + if (a->addr < b->addr) + return -1; + return a->addr > b->addr; +} + +static int parse_symbols(const char *fname) +{ + FILE *fp; + char addr_str[20]; /* Only need 17, but round up to next int size */ + char size_str[20]; + char type; + + fp = fopen(fname, "r"); + if (!fp) { + perror(fname); + return -1; + } + + while (fscanf(fp, "%16s %16s %c %*s\n", addr_str, size_str, &type) == 3) { + uint64_t addr; + uint64_t size; + + /* Only care about functions */ + if (type != 't' && type != 'T' && type != 'W') + continue; + + addr = strtoull(addr_str, NULL, 16); + size = strtoull(size_str, NULL, 16); + if (add_field(addr, size) < 0) + return -1; + } + fclose(fp); + + qsort(function_list, function_list_size, sizeof(struct func_info), cmp_funcs); + + return 0; +} + static pthread_t mcount_sort_thread; static bool sort_reloc; @@ -752,6 +844,21 @@ static void *sort_mcount_loc(void *arg) goto out; } + /* zero out any locations not found by function list */ + if (function_list_size) { + for (void *ptr = vals; ptr < vals + size; ptr += long_size) { + uint64_t key; + + key = long_size == 4 ? r((uint32_t *)ptr) : r8((uint64_t *)ptr); + if (!find_func(key)) { + if (long_size == 4) + *(uint32_t *)ptr = 0; + else + *(uint64_t *)ptr = 0; + } + } + } + compare_values = long_size == 4 ? compare_values_32 : compare_values_64; qsort(vals, count, long_size, compare_values); @@ -801,6 +908,8 @@ static void get_mcount_loc(struct elf_mcount_loc *emloc, Elf_Shdr *symtab_sec, return; } } +#else /* MCOUNT_SORT_ENABLED */ +static inline int parse_symbols(const char *fname) { return 0; } #endif static int do_sort(Elf_Ehdr *ehdr, @@ -1256,14 +1365,29 @@ int main(int argc, char *argv[]) int i, n_error = 0; /* gcc-4.3.0 false positive complaint */ size_t size = 0; void *addr = NULL; + int c; + + while ((c = getopt(argc, argv, "s:")) >= 0) { + switch (c) { + case 's': + if (parse_symbols(optarg) < 0) { + fprintf(stderr, "Could not parse %s\n", optarg); + return -1; + } + break; + default: + fprintf(stderr, "usage: sorttable [-s nm-file] vmlinux...\n"); + return 0; + } + } - if (argc < 2) { + if ((argc - optind) < 1) { fprintf(stderr, "usage: sorttable vmlinux...\n"); return 0; } /* Process each file in turn, allowing deep failure. */ - for (i = 1; i < argc; i++) { + for (i = optind; i < argc; i++) { addr = mmap_file(argv[i], &size); if (!addr) { ++n_error; From patchwork Tue Feb 18 19:59:23 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 13980832 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B8B5AC021AD for ; Tue, 18 Feb 2025 20:34:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:Subject:Cc:To:From:Date:Message-ID:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: List-Owner; bh=fZojzVLy2tMfil75IXAXZ03S3XDVrP0r5Q56Ndp4Rwk=; b=UstUycZktekPAA w0Px/cBXkcwAy3jSrGLyPu2qSQ7e2KIvP11lRQKxA6Qq9iTo342yPBy3PjeL0wveeRX4Ctpin2a+4 3y6cjIlUpUzBWmipiSUlCW7qjkJrydTNFbUKNbOTojIZK5xZz83PibNq3BpqOCvsShHp986qsGR9Z ysqLI/1fUNcpMuKvJmBiECrrwJqQEds3OdE+5QN+BE5/DmPGozwo+uE3ZTVfEv+OK26Tj0kHuMCo7 i/ScUUlz1BxgmH1Fm3NAzvB/HVMPqqhUD8rYgaWFGjCensBhTtVzdYPOTgfNDQjKVjnw5KFi7KU2e T5mE4c07FaEsbnux6Xew==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tkUIF-00000009n9J-2Nr8; Tue, 18 Feb 2025 20:34:11 +0000 Received: from nyc.source.kernel.org ([2604:1380:45d1:ec00::3]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tkTlC-00000009fnG-1J27 for linux-arm-kernel@lists.infradead.org; Tue, 18 Feb 2025 20:00:06 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id 2461FA41AFD; Tue, 18 Feb 2025 19:58:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 7FB78C4CEF5; Tue, 18 Feb 2025 20:00:00 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.98) (envelope-from ) id 1tkTlX-00000004ArJ-0q9Y; Tue, 18 Feb 2025 15:00:23 -0500 Message-ID: <20250218200023.055162048@goodmis.org> User-Agent: quilt/0.68 Date: Tue, 18 Feb 2025 14:59:23 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, bpf , linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Peter Zijlstra , Linus Torvalds , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Zheng Yejian , Martin Kelly , Christophe Leroy , Josh Poimboeuf , Heiko Carstens , Catalin Marinas , Will Deacon , Vasily Gorbik , Alexander Gordeev Subject: [PATCH v5 5/6] ftrace: Update the mcount_loc check of skipped entries References: <20250218195918.255228630@goodmis.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250218_120002_427395_31130063 X-CRM114-Status: GOOD ( 14.17 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Steven Rostedt Now that weak functions turn into skipped entries, update the check to make sure the amount that was allocated would fit both the entries that were allocated as well as those that were skipped. Signed-off-by: Steven Rostedt (Google) --- Changes since v4: https://lore.kernel.org/20250217153453.792481985@goodmis.org - Initialized variable "remaining" to zero kernel/trace/ftrace.c | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index e3f89924f603..e657013424aa 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -7111,7 +7111,28 @@ static int ftrace_process_locs(struct module *mod, /* We should have used all pages unless we skipped some */ if (pg_unuse) { - WARN_ON(!skipped); + unsigned long pg_remaining, remaining = 0; + unsigned long skip; + + /* Count the number of entries unused and compare it to skipped. */ + pg_remaining = (ENTRIES_PER_PAGE << pg->order) - pg->index; + + if (!WARN(skipped < pg_remaining, "Extra allocated pages for ftrace")) { + + skip = skipped - pg_remaining; + + for (pg = pg_unuse; pg; pg = pg->next) + remaining += 1 << pg->order; + + skip = DIV_ROUND_UP(skip, ENTRIES_PER_PAGE); + + /* + * Check to see if the number of pages remaining would + * just fit the number of entries skipped. + */ + WARN(skip != remaining, "Extra allocated pages for ftrace: %lu with %lu skipped", + remaining, skipped); + } /* Need to synchronize with ftrace_location_range() */ synchronize_rcu(); ftrace_free_pages(pg_unuse); From patchwork Tue Feb 18 19:59:24 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Steven Rostedt X-Patchwork-Id: 13980831 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BB535C021AA for ; Tue, 18 Feb 2025 20:32:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:Subject:Cc:To:From:Date:Message-ID:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To: List-Owner; bh=dor0GCQpDJiIiZLkdo5/Mz1qqle8mYzzA6GzARE0hz8=; b=g7XnyTUlBAWy1l 6Ls7VwitRbYdqTcz4ro9wztPG5Y1B25OFoD+lNIKiOQ4UQGZ0uWYJH+8BqOIU9C3dhm+4OscjSjEF qeWivFfVA6V/oEPhSDDSCPTtsYAOn73I0qdi/sYx+JK2DN4BfLW6pU9VNYZCNSnCfsHUGzsBxwQ+z V6h5vAVkWliiJ8eoqNhOsu5JrKBzw0VStOoY/SGrR0xzdPnMih6HYRYnU8sCygg/Pk7+1rhtHz3nS 40jAXVnRm8+zjJQo/SKT7dBPsqvKyVPuQGU67AgG09iqVxVo9pe2F+XUu8Dhsmxk9rTmALTQpIVRC Y3exQi7+jQg+wbk+0kKg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tkUGn-00000009mpo-3bVf; Tue, 18 Feb 2025 20:32:41 +0000 Received: from dfw.source.kernel.org ([139.178.84.217]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tkTlC-00000009fnF-0ewP for linux-arm-kernel@lists.infradead.org; Tue, 18 Feb 2025 20:00:05 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id 13E0B5C6273; Tue, 18 Feb 2025 19:59:22 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ABA4DC4CEEA; Tue, 18 Feb 2025 20:00:00 +0000 (UTC) Received: from rostedt by gandalf with local (Exim 4.98) (envelope-from ) id 1tkTlX-00000004Arp-1YRM; Tue, 18 Feb 2025 15:00:23 -0500 Message-ID: <20250218200023.221100846@goodmis.org> User-Agent: quilt/0.68 Date: Tue, 18 Feb 2025 14:59:24 -0500 From: Steven Rostedt To: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-kbuild@vger.kernel.org, bpf , linux-arm-kernel@lists.infradead.org, linux-s390@vger.kernel.org Cc: Masami Hiramatsu , Mark Rutland , Mathieu Desnoyers , Andrew Morton , Peter Zijlstra , Linus Torvalds , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Zheng Yejian , Martin Kelly , Christophe Leroy , Josh Poimboeuf , Heiko Carstens , Catalin Marinas , Will Deacon , Vasily Gorbik , Alexander Gordeev Subject: [PATCH v5 6/6] ftrace: Have ftrace pages output reflect freed pages References: <20250218195918.255228630@goodmis.org> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20250218_120002_287341_ADC757D5 X-CRM114-Status: GOOD ( 16.86 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org From: Steven Rostedt The amount of memory that ftrace uses to save the descriptors to manage the functions it can trace is shown at output. But if there are a lot of functions that are skipped because they were weak or the architecture added holes into the tables, then the extra pages that were allocated are freed. But these freed pages are not reflected in the numbers shown, and they can even be inconsistent with what is reported: ftrace: allocating 57482 entries in 225 pages ftrace: allocated 224 pages with 3 groups The above shows the number of original entries that are in the mcount_loc section and the pages needed to save them (225), but the second output reflects the number of pages that were actually used. The two should be consistent as: ftrace: allocating 56739 entries in 224 pages ftrace: allocated 224 pages with 3 groups The above also shows the accurate number of entires that were actually stored and does not include the entries that were removed. Signed-off-by: Steven Rostedt (Google) --- kernel/trace/ftrace.c | 15 ++++++++++++--- 1 file changed, 12 insertions(+), 3 deletions(-) diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index e657013424aa..27c8def2139d 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -7006,6 +7006,7 @@ static int ftrace_process_locs(struct module *mod, unsigned long addr; unsigned long kaslr; unsigned long flags = 0; /* Shut up gcc */ + unsigned long pages; int ret = -ENOMEM; count = end - start; @@ -7013,6 +7014,8 @@ static int ftrace_process_locs(struct module *mod, if (!count) return 0; + pages = DIV_ROUND_UP(count, ENTRIES_PER_PAGE); + /* * Sorting mcount in vmlinux at build time depend on * CONFIG_BUILDTIME_MCOUNT_SORT, while mcount loc in @@ -7124,6 +7127,8 @@ static int ftrace_process_locs(struct module *mod, for (pg = pg_unuse; pg; pg = pg->next) remaining += 1 << pg->order; + pages -= remaining; + skip = DIV_ROUND_UP(skip, ENTRIES_PER_PAGE); /* @@ -7137,6 +7142,13 @@ static int ftrace_process_locs(struct module *mod, synchronize_rcu(); ftrace_free_pages(pg_unuse); } + + if (!mod) { + count -= skipped; + pr_info("ftrace: allocating %ld entries in %ld pages\n", + count, pages); + } + return ret; } @@ -7782,9 +7794,6 @@ void __init ftrace_init(void) goto failed; } - pr_info("ftrace: allocating %ld entries in %ld pages\n", - count, DIV_ROUND_UP(count, ENTRIES_PER_PAGE)); - ret = ftrace_process_locs(NULL, __start_mcount_loc, __stop_mcount_loc);