From patchwork Thu Apr 11 16:05:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 13626229 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 586D3C4345F for ; Thu, 11 Apr 2024 16:06:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id DFFFE6B00BA; Thu, 11 Apr 2024 12:06:15 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id D87AD6B00BB; Thu, 11 Apr 2024 12:06:15 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B64876B00BC; Thu, 11 Apr 2024 12:06:15 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 90B6B6B00BA for ; Thu, 11 Apr 2024 12:06:15 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 62DD6A076B for ; Thu, 11 Apr 2024 16:06:15 +0000 (UTC) X-FDA: 81997727910.03.D92B269 Received: from sin.source.kernel.org (sin.source.kernel.org [145.40.73.55]) by imf28.hostedemail.com (Postfix) with ESMTP id 2F2BFC000A for ; Thu, 11 Apr 2024 16:06:12 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="LnrlFS/9"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf28.hostedemail.com: domain of rppt@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712851573; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tvZ3pJir6PnrCk2h2lDJAjGXD7k5QQFXBKSzv2g0JZc=; b=7PciFbdC+S9RVN/z3Xa1nhQ0kuHhVS6S/0uDMOeaSdpAKtCJ4Wek+nMgGbQS5L5qHdIQ4Z GqsoeCYLubLEoL6ZDnNwn5ieKIJ8A957MrpdebOlQ8cCA/YXX21Tbf/O55mRDEc0x8BHjb rbtnVTWkCkqycSbosM8eSyz4tprQzoM= ARC-Authentication-Results: i=1; imf28.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="LnrlFS/9"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf28.hostedemail.com: domain of rppt@kernel.org designates 145.40.73.55 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712851573; a=rsa-sha256; cv=none; b=JsLX7TgjKu+5mpV0FBH3BuKgoqZpbTkSBnVH0ZyYxOtLup3yrJFma9kZiTLlt0YXtIWicZ cTRKjZrSSVKug0IjqRhafsSmaLIQss4p1rWLK63fGh9rE7vfKArhStqVnQkNRkH5/RG45k Ws2U00wmXPViKC+fm8Y6jGZyr3rky98= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by sin.source.kernel.org (Postfix) with ESMTP id 309BACE1773; Thu, 11 Apr 2024 16:06:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id C4568C113CD; Thu, 11 Apr 2024 16:06:01 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712851569; bh=+blYsU+yhHAK3ZWSR/4vIF4tppBrgYZyNx/i+3pCeM4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=LnrlFS/9iawvR+8bxHsr0M/9FPe5IQYw7I6C8K+CutHZsK/WxpkdBuz8ri/GGmuzl r+YEuwwyYjVOMT6kHDRhLZHoWkCWbfKD1DUY/DQfdhUA3AliFnvkr79QIUHmsvn+Xh 8nQ2Zf21W2ZTSIBApvBf5palrBojvjhjJN+vNV+WOBp209WK4NciwA1uCxRhQWtp/l dk8faa1HtHFiDzAnDJrLHODKg4hNxxZO15Z6AeQHYQzGk2/htmQdZY7Lw8whK5MsKr Hh7c30KTH5SVkSL94fhMmpH3MZ1m8TvZt5cNCyQpbBzxG9VGORMa3Cth0VaWEvMTUH uU+b9ZYvJFIwA== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Andy Lutomirski , Arnd Bergmann , Catalin Marinas , Christoph Hellwig , Helge Deller , Lorenzo Stoakes , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mathieu Desnoyers , Michael Ellerman , Mike Rapoport , Palmer Dabbelt , Peter Zijlstra , Russell King , Song Liu , Steven Rostedt , Thomas Gleixner , Uladzislau Rezki , Will Deacon , bpf@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [RFC PATCH 3/7] module: prepare to handle ROX allocations for text Date: Thu, 11 Apr 2024 19:05:22 +0300 Message-ID: <20240411160526.2093408-4-rppt@kernel.org> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240411160526.2093408-1-rppt@kernel.org> References: <20240411160526.2093408-1-rppt@kernel.org> MIME-Version: 1.0 X-Rspamd-Queue-Id: 2F2BFC000A X-Rspam-User: X-Rspamd-Server: rspam04 X-Stat-Signature: yp6gopd17y8z3mwrq791siqu4tuqgoa7 X-HE-Tag: 1712851572-897645 X-HE-Meta: U2FsdGVkX1/MeFBSrCM8oqghggJJD6PZl8H20GLstrLEH7i1Oj5fvzpLMxdnShiYzx8O9MtDXAjwbWBI4sLi/+9FGUZ3CX/Cy8s8NJHBCUz3m9GoyWqOxSCv/Lj96ImlTajEJ0SsixNHNZCswO+KrzAJINz3r/cD8dijUGOxJzAzbmjHnuaiEcXRj1uAONXfwtoI+AzIND22lF4AKdHE7/+wfnJxpPCIho8PFmYEygdBb6eFO8wCNvxR7NnTLI+Ly2fu6QqI/RKG63GQ+nhOEWQeLmdglN/4zWSmFYZ7MBhGffqBj1OSozHyE+VM+2ULqqXCPGqL5QOX0HuHnxUu5gicg7fIVEnAtohZ5ZptkYNTtKW1dAV51GNWmk1HPtGYa6bbiJFDVQV/8LlN4zRhiOHwym3G7gVhtRd1XGTdBB/mWHZtvLmcydXL2R3+USln1Q8zF0qNHicDIqd41MmN8NpLkOgKUNE2aMC39684UXW4jr/KxxqNt3M4tz3/GAsiT7e5pyeJi2uuusMaIB23rlgv2xluW4E5XLHt59+t3fqHj7/LWfWDxpppB3k5fVowxtSO5B5g9mxhGetCi8FJTV682HnVn5I9r4nsRTdO05pq6RGvPq1rrtJd+pJ9CGiZZ4KMTC0JFPn+AB0GPIQ/N+X4sqsKAoQ2nORqC9mWRXR8BX8HmwNsSi8//Ro9e01RgbRqltHJ9ocPJCj8AW4z3YRfzeCms+900VOAdgBQKxozbzNWMWBEipvAWoNTI+78tpAAxF3Fk2n7X4hRMpayCDkfQ//hjpDbWzRJfoKZetQ4wiXLvyH3yyZMiz0qXl6KrV72sa5L4BeUpcTQih8qEgJKQFFDnQWBqiHTZhKpML4BZMmi+jJtp5PpiMTZG9ChgH/bHgvfrMmV2ZiiNX2biyfg1aQZ6tAOS+qL3t13wPr1fZ9J238AZg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (IBM)" In order to support ROX allocations for module text, it is necessary to handle modifications to the code, such as relocations and alternatives patching, without write access to that memory. One option is to use text patching, but this would make module loading extremely slow and will expose executable code that is not finally formed. A better way is to have memory allocated with ROX permissions contain invalid instructions and keep a writable, but not executable copy of the module text. The relocations and alternative patches would be done on the writable copy using the addresses of the ROX memory. Once the module is completely ready, the updated text will be copied to ROX memory using text patching in one go and the writable copy will be freed. Add support for that to module initialization code and provide necessary interfaces in execmem. --- include/linux/execmem.h | 23 +++++++++++++ include/linux/module.h | 11 ++++++ kernel/module/main.c | 70 ++++++++++++++++++++++++++++++++++---- kernel/module/strict_rwx.c | 3 ++ mm/execmem.c | 11 ++++++ 5 files changed, 111 insertions(+), 7 deletions(-) diff --git a/include/linux/execmem.h b/include/linux/execmem.h index ffd0d12feef5..9d22999dbd7d 100644 --- a/include/linux/execmem.h +++ b/include/linux/execmem.h @@ -46,9 +46,11 @@ enum execmem_type { /** * enum execmem_range_flags - options for executable memory allocations * @EXECMEM_KASAN_SHADOW: allocate kasan shadow + * @EXECMEM_READ_ONLY: allocated memory should be mapped as read only */ enum execmem_range_flags { EXECMEM_KASAN_SHADOW = (1 << 0), + EXECMEM_ROX_CACHE = (1 << 1), }; /** @@ -123,6 +125,27 @@ void *execmem_alloc(enum execmem_type type, size_t size); */ void execmem_free(void *ptr); +/** + * execmem_update_copy - copy an update to executable memory + * @dst: destination address to update + * @src: source address containing the data + * @size: how many bytes of memory shold be copied + * + * Copy @size bytes from @src to @dst using text poking if the memory at + * @dst is read-only. + * + * Return: a pointer to @dst or NULL on error + */ +void *execmem_update_copy(void *dst, const void *src, size_t size); + +/** + * execmem_is_rox - check if execmem is read-only + * @type - the execmem type to check + * + * Return: %true if the @type is read-only, %false if it's writable + */ +bool execmem_is_rox(enum execmem_type type); + #ifdef CONFIG_ARCH_WANTS_EXECMEM_EARLY void execmem_early_init(void); #else diff --git a/include/linux/module.h b/include/linux/module.h index 1153b0d99a80..3df3202680a2 100644 --- a/include/linux/module.h +++ b/include/linux/module.h @@ -361,6 +361,8 @@ enum mod_mem_type { struct module_memory { void *base; + void *rw_copy; + bool is_rox; unsigned int size; #ifdef CONFIG_MODULES_TREE_LOOKUP @@ -368,6 +370,15 @@ struct module_memory { #endif }; +#ifdef CONFIG_MODULES +unsigned long module_writable_offset(struct module *mod, void *loc); +#else +static inline unsigned long module_writable_offset(struct module *mod, void *loc) +{ + return 0; +} +#endif + #ifdef CONFIG_MODULES_TREE_LOOKUP /* Only touch one cacheline for common rbtree-for-core-layout case. */ #define __module_memory_align ____cacheline_aligned diff --git a/kernel/module/main.c b/kernel/module/main.c index 91e185607d4b..f83fbb9c95ee 100644 --- a/kernel/module/main.c +++ b/kernel/module/main.c @@ -1188,6 +1188,21 @@ void __weak module_arch_freeing_init(struct module *mod) { } +unsigned long module_writable_offset(struct module *mod, void *loc) +{ + if (!mod) + return 0; + + for_class_mod_mem_type(type, text) { + struct module_memory *mem = &mod->mem[type]; + + if (loc >= mem->base && loc < mem->base + mem->size) + return (unsigned long)(mem->rw_copy - mem->base); + } + + return 0; +} + static int module_memory_alloc(struct module *mod, enum mod_mem_type type) { unsigned int size = PAGE_ALIGN(mod->mem[type].size); @@ -1205,6 +1220,23 @@ static int module_memory_alloc(struct module *mod, enum mod_mem_type type) if (!ptr) return -ENOMEM; + mod->mem[type].base = ptr; + + if (execmem_is_rox(execmem_type)) { + ptr = vzalloc(size); + + if (!ptr) { + execmem_free(mod->mem[type].base); + return -ENOMEM; + } + + mod->mem[type].rw_copy = ptr; + mod->mem[type].is_rox = true; + } else { + mod->mem[type].rw_copy = mod->mem[type].base; + memset(mod->mem[type].base, 0, size); + } + /* * The pointer to these blocks of memory are stored on the module * structure and we keep that around so long as the module is @@ -1218,15 +1250,16 @@ static int module_memory_alloc(struct module *mod, enum mod_mem_type type) */ kmemleak_not_leak(ptr); - memset(ptr, 0, size); - mod->mem[type].base = ptr; - return 0; } static void module_memory_free(struct module *mod, enum mod_mem_type type) { - void *ptr = mod->mem[type].base; + struct module_memory *mem = &mod->mem[type]; + void *ptr = mem->base; + + if (mem->is_rox) + vfree(mem->rw_copy); execmem_free(ptr); } @@ -2237,6 +2270,7 @@ static int move_module(struct module *mod, struct load_info *info) for_each_mod_mem_type(type) { if (!mod->mem[type].size) { mod->mem[type].base = NULL; + mod->mem[type].rw_copy = NULL; continue; } @@ -2253,11 +2287,14 @@ static int move_module(struct module *mod, struct load_info *info) void *dest; Elf_Shdr *shdr = &info->sechdrs[i]; enum mod_mem_type type = shdr->sh_entsize >> SH_ENTSIZE_TYPE_SHIFT; + unsigned long offset = shdr->sh_entsize & SH_ENTSIZE_OFFSET_MASK; + unsigned long addr; if (!(shdr->sh_flags & SHF_ALLOC)) continue; - dest = mod->mem[type].base + (shdr->sh_entsize & SH_ENTSIZE_OFFSET_MASK); + addr = (unsigned long)mod->mem[type].base + offset; + dest = mod->mem[type].rw_copy + offset; if (shdr->sh_type != SHT_NOBITS) { /* @@ -2279,7 +2316,7 @@ static int move_module(struct module *mod, struct load_info *info) * users of info can keep taking advantage and using the newly * minted official memory area. */ - shdr->sh_addr = (unsigned long)dest; + shdr->sh_addr = addr; pr_debug("\t0x%lx 0x%.8lx %s\n", (long)shdr->sh_addr, (long)shdr->sh_size, info->secstrings + shdr->sh_name); } @@ -2429,6 +2466,8 @@ int __weak module_finalize(const Elf_Ehdr *hdr, static int post_relocation(struct module *mod, const struct load_info *info) { + int ret; + /* Sort exception table now relocations are done. */ sort_extable(mod->extable, mod->extable + mod->num_exentries); @@ -2440,7 +2479,24 @@ static int post_relocation(struct module *mod, const struct load_info *info) add_kallsyms(mod, info); /* Arch-specific module finalizing. */ - return module_finalize(info->hdr, info->sechdrs, mod); + ret = module_finalize(info->hdr, info->sechdrs, mod); + if (ret) + return ret; + + for_each_mod_mem_type(type) { + struct module_memory *mem = &mod->mem[type]; + + if (mem->is_rox) { + if (!execmem_update_copy(mem->base, mem->rw_copy, + mem->size)) + return -ENOMEM; + + vfree(mem->rw_copy); + mem->rw_copy = NULL; + } + } + + return 0; } /* Call module constructors. */ diff --git a/kernel/module/strict_rwx.c b/kernel/module/strict_rwx.c index c45caa4690e5..239e5013359d 100644 --- a/kernel/module/strict_rwx.c +++ b/kernel/module/strict_rwx.c @@ -34,6 +34,9 @@ int module_enable_text_rox(const struct module *mod) for_class_mod_mem_type(type, text) { int ret; + if (mod->mem[type].is_rox) + continue; + if (IS_ENABLED(CONFIG_STRICT_MODULE_RWX)) ret = module_set_memory(mod, type, set_memory_rox); else diff --git a/mm/execmem.c b/mm/execmem.c index aabc0afabdbc..c920d2b5a721 100644 --- a/mm/execmem.c +++ b/mm/execmem.c @@ -4,6 +4,7 @@ #include #include #include +#include static struct execmem_info *execmem_info __ro_after_init; static struct execmem_info default_execmem_info __ro_after_init; @@ -63,6 +64,16 @@ void execmem_free(void *ptr) vfree(ptr); } +void *execmem_update_copy(void *dst, const void *src, size_t size) +{ + return text_poke_copy(dst, src, size); +} + +bool execmem_is_rox(enum execmem_type type) +{ + return !!(execmem_info->ranges[type].flags & EXECMEM_ROX_CACHE); +} + static bool execmem_validate(struct execmem_info *info) { struct execmem_range *r = &info->ranges[EXECMEM_DEFAULT];