From patchwork Thu Apr 11 16:05:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 13626226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 61929C4345F for ; Thu, 11 Apr 2024 16:05:48 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 01BC96B00B3; Thu, 11 Apr 2024 12:05:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id EC03C6B00B5; Thu, 11 Apr 2024 12:05:47 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D12BB6B00B6; Thu, 11 Apr 2024 12:05:47 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id ACCA36B00B3 for ; Thu, 11 Apr 2024 12:05:47 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 6F198C0C2B for ; Thu, 11 Apr 2024 16:05:47 +0000 (UTC) X-FDA: 81997726734.26.8A0A4C8 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf27.hostedemail.com (Postfix) with ESMTP id AECC54001D for ; Thu, 11 Apr 2024 16:05:45 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=nUquO4a1; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf27.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712851545; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=FOv8sFpN1l1aG5Cz5ttusdbxwZKN2VFsPm1+5tEpgoI=; b=CrQsulfIBIBPcPUPqhDS4k6NfZd62plkHQJv4jdRZ0Q41bvhu0zehBUx1m+F9wr3+AY2lS THAGXO+Wlt5YFHDn4KRUR/V6/WzmsvnoityBvlrVsW2FZjBUNszOix62Ccq62m2xH6GUUo IeJzXRT7d9M5wsVTTRWNKxf3QUb7SjU= ARC-Authentication-Results: i=1; imf27.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=nUquO4a1; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf27.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712851545; a=rsa-sha256; cv=none; b=lQhyinudW7MxGnNc7iXpNwxow03eiDcWWkFy0YGDRUjzixqKsSto212spzv8V9eCbf3qMP HwmZ4Fyk8RZl5Bl9FlxtB8alHBhb2mipt1GqoYPEZbO7JDIQaa7Sts1RtmKih90rZsP0Aj hKPyTQSj4Zwahpkf7CRhRmZLWeqwpAo= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id E47776206D; Thu, 11 Apr 2024 16:05:44 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E6739C113CE; Thu, 11 Apr 2024 16:05:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1712851544; bh=B2UPTwvZ8YRZSWkmTYRU/Sp3RonIqh19BHZ95160r7o=; h=From:To:Cc:Subject:Date:From; b=nUquO4a1/6ZTCNZbRLuv4NNQcqijBDkbIKlsZevPGl461/ZAm/d/negh5wrvgBCEY R1teI17GnUQ8WHezjocLnqxNY/YMCaQzL6Atj23S2xvmsKGYIJpQN2PbQLQvmn3ZpL fXgFWWrrB1dxwNQnNpbdq+kT/DxZA0kF5nLY58SYRgrOrT5+pwun09BfqFJZQ4nrSU lBV8GXmV0rOpfqCddCsAahKorRytLs4uqJ0ZtXVAw0IqI62FLkfq3S8CI4TwHztRUU JnFs5quLmmnkCSs0ER4U9IQFddPY6ug8EGp9CfxVibDAZWQYbaUOr4gCPUVMw+WuAe c6XnkSextJR5g== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , Andy Lutomirski , Arnd Bergmann , Catalin Marinas , Christoph Hellwig , Helge Deller , Lorenzo Stoakes , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Mathieu Desnoyers , Michael Ellerman , Mike Rapoport , Palmer Dabbelt , Peter Zijlstra , Russell King , Song Liu , Steven Rostedt , Thomas Gleixner , Uladzislau Rezki , Will Deacon , bpf@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, x86@kernel.org Subject: [RFC PATCH 0/7] x86/module: use large ROX pages for text allocations Date: Thu, 11 Apr 2024 19:05:19 +0300 Message-ID: <20240411160526.2093408-1-rppt@kernel.org> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Rspamd-Queue-Id: AECC54001D X-Rspam-User: X-Rspamd-Server: rspam02 X-Stat-Signature: idbirjkjoyt7njm1m6mt18mojas4e9qf X-HE-Tag: 1712851545-35913 X-HE-Meta: U2FsdGVkX19HKVlk7C0ZTO7qWtkjbyG1qKhnHXx7rxijdHxzKr/dbbf7QjtOBh7DyCvGbqJsVDxm7EDMgQNWyjHR3uNWdDLurbFvuvTZ2cTfDSNlxr9KA8YujTshVhhUg0Y2rEjgipxSWQqWR1dcCtVxzk6K2e9lQxTUw7NjPkL1HMN7dADwZ8spiX0o+llrL3O01n8wTSAFw5uYOAv4Ec1BLf4xvqFhGZlpAFm9NBLR4CZ9sNRI2W+pww+eWbnjePYMwBbXiNNRImFv9GsWPex3M3UwlR32Gt/seJ6LBjVqSoveQASIxzJDufFYry6BesnEri4Kz/pmzYTh8r8nEmw7rFdpkVsb5z1jpaPHKh0p3oU3VwAa3iXsyylS1lPcr0b2AE6eGcfkgdSZrb2pJ5ge6UM9+SNDBW6UfCdSe0XCAJTU2aCXZQrs6IgGm5QIFpBrn/fGzwG7F5gsOlPaZXf7TABc5Ji47YbQdY3qc4SDRaiIaCfkXUMN6iQAUxOzlrZGnXTKcMwfWuI1vZDeFtEuymFRZYaNS98ItlKh625/P9wCZKoW7KylWVhEfR171Ej6lsWRMEjVj9uRN+dgzseC6ZTzsog4VjOq61XV2V26m9GIzVzM3fbRRlqVVrdbzFO4owgZxEKHsoEkbbBVUFzX9sZYG/1kNr9gU09g1GujzZU/UIdFlvc2xAu8fNr3k6Nyhz0TZlLuzuJ81b8cOP90p+ZwdUIY1QjvA2Lm9mCmLAkWd6RtyPPdjD61q1ZGWf+ScXXt5YDs72z5Jc+dSNxr8ikazXnJxa7t9K4Tc9v1/vHUF5XcTwzFOPrEOx4M/mF7THQLWBM5UwvB545DIM4Ls4PEcuS42h/ZYjzKc0jOws9wqcbzAGEsVOYamPsG2Sn+ynDukE+taCF3A93JfeLkwUrW0LC9QA0S7K0W3jUKfP6ODQthkMSgyL88yXKWD24kpwIuHCSGZoBDQH7 AcL0g75B k/0ATIwu6Dt9JGrDTNXeQ62SXD3awwgcKe8meXDH8fkPwZHVmtsKSRqCUkR+jM9KwLEZ9mCAkpOwyw/kaDHVPwoZeqxcyJ1sqZGxCQCo1TZKOmeXiHVU3TwCmtPD9as64D3eX6m4RgtmJwn4k1e27hxzRACJht6T1fMjMF67JyFdPqYW+h/2tvkroLRkMelStg0vaxH01ITQ/97o/WEYWpOYM1Wi7Mihv+s4960AmFKQihP6ClCJsPXL/q8j+l/jjaL2J0j0VOeMTvQowL5gLrcFf2UNgmbl7Umhurnt8nQSvuuU= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (IBM)" Hi, These patches add support for using large ROX pages for allocations of executable memory on x86. They address Andy's comments [1] about having executable mappings for code that was not completely formed. The approach taken is to allocate ROX memory along with writable but not executable memory and use the writable copy to perform relocations and alternatives patching. After the module text gets into its final shape, the contents of the writable memory is copied into the actual ROX location using text poking. The allocations of the ROX memory use vmalloc(VMAP_ALLOW_HUGE_MAP) to allocate PMD aligned memory, fill that memory with invalid instructions and in the end remap it as ROX. Portions of these large pages are handed out to execmem_alloc() callers without any changes to the permissions. When the memory is freed with execmem_free() it is invalidated again so that it won't contain stale instructions. The module memory allocation, x86 code dealing with relocations and alternatives patching takes into account the existence of the two copies, the writable memory and the ROX memory at the actual allocated virtual address. This is an early RFC, it is not well tested and there is a lot of room for improvement. For example, the locking of execmem_cache can be made more fine grained, freeing of PMD-sized pages from execmem_cache can be implemented with a shrinker, the large pages can be removed from the direct map when they are added to the cache and restored there when they are free from the cache. Still, I'd like to hear feedback on the approach in general before moving forward with polishing the details. The series applies on top of v4 of "jit/text allocator" [2] and also available at git: https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=execmem/v4%2bx86-rox [1] https://lore.kernel.org/all/a17c65c6-863f-4026-9c6f-a04b659e9ab4@app.fastmail.com [2] https://lore.kernel.org/linux-mm/20240411160051.2093261-1-rppt@kernel.org Mike Rapoport (IBM) (6): asm-generic: introduce text-patching.h mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations module: prepare to handle ROX allocations for text x86/module: perpare module loading for ROX allocations of text execmem: add support for cache of large ROX pages x86/module: enable ROX caches for module text Song Liu (1): ftrace: Add swap_func to ftrace_process_locs() arch/alpha/include/asm/Kbuild | 1 + arch/arc/include/asm/Kbuild | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/arm/kernel/ftrace.c | 2 +- arch/arm/kernel/jump_label.c | 2 +- arch/arm/kernel/kgdb.c | 2 +- arch/arm/kernel/patch.c | 2 +- arch/arm/probes/kprobes/core.c | 2 +- arch/arm/probes/kprobes/opt-arm.c | 2 +- .../asm/{patching.h => text-patching.h} | 0 arch/arm64/kernel/ftrace.c | 2 +- arch/arm64/kernel/jump_label.c | 2 +- arch/arm64/kernel/kgdb.c | 2 +- arch/arm64/kernel/patching.c | 2 +- arch/arm64/kernel/probes/kprobes.c | 2 +- arch/arm64/kernel/traps.c | 2 +- arch/arm64/net/bpf_jit_comp.c | 2 +- arch/csky/include/asm/Kbuild | 1 + arch/hexagon/include/asm/Kbuild | 1 + arch/loongarch/include/asm/Kbuild | 1 + arch/m68k/include/asm/Kbuild | 1 + arch/microblaze/include/asm/Kbuild | 1 + arch/mips/include/asm/Kbuild | 1 + arch/nios2/include/asm/Kbuild | 1 + arch/openrisc/include/asm/Kbuild | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/parisc/kernel/ftrace.c | 2 +- arch/parisc/kernel/jump_label.c | 2 +- arch/parisc/kernel/kgdb.c | 2 +- arch/parisc/kernel/kprobes.c | 2 +- arch/parisc/kernel/patch.c | 2 +- arch/powerpc/include/asm/kprobes.h | 2 +- .../asm/{code-patching.h => text-patching.h} | 0 arch/powerpc/kernel/crash_dump.c | 2 +- arch/powerpc/kernel/epapr_paravirt.c | 2 +- arch/powerpc/kernel/jump_label.c | 2 +- arch/powerpc/kernel/kgdb.c | 2 +- arch/powerpc/kernel/kprobes.c | 2 +- arch/powerpc/kernel/module_32.c | 2 +- arch/powerpc/kernel/module_64.c | 2 +- arch/powerpc/kernel/optprobes.c | 2 +- arch/powerpc/kernel/process.c | 2 +- arch/powerpc/kernel/security.c | 2 +- arch/powerpc/kernel/setup_32.c | 2 +- arch/powerpc/kernel/setup_64.c | 2 +- arch/powerpc/kernel/static_call.c | 2 +- arch/powerpc/kernel/trace/ftrace.c | 2 +- arch/powerpc/kernel/trace/ftrace_64_pg.c | 2 +- arch/powerpc/lib/code-patching.c | 2 +- arch/powerpc/lib/feature-fixups.c | 2 +- arch/powerpc/lib/test-code-patching.c | 2 +- arch/powerpc/lib/test_emulate_step.c | 2 +- arch/powerpc/mm/book3s32/mmu.c | 2 +- arch/powerpc/mm/book3s64/hash_utils.c | 2 +- arch/powerpc/mm/book3s64/slb.c | 2 +- arch/powerpc/mm/kasan/init_32.c | 2 +- arch/powerpc/mm/mem.c | 2 +- arch/powerpc/mm/nohash/44x.c | 2 +- arch/powerpc/mm/nohash/book3e_pgtable.c | 2 +- arch/powerpc/mm/nohash/tlb.c | 2 +- arch/powerpc/net/bpf_jit_comp.c | 2 +- arch/powerpc/perf/8xx-pmu.c | 2 +- arch/powerpc/perf/core-book3s.c | 2 +- arch/powerpc/platforms/85xx/smp.c | 2 +- arch/powerpc/platforms/86xx/mpc86xx_smp.c | 2 +- arch/powerpc/platforms/cell/smp.c | 2 +- arch/powerpc/platforms/powermac/smp.c | 2 +- arch/powerpc/platforms/powernv/idle.c | 2 +- arch/powerpc/platforms/powernv/smp.c | 2 +- arch/powerpc/platforms/pseries/smp.c | 2 +- arch/powerpc/xmon/xmon.c | 2 +- arch/riscv/errata/andes/errata.c | 2 +- arch/riscv/errata/sifive/errata.c | 2 +- arch/riscv/errata/thead/errata.c | 2 +- .../include/asm/{patch.h => text-patching.h} | 0 arch/riscv/include/asm/uprobes.h | 2 +- arch/riscv/kernel/alternative.c | 2 +- arch/riscv/kernel/cpufeature.c | 3 +- arch/riscv/kernel/ftrace.c | 2 +- arch/riscv/kernel/jump_label.c | 2 +- arch/riscv/kernel/patch.c | 2 +- arch/riscv/kernel/probes/kprobes.c | 2 +- arch/riscv/net/bpf_jit_comp64.c | 2 +- arch/riscv/net/bpf_jit_core.c | 2 +- arch/sh/include/asm/Kbuild | 1 + arch/sparc/include/asm/Kbuild | 1 + arch/um/kernel/um_arch.c | 16 +- arch/x86/entry/vdso/vma.c | 3 +- arch/x86/include/asm/alternative.h | 14 +- arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 152 ++++++---- arch/x86/kernel/ftrace.c | 41 ++- arch/x86/kernel/module.c | 17 +- arch/x86/mm/init.c | 29 +- arch/xtensa/include/asm/Kbuild | 1 + include/asm-generic/text-patching.h | 5 + include/linux/execmem.h | 25 ++ include/linux/ftrace.h | 2 + include/linux/module.h | 11 + include/linux/text-patching.h | 15 + kernel/module/main.c | 70 ++++- kernel/module/strict_rwx.c | 3 + kernel/trace/ftrace.c | 13 +- mm/execmem.c | 278 +++++++++++++++++- mm/vmalloc.c | 9 +- 105 files changed, 663 insertions(+), 193 deletions(-) rename arch/arm/include/asm/{patch.h => text-patching.h} (100%) rename arch/arm64/include/asm/{patching.h => text-patching.h} (100%) rename arch/parisc/include/asm/{patch.h => text-patching.h} (100%) rename arch/powerpc/include/asm/{code-patching.h => text-patching.h} (100%) rename arch/riscv/include/asm/{patch.h => text-patching.h} (100%) create mode 100644 include/asm-generic/text-patching.h create mode 100644 include/linux/text-patching.h