From patchwork Mon Sep 9 06:47:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 13795957 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 66DC9CE7AB0 for ; Mon, 9 Sep 2024 06:48:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 79B6D6B0113; Mon, 9 Sep 2024 02:47:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 724E96B0114; Mon, 9 Sep 2024 02:47:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 576AC6B0115; Mon, 9 Sep 2024 02:47:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 36CE06B0113 for ; Mon, 9 Sep 2024 02:47:59 -0400 (EDT) Received: from smtpin23.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id AD533140D6D for ; Mon, 9 Sep 2024 06:47:58 +0000 (UTC) X-FDA: 82544269836.23.57F14F5 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by imf18.hostedemail.com (Postfix) with ESMTP id 17E1B1C0010 for ; Mon, 9 Sep 2024 06:47:56 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GZTJc1yo; spf=pass (imf18.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1725864450; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=fj/Brd6CM2lVYtEeZ0hLWl99bsqqAwE5Arq7Euy3Ec4=; b=Orhu27XY0l0n6uZRfmk2YMuG8TpzwAGuoyfOsFeVRT+gsREXgxw/Ta/Tg6y7S2tSFG7gCa Hb+GH/NhfT4C0G+Z6kqEXdzX9PhGZp5Ux5DryxBSNVco7epiCRvBvQs86OHPDWeCSSGCTu DvVCTTXsBWXUmAg/u7RN5+6pWT9nPmY= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b=GZTJc1yo; spf=pass (imf18.hostedemail.com: domain of rppt@kernel.org designates 139.178.84.217 as permitted sender) smtp.mailfrom=rppt@kernel.org; dmarc=pass (policy=quarantine) header.from=kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1725864450; a=rsa-sha256; cv=none; b=MpJ4wM8/m41Sm5qeHUBTDZS7q5LfuA8vgk6Iv3LLacDVxv78Ok4vCVxTbRUqgyC0TcjCP5 q8VgWvXEL5TQhpDYiyvOrM8mQcjId3Qj9QWrUvXvc5dZ4LsY5BTab69/yuF1+Wx50Yy5hs Q+qtm/rPRmr6uJt1C2dSn77NHUBqKYU= Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by dfw.source.kernel.org (Postfix) with ESMTP id D49E05C483E; Mon, 9 Sep 2024 06:47:51 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 5F127C4CEC5; Mon, 9 Sep 2024 06:47:40 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1725864475; bh=B9x+q6EZd7K9Ar6u2d/l/9Mk8JmglYJFdX3xwumRUgY=; h=From:To:Cc:Subject:Date:From; b=GZTJc1yoHgAC1vrKmkrTik6bpeO/Ba3QeazSMwPEzdJE6v6r2E7ixgB1ws7HJXN1F HRwEzExXYATn3fg6VyMu83WZRGIqkCuTqIpvYBgI7GMvZ2fNpZanOxVPd3ZxuoqB0K kwTmCftX0HBa5qu7J42H4mXCg8BICIgqgeYhEDOMsEPSqM10sT99D9eEaHA26BQrpc fWMQM7MbpHVOJx1YrHAuv2j7+cuQdZMv1YBYBeSYPLwSZAqusrJbcD9XnNmcUVYqMp 6acnhTwIIPKoUx5UMRy32HTx9KDEvPL+OqTadSOplha4y8PQUwonrxBocNmxw0Twq/ 7gmEoI8OZn75Q== From: Mike Rapoport To: Andrew Morton Cc: Andreas Larsson , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Brian Cain , Catalin Marinas , Christoph Hellwig , Christophe Leroy , Dave Hansen , Dinh Nguyen , Geert Uytterhoeven , Guo Ren , Helge Deller , Huacai Chen , Ingo Molnar , Johannes Berg , John Paul Adrian Glaubitz , Kent Overstreet , "Liam R. Howlett" , Luis Chamberlain , Mark Rutland , Masami Hiramatsu , Matt Turner , Max Filippov , Michael Ellerman , Michal Simek , Mike Rapoport , Oleg Nesterov , Palmer Dabbelt , Peter Zijlstra , Richard Weinberger , Russell King , Song Liu , Stafford Horne , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Uladzislau Rezki , Vineet Gupta , Will Deacon , bpf@vger.kernel.org, linux-alpha@vger.kernel.org, linux-arch@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-csky@vger.kernel.org, linux-hexagon@vger.kernel.org, linux-kernel@vger.kernel.org, linux-m68k@lists.linux-m68k.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-openrisc@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-sh@vger.kernel.org, linux-snps-arc@lists.infradead.org, linux-trace-kernel@vger.kernel.org, linux-um@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 0/8] x86/module: use large ROX pages for text allocations Date: Mon, 9 Sep 2024 09:47:22 +0300 Message-ID: <20240909064730.3290724-1-rppt@kernel.org> X-Mailer: git-send-email 2.43.0 MIME-Version: 1.0 X-Rspam-User: X-Stat-Signature: y5k68ejnt84emyyyruxpf1yoqqus69f3 X-Rspamd-Queue-Id: 17E1B1C0010 X-Rspamd-Server: rspam11 X-HE-Tag: 1725864476-409850 X-HE-Meta: U2FsdGVkX1/quBkI6RgbztfzJ6N9XlYYZrJQusGuzH0/BIW7fIGK8q0jHx3+6x0tUnTxWBdC8wy3DuCc4ey6yGr7PCWCUZpPNJwdqPdO1IRl2s4kBqyN8zxIgRyxYE+f3ZYZrboSGVK51CtEFcWWdraaUEnWgIYK1iBju86KYDh/dhF1gPvCEQDRAqPs8oHwANfPukUgrIH53sJ8YBRKJOz+G2DLGxm6pD2hcETomsq41XbydeHzxcbcN6k8+3lyiGus2iMETi1UKEWFqNtG9QIhW6xl6dz7dFvazlPNazwid8j7S1MIhTms4zqPRb/nFRPERMeXKZTVF4yg7SfK2lQOY6ojShYwqixNYnfg/UKN47+eSYlSRwTJG6Mz8tud8gavHxvgPBYEwpreFNVonyzlox0krfw8KnkedonK1H0YzuDkHRNDvLBmH1Jr0HIkj3BqWss0mtYGNz790lhJFLKqeM70/IWYTzHR/WFCcId/0YHEXko5EOrQeH9EouOgDBgmw6DcGIfCQeoc86g72Q6HE+o9wmJ9LJ6jEuA/5IY1WRN+FCPElW1e/DWjKlK8pw5Id48Gp+jc2ilw+9149DSycgfNDfw36PxrOtImx55HMSiII/Mn3Yg+Ebh/YISOnXa5eVU3WuOL8aFax9ZSCX8AYY2VF5tRL+3asgjDp4REAmtHHb7YdYoWGmH8J0NQucxdgz+exSboc1STrg++x28Cv49fXaC0rW164vhpp9+/onic5rVaUfT+v/CQ8x0AEkdMnPRNIoRli3LZNZ86c88QDoEP7k5Ri28yp8q4Y23LNn3BgXYVbnUPWsGRyprYKzejnyRMaqOgmUuGi/bvIgKTozRT043RggOarMa0yD2yiFSJkQa43RnQFUCJ2qeC9U49NKLXlrSHBJtcEMeT5ifYs1Ls/53HeXRt9BFFQ1Rb9wFoMdqH2LZ1cOa/Gmhlhse7WMVg2CLQ07Xjhh/ f9EDCJax /nqygkbQgCrivW2TUtyaI2jrDuaCUCHGxy3ANl2fTK59YytT8kXMxjJTKXi6VF9fqizHs5oeVGK0RcPcg5ISzfy21sDVfd+39dEsyPieSEylQ4N+i6dZHeFTFU50a1kvED3Io8kA801xTBFZ1qIVL1Ow9OUPAISphCCdu4SilH8OBEG9Fr9fKJkQf86rlpDRPcAzbXPLrb+bS0HhHm3Y+TNP74cXiPrF3FPbpdXPeJwb8kXGBZZVxYKh3jXGqYtWanC+5D1V33RnIRFeA9i24YTDH490JwON1L3o1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: "Mike Rapoport (Microsoft)" Hi, These patches add support for using large ROX pages for allocations of executable memory on x86. They address Andy's comments [1] about having executable mappings for code that was not completely formed. The approach taken is to allocate ROX memory along with writable but not executable memory and use the writable copy to perform relocations and alternatives patching. After the module text gets into its final shape, the contents of the writable memory is copied into the actual ROX location using text poking. The allocations of the ROX memory use vmalloc(VMAP_ALLOW_HUGE_MAP) to allocate PMD aligned memory, fill that memory with invalid instructions and in the end remap it as ROX. Portions of these large pages are handed out to execmem_alloc() callers without any changes to the permissions. When the memory is freed with execmem_free() it is invalidated again so that it won't contain stale instructions. The module memory allocation, x86 code dealing with relocations and alternatives patching take into account the existence of the two copies, the writable memory and the ROX memory at the actual allocated virtual address. The patches are available at git: https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=execmem/x86-rox/v2 [1] https://lore.kernel.org/all/a17c65c6-863f-4026-9c6f-a04b659e9ab4@app.fastmail.com v2: https://lore.kernel.org/all/20240826065532.2618273-1-rppt@kernel.org * add comment why ftrace_swap_func() is needed (Steve) Since RFC: https://lore.kernel.org/all/20240411160526.2093408-1-rppt@kernel.org * update changelog about HUGE_VMAP allocations (Christophe) * move module_writable_address() from x86 to modules core (Ingo) * rename execmem_invalidate() to execmem_fill_trapping_insns() (Peter) * call alternatives_smp_unlock() after module text in-place is up to date (Nadav) Mike Rapoport (Microsoft) (7): mm: vmalloc: group declarations depending on CONFIG_MMU together mm: vmalloc: don't account for number of nodes for HUGE_VMAP allocations asm-generic: introduce text-patching.h module: prepare to handle ROX allocations for text x86/module: perpare module loading for ROX allocations of text execmem: add support for cache of large ROX pages x86/module: enable ROX caches for module text Song Liu (1): ftrace: Add swap_func to ftrace_process_locs() arch/alpha/include/asm/Kbuild | 1 + arch/arc/include/asm/Kbuild | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/arm/kernel/ftrace.c | 2 +- arch/arm/kernel/jump_label.c | 2 +- arch/arm/kernel/kgdb.c | 2 +- arch/arm/kernel/patch.c | 2 +- arch/arm/probes/kprobes/core.c | 2 +- arch/arm/probes/kprobes/opt-arm.c | 2 +- .../asm/{patching.h => text-patching.h} | 0 arch/arm64/kernel/ftrace.c | 2 +- arch/arm64/kernel/jump_label.c | 2 +- arch/arm64/kernel/kgdb.c | 2 +- arch/arm64/kernel/patching.c | 2 +- arch/arm64/kernel/probes/kprobes.c | 2 +- arch/arm64/kernel/traps.c | 2 +- arch/arm64/net/bpf_jit_comp.c | 2 +- arch/csky/include/asm/Kbuild | 1 + arch/hexagon/include/asm/Kbuild | 1 + arch/loongarch/include/asm/Kbuild | 1 + arch/m68k/include/asm/Kbuild | 1 + arch/microblaze/include/asm/Kbuild | 1 + arch/mips/include/asm/Kbuild | 1 + arch/nios2/include/asm/Kbuild | 1 + arch/openrisc/include/asm/Kbuild | 1 + .../include/asm/{patch.h => text-patching.h} | 0 arch/parisc/kernel/ftrace.c | 2 +- arch/parisc/kernel/jump_label.c | 2 +- arch/parisc/kernel/kgdb.c | 2 +- arch/parisc/kernel/kprobes.c | 2 +- arch/parisc/kernel/patch.c | 2 +- arch/powerpc/include/asm/kprobes.h | 2 +- .../asm/{code-patching.h => text-patching.h} | 0 arch/powerpc/kernel/crash_dump.c | 2 +- arch/powerpc/kernel/epapr_paravirt.c | 2 +- arch/powerpc/kernel/jump_label.c | 2 +- arch/powerpc/kernel/kgdb.c | 2 +- arch/powerpc/kernel/kprobes.c | 2 +- arch/powerpc/kernel/module_32.c | 2 +- arch/powerpc/kernel/module_64.c | 2 +- arch/powerpc/kernel/optprobes.c | 2 +- arch/powerpc/kernel/process.c | 2 +- arch/powerpc/kernel/security.c | 2 +- arch/powerpc/kernel/setup_32.c | 2 +- arch/powerpc/kernel/setup_64.c | 2 +- arch/powerpc/kernel/static_call.c | 2 +- arch/powerpc/kernel/trace/ftrace.c | 2 +- arch/powerpc/kernel/trace/ftrace_64_pg.c | 2 +- arch/powerpc/lib/code-patching.c | 2 +- arch/powerpc/lib/feature-fixups.c | 2 +- arch/powerpc/lib/test-code-patching.c | 2 +- arch/powerpc/lib/test_emulate_step.c | 2 +- arch/powerpc/mm/book3s32/mmu.c | 2 +- arch/powerpc/mm/book3s64/hash_utils.c | 2 +- arch/powerpc/mm/book3s64/slb.c | 2 +- arch/powerpc/mm/kasan/init_32.c | 2 +- arch/powerpc/mm/mem.c | 2 +- arch/powerpc/mm/nohash/44x.c | 2 +- arch/powerpc/mm/nohash/book3e_pgtable.c | 2 +- arch/powerpc/mm/nohash/tlb.c | 2 +- arch/powerpc/mm/nohash/tlb_64e.c | 2 +- arch/powerpc/net/bpf_jit_comp.c | 2 +- arch/powerpc/perf/8xx-pmu.c | 2 +- arch/powerpc/perf/core-book3s.c | 2 +- arch/powerpc/platforms/85xx/smp.c | 2 +- arch/powerpc/platforms/86xx/mpc86xx_smp.c | 2 +- arch/powerpc/platforms/cell/smp.c | 2 +- arch/powerpc/platforms/powermac/smp.c | 2 +- arch/powerpc/platforms/powernv/idle.c | 2 +- arch/powerpc/platforms/powernv/smp.c | 2 +- arch/powerpc/platforms/pseries/smp.c | 2 +- arch/powerpc/xmon/xmon.c | 2 +- arch/riscv/errata/andes/errata.c | 2 +- arch/riscv/errata/sifive/errata.c | 2 +- arch/riscv/errata/thead/errata.c | 2 +- .../include/asm/{patch.h => text-patching.h} | 0 arch/riscv/include/asm/uprobes.h | 2 +- arch/riscv/kernel/alternative.c | 2 +- arch/riscv/kernel/cpufeature.c | 3 +- arch/riscv/kernel/ftrace.c | 2 +- arch/riscv/kernel/jump_label.c | 2 +- arch/riscv/kernel/patch.c | 2 +- arch/riscv/kernel/probes/kprobes.c | 2 +- arch/riscv/net/bpf_jit_comp64.c | 2 +- arch/riscv/net/bpf_jit_core.c | 2 +- arch/sh/include/asm/Kbuild | 1 + arch/sparc/include/asm/Kbuild | 1 + arch/um/kernel/um_arch.c | 16 +- arch/x86/entry/vdso/vma.c | 3 +- arch/x86/include/asm/alternative.h | 14 +- arch/x86/include/asm/text-patching.h | 1 + arch/x86/kernel/alternative.c | 160 ++++++---- arch/x86/kernel/ftrace.c | 41 ++- arch/x86/kernel/module.c | 45 ++- arch/x86/mm/init.c | 26 +- arch/xtensa/include/asm/Kbuild | 1 + include/asm-generic/text-patching.h | 5 + include/linux/execmem.h | 25 ++ include/linux/ftrace.h | 2 + include/linux/module.h | 9 + include/linux/moduleloader.h | 4 + include/linux/text-patching.h | 15 + include/linux/vmalloc.h | 60 ++-- kernel/module/main.c | 77 ++++- kernel/module/strict_rwx.c | 3 + kernel/trace/ftrace.c | 19 +- mm/execmem.c | 300 +++++++++++++++++- mm/vmalloc.c | 9 +- 108 files changed, 752 insertions(+), 236 deletions(-) rename arch/arm/include/asm/{patch.h => text-patching.h} (100%) rename arch/arm64/include/asm/{patching.h => text-patching.h} (100%) rename arch/parisc/include/asm/{patch.h => text-patching.h} (100%) rename arch/powerpc/include/asm/{code-patching.h => text-patching.h} (100%) rename arch/riscv/include/asm/{patch.h => text-patching.h} (100%) create mode 100644 include/asm-generic/text-patching.h create mode 100644 include/linux/text-patching.h base-commit: 47ac09b91befbb6a235ab620c32af719f8208399