From patchwork Mon Sep 18 07:29:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Rapoport X-Patchwork-Id: 13388969 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 16E22C46CA1 for ; Mon, 18 Sep 2023 07:30:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9D7576B0274; Mon, 18 Sep 2023 03:30:20 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 95E4D6B0276; Mon, 18 Sep 2023 03:30:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7D7FA6B0277; Mon, 18 Sep 2023 03:30:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6789F6B0274 for ; Mon, 18 Sep 2023 03:30:20 -0400 (EDT) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 3CDC44032F for ; Mon, 18 Sep 2023 07:30:20 +0000 (UTC) X-FDA: 81248895000.10.A5DE3B2 Received: from ams.source.kernel.org (ams.source.kernel.org [145.40.68.75]) by imf17.hostedemail.com (Postfix) with ESMTP id 224B540023 for ; Mon, 18 Sep 2023 07:30:17 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="XNmM3/b0"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of rppt@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695022218; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=a5CeRUd4x3iHmNCACB2XlzGscXJQEwvcpYcuiksIRAU=; b=6mwqNe0AKeFQsGSJwrk/WHd9cnMfdL1L5xjT8V5Ta6q2nW01Fg8DR2S8Nv1SQ8engygVTe 5BWbsavob4IPtoPr9H+CAOm/SADsSsNLYGc7pRqRuKvfanNMCTRDF0LErFiTLkvmAeBnBc WfWHFGfvAU2eWF7Hy481BQe+j3z/M14= ARC-Authentication-Results: i=1; imf17.hostedemail.com; dkim=pass header.d=kernel.org header.s=k20201202 header.b="XNmM3/b0"; dmarc=pass (policy=none) header.from=kernel.org; spf=pass (imf17.hostedemail.com: domain of rppt@kernel.org designates 145.40.68.75 as permitted sender) smtp.mailfrom=rppt@kernel.org ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1695022218; a=rsa-sha256; cv=none; b=nzjiucFCua4CAWZ8Sfx3b/sK9Y66CdhfY0RaO7RqUcdDi1S4GyIB0FKAfJS7HZHNvsFfMN k+A2+K89DJljSSARf0Z93Fwu3WXKq2jNiwoYbTaEizfHQgVS5wE1LmTgFZYFEcMZI+HK5Q M4ucS586FGYbTkvGXYrE0fnnuoPiXhc= Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 361F0B80BEC; Mon, 18 Sep 2023 07:30:16 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 8DF90C433C7; Mon, 18 Sep 2023 07:30:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1695022214; bh=FqY2R7Jtz2z1dS35nfgqn/BnUCyRu2xGf4mJU4r0njM=; h=From:To:Cc:Subject:Date:From; b=XNmM3/b0JFbkgOm/zfpoekf5cae26/g6XLPaphd0+yZhWzNrdc4u19ycIPTWNFm6m 9Celv2Wb3Q5nRFcdMvHFJ4GRqO2qyDDZz1jP7wIEZ3Y8JHy68r3oUsQ4IEO0Vtjd5l MWP6ADW37I33+ugGA7YfuegWUH+Gn366DxorgNUDiJ4JU0hFytzxH6zJ4ahVC4OF7o i2CGqshElLIQD67W0Y/co2EvMmfkgVCBMPR9hHDUkZjT/Awusm1X0DUINdeDGtq7ce deuuDA2yx7KoWsRzJZri3bCH7lilJOu6sbE4PTUNL0yHtV0Ghtcx/Pg5zFymPEenhw p/MI2wjlDBs5Q== From: Mike Rapoport To: linux-kernel@vger.kernel.org Cc: Andrew Morton , =?utf-8?b?QmrDtnJuIFTDtnBl?= =?utf-8?b?bA==?= , Catalin Marinas , Christophe Leroy , "David S. Miller" , Dinh Nguyen , Heiko Carstens , Helge Deller , Huacai Chen , Kent Overstreet , Luis Chamberlain , Mark Rutland , Michael Ellerman , Mike Rapoport , Nadav Amit , "Naveen N. Rao" , Palmer Dabbelt , Puranjay Mohan , Rick Edgecombe , Russell King , Song Liu , Steven Rostedt , Thomas Bogendoerfer , Thomas Gleixner , Will Deacon , bpf@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-mips@vger.kernel.org, linux-mm@kvack.org, linux-modules@vger.kernel.org, linux-parisc@vger.kernel.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, loongarch@lists.linux.dev, netdev@vger.kernel.org, sparclinux@vger.kernel.org, x86@kernel.org Subject: [PATCH v3 00/13] mm: jit/text allocator Date: Mon, 18 Sep 2023 10:29:42 +0300 Message-Id: <20230918072955.2507221-1-rppt@kernel.org> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 224B540023 X-Stat-Signature: behzfoabowwyq6893uwz3z85inohz4w3 X-HE-Tag: 1695022217-404049 X-HE-Meta: U2FsdGVkX1/QyCaZATTVJ4tWD1EgCZAaW2372+Ua8ZgwsECRXZis9noh1Dv6pDxtmPd6S3lu1eSbBRzPk1jTbYNH6TThSQ1W98m10pMvlld1bex/Q9b4KJL8XgdQFeEBQf4rlFQMwsJUHDFlBBAL8SiHWVFDwj7D89V3sMdrsG3Ai0jNLbHr9BvqV+anEwdfP9DilMMOLxzWjY3JnFAti4OSMKLSUayi/bh6jfa8BlHAscVG1jZS4ePQqQsAStuAnj91OC424baNe7eBsL05RnFAeHesAtXyCAi3OAg+epeWhlnrhmiWINKBFhSyTyZeUOm4Hg/zk7NR0Z1Hq8Chvmu/GDCodX5iWjQXC5OZPNUOsdgMeuD4j77WFmmqYIdJV3TUmiS5Pf+6mNMXEDueq9AS7erCfRhmMk7d88m0N4au0sbDgdiZrqWSZFDysvPUT4uew6mkq+93kX7QXDU7DdqWdRRcCRzr3rp6gidezM6DtNt/nR/myqtDA+nKewLoeqprxwoG1mCRHRITL7RRrjJiz2pWJC2xfkBUpfRGVxX30S2UXWa5KB5Nib3ev38qA70+EvrzSTz2ckfZX1Q1N64VYhIjileZH2nyGZt5HuA2zuQl8BnS2nDL50VGRf7IEUjMi1GR8JAZ4WH8LGh8M5GziRlCzJbG8eZYCG99x80hxYA0OLW573GoS0a2gISMPQ7pj1oiMB1eqxbtKBP47E/Gzh5Er0zg4M7PIhcmJjJ+OI367jce/O8A/rpGAnktEnNCeYvZlRNR5/Clobsrf0rmF953z6gjXUBOdflgAX4rXsnc+/dmWVu3rwp+1BXJWv1Wk0BsAVE6ugCBsJJZi9jqhHsMWO79vppLk6Fd+oImBIhYC7YBzBB3/QuC2VDcfDgo0j1sJCtVp4gFl8TaMqB3uzeNriQdZ7fWMQd6NzdHm61GZREzySq74QE0az2qznDfv8OeHTj2rN1+DrY AfYqDtnv RscSTqjnUPt2jidJWCpTJ78y1kU2fqfaOF7cYAaQYcOLbLgQxB60KDMtmAYtfuydlEtKm/zpVDgUQLQ/34fGLg4zQtn6SN5z/RvW7DsYMBQ6tMc/iLj/wHlb3EXiCWGmXOv7AjQcqv7jQ4E/53vBZG3C2Fc+WzS07jZ33rRA44UL0DCplQLfzYO2m99yg2jq6OVbn+VJAPdYaMNLrvZ88XxH4MMpXDWAEODjGQZjfvTLWsQCehcvmV35HeII//lJnBnQTBFYqPlb53OZOsxzK0Gt9hYQiDqLRaIqZiN6KaXrVRXW+YGDmBVQsbNgaIZCAAziOYFtwFfL1TkgtAwB/6SzqYpusmOWb2lFU4/f7EaBagA0Wfb/LZK1YKjgALNRnCR8ZS3LCCgYNNyklYq4TqRdpXNkeyWd5wc7LCFJ5f0RRzlOWk9Ai4rVrajXIN6Sr3x3m X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: "Mike Rapoport (IBM)" Hi, module_alloc() is used everywhere as a mean to allocate memory for code. Beside being semantically wrong, this unnecessarily ties all subsystmes that need to allocate code, such as ftrace, kprobes and BPF to modules and puts the burden of code allocation to the modules code. Several architectures override module_alloc() because of various constraints where the executable memory can be located and this causes additional obstacles for improvements of code allocation. A centralized infrastructure for code allocation allows allocations of executable memory as ROX, and future optimizations such as caching large pages for better iTLB performance and providing sub-page allocations for users that only need small jit code snippets. Rick Edgecombe proposed perm_alloc extension to vmalloc [1] and Song Liu proposed execmem_alloc [2], but both these approaches were targeting BPF allocations and lacked the ground work to abstract executable allocations and split them from the modules core. Thomas Gleixner suggested to express module allocation restrictions and requirements as struct mod_alloc_type_params [3] that would define ranges, protections and other parameters for different types of allocations used by modules and following that suggestion Song separated allocations of different types in modules (commit ac3b43283923 ("module: replace module_layout with module_memory")) and posted "Type aware module allocator" set [4]. I liked the idea of parametrising code allocation requirements as a structure, but I believe the original proposal and Song's module allocator was too module centric, so I came up with these patches. This set splits code allocation from modules by introducing execmem_text_alloc(), execmem_data_alloc() and execmem_free(), APIs, replaces call sites of module_alloc() and module_memfree() with the new APIs and implements core text and related allocations in a central place. Instead of architecture specific overrides for module_alloc(), the architectures that require non-default behaviour for text allocation must fill execmem_alloc_params structure and implement execmem_arch_params() that returns a pointer to that structure. If an architecture does not implement execmem_arch_params(), the defaults compatible with the current modules::module_alloc() are used. Since architectures define different restrictions on placement, permissions, alignment and other parameters for memory that can be used by different subsystems that allocate executable memory, execmem APIs take a type argument, that will be used to identify the calling subsystem and to allow architectures to define parameters for ranges suitable for that subsystem. The new infrastructure allows decoupling of BPF, kprobes and ftrace from modules, and most importantly it paves the way for ROX allocations for executable memory. [1] https://lore.kernel.org/lkml/20201120202426.18009-1-rick.p.edgecombe@intel.com/ [2] https://lore.kernel.org/all/20221107223921.3451913-1-song@kernel.org/ [3] https://lore.kernel.org/all/87v8mndy3y.ffs@tglx/ [4] https://lore.kernel.org/all/20230526051529.3387103-1-song@kernel.org v3 changes: * add type parameter to execmem allocation APIs * remove BPF dependency on modules v2: https://lore.kernel.org/all/20230616085038.4121892-1-rppt@kernel.org * Separate "module" and "others" allocations with execmem_text_alloc() and jit_text_alloc() * Drop ROX entablement on x86 * Add ack for nios2 changes, thanks Dinh Nguyen v1: https://lore.kernel.org/all/20230601101257.530867-1-rppt@kernel.org Mike Rapoport (IBM) (13): nios2: define virtual address space for modules mm: introduce execmem_text_alloc() and execmem_free() mm/execmem, arch: convert simple overrides of module_alloc to execmem mm/execmem, arch: convert remaining overrides of module_alloc to execmem modules, execmem: drop module_alloc mm/execmem: introduce execmem_data_alloc() arm64, execmem: extend execmem_params for generated code allocations riscv: extend execmem_params for generated code allocations powerpc: extend execmem_params for kprobes allocations arch: make execmem setup available regardless of CONFIG_MODULES x86/ftrace: enable dynamic ftrace without CONFIG_MODULES kprobes: remove dependency on CONFIG_MODULES bpf: remove CONFIG_BPF_JIT dependency on CONFIG_MODULES of arch/Kconfig | 2 +- arch/arm/kernel/module.c | 32 ------- arch/arm/mm/init.c | 38 ++++++++ arch/arm64/kernel/module.c | 124 ------------------------- arch/arm64/kernel/probes/kprobes.c | 7 -- arch/arm64/mm/init.c | 132 +++++++++++++++++++++++++++ arch/arm64/net/bpf_jit_comp.c | 11 --- arch/loongarch/kernel/module.c | 6 -- arch/loongarch/mm/init.c | 20 ++++ arch/mips/kernel/module.c | 10 +- arch/mips/mm/init.c | 20 ++++ arch/nios2/include/asm/pgtable.h | 5 +- arch/nios2/kernel/module.c | 28 +++--- arch/parisc/kernel/module.c | 12 +-- arch/parisc/mm/init.c | 22 ++++- arch/powerpc/kernel/kprobes.c | 16 +--- arch/powerpc/kernel/module.c | 37 -------- arch/powerpc/mm/mem.c | 62 +++++++++++++ arch/riscv/kernel/module.c | 10 -- arch/riscv/kernel/probes/kprobes.c | 10 -- arch/riscv/mm/init.c | 39 ++++++++ arch/riscv/net/bpf_jit_core.c | 13 --- arch/s390/kernel/ftrace.c | 4 +- arch/s390/kernel/kprobes.c | 4 +- arch/s390/kernel/module.c | 42 +-------- arch/s390/mm/init.c | 28 ++++++ arch/sparc/kernel/module.c | 33 +------ arch/sparc/mm/Makefile | 2 + arch/sparc/mm/execmem.c | 25 +++++ arch/sparc/net/bpf_jit_comp_32.c | 8 +- arch/x86/Kconfig | 1 + arch/x86/kernel/ftrace.c | 16 +--- arch/x86/kernel/kprobes/core.c | 4 +- arch/x86/kernel/module.c | 51 ----------- arch/x86/mm/init.c | 29 ++++++ include/linux/execmem.h | 141 ++++++++++++++++++++++++++++ include/linux/moduleloader.h | 15 --- kernel/bpf/Kconfig | 2 +- kernel/bpf/core.c | 6 +- kernel/kprobes.c | 51 ++++++----- kernel/module/Kconfig | 1 + kernel/module/main.c | 45 ++------- kernel/trace/trace_kprobe.c | 11 +++ mm/Kconfig | 3 + mm/Makefile | 1 + mm/execmem.c | 142 +++++++++++++++++++++++++++++ mm/mm_init.c | 2 + 47 files changed, 801 insertions(+), 522 deletions(-) create mode 100644 arch/sparc/mm/execmem.c create mode 100644 include/linux/execmem.h create mode 100644 mm/execmem.c base-commit: 0bb80ecc33a8fb5a682236443c1e740d5c917d1d