| Message ID | ZEbkqq1tvm1WHVHw@bombadil.infradead.org (mailing list archive) |
|---|---|
| State | New, archived |
| Headers | show |
| Series | [GIT,PULL] modules changes for v6.4-rc1 | expand |
The pull request you sent on Mon, 24 Apr 2023 13:20:58 -0700:
> git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/modules-6.4-rc1
has been merged into torvalds/linux.git:
https://git.kernel.org/torvalds/c/b6a7828502dc769e1a5329027bc5048222fa210a
Thank you!
The following changes since commit fe15c26ee26efa11741a7b632e9f23b01aca4cc6: Linux 6.3-rc1 (2023-03-05 14:52:03 -0800) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/mcgrof/linux.git/ tags/modules-6.4-rc1 for you to fetch changes up to 8660484ed1cf3261e89e0bad94c6395597e87599: module: add debugging auto-load duplicate module support (2023-04-19 17:26:01 -0700) ---------------------------------------------------------------- modules-6.4-rc1 The summary of the changes for this pull requests is: * Song Liu's new struct module_memory replacement * Nick Alcock's MODULE_LICENSE() removal for non-modules * My cleanups and enhancements to reduce the areas where we vmalloc module memory for duplicates, and the respective debug code which proves the remaining vmalloc pressure comes from userspace. Most of the changes have been in linux-next for quite some time except the minor fixes I made to check if a module was already loaded prior to allocating the final module memory with vmalloc and the respective debug code it introduces to help clarify the issue. Although the functional change is small it is rather safe as it can only *help* reduce vmalloc space for duplicates and is confirmed to fix a bootup issue with over 400 CPUs with KASAN enabled. I don't expect stable kernels to pick up that fix as the cleanups would have also had to have been picked up. Folks on larger CPU systems with modules will want to just upgrade if vmalloc space has been an issue on bootup. Given the size of this request, here's some more elaborate details on this pull request. The functional change change in this pull request is the very first patch from Song Liu which replaces the struct module_layout with a new struct module memory. The old data structure tried to put together all types of supported module memory types in one data structure, the new one abstracts the differences in memory types in a module to allow each one to provide their own set of details. This paves the way in the future so we can deal with them in a cleaner way. If you look at changes they also provide a nice cleanup of how we handle these different memory areas in a module. This change has been in linux-next since before the merge window opened for v6.3 so to provide more than a full kernel cycle of testing. It's a good thing as quite a bit of fixes have been found for it. Jason Baron then made dynamic debug a first class citizen module user by using module notifier callbacks to allocate / remove module specific dynamic debug information. Nick Alcock has done quite a bit of work cross-tree to remove module license tags from things which cannot possibly be module at my request so to: a) help him with his longer term tooling goals which require a deterministic evaluation if a piece a symbol code could ever be part of a module or not. But quite recently it is has been made clear that tooling is not the only one that would benefit. Disambiguating symbols also helps efforts such as live patching, kprobes and BPF, but for other reasons and R&D on this area is active with no clear solution in sight. b) help us inch closer to the now generally accepted long term goal of automating all the MODULE_LICENSE() tags from SPDX license tags In so far as a) is concerned, although module license tags are a no-op for non-modules, tools which would want create a mapping of possible modules can only rely on the module license tag after the commit 8b41fc4454e ("kbuild: create modules.builtin without Makefile.modbuiltin or tristate.conf"). Nick has been working on this *for years* and AFAICT I was the only one to suggest two alternatives to this approach for tooling. The complexity in one of my suggested approaches lies in that we'd need a possible-obj-m and a could-be-module which would check if the object being built is part of any kconfig build which could ever lead to it being part of a module, and if so define a new define -DPOSSIBLE_MODULE [0]. A more obvious yet theoretical approach I've suggested would be to have a tristate in kconfig imply the same new -DPOSSIBLE_MODULE as well but that means getting kconfig symbol names mapping to modules always, and I don't think that's the case today. I am not aware of Nick or anyone exploring either of these options. Quite recently Josh Poimboeuf has pointed out that live patching, kprobes and BPF would benefit from resolving some part of the disambiguation as well but for other reasons. The function granularity KASLR (fgkaslr) patches were mentioned but Joe Lawrence has clarified this effort has been dropped with no clear solution in sight [1]. In the meantime removing module license tags from code which could never be modules is welcomed for both objectives mentioned above. Some developers have also welcomed these changes as it has helped clarify when a module was never possible and they forgot to clean this up, and so you'll see quite a bit of Nick's patches in other pull requests for this merge window. I just picked up the stragglers after rc3. LWN has good coverage on the motivation behind this work [2] and the typical cross-tree issues he ran into along the way. The only concrete blocker issue he ran into was that we should not remove the MODULE_LICENSE() tags from files which have no SPDX tags yet, even if they can never be modules. Nick ended up giving up on his efforts due to having to do this vetting and backlash he ran into from folks who really did *not understand* the core of the issue nor were providing any alternative / guidance. I've gone through his changes and dropped the patches which dropped the module license tags where an SPDX license tag was missing, it only consisted of 11 drivers. To see if a pull request deals with a file which lacks SPDX tags you can just use: ./scripts/spdxcheck.py -f \ $(git diff --name-only commid-id | xargs echo) You'll see a core module file in this pull request for the above, but that's not related to his changes. WE just need to add the SPDX license tag for the kernel/module/kmod.c file in the future but it demonstrates the effectiveness of the script. Most of Nick's changes were spread out through different trees, and I just picked up the slack after rc3 for the last kernel was out. Those changes have been in linux-next for over two weeks. The cleanups, debug code I added and final fix I added for modules were motivated by David Hildenbrand's report of boot failing on a systems with over 400 CPUs when KASAN was enabled due to running out of virtual memory space. Although the functional change only consists of 3 lines in the patch "module: avoid allocation if module is already present and ready", proving that this was the best we can do on the modules side took quite a bit of effort and new debug code. The initial cleanups I did on the modules side of things has been in linux-next since around rc3 of the last kernel, the actual final fix for and debug code however have only been in linux-next for about a week or so but I think it is worth getting that code in for this merge window as it does help fix / prove / evaluate the issues reported with larger number of CPUs. Userspace is not yet fixed as it is taking a bit of time for folks to understand the crux of the issue and find a proper resolution. Worst come to worst, I have a kludge-of-concept [3] of how to make kernel_read*() calls for modules unique / converge them, but I'm currently inclined to just see if userspace can fix this instead. [0] https://lore.kernel.org/all/Y/kXDqW+7d71C4wz@bombadil.infradead.org/ [1] https://lkml.kernel.org/r/025f2151-ce7c-5630-9b90-98742c97ac65@redhat.com [2] https://lwn.net/Articles/927569/ [3] https://lkml.kernel.org/r/20230414052840.1994456-3-mcgrof@kernel.org ---------------------------------------------------------------- Arnd Bergmann (4): ARM: dyndbg: allow including dyndbg.h in decompressor module: stats: include uapi/linux/module.h module: fix building stats for 32-bit targets module: stats: fix invalid_mod_bytes typo Bagas Sanjaya (1): Documentation: core-api: update kernel-doc reference to kmod.c Fabio M. De Francesco (1): module/decompress: Never use kunmap() for local un-mappings Jason Baron (3): dyndbg: remove unused 'base' arg from __ddebug_add_module() dyndbg: use the module notifier callbacks dyndbg: cleanup dynamic usage in ib_srp.c Jiapeng Chong (1): module: Remove the unused function within Jim Cromie (4): module: in layout_sections, move_module: add the modname module: add symbol-name to pr_debug Absolute symbol module: add section-size to move_module pr_debug module: already_uses() - reduce pr_debug output volume Luis Chamberlain (24): kernel/module: add documentation for try_module_get() module: move get_modinfo() helpers all above module: rename next_string() to module_next_tag_pair() module: add a for_each_modinfo_entry() module: move early sanity checks into a helper module: move check_modinfo() early to early_mod_check() module: rename set_license() to module_license_taint_check() module: split taint work out of check_modinfo_livepatch() module: split taint adding with info checking module: move tainting until after a module hits our linked list module: move signature taint to module_augment_kernel_taints() module: converge taint work together module: rename check_module_license_and_versions() to check_export_symbol_versions() module: add sanity check for ELF module section module: add stop-grap sanity check on module memcpy() module: move more elf validity checks to elf_validity_check() module: merge remnants of setup_load_info() to elf validation module: fold usermode helper kmod into modules directory module: fix kmemleak annotations for non init ELF sections modules/kmod: replace implementation with a semaphore module: extract patient module check into helper module: add debug stats to help identify memory pressure module: avoid allocation if module is already present and ready module: add debugging auto-load duplicate module support Nick Alcock (73): soc: fujitsu: remove MODULE_LICENSE in non-modules mfd: altera-sysmgr: remove MODULE_LICENSE in non-modules irqchip/al-fic: remove MODULE_LICENSE in non-modules bus: arm-integrator-lm: remove MODULE_LICENSE in non-modules drivers/perf: remove MODULE_LICENSE in non-modules ARM: tegra: remove MODULE_LICENSE in non-modules pinctrl: actions: remove MODULE_LICENSE in non-modules soc: apple: apple-pmgr-pwrstate: remove MODULE_LICENSE in non-modules clk: bm1880: remove MODULE_LICENSE in non-modules bus: ixp4xx: remove MODULE_LICENSE in non-modules pinctrl: nuvoton: npcm7xx: remove MODULE_LICENSE in non-modules bus: qcom: remove MODULE_LICENSE in non-modules bus: remove MODULE_LICENSE in non-modules EDAC, altera: remove MODULE_LICENSE in non-modules power: reset: keystone-reset: remove MODULE_LICENSE in non-modules video: fbdev: remove MODULE_LICENSE in non-modules KEYS: remove MODULE_LICENSE in non-modules pinctrl: bcm: ns: remove MODULE_LICENSE in non-modules MIPS: BCM47XX: remove MODULE_LICENSE in non-modules clocksource: remove MODULE_LICENSE in non-modules clocksource/drivers/timer-tegra186: remove MODULE_LICENSE in non-modules clocksource: remove MODULE_LICENSE in non-modules crypto: remove MODULE_LICENSE in non-modules crypto: blake2s: remove module-related code crypto: remove MODULE_LICENSE in non-modules crypto: blake2s: remove module_init and module.h inclusion dmaengine: stm32-mdma: remove MODULE_LICENSE in non-modules dmaengine: remove MODULE_LICENSE in non-modules dma-mapping: benchmark: remove MODULE_LICENSE in non-modules dma-buf: heaps: remove MODULE_LICENSE in non-modules binfmt_elf: remove MODULE_LICENSE in non-modules phy: intel: remove MODULE_LICENSE in non-modules hwspinlock: remove MODULE_LICENSE in non-modules iommu/sun50i: remove MODULE_LICENSE in non-modules irqchip: remove MODULE_LICENSE in non-modules irqchip: remove MODULE_LICENSE in non-modules mailbox: rockchip: remove MODULE_LICENSE in non-modules mailbox: zynq: make modular power: reset: mt6397: remove MODULE_LICENSE in non-modules irqchip/mchp-eic: remove MODULE_LICENSE in non-modules NFSv4.2: remove MODULE_LICENSE in non-modules nvmem: core: remove MODULE_LICENSE in non-modules perf/hw_breakpoint: remove MODULE_LICENSE in non-modules pinctrl: amd: remove MODULE_LICENSE in non-modules pinctrl: mediatek: remove MODULE_LICENSE in non-modules pinctrl: renesas: remove MODULE_LICENSE in non-modules lib: remove MODULE_LICENSE in non-modules power: supply: remove MODULE_LICENSE in non-modules remoteproc: remove MODULE_LICENSE in non-modules clk: renesas: remove MODULE_LICENSE in non-modules reset: mchp: sparx5: remove MODULE_LICENSE in non-modules reset: lantiq: remove MODULE_LICENSE in non-modules clk: microchip: mpfs: remove MODULE_LICENSE in non-modules reset: mpfs: remove MODULE_LICENSE in non-modules rv/reactor: remove MODULE_LICENSE in non-modules irqchip/irq-sl28cpld: remove MODULE_LICENSE in non-modules power: reset: remove MODULE_LICENSE in non-modules soc/tegra: cbb: remove MODULE_LICENSE in non-modules irqchip: remove MODULE_LICENSE in non-modules bus: remove MODULE_LICENSE in non-modules braille_console: remove MODULE_LICENSE in non-modules drivers: bus: simple-pm-bus: remove MODULE_LICENSE in non-modules watch_queue: remove MODULE_LICENSE in non-modules btree: remove MODULE_LICENSE in non-modules treewide: remove MODULE_LICENSE in non-modules unicode: remove MODULE_LICENSE in non-modules udmabuf: remove MODULE_LICENSE in non-modules regulator: stm32-pwr: remove MODULE_LICENSE in non-modules x86/mm/dump_pagetables: remove MODULE_LICENSE in non-modules zpool: remove MODULE_LICENSE in non-modules zswap: remove MODULE_LICENSE in non-modules interconnect: remove MODULE_LICENSE in non-modules interconnect: remove module-related code Pankaj Raghav (1): scripts/gdb: use mem instead of core_layout to get the module address Peter Zijlstra (1): Change DEFINE_SEMAPHORE() to take a number argument Randy Dunlap (1): module: fix MIPS module_layout -> module_memory Song Liu (1): module: replace module_layout with module_memory Thomas Weißschuh (1): module: make module_ktype structure constant Tiezhu Yang (3): module: Sync code of is_arm_mapping_symbol() module: Move is_arm_mapping_symbol() to module_symbol.h module: Ignore L0 and rename is_arm_mapping_symbol() Tom Rix (1): module: remove use of uninitialized variable len Zhen Lei (1): kallsyms: Delete an unused parameter related to {module_}kallsyms_on_each_symbol() Documentation/admin-guide/kernel-parameters.txt | 6 + Documentation/core-api/kernel-api.rst | 24 +- .../translations/zh_CN/core-api/kernel-api.rst | 2 +- MAINTAINERS | 13 +- arch/arc/kernel/unwind.c | 12 +- arch/arm/boot/compressed/decompress.c | 1 + arch/arm/kernel/module-plts.c | 9 +- arch/arm64/kernel/module-plts.c | 13 +- arch/ia64/kernel/module.c | 24 +- arch/mips/cavium-octeon/setup.c | 2 +- arch/mips/kernel/vpe.c | 11 +- arch/parisc/kernel/module.c | 51 +- arch/powerpc/kernel/module_32.c | 7 +- arch/s390/kernel/module.c | 26 +- arch/x86/crypto/blake2s-glue.c | 5 +- arch/x86/kernel/callthunks.c | 4 +- arch/x86/kernel/cpu/intel.c | 2 +- arch/x86/kernel/module.c | 4 +- arch/x86/mm/debug_pagetables.c | 1 - crypto/asymmetric_keys/asymmetric_type.c | 1 - drivers/accessibility/braille/braille_console.c | 1 - drivers/amba/tegra-ahb.c | 1 - drivers/bus/arm-integrator-lm.c | 1 - drivers/bus/bt1-apb.c | 1 - drivers/bus/bt1-axi.c | 1 - drivers/bus/intel-ixp4xx-eb.c | 1 - drivers/bus/qcom-ebi2.c | 1 - drivers/bus/qcom-ssc-block-bus.c | 1 - drivers/bus/simple-pm-bus.c | 2 +- drivers/clk/clk-bm1880.c | 1 - drivers/clk/microchip/clk-mpfs.c | 1 - drivers/clk/renesas/renesas-cpg-mssr.c | 1 - drivers/clk/renesas/rzg2l-cpg.c | 1 - drivers/clocksource/em_sti.c | 1 - drivers/clocksource/sh_cmt.c | 1 - drivers/clocksource/sh_mtu2.c | 1 - drivers/clocksource/sh_tmu.c | 1 - drivers/clocksource/timer-stm32-lp.c | 1 - drivers/clocksource/timer-tegra186.c | 1 - drivers/clocksource/timer-ti-dm.c | 1 - drivers/dma-buf/heaps/cma_heap.c | 1 - drivers/dma-buf/heaps/system_heap.c | 1 - drivers/dma-buf/udmabuf.c | 1 - drivers/dma/ep93xx_dma.c | 1 - drivers/dma/ipu/ipu_idmac.c | 1 - drivers/dma/mv_xor_v2.c | 1 - drivers/dma/sh/shdma-base.c | 1 - drivers/dma/stm32-dmamux.c | 1 - drivers/dma/stm32-mdma.c | 1 - drivers/edac/altera_edac.c | 1 - drivers/firmware/broadcom/bcm47xx_nvram.c | 1 - drivers/firmware/efi/runtime-wrappers.c | 2 +- drivers/firmware/efi/vars.c | 2 +- drivers/hwspinlock/hwspinlock_core.c | 1 - drivers/infiniband/ulp/srp/ib_srp.c | 5 - drivers/interconnect/core.c | 5 - drivers/iommu/sun50i-iommu.c | 1 - drivers/irqchip/irq-al-fic.c | 1 - drivers/irqchip/irq-ls-scfg-msi.c | 1 - drivers/irqchip/irq-mbigen.c | 1 - drivers/irqchip/irq-mchp-eic.c | 1 - drivers/irqchip/irq-renesas-intc-irqpin.c | 1 - drivers/irqchip/irq-renesas-irqc.c | 1 - drivers/irqchip/irq-renesas-rza1.c | 1 - drivers/irqchip/irq-renesas-rzg2l.c | 1 - drivers/irqchip/irq-sl28cpld.c | 1 - drivers/irqchip/irq-ti-sci-inta.c | 1 - drivers/irqchip/irq-ti-sci-intr.c | 1 - drivers/macintosh/adb.c | 2 +- drivers/mailbox/Kconfig | 2 +- drivers/mailbox/rockchip-mailbox.c | 1 - drivers/mfd/altera-sysmgr.c | 1 - drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c | 2 +- drivers/nvmem/core.c | 1 - drivers/perf/apple_m1_cpu_pmu.c | 1 - drivers/phy/intel/phy-intel-lgm-combo.c | 1 - drivers/pinctrl/actions/pinctrl-s500.c | 1 - drivers/pinctrl/actions/pinctrl-s700.c | 1 - drivers/pinctrl/actions/pinctrl-s900.c | 1 - drivers/pinctrl/bcm/pinctrl-ns.c | 1 - drivers/pinctrl/mediatek/pinctrl-mt8188.c | 1 - drivers/pinctrl/mediatek/pinctrl-mt8192.c | 1 - drivers/pinctrl/mediatek/pinctrl-mt8365.c | 1 - drivers/pinctrl/nuvoton/pinctrl-npcm7xx.c | 1 - drivers/pinctrl/pinctrl-amd.c | 1 - drivers/pinctrl/renesas/pinctrl-rza1.c | 1 - drivers/pinctrl/renesas/pinctrl-rza2.c | 1 - drivers/pinctrl/renesas/pinctrl-rzg2l.c | 1 - drivers/pinctrl/renesas/pinctrl-rzn1.c | 1 - drivers/pinctrl/renesas/pinctrl-rzv2m.c | 1 - drivers/platform/x86/intel/ifs/sysfs.c | 2 +- drivers/power/reset/as3722-poweroff.c | 1 - drivers/power/reset/gpio-poweroff.c | 1 - drivers/power/reset/gpio-restart.c | 1 - drivers/power/reset/keystone-reset.c | 1 - drivers/power/reset/ltc2952-poweroff.c | 1 - drivers/power/reset/mt6323-poweroff.c | 1 - drivers/power/reset/regulator-poweroff.c | 1 - drivers/power/reset/restart-poweroff.c | 1 - drivers/power/reset/tps65086-restart.c | 1 - drivers/power/supply/power_supply_core.c | 1 - drivers/power/supply/wm97xx_battery.c | 1 - drivers/regulator/stm32-pwr.c | 1 - drivers/remoteproc/remoteproc_core.c | 1 - drivers/reset/reset-lantiq.c | 1 - drivers/reset/reset-microchip-sparx5.c | 1 - drivers/reset/reset-mpfs.c | 1 - drivers/scsi/esas2r/esas2r_ioctl.c | 2 +- drivers/soc/apple/apple-pmgr-pwrstate.c | 1 - drivers/soc/fujitsu/a64fx-diag.c | 1 - drivers/soc/tegra/cbb/tegra194-cbb.c | 1 - drivers/soc/tegra/cbb/tegra234-cbb.c | 1 - .../vc04_services/interface/vchiq_arm/vchiq_arm.c | 2 +- drivers/video/fbdev/wm8505fb.c | 1 - drivers/video/fbdev/wmt_ge_rops.c | 1 - fs/binfmt_elf.c | 1 - fs/nfs_common/nfs_ssc.c | 1 - fs/unicode/utf8-core.c | 1 - include/linux/dynamic_debug.h | 68 +- include/linux/kallsyms.h | 7 +- include/linux/module.h | 141 ++- include/linux/module_symbol.h | 17 + include/linux/semaphore.h | 10 +- kernel/Makefile | 1 - kernel/dma/map_benchmark.c | 1 - kernel/events/hw_breakpoint_test.c | 1 - kernel/kallsyms.c | 5 +- kernel/kallsyms_selftest.c | 6 +- kernel/livepatch/core.c | 3 +- kernel/module/Kconfig | 100 +- kernel/module/Makefile | 6 +- kernel/module/decompress.c | 6 +- kernel/module/dups.c | 246 +++++ kernel/module/internal.h | 140 ++- kernel/module/kallsyms.c | 78 +- kernel/module/kdb.c | 17 +- kernel/{ => module}/kmod.c | 49 +- kernel/module/main.c | 1091 +++++++++++--------- kernel/module/procfs.c | 16 +- kernel/module/stats.c | 430 ++++++++ kernel/module/strict_rwx.c | 99 +- kernel/module/tracking.c | 7 +- kernel/module/tree_lookup.c | 39 +- kernel/params.c | 2 +- kernel/printk/printk.c | 2 +- kernel/trace/ftrace.c | 3 +- kernel/trace/rv/reactor_panic.c | 1 - kernel/trace/rv/reactor_printk.c | 1 - kernel/watch_queue.c | 1 - lib/Makefile | 3 + lib/btree.c | 1 - lib/crypto/blake2s-generic.c | 5 - lib/crypto/blake2s.c | 1 - lib/dynamic_debug.c | 51 +- lib/pldmfw/pldmfw.c | 1 - lib/test_fprobe.c | 1 - mm/zpool.c | 1 - mm/zswap.c | 1 - net/rxrpc/call_object.c | 6 +- scripts/gdb/linux/constants.py.in | 3 + scripts/gdb/linux/modules.py | 4 +- scripts/gdb/linux/symbols.py | 4 +- scripts/mod/modpost.c | 12 +- 163 files changed, 1982 insertions(+), 1039 deletions(-) create mode 100644 include/linux/module_symbol.h create mode 100644 kernel/module/dups.c rename kernel/{ => module}/kmod.c (77%) create mode 100644 kernel/module/stats.c