| Message ID | 20221121171202.22080-1-vbabka@suse.cz (mailing list archive) |
|---|---|
| Headers | show |
| Series | Introduce CONFIG_SLUB_TINY and deprecate SLOB | expand |
On Mon, Nov 21, 2022, at 18:11, Vlastimil Babka wrote: > > this continues the discussion from [1]. Reasons to remove SLOB are > outlined there and no-one has objected so far. The last patch of this > series therefore deprecates CONFIG_SLOB and updates all the defconfigs > using CONFIG_SLOB=y in the tree. > > There is a k210 board with 8MB RAM where switching to SLUB caused issues > [2] and the lkp bot wasn't also happy about code bloat [3]. To address > both, this series introduces CONFIG_SLUB_TINY to perform some rather > low-hanging fruit modifications to SLUB to reduce its memory overhead. > This seems to have been successful at least in the k210 case [4]. I > consider this as an acceptable tradeoff for getting rid of SLOB. I agree that this is a great success for replacing SLOB on the smallest machines that have 32MB or less and have to run a a highly customized kernel, and this is probably enough to have a drop-in replacement without making any currently working system worse. On the other hand, I have the feeling that we may want something a bit less aggressive than this for machines that are slightly less constrained, in particular when a single kernel needs to scale from 64MB to 512MB, which can happen e.g. on OpenWRT. I have seen a number of reports over the years that suggest that new kernels handle fragmentation and low memory worse than old ones, and it would be great to improve that again. I can imagine those machines wanting to use sysfs in general but not for the slab caches, so having a separate knob to configure out the sysfs stuff could be useful without having to go all the way to SLUB_TINY. For the options that trade off performance against lower fragmentation (MIN/MAX_PARTIAL, KMALLOC_RECLAIM, percpu slabs), I wonder if it's possible to have a boot time default based on the amount of RAM per CPU to have a better tuned system on most cases, rather than having to go to one extreme or the other at compile time. Arnd https://openwrt.org/toh/views/toh_standard_all?datasrt=target&dataflt%5B0%5D=availability_%3DAvailable%202021
On 11/22/22 17:33, Arnd Bergmann wrote: > On Mon, Nov 21, 2022, at 18:11, Vlastimil Babka wrote: >> >> this continues the discussion from [1]. Reasons to remove SLOB are >> outlined there and no-one has objected so far. The last patch of this >> series therefore deprecates CONFIG_SLOB and updates all the defconfigs >> using CONFIG_SLOB=y in the tree. >> >> There is a k210 board with 8MB RAM where switching to SLUB caused issues >> [2] and the lkp bot wasn't also happy about code bloat [3]. To address >> both, this series introduces CONFIG_SLUB_TINY to perform some rather >> low-hanging fruit modifications to SLUB to reduce its memory overhead. >> This seems to have been successful at least in the k210 case [4]. I >> consider this as an acceptable tradeoff for getting rid of SLOB. > > I agree that this is a great success for replacing SLOB on the > smallest machines that have 32MB or less and have to run a > a highly customized kernel, and this is probably enough to > have a drop-in replacement without making any currently working > system worse. > > On the other hand, I have the feeling that we may want something > a bit less aggressive than this for machines that are slightly > less constrained, in particular when a single kernel needs to > scale from 64MB to 512MB, which can happen e.g. on OpenWRT. > I have seen a number of reports over the years that suggest > that new kernels handle fragmentation and low memory worse than > old ones, and it would be great to improve that again. I see. That would need to study such reports and see if the problem there is actually SLUB, or the page allocator or something else entirely. > I can imagine those machines wanting to use sysfs in general > but not for the slab caches, so having a separate knob to > configure out the sysfs stuff could be useful without having > to go all the way to SLUB_TINY. Right, but AFAIK that wouldn't save much except some text size and kobjects, so probably negligible for >32MB? > For the options that trade off performance against lower > fragmentation (MIN/MAX_PARTIAL, KMALLOC_RECLAIM, percpu > slabs), I wonder if it's possible to have a boot time > default based on the amount of RAM per CPU to have a better > tuned system on most cases, rather than having to go > to one extreme or the other at compile time. Possible for some of these things, but for others that brings us back to the question what are the actual observed issues. If it's low memory in absolute number of pages, these can help, but if it's fragmentation (and the kind if RAM sizes should have page grouping by mobility enabled), ditching e.g. the KMALLOC_RECLAIM could make it worse. Unfortunately some of these tradeoffs can be rather unpredictable. Thanks, Vlastimil > Arnd > > https://openwrt.org/toh/views/toh_standard_all?datasrt=target&dataflt%5B0%5D=availability_%3DAvailable%202021
On Tue, Nov 22, 2022, at 17:59, Vlastimil Babka wrote: > On 11/22/22 17:33, Arnd Bergmann wrote: >> On Mon, Nov 21, 2022, at 18:11, Vlastimil Babka wrote: >> I can imagine those machines wanting to use sysfs in general >> but not for the slab caches, so having a separate knob to >> configure out the sysfs stuff could be useful without having >> to go all the way to SLUB_TINY. > > Right, but AFAIK that wouldn't save much except some text size and kobjects, > so probably negligible for >32MB? Makes sense, I assume you have a better idea of how much this could save. I'm not at all worried about the .text size, but my initial guess was that the metadata for sysfs would be noticeable. >> For the options that trade off performance against lower >> fragmentation (MIN/MAX_PARTIAL, KMALLOC_RECLAIM, percpu >> slabs), I wonder if it's possible to have a boot time >> default based on the amount of RAM per CPU to have a better >> tuned system on most cases, rather than having to go >> to one extreme or the other at compile time. > > Possible for some of these things, but for others that brings us back to the > question what are the actual observed issues. If it's low memory in absolute > number of pages, these can help, but if it's fragmentation (and the kind if > RAM sizes should have page grouping by mobility enabled), ditching e.g. the > KMALLOC_RECLAIM could make it worse. Unfortunately some of these tradeoffs > can be rather unpredictable. Are there any obvious wins on memory uage? I would guess that it would be safe to e.g. ditch percpu slabs when running with less 128MB per CPU, and the MIN/MAX_PARTIAL values could easily be a function of the number of pages in total or per cpu, whichever makes most sense. As a side-effect, those could also grow slightly larger on huge systems by scaling them with log2(totalpages). Arnd
On Mon, Nov 21, 2022 at 06:11:50PM +0100, Vlastimil Babka wrote: > Hi, > > this continues the discussion from [1]. Reasons to remove SLOB are > outlined there and no-one has objected so far. The last patch of this > series therefore deprecates CONFIG_SLOB and updates all the defconfigs > using CONFIG_SLOB=y in the tree. > > There is a k210 board with 8MB RAM where switching to SLUB caused issues > [2] and the lkp bot wasn't also happy about code bloat [3]. To address > both, this series introduces CONFIG_SLUB_TINY to perform some rather > low-hanging fruit modifications to SLUB to reduce its memory overhead. > This seems to have been successful at least in the k210 case [4]. I > consider this as an acceptable tradeoff for getting rid of SLOB. > > The series is also available in git: > https://git.kernel.org/pub/scm/linux/kernel/git/vbabka/linux.git/log/?h=slub-tiny-v1r2 > > [1] https://lore.kernel.org/all/b35c3f82-f67b-2103-7d82-7a7ba7521439@suse.cz/ > [2] https://lore.kernel.org/all/a5bba3ca-da19-293c-c01b-a28291533466@opensource.wdc.com/ > [3] https://lore.kernel.org/all/Y25E9cJbhDAKi1vd@99bb1221be19/ > [4] https://lore.kernel.org/all/6a1883c4-4c3f-545a-90e8-2cd805bcf4ae@opensource.wdc.com/ > > Vlastimil Babka (12): > mm, slab: ignore hardened usercopy parameters when disabled > mm, slub: add CONFIG_SLUB_TINY > mm, slub: disable SYSFS support with CONFIG_SLUB_TINY > mm, slub: retain no free slabs on partial list with CONFIG_SLUB_TINY > mm, slub: lower the default slub_max_order with CONFIG_SLUB_TINY > mm, slub: don't create kmalloc-rcl caches with CONFIG_SLUB_TINY > mm, slab: ignore SLAB_RECLAIM_ACCOUNT with CONFIG_SLUB_TINY > mm, slub: refactor free debug processing > mm, slub: split out allocations from pre/post hooks > mm, slub: remove percpu slabs with CONFIG_SLUB_TINY > mm, slub: don't aggressively inline with CONFIG_SLUB_TINY > mm, slob: rename CONFIG_SLOB to CONFIG_SLOB_DEPRECATED > > arch/arm/configs/clps711x_defconfig | 3 +- > arch/arm/configs/collie_defconfig | 3 +- > arch/arm/configs/multi_v4t_defconfig | 3 +- > arch/arm/configs/omap1_defconfig | 3 +- > arch/arm/configs/pxa_defconfig | 3 +- > arch/arm/configs/tct_hammer_defconfig | 3 +- > arch/arm/configs/xcep_defconfig | 3 +- > arch/openrisc/configs/or1ksim_defconfig | 3 +- > arch/openrisc/configs/simple_smp_defconfig | 3 +- > arch/riscv/configs/nommu_k210_defconfig | 3 +- > .../riscv/configs/nommu_k210_sdcard_defconfig | 3 +- > arch/riscv/configs/nommu_virt_defconfig | 3 +- > arch/sh/configs/rsk7201_defconfig | 3 +- > arch/sh/configs/rsk7203_defconfig | 3 +- > arch/sh/configs/se7206_defconfig | 3 +- > arch/sh/configs/shmin_defconfig | 3 +- > arch/sh/configs/shx3_defconfig | 3 +- > include/linux/slab.h | 8 + > include/linux/slub_def.h | 6 +- > kernel/configs/tiny.config | 5 +- > mm/Kconfig | 38 +- > mm/Kconfig.debug | 2 +- > mm/slab_common.c | 16 +- > mm/slub.c | 415 ++++++++++++------ > 24 files changed, 377 insertions(+), 164 deletions(-) For the series Acked-by: Mike Rapoport <rppt@linux.ibm.com>