| Message ID | 20250416180653.3438158-1-usamaarif642@gmail.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | alloc_tag: introduce Kconfig option for default compressed profiling | expand |
On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: > > With this Kconfig option enabled, the kernel stores allocation tag references > in the page flags by default. > > There are 2 reasons to introduce this: > - As mentioned in [1], compressed tags dont have system memory overhead > and much lower performance overhead. It would be preferrable to have this as > the default option, and to be able to switch it at compile time. Another > option is to just declare the static key as true by default? > - As compressed option is the best one, it doesn't make sense to have to > change both defconfig and command line options to enable memory > allocation profiling. Changing commandline across a large number of services > can result in signifcant work, which shouldn't be needed if the kernel > defconfig needs to be changed anyways. The reason tag compression is not the default option is because it works only if there are enough free bits in the page flags to store a tag index. If you configure it to use page flags and your build does not have enough free bits, the profiling will be disabled (see alloc_tag_sec_init()). IOW there is no graceful fallback to use page extensions. Therefore, the current default option is not the most performant but the one which works on all builds. Instead of this just set sysctl.vm.mem_profiling boot parameter in your config file. Your change effectively changes the default value of mem_profiling_compressed and I don't see why you need to introduce a new config option for that. But that really does not matter because changing default to compressed tags is not the right choice IMO. > > [1] https://lore.kernel.org/all/20241023170759.999909-7-surenb@google.com/T/#m0da08879435f7673eaa10871a6e9d1be4f605ac8 > > Signed-off-by: Usama Arif <usamaarif642@gmail.com> > --- > include/linux/pgalloc_tag.h | 4 ++++ > lib/Kconfig.debug | 5 +++++ > lib/alloc_tag.c | 4 ++++ > 3 files changed, 13 insertions(+) > > diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h > index c74077977830..0226059bcf00 100644 > --- a/include/linux/pgalloc_tag.h > +++ b/include/linux/pgalloc_tag.h > @@ -16,7 +16,11 @@ extern unsigned long alloc_tag_ref_mask; > extern int alloc_tag_ref_offs; > extern struct alloc_tag_kernel_section kernel_tags; > > +#ifdef CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT > +DECLARE_STATIC_KEY_TRUE(mem_profiling_compressed); > +#else > DECLARE_STATIC_KEY_FALSE(mem_profiling_compressed); > +#endif > > typedef u16 pgalloc_tag_idx; > > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > index 9fe4d8dfe578..66d8995f3514 100644 > --- a/lib/Kconfig.debug > +++ b/lib/Kconfig.debug > @@ -1028,6 +1028,11 @@ config MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT > default y > depends on MEM_ALLOC_PROFILING > > +config MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT > + bool "store page allocation tag references in the page flags by default" > + default y > + depends on MEM_ALLOC_PROFILING > + > config MEM_ALLOC_PROFILING_DEBUG > bool "Memory allocation profiler debugging" > default n > diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c > index 25ecc1334b67..30adad5630dd 100644 > --- a/lib/alloc_tag.c > +++ b/lib/alloc_tag.c > @@ -31,7 +31,11 @@ DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT, > mem_alloc_profiling_key); > EXPORT_SYMBOL(mem_alloc_profiling_key); > > +#ifdef CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT > +DEFINE_STATIC_KEY_TRUE(mem_profiling_compressed); > +#else > DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed); > +#endif > > struct alloc_tag_kernel_section kernel_tags = { NULL, 0 }; > unsigned long alloc_tag_ref_mask; > -- > 2.47.1 >
On Wed, Apr 16, 2025 at 02:08:31PM -0700, Suren Baghdasaryan wrote: > On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: > > > > With this Kconfig option enabled, the kernel stores allocation tag references > > in the page flags by default. > > > > There are 2 reasons to introduce this: > > - As mentioned in [1], compressed tags dont have system memory overhead > > and much lower performance overhead. It would be preferrable to have this as > > the default option, and to be able to switch it at compile time. Another > > option is to just declare the static key as true by default? > > - As compressed option is the best one, it doesn't make sense to have to > > change both defconfig and command line options to enable memory > > allocation profiling. Changing commandline across a large number of services > > can result in signifcant work, which shouldn't be needed if the kernel > > defconfig needs to be changed anyways. > > The reason tag compression is not the default option is because it > works only if there are enough free bits in the page flags to store a > tag index. If you configure it to use page flags and your build does > not have enough free bits, the profiling will be disabled (see > alloc_tag_sec_init()). Is it possible to fail the build in that case i.e. check the page flags availability at build time?
On 16/04/2025 22:08, Suren Baghdasaryan wrote: > On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: >> >> With this Kconfig option enabled, the kernel stores allocation tag references >> in the page flags by default. >> >> There are 2 reasons to introduce this: >> - As mentioned in [1], compressed tags dont have system memory overhead >> and much lower performance overhead. It would be preferrable to have this as >> the default option, and to be able to switch it at compile time. Another >> option is to just declare the static key as true by default? >> - As compressed option is the best one, it doesn't make sense to have to >> change both defconfig and command line options to enable memory >> allocation profiling. Changing commandline across a large number of services >> can result in signifcant work, which shouldn't be needed if the kernel >> defconfig needs to be changed anyways. > > The reason tag compression is not the default option is because it > works only if there are enough free bits in the page flags to store a > tag index. If you configure it to use page flags and your build does > not have enough free bits, the profiling will be disabled (see > alloc_tag_sec_init()). IOW there is no graceful fallback to use page > extensions. Therefore, the current default option is not the most > performant but the one which works on all builds. Instead of this just > set sysctl.vm.mem_profiling boot parameter in your config file. Hi Suren, Thanks for the review! The main reason is to not have to make a change in both defconfig and kernel command line while deploying it. We can ofcourse set the commandline as well, but just makes deployment more tedious, and adds an extra commandline parameter. In our case, we only want to deploy compressed tags, and if there aren't enough free bits, we would prefer to disable memory allocation profiling than to take the memory and performance hit. Would keeping the default value of this config disabled be an acceptable option? i.e. the below diff on top of this patch? diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 66d8995f3514..163ffcece47a 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1030,7 +1030,7 @@ config MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT config MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT bool "store page allocation tag references in the page flags by default" - default y + default n depends on MEM_ALLOC_PROFILING config MEM_ALLOC_PROFILING_DEBUG Thanks, Usama > > Your change effectively changes the default value of > mem_profiling_compressed and I don't see why you need to introduce a > new config option for that. But that really does not matter because > changing default to compressed tags is not the right choice IMO. > >> >> [1] https://lore.kernel.org/all/20241023170759.999909-7-surenb@google.com/T/#m0da08879435f7673eaa10871a6e9d1be4f605ac8 >> >> Signed-off-by: Usama Arif <usamaarif642@gmail.com> >> --- >> include/linux/pgalloc_tag.h | 4 ++++ >> lib/Kconfig.debug | 5 +++++ >> lib/alloc_tag.c | 4 ++++ >> 3 files changed, 13 insertions(+) >> >> diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h >> index c74077977830..0226059bcf00 100644 >> --- a/include/linux/pgalloc_tag.h >> +++ b/include/linux/pgalloc_tag.h >> @@ -16,7 +16,11 @@ extern unsigned long alloc_tag_ref_mask; >> extern int alloc_tag_ref_offs; >> extern struct alloc_tag_kernel_section kernel_tags; >> >> +#ifdef CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT >> +DECLARE_STATIC_KEY_TRUE(mem_profiling_compressed); >> +#else >> DECLARE_STATIC_KEY_FALSE(mem_profiling_compressed); >> +#endif >> >> typedef u16 pgalloc_tag_idx; >> >> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug >> index 9fe4d8dfe578..66d8995f3514 100644 >> --- a/lib/Kconfig.debug >> +++ b/lib/Kconfig.debug >> @@ -1028,6 +1028,11 @@ config MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT >> default y >> depends on MEM_ALLOC_PROFILING >> >> +config MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT >> + bool "store page allocation tag references in the page flags by default" >> + default y >> + depends on MEM_ALLOC_PROFILING >> + >> config MEM_ALLOC_PROFILING_DEBUG >> bool "Memory allocation profiler debugging" >> default n >> diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c >> index 25ecc1334b67..30adad5630dd 100644 >> --- a/lib/alloc_tag.c >> +++ b/lib/alloc_tag.c >> @@ -31,7 +31,11 @@ DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT, >> mem_alloc_profiling_key); >> EXPORT_SYMBOL(mem_alloc_profiling_key); >> >> +#ifdef CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT >> +DEFINE_STATIC_KEY_TRUE(mem_profiling_compressed); >> +#else >> DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed); >> +#endif >> >> struct alloc_tag_kernel_section kernel_tags = { NULL, 0 }; >> unsigned long alloc_tag_ref_mask; >> -- >> 2.47.1 >>
On Wed, Apr 16, 2025 at 2:41 PM Shakeel Butt <shakeel.butt@linux.dev> wrote: > > On Wed, Apr 16, 2025 at 02:08:31PM -0700, Suren Baghdasaryan wrote: > > On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: > > > > > > With this Kconfig option enabled, the kernel stores allocation tag references > > > in the page flags by default. > > > > > > There are 2 reasons to introduce this: > > > - As mentioned in [1], compressed tags dont have system memory overhead > > > and much lower performance overhead. It would be preferrable to have this as > > > the default option, and to be able to switch it at compile time. Another > > > option is to just declare the static key as true by default? > > > - As compressed option is the best one, it doesn't make sense to have to > > > change both defconfig and command line options to enable memory > > > allocation profiling. Changing commandline across a large number of services > > > can result in signifcant work, which shouldn't be needed if the kernel > > > defconfig needs to be changed anyways. > > > > The reason tag compression is not the default option is because it > > works only if there are enough free bits in the page flags to store a > > tag index. If you configure it to use page flags and your build does > > not have enough free bits, the profiling will be disabled (see > > alloc_tag_sec_init()). > > Is it possible to fail the build in that case i.e. check the page flags > availability at build time? The difficulty is finding out the number of allocation tags in the kernel before it gets built. Maybe there is a way to add an additional post-build stage to run that check. But even then making this option default and causing build failures does not seem like a good idea to me but maybe I'm being too cautious?
On Wed, Apr 16, 2025 at 2:52 PM Usama Arif <usamaarif642@gmail.com> wrote: > > > > On 16/04/2025 22:08, Suren Baghdasaryan wrote: > > On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: > >> > >> With this Kconfig option enabled, the kernel stores allocation tag references > >> in the page flags by default. > >> > >> There are 2 reasons to introduce this: > >> - As mentioned in [1], compressed tags dont have system memory overhead > >> and much lower performance overhead. It would be preferrable to have this as > >> the default option, and to be able to switch it at compile time. Another > >> option is to just declare the static key as true by default? > >> - As compressed option is the best one, it doesn't make sense to have to > >> change both defconfig and command line options to enable memory > >> allocation profiling. Changing commandline across a large number of services > >> can result in signifcant work, which shouldn't be needed if the kernel > >> defconfig needs to be changed anyways. > > > > The reason tag compression is not the default option is because it > > works only if there are enough free bits in the page flags to store a > > tag index. If you configure it to use page flags and your build does > > not have enough free bits, the profiling will be disabled (see > > alloc_tag_sec_init()). IOW there is no graceful fallback to use page > > extensions. Therefore, the current default option is not the most > > performant but the one which works on all builds. Instead of this just > > set sysctl.vm.mem_profiling boot parameter in your config file. > > Hi Suren, Hi Usama, > > Thanks for the review! The main reason is to not have to make a change in > both defconfig and kernel command line while deploying it. We can ofcourse > set the commandline as well, but just makes deployment more tedious, and > adds an extra commandline parameter. In our case, we only want to deploy > compressed tags, and if there aren't enough free bits, we would prefer to > disable memory allocation profiling than to take the memory and performance > hit. > > Would keeping the default value of this config disabled be an acceptable option? > i.e. the below diff on top of this patch? Well, in that case I fail to see why CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT=y is better than CONFIG_CMDLINE="sysctl.vm.mem_profiling=1,compressed" ? Either way you need to change the config file, no? > > diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > index 66d8995f3514..163ffcece47a 100644 > --- a/lib/Kconfig.debug > +++ b/lib/Kconfig.debug > @@ -1030,7 +1030,7 @@ config MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT > > config MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT > bool "store page allocation tag references in the page flags by default" > - default y > + default n > depends on MEM_ALLOC_PROFILING > > config MEM_ALLOC_PROFILING_DEBUG > > > Thanks, > Usama > > > > Your change effectively changes the default value of > > mem_profiling_compressed and I don't see why you need to introduce a > > new config option for that. But that really does not matter because > > changing default to compressed tags is not the right choice IMO. > > > >> > >> [1] https://lore.kernel.org/all/20241023170759.999909-7-surenb@google.com/T/#m0da08879435f7673eaa10871a6e9d1be4f605ac8 > >> > >> Signed-off-by: Usama Arif <usamaarif642@gmail.com> > >> --- > >> include/linux/pgalloc_tag.h | 4 ++++ > >> lib/Kconfig.debug | 5 +++++ > >> lib/alloc_tag.c | 4 ++++ > >> 3 files changed, 13 insertions(+) > >> > >> diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h > >> index c74077977830..0226059bcf00 100644 > >> --- a/include/linux/pgalloc_tag.h > >> +++ b/include/linux/pgalloc_tag.h > >> @@ -16,7 +16,11 @@ extern unsigned long alloc_tag_ref_mask; > >> extern int alloc_tag_ref_offs; > >> extern struct alloc_tag_kernel_section kernel_tags; > >> > >> +#ifdef CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT > >> +DECLARE_STATIC_KEY_TRUE(mem_profiling_compressed); > >> +#else > >> DECLARE_STATIC_KEY_FALSE(mem_profiling_compressed); > >> +#endif > >> > >> typedef u16 pgalloc_tag_idx; > >> > >> diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug > >> index 9fe4d8dfe578..66d8995f3514 100644 > >> --- a/lib/Kconfig.debug > >> +++ b/lib/Kconfig.debug > >> @@ -1028,6 +1028,11 @@ config MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT > >> default y > >> depends on MEM_ALLOC_PROFILING > >> > >> +config MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT > >> + bool "store page allocation tag references in the page flags by default" > >> + default y > >> + depends on MEM_ALLOC_PROFILING > >> + > >> config MEM_ALLOC_PROFILING_DEBUG > >> bool "Memory allocation profiler debugging" > >> default n > >> diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c > >> index 25ecc1334b67..30adad5630dd 100644 > >> --- a/lib/alloc_tag.c > >> +++ b/lib/alloc_tag.c > >> @@ -31,7 +31,11 @@ DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT, > >> mem_alloc_profiling_key); > >> EXPORT_SYMBOL(mem_alloc_profiling_key); > >> > >> +#ifdef CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT > >> +DEFINE_STATIC_KEY_TRUE(mem_profiling_compressed); > >> +#else > >> DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed); > >> +#endif > >> > >> struct alloc_tag_kernel_section kernel_tags = { NULL, 0 }; > >> unsigned long alloc_tag_ref_mask; > >> -- > >> 2.47.1 > >> >
On 17/04/2025 01:15, Suren Baghdasaryan wrote: > On Wed, Apr 16, 2025 at 2:52 PM Usama Arif <usamaarif642@gmail.com> wrote: >> >> >> >> On 16/04/2025 22:08, Suren Baghdasaryan wrote: >>> On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: >>>> >>>> With this Kconfig option enabled, the kernel stores allocation tag references >>>> in the page flags by default. >>>> >>>> There are 2 reasons to introduce this: >>>> - As mentioned in [1], compressed tags dont have system memory overhead >>>> and much lower performance overhead. It would be preferrable to have this as >>>> the default option, and to be able to switch it at compile time. Another >>>> option is to just declare the static key as true by default? >>>> - As compressed option is the best one, it doesn't make sense to have to >>>> change both defconfig and command line options to enable memory >>>> allocation profiling. Changing commandline across a large number of services >>>> can result in signifcant work, which shouldn't be needed if the kernel >>>> defconfig needs to be changed anyways. >>> >>> The reason tag compression is not the default option is because it >>> works only if there are enough free bits in the page flags to store a >>> tag index. If you configure it to use page flags and your build does >>> not have enough free bits, the profiling will be disabled (see >>> alloc_tag_sec_init()). IOW there is no graceful fallback to use page >>> extensions. Therefore, the current default option is not the most >>> performant but the one which works on all builds. Instead of this just >>> set sysctl.vm.mem_profiling boot parameter in your config file. >> >> Hi Suren, > > Hi Usama, > >> >> Thanks for the review! The main reason is to not have to make a change in >> both defconfig and kernel command line while deploying it. We can ofcourse >> set the commandline as well, but just makes deployment more tedious, and >> adds an extra commandline parameter. In our case, we only want to deploy >> compressed tags, and if there aren't enough free bits, we would prefer to >> disable memory allocation profiling than to take the memory and performance >> hit. >> >> Would keeping the default value of this config disabled be an acceptable option? >> i.e. the below diff on top of this patch? > > Well, in that case I fail to see why > CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT=y is better > than CONFIG_CMDLINE="sysctl.vm.mem_profiling=1,compressed" ? Either > way you need to change the config file, no? > Ah ok, I always thought CONFIG_CMDLINE overrides what the kernel gets from the bootloader, but I just tried it and it appends, as long as CONFIG_CMDLINE_OVERRIDE is not set. Your above suggestion works as well. Thanks!
On Wed, Apr 16, 2025 at 05:11:11PM -0700, Suren Baghdasaryan wrote: > On Wed, Apr 16, 2025 at 2:41 PM Shakeel Butt <shakeel.butt@linux.dev> wrote: > > > > On Wed, Apr 16, 2025 at 02:08:31PM -0700, Suren Baghdasaryan wrote: > > > On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: > > > > > > > > With this Kconfig option enabled, the kernel stores allocation tag references > > > > in the page flags by default. > > > > > > > > There are 2 reasons to introduce this: > > > > - As mentioned in [1], compressed tags dont have system memory overhead > > > > and much lower performance overhead. It would be preferrable to have this as > > > > the default option, and to be able to switch it at compile time. Another > > > > option is to just declare the static key as true by default? > > > > - As compressed option is the best one, it doesn't make sense to have to > > > > change both defconfig and command line options to enable memory > > > > allocation profiling. Changing commandline across a large number of services > > > > can result in signifcant work, which shouldn't be needed if the kernel > > > > defconfig needs to be changed anyways. > > > > > > The reason tag compression is not the default option is because it > > > works only if there are enough free bits in the page flags to store a > > > tag index. If you configure it to use page flags and your build does > > > not have enough free bits, the profiling will be disabled (see > > > alloc_tag_sec_init()). > > > > Is it possible to fail the build in that case i.e. check the page flags > > availability at build time? > > The difficulty is finding out the number of allocation tags in the > kernel before it gets built. Maybe there is a way to add an additional > post-build stage to run that check. Yeah that would be good to have. > But even then making this option > default and causing build failures does not seem like a good idea to > me but maybe I'm being too cautious? Oh my question was orthogonal to the patch. Basically some users may want build time guarantee for this and they can enable such build-failing opt-in config/check.
On Thu, Apr 17, 2025 at 8:47 AM Shakeel Butt <shakeel.butt@linux.dev> wrote: > > On Wed, Apr 16, 2025 at 05:11:11PM -0700, Suren Baghdasaryan wrote: > > On Wed, Apr 16, 2025 at 2:41 PM Shakeel Butt <shakeel.butt@linux.dev> wrote: > > > > > > On Wed, Apr 16, 2025 at 02:08:31PM -0700, Suren Baghdasaryan wrote: > > > > On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: > > > > > > > > > > With this Kconfig option enabled, the kernel stores allocation tag references > > > > > in the page flags by default. > > > > > > > > > > There are 2 reasons to introduce this: > > > > > - As mentioned in [1], compressed tags dont have system memory overhead > > > > > and much lower performance overhead. It would be preferrable to have this as > > > > > the default option, and to be able to switch it at compile time. Another > > > > > option is to just declare the static key as true by default? > > > > > - As compressed option is the best one, it doesn't make sense to have to > > > > > change both defconfig and command line options to enable memory > > > > > allocation profiling. Changing commandline across a large number of services > > > > > can result in signifcant work, which shouldn't be needed if the kernel > > > > > defconfig needs to be changed anyways. > > > > > > > > The reason tag compression is not the default option is because it > > > > works only if there are enough free bits in the page flags to store a > > > > tag index. If you configure it to use page flags and your build does > > > > not have enough free bits, the profiling will be disabled (see > > > > alloc_tag_sec_init()). > > > > > > Is it possible to fail the build in that case i.e. check the page flags > > > availability at build time? > > > > The difficulty is finding out the number of allocation tags in the > > kernel before it gets built. Maybe there is a way to add an additional > > post-build stage to run that check. > > Yeah that would be good to have. > > > But even then making this option > > default and causing build failures does not seem like a good idea to > > me but maybe I'm being too cautious? > > Oh my question was orthogonal to the patch. Basically some users may > want build time guarantee for this and they can enable such > build-failing opt-in config/check. Yes, that would require the post-build step to check the number of tags vs the number of available page flag bits. I'll add it to my TODO list but it won't be at the top, sorry :) Volunteers to help with that would be highly appreciated.
On 17/04/2025 17:00, Suren Baghdasaryan wrote: > On Thu, Apr 17, 2025 at 8:47 AM Shakeel Butt <shakeel.butt@linux.dev> wrote: >> >> On Wed, Apr 16, 2025 at 05:11:11PM -0700, Suren Baghdasaryan wrote: >>> On Wed, Apr 16, 2025 at 2:41 PM Shakeel Butt <shakeel.butt@linux.dev> wrote: >>>> >>>> On Wed, Apr 16, 2025 at 02:08:31PM -0700, Suren Baghdasaryan wrote: >>>>> On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: >>>>>> >>>>>> With this Kconfig option enabled, the kernel stores allocation tag references >>>>>> in the page flags by default. >>>>>> >>>>>> There are 2 reasons to introduce this: >>>>>> - As mentioned in [1], compressed tags dont have system memory overhead >>>>>> and much lower performance overhead. It would be preferrable to have this as >>>>>> the default option, and to be able to switch it at compile time. Another >>>>>> option is to just declare the static key as true by default? >>>>>> - As compressed option is the best one, it doesn't make sense to have to >>>>>> change both defconfig and command line options to enable memory >>>>>> allocation profiling. Changing commandline across a large number of services >>>>>> can result in signifcant work, which shouldn't be needed if the kernel >>>>>> defconfig needs to be changed anyways. >>>>> >>>>> The reason tag compression is not the default option is because it >>>>> works only if there are enough free bits in the page flags to store a >>>>> tag index. If you configure it to use page flags and your build does >>>>> not have enough free bits, the profiling will be disabled (see >>>>> alloc_tag_sec_init()). >>>> >>>> Is it possible to fail the build in that case i.e. check the page flags >>>> availability at build time? >>> >>> The difficulty is finding out the number of allocation tags in the >>> kernel before it gets built. Maybe there is a way to add an additional >>> post-build stage to run that check. >> >> Yeah that would be good to have. >> >>> But even then making this option >>> default and causing build failures does not seem like a good idea to >>> me but maybe I'm being too cautious? >> >> Oh my question was orthogonal to the patch. Basically some users may >> want build time guarantee for this and they can enable such >> build-failing opt-in config/check. > > Yes, that would require the post-build step to check the number of > tags vs the number of available page flag bits. I'll add it to my TODO > list but it won't be at the top, sorry :) Volunteers to help with that > would be highly appreciated. Hi Suren, A question orthogonal to the patch, the defconfig entry is defined as below: config MEM_ALLOC_PROFILING bool "Enable memory allocation profiling" default n depends on MMU depends on PROC_FS depends on !DEBUG_FORCE_WEAK_PER_CPU select CODE_TAGGING select PAGE_EXTENSION select SLAB_OBJ_EXT i.e. we select PAGE_EXTENSION even if we use compressed profiling and use page flags instead of page extension. Which means the 0.2% (8 bytes per struct page) memory overhead will still exist even when we dont need it? Should we have some defconfig option (happy with any other way) that only allows compressed profiling (otherwise nothing), so that we don't have the dependency on page extension and thus not have the overhead if we only plan to use compressed profiling? Thanks
On 17/04/2025 18:50, Usama Arif wrote: > > > On 17/04/2025 17:00, Suren Baghdasaryan wrote: >> On Thu, Apr 17, 2025 at 8:47 AM Shakeel Butt <shakeel.butt@linux.dev> wrote: >>> >>> On Wed, Apr 16, 2025 at 05:11:11PM -0700, Suren Baghdasaryan wrote: >>>> On Wed, Apr 16, 2025 at 2:41 PM Shakeel Butt <shakeel.butt@linux.dev> wrote: >>>>> >>>>> On Wed, Apr 16, 2025 at 02:08:31PM -0700, Suren Baghdasaryan wrote: >>>>>> On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: >>>>>>> >>>>>>> With this Kconfig option enabled, the kernel stores allocation tag references >>>>>>> in the page flags by default. >>>>>>> >>>>>>> There are 2 reasons to introduce this: >>>>>>> - As mentioned in [1], compressed tags dont have system memory overhead >>>>>>> and much lower performance overhead. It would be preferrable to have this as >>>>>>> the default option, and to be able to switch it at compile time. Another >>>>>>> option is to just declare the static key as true by default? >>>>>>> - As compressed option is the best one, it doesn't make sense to have to >>>>>>> change both defconfig and command line options to enable memory >>>>>>> allocation profiling. Changing commandline across a large number of services >>>>>>> can result in signifcant work, which shouldn't be needed if the kernel >>>>>>> defconfig needs to be changed anyways. >>>>>> >>>>>> The reason tag compression is not the default option is because it >>>>>> works only if there are enough free bits in the page flags to store a >>>>>> tag index. If you configure it to use page flags and your build does >>>>>> not have enough free bits, the profiling will be disabled (see >>>>>> alloc_tag_sec_init()). >>>>> >>>>> Is it possible to fail the build in that case i.e. check the page flags >>>>> availability at build time? >>>> >>>> The difficulty is finding out the number of allocation tags in the >>>> kernel before it gets built. Maybe there is a way to add an additional >>>> post-build stage to run that check. >>> >>> Yeah that would be good to have. >>> >>>> But even then making this option >>>> default and causing build failures does not seem like a good idea to >>>> me but maybe I'm being too cautious? >>> >>> Oh my question was orthogonal to the patch. Basically some users may >>> want build time guarantee for this and they can enable such >>> build-failing opt-in config/check. >> >> Yes, that would require the post-build step to check the number of >> tags vs the number of available page flag bits. I'll add it to my TODO >> list but it won't be at the top, sorry :) Volunteers to help with that >> would be highly appreciated. > > Hi Suren, > > A question orthogonal to the patch, the defconfig entry is defined as below: > > config MEM_ALLOC_PROFILING > bool "Enable memory allocation profiling" > default n > depends on MMU > depends on PROC_FS > depends on !DEBUG_FORCE_WEAK_PER_CPU > select CODE_TAGGING > select PAGE_EXTENSION > select SLAB_OBJ_EXT > > i.e. we select PAGE_EXTENSION even if we use compressed profiling and use page flags > instead of page extension. Which means the 0.2% (8 bytes per struct page) memory overhead > will still exist even when we dont need it? > > Should we have some defconfig option (happy with any other way) that only allows compressed > profiling (otherwise nothing), so that we don't have the dependency on page extension > and thus not have the overhead if we only plan to use compressed profiling? > > Thanks Johannes pointed out the .need function of page_alloc_tagging_ops, i.e. need_page_alloc_tagging, so page extensions wouldn't be enabled and hopefully there is no memory overhead when mem_profiling_compressed is true. Let me know if my understanding is wrong. And sorry for the noise!
On Thu, Apr 17, 2025 at 11:35 AM Usama Arif <usamaarif642@gmail.com> wrote: > > > > On 17/04/2025 18:50, Usama Arif wrote: > > > > > > On 17/04/2025 17:00, Suren Baghdasaryan wrote: > >> On Thu, Apr 17, 2025 at 8:47 AM Shakeel Butt <shakeel.butt@linux.dev> wrote: > >>> > >>> On Wed, Apr 16, 2025 at 05:11:11PM -0700, Suren Baghdasaryan wrote: > >>>> On Wed, Apr 16, 2025 at 2:41 PM Shakeel Butt <shakeel.butt@linux.dev> wrote: > >>>>> > >>>>> On Wed, Apr 16, 2025 at 02:08:31PM -0700, Suren Baghdasaryan wrote: > >>>>>> On Wed, Apr 16, 2025 at 11:06 AM Usama Arif <usamaarif642@gmail.com> wrote: > >>>>>>> > >>>>>>> With this Kconfig option enabled, the kernel stores allocation tag references > >>>>>>> in the page flags by default. > >>>>>>> > >>>>>>> There are 2 reasons to introduce this: > >>>>>>> - As mentioned in [1], compressed tags dont have system memory overhead > >>>>>>> and much lower performance overhead. It would be preferrable to have this as > >>>>>>> the default option, and to be able to switch it at compile time. Another > >>>>>>> option is to just declare the static key as true by default? > >>>>>>> - As compressed option is the best one, it doesn't make sense to have to > >>>>>>> change both defconfig and command line options to enable memory > >>>>>>> allocation profiling. Changing commandline across a large number of services > >>>>>>> can result in signifcant work, which shouldn't be needed if the kernel > >>>>>>> defconfig needs to be changed anyways. > >>>>>> > >>>>>> The reason tag compression is not the default option is because it > >>>>>> works only if there are enough free bits in the page flags to store a > >>>>>> tag index. If you configure it to use page flags and your build does > >>>>>> not have enough free bits, the profiling will be disabled (see > >>>>>> alloc_tag_sec_init()). > >>>>> > >>>>> Is it possible to fail the build in that case i.e. check the page flags > >>>>> availability at build time? > >>>> > >>>> The difficulty is finding out the number of allocation tags in the > >>>> kernel before it gets built. Maybe there is a way to add an additional > >>>> post-build stage to run that check. > >>> > >>> Yeah that would be good to have. > >>> > >>>> But even then making this option > >>>> default and causing build failures does not seem like a good idea to > >>>> me but maybe I'm being too cautious? > >>> > >>> Oh my question was orthogonal to the patch. Basically some users may > >>> want build time guarantee for this and they can enable such > >>> build-failing opt-in config/check. > >> > >> Yes, that would require the post-build step to check the number of > >> tags vs the number of available page flag bits. I'll add it to my TODO > >> list but it won't be at the top, sorry :) Volunteers to help with that > >> would be highly appreciated. > > > > Hi Suren, > > > > A question orthogonal to the patch, the defconfig entry is defined as below: > > > > config MEM_ALLOC_PROFILING > > bool "Enable memory allocation profiling" > > default n > > depends on MMU > > depends on PROC_FS > > depends on !DEBUG_FORCE_WEAK_PER_CPU > > select CODE_TAGGING > > select PAGE_EXTENSION > > select SLAB_OBJ_EXT > > > > i.e. we select PAGE_EXTENSION even if we use compressed profiling and use page flags > > instead of page extension. Which means the 0.2% (8 bytes per struct page) memory overhead > > will still exist even when we dont need it? > > > > Should we have some defconfig option (happy with any other way) that only allows compressed > > profiling (otherwise nothing), so that we don't have the dependency on page extension > > and thus not have the overhead if we only plan to use compressed profiling? > > > > Thanks > > > Johannes pointed out the .need function of page_alloc_tagging_ops, i.e. need_page_alloc_tagging, > so page extensions wouldn't be enabled and hopefully there is no memory overhead when > mem_profiling_compressed is true. Let me know if my understanding is wrong. > And sorry for the noise! Sorry for the delay. Meetings... Yep, Johannes is right, need_page_alloc_tagging() should disable that overhead.
diff --git a/include/linux/pgalloc_tag.h b/include/linux/pgalloc_tag.h index c74077977830..0226059bcf00 100644 --- a/include/linux/pgalloc_tag.h +++ b/include/linux/pgalloc_tag.h @@ -16,7 +16,11 @@ extern unsigned long alloc_tag_ref_mask; extern int alloc_tag_ref_offs; extern struct alloc_tag_kernel_section kernel_tags; +#ifdef CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT +DECLARE_STATIC_KEY_TRUE(mem_profiling_compressed); +#else DECLARE_STATIC_KEY_FALSE(mem_profiling_compressed); +#endif typedef u16 pgalloc_tag_idx; diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug index 9fe4d8dfe578..66d8995f3514 100644 --- a/lib/Kconfig.debug +++ b/lib/Kconfig.debug @@ -1028,6 +1028,11 @@ config MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT default y depends on MEM_ALLOC_PROFILING +config MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT + bool "store page allocation tag references in the page flags by default" + default y + depends on MEM_ALLOC_PROFILING + config MEM_ALLOC_PROFILING_DEBUG bool "Memory allocation profiler debugging" default n diff --git a/lib/alloc_tag.c b/lib/alloc_tag.c index 25ecc1334b67..30adad5630dd 100644 --- a/lib/alloc_tag.c +++ b/lib/alloc_tag.c @@ -31,7 +31,11 @@ DEFINE_STATIC_KEY_MAYBE(CONFIG_MEM_ALLOC_PROFILING_ENABLED_BY_DEFAULT, mem_alloc_profiling_key); EXPORT_SYMBOL(mem_alloc_profiling_key); +#ifdef CONFIG_MEM_ALLOC_PROFILING_COMPRESSED_ENABLED_BY_DEFAULT +DEFINE_STATIC_KEY_TRUE(mem_profiling_compressed); +#else DEFINE_STATIC_KEY_FALSE(mem_profiling_compressed); +#endif struct alloc_tag_kernel_section kernel_tags = { NULL, 0 }; unsigned long alloc_tag_ref_mask;
With this Kconfig option enabled, the kernel stores allocation tag references in the page flags by default. There are 2 reasons to introduce this: - As mentioned in [1], compressed tags dont have system memory overhead and much lower performance overhead. It would be preferrable to have this as the default option, and to be able to switch it at compile time. Another option is to just declare the static key as true by default? - As compressed option is the best one, it doesn't make sense to have to change both defconfig and command line options to enable memory allocation profiling. Changing commandline across a large number of services can result in signifcant work, which shouldn't be needed if the kernel defconfig needs to be changed anyways. [1] https://lore.kernel.org/all/20241023170759.999909-7-surenb@google.com/T/#m0da08879435f7673eaa10871a6e9d1be4f605ac8 Signed-off-by: Usama Arif <usamaarif642@gmail.com> --- include/linux/pgalloc_tag.h | 4 ++++ lib/Kconfig.debug | 5 +++++ lib/alloc_tag.c | 4 ++++ 3 files changed, 13 insertions(+)