| Message ID | 20231204163254.2636289-1-dmaluka@chromium.org (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | mm/thp: add CONFIG_TRANSPARENT_HUGEPAGE_NEVER option | expand |
On Mon, 4 Dec 2023 17:32:54 +0100 Dmytro Maluka <dmaluka@chromium.org> wrote: > Add an option to disable transparent hugepages by default, in line with > the existing transparent_hugepage=never command line setting. > > Rationale: khugepaged has its own non-negligible memory cost even if it > is not used by any applications, since it bumps up vm.min_free_kbytes to > its own required minimum in set_recommended_min_free_kbytes(). For > example, on a machine with 4GB RAM, with 3 mm zones and pageblock_order > == MAX_ORDER, starting khugepaged causes vm.min_free_kbytes increase > from 8MB to 132MB. > > So if we use THP on machines with e.g. >=8GB of memory for better > performance, but avoid using it on lower-memory machines to avoid its > memory overhead, then for the same reason we also want to avoid even > starting khugepaged on those <8GB machines. So with > CONFIG_TRANSPARENT_HUGEPAGE_NEVER we can use the same kernel image on > both >=8GB and <8GB machines, with THP support enabled but khugepaged > not started by default. The userspace can then decide to enable THP > (i.e. start khugepaged) via sysfs if needed, based on the total amount > of memory. > > This could also be achieved with the existing transparent_hugepage=never > setting in the kernel command line instead. But it seems cleaner to > avoid tweaking the command line for such a basic setting. > > P.S. I see that CONFIG_TRANSPARENT_HUGEPAGE_NEVER was already proposed > in the past [1] but without an explanation of the purpose. > > ... > > --- a/mm/Kconfig > +++ b/mm/Kconfig > @@ -859,6 +859,12 @@ choice > madvise(MADV_HUGEPAGE) but it won't risk to increase the > memory footprint of applications without a guaranteed > benefit. > + > + config TRANSPARENT_HUGEPAGE_NEVER > + bool "never" > + help > + Disabling Transparent Hugepage by default. It can still be s/Disabling/Disable/ > + enabled at runtime via sysfs. > endchoice The patch adds the config option but doesn't use it?
On Mon, Dec 04, 2023 at 11:13:01AM -0800, Andrew Morton wrote: > On Mon, 4 Dec 2023 17:32:54 +0100 Dmytro Maluka <dmaluka@chromium.org> wrote: > > > Add an option to disable transparent hugepages by default, in line with > > the existing transparent_hugepage=never command line setting. > > > > Rationale: khugepaged has its own non-negligible memory cost even if it > > is not used by any applications, since it bumps up vm.min_free_kbytes to > > its own required minimum in set_recommended_min_free_kbytes(). For > > example, on a machine with 4GB RAM, with 3 mm zones and pageblock_order > > == MAX_ORDER, starting khugepaged causes vm.min_free_kbytes increase > > from 8MB to 132MB. > > > > So if we use THP on machines with e.g. >=8GB of memory for better > > performance, but avoid using it on lower-memory machines to avoid its > > memory overhead, then for the same reason we also want to avoid even > > starting khugepaged on those <8GB machines. So with > > CONFIG_TRANSPARENT_HUGEPAGE_NEVER we can use the same kernel image on > > both >=8GB and <8GB machines, with THP support enabled but khugepaged > > not started by default. The userspace can then decide to enable THP > > (i.e. start khugepaged) via sysfs if needed, based on the total amount > > of memory. > > > > This could also be achieved with the existing transparent_hugepage=never > > setting in the kernel command line instead. But it seems cleaner to > > avoid tweaking the command line for such a basic setting. > > > > P.S. I see that CONFIG_TRANSPARENT_HUGEPAGE_NEVER was already proposed > > in the past [1] but without an explanation of the purpose. > > > > ... > > > > --- a/mm/Kconfig > > +++ b/mm/Kconfig > > @@ -859,6 +859,12 @@ choice > > madvise(MADV_HUGEPAGE) but it won't risk to increase the > > memory footprint of applications without a guaranteed > > benefit. > > + > > + config TRANSPARENT_HUGEPAGE_NEVER > > + bool "never" > > + help > > + Disabling Transparent Hugepage by default. It can still be > > s/Disabling/Disable/ It is in line with the descriptions of TRANSPARENT_HUGEPAGE_ALWAYS and TRANSPARENT_HUGEPAGE_MADVISE: "Enabling Transparent Hugepage ..." > > + enabled at runtime via sysfs. > > endchoice > > The patch adds the config option but doesn't use it? I should have been more precise: it is not a new option but a new choice for CONFIG_TRANSPARENT_HUGEPAGE, in addition to the existing ALWAYS and MADVISE choices. In mm/huge_memory.c in the declaration of the transparent_hugepage_flags variable, if either ALWAYS or MADVISE is chosen, transparent_hugepage_flags is initialized with such a value that makes khugepaged being started by default during bootup. This patch allows enabling CONFIG_TRANSPARENT_HUGEPAGE without setting either ALWAYS or MADVISE, so that transparent_hugepage_flags is initialized with such a value that khugepaged is not started by default.
On Mon, 4 Dec 2023 20:57:33 +0100 Dmytro Maluka <dmaluka@chromium.org> wrote: > On Mon, Dec 04, 2023 at 11:13:01AM -0800, Andrew Morton wrote: > > On Mon, 4 Dec 2023 17:32:54 +0100 Dmytro Maluka <dmaluka@chromium.org> wrote: > > > > > Add an option to disable transparent hugepages by default, in line with > > > the existing transparent_hugepage=never command line setting. > > > > > > Rationale: khugepaged has its own non-negligible memory cost even if it > > > is not used by any applications, since it bumps up vm.min_free_kbytes to > > > its own required minimum in set_recommended_min_free_kbytes(). For > > > example, on a machine with 4GB RAM, with 3 mm zones and pageblock_order > > > == MAX_ORDER, starting khugepaged causes vm.min_free_kbytes increase > > > from 8MB to 132MB. > > > > > > So if we use THP on machines with e.g. >=8GB of memory for better > > > performance, but avoid using it on lower-memory machines to avoid its > > > memory overhead, then for the same reason we also want to avoid even > > > starting khugepaged on those <8GB machines. So with > > > CONFIG_TRANSPARENT_HUGEPAGE_NEVER we can use the same kernel image on > > > both >=8GB and <8GB machines, with THP support enabled but khugepaged > > > not started by default. The userspace can then decide to enable THP > > > (i.e. start khugepaged) via sysfs if needed, based on the total amount > > > of memory. > > > > > > This could also be achieved with the existing transparent_hugepage=never > > > setting in the kernel command line instead. But it seems cleaner to > > > avoid tweaking the command line for such a basic setting. > > > > > > P.S. I see that CONFIG_TRANSPARENT_HUGEPAGE_NEVER was already proposed > > > in the past [1] but without an explanation of the purpose. > > > > > > ... > > > > > > --- a/mm/Kconfig > > > +++ b/mm/Kconfig > > > @@ -859,6 +859,12 @@ choice > > > madvise(MADV_HUGEPAGE) but it won't risk to increase the > > > memory footprint of applications without a guaranteed > > > benefit. > > > + > > > + config TRANSPARENT_HUGEPAGE_NEVER > > > + bool "never" > > > + help > > > + Disabling Transparent Hugepage by default. It can still be > > > > s/Disabling/Disable/ > > It is in line with the descriptions of TRANSPARENT_HUGEPAGE_ALWAYS and > TRANSPARENT_HUGEPAGE_MADVISE: "Enabling Transparent Hugepage ..." Those are incorrect also. > > > + enabled at runtime via sysfs. > > > endchoice > > > > The patch adds the config option but doesn't use it? > > I should have been more precise: it is not a new option but a new choice > for CONFIG_TRANSPARENT_HUGEPAGE, in addition to the existing ALWAYS and > MADVISE choices. In mm/huge_memory.c in the declaration of the > transparent_hugepage_flags variable, if either ALWAYS or MADVISE is > chosen, transparent_hugepage_flags is initialized with such a value > that makes khugepaged being started by default during bootup. This patch > allows enabling CONFIG_TRANSPARENT_HUGEPAGE without setting either > ALWAYS or MADVISE, so that transparent_hugepage_flags is initialized > with such a value that khugepaged is not started by default. OK, thanks.
On Mon, Dec 04, 2023 at 12:15:24PM -0800, Andrew Morton wrote: > On Mon, 4 Dec 2023 20:57:33 +0100 Dmytro Maluka <dmaluka@chromium.org> wrote: > > > On Mon, Dec 04, 2023 at 11:13:01AM -0800, Andrew Morton wrote: > > > On Mon, 4 Dec 2023 17:32:54 +0100 Dmytro Maluka <dmaluka@chromium.org> wrote: > > > > > > > Add an option to disable transparent hugepages by default, in line with > > > > the existing transparent_hugepage=never command line setting. > > > > > > > > Rationale: khugepaged has its own non-negligible memory cost even if it > > > > is not used by any applications, since it bumps up vm.min_free_kbytes to > > > > its own required minimum in set_recommended_min_free_kbytes(). For > > > > example, on a machine with 4GB RAM, with 3 mm zones and pageblock_order > > > > == MAX_ORDER, starting khugepaged causes vm.min_free_kbytes increase > > > > from 8MB to 132MB. > > > > > > > > So if we use THP on machines with e.g. >=8GB of memory for better > > > > performance, but avoid using it on lower-memory machines to avoid its > > > > memory overhead, then for the same reason we also want to avoid even > > > > starting khugepaged on those <8GB machines. So with > > > > CONFIG_TRANSPARENT_HUGEPAGE_NEVER we can use the same kernel image on > > > > both >=8GB and <8GB machines, with THP support enabled but khugepaged > > > > not started by default. The userspace can then decide to enable THP > > > > (i.e. start khugepaged) via sysfs if needed, based on the total amount > > > > of memory. > > > > > > > > This could also be achieved with the existing transparent_hugepage=never > > > > setting in the kernel command line instead. But it seems cleaner to > > > > avoid tweaking the command line for such a basic setting. > > > > > > > > P.S. I see that CONFIG_TRANSPARENT_HUGEPAGE_NEVER was already proposed > > > > in the past [1] but without an explanation of the purpose. > > > > > > > > ... > > > > > > > > --- a/mm/Kconfig > > > > +++ b/mm/Kconfig > > > > @@ -859,6 +859,12 @@ choice > > > > madvise(MADV_HUGEPAGE) but it won't risk to increase the > > > > memory footprint of applications without a guaranteed > > > > benefit. > > > > + > > > > + config TRANSPARENT_HUGEPAGE_NEVER > > > > + bool "never" > > > > + help > > > > + Disabling Transparent Hugepage by default. It can still be > > > > > > s/Disabling/Disable/ > > > > It is in line with the descriptions of TRANSPARENT_HUGEPAGE_ALWAYS and > > TRANSPARENT_HUGEPAGE_MADVISE: "Enabling Transparent Hugepage ..." > > Those are incorrect also. Ok, corrected in v2. Also clarified the changelog wrt your 2nd question. > > > > + enabled at runtime via sysfs. > > > > endchoice > > > > > > The patch adds the config option but doesn't use it? > > > > I should have been more precise: it is not a new option but a new choice > > for CONFIG_TRANSPARENT_HUGEPAGE, in addition to the existing ALWAYS and > > MADVISE choices. In mm/huge_memory.c in the declaration of the > > transparent_hugepage_flags variable, if either ALWAYS or MADVISE is > > chosen, transparent_hugepage_flags is initialized with such a value > > that makes khugepaged being started by default during bootup. This patch > > allows enabling CONFIG_TRANSPARENT_HUGEPAGE without setting either > > ALWAYS or MADVISE, so that transparent_hugepage_flags is initialized > > with such a value that khugepaged is not started by default. > > OK, thanks.
diff --git a/mm/Kconfig b/mm/Kconfig index 89971a894b60..ec2d5841c9dc 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -859,6 +859,12 @@ choice madvise(MADV_HUGEPAGE) but it won't risk to increase the memory footprint of applications without a guaranteed benefit. + + config TRANSPARENT_HUGEPAGE_NEVER + bool "never" + help + Disabling Transparent Hugepage by default. It can still be + enabled at runtime via sysfs. endchoice config THP_SWAP
Add an option to disable transparent hugepages by default, in line with the existing transparent_hugepage=never command line setting. Rationale: khugepaged has its own non-negligible memory cost even if it is not used by any applications, since it bumps up vm.min_free_kbytes to its own required minimum in set_recommended_min_free_kbytes(). For example, on a machine with 4GB RAM, with 3 mm zones and pageblock_order == MAX_ORDER, starting khugepaged causes vm.min_free_kbytes increase from 8MB to 132MB. So if we use THP on machines with e.g. >=8GB of memory for better performance, but avoid using it on lower-memory machines to avoid its memory overhead, then for the same reason we also want to avoid even starting khugepaged on those <8GB machines. So with CONFIG_TRANSPARENT_HUGEPAGE_NEVER we can use the same kernel image on both >=8GB and <8GB machines, with THP support enabled but khugepaged not started by default. The userspace can then decide to enable THP (i.e. start khugepaged) via sysfs if needed, based on the total amount of memory. This could also be achieved with the existing transparent_hugepage=never setting in the kernel command line instead. But it seems cleaner to avoid tweaking the command line for such a basic setting. P.S. I see that CONFIG_TRANSPARENT_HUGEPAGE_NEVER was already proposed in the past [1] but without an explanation of the purpose. [1] https://lore.kernel.org/all/202211301651462590168@zte.com.cn/ Signed-off-by: Dmytro Maluka <dmaluka@chromium.org> --- mm/Kconfig | 6 ++++++ 1 file changed, 6 insertions(+)