| Message ID | 20240717220219.3743374-3-peterx@redhat.com (mailing list archive) |
|---|---|
| State | RFC |
| Headers | show |
| Series | mm: THP-agnostic refactor on huge mappings | expand |
| Context | Check | Description |
|---|---|---|
| conchuod/vmtest-fixes-PR | fail | merge-conflict |
Le 18/07/2024 à 00:02, Peter Xu a écrit : > Introduce two more sub-options for PGTABLE_HAS_HUGE_LEAVES: > > - PGTABLE_HAS_PMD_LEAVES: set when there can be PMD mappings > - PGTABLE_HAS_PUD_LEAVES: set when there can be PUD mappings > > It will help to identify whether the current build may only want PMD > helpers but not PUD ones, as these sub-options will also check against the > arch support over HAVE_ARCH_TRANSPARENT_HUGEPAGE[_PUD]. > > Note that having them depend on HAVE_ARCH_TRANSPARENT_HUGEPAGE[_PUD] is > still some intermediate step. The best way is to have an option say > "whether arch XXX supports PMD/PUD mappings" and so on. However let's > leave that for later as that's the easy part. So far, we use these options > to stably detect per-arch huge mapping support. > > Signed-off-by: Peter Xu <peterx@redhat.com> > --- > include/linux/huge_mm.h | 10 +++++++--- > mm/Kconfig | 6 ++++++ > 2 files changed, 13 insertions(+), 3 deletions(-) > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > index 711632df7edf..37482c8445d1 100644 > --- a/include/linux/huge_mm.h > +++ b/include/linux/huge_mm.h > @@ -96,14 +96,18 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr; > #define thp_vma_allowable_order(vma, vm_flags, tva_flags, order) \ > (!!thp_vma_allowable_orders(vma, vm_flags, tva_flags, BIT(order))) > > -#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES > -#define HPAGE_PMD_SHIFT PMD_SHIFT > +#ifdef CONFIG_PGTABLE_HAS_PUD_LEAVES > #define HPAGE_PUD_SHIFT PUD_SHIFT > #else > -#define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) > #define HPAGE_PUD_SHIFT ({ BUILD_BUG(); 0; }) > #endif > > +#ifdef CONFIG_PGTABLE_HAS_PMD_LEAVES > +#define HPAGE_PMD_SHIFT PMD_SHIFT > +#else > +#define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) > +#endif > + > #define HPAGE_PMD_ORDER (HPAGE_PMD_SHIFT-PAGE_SHIFT) > #define HPAGE_PMD_NR (1<<HPAGE_PMD_ORDER) > #define HPAGE_PMD_MASK (~(HPAGE_PMD_SIZE - 1)) > diff --git a/mm/Kconfig b/mm/Kconfig > index 60796402850e..2dbdc088dee8 100644 > --- a/mm/Kconfig > +++ b/mm/Kconfig > @@ -860,6 +860,12 @@ endif # TRANSPARENT_HUGEPAGE > config PGTABLE_HAS_HUGE_LEAVES > def_bool TRANSPARENT_HUGEPAGE || HUGETLB_PAGE > > +config PGTABLE_HAS_PMD_LEAVES > + def_bool HAVE_ARCH_TRANSPARENT_HUGEPAGE && PGTABLE_HAS_HUGE_LEAVES > + > +config PGTABLE_HAS_PUD_LEAVES > + def_bool HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD && PGTABLE_HAS_HUGE_LEAVES > + What if an architecture has hugepages at PMD and/or PUD level and doesn't support THP ? Christophe > # > # UP and nommu archs use km based percpu allocator > #
On Thu, Aug 22, 2024 at 05:22:03PM +0000, LEROY Christophe wrote: > > > Le 18/07/2024 à 00:02, Peter Xu a écrit : > > Introduce two more sub-options for PGTABLE_HAS_HUGE_LEAVES: > > > > - PGTABLE_HAS_PMD_LEAVES: set when there can be PMD mappings > > - PGTABLE_HAS_PUD_LEAVES: set when there can be PUD mappings > > > > It will help to identify whether the current build may only want PMD > > helpers but not PUD ones, as these sub-options will also check against the > > arch support over HAVE_ARCH_TRANSPARENT_HUGEPAGE[_PUD]. > > > > Note that having them depend on HAVE_ARCH_TRANSPARENT_HUGEPAGE[_PUD] is > > still some intermediate step. The best way is to have an option say > > "whether arch XXX supports PMD/PUD mappings" and so on. However let's > > leave that for later as that's the easy part. So far, we use these options > > to stably detect per-arch huge mapping support. > > > > Signed-off-by: Peter Xu <peterx@redhat.com> > > --- > > include/linux/huge_mm.h | 10 +++++++--- > > mm/Kconfig | 6 ++++++ > > 2 files changed, 13 insertions(+), 3 deletions(-) > > > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > > index 711632df7edf..37482c8445d1 100644 > > --- a/include/linux/huge_mm.h > > +++ b/include/linux/huge_mm.h > > @@ -96,14 +96,18 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr; > > #define thp_vma_allowable_order(vma, vm_flags, tva_flags, order) \ > > (!!thp_vma_allowable_orders(vma, vm_flags, tva_flags, BIT(order))) > > > > -#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES > > -#define HPAGE_PMD_SHIFT PMD_SHIFT > > +#ifdef CONFIG_PGTABLE_HAS_PUD_LEAVES > > #define HPAGE_PUD_SHIFT PUD_SHIFT > > #else > > -#define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) > > #define HPAGE_PUD_SHIFT ({ BUILD_BUG(); 0; }) > > #endif > > > > +#ifdef CONFIG_PGTABLE_HAS_PMD_LEAVES > > +#define HPAGE_PMD_SHIFT PMD_SHIFT > > +#else > > +#define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) > > +#endif > > + > > #define HPAGE_PMD_ORDER (HPAGE_PMD_SHIFT-PAGE_SHIFT) > > #define HPAGE_PMD_NR (1<<HPAGE_PMD_ORDER) > > #define HPAGE_PMD_MASK (~(HPAGE_PMD_SIZE - 1)) > > diff --git a/mm/Kconfig b/mm/Kconfig > > index 60796402850e..2dbdc088dee8 100644 > > --- a/mm/Kconfig > > +++ b/mm/Kconfig > > @@ -860,6 +860,12 @@ endif # TRANSPARENT_HUGEPAGE > > config PGTABLE_HAS_HUGE_LEAVES > > def_bool TRANSPARENT_HUGEPAGE || HUGETLB_PAGE > > > > +config PGTABLE_HAS_PMD_LEAVES > > + def_bool HAVE_ARCH_TRANSPARENT_HUGEPAGE && PGTABLE_HAS_HUGE_LEAVES > > + > > +config PGTABLE_HAS_PUD_LEAVES > > + def_bool HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD && PGTABLE_HAS_HUGE_LEAVES > > + > > What if an architecture has hugepages at PMD and/or PUD level and > doesn't support THP ? What's the arch to be discussed here? The whole purpose of this series so far is trying to make some pmd/pud helpers that only defined with CONFIG_THP=on to be available even if not. It means this series alone (or any future plan) shouldn't affect any arch that has CONFIG_THP=off always. But logically I think we should need some config option just to say "this arch supports pmd mappings" indeed, even if CONFIG_THP=off. When that's there, we should perhaps add that option into this equation so PGTABLE_HAS_*_LEAVES will also be selected in that case.
Le 22/08/2024 à 21:16, Peter Xu a écrit : > On Thu, Aug 22, 2024 at 05:22:03PM +0000, LEROY Christophe wrote: >> >> >> Le 18/07/2024 à 00:02, Peter Xu a écrit : >>> Introduce two more sub-options for PGTABLE_HAS_HUGE_LEAVES: >>> >>> - PGTABLE_HAS_PMD_LEAVES: set when there can be PMD mappings >>> - PGTABLE_HAS_PUD_LEAVES: set when there can be PUD mappings >>> >>> It will help to identify whether the current build may only want PMD >>> helpers but not PUD ones, as these sub-options will also check against the >>> arch support over HAVE_ARCH_TRANSPARENT_HUGEPAGE[_PUD]. >>> >>> Note that having them depend on HAVE_ARCH_TRANSPARENT_HUGEPAGE[_PUD] is >>> still some intermediate step. The best way is to have an option say >>> "whether arch XXX supports PMD/PUD mappings" and so on. However let's >>> leave that for later as that's the easy part. So far, we use these options >>> to stably detect per-arch huge mapping support. >>> >>> Signed-off-by: Peter Xu <peterx@redhat.com> >>> --- >>> include/linux/huge_mm.h | 10 +++++++--- >>> mm/Kconfig | 6 ++++++ >>> 2 files changed, 13 insertions(+), 3 deletions(-) >>> >>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h >>> index 711632df7edf..37482c8445d1 100644 >>> --- a/include/linux/huge_mm.h >>> +++ b/include/linux/huge_mm.h >>> @@ -96,14 +96,18 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr; >>> #define thp_vma_allowable_order(vma, vm_flags, tva_flags, order) \ >>> (!!thp_vma_allowable_orders(vma, vm_flags, tva_flags, BIT(order))) >>> >>> -#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES >>> -#define HPAGE_PMD_SHIFT PMD_SHIFT >>> +#ifdef CONFIG_PGTABLE_HAS_PUD_LEAVES >>> #define HPAGE_PUD_SHIFT PUD_SHIFT >>> #else >>> -#define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) >>> #define HPAGE_PUD_SHIFT ({ BUILD_BUG(); 0; }) >>> #endif >>> >>> +#ifdef CONFIG_PGTABLE_HAS_PMD_LEAVES >>> +#define HPAGE_PMD_SHIFT PMD_SHIFT >>> +#else >>> +#define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) >>> +#endif >>> + >>> #define HPAGE_PMD_ORDER (HPAGE_PMD_SHIFT-PAGE_SHIFT) >>> #define HPAGE_PMD_NR (1<<HPAGE_PMD_ORDER) >>> #define HPAGE_PMD_MASK (~(HPAGE_PMD_SIZE - 1)) >>> diff --git a/mm/Kconfig b/mm/Kconfig >>> index 60796402850e..2dbdc088dee8 100644 >>> --- a/mm/Kconfig >>> +++ b/mm/Kconfig >>> @@ -860,6 +860,12 @@ endif # TRANSPARENT_HUGEPAGE >>> config PGTABLE_HAS_HUGE_LEAVES >>> def_bool TRANSPARENT_HUGEPAGE || HUGETLB_PAGE >>> >>> +config PGTABLE_HAS_PMD_LEAVES >>> + def_bool HAVE_ARCH_TRANSPARENT_HUGEPAGE && PGTABLE_HAS_HUGE_LEAVES >>> + >>> +config PGTABLE_HAS_PUD_LEAVES >>> + def_bool HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD && PGTABLE_HAS_HUGE_LEAVES >>> + >> >> What if an architecture has hugepages at PMD and/or PUD level and >> doesn't support THP ? > > What's the arch to be discussed here? It is LOONGARCH and MIPS, they provide pud_leaf() that can return true even when they have no PUD. > > The whole purpose of this series so far is trying to make some pmd/pud > helpers that only defined with CONFIG_THP=on to be available even if not. > It means this series alone (or any future plan) shouldn't affect any arch > that has CONFIG_THP=off always. > > But logically I think we should need some config option just to say "this > arch supports pmd mappings" indeed, even if CONFIG_THP=off. When that's > there, we should perhaps add that option into this equation so > PGTABLE_HAS_*_LEAVES will also be selected in that case. > Why is an option needed for that ? If pmd_leaf() returns always false, it means the arch doesn't support pmd mappings and if properly used all related code should fold away without a config option, shouldn't it ?
On Fri, Aug 23, 2024 at 06:19:52AM +0000, LEROY Christophe wrote: > Why is an option needed for that ? If pmd_leaf() returns always false, > it means the arch doesn't support pmd mappings and if properly used all > related code should fold away without a config option, shouldn't it ? It's not always easy to leverage an "if" clause there, IIUC. Take the case of when a driver wants to inject a pmd pfnmap, we may want something like: if (pmd_leaf_supported()) inject_pmd_leaf(&pmd); We don't have a pmd entry to reference at the point of pmd_leaf_supported() when making the decision. Thanks,
diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 711632df7edf..37482c8445d1 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -96,14 +96,18 @@ extern struct kobj_attribute thpsize_shmem_enabled_attr; #define thp_vma_allowable_order(vma, vm_flags, tva_flags, order) \ (!!thp_vma_allowable_orders(vma, vm_flags, tva_flags, BIT(order))) -#ifdef CONFIG_PGTABLE_HAS_HUGE_LEAVES -#define HPAGE_PMD_SHIFT PMD_SHIFT +#ifdef CONFIG_PGTABLE_HAS_PUD_LEAVES #define HPAGE_PUD_SHIFT PUD_SHIFT #else -#define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) #define HPAGE_PUD_SHIFT ({ BUILD_BUG(); 0; }) #endif +#ifdef CONFIG_PGTABLE_HAS_PMD_LEAVES +#define HPAGE_PMD_SHIFT PMD_SHIFT +#else +#define HPAGE_PMD_SHIFT ({ BUILD_BUG(); 0; }) +#endif + #define HPAGE_PMD_ORDER (HPAGE_PMD_SHIFT-PAGE_SHIFT) #define HPAGE_PMD_NR (1<<HPAGE_PMD_ORDER) #define HPAGE_PMD_MASK (~(HPAGE_PMD_SIZE - 1)) diff --git a/mm/Kconfig b/mm/Kconfig index 60796402850e..2dbdc088dee8 100644 --- a/mm/Kconfig +++ b/mm/Kconfig @@ -860,6 +860,12 @@ endif # TRANSPARENT_HUGEPAGE config PGTABLE_HAS_HUGE_LEAVES def_bool TRANSPARENT_HUGEPAGE || HUGETLB_PAGE +config PGTABLE_HAS_PMD_LEAVES + def_bool HAVE_ARCH_TRANSPARENT_HUGEPAGE && PGTABLE_HAS_HUGE_LEAVES + +config PGTABLE_HAS_PUD_LEAVES + def_bool HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD && PGTABLE_HAS_HUGE_LEAVES + # # UP and nommu archs use km based percpu allocator #
Introduce two more sub-options for PGTABLE_HAS_HUGE_LEAVES: - PGTABLE_HAS_PMD_LEAVES: set when there can be PMD mappings - PGTABLE_HAS_PUD_LEAVES: set when there can be PUD mappings It will help to identify whether the current build may only want PMD helpers but not PUD ones, as these sub-options will also check against the arch support over HAVE_ARCH_TRANSPARENT_HUGEPAGE[_PUD]. Note that having them depend on HAVE_ARCH_TRANSPARENT_HUGEPAGE[_PUD] is still some intermediate step. The best way is to have an option say "whether arch XXX supports PMD/PUD mappings" and so on. However let's leave that for later as that's the easy part. So far, we use these options to stably detect per-arch huge mapping support. Signed-off-by: Peter Xu <peterx@redhat.com> --- include/linux/huge_mm.h | 10 +++++++--- mm/Kconfig | 6 ++++++ 2 files changed, 13 insertions(+), 3 deletions(-)