[V13] mm/debug: Add tests validating architecture page table helpers

Commit Message

Anshuman Khandual Feb. 5, 2020, 10:14 a.m. UTC

This adds tests which will validate architecture page table helpers and
other accessors in their compliance with expected generic MM semantics.
This will help various architectures in validating changes to existing
page table helpers or addition of new ones.

This test covers basic page table entry transformations including but not
limited to old, young, dirty, clean, write, write protect etc at various
level along with populating intermediate entries with next page table page
and validating them.

Test page table pages are allocated from system memory with required size
and alignments. The mapped pfns at page table levels are derived from a
real pfn representing a valid kernel text symbol. This test gets called
inside kernel_init() right after async_synchronize_full().

This test gets built and run when CONFIG_DEBUG_VM_PGTABLE is selected. Any
architecture, which is willing to subscribe this test will need to select
ARCH_HAS_DEBUG_VM_PGTABLE. For now this is limited to arc, arm64, x86, s390
and ppc32 platforms where the test is known to build and run successfully.
Going forward, other architectures too can subscribe the test after fixing
any build or runtime problems with their page table helpers. Meanwhile for
better platform coverage, the test can also be enabled with CONFIG_EXPERT
even without ARCH_HAS_DEBUG_VM_PGTABLE.

Folks interested in making sure that a given platform's page table helpers
conform to expected generic MM semantics should enable the above config
which will just trigger this test during boot. Any non conformity here will
be reported as an warning which would need to be fixed. This test will help
catch any changes to the agreed upon semantics expected from generic MM and
enable platforms to accommodate it thereafter.

Cc: Andrew Morton <[e-mail]>
Cc: Vlastimil Babka <[e-mail]>
Cc: Greg Kroah-Hartman <[e-mail]>
Cc: Thomas Gleixner <[e-mail]>
Cc: Mike Rapoport <[e-mail]>
Cc: Jason Gunthorpe <[e-mail]>
Cc: Dan Williams <[e-mail]>
Cc: Peter Zijlstra <[e-mail]>
Cc: Michal Hocko <[e-mail]>
Cc: Mark Rutland <[e-mail]>
Cc: Mark Brown <[e-mail]>
Cc: Steven Price <[e-mail]>
Cc: Ard Biesheuvel <[e-mail]>
Cc: Masahiro Yamada <[e-mail]>
Cc: Kees Cook <[e-mail]>
Cc: Tetsuo Handa <[e-mail]>
Cc: Matthew Wilcox <[e-mail]>
Cc: Dave Hansen <[e-mail]>
Cc: Russell King - ARM Linux <[e-mail]>
Cc: Michael Ellerman <[e-mail]>
Cc: Paul Mackerras <[e-mail]>
Cc: Martin Schwidefsky <[e-mail]>
Cc: Heiko Carstens <[e-mail]>
Cc: "David S. Miller" <[e-mail]>
Cc: Vineet Gupta <[e-mail]>
Cc: James Hogan <[e-mail]>
Cc: Paul Burton <[e-mail]>
Cc: Ralf Baechle <[e-mail]>
Cc: Kirill A. Shutemov <[e-mail]>
Cc: Gerald Schaefer <[e-mail]>
Cc: Christophe Leroy <[e-mail]>
Cc: Ingo Molnar <[e-mail]>
Cc: Richard Henderson <[e-mail]>
Cc: Ivan Kokshaysky <[e-mail]>
Cc: Matt Turner <[e-mail]>
Cc: Mark Salter <[e-mail]>
Cc: Aurelien Jacquiot <[e-mail]>
Cc: Yoshinori Sato <[e-mail]>
Cc: Brian Cain <[e-mail]>
Cc: Geert Uytterhoeven <[e-mail]>
Cc: Michal Simek <[e-mail]>
Cc: Paul Walmsley <[e-mail]>
Cc: Palmer Dabbelt <[e-mail]>
Cc: Guan Xuetao <[e-mail]>
Cc: Chris Zankel <[e-mail]>
Cc: Max Filippov <[e-mail]>
Cc: Qian Cai <[e-mail]>
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]
Cc: [e-mail]

Suggested-by: Catalin Marinas <[e-mail]>
Reviewed-by: Ingo Molnar <[e-mail]>
Tested-by: Gerald Schaefer <[e-mail]> # s390
Tested-by: Christophe Leroy <[e-mail]>	# ppc32
Signed-off-by: Andrew Morton <[e-mail]>
Signed-off-by: Christophe Leroy <[e-mail]>
Signed-off-by: Anshuman Khandual <[e-mail]>
---
This adds a test validation for architecture exported page table helpers.
Patch adds basic transformation tests at various levels of the page table.

This test was originally suggested by Catalin during arm64 THP migration
RFC discussion earlier. Going forward it can include more specific tests
with respect to various generic MM functions like THP, HugeTLB etc and
platform specific tests.

https://lore.kernel.org/linux-mm/[e-mail]/

Needs to be applied on linux V5.5

Changes in V13:

- Subscribed s390 platform and updated debug-vm-pgtable/arch-support.txt per Gerald
- Dropped keyword 'extern' from debug_vm_pgtable() declaration per Christophe
- Moved debug_vm_pgtable() declarations to <linux/mmdebug.h> per Christophe
- Moved debug_vm_pgtable() call site into kernel_init() per Christophe
- Changed CONFIG_DEBUG_VM_PGTABLE rules per Christophe
- Updated commit to include new supported platforms and changed config selection

Changes in V12: (https://patchwork.kernel.org/project/linux-mm/list/?series=233905)

- Replaced __mmdrop() with mmdrop()
- Enable ARCH_HAS_DEBUG_VM_PGTABLE on X86 for non CONFIG_X86_PAE platforms as the
  test procedure interfere with pre-allocated PMDs attached to the PGD resulting
  in runtime failures with VM_BUG_ON()

Changes in V11: (https://patchwork.kernel.org/project/linux-mm/list/?series=221135)

- Rebased the patch on V5.4

Changes in V10: (https://patchwork.kernel.org/project/linux-mm/list/?series=205529)

- Always enable DEBUG_VM_PGTABLE when DEBUG_VM is enabled per Ingo
- Added tags from Ingo

Changes in V9: (https://patchwork.kernel.org/project/linux-mm/list/?series=201429)

- Changed feature support enumeration for powerpc platforms per Christophe
- Changed config wrapper for basic_[pmd|pud]_tests() to enable ARC platform
- Enabled the test on ARC platform

Changes in V8: (https://patchwork.kernel.org/project/linux-mm/list/?series=194297)

- Enabled ARCH_HAS_DEBUG_VM_PGTABLE on PPC32 platform per Christophe
- Updated feature documentation as DEBUG_VM_PGTABLE is now enabled on PPC32 platform
- Moved ARCH_HAS_DEBUG_VM_PGTABLE earlier to indent it with DEBUG_VM per Christophe
- Added an information message in debug_vm_pgtable() per Christophe
- Dropped random_vaddr boundary condition checks per Christophe and Qian
- Replaced virt_addr_valid() check with pfn_valid() check in debug_vm_pgtable()
- Slightly changed pr_fmt(fmt) information

Changes in V7: (https://patchwork.kernel.org/project/linux-mm/list/?series=193051)

- Memory allocation and free routines for mapped pages have been droped
- Mapped pfns are derived from standard kernel text symbol per Matthew
- Moved debug_vm_pgtaable() after page_alloc_init_late() per Michal and Qian 
- Updated the commit message per Michal
- Updated W=1 GCC warning problem on x86 per Qian Cai
- Addition of new alloc_contig_pages() helper has been submitted separately

Changes in V6: (https://patchwork.kernel.org/project/linux-mm/list/?series=187589)

- Moved alloc_gigantic_page_order() into mm/page_alloc.c per Michal
- Moved alloc_gigantic_page_order() within CONFIG_CONTIG_ALLOC in the test
- Folded Andrew's include/asm-generic/pgtable.h fix into the test patch 2/2

Changes in V5: (https://patchwork.kernel.org/project/linux-mm/list/?series=185991)

- Redefined and moved X86 mm_p4d_folded() into a different header per Kirill/Ingo
- Updated the config option comment per Ingo and dropped 'kernel module' reference
- Updated the commit message and dropped 'kernel module' reference
- Changed DEBUG_ARCH_PGTABLE_TEST into DEBUG_VM_PGTABLE per Ingo
- Moved config option from mm/Kconfig.debug into lib/Kconfig.debug
- Renamed core test function arch_pgtable_tests() as debug_vm_pgtable()
- Renamed mm/arch_pgtable_test.c as mm/debug_vm_pgtable.c
- debug_vm_pgtable() gets called from kernel_init_freeable() after init_mm_internals()
- Added an entry in Documentation/features/debug/ per Ingo
- Enabled the test on arm64 and x86 platforms for now

Changes in V4: (https://patchwork.kernel.org/project/linux-mm/list/?series=183465)

- Disable DEBUG_ARCH_PGTABLE_TEST for ARM and IA64 platforms

Changes in V3: (https://lore.kernel.org/patchwork/project/lkml/list/?series=411216)

- Changed test trigger from module format into late_initcall()
- Marked all functions with __init to be freed after completion
- Changed all __PGTABLE_PXX_FOLDED checks as mm_pxx_folded()
- Folded in PPC32 fixes from Christophe

Changes in V2:

https://lore.kernel.org/linux-mm/[e-mail]/T/#t

- Fixed small typo error in MODULE_DESCRIPTION()
- Fixed m64k build problems for lvalue concerns in pmd_xxx_tests()
- Fixed dynamic page table level folding problems on x86 as per Kirril
- Fixed second pointers during pxx_populate_tests() per Kirill and Gerald
- Allocate and free pte table with pte_alloc_one/pte_free per Kirill
- Modified pxx_clear_tests() to accommodate s390 lower 12 bits situation
- Changed RANDOM_NZVALUE value from 0xbe to 0xff
- Changed allocation, usage, free sequence for saved_ptep
- Renamed VMA_FLAGS as VMFLAGS
- Implemented a new method for random vaddr generation
- Implemented some other cleanups
- Dropped extern reference to mm_alloc()
- Created and exported new alloc_gigantic_page_order()
- Dropped the custom allocator and used new alloc_gigantic_page_order()

Changes in V1:

https://lore.kernel.org/linux-mm/[e-mail]/

- Added fallback mechanism for PMD aligned memory allocation failure

Changes in RFC V2:

https://lore.kernel.org/linux-mm/[e-mail]/T/#u

- Moved test module and it's config from lib/ to mm/
- Renamed config TEST_ARCH_PGTABLE as DEBUG_ARCH_PGTABLE_TEST
- Renamed file from test_arch_pgtable.c to arch_pgtable_test.c
- Added relevant MODULE_DESCRIPTION() and MODULE_AUTHOR() details
- Dropped loadable module config option
- Basic tests now use memory blocks with required size and alignment
- PUD aligned memory block gets allocated with alloc_contig_range()
- If PUD aligned memory could not be allocated it falls back on PMD aligned
  memory block from page allocator and pud_* tests are skipped
- Clear and populate tests now operate on real in memory page table entries
- Dummy mm_struct gets allocated with mm_alloc()
- Dummy page table entries get allocated with [pud|pmd|pte]_alloc_[map]()
- Simplified [p4d|pgd]_basic_tests(), now has random values in the entries

Original RFC V1:

https://lore.kernel.org/linux-mm/[e-mail]/

 .../debug/debug-vm-pgtable/arch-support.txt   |  35 ++
 arch/arc/Kconfig                              |   1 +
 arch/arm64/Kconfig                            |   1 +
 arch/powerpc/Kconfig                          |   1 +
 arch/s390/Kconfig                             |   1 +
 arch/x86/Kconfig                              |   1 +
 arch/x86/include/asm/pgtable_64.h             |   6 +
 include/linux/mmdebug.h                       |   5 +
 init/main.c                                   |   2 +
 lib/Kconfig.debug                             |  22 +
 mm/Makefile                                   |   1 +
 mm/debug_vm_pgtable.c                         | 388 ++++++++++++++++++
 12 files changed, 464 insertions(+)
 create mode 100644 Documentation/features/debug/debug-vm-pgtable/arch-support.txt
 create mode 100644 mm/debug_vm_pgtable.c

Comments

kernel test robot Feb. 5, 2020, 11:10 p.m. UTC | #1

Hi Anshuman,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on powerpc/next]
[also build test ERROR on s390/features linus/master arc/for-next v5.5]
[cannot apply to mmotm/master tip/x86/core arm64/for-next/core next-20200205]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]

url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20200205-215507
base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
config: ia64-allmodconfig (attached as .config)
compiler: ia64-linux-gcc (GCC) 7.5.0
reproduce:
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # save the attached .config to linux build tree
        GCC_VERSION=7.5.0 make.cross ARCH=ia64 

If you fix the issue, kindly add following tag
Reported-by: kbuild test robot <[e-mail]>

All error/warnings (new ones prefixed by >>):

   In file included from include/asm-generic/pgtable-nopud.h:8:0,
                    from arch/ia64/include/asm/pgtable.h:586,
                    from include/linux/mm.h:99,
                    from include/linux/highmem.h:8,
                    from mm/debug_vm_pgtable.c:14:
   mm/debug_vm_pgtable.c: In function 'pud_clear_tests':
>> include/asm-generic/pgtable-nop4d-hack.h:47:32: error: implicit declaration of function '__pgd'; did you mean '__p4d'? [-Werror=implicit-function-declaration]
    #define __pud(x)    ((pud_t) { __pgd(x) })
                                   ^
>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
     pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
           ^~~~~
>> include/asm-generic/pgtable-nop4d-hack.h:47:22: warning: missing braces around initializer [-Wmissing-braces]
    #define __pud(x)    ((pud_t) { __pgd(x) })
                         ^
>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
     pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
           ^~~~~
   cc1: some warnings being treated as errors

vim +47 include/asm-generic/pgtable-nop4d-hack.h

30ec842660bd0d Kirill A. Shutemov 2017-03-09  45  
30ec842660bd0d Kirill A. Shutemov 2017-03-09  46  #define pud_val(x)				(pgd_val((x).pgd))
30ec842660bd0d Kirill A. Shutemov 2017-03-09 @47  #define __pud(x)				((pud_t) { __pgd(x) })
30ec842660bd0d Kirill A. Shutemov 2017-03-09  48  

:::::: The code at line 47 was first introduced by commit
:::::: 30ec842660bd0d056d4a7028ac5bd4a82b113d4f asm-generic: introduce __ARCH_USE_5LEVEL_HACK

:::::: TO: Kirill A. Shutemov <[e-mail]>
:::::: CC: Linus Torvalds <[e-mail]>

---
0-DAY kernel test infrastructure                 Open Source Technology Center
https://lists.01.org/hyperkitty/list/[e-mail] Intel Corporation

Anshuman Khandual Feb. 6, 2020, 8:19 a.m. UTC | #2

On 02/06/2020 04:40 AM, kbuild test robot wrote:
> Hi Anshuman,
> 
> Thank you for the patch! Yet something to improve:
> 
> [auto build test ERROR on powerpc/next]
> [also build test ERROR on s390/features linus/master arc/for-next v5.5]
> [cannot apply to mmotm/master tip/x86/core arm64/for-next/core next-20200205]
> [if your patch is applied to the wrong git tree, please drop us a note to help
> improve the system. BTW, we also suggest to use '--base' option to specify the
> base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
> 
> url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20200205-215507
> base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
> config: ia64-allmodconfig (attached as .config)
> compiler: ia64-linux-gcc (GCC) 7.5.0
> reproduce:
>         wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>         chmod +x ~/bin/make.cross
>         # save the attached .config to linux build tree
>         GCC_VERSION=7.5.0 make.cross ARCH=ia64 
> 
> If you fix the issue, kindly add following tag
> Reported-by: kbuild test robot <[e-mail]>
> 
> All error/warnings (new ones prefixed by >>):
> 
>    In file included from include/asm-generic/pgtable-nopud.h:8:0,
>                     from arch/ia64/include/asm/pgtable.h:586,
>                     from include/linux/mm.h:99,
>                     from include/linux/highmem.h:8,
>                     from mm/debug_vm_pgtable.c:14:
>    mm/debug_vm_pgtable.c: In function 'pud_clear_tests':
>>> include/asm-generic/pgtable-nop4d-hack.h:47:32: error: implicit declaration of function '__pgd'; did you mean '__p4d'? [-Werror=implicit-function-declaration]
>     #define __pud(x)    ((pud_t) { __pgd(x) })
>                                    ^
>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>      pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>            ^~~~~
>>> include/asm-generic/pgtable-nop4d-hack.h:47:22: warning: missing braces around initializer [-Wmissing-braces]
>     #define __pud(x)    ((pud_t) { __pgd(x) })
>                          ^
>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>      pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>            ^~~~~
>    cc1: some warnings being treated as errors

This build failure is expected now given that we have allowed DEBUG_VM_PGTABLE
with EXPERT without platform requiring ARCH_HAS_DEBUG_VM_PGTABLE. This problem
i.e build failure caused without a platform __pgd(), is known to exist both on
ia64 and arm (32bit) platforms. Please refer https://lkml.org/lkml/2019/9/24/314
for details where this was discussed earlier.

- Anshuman

Andrew Morton Feb. 10, 2020, 4:52 a.m. UTC | #3

On Thu, 6 Feb 2020 13:49:35 +0530 Anshuman Khandual <[e-mail]> wrote:

> 
> On 02/06/2020 04:40 AM, kbuild test robot wrote:
> > Hi Anshuman,
> > 
> > Thank you for the patch! Yet something to improve:
> > 
> > [auto build test ERROR on powerpc/next]
> > [also build test ERROR on s390/features linus/master arc/for-next v5.5]
> > [cannot apply to mmotm/master tip/x86/core arm64/for-next/core next-20200205]
> > [if your patch is applied to the wrong git tree, please drop us a note to help
> > improve the system. BTW, we also suggest to use '--base' option to specify the
> > base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
> > 
> > url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20200205-215507
> > base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
> > config: ia64-allmodconfig (attached as .config)
> > compiler: ia64-linux-gcc (GCC) 7.5.0
> > reproduce:
> >         wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> >         chmod +x ~/bin/make.cross
> >         # save the attached .config to linux build tree
> >         GCC_VERSION=7.5.0 make.cross ARCH=ia64 
> > 
> > If you fix the issue, kindly add following tag
> > Reported-by: kbuild test robot <[e-mail]>
> > 
> > All error/warnings (new ones prefixed by >>):
> > 
> >    In file included from include/asm-generic/pgtable-nopud.h:8:0,
> >                     from arch/ia64/include/asm/pgtable.h:586,
> >                     from include/linux/mm.h:99,
> >                     from include/linux/highmem.h:8,
> >                     from mm/debug_vm_pgtable.c:14:
> >    mm/debug_vm_pgtable.c: In function 'pud_clear_tests':
> >>> include/asm-generic/pgtable-nop4d-hack.h:47:32: error: implicit declaration of function '__pgd'; did you mean '__p4d'? [-Werror=implicit-function-declaration]
> >     #define __pud(x)    ((pud_t) { __pgd(x) })
> >                                    ^
> >>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
> >      pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
> >            ^~~~~
> >>> include/asm-generic/pgtable-nop4d-hack.h:47:22: warning: missing braces around initializer [-Wmissing-braces]
> >     #define __pud(x)    ((pud_t) { __pgd(x) })
> >                          ^
> >>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
> >      pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
> >            ^~~~~
> >    cc1: some warnings being treated as errors
> 
> This build failure is expected now given that we have allowed DEBUG_VM_PGTABLE
> with EXPERT without platform requiring ARCH_HAS_DEBUG_VM_PGTABLE. This problem
> i.e build failure caused without a platform __pgd(), is known to exist both on
> ia64 and arm (32bit) platforms. Please refer https://lkml.org/lkml/2019/9/24/314
> for details where this was discussed earlier.
> 

I'd prefer not to merge a patch which is known to cause build
regressions.  Is there some temporary thing we can do to prevent these
errors until arch maintainers(?) get around to implementing the
long-term fixes?

Anshuman Khandual Feb. 10, 2020, 5:35 a.m. UTC | #4

On 02/10/2020 10:22 AM, Andrew Morton wrote:
> On Thu, 6 Feb 2020 13:49:35 +0530 Anshuman Khandual <[e-mail]> wrote:
> 
>>
>> On 02/06/2020 04:40 AM, kbuild test robot wrote:
>>> Hi Anshuman,
>>>
>>> Thank you for the patch! Yet something to improve:
>>>
>>> [auto build test ERROR on powerpc/next]
>>> [also build test ERROR on s390/features linus/master arc/for-next v5.5]
>>> [cannot apply to mmotm/master tip/x86/core arm64/for-next/core next-20200205]
>>> [if your patch is applied to the wrong git tree, please drop us a note to help
>>> improve the system. BTW, we also suggest to use '--base' option to specify the
>>> base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
>>>
>>> url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20200205-215507
>>> base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
>>> config: ia64-allmodconfig (attached as .config)
>>> compiler: ia64-linux-gcc (GCC) 7.5.0
>>> reproduce:
>>>         wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>>>         chmod +x ~/bin/make.cross
>>>         # save the attached .config to linux build tree
>>>         GCC_VERSION=7.5.0 make.cross ARCH=ia64 
>>>
>>> If you fix the issue, kindly add following tag
>>> Reported-by: kbuild test robot <[e-mail]>
>>>
>>> All error/warnings (new ones prefixed by >>):
>>>
>>>    In file included from include/asm-generic/pgtable-nopud.h:8:0,
>>>                     from arch/ia64/include/asm/pgtable.h:586,
>>>                     from include/linux/mm.h:99,
>>>                     from include/linux/highmem.h:8,
>>>                     from mm/debug_vm_pgtable.c:14:
>>>    mm/debug_vm_pgtable.c: In function 'pud_clear_tests':
>>>>> include/asm-generic/pgtable-nop4d-hack.h:47:32: error: implicit declaration of function '__pgd'; did you mean '__p4d'? [-Werror=implicit-function-declaration]
>>>     #define __pud(x)    ((pud_t) { __pgd(x) })
>>>                                    ^
>>>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>>>      pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>>>            ^~~~~
>>>>> include/asm-generic/pgtable-nop4d-hack.h:47:22: warning: missing braces around initializer [-Wmissing-braces]
>>>     #define __pud(x)    ((pud_t) { __pgd(x) })
>>>                          ^
>>>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>>>      pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>>>            ^~~~~
>>>    cc1: some warnings being treated as errors
>>
>> This build failure is expected now given that we have allowed DEBUG_VM_PGTABLE
>> with EXPERT without platform requiring ARCH_HAS_DEBUG_VM_PGTABLE. This problem
>> i.e build failure caused without a platform __pgd(), is known to exist both on
>> ia64 and arm (32bit) platforms. Please refer https://lkml.org/lkml/2019/9/24/314
>> for details where this was discussed earlier.
>>
> 
> I'd prefer not to merge a patch which is known to cause build
> regressions.  Is there some temporary thing we can do to prevent these
> errors until arch maintainers(?) get around to implementing the
> long-term fixes?

We could explicitly disable CONFIG_DEBUG_VM_PGTABLE on ia64 and arm platforms
which will ensure that others can still use the EXPERT path.

config DEBUG_VM_PGTABLE
	bool "Debug arch page table for semantics compliance"
	depends on MMU
	depends on !(IA64 || ARM)
	depends on ARCH_HAS_DEBUG_VM_PGTABLE || EXPERT
	default n if !ARCH_HAS_DEBUG_VM_PGTABLE
	default y if DEBUG_VM

Christophe Leroy Feb. 10, 2020, 6:38 a.m. UTC | #5

Le 10/02/2020 à 06:35, Anshuman Khandual a écrit :
> 
> 
> On 02/10/2020 10:22 AM, Andrew Morton wrote:
>> On Thu, 6 Feb 2020 13:49:35 +0530 Anshuman Khandual <[e-mail]> wrote:
>>
>>>
>>> On 02/06/2020 04:40 AM, kbuild test robot wrote:
>>>> Hi Anshuman,
>>>>
>>>> Thank you for the patch! Yet something to improve:
>>>>
>>>> [auto build test ERROR on powerpc/next]
>>>> [also build test ERROR on s390/features linus/master arc/for-next v5.5]
>>>> [cannot apply to mmotm/master tip/x86/core arm64/for-next/core next-20200205]
>>>> [if your patch is applied to the wrong git tree, please drop us a note to help
>>>> improve the system. BTW, we also suggest to use '--base' option to specify the
>>>> base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
>>>>
>>>> url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20200205-215507
>>>> base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
>>>> config: ia64-allmodconfig (attached as .config)
>>>> compiler: ia64-linux-gcc (GCC) 7.5.0
>>>> reproduce:
>>>>          wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>>>>          chmod +x ~/bin/make.cross
>>>>          # save the attached .config to linux build tree
>>>>          GCC_VERSION=7.5.0 make.cross ARCH=ia64
>>>>
>>>> If you fix the issue, kindly add following tag
>>>> Reported-by: kbuild test robot <[e-mail]>
>>>>
>>>> All error/warnings (new ones prefixed by >>):
>>>>
>>>>     In file included from include/asm-generic/pgtable-nopud.h:8:0,
>>>>                      from arch/ia64/include/asm/pgtable.h:586,
>>>>                      from include/linux/mm.h:99,
>>>>                      from include/linux/highmem.h:8,
>>>>                      from mm/debug_vm_pgtable.c:14:
>>>>     mm/debug_vm_pgtable.c: In function 'pud_clear_tests':
>>>>>> include/asm-generic/pgtable-nop4d-hack.h:47:32: error: implicit declaration of function '__pgd'; did you mean '__p4d'? [-Werror=implicit-function-declaration]
>>>>      #define __pud(x)    ((pud_t) { __pgd(x) })
>>>>                                     ^
>>>>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>>>>       pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>>>>             ^~~~~
>>>>>> include/asm-generic/pgtable-nop4d-hack.h:47:22: warning: missing braces around initializer [-Wmissing-braces]
>>>>      #define __pud(x)    ((pud_t) { __pgd(x) })
>>>>                           ^
>>>>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>>>>       pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>>>>             ^~~~~
>>>>     cc1: some warnings being treated as errors
>>>
>>> This build failure is expected now given that we have allowed DEBUG_VM_PGTABLE
>>> with EXPERT without platform requiring ARCH_HAS_DEBUG_VM_PGTABLE. This problem
>>> i.e build failure caused without a platform __pgd(), is known to exist both on
>>> ia64 and arm (32bit) platforms. Please refer https://lkml.org/lkml/2019/9/24/314
>>> for details where this was discussed earlier.
>>>
>>
>> I'd prefer not to merge a patch which is known to cause build
>> regressions.  Is there some temporary thing we can do to prevent these
>> errors until arch maintainers(?) get around to implementing the
>> long-term fixes?
> 
> We could explicitly disable CONFIG_DEBUG_VM_PGTABLE on ia64 and arm platforms
> which will ensure that others can still use the EXPERT path.
> 
> config DEBUG_VM_PGTABLE
> 	bool "Debug arch page table for semantics compliance"
> 	depends on MMU
> 	depends on !(IA64 || ARM)
> 	depends on ARCH_HAS_DEBUG_VM_PGTABLE || EXPERT
> 	default n if !ARCH_HAS_DEBUG_VM_PGTABLE
> 	default y if DEBUG_VM
> 

On both ia32 and arm, the fix is trivial.

Can we include the fix within this patch, just the same way as the 
mm_p4d_folded() fix for x86 ?

Christophe

Russell King (Oracle) Feb. 10, 2020, 10:02 a.m. UTC | #6

On Mon, Feb 10, 2020 at 07:38:38AM +0100, Christophe Leroy wrote:
> 
> 
> Le 10/02/2020 à 06:35, Anshuman Khandual a écrit :
> > 
> > 
> > On 02/10/2020 10:22 AM, Andrew Morton wrote:
> > > On Thu, 6 Feb 2020 13:49:35 +0530 Anshuman Khandual <[e-mail]> wrote:
> > > 
> > > > 
> > > > On 02/06/2020 04:40 AM, kbuild test robot wrote:
> > > > > Hi Anshuman,
> > > > > 
> > > > > Thank you for the patch! Yet something to improve:
> > > > > 
> > > > > [auto build test ERROR on powerpc/next]
> > > > > [also build test ERROR on s390/features linus/master arc/for-next v5.5]
> > > > > [cannot apply to mmotm/master tip/x86/core arm64/for-next/core next-20200205]
> > > > > [if your patch is applied to the wrong git tree, please drop us a note to help
> > > > > improve the system. BTW, we also suggest to use '--base' option to specify the
> > > > > base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
> > > > > 
> > > > > url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20200205-215507
> > > > > base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
> > > > > config: ia64-allmodconfig (attached as .config)
> > > > > compiler: ia64-linux-gcc (GCC) 7.5.0
> > > > > reproduce:
> > > > >          wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> > > > >          chmod +x ~/bin/make.cross
> > > > >          # save the attached .config to linux build tree
> > > > >          GCC_VERSION=7.5.0 make.cross ARCH=ia64
> > > > > 
> > > > > If you fix the issue, kindly add following tag
> > > > > Reported-by: kbuild test robot <[e-mail]>
> > > > > 
> > > > > All error/warnings (new ones prefixed by >>):
> > > > > 
> > > > >     In file included from include/asm-generic/pgtable-nopud.h:8:0,
> > > > >                      from arch/ia64/include/asm/pgtable.h:586,
> > > > >                      from include/linux/mm.h:99,
> > > > >                      from include/linux/highmem.h:8,
> > > > >                      from mm/debug_vm_pgtable.c:14:
> > > > >     mm/debug_vm_pgtable.c: In function 'pud_clear_tests':
> > > > > > > include/asm-generic/pgtable-nop4d-hack.h:47:32: error: implicit declaration of function '__pgd'; did you mean '__p4d'? [-Werror=implicit-function-declaration]
> > > > >      #define __pud(x)    ((pud_t) { __pgd(x) })
> > > > >                                     ^
> > > > > > > mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
> > > > >       pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
> > > > >             ^~~~~
> > > > > > > include/asm-generic/pgtable-nop4d-hack.h:47:22: warning: missing braces around initializer [-Wmissing-braces]
> > > > >      #define __pud(x)    ((pud_t) { __pgd(x) })
> > > > >                           ^
> > > > > > > mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
> > > > >       pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
> > > > >             ^~~~~
> > > > >     cc1: some warnings being treated as errors
> > > > 
> > > > This build failure is expected now given that we have allowed DEBUG_VM_PGTABLE
> > > > with EXPERT without platform requiring ARCH_HAS_DEBUG_VM_PGTABLE. This problem
> > > > i.e build failure caused without a platform __pgd(), is known to exist both on
> > > > ia64 and arm (32bit) platforms. Please refer https://lkml.org/lkml/2019/9/24/314
> > > > for details where this was discussed earlier.
> > > > 
> > > 
> > > I'd prefer not to merge a patch which is known to cause build
> > > regressions.  Is there some temporary thing we can do to prevent these
> > > errors until arch maintainers(?) get around to implementing the
> > > long-term fixes?
> > 
> > We could explicitly disable CONFIG_DEBUG_VM_PGTABLE on ia64 and arm platforms
> > which will ensure that others can still use the EXPERT path.
> > 
> > config DEBUG_VM_PGTABLE
> > 	bool "Debug arch page table for semantics compliance"
> > 	depends on MMU
> > 	depends on !(IA64 || ARM)
> > 	depends on ARCH_HAS_DEBUG_VM_PGTABLE || EXPERT
> > 	default n if !ARCH_HAS_DEBUG_VM_PGTABLE
> > 	default y if DEBUG_VM
> > 
> 
> On both ia32 and arm, the fix is trivial.
> 
> Can we include the fix within this patch, just the same way as the
> mm_p4d_folded() fix for x86 ?

Why should arm include a macro for something that nothing (apart from
this checker) requires?  If the checker requires it but the rest of
the kernel does not, it suggests that the checker isn't actually
correct, and the results can't be relied upon.

Christophe Leroy Feb. 10, 2020, 10:46 a.m. UTC | #7

Le 10/02/2020 à 11:02, Russell King - ARM Linux admin a écrit :
> On Mon, Feb 10, 2020 at 07:38:38AM +0100, Christophe Leroy wrote:
>>
>>
>> Le 10/02/2020 à 06:35, Anshuman Khandual a écrit :
>>>
>>>
>>> On 02/10/2020 10:22 AM, Andrew Morton wrote:
>>>> On Thu, 6 Feb 2020 13:49:35 +0530 Anshuman Khandual <[e-mail]> wrote:
>>>>
>>>>>
>>>>> On 02/06/2020 04:40 AM, kbuild test robot wrote:
>>>>>> Hi Anshuman,
>>>>>>
>>>>>> Thank you for the patch! Yet something to improve:
>>>>>>
>>>>>> [auto build test ERROR on powerpc/next]
>>>>>> [also build test ERROR on s390/features linus/master arc/for-next v5.5]
>>>>>> [cannot apply to mmotm/master tip/x86/core arm64/for-next/core next-20200205]
>>>>>> [if your patch is applied to the wrong git tree, please drop us a note to help
>>>>>> improve the system. BTW, we also suggest to use '--base' option to specify the
>>>>>> base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
>>>>>>
>>>>>> url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20200205-215507
>>>>>> base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
>>>>>> config: ia64-allmodconfig (attached as .config)
>>>>>> compiler: ia64-linux-gcc (GCC) 7.5.0
>>>>>> reproduce:
>>>>>>           wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>>>>>>           chmod +x ~/bin/make.cross
>>>>>>           # save the attached .config to linux build tree
>>>>>>           GCC_VERSION=7.5.0 make.cross ARCH=ia64
>>>>>>
>>>>>> If you fix the issue, kindly add following tag
>>>>>> Reported-by: kbuild test robot <[e-mail]>
>>>>>>
>>>>>> All error/warnings (new ones prefixed by >>):
>>>>>>
>>>>>>      In file included from include/asm-generic/pgtable-nopud.h:8:0,
>>>>>>                       from arch/ia64/include/asm/pgtable.h:586,
>>>>>>                       from include/linux/mm.h:99,
>>>>>>                       from include/linux/highmem.h:8,
>>>>>>                       from mm/debug_vm_pgtable.c:14:
>>>>>>      mm/debug_vm_pgtable.c: In function 'pud_clear_tests':
>>>>>>>> include/asm-generic/pgtable-nop4d-hack.h:47:32: error: implicit declaration of function '__pgd'; did you mean '__p4d'? [-Werror=implicit-function-declaration]
>>>>>>       #define __pud(x)    ((pud_t) { __pgd(x) })
>>>>>>                                      ^
>>>>>>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>>>>>>        pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>>>>>>              ^~~~~
>>>>>>>> include/asm-generic/pgtable-nop4d-hack.h:47:22: warning: missing braces around initializer [-Wmissing-braces]
>>>>>>       #define __pud(x)    ((pud_t) { __pgd(x) })
>>>>>>                            ^
>>>>>>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>>>>>>        pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>>>>>>              ^~~~~
>>>>>>      cc1: some warnings being treated as errors
>>>>>
>>>>> This build failure is expected now given that we have allowed DEBUG_VM_PGTABLE
>>>>> with EXPERT without platform requiring ARCH_HAS_DEBUG_VM_PGTABLE. This problem
>>>>> i.e build failure caused without a platform __pgd(), is known to exist both on
>>>>> ia64 and arm (32bit) platforms. Please refer https://lkml.org/lkml/2019/9/24/314
>>>>> for details where this was discussed earlier.
>>>>>
>>>>
>>>> I'd prefer not to merge a patch which is known to cause build
>>>> regressions.  Is there some temporary thing we can do to prevent these
>>>> errors until arch maintainers(?) get around to implementing the
>>>> long-term fixes?
>>>
>>> We could explicitly disable CONFIG_DEBUG_VM_PGTABLE on ia64 and arm platforms
>>> which will ensure that others can still use the EXPERT path.
>>>
>>> config DEBUG_VM_PGTABLE
>>> 	bool "Debug arch page table for semantics compliance"
>>> 	depends on MMU
>>> 	depends on !(IA64 || ARM)
>>> 	depends on ARCH_HAS_DEBUG_VM_PGTABLE || EXPERT
>>> 	default n if !ARCH_HAS_DEBUG_VM_PGTABLE
>>> 	default y if DEBUG_VM
>>>
>>
>> On both ia32 and arm, the fix is trivial.
>>
>> Can we include the fix within this patch, just the same way as the
>> mm_p4d_folded() fix for x86 ?
> 
> Why should arm include a macro for something that nothing (apart from
> this checker) requires?  If the checker requires it but the rest of
> the kernel does not, it suggests that the checker isn't actually
> correct, and the results can't be relied upon.
> 

As far as I can see, the problem is that arm opencodes part of the API 
instead of including asm-generic/pgtable-nopmd.h

Here, the ARM has 2 levels, ie only PGD and PTE. But instead of defining 
__pgd and __pte and getting everything else from asm-generic, it defines 
a __pmd then redefines the folded levels like the pud, etc ...

That's exactly what the checker aims at detecting: architectures than do 
not properly use the standard linux page table structures.

Christophe

Russell King (Oracle) Feb. 10, 2020, 11:06 a.m. UTC | #8

On Mon, Feb 10, 2020 at 11:46:23AM +0100, Christophe Leroy wrote:
> 
> 
> Le 10/02/2020 à 11:02, Russell King - ARM Linux admin a écrit :
> > On Mon, Feb 10, 2020 at 07:38:38AM +0100, Christophe Leroy wrote:
> > > 
> > > 
> > > Le 10/02/2020 à 06:35, Anshuman Khandual a écrit :
> > > > 
> > > > 
> > > > On 02/10/2020 10:22 AM, Andrew Morton wrote:
> > > > > On Thu, 6 Feb 2020 13:49:35 +0530 Anshuman Khandual <[e-mail]> wrote:
> > > > > 
> > > > > > 
> > > > > > On 02/06/2020 04:40 AM, kbuild test robot wrote:
> > > > > > > Hi Anshuman,
> > > > > > > 
> > > > > > > Thank you for the patch! Yet something to improve:
> > > > > > > 
> > > > > > > [auto build test ERROR on powerpc/next]
> > > > > > > [also build test ERROR on s390/features linus/master arc/for-next v5.5]
> > > > > > > [cannot apply to mmotm/master tip/x86/core arm64/for-next/core next-20200205]
> > > > > > > [if your patch is applied to the wrong git tree, please drop us a note to help
> > > > > > > improve the system. BTW, we also suggest to use '--base' option to specify the
> > > > > > > base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
> > > > > > > 
> > > > > > > url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20200205-215507
> > > > > > > base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
> > > > > > > config: ia64-allmodconfig (attached as .config)
> > > > > > > compiler: ia64-linux-gcc (GCC) 7.5.0
> > > > > > > reproduce:
> > > > > > >           wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
> > > > > > >           chmod +x ~/bin/make.cross
> > > > > > >           # save the attached .config to linux build tree
> > > > > > >           GCC_VERSION=7.5.0 make.cross ARCH=ia64
> > > > > > > 
> > > > > > > If you fix the issue, kindly add following tag
> > > > > > > Reported-by: kbuild test robot <[e-mail]>
> > > > > > > 
> > > > > > > All error/warnings (new ones prefixed by >>):
> > > > > > > 
> > > > > > >      In file included from include/asm-generic/pgtable-nopud.h:8:0,
> > > > > > >                       from arch/ia64/include/asm/pgtable.h:586,
> > > > > > >                       from include/linux/mm.h:99,
> > > > > > >                       from include/linux/highmem.h:8,
> > > > > > >                       from mm/debug_vm_pgtable.c:14:
> > > > > > >      mm/debug_vm_pgtable.c: In function 'pud_clear_tests':
> > > > > > > > > include/asm-generic/pgtable-nop4d-hack.h:47:32: error: implicit declaration of function '__pgd'; did you mean '__p4d'? [-Werror=implicit-function-declaration]
> > > > > > >       #define __pud(x)    ((pud_t) { __pgd(x) })
> > > > > > >                                      ^
> > > > > > > > > mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
> > > > > > >        pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
> > > > > > >              ^~~~~
> > > > > > > > > include/asm-generic/pgtable-nop4d-hack.h:47:22: warning: missing braces around initializer [-Wmissing-braces]
> > > > > > >       #define __pud(x)    ((pud_t) { __pgd(x) })
> > > > > > >                            ^
> > > > > > > > > mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
> > > > > > >        pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
> > > > > > >              ^~~~~
> > > > > > >      cc1: some warnings being treated as errors
> > > > > > 
> > > > > > This build failure is expected now given that we have allowed DEBUG_VM_PGTABLE
> > > > > > with EXPERT without platform requiring ARCH_HAS_DEBUG_VM_PGTABLE. This problem
> > > > > > i.e build failure caused without a platform __pgd(), is known to exist both on
> > > > > > ia64 and arm (32bit) platforms. Please refer https://lkml.org/lkml/2019/9/24/314
> > > > > > for details where this was discussed earlier.
> > > > > > 
> > > > > 
> > > > > I'd prefer not to merge a patch which is known to cause build
> > > > > regressions.  Is there some temporary thing we can do to prevent these
> > > > > errors until arch maintainers(?) get around to implementing the
> > > > > long-term fixes?
> > > > 
> > > > We could explicitly disable CONFIG_DEBUG_VM_PGTABLE on ia64 and arm platforms
> > > > which will ensure that others can still use the EXPERT path.
> > > > 
> > > > config DEBUG_VM_PGTABLE
> > > > 	bool "Debug arch page table for semantics compliance"
> > > > 	depends on MMU
> > > > 	depends on !(IA64 || ARM)
> > > > 	depends on ARCH_HAS_DEBUG_VM_PGTABLE || EXPERT
> > > > 	default n if !ARCH_HAS_DEBUG_VM_PGTABLE
> > > > 	default y if DEBUG_VM
> > > > 
> > > 
> > > On both ia32 and arm, the fix is trivial.
> > > 
> > > Can we include the fix within this patch, just the same way as the
> > > mm_p4d_folded() fix for x86 ?
> > 
> > Why should arm include a macro for something that nothing (apart from
> > this checker) requires?  If the checker requires it but the rest of
> > the kernel does not, it suggests that the checker isn't actually
> > correct, and the results can't be relied upon.
> > 
> 
> As far as I can see, the problem is that arm opencodes part of the API
> instead of including asm-generic/pgtable-nopmd.h
> 
> Here, the ARM has 2 levels, ie only PGD and PTE. But instead of defining
> __pgd and __pte and getting everything else from asm-generic, it defines a
> __pmd then redefines the folded levels like the pud, etc ...
> 
> That's exactly what the checker aims at detecting: architectures than do not
> properly use the standard linux page table structures.

There are good reasons for the way ARM does stuff.  The generic crap was
written without regard for the circumstances that ARM has, and thus is
entirely unsuitable for 32-bit ARM.

Anshuman Khandual Feb. 11, 2020, 2:25 a.m. UTC | #9

On 02/10/2020 04:36 PM, Russell King - ARM Linux admin wrote:
> On Mon, Feb 10, 2020 at 11:46:23AM +0100, Christophe Leroy wrote:
>>
>>
>> Le 10/02/2020 à 11:02, Russell King - ARM Linux admin a écrit :
>>> On Mon, Feb 10, 2020 at 07:38:38AM +0100, Christophe Leroy wrote:
>>>>
>>>>
>>>> Le 10/02/2020 à 06:35, Anshuman Khandual a écrit :
>>>>>
>>>>>
>>>>> On 02/10/2020 10:22 AM, Andrew Morton wrote:
>>>>>> On Thu, 6 Feb 2020 13:49:35 +0530 Anshuman Khandual <[e-mail]> wrote:
>>>>>>
>>>>>>>
>>>>>>> On 02/06/2020 04:40 AM, kbuild test robot wrote:
>>>>>>>> Hi Anshuman,
>>>>>>>>
>>>>>>>> Thank you for the patch! Yet something to improve:
>>>>>>>>
>>>>>>>> [auto build test ERROR on powerpc/next]
>>>>>>>> [also build test ERROR on s390/features linus/master arc/for-next v5.5]
>>>>>>>> [cannot apply to mmotm/master tip/x86/core arm64/for-next/core next-20200205]
>>>>>>>> [if your patch is applied to the wrong git tree, please drop us a note to help
>>>>>>>> improve the system. BTW, we also suggest to use '--base' option to specify the
>>>>>>>> base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
>>>>>>>>
>>>>>>>> url:    https://github.com/0day-ci/linux/commits/Anshuman-Khandual/mm-debug-Add-tests-validating-architecture-page-table-helpers/20200205-215507
>>>>>>>> base:   https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git next
>>>>>>>> config: ia64-allmodconfig (attached as .config)
>>>>>>>> compiler: ia64-linux-gcc (GCC) 7.5.0
>>>>>>>> reproduce:
>>>>>>>>           wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
>>>>>>>>           chmod +x ~/bin/make.cross
>>>>>>>>           # save the attached .config to linux build tree
>>>>>>>>           GCC_VERSION=7.5.0 make.cross ARCH=ia64
>>>>>>>>
>>>>>>>> If you fix the issue, kindly add following tag
>>>>>>>> Reported-by: kbuild test robot <[e-mail]>
>>>>>>>>
>>>>>>>> All error/warnings (new ones prefixed by >>):
>>>>>>>>
>>>>>>>>      In file included from include/asm-generic/pgtable-nopud.h:8:0,
>>>>>>>>                       from arch/ia64/include/asm/pgtable.h:586,
>>>>>>>>                       from include/linux/mm.h:99,
>>>>>>>>                       from include/linux/highmem.h:8,
>>>>>>>>                       from mm/debug_vm_pgtable.c:14:
>>>>>>>>      mm/debug_vm_pgtable.c: In function 'pud_clear_tests':
>>>>>>>>>> include/asm-generic/pgtable-nop4d-hack.h:47:32: error: implicit declaration of function '__pgd'; did you mean '__p4d'? [-Werror=implicit-function-declaration]
>>>>>>>>       #define __pud(x)    ((pud_t) { __pgd(x) })
>>>>>>>>                                      ^
>>>>>>>>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>>>>>>>>        pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>>>>>>>>              ^~~~~
>>>>>>>>>> include/asm-generic/pgtable-nop4d-hack.h:47:22: warning: missing braces around initializer [-Wmissing-braces]
>>>>>>>>       #define __pud(x)    ((pud_t) { __pgd(x) })
>>>>>>>>                            ^
>>>>>>>>>> mm/debug_vm_pgtable.c:141:8: note: in expansion of macro '__pud'
>>>>>>>>        pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
>>>>>>>>              ^~~~~
>>>>>>>>      cc1: some warnings being treated as errors
>>>>>>>
>>>>>>> This build failure is expected now given that we have allowed DEBUG_VM_PGTABLE
>>>>>>> with EXPERT without platform requiring ARCH_HAS_DEBUG_VM_PGTABLE. This problem
>>>>>>> i.e build failure caused without a platform __pgd(), is known to exist both on
>>>>>>> ia64 and arm (32bit) platforms. Please refer https://lkml.org/lkml/2019/9/24/314
>>>>>>> for details where this was discussed earlier.
>>>>>>>
>>>>>>
>>>>>> I'd prefer not to merge a patch which is known to cause build
>>>>>> regressions.  Is there some temporary thing we can do to prevent these
>>>>>> errors until arch maintainers(?) get around to implementing the
>>>>>> long-term fixes?
>>>>>
>>>>> We could explicitly disable CONFIG_DEBUG_VM_PGTABLE on ia64 and arm platforms
>>>>> which will ensure that others can still use the EXPERT path.
>>>>>
>>>>> config DEBUG_VM_PGTABLE
>>>>> 	bool "Debug arch page table for semantics compliance"
>>>>> 	depends on MMU
>>>>> 	depends on !(IA64 || ARM)
>>>>> 	depends on ARCH_HAS_DEBUG_VM_PGTABLE || EXPERT
>>>>> 	default n if !ARCH_HAS_DEBUG_VM_PGTABLE
>>>>> 	default y if DEBUG_VM
>>>>>
>>>>
>>>> On both ia32 and arm, the fix is trivial.
>>>>
>>>> Can we include the fix within this patch, just the same way as the
>>>> mm_p4d_folded() fix for x86 ?
>>>
>>> Why should arm include a macro for something that nothing (apart from
>>> this checker) requires?  If the checker requires it but the rest of
>>> the kernel does not, it suggests that the checker isn't actually
>>> correct, and the results can't be relied upon.
>>>
>>
>> As far as I can see, the problem is that arm opencodes part of the API
>> instead of including asm-generic/pgtable-nopmd.h
>>
>> Here, the ARM has 2 levels, ie only PGD and PTE. But instead of defining
>> __pgd and __pte and getting everything else from asm-generic, it defines a
>> __pmd then redefines the folded levels like the pud, etc ...
>>
>> That's exactly what the checker aims at detecting: architectures than do not
>> properly use the standard linux page table structures.
> 
> There are good reasons for the way ARM does stuff.  The generic crap was
> written without regard for the circumstances that ARM has, and thus is
> entirely unsuitable for 32-bit ARM.

Since we dont have an agreement here, lets just settle with disabling the
test for now on platforms where the build fails. CONFIG_EXPERT is enabling
this test for better adaptability and coverage, hence how about re framing
the config like this ? This at the least conveys the fact that EXPERT only
works when platform is neither IA64 or ARM.

config DEBUG_VM_PGTABLE
	bool "Debug arch page table for semantics compliance"
	depends on MMU
	depends on ARCH_HAS_DEBUG_VM_PGTABLE || (EXPERT &&  !(IA64 || ARM))
	default n if !ARCH_HAS_DEBUG_VM_PGTABLE
	default y if DEBUG_VM

Christophe Leroy Feb. 11, 2020, 5:33 a.m. UTC | #10

Le 11/02/2020 à 03:25, Anshuman Khandual a écrit :
> 
> 
> On 02/10/2020 04:36 PM, Russell King - ARM Linux admin wrote:
>> There are good reasons for the way ARM does stuff.  The generic crap was
>> written without regard for the circumstances that ARM has, and thus is
>> entirely unsuitable for 32-bit ARM.
> 
> Since we dont have an agreement here, lets just settle with disabling the
> test for now on platforms where the build fails. CONFIG_EXPERT is enabling
> this test for better adaptability and coverage, hence how about re framing
> the config like this ? This at the least conveys the fact that EXPERT only
> works when platform is neither IA64 or ARM.

Agreed

> 
> config DEBUG_VM_PGTABLE
> 	bool "Debug arch page table for semantics compliance"
> 	depends on MMU
> 	depends on ARCH_HAS_DEBUG_VM_PGTABLE || (EXPERT &&  !(IA64 || ARM))

I think it's maybe better to have a dedicated depends line:

depends on !IA64 && !ARM
depends on ARCH_HAS_DEBUG_VM_PGTABLE || EXPERT

The day arm and/or ia64 is ready for building the test, we can remove 
that depends.

> 	default n if !ARCH_HAS_DEBUG_VM_PGTABLE
> 	default y if DEBUG_VM
> 

Christophe

Russell King (Oracle) Feb. 11, 2020, 9:24 a.m. UTC | #11

On Tue, Feb 11, 2020 at 06:33:47AM +0100, Christophe Leroy wrote:
> 
> 
> Le 11/02/2020 à 03:25, Anshuman Khandual a écrit :
> > 
> > 
> > On 02/10/2020 04:36 PM, Russell King - ARM Linux admin wrote:
> > > There are good reasons for the way ARM does stuff.  The generic crap was
> > > written without regard for the circumstances that ARM has, and thus is
> > > entirely unsuitable for 32-bit ARM.
> > 
> > Since we dont have an agreement here, lets just settle with disabling the
> > test for now on platforms where the build fails. CONFIG_EXPERT is enabling
> > this test for better adaptability and coverage, hence how about re framing
> > the config like this ? This at the least conveys the fact that EXPERT only
> > works when platform is neither IA64 or ARM.
> 
> Agreed
> 
> > 
> > config DEBUG_VM_PGTABLE
> > 	bool "Debug arch page table for semantics compliance"
> > 	depends on MMU
> > 	depends on ARCH_HAS_DEBUG_VM_PGTABLE || (EXPERT &&  !(IA64 || ARM))
> 
> I think it's maybe better to have a dedicated depends line:
> 
> depends on !IA64 && !ARM
> depends on ARCH_HAS_DEBUG_VM_PGTABLE || EXPERT
> 
> The day arm and/or ia64 is ready for building the test, we can remove that
> depends.

Never going to happen as its technically infeasible, sorry.

Patch

diff --git a/Documentation/features/debug/debug-vm-pgtable/arch-support.txt b/Documentation/features/debug/debug-vm-pgtable/arch-support.txt
new file mode 100644
index 000000000000..64d0f9b15c49
--- /dev/null
+++ b/Documentation/features/debug/debug-vm-pgtable/arch-support.txt
@@ -0,0 +1,35 @@ 
+#
+# Feature name:          debug-vm-pgtable
+#         Kconfig:       ARCH_HAS_DEBUG_VM_PGTABLE
+#         description:   arch supports pgtable tests for semantics compliance
+#
+    -----------------------
+    |         arch |status|
+    -----------------------
+    |       alpha: | TODO |
+    |         arc: |  ok  |
+    |         arm: | TODO |
+    |       arm64: |  ok  |
+    |         c6x: | TODO |
+    |        csky: | TODO |
+    |       h8300: | TODO |
+    |     hexagon: | TODO |
+    |        ia64: | TODO |
+    |        m68k: | TODO |
+    |  microblaze: | TODO |
+    |        mips: | TODO |
+    |       nds32: | TODO |
+    |       nios2: | TODO |
+    |    openrisc: | TODO |
+    |      parisc: | TODO |
+    |  powerpc/32: |  ok  |
+    |  powerpc/64: | TODO |
+    |       riscv: | TODO |
+    |        s390: |  ok  |
+    |          sh: | TODO |
+    |       sparc: | TODO |
+    |          um: | TODO |
+    |   unicore32: | TODO |
+    |         x86: |  ok  |
+    |      xtensa: | TODO |
+    -----------------------
diff --git a/arch/arc/Kconfig b/arch/arc/Kconfig
index 26108ea785c2..6a76f3b609b2 100644
--- a/arch/arc/Kconfig
+++ b/arch/arc/Kconfig
@@ -6,6 +6,7 @@ 
 config ARC
 	def_bool y
 	select ARC_TIMERS
+	select ARCH_HAS_DEBUG_VM_PGTABLE
 	select ARCH_HAS_DMA_PREP_COHERENT
 	select ARCH_HAS_PTE_SPECIAL
 	select ARCH_HAS_SETUP_DMA_OPS
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index e688dfad0b72..876c9a672db1 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -11,6 +11,7 @@  config ARM64
 	select ACPI_PPTT if ACPI
 	select ARCH_CLOCKSOURCE_DATA
 	select ARCH_HAS_DEBUG_VIRTUAL
+	select ARCH_HAS_DEBUG_VM_PGTABLE
 	select ARCH_HAS_DEVMEM_IS_ALLOWED
 	select ARCH_HAS_DMA_PREP_COHERENT
 	select ARCH_HAS_ACPI_TABLE_UPGRADE if ACPI
diff --git a/arch/powerpc/Kconfig b/arch/powerpc/Kconfig
index e2a412113359..e229e2a86518 100644
--- a/arch/powerpc/Kconfig
+++ b/arch/powerpc/Kconfig
@@ -120,6 +120,7 @@  config PPC
 	#
 	select ARCH_32BIT_OFF_T if PPC32
 	select ARCH_HAS_DEBUG_VIRTUAL
+	select ARCH_HAS_DEBUG_VM_PGTABLE if PPC32
 	select ARCH_HAS_DEVMEM_IS_ALLOWED
 	select ARCH_HAS_ELF_RANDOMIZE
 	select ARCH_HAS_FORTIFY_SOURCE
diff --git a/arch/s390/Kconfig b/arch/s390/Kconfig
index bc88841d335d..2ea962ead27e 100644
--- a/arch/s390/Kconfig
+++ b/arch/s390/Kconfig
@@ -59,6 +59,7 @@  config KASAN_SHADOW_OFFSET
 config S390
 	def_bool y
 	select ARCH_BINFMT_ELF_STATE
+	select ARCH_HAS_DEBUG_VM_PGTABLE
 	select ARCH_HAS_DEVMEM_IS_ALLOWED
 	select ARCH_HAS_ELF_RANDOMIZE
 	select ARCH_HAS_FORTIFY_SOURCE
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index 5e8949953660..218536a4581d 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -61,6 +61,7 @@  config X86
 	select ARCH_CLOCKSOURCE_INIT
 	select ARCH_HAS_ACPI_TABLE_UPGRADE	if ACPI
 	select ARCH_HAS_DEBUG_VIRTUAL
+	select ARCH_HAS_DEBUG_VM_PGTABLE	if !X86_PAE
 	select ARCH_HAS_DEVMEM_IS_ALLOWED
 	select ARCH_HAS_ELF_RANDOMIZE
 	select ARCH_HAS_FAST_MULTIPLIER
diff --git a/arch/x86/include/asm/pgtable_64.h b/arch/x86/include/asm/pgtable_64.h
index 0b6c4042942a..fb0e76d254b3 100644
--- a/arch/x86/include/asm/pgtable_64.h
+++ b/arch/x86/include/asm/pgtable_64.h
@@ -53,6 +53,12 @@  static inline void sync_initial_page_table(void) { }
 
 struct mm_struct;
 
+#define mm_p4d_folded mm_p4d_folded
+static inline bool mm_p4d_folded(struct mm_struct *mm)
+{
+	return !pgtable_l5_enabled();
+}
+
 void set_pte_vaddr_p4d(p4d_t *p4d_page, unsigned long vaddr, pte_t new_pte);
 void set_pte_vaddr_pud(pud_t *pud_page, unsigned long vaddr, pte_t new_pte);
 
diff --git a/include/linux/mmdebug.h b/include/linux/mmdebug.h
index 2ad72d2c8cc5..5339aa14b749 100644
--- a/include/linux/mmdebug.h
+++ b/include/linux/mmdebug.h
@@ -64,4 +64,9 @@  void dump_mm(const struct mm_struct *mm);
 #define VM_BUG_ON_PGFLAGS(cond, page) BUILD_BUG_ON_INVALID(cond)
 #endif
 
+#ifdef CONFIG_DEBUG_VM_PGTABLE
+void debug_vm_pgtable(void);
+#else
+static inline void debug_vm_pgtable(void) { }
+#endif
 #endif
diff --git a/init/main.c b/init/main.c
index da1bc0b60a7d..e2999239bb8c 100644
--- a/init/main.c
+++ b/init/main.c
@@ -93,6 +93,7 @@ 
 #include <linux/rodata_test.h>
 #include <linux/jump_label.h>
 #include <linux/mem_encrypt.h>
+#include <linux/mmdebug.h>
 
 #include <asm/io.h>
 #include <asm/bugs.h>
@@ -1110,6 +1111,7 @@  static int __ref kernel_init(void *unused)
 	kernel_init_freeable();
 	/* need to finish all async __init code before freeing the memory */
 	async_synchronize_full();
+	debug_vm_pgtable();
 	ftrace_free_init_mem();
 	free_initmem();
 	mark_readonly();
diff --git a/lib/Kconfig.debug b/lib/Kconfig.debug
index 5ffe144c9794..64c80f7af904 100644
--- a/lib/Kconfig.debug
+++ b/lib/Kconfig.debug
@@ -653,6 +653,12 @@  config SCHED_STACK_END_CHECK
 	  data corruption or a sporadic crash at a later stage once the region
 	  is examined. The runtime overhead introduced is minimal.
 
+config ARCH_HAS_DEBUG_VM_PGTABLE
+	bool
+	help
+	  An architecture should select this when it can successfully
+	  build and run DEBUG_VM_PGTABLE.
+
 config DEBUG_VM
 	bool "Debug VM"
 	depends on DEBUG_KERNEL
@@ -688,6 +694,22 @@  config DEBUG_VM_PGFLAGS
 
 	  If unsure, say N.
 
+config DEBUG_VM_PGTABLE
+	bool "Debug arch page table for semantics compliance"
+	depends on MMU
+	depends on ARCH_HAS_DEBUG_VM_PGTABLE || EXPERT
+	default n if !ARCH_HAS_DEBUG_VM_PGTABLE
+	default y if DEBUG_VM
+	help
+	  This option provides a debug method which can be used to test
+	  architecture page table helper functions on various platforms in
+	  verifying if they comply with expected generic MM semantics. This
+	  will help architecture code in making sure that any changes or
+	  new additions of these helpers still conform to expected
+	  semantics of the generic MM.
+
+	  If unsure, say N.
+
 config ARCH_HAS_DEBUG_VIRTUAL
 	bool
 
diff --git a/mm/Makefile b/mm/Makefile
index 1937cc251883..eba423a26b19 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -86,6 +86,7 @@  obj-$(CONFIG_HWPOISON_INJECT) += hwpoison-inject.o
 obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o
 obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak-test.o
 obj-$(CONFIG_DEBUG_RODATA_TEST) += rodata_test.o
+obj-$(CONFIG_DEBUG_VM_PGTABLE) += debug_vm_pgtable.o
 obj-$(CONFIG_PAGE_OWNER) += page_owner.o
 obj-$(CONFIG_CLEANCACHE) += cleancache.o
 obj-$(CONFIG_MEMORY_ISOLATION) += page_isolation.o
diff --git a/mm/debug_vm_pgtable.c b/mm/debug_vm_pgtable.c
new file mode 100644
index 000000000000..0f37f32d15f1
--- /dev/null
+++ b/mm/debug_vm_pgtable.c
@@ -0,0 +1,388 @@ 
+// SPDX-License-Identifier: GPL-2.0-only
+/*
+ * This kernel test validates architecture page table helpers and
+ * accessors and helps in verifying their continued compliance with
+ * expected generic MM semantics.
+ *
+ * Copyright (C) 2019 ARM Ltd.
+ *
+ * Author: Anshuman Khandual <[e-mail]>
+ */
+#define pr_fmt(fmt) "debug_vm_pgtable: %s: " fmt, __func__
+
+#include <linux/gfp.h>
+#include <linux/highmem.h>
+#include <linux/hugetlb.h>
+#include <linux/kernel.h>
+#include <linux/kconfig.h>
+#include <linux/mm.h>
+#include <linux/mman.h>
+#include <linux/mm_types.h>
+#include <linux/module.h>
+#include <linux/pfn_t.h>
+#include <linux/printk.h>
+#include <linux/random.h>
+#include <linux/spinlock.h>
+#include <linux/swap.h>
+#include <linux/swapops.h>
+#include <linux/start_kernel.h>
+#include <linux/sched/mm.h>
+#include <asm/pgalloc.h>
+#include <asm/pgtable.h>
+
+/*
+ * Basic operations
+ *
+ * mkold(entry)			= An old and not a young entry
+ * mkyoung(entry)		= A young and not an old entry
+ * mkdirty(entry)		= A dirty and not a clean entry
+ * mkclean(entry)		= A clean and not a dirty entry
+ * mkwrite(entry)		= A write and not a write protected entry
+ * wrprotect(entry)		= A write protected and not a write entry
+ * pxx_bad(entry)		= A mapped and non-table entry
+ * pxx_same(entry1, entry2)	= Both entries hold the exact same value
+ */
+#define VMFLAGS	(VM_READ|VM_WRITE|VM_EXEC)
+
+/*
+ * On s390 platform, the lower 12 bits are used to identify given page table
+ * entry type and for other arch specific requirements. But these bits might
+ * affect the ability to clear entries with pxx_clear(). So while loading up
+ * the entries skip all lower 12 bits in order to accommodate s390 platform.
+ * It does not have affect any other platform.
+ */
+#define RANDOM_ORVALUE	(0xfffffffffffff000UL)
+#define RANDOM_NZVALUE	(0xff)
+
+static void __init pte_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+	pte_t pte = pfn_pte(pfn, prot);
+
+	WARN_ON(!pte_same(pte, pte));
+	WARN_ON(!pte_young(pte_mkyoung(pte)));
+	WARN_ON(!pte_dirty(pte_mkdirty(pte)));
+	WARN_ON(!pte_write(pte_mkwrite(pte)));
+	WARN_ON(pte_young(pte_mkold(pte)));
+	WARN_ON(pte_dirty(pte_mkclean(pte)));
+	WARN_ON(pte_write(pte_wrprotect(pte)));
+}
+
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+static void __init pmd_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+	pmd_t pmd = pfn_pmd(pfn, prot);
+
+	WARN_ON(!pmd_same(pmd, pmd));
+	WARN_ON(!pmd_young(pmd_mkyoung(pmd)));
+	WARN_ON(!pmd_dirty(pmd_mkdirty(pmd)));
+	WARN_ON(!pmd_write(pmd_mkwrite(pmd)));
+	WARN_ON(pmd_young(pmd_mkold(pmd)));
+	WARN_ON(pmd_dirty(pmd_mkclean(pmd)));
+	WARN_ON(pmd_write(pmd_wrprotect(pmd)));
+	/*
+	 * A huge page does not point to next level page table
+	 * entry. Hence this must qualify as pmd_bad().
+	 */
+	WARN_ON(!pmd_bad(pmd_mkhuge(pmd)));
+}
+
+#ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD
+static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+	pud_t pud = pfn_pud(pfn, prot);
+
+	WARN_ON(!pud_same(pud, pud));
+	WARN_ON(!pud_young(pud_mkyoung(pud)));
+	WARN_ON(!pud_write(pud_mkwrite(pud)));
+	WARN_ON(pud_write(pud_wrprotect(pud)));
+	WARN_ON(pud_young(pud_mkold(pud)));
+
+	if (mm_pmd_folded(mm) || __is_defined(ARCH_HAS_4LEVEL_HACK))
+		return;
+
+	/*
+	 * A huge page does not point to next level page table
+	 * entry. Hence this must qualify as pud_bad().
+	 */
+	WARN_ON(!pud_bad(pud_mkhuge(pud)));
+}
+#else
+static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot) { }
+#endif
+#else
+static void __init pmd_basic_tests(unsigned long pfn, pgprot_t prot) { }
+static void __init pud_basic_tests(unsigned long pfn, pgprot_t prot) { }
+#endif
+
+static void __init p4d_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+	p4d_t p4d;
+
+	memset(&p4d, RANDOM_NZVALUE, sizeof(p4d_t));
+	WARN_ON(!p4d_same(p4d, p4d));
+}
+
+static void __init pgd_basic_tests(unsigned long pfn, pgprot_t prot)
+{
+	pgd_t pgd;
+
+	memset(&pgd, RANDOM_NZVALUE, sizeof(pgd_t));
+	WARN_ON(!pgd_same(pgd, pgd));
+}
+
+#ifndef __ARCH_HAS_4LEVEL_HACK
+static void __init pud_clear_tests(struct mm_struct *mm, pud_t *pudp)
+{
+	pud_t pud = READ_ONCE(*pudp);
+
+	if (mm_pmd_folded(mm))
+		return;
+
+	pud = __pud(pud_val(pud) | RANDOM_ORVALUE);
+	WRITE_ONCE(*pudp, pud);
+	pud_clear(pudp);
+	pud = READ_ONCE(*pudp);
+	WARN_ON(!pud_none(pud));
+}
+
+static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
+				      pmd_t *pmdp)
+{
+	pud_t pud;
+
+	if (mm_pmd_folded(mm))
+		return;
+	/*
+	 * This entry points to next level page table page.
+	 * Hence this must not qualify as pud_bad().
+	 */
+	pmd_clear(pmdp);
+	pud_clear(pudp);
+	pud_populate(mm, pudp, pmdp);
+	pud = READ_ONCE(*pudp);
+	WARN_ON(pud_bad(pud));
+}
+#else
+static void __init pud_clear_tests(struct mm_struct *mm, pud_t *pudp) { }
+static void __init pud_populate_tests(struct mm_struct *mm, pud_t *pudp,
+				      pmd_t *pmdp)
+{
+}
+#endif
+
+#ifndef __ARCH_HAS_5LEVEL_HACK
+static void __init p4d_clear_tests(struct mm_struct *mm, p4d_t *p4dp)
+{
+	p4d_t p4d = READ_ONCE(*p4dp);
+
+	if (mm_pud_folded(mm))
+		return;
+
+	p4d = __p4d(p4d_val(p4d) | RANDOM_ORVALUE);
+	WRITE_ONCE(*p4dp, p4d);
+	p4d_clear(p4dp);
+	p4d = READ_ONCE(*p4dp);
+	WARN_ON(!p4d_none(p4d));
+}
+
+static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
+				      pud_t *pudp)
+{
+	p4d_t p4d;
+
+	if (mm_pud_folded(mm))
+		return;
+
+	/*
+	 * This entry points to next level page table page.
+	 * Hence this must not qualify as p4d_bad().
+	 */
+	pud_clear(pudp);
+	p4d_clear(p4dp);
+	p4d_populate(mm, p4dp, pudp);
+	p4d = READ_ONCE(*p4dp);
+	WARN_ON(p4d_bad(p4d));
+}
+
+static void __init pgd_clear_tests(struct mm_struct *mm, pgd_t *pgdp)
+{
+	pgd_t pgd = READ_ONCE(*pgdp);
+
+	if (mm_p4d_folded(mm))
+		return;
+
+	pgd = __pgd(pgd_val(pgd) | RANDOM_ORVALUE);
+	WRITE_ONCE(*pgdp, pgd);
+	pgd_clear(pgdp);
+	pgd = READ_ONCE(*pgdp);
+	WARN_ON(!pgd_none(pgd));
+}
+
+static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
+				      p4d_t *p4dp)
+{
+	pgd_t pgd;
+
+	if (mm_p4d_folded(mm))
+		return;
+
+	/*
+	 * This entry points to next level page table page.
+	 * Hence this must not qualify as pgd_bad().
+	 */
+	p4d_clear(p4dp);
+	pgd_clear(pgdp);
+	pgd_populate(mm, pgdp, p4dp);
+	pgd = READ_ONCE(*pgdp);
+	WARN_ON(pgd_bad(pgd));
+}
+#else
+static void __init p4d_clear_tests(struct mm_struct *mm, p4d_t *p4dp) { }
+static void __init pgd_clear_tests(struct mm_struct *mm, pgd_t *pgdp) { }
+static void __init p4d_populate_tests(struct mm_struct *mm, p4d_t *p4dp,
+				      pud_t *pudp)
+{
+}
+static void __init pgd_populate_tests(struct mm_struct *mm, pgd_t *pgdp,
+				      p4d_t *p4dp)
+{
+}
+#endif
+
+static void __init pte_clear_tests(struct mm_struct *mm, pte_t *ptep)
+{
+	pte_t pte = READ_ONCE(*ptep);
+
+	pte = __pte(pte_val(pte) | RANDOM_ORVALUE);
+	WRITE_ONCE(*ptep, pte);
+	pte_clear(mm, 0, ptep);
+	pte = READ_ONCE(*ptep);
+	WARN_ON(!pte_none(pte));
+}
+
+static void __init pmd_clear_tests(struct mm_struct *mm, pmd_t *pmdp)
+{
+	pmd_t pmd = READ_ONCE(*pmdp);
+
+	pmd = __pmd(pmd_val(pmd) | RANDOM_ORVALUE);
+	WRITE_ONCE(*pmdp, pmd);
+	pmd_clear(pmdp);
+	pmd = READ_ONCE(*pmdp);
+	WARN_ON(!pmd_none(pmd));
+}
+
+static void __init pmd_populate_tests(struct mm_struct *mm, pmd_t *pmdp,
+				      pgtable_t pgtable)
+{
+	pmd_t pmd;
+
+	/*
+	 * This entry points to next level page table page.
+	 * Hence this must not qualify as pmd_bad().
+	 */
+	pmd_clear(pmdp);
+	pmd_populate(mm, pmdp, pgtable);
+	pmd = READ_ONCE(*pmdp);
+	WARN_ON(pmd_bad(pmd));
+}
+
+static unsigned long __init get_random_vaddr(void)
+{
+	unsigned long random_vaddr, random_pages, total_user_pages;
+
+	total_user_pages = (TASK_SIZE - FIRST_USER_ADDRESS) / PAGE_SIZE;
+
+	random_pages = get_random_long() % total_user_pages;
+	random_vaddr = FIRST_USER_ADDRESS + random_pages * PAGE_SIZE;
+
+	return random_vaddr;
+}
+
+void __init debug_vm_pgtable(void)
+{
+	struct mm_struct *mm;
+	pgd_t *pgdp;
+	p4d_t *p4dp, *saved_p4dp;
+	pud_t *pudp, *saved_pudp;
+	pmd_t *pmdp, *saved_pmdp, pmd;
+	pte_t *ptep;
+	pgtable_t saved_ptep;
+	pgprot_t prot;
+	phys_addr_t paddr;
+	unsigned long vaddr, pte_aligned, pmd_aligned;
+	unsigned long pud_aligned, p4d_aligned, pgd_aligned;
+
+	pr_info("Validating architecture page table helpers\n");
+	prot = vm_get_page_prot(VMFLAGS);
+	vaddr = get_random_vaddr();
+	mm = mm_alloc();
+	if (!mm) {
+		pr_err("mm_struct allocation failed\n");
+		return;
+	}
+
+	/*
+	 * PFN for mapping at PTE level is determined from a standard kernel
+	 * text symbol. But pfns for higher page table levels are derived by
+	 * masking lower bits of this real pfn. These derived pfns might not
+	 * exist on the platform but that does not really matter as pfn_pxx()
+	 * helpers will still create appropriate entries for the test. This
+	 * helps avoid large memory block allocations to be used for mapping
+	 * at higher page table levels.
+	 */
+	paddr = __pa(&start_kernel);
+
+	pte_aligned = (paddr & PAGE_MASK) >> PAGE_SHIFT;
+	pmd_aligned = (paddr & PMD_MASK) >> PAGE_SHIFT;
+	pud_aligned = (paddr & PUD_MASK) >> PAGE_SHIFT;
+	p4d_aligned = (paddr & P4D_MASK) >> PAGE_SHIFT;
+	pgd_aligned = (paddr & PGDIR_MASK) >> PAGE_SHIFT;
+	WARN_ON(!pfn_valid(pte_aligned));
+
+	pgdp = pgd_offset(mm, vaddr);
+	p4dp = p4d_alloc(mm, pgdp, vaddr);
+	pudp = pud_alloc(mm, p4dp, vaddr);
+	pmdp = pmd_alloc(mm, pudp, vaddr);
+	ptep = pte_alloc_map(mm, pmdp, vaddr);
+
+	/*
+	 * Save all the page table page addresses as the page table
+	 * entries will be used for testing with random or garbage
+	 * values. These saved addresses will be used for freeing
+	 * page table pages.
+	 */
+	pmd = READ_ONCE(*pmdp);
+	saved_p4dp = p4d_offset(pgdp, 0UL);
+	saved_pudp = pud_offset(p4dp, 0UL);
+	saved_pmdp = pmd_offset(pudp, 0UL);
+	saved_ptep = pmd_pgtable(pmd);
+
+	pte_basic_tests(pte_aligned, prot);
+	pmd_basic_tests(pmd_aligned, prot);
+	pud_basic_tests(pud_aligned, prot);
+	p4d_basic_tests(p4d_aligned, prot);
+	pgd_basic_tests(pgd_aligned, prot);
+
+	pte_clear_tests(mm, ptep);
+	pmd_clear_tests(mm, pmdp);
+	pud_clear_tests(mm, pudp);
+	p4d_clear_tests(mm, p4dp);
+	pgd_clear_tests(mm, pgdp);
+
+	pte_unmap(ptep);
+
+	pmd_populate_tests(mm, pmdp, saved_ptep);
+	pud_populate_tests(mm, pudp, saved_pmdp);
+	p4d_populate_tests(mm, p4dp, saved_pudp);
+	pgd_populate_tests(mm, pgdp, saved_p4dp);
+
+	p4d_free(mm, saved_p4dp);
+	pud_free(mm, saved_pudp);
+	pmd_free(mm, saved_pmdp);
+	pte_free(mm, saved_ptep);
+
+	mm_dec_nr_puds(mm);
+	mm_dec_nr_pmds(mm);
+	mm_dec_nr_ptes(mm);
+	mmdrop(mm);
+}