| Message ID | 20231220054123.1266001-1-maskray@google.com (mailing list archive) |
|---|---|
| State | New |
| Headers | show |
| Series | mm: remove VM_EXEC requirement for THP eligibility | expand |
On Tue, Dec 19, 2023 at 9:41 PM Fangrui Song <maskray@google.com> wrote: > > Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP > checking in transparent_hugepage_enabled()") introduced the VM_EXEC > requirement, which is not strictly needed. > > lld's default --rosegment option and GNU ld's -z separate-code option > (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD > segment without the PF_X flag, which should be eligible for THP. > > Certain architectures support medium and large code models, where > .lrodata may be placed in a separate read-only PT_LOAD segment, which > should be eligible for THP as well. Yeah, it doesn't have to be VM_EXEC. The original implementation was restricted to VM_EXEC to minimize the blast radius and the targe use case is for large text segments. Out of curiosity, did you see any noticeable improvement with this change? > > Signed-off-by: Fangrui Song <maskray@google.com> > --- > include/linux/huge_mm.h | 1 - > 1 file changed, 1 deletion(-) > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > index fa0350b0812a..4c9e67e9000f 100644 > --- a/include/linux/huge_mm.h > +++ b/include/linux/huge_mm.h > @@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) > inode = vma->vm_file->f_inode; > > return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) && > - (vma->vm_flags & VM_EXEC) && > !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); > } > > -- > 2.43.0.472.g3155946c3a-goog >
On Wed, Dec 20, 2023 at 3:42 PM Yang Shi <shy828301@gmail.com> wrote: > > On Tue, Dec 19, 2023 at 9:41 PM Fangrui Song <maskray@google.com> wrote: > > > > Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP > > checking in transparent_hugepage_enabled()") introduced the VM_EXEC > > requirement, which is not strictly needed. > > > > lld's default --rosegment option and GNU ld's -z separate-code option > > (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD > > segment without the PF_X flag, which should be eligible for THP. > > > > Certain architectures support medium and large code models, where > > .lrodata may be placed in a separate read-only PT_LOAD segment, which > > should be eligible for THP as well. > > Yeah, it doesn't have to be VM_EXEC. The original implementation was > restricted to VM_EXEC to minimize the blast radius and the targe use > case is for large text segments. Out of curiosity, did you see any > noticeable improvement with this change? Hi Yang, Thanks for the comment. Frankly, I am not familiar with huge pages... I noticed this VM_EXEC condition when I was writing this hugepage-related section in https://maskray.me/blog/2023-12-17-exploring-the-section-layout-in-linker-output#transparent-huge-pages-for-mapped-files (Thanks to Alexander Monakov's comment about CONFIG_READ_ONLY_THP_FOR_FS in https://mazzo.li/posts/check-huge-page.html). As dTLB for read-only data is also an important optimization of file-backed THP, it seems straightforward that we should drop the VM_EXEC condition :) On my Arch linux machine, the r--p page gets split if I invoke madvise(__ehdr_start, HPAGE_SIZE, MADV_HUGEPAGE) I haven't figured out why it behaves so in the presence of the VM_EXEC check. % g++ test.cc -o ~/tmp/test -O2 -fuse-ld=lld -Wl,-z,max-page-size=2097152 && sudo ~/tmp/test __ehdr_start: 0x55f3b1c00000 55f3b1c00000-55f3b1e00000 r--p 00000000 103:03 555277119 /home/ray/tmp/test 55f3b1e00000-55f3b1e01000 r--p 00200000 103:03 555277119 /home/ray/tmp/test 55f3b2000000-55f3b2002000 r-xp 00200000 103:03 555277119 /home/ray/tmp/test 55f3b2201000-55f3b2202000 r--p 00201000 103:03 555277119 /home/ray/tmp/test 55f3b2401000-55f3b2402000 rw-p 00201000 103:03 555277119 /home/ray/tmp/test 55f3b3a9a000-55f3b3abb000 rw-p 00000000 00:00 0 [heap] It'd be greatly appreciated if someone familiar with CONFIG_READ_ONLY_THP_FOR_FS could provide some notes on how to use this feature:) > > > > Signed-off-by: Fangrui Song <maskray@google.com> > > --- > > include/linux/huge_mm.h | 1 - > > 1 file changed, 1 deletion(-) > > > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > > index fa0350b0812a..4c9e67e9000f 100644 > > --- a/include/linux/huge_mm.h > > +++ b/include/linux/huge_mm.h > > @@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) > > inode = vma->vm_file->f_inode; > > > > return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) && > > - (vma->vm_flags & VM_EXEC) && > > !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); > > } > > > > -- > > 2.43.0.472.g3155946c3a-goog > >
On Wed, Dec 20, 2023 at 8:53 PM Fangrui Song <maskray@google.com> wrote: > > On Wed, Dec 20, 2023 at 3:42 PM Yang Shi <shy828301@gmail.com> wrote: > > > > On Tue, Dec 19, 2023 at 9:41 PM Fangrui Song <maskray@google.com> wrote: > > > > > > Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP > > > checking in transparent_hugepage_enabled()") introduced the VM_EXEC > > > requirement, which is not strictly needed. > > > > > > lld's default --rosegment option and GNU ld's -z separate-code option > > > (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD > > > segment without the PF_X flag, which should be eligible for THP. > > > > > > Certain architectures support medium and large code models, where > > > .lrodata may be placed in a separate read-only PT_LOAD segment, which > > > should be eligible for THP as well. > > > > Yeah, it doesn't have to be VM_EXEC. The original implementation was > > restricted to VM_EXEC to minimize the blast radius and the targe use > > case is for large text segments. Out of curiosity, did you see any > > noticeable improvement with this change? > > Hi Yang, > > Thanks for the comment. Frankly, I am not familiar with huge pages... > I noticed this VM_EXEC condition when I was writing this > hugepage-related section in > https://maskray.me/blog/2023-12-17-exploring-the-section-layout-in-linker-output#transparent-huge-pages-for-mapped-files > (Thanks to Alexander Monakov's comment about > CONFIG_READ_ONLY_THP_FOR_FS in > https://mazzo.li/posts/check-huge-page.html). Thanks for sharing the article, learnt something about linker and loader. > > As dTLB for read-only data is also an important optimization of > file-backed THP, it seems straightforward that we should drop the > VM_EXEC condition :) Yeah, as long as the use case is valid, it is definitely fine to lift the restriction. > > On my Arch linux machine, the r--p page gets split if I invoke > madvise(__ehdr_start, HPAGE_SIZE, MADV_HUGEPAGE) I haven't figured out > why it behaves so in the presence of the VM_EXEC check. What do you mean about "split"? THP got split into small pages? It depends on the address of __ehdr_start. If it is in the middle of a VMA, the VMA is going to be split due to the different huge page attributes. > > % g++ test.cc -o ~/tmp/test -O2 -fuse-ld=lld > -Wl,-z,max-page-size=2097152 && sudo ~/tmp/test > __ehdr_start: 0x55f3b1c00000 > 55f3b1c00000-55f3b1e00000 r--p 00000000 103:03 555277119 > /home/ray/tmp/test > 55f3b1e00000-55f3b1e01000 r--p 00200000 103:03 555277119 > /home/ray/tmp/test > 55f3b2000000-55f3b2002000 r-xp 00200000 103:03 555277119 > /home/ray/tmp/test > 55f3b2201000-55f3b2202000 r--p 00201000 103:03 555277119 > /home/ray/tmp/test > 55f3b2401000-55f3b2402000 rw-p 00201000 103:03 555277119 > /home/ray/tmp/test > 55f3b3a9a000-55f3b3abb000 rw-p 00000000 00:00 0 [heap] > > > It'd be greatly appreciated if someone familiar with > CONFIG_READ_ONLY_THP_FOR_FS could provide some notes on how to use > this feature:) I think your blog covered all the points. If you don't mind, you could add some notes in Documentation/admin-guide/mm/transhuge.rst. > > > > > > > Signed-off-by: Fangrui Song <maskray@google.com> > > > --- > > > include/linux/huge_mm.h | 1 - > > > 1 file changed, 1 deletion(-) > > > > > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > > > index fa0350b0812a..4c9e67e9000f 100644 > > > --- a/include/linux/huge_mm.h > > > +++ b/include/linux/huge_mm.h > > > @@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) > > > inode = vma->vm_file->f_inode; > > > > > > return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) && > > > - (vma->vm_flags & VM_EXEC) && > > > !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); > > > } > > > > > > -- > > > 2.43.0.472.g3155946c3a-goog > > > > > > > -- > 宋方睿
On Thu, Dec 21, 2023 at 11:31 AM Yang Shi <shy828301@gmail.com> wrote: > > On Wed, Dec 20, 2023 at 8:53 PM Fangrui Song <maskray@google.com> wrote: > > > > On Wed, Dec 20, 2023 at 3:42 PM Yang Shi <shy828301@gmail.com> wrote: > > > > > > On Tue, Dec 19, 2023 at 9:41 PM Fangrui Song <maskray@google.com> wrote: > > > > > > > > Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP > > > > checking in transparent_hugepage_enabled()") introduced the VM_EXEC > > > > requirement, which is not strictly needed. > > > > > > > > lld's default --rosegment option and GNU ld's -z separate-code option > > > > (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD > > > > segment without the PF_X flag, which should be eligible for THP. > > > > > > > > Certain architectures support medium and large code models, where > > > > .lrodata may be placed in a separate read-only PT_LOAD segment, which > > > > should be eligible for THP as well. > > > > > > Yeah, it doesn't have to be VM_EXEC. The original implementation was > > > restricted to VM_EXEC to minimize the blast radius and the targe use > > > case is for large text segments. Out of curiosity, did you see any > > > noticeable improvement with this change? > > > > Hi Yang, > > > > Thanks for the comment. Frankly, I am not familiar with huge pages... > > I noticed this VM_EXEC condition when I was writing this > > hugepage-related section in > > https://maskray.me/blog/2023-12-17-exploring-the-section-layout-in-linker-output#transparent-huge-pages-for-mapped-files > > (Thanks to Alexander Monakov's comment about > > CONFIG_READ_ONLY_THP_FOR_FS in > > https://mazzo.li/posts/check-huge-page.html). > > Thanks for sharing the article, learnt something about linker and loader. BTW, kernel should try to map the segments (the size has to be >= 2M) to 2M aligned address even though the loading address is not 2M aligned for ext4/xfs/btrfs since v5.18. See commit 1854bc6e2420 ("mm/readahead: Align file mappings for non-DAX"). Did you see this behavior? > > > > > As dTLB for read-only data is also an important optimization of > > file-backed THP, it seems straightforward that we should drop the > > VM_EXEC condition :) > > Yeah, as long as the use case is valid, it is definitely fine to lift > the restriction. > > > > > On my Arch linux machine, the r--p page gets split if I invoke > > madvise(__ehdr_start, HPAGE_SIZE, MADV_HUGEPAGE) I haven't figured out > > why it behaves so in the presence of the VM_EXEC check. > > What do you mean about "split"? THP got split into small pages? It > depends on the address of __ehdr_start. If it is in the middle of a > VMA, the VMA is going to be split due to the different huge page > attributes. > > > > > % g++ test.cc -o ~/tmp/test -O2 -fuse-ld=lld > > -Wl,-z,max-page-size=2097152 && sudo ~/tmp/test > > __ehdr_start: 0x55f3b1c00000 > > 55f3b1c00000-55f3b1e00000 r--p 00000000 103:03 555277119 > > /home/ray/tmp/test > > 55f3b1e00000-55f3b1e01000 r--p 00200000 103:03 555277119 > > /home/ray/tmp/test > > 55f3b2000000-55f3b2002000 r-xp 00200000 103:03 555277119 > > /home/ray/tmp/test > > 55f3b2201000-55f3b2202000 r--p 00201000 103:03 555277119 > > /home/ray/tmp/test > > 55f3b2401000-55f3b2402000 rw-p 00201000 103:03 555277119 > > /home/ray/tmp/test > > 55f3b3a9a000-55f3b3abb000 rw-p 00000000 00:00 0 [heap] > > > > > > It'd be greatly appreciated if someone familiar with > > CONFIG_READ_ONLY_THP_FOR_FS could provide some notes on how to use > > this feature:) > > I think your blog covered all the points. If you don't mind, you could > add some notes in Documentation/admin-guide/mm/transhuge.rst. > > > > > > > > > > > Signed-off-by: Fangrui Song <maskray@google.com> > > > > --- > > > > include/linux/huge_mm.h | 1 - > > > > 1 file changed, 1 deletion(-) > > > > > > > > diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h > > > > index fa0350b0812a..4c9e67e9000f 100644 > > > > --- a/include/linux/huge_mm.h > > > > +++ b/include/linux/huge_mm.h > > > > @@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) > > > > inode = vma->vm_file->f_inode; > > > > > > > > return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) && > > > > - (vma->vm_flags & VM_EXEC) && > > > > !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); > > > > } > > > > > > > > -- > > > > 2.43.0.472.g3155946c3a-goog > > > > > > > > > > > > -- > > 宋方睿
On Wed, Dec 20, 2023 at 08:53:38PM -0800, Fangrui Song wrote: > Thanks for the comment. Frankly, I am not familiar with huge pages... > I noticed this VM_EXEC condition when I was writing this > hugepage-related section in > https://maskray.me/blog/2023-12-17-exploring-the-section-layout-in-linker-output#transparent-huge-pages-for-mapped-files > (Thanks to Alexander Monakov's comment about > CONFIG_READ_ONLY_THP_FOR_FS in > https://mazzo.li/posts/check-huge-page.html). CONFIG_READ_ONLY_THP_FOR_FS is a preliminary hack which solves some problems. The real solution is using large folios, which at the moment means that you should test on XFS or AFS; filesystem authors have not been enthusiastic about adding support to their filesystems so far. In your blog, you write: : In -z noseparate-code layouts, the file content starts somewhere at : the first page, potentially wasting half a huge page on unrelated : content. Switching to -z separate-code allows reclaiming the benefits : of the half huge page but increases the file size. Balancing : these aspects poses a challenge. One potential solution is using : fallocate(FALLOC_FL_PUNCH_HOLE), which introduces complexity into the : linker. However, this approach feels like a workaround to address a : kernel limitation. It would be preferable if a file-backed huge page : didn't necessitate a file offset aligned to a huge page boundary. You should distinguish between file size (ie st_size in stat(3)) and amount of space occupied on storage (st_blocks). The linker should be fine with creating a sparse file. If it doesn't, cp --sparse will do the trick. Yes, it's a kernel limitation that folios have to be aligned within the file as well as in both virtual and physical address space. It's a huge complexity win to do that; I don't think we'd be able to tile the page cache effectively if we allowed folios to be placed at arbitrary offsets (I think it turns into a knapsack problem at that point). > As dTLB for read-only data is also an important optimization of > file-backed THP, it seems straightforward that we should drop the > VM_EXEC condition :) I'm not particularly enthusiastic about making CONFIG_READ_ONLY_THP_FOR_FS better. Large folios are the future. Indeed, I'd like to see CONFIG_READ_ONLY_THP_FOR_FS go away in the next year or two once btrfs and ext4 have support for large folios.
diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index fa0350b0812a..4c9e67e9000f 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -126,7 +126,6 @@ static inline bool file_thp_enabled(struct vm_area_struct *vma) inode = vma->vm_file->f_inode; return (IS_ENABLED(CONFIG_READ_ONLY_THP_FOR_FS)) && - (vma->vm_flags & VM_EXEC) && !inode_is_open_for_write(inode) && S_ISREG(inode->i_mode); }
Commit e6be37b2e7bd ("mm/huge_memory.c: add missing read-only THP checking in transparent_hugepage_enabled()") introduced the VM_EXEC requirement, which is not strictly needed. lld's default --rosegment option and GNU ld's -z separate-code option (default on Linux/x86 since binutils 2.31) create a read-only PT_LOAD segment without the PF_X flag, which should be eligible for THP. Certain architectures support medium and large code models, where .lrodata may be placed in a separate read-only PT_LOAD segment, which should be eligible for THP as well. Signed-off-by: Fangrui Song <maskray@google.com> --- include/linux/huge_mm.h | 1 - 1 file changed, 1 deletion(-)