| Message ID | 1434493710-11138-1-git-send-email-toshi.kani@hp.com (mailing list archive) |
|---|---|
| State | Not Applicable |
| Headers | show |
On Tue, Jun 16, 2015 at 04:28:30PM -0600, Toshi Kani wrote: > DAX has the following issues in a shared or read-only private > mmap'd file. > - mmap(MAP_POPULATE) does not pre-fault > - mlock() fails with -ENOMEM > > DAX uses VM_MIXEDMAP for mmap'd files, which do not have struct > page associated with the ranges. Both MAP_POPULATE and mlock() > call __mm_populate(), which in turn calls __get_user_pages(). > Because __get_user_pages() requires a valid page returned from > follow_page_mask(), MAP_POPULATE and mlock(), i.e. FOLL_POPULATE, > fail in the first page. > > Change __get_user_pages() to proceed FOLL_POPULATE when the > translation is set but its page does not exist (-EFAULT), and > @pages is not requested. With that, MAP_POPULATE and mlock() > set translations to the requested range and complete successfully. > > MAP_POPULATE still provides a major performance improvement to > DAX as it will avoid page faults during initial access to the > pages. > > mlock() continues to set VM_LOCKED to vma and populate the range. > Since there is no struct page, the range is pinned without marking > pages mlocked. > > Note, MAP_POPULATE and mlock() already work for a write-able > private mmap'd file on DAX since populate_vma_page_range() breaks > COW, which allocates page caches. I don't think that's true in all cases. We would fail to break COW for mlock() if the mapping is populated with read-only entries by the mlock() time. In this case follow_page_mask() would fail with -EFAULT and faultin_page() will never executed.
On Sat, 2015-06-20 at 22:46 +0300, Kirill A. Shutemov wrote: > On Tue, Jun 16, 2015 at 04:28:30PM -0600, Toshi Kani wrote: > > DAX has the following issues in a shared or read-only private > > mmap'd file. > > - mmap(MAP_POPULATE) does not pre-fault > > - mlock() fails with -ENOMEM > > > > DAX uses VM_MIXEDMAP for mmap'd files, which do not have struct > > page associated with the ranges. Both MAP_POPULATE and mlock() > > call __mm_populate(), which in turn calls __get_user_pages(). > > Because __get_user_pages() requires a valid page returned from > > follow_page_mask(), MAP_POPULATE and mlock(), i.e. FOLL_POPULATE, > > fail in the first page. > > > > Change __get_user_pages() to proceed FOLL_POPULATE when the > > translation is set but its page does not exist (-EFAULT), and > > @pages is not requested. With that, MAP_POPULATE and mlock() > > set translations to the requested range and complete successfully. > > > > MAP_POPULATE still provides a major performance improvement to > > DAX as it will avoid page faults during initial access to the > > pages. > > > > mlock() continues to set VM_LOCKED to vma and populate the range. > > Since there is no struct page, the range is pinned without marking > > pages mlocked. > > > > Note, MAP_POPULATE and mlock() already work for a write-able > > private mmap'd file on DAX since populate_vma_page_range() breaks > > COW, which allocates page caches. > > I don't think that's true in all cases. > > We would fail to break COW for mlock() if the mapping is populated with > read-only entries by the mlock() time. In this case follow_page_mask() > would fail with -EFAULT and faultin_page() will never executed. No, mlock() always breaks COW as populate_vma_page_range() sets FOLL_WRITE in case of write-able private mmap. /* * We want to touch writable mappings with a write fault in order * to break COW, except for shared mappings because these don't COW * and we would not want to dirty them for nothing. */ if ((vma->vm_flags & (VM_WRITE | VM_SHARED)) == VM_WRITE) gup_flags |= FOLL_WRITE; Thanks, -Toshi
diff --git a/mm/gup.c b/mm/gup.c index 6297f6b..16d536f 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -490,8 +490,20 @@ retry: } BUG(); } - if (IS_ERR(page)) + if (IS_ERR(page)) { + /* + * No page may be associated with VM_MIXEDMAP. Proceed + * FOLL_POPULATE when the translation is set but its + * page does not exist (-EFAULT), and @pages is not + * requested by the caller. + */ + if ((PTR_ERR(page) == -EFAULT) && (!pages) && + (gup_flags & FOLL_POPULATE) && + (vma->vm_flags & VM_MIXEDMAP)) + goto next_page; + return i ? i : PTR_ERR(page); + } if (pages) { pages[i] = page; flush_anon_page(vma, page, start);
DAX has the following issues in a shared or read-only private mmap'd file. - mmap(MAP_POPULATE) does not pre-fault - mlock() fails with -ENOMEM DAX uses VM_MIXEDMAP for mmap'd files, which do not have struct page associated with the ranges. Both MAP_POPULATE and mlock() call __mm_populate(), which in turn calls __get_user_pages(). Because __get_user_pages() requires a valid page returned from follow_page_mask(), MAP_POPULATE and mlock(), i.e. FOLL_POPULATE, fail in the first page. Change __get_user_pages() to proceed FOLL_POPULATE when the translation is set but its page does not exist (-EFAULT), and @pages is not requested. With that, MAP_POPULATE and mlock() set translations to the requested range and complete successfully. MAP_POPULATE still provides a major performance improvement to DAX as it will avoid page faults during initial access to the pages. mlock() continues to set VM_LOCKED to vma and populate the range. Since there is no struct page, the range is pinned without marking pages mlocked. Note, MAP_POPULATE and mlock() already work for a write-able private mmap'd file on DAX since populate_vma_page_range() breaks COW, which allocates page caches. Signed-off-by: Toshi Kani <toshi.kani@hp.com> --- mm/gup.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-)