diff mbox series

[16/29] mm: add pte_map_lock() and pte_spinlock()

Message ID 20210430195232.30491-17-michel@lespinasse.org (mailing list archive)
State New, archived
Headers show
Series [01/29] mm: export dump_mm | expand

Commit Message

Michel Lespinasse April 30, 2021, 7:52 p.m. UTC 
pte_map_lock() and pte_spinlock() are used by fault handlers to ensure
the pte is mapped and locked before they commit the faulted page to the
mm's address space at the end of the fault.

The functions differ in their preconditions; pte_map_lock() expects
the pte to be unmapped prior to the call, while pte_spinlock() expects
it to be already mapped.

In the speculative fault case, the functions verify, after locking the pte,
that the mmap sequence count has not changed since the start of the fault,
and thus that no mmap lock writers have been running concurrently with
the fault. After that point the page table lock serializes any further
races with concurrent mmap lock writers.

If the mmap sequence count check fails, both functions will return false
with the pte being left unmapped and unlocked.

Signed-off-by: Michel Lespinasse <michel@lespinasse.org>
---
 include/linux/mm.h | 36 +++++++++++++++++++++++++
 mm/memory.c        | 66 ++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 102 insertions(+)

Comments

kernel test robot April 30, 2021, 11:33 p.m. UTC | #1 
Hi Michel,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on tip/x86/mm]
[also build test ERROR on arm64/for-next/core linus/master v5.12]
[cannot apply to hnaz-linux-mm/master next-20210430]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Michel-Lespinasse/Speculative-page-faults-anon-vmas-only/20210501-035602
base:   https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git a500fc918f7b8dc3dff2e6c74f3e73e856c18248
config: i386-allyesconfig (attached as .config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/284898f9c11d755d2b231794fc7529d562f8e918
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Michel-Lespinasse/Speculative-page-faults-anon-vmas-only/20210501-035602
        git checkout 284898f9c11d755d2b231794fc7529d562f8e918
        # save the attached .config to linux build tree
        make W=1 W=1 ARCH=i386 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from include/linux/mm.h:33,
                    from include/linux/kallsyms.h:12,
                    from include/linux/bpf.h:20,
                    from include/linux/bpf-cgroup.h:5,
                    from include/linux/cgroup-defs.h:22,
                    from include/linux/cgroup.h:28,
                    from include/linux/memcontrol.h:13,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/mm.h: In function 'pte_map_lock':
   include/linux/pgtable.h:79:12: error: implicit declaration of function 'kmap_atomic'; did you mean 'in_atomic'? [-Werror=implicit-function-declaration]
      79 |  ((pte_t *)kmap_atomic(pmd_page(*(dir))) +  \
         |            ^~~~~~~~~~~
   include/linux/mm.h:2205:17: note: in expansion of macro 'pte_offset_map'
    2205 |  pte_t *__pte = pte_offset_map(pmd, address); \
         |                 ^~~~~~~~~~~~~~
   include/linux/mm.h:3174:13: note: in expansion of macro 'pte_offset_map_lock'
    3174 |  vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
         |             ^~~~~~~~~~~~~~~~~~~
   In file included from include/linux/highmem.h:14,
                    from include/linux/pagemap.h:11,
                    from include/linux/blkdev.h:14,
                    from include/linux/blk-cgroup.h:23,
                    from include/linux/writeback.h:14,
                    from include/linux/memcontrol.h:22,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/highmem-internal.h: At top level:
>> include/linux/highmem-internal.h:98:21: error: conflicting types for 'kmap_atomic'
      98 | static inline void *kmap_atomic(struct page *page)
         |                     ^~~~~~~~~~~
   In file included from include/linux/mm.h:33,
                    from include/linux/kallsyms.h:12,
                    from include/linux/bpf.h:20,
                    from include/linux/bpf-cgroup.h:5,
                    from include/linux/cgroup-defs.h:22,
                    from include/linux/cgroup.h:28,
                    from include/linux/memcontrol.h:13,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/pgtable.h:79:12: note: previous implicit declaration of 'kmap_atomic' was here
      79 |  ((pte_t *)kmap_atomic(pmd_page(*(dir))) +  \
         |            ^~~~~~~~~~~
   include/linux/mm.h:2205:17: note: in expansion of macro 'pte_offset_map'
    2205 |  pte_t *__pte = pte_offset_map(pmd, address); \
         |                 ^~~~~~~~~~~~~~
   include/linux/mm.h:3174:13: note: in expansion of macro 'pte_offset_map_lock'
    3174 |  vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
         |             ^~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
--
   In file included from include/linux/mm.h:33,
                    from include/linux/kallsyms.h:12,
                    from include/linux/bpf.h:20,
                    from include/linux/bpf-cgroup.h:5,
                    from include/linux/cgroup-defs.h:22,
                    from include/linux/cgroup.h:28,
                    from include/linux/memcontrol.h:13,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/mm.h: In function 'pte_map_lock':
   include/linux/pgtable.h:79:12: error: implicit declaration of function 'kmap_atomic'; did you mean 'in_atomic'? [-Werror=implicit-function-declaration]
      79 |  ((pte_t *)kmap_atomic(pmd_page(*(dir))) +  \
         |            ^~~~~~~~~~~
   include/linux/mm.h:2205:17: note: in expansion of macro 'pte_offset_map'
    2205 |  pte_t *__pte = pte_offset_map(pmd, address); \
         |                 ^~~~~~~~~~~~~~
   include/linux/mm.h:3174:13: note: in expansion of macro 'pte_offset_map_lock'
    3174 |  vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
         |             ^~~~~~~~~~~~~~~~~~~
   In file included from include/linux/highmem.h:14,
                    from include/linux/pagemap.h:11,
                    from include/linux/blkdev.h:14,
                    from include/linux/blk-cgroup.h:23,
                    from include/linux/writeback.h:14,
                    from include/linux/memcontrol.h:22,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/highmem-internal.h: At top level:
>> include/linux/highmem-internal.h:98:21: error: conflicting types for 'kmap_atomic'
      98 | static inline void *kmap_atomic(struct page *page)
         |                     ^~~~~~~~~~~
   In file included from include/linux/mm.h:33,
                    from include/linux/kallsyms.h:12,
                    from include/linux/bpf.h:20,
                    from include/linux/bpf-cgroup.h:5,
                    from include/linux/cgroup-defs.h:22,
                    from include/linux/cgroup.h:28,
                    from include/linux/memcontrol.h:13,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/pgtable.h:79:12: note: previous implicit declaration of 'kmap_atomic' was here
      79 |  ((pte_t *)kmap_atomic(pmd_page(*(dir))) +  \
         |            ^~~~~~~~~~~
   include/linux/mm.h:2205:17: note: in expansion of macro 'pte_offset_map'
    2205 |  pte_t *__pte = pte_offset_map(pmd, address); \
         |                 ^~~~~~~~~~~~~~
   include/linux/mm.h:3174:13: note: in expansion of macro 'pte_offset_map_lock'
    3174 |  vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
         |             ^~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
   make[2]: *** [scripts/Makefile.build:116: arch/x86/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1233: prepare0] Error 2
   make[1]: Target 'modules_prepare' not remade because of errors.
   make: *** [Makefile:215: __sub-make] Error 2
   make: Target 'modules_prepare' not remade because of errors.
--
   In file included from include/linux/mm.h:33,
                    from include/linux/kallsyms.h:12,
                    from include/linux/bpf.h:20,
                    from include/linux/bpf-cgroup.h:5,
                    from include/linux/cgroup-defs.h:22,
                    from include/linux/cgroup.h:28,
                    from include/linux/memcontrol.h:13,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/mm.h: In function 'pte_map_lock':
   include/linux/pgtable.h:79:12: error: implicit declaration of function 'kmap_atomic'; did you mean 'in_atomic'? [-Werror=implicit-function-declaration]
      79 |  ((pte_t *)kmap_atomic(pmd_page(*(dir))) +  \
         |            ^~~~~~~~~~~
   include/linux/mm.h:2205:17: note: in expansion of macro 'pte_offset_map'
    2205 |  pte_t *__pte = pte_offset_map(pmd, address); \
         |                 ^~~~~~~~~~~~~~
   include/linux/mm.h:3174:13: note: in expansion of macro 'pte_offset_map_lock'
    3174 |  vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
         |             ^~~~~~~~~~~~~~~~~~~
   In file included from include/linux/highmem.h:14,
                    from include/linux/pagemap.h:11,
                    from include/linux/blkdev.h:14,
                    from include/linux/blk-cgroup.h:23,
                    from include/linux/writeback.h:14,
                    from include/linux/memcontrol.h:22,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/highmem-internal.h: At top level:
>> include/linux/highmem-internal.h:98:21: error: conflicting types for 'kmap_atomic'
      98 | static inline void *kmap_atomic(struct page *page)
         |                     ^~~~~~~~~~~
   In file included from include/linux/mm.h:33,
                    from include/linux/kallsyms.h:12,
                    from include/linux/bpf.h:20,
                    from include/linux/bpf-cgroup.h:5,
                    from include/linux/cgroup-defs.h:22,
                    from include/linux/cgroup.h:28,
                    from include/linux/memcontrol.h:13,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/pgtable.h:79:12: note: previous implicit declaration of 'kmap_atomic' was here
      79 |  ((pte_t *)kmap_atomic(pmd_page(*(dir))) +  \
         |            ^~~~~~~~~~~
   include/linux/mm.h:2205:17: note: in expansion of macro 'pte_offset_map'
    2205 |  pte_t *__pte = pte_offset_map(pmd, address); \
         |                 ^~~~~~~~~~~~~~
   include/linux/mm.h:3174:13: note: in expansion of macro 'pte_offset_map_lock'
    3174 |  vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
         |             ^~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
   make[2]: *** [scripts/Makefile.build:116: arch/x86/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1233: prepare0] Error 2
   make[1]: Target 'prepare' not remade because of errors.
   make: *** [Makefile:215: __sub-make] Error 2
   make: Target 'prepare' not remade because of errors.


vim +/kmap_atomic +98 include/linux/highmem-internal.h

13f876ba77ebd5 Thomas Gleixner 2020-11-03   97  
13f876ba77ebd5 Thomas Gleixner 2020-11-03  @98  static inline void *kmap_atomic(struct page *page)
13f876ba77ebd5 Thomas Gleixner 2020-11-03   99  {
13f876ba77ebd5 Thomas Gleixner 2020-11-03  100  	return kmap_atomic_prot(page, kmap_prot);
13f876ba77ebd5 Thomas Gleixner 2020-11-03  101  }
13f876ba77ebd5 Thomas Gleixner 2020-11-03  102  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
kernel test robot April 30, 2021, 11:45 p.m. UTC | #2 
Hi Michel,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on tip/x86/mm]
[also build test ERROR on arm64/for-next/core v5.12]
[cannot apply to hnaz-linux-mm/master linus/master next-20210430]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Michel-Lespinasse/Speculative-page-faults-anon-vmas-only/20210501-035602
base:   https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git a500fc918f7b8dc3dff2e6c74f3e73e856c18248
config: arm-defconfig (attached as .config)
compiler: arm-linux-gnueabi-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/0day-ci/linux/commit/284898f9c11d755d2b231794fc7529d562f8e918
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Michel-Lespinasse/Speculative-page-faults-anon-vmas-only/20210501-035602
        git checkout 284898f9c11d755d2b231794fc7529d562f8e918
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross W=1 ARCH=arm 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from include/linux/mm.h:33,
                    from arch/arm/kernel/asm-offsets.c:12:
   include/linux/mm.h: In function 'pte_map_lock':
>> include/linux/pgtable.h:79:12: error: implicit declaration of function 'kmap_atomic'; did you mean 'in_atomic'? [-Werror=implicit-function-declaration]
      79 |  ((pte_t *)kmap_atomic(pmd_page(*(dir))) +  \
         |            ^~~~~~~~~~~
   include/linux/mm.h:2205:17: note: in expansion of macro 'pte_offset_map'
    2205 |  pte_t *__pte = pte_offset_map(pmd, address); \
         |                 ^~~~~~~~~~~~~~
   include/linux/mm.h:3174:13: note: in expansion of macro 'pte_offset_map_lock'
    3174 |  vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
         |             ^~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
--
   In file included from include/linux/mm.h:33,
                    from arch/arm/kernel/asm-offsets.c:12:
   include/linux/mm.h: In function 'pte_map_lock':
>> include/linux/pgtable.h:79:12: error: implicit declaration of function 'kmap_atomic'; did you mean 'in_atomic'? [-Werror=implicit-function-declaration]
      79 |  ((pte_t *)kmap_atomic(pmd_page(*(dir))) +  \
         |            ^~~~~~~~~~~
   include/linux/mm.h:2205:17: note: in expansion of macro 'pte_offset_map'
    2205 |  pte_t *__pte = pte_offset_map(pmd, address); \
         |                 ^~~~~~~~~~~~~~
   include/linux/mm.h:3174:13: note: in expansion of macro 'pte_offset_map_lock'
    3174 |  vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
         |             ^~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
   make[2]: *** [scripts/Makefile.build:116: arch/arm/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1233: prepare0] Error 2
   make[1]: Target 'modules_prepare' not remade because of errors.
   make: *** [Makefile:215: __sub-make] Error 2
   make: Target 'modules_prepare' not remade because of errors.
--
   In file included from include/linux/mm.h:33,
                    from arch/arm/kernel/asm-offsets.c:12:
   include/linux/mm.h: In function 'pte_map_lock':
>> include/linux/pgtable.h:79:12: error: implicit declaration of function 'kmap_atomic'; did you mean 'in_atomic'? [-Werror=implicit-function-declaration]
      79 |  ((pte_t *)kmap_atomic(pmd_page(*(dir))) +  \
         |            ^~~~~~~~~~~
   include/linux/mm.h:2205:17: note: in expansion of macro 'pte_offset_map'
    2205 |  pte_t *__pte = pte_offset_map(pmd, address); \
         |                 ^~~~~~~~~~~~~~
   include/linux/mm.h:3174:13: note: in expansion of macro 'pte_offset_map_lock'
    3174 |  vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
         |             ^~~~~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
   make[2]: *** [scripts/Makefile.build:116: arch/arm/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1233: prepare0] Error 2
   make[1]: Target 'prepare' not remade because of errors.
   make: *** [Makefile:215: __sub-make] Error 2
   make: Target 'prepare' not remade because of errors.


vim +79 include/linux/pgtable.h

974b9b2c68f3d3 Mike Rapoport 2020-06-08  76  
974b9b2c68f3d3 Mike Rapoport 2020-06-08  77  #if defined(CONFIG_HIGHPTE)
974b9b2c68f3d3 Mike Rapoport 2020-06-08  78  #define pte_offset_map(dir, address)				\
974b9b2c68f3d3 Mike Rapoport 2020-06-08 @79  	((pte_t *)kmap_atomic(pmd_page(*(dir))) +		\
974b9b2c68f3d3 Mike Rapoport 2020-06-08  80  	 pte_index((address)))
974b9b2c68f3d3 Mike Rapoport 2020-06-08  81  #define pte_unmap(pte) kunmap_atomic((pte))
974b9b2c68f3d3 Mike Rapoport 2020-06-08  82  #else
974b9b2c68f3d3 Mike Rapoport 2020-06-08  83  #define pte_offset_map(dir, address)	pte_offset_kernel((dir), (address))
974b9b2c68f3d3 Mike Rapoport 2020-06-08  84  #define pte_unmap(pte) ((void)(pte))	/* NOP */
974b9b2c68f3d3 Mike Rapoport 2020-06-08  85  #endif
974b9b2c68f3d3 Mike Rapoport 2020-06-08  86  

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
diff mbox series

Patch

diff --git a/include/linux/mm.h b/include/linux/mm.h
index dee8a4833779..8124cd53ce15 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -3183,5 +3183,41 @@  extern int sysctl_nr_trim_pages;
 
 void mem_dump_obj(void *object);
 
+#ifdef CONFIG_MMU
+#ifdef CONFIG_SPECULATIVE_PAGE_FAULT
+
+bool __pte_map_lock(struct vm_fault *vmf);
+
+static inline bool pte_map_lock(struct vm_fault *vmf)
+{
+	VM_BUG_ON(vmf->pte);
+	return __pte_map_lock(vmf);
+}
+
+static inline bool pte_spinlock(struct vm_fault *vmf)
+{
+	VM_BUG_ON(!vmf->pte);
+	return __pte_map_lock(vmf);
+}
+
+#else	/* !CONFIG_SPECULATIVE_PAGE_FAULT */
+
+static inline bool pte_map_lock(struct vm_fault *vmf)
+{
+	vmf->pte = pte_offset_map_lock(vmf->vma->vm_mm, vmf->pmd, vmf->address,
+				       &vmf->ptl);
+	return true;
+}
+
+static inline bool pte_spinlock(struct vm_fault *vmf)
+{
+	vmf->ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd);
+	spin_lock(vmf->ptl);
+	return true;
+}
+
+#endif	/* CONFIG_SPECULATIVE_PAGE_FAULT */
+#endif	/* CONFIG_MMU */
+
 #endif /* __KERNEL__ */
 #endif /* _LINUX_MM_H */
diff --git a/mm/memory.c b/mm/memory.c
index 3f5c3d6c0197..e2f9e4c096dd 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -2586,6 +2586,72 @@  EXPORT_SYMBOL_GPL(apply_to_existing_page_range);
 #define speculative_page_walk_end()   local_irq_enable()
 #endif
 
+bool __pte_map_lock(struct vm_fault *vmf)
+{
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+	pmd_t pmdval;
+#endif
+	pte_t *pte = vmf->pte;
+	spinlock_t *ptl;
+
+	if (!(vmf->flags & FAULT_FLAG_SPECULATIVE)) {
+		vmf->ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd);
+		if (!pte)
+			vmf->pte = pte_offset_map(vmf->pmd, vmf->address);
+		spin_lock(vmf->ptl);
+		return true;
+	}
+
+	speculative_page_walk_begin();
+	if (!mmap_seq_read_check(vmf->vma->vm_mm, vmf->seq))
+		goto fail;
+	/*
+	 * The mmap sequence count check guarantees that the page
+	 * tables are still valid at that point, and
+	 * speculative_page_walk_begin() ensures that they stay around.
+	 */
+#ifdef CONFIG_TRANSPARENT_HUGEPAGE
+	/*
+	 * We check if the pmd value is still the same to ensure that there
+	 * is not a huge collapse operation in progress in our back.
+	 */
+	pmdval = READ_ONCE(*vmf->pmd);
+	if (!pmd_same(pmdval, vmf->orig_pmd))
+		goto fail;
+#endif
+	ptl = pte_lockptr(vmf->vma->vm_mm, vmf->pmd);
+	if (!pte)
+		pte = pte_offset_map(vmf->pmd, vmf->address);
+	/*
+	 * Try locking the page table.
+	 *
+	 * Note that we might race against zap_pte_range() which
+	 * invalidates TLBs while holding the page table lock.
+	 * We are still under the speculative_page_walk_begin() section,
+	 * and zap_pte_range() could thus deadlock with us if we tried
+	 * using spin_lock() here.
+	 *
+	 * We also don't want to retry until spin_trylock() succeeds,
+	 * because of the starvation potential against a stream of lockers.
+	 */
+	if (unlikely(!spin_trylock(ptl)))
+		goto fail;
+	if (!mmap_seq_read_check(vmf->vma->vm_mm, vmf->seq))
+		goto unlock_fail;
+	speculative_page_walk_end();
+	vmf->pte = pte;
+	vmf->ptl = ptl;
+	return true;
+
+unlock_fail:
+	spin_unlock(ptl);
+fail:
+	if (pte)
+		pte_unmap(pte);
+	speculative_page_walk_end();
+	return false;
+}
+
 #endif	/* CONFIG_SPECULATIVE_PAGE_FAULT */
 
 /*