| Message ID | 20240117223729.1444522-1-lokeshgidra@google.com (mailing list archive) |
|---|---|
| State | Handled Elsewhere |
| Headers | show |
| Series | userfaultfd: fix mmap_changing checking in mfill_atomic_hugetlb | expand |
On Wed, 17 Jan 2024 14:37:29 -0800 Lokesh Gidra <lokeshgidra@google.com> wrote: > In mfill_atomic_hugetlb(), mmap_changing isn't being checked > again if we drop mmap_lock and reacquire it. When the lock is not held, > mmap_changing could have been incremented. This is also inconsistent > with the behavior in mfill_atomic(). Thanks. Could you and reviewers please consider - what might be the userspace-visible runtime effects? - Should the fix be backported into earlier kernels? - A suitable Fixes: target?
Apologies, I had forgotten to re-check the "send plaintext" checkbox in my e-mail client, so the mailing lists rejected my previous mail. I am duly ashamed. Allow me to try once more. :) On Thu, Jan 18, 2024 at 1:59 PM Andrew Morton <akpm@linux-foundation.org> wrote: > > On Wed, 17 Jan 2024 14:37:29 -0800 Lokesh Gidra <lokeshgidra@google.com> wrote: > > > In mfill_atomic_hugetlb(), mmap_changing isn't being checked > > again if we drop mmap_lock and reacquire it. When the lock is not held, > > mmap_changing could have been incremented. This is also inconsistent > > with the behavior in mfill_atomic(). The change looks reasonable to me. I'm not sure I can conclusively say there isn't some other mechanism specific to hugetlbfs which means this isn't needed, though. > > Thanks. Could you and reviewers please consider > > - what might be the userspace-visible runtime effects? > > - Should the fix be backported into earlier kernels? > > - A suitable Fixes: target? Hmm, 60d4d2d2b40e4 added __mcopy_atomic_hugetlb without this. But, at that point in history, none of the other functions had mmap_changing either. So, I think the right Fixes: target is df2cc96e77011 ("userfaultfd: prevent non-cooperative events vs mcopy_atomic races") ? It seems to have missed the hugetlb path. This was introduced in 4.18. Based on that commit's message, essentially what can happen if the race "succeeds" is, memory can be accessed without userfaultfd being notified of this fact. Depending on what userfaultfd is being used for, from userspace's perspective this can appear like memory corruption for example. So, based on that it seems to me reasonable to backport this to stable kernels (4.19+).
On Thu, Jan 18, 2024 at 03:17:14PM -0800, Axel Rasmussen wrote: > > On Thu, Jan 18, 2024 at 1:59 PM Andrew Morton <akpm@linux-foundation.org> > wrote: > > On Wed, 17 Jan 2024 14:37:29 -0800 Lokesh Gidra <lokeshgidra@google.com> > wrote: > > > In mfill_atomic_hugetlb(), mmap_changing isn't being checked > > again if we drop mmap_lock and reacquire it. When the lock is not held, > > mmap_changing could have been incremented. This is also inconsistent > > with the behavior in mfill_atomic(). > > > The change looks reasonable to me. I'm not sure I can conclusively say there > isn't some other mechanism specific to hugetlbfs which means this isn't needed, > though. There's nothing specific to hugetlb, if a non-cooperative uffdio_copy races with mremap/fork etc, the vma under it may change > Thanks. Could you and reviewers please consider > > - what might be the userspace-visible runtime effects? For users of non-cooperative uffd with hugetlb, this would fix crashes caused by races between uffd operations that update memory and the operations that change the VM layout. Pretty much the same fix as df2cc96e77011 ("userfaultfd: prevent non-cooperative events vs mcopy_atomic races") for !hugetlb memory. I doubt such users exist, though... > - Should the fix be backported into earlier kernels? > - A suitable Fixes: target? > > Hmm, 60d4d2d2b40e4 added __mcopy_atomic_hugetlb without this. But, at that > point in history, none of the other functions had mmap_changing either. > > So, I think the right Fixes: target is df2cc96e77011 ("userfaultfd: prevent > non-cooperative events vs mcopy_atomic races") ? It seems to have missed the > hugetlb path. This was introduced in 4.18. > > Based on that commit's message, essentially what can happen if the race > "succeeds" is, memory can be accessed without userfaultfd being notified of > this fact. Depending on what userfaultfd is being used for, from > userspace's perspective this can appear like memory corruption for example. So, > based on that it seems to me reasonable to backport this to stable kernels > (4.19+). I agree with Axel, Fixes: df2cc96e77011 ("userfaultfd: prevent non-cooperative events vs mcopy_atomic races") seems appropriate.
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 20e3b0d9cf7e..75fcf1f783bc 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -357,6 +357,7 @@ static __always_inline ssize_t mfill_atomic_hugetlb( unsigned long dst_start, unsigned long src_start, unsigned long len, + atomic_t *mmap_changing, uffd_flags_t flags) { struct mm_struct *dst_mm = dst_vma->vm_mm; @@ -472,6 +473,15 @@ static __always_inline ssize_t mfill_atomic_hugetlb( goto out; } mmap_read_lock(dst_mm); + /* + * If memory mappings are changing because of non-cooperative + * operation (e.g. mremap) running in parallel, bail out and + * request the user to retry later + */ + if (mmap_changing && atomic_read(mmap_changing)) { + err = -EAGAIN; + break; + } dst_vma = NULL; goto retry; @@ -506,6 +516,7 @@ extern ssize_t mfill_atomic_hugetlb(struct vm_area_struct *dst_vma, unsigned long dst_start, unsigned long src_start, unsigned long len, + atomic_t *mmap_changing, uffd_flags_t flags); #endif /* CONFIG_HUGETLB_PAGE */ @@ -622,8 +633,8 @@ static __always_inline ssize_t mfill_atomic(struct mm_struct *dst_mm, * If this is a HUGETLB vma, pass off to appropriate routine */ if (is_vm_hugetlb_page(dst_vma)) - return mfill_atomic_hugetlb(dst_vma, dst_start, - src_start, len, flags); + return mfill_atomic_hugetlb(dst_vma, dst_start, src_start, + len, mmap_changing, flags); if (!vma_is_anonymous(dst_vma) && !vma_is_shmem(dst_vma)) goto out_unlock;
In mfill_atomic_hugetlb(), mmap_changing isn't being checked again if we drop mmap_lock and reacquire it. When the lock is not held, mmap_changing could have been incremented. This is also inconsistent with the behavior in mfill_atomic(). Signed-off-by: Lokesh Gidra <lokeshgidra@google.com> --- mm/userfaultfd.c | 15 +++++++++++++-- 1 file changed, 13 insertions(+), 2 deletions(-)