From patchwork Mon Mar 6 22:50:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Axel Rasmussen X-Patchwork-Id: 13162501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id F30D0C64EC4 for ; Mon, 6 Mar 2023 22:50:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3955B6B0072; Mon, 6 Mar 2023 17:50:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 344C26B0073; Mon, 6 Mar 2023 17:50:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1E636280001; Mon, 6 Mar 2023 17:50:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 114896B0072 for ; Mon, 6 Mar 2023 17:50:38 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B7A5C1C5AFA for ; Mon, 6 Mar 2023 22:50:37 +0000 (UTC) X-FDA: 80539969314.15.D1E270E Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) by imf11.hostedemail.com (Postfix) with ESMTP id EF00F4000C for ; Mon, 6 Mar 2023 22:50:35 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=hXXtqK+n; spf=pass (imf11.hostedemail.com: domain of 3O24GZA0KCFg0N4BH0ICKII4D6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--axelrasmussen.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3O24GZA0KCFg0N4BH0ICKII4D6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678143036; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=/rzLjkwC2u5MGzDlepslQhV0v6WC0U/wtR0QDlZObA4=; b=P3EZB423MH5CJ1sp6uH39EsdP5xQXz8hewJmv1FxUY0jtJEBUtZiRoL7sXER6Q+9oaFfhA b3wMdujuYiPEMwZCS70oQBY1YcBWYOrth/nrHSR+0nOA4LcSjjSxTD1jAqadceid5m2Y2z qDlj7As4uBGYLl+DJxEZW21rvQQcMgo= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=pass header.d=google.com header.s=20210112 header.b=hXXtqK+n; spf=pass (imf11.hostedemail.com: domain of 3O24GZA0KCFg0N4BH0ICKII4D6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--axelrasmussen.bounces.google.com designates 209.85.219.202 as permitted sender) smtp.mailfrom=3O24GZA0KCFg0N4BH0ICKII4D6EE6B4.2ECB8DKN-CCAL02A.EH6@flex--axelrasmussen.bounces.google.com; dmarc=pass (policy=reject) header.from=google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678143036; a=rsa-sha256; cv=none; b=E5nMPFdcQ5pivR7/Os9pSBDrPNLaWasOSeNy/tUQc7KRvTScvTaqnkcds3HvhyA1/WJYdS 086+I40acizdYVrFOxsPpFeqRu4stUADBWK+erajympabIa47NO4P6ZBQqC1q4SQTr6jiF 8f6L6h1v2V1v8JJdw1VFj01pTmyrfeY= Received: by mail-yb1-f202.google.com with SMTP id m6-20020a056902118600b00aeb1e3dbd1bso11829634ybu.9 for ; Mon, 06 Mar 2023 14:50:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; t=1678143035; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=/rzLjkwC2u5MGzDlepslQhV0v6WC0U/wtR0QDlZObA4=; b=hXXtqK+nObSBbk42X4186S/BuUW6h0+rcZvKUrSxrvjnBGNWu6kE+X3QL9KMppgwkZ cuh1YBOO+eCLPDlppXJPZndpWMbOuEEmYaEKfHUoqTWdLLK76APb/y9tNJbv41deQ7+Z fivExsyWtYEh8bIr/ChfAmzLVA0nmJXzMUZYz7vUufxZcmaTdDM8an5SSCygd9vV1Kxs Jg1p+AwJhArnUKcadA0/C/h+BoRNu2yPkZDlr/m6RXdQAhVTfRNLM8JyX1+L//GggnLU P0fGdPhXmTAA+ktZrWjMr1sfWlWM5Udn21ll3WyMUz3JYj3+Fk8T7gXGq7p5eYgA2Z5D 9u8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; t=1678143035; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=/rzLjkwC2u5MGzDlepslQhV0v6WC0U/wtR0QDlZObA4=; b=aaw/PBXNjQee6aG006tuAVtn4m9pkR7dMmPgGeZIdPxoPPs3elBu2lB97ivfVmon1d URMOALmfcLMQ6UZ1F6dpx/1JW2JFHST/aEFSIxYBOB3Q5ff4YIXCuZc2Xc0hypxOYtLt k5lmVrtWnULUMdZG7Rc7r8nzDx1RodCiI1MJGVLX6GugLxbvWbUZL/gquOuPKse96jmC gaHGpjX+QqAPDsPKg1EJ+THgHpdRHwv7YmI+0+T37cNJ6Moi5nyW6eomg5WN058mL2QV Sjlx4hLLVUw0YODwCOIy2UuQRXM+MvbRYkM/1i+63p4KzRffsxjIWlmpzYYNdomvihPb ABZQ== X-Gm-Message-State: AO0yUKVr29aaaflH0cCXbJayZjr1L6nT7rIx9ud2js7/BMOUtd0PaYRl x75zba06jBXp9hrnGyCKGWnbclnUnO8SOXQucvWy X-Google-Smtp-Source: AK7set87bWVquSBcuZW8Ny+8Trz9/SMPleDpObSKtz0DFZyFj7iwR7+DMndg7acdQz2mXuY7FZwt1m8s/MP7pGIMbhYV X-Received: from axel.svl.corp.google.com ([2620:15c:2d4:203:17e9:c330:41ce:6b08]) (user=axelrasmussen job=sendgmr) by 2002:a25:8609:0:b0:a09:314f:9f09 with SMTP id y9-20020a258609000000b00a09314f9f09mr7650365ybk.6.1678143035096; Mon, 06 Mar 2023 14:50:35 -0800 (PST) Date: Mon, 6 Mar 2023 14:50:20 -0800 In-Reply-To: <20230306225024.264858-1-axelrasmussen@google.com> Mime-Version: 1.0 References: <20230306225024.264858-1-axelrasmussen@google.com> X-Mailer: git-send-email 2.40.0.rc0.216.gc4246ad0f0-goog Message-ID: <20230306225024.264858-2-axelrasmussen@google.com> Subject: [PATCH v3 1/5] mm: userfaultfd: rename functions for clarity + consistency From: Axel Rasmussen To: Alexander Viro , Andrew Morton , Hugh Dickins , Jan Kara , "Liam R. Howlett" , Matthew Wilcox , Mike Kravetz , Mike Rapoport , Muchun Song , Nadav Amit , Peter Xu , Shuah Khan Cc: James Houghton , linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-kselftest@vger.kernel.org, Axel Rasmussen X-Stat-Signature: ppk9g85ascc4ejq4ksf3rh9jt3w7kqi1 X-Rspam-User: X-Rspamd-Queue-Id: EF00F4000C X-Rspamd-Server: rspam06 X-HE-Tag: 1678143035-485936 X-HE-Meta: U2FsdGVkX1+jrB3A9BdEfjgpfa6/SB1wz0C+Lj9e9DJKgGYyhrj8XFYbh7oZA8aWf2XYKj1W7i1tF5uKCyUL1bxTSC6kVD7GPYIoYc72CADvcji6ESv67vIgefxjFL88zKDLSeno++oMa2bgcUrOBIR9o1SLZLj9dH8jCLrpNiKRz0ZH4oc5XHMxxPEsaQyrjKexMY8Pn4siwmHtDHR9MKwAyslP4thkFR2As8OHoxiVH891e9IU4H/2Y9P9N7vKF6gDwzboY0WbwxQqrYpNCB0drruZEBTl1D+jNR+TOi5NQnDZVFnxQi/89chtIpO5adyEXj1I1SX3f2houi/2XrrvYcK076G6NqM8vzqhr1drKDmYeZhd9FRXcQaNim8vpWHSwyG2NA3R2LeDjKYhZiqoB4Bqk7AMVyUjkk9jVDGA6cPt4cGBSN5UXS+GRdb3N+xuEhKPEXUCxKv8JADeRpa185pp0GVmlJM43MpDbebVqcgN6f66kJH75iRUnccjegEetdc4pZKADNhlHmzu9FeYxPO//fMDpl12/RITY9Urx1nXR+xK/TH14Re1HXfNR2mYFS95CSTmOTPs92ZcRGDTbl0tISnqeyehj9r33TrLYIuvoOLiFwTVEachpTPIu3j4oxSMv6ZHshxSC5b5MrsNZu4qVkXq5PKUlidz/yGSeqYRBiQxgsaA0Oz4ikx0qzqpwWhHEOqy2KKPxa797y9WX33OfskAdZs6leyzlDIn9D0Xx30XchXpDNs1jKPYjcR8YQXqtfsz+ukNzJlhGHPemNVVDTrorOYotU4ul2/gPywzV5B4rbEUjez2hBbO/e0FrQkG5/DageymjFeLpi7WkAeez5CkCkpRiOy1GAOhw6LQmnoTjzey+E3a+mpK5IsYF9kNSsINBvK+itibDszbcS9AXFGo1a+Dg2JbkfAQAanEW/zXWHBCzjFglkFVR2Mekau34KowsR4dypo PgyBNGGh Q17u3u1HWP+H2bJhBK8S3OCYTjI8n6BJ5LuZA4twdeLZsS8A3imS6q+CSLHJLYBuOAEqeNSnkuQ65+n+MsPF2JdKoD5clAW3dn8ytIT8UEef52zmBk1SwWW5eMN7foS9zQLP8O5DJ2Hu/wVusPESlyZPcPf2MxvsEyduKdfjsb5avvMIOpdINvXJTUIyVmE6nalHpcpP85s0w4YwdVvzDZEhpvsYNDZGbRwQVY3GxRTory8p6iH61nom51aMni+kuVMr0aiWf2CoKUOQmU/mF4UkQoQIobNLD6/RZ8OsdEmnajeP+h6pp7zoL+cKtDxreZx1Q5LagQNndzYrH2VJBRNt7PkurkZmirZIBSkzMY+O8zothdmD/KnVR0vbOyPoRt6otsck+f6KdyCRFTlupcCWSNdzCpf30MWE1GVZlN3xflD7eqsX1mVQLQj9ZNWeLjoIUuWbC/5JUYtcvuEC3+KLAiCZIWYNepj3u8gLe7A27UOJLDRKpOAKtJLGW8n9Qv2SBbnrW/MVqx3RAeSus7O6YGsSQK+3v7Wi5Cze1U3fSaV47EKIvJdRtQ6tOdG99+0HyjyKwwCjf3XwpkYla72/woYpX7CA1Ffqi06LT6sdytpQPvCmbhAEzU0vHKwSJdjwl/hd3+JHoCs7XrrgaXs5orCRai+mBvPjR7or78FBea7cGuXhkwGhePg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The basic problem is, over time we've added new userfaultfd ioctls, and we've refactored the code so functions which used to handle only one case are now re-used to deal with several cases. While this happened, we didn't bother to rename the functions. Similarly, as we added new functions, we cargo-culted pieces of the now-inconsistent naming scheme, so those functions too ended up with names that don't make a lot of sense. A key point here is, "copy" in most userfaultfd code refers specifically to UFFDIO_COPY, where we allocate a new page and copy its contents from userspace. There are many functions with "copy" in the name that don't actually do this (at least in some cases). So, rename things into a consistent scheme. The high level idea is that the call stack for userfaultfd ioctls becomes: userfaultfd_ioctl -> userfaultfd_(particular ioctl) -> mfill_atomic_(particular kind of fill operation) -> mfill_atomic /* loops over pages in range */ -> mfill_atomic_pte /* deals with single pages */ -> mfill_atomic_pte_(particular kind of fill operation) -> mfill_atomic_install_pte There are of course some special cases (shmem, hugetlb), but this is the general structure which all function names now adhere to. Signed-off-by: Axel Rasmussen Acked-by: Peter Xu --- fs/userfaultfd.c | 18 +++---- include/linux/hugetlb.h | 30 +++++------ include/linux/userfaultfd_k.h | 18 +++---- mm/hugetlb.c | 20 +++---- mm/userfaultfd.c | 98 +++++++++++++++++------------------ 5 files changed, 92 insertions(+), 92 deletions(-) diff --git a/fs/userfaultfd.c b/fs/userfaultfd.c index 44d1ee429eb0..365bf00dd8dd 100644 --- a/fs/userfaultfd.c +++ b/fs/userfaultfd.c @@ -1741,9 +1741,9 @@ static int userfaultfd_copy(struct userfaultfd_ctx *ctx, if (uffdio_copy.mode & ~(UFFDIO_COPY_MODE_DONTWAKE|UFFDIO_COPY_MODE_WP)) goto out; if (mmget_not_zero(ctx->mm)) { - ret = mcopy_atomic(ctx->mm, uffdio_copy.dst, uffdio_copy.src, - uffdio_copy.len, &ctx->mmap_changing, - uffdio_copy.mode); + ret = mfill_atomic_copy(ctx->mm, uffdio_copy.dst, uffdio_copy.src, + uffdio_copy.len, &ctx->mmap_changing, + uffdio_copy.mode); mmput(ctx->mm); } else { return -ESRCH; @@ -1793,9 +1793,9 @@ static int userfaultfd_zeropage(struct userfaultfd_ctx *ctx, goto out; if (mmget_not_zero(ctx->mm)) { - ret = mfill_zeropage(ctx->mm, uffdio_zeropage.range.start, - uffdio_zeropage.range.len, - &ctx->mmap_changing); + ret = mfill_atomic_zeropage(ctx->mm, uffdio_zeropage.range.start, + uffdio_zeropage.range.len, + &ctx->mmap_changing); mmput(ctx->mm); } else { return -ESRCH; @@ -1903,9 +1903,9 @@ static int userfaultfd_continue(struct userfaultfd_ctx *ctx, unsigned long arg) goto out; if (mmget_not_zero(ctx->mm)) { - ret = mcopy_continue(ctx->mm, uffdio_continue.range.start, - uffdio_continue.range.len, - &ctx->mmap_changing); + ret = mfill_atomic_continue(ctx->mm, uffdio_continue.range.start, + uffdio_continue.range.len, + &ctx->mmap_changing); mmput(ctx->mm); } else { return -ESRCH; diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h index 7c977d234aba..8f0467bf1cbd 100644 --- a/include/linux/hugetlb.h +++ b/include/linux/hugetlb.h @@ -158,13 +158,13 @@ unsigned long hugetlb_total_pages(void); vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long address, unsigned int flags); #ifdef CONFIG_USERFAULTFD -int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, pte_t *dst_pte, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - enum mcopy_atomic_mode mode, - struct page **pagep, - bool wp_copy); +int hugetlb_mfill_atomic_pte(struct mm_struct *dst_mm, pte_t *dst_pte, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + enum mcopy_atomic_mode mode, + struct page **pagep, + bool wp_copy); #endif /* CONFIG_USERFAULTFD */ bool hugetlb_reserve_pages(struct inode *inode, long from, long to, struct vm_area_struct *vma, @@ -393,14 +393,14 @@ static inline void hugetlb_free_pgd_range(struct mmu_gather *tlb, } #ifdef CONFIG_USERFAULTFD -static inline int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, - pte_t *dst_pte, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - enum mcopy_atomic_mode mode, - struct page **pagep, - bool wp_copy) +static inline int hugetlb_mfill_atomic_pte(struct mm_struct *dst_mm, + pte_t *dst_pte, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + enum mcopy_atomic_mode mode, + struct page **pagep, + bool wp_copy) { BUG(); return 0; diff --git a/include/linux/userfaultfd_k.h b/include/linux/userfaultfd_k.h index 3767f18114ef..468080125612 100644 --- a/include/linux/userfaultfd_k.h +++ b/include/linux/userfaultfd_k.h @@ -61,15 +61,15 @@ extern int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, unsigned long dst_addr, struct page *page, bool newly_allocated, bool wp_copy); -extern ssize_t mcopy_atomic(struct mm_struct *dst_mm, unsigned long dst_start, - unsigned long src_start, unsigned long len, - atomic_t *mmap_changing, __u64 mode); -extern ssize_t mfill_zeropage(struct mm_struct *dst_mm, - unsigned long dst_start, - unsigned long len, - atomic_t *mmap_changing); -extern ssize_t mcopy_continue(struct mm_struct *dst_mm, unsigned long dst_start, - unsigned long len, atomic_t *mmap_changing); +extern ssize_t mfill_atomic_copy(struct mm_struct *dst_mm, unsigned long dst_start, + unsigned long src_start, unsigned long len, + atomic_t *mmap_changing, __u64 mode); +extern ssize_t mfill_atomic_zeropage(struct mm_struct *dst_mm, + unsigned long dst_start, + unsigned long len, + atomic_t *mmap_changing); +extern ssize_t mfill_atomic_continue(struct mm_struct *dst_mm, unsigned long dst_start, + unsigned long len, atomic_t *mmap_changing); extern int mwriteprotect_range(struct mm_struct *dst_mm, unsigned long start, unsigned long len, bool enable_wp, atomic_t *mmap_changing); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 07abcb6eb203..4c9276549394 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6154,17 +6154,17 @@ vm_fault_t hugetlb_fault(struct mm_struct *mm, struct vm_area_struct *vma, #ifdef CONFIG_USERFAULTFD /* - * Used by userfaultfd UFFDIO_COPY. Based on mcopy_atomic_pte with - * modifications for huge pages. + * Used by userfaultfd UFFDIO_* ioctls. Based on userfaultfd's mfill_atomic_pte + * with modifications for hugetlb pages. */ -int hugetlb_mcopy_atomic_pte(struct mm_struct *dst_mm, - pte_t *dst_pte, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - enum mcopy_atomic_mode mode, - struct page **pagep, - bool wp_copy) +int hugetlb_mfill_atomic_pte(struct mm_struct *dst_mm, + pte_t *dst_pte, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + enum mcopy_atomic_mode mode, + struct page **pagep, + bool wp_copy) { bool is_continue = (mode == MCOPY_ATOMIC_CONTINUE); struct hstate *h = hstate_vma(dst_vma); diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c index 53c3d916ff66..84db5b2fad3a 100644 --- a/mm/userfaultfd.c +++ b/mm/userfaultfd.c @@ -127,13 +127,13 @@ int mfill_atomic_install_pte(struct mm_struct *dst_mm, pmd_t *dst_pmd, return ret; } -static int mcopy_atomic_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - unsigned long src_addr, - struct page **pagep, - bool wp_copy) +static int mfill_atomic_pte_copy(struct mm_struct *dst_mm, + pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + unsigned long src_addr, + struct page **pagep, + bool wp_copy) { void *page_kaddr; int ret; @@ -204,10 +204,10 @@ static int mcopy_atomic_pte(struct mm_struct *dst_mm, goto out; } -static int mfill_zeropage_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr) +static int mfill_atomic_pte_zeropage(struct mm_struct *dst_mm, + pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr) { pte_t _dst_pte, *dst_pte; spinlock_t *ptl; @@ -240,11 +240,11 @@ static int mfill_zeropage_pte(struct mm_struct *dst_mm, } /* Handles UFFDIO_CONTINUE for all shmem VMAs (shared or private). */ -static int mcontinue_atomic_pte(struct mm_struct *dst_mm, - pmd_t *dst_pmd, - struct vm_area_struct *dst_vma, - unsigned long dst_addr, - bool wp_copy) +static int mfill_atomic_pte_continue(struct mm_struct *dst_mm, + pmd_t *dst_pmd, + struct vm_area_struct *dst_vma, + unsigned long dst_addr, + bool wp_copy) { struct inode *inode = file_inode(dst_vma->vm_file); pgoff_t pgoff = linear_page_index(dst_vma, dst_addr); @@ -307,10 +307,10 @@ static pmd_t *mm_alloc_pmd(struct mm_struct *mm, unsigned long address) #ifdef CONFIG_HUGETLB_PAGE /* - * __mcopy_atomic processing for HUGETLB vmas. Note that this routine is + * mfill_atomic processing for HUGETLB vmas. Note that this routine is * called with mmap_lock held, it will release mmap_lock before returning. */ -static __always_inline ssize_t __mcopy_atomic_hugetlb(struct mm_struct *dst_mm, +static __always_inline ssize_t mfill_atomic_hugetlb(struct mm_struct *dst_mm, struct vm_area_struct *dst_vma, unsigned long dst_start, unsigned long src_start, @@ -411,7 +411,7 @@ static __always_inline ssize_t __mcopy_atomic_hugetlb(struct mm_struct *dst_mm, goto out_unlock; } - err = hugetlb_mcopy_atomic_pte(dst_mm, dst_pte, dst_vma, + err = hugetlb_mfill_atomic_pte(dst_mm, dst_pte, dst_vma, dst_addr, src_addr, mode, &page, wp_copy); @@ -463,7 +463,7 @@ static __always_inline ssize_t __mcopy_atomic_hugetlb(struct mm_struct *dst_mm, } #else /* !CONFIG_HUGETLB_PAGE */ /* fail at build time if gcc attempts to use this */ -extern ssize_t __mcopy_atomic_hugetlb(struct mm_struct *dst_mm, +extern ssize_t mfill_atomic_hugetlb(struct mm_struct *dst_mm, struct vm_area_struct *dst_vma, unsigned long dst_start, unsigned long src_start, @@ -484,8 +484,8 @@ static __always_inline ssize_t mfill_atomic_pte(struct mm_struct *dst_mm, ssize_t err; if (mode == MCOPY_ATOMIC_CONTINUE) { - return mcontinue_atomic_pte(dst_mm, dst_pmd, dst_vma, dst_addr, - wp_copy); + return mfill_atomic_pte_continue(dst_mm, dst_pmd, dst_vma, + dst_addr, wp_copy); } /* @@ -500,11 +500,11 @@ static __always_inline ssize_t mfill_atomic_pte(struct mm_struct *dst_mm, */ if (!(dst_vma->vm_flags & VM_SHARED)) { if (mode == MCOPY_ATOMIC_NORMAL) - err = mcopy_atomic_pte(dst_mm, dst_pmd, dst_vma, - dst_addr, src_addr, page, - wp_copy); + err = mfill_atomic_pte_copy(dst_mm, dst_pmd, dst_vma, + dst_addr, src_addr, page, + wp_copy); else - err = mfill_zeropage_pte(dst_mm, dst_pmd, + err = mfill_atomic_pte_zeropage(dst_mm, dst_pmd, dst_vma, dst_addr); } else { err = shmem_mfill_atomic_pte(dst_mm, dst_pmd, dst_vma, @@ -516,13 +516,13 @@ static __always_inline ssize_t mfill_atomic_pte(struct mm_struct *dst_mm, return err; } -static __always_inline ssize_t __mcopy_atomic(struct mm_struct *dst_mm, - unsigned long dst_start, - unsigned long src_start, - unsigned long len, - enum mcopy_atomic_mode mcopy_mode, - atomic_t *mmap_changing, - __u64 mode) +static __always_inline ssize_t mfill_atomic(struct mm_struct *dst_mm, + unsigned long dst_start, + unsigned long src_start, + unsigned long len, + enum mcopy_atomic_mode mcopy_mode, + atomic_t *mmap_changing, + __u64 mode) { struct vm_area_struct *dst_vma; ssize_t err; @@ -588,9 +588,9 @@ static __always_inline ssize_t __mcopy_atomic(struct mm_struct *dst_mm, * If this is a HUGETLB vma, pass off to appropriate routine */ if (is_vm_hugetlb_page(dst_vma)) - return __mcopy_atomic_hugetlb(dst_mm, dst_vma, dst_start, - src_start, len, mcopy_mode, - wp_copy); + return mfill_atomic_hugetlb(dst_mm, dst_vma, dst_start, + src_start, len, mcopy_mode, + wp_copy); if (!vma_is_anonymous(dst_vma) && !vma_is_shmem(dst_vma)) goto out_unlock; @@ -688,26 +688,26 @@ static __always_inline ssize_t __mcopy_atomic(struct mm_struct *dst_mm, return copied ? copied : err; } -ssize_t mcopy_atomic(struct mm_struct *dst_mm, unsigned long dst_start, - unsigned long src_start, unsigned long len, - atomic_t *mmap_changing, __u64 mode) +ssize_t mfill_atomic_copy(struct mm_struct *dst_mm, unsigned long dst_start, + unsigned long src_start, unsigned long len, + atomic_t *mmap_changing, __u64 mode) { - return __mcopy_atomic(dst_mm, dst_start, src_start, len, - MCOPY_ATOMIC_NORMAL, mmap_changing, mode); + return mfill_atomic(dst_mm, dst_start, src_start, len, + MCOPY_ATOMIC_NORMAL, mmap_changing, mode); } -ssize_t mfill_zeropage(struct mm_struct *dst_mm, unsigned long start, - unsigned long len, atomic_t *mmap_changing) +ssize_t mfill_atomic_zeropage(struct mm_struct *dst_mm, unsigned long start, + unsigned long len, atomic_t *mmap_changing) { - return __mcopy_atomic(dst_mm, start, 0, len, MCOPY_ATOMIC_ZEROPAGE, - mmap_changing, 0); + return mfill_atomic(dst_mm, start, 0, len, MCOPY_ATOMIC_ZEROPAGE, + mmap_changing, 0); } -ssize_t mcopy_continue(struct mm_struct *dst_mm, unsigned long start, - unsigned long len, atomic_t *mmap_changing) +ssize_t mfill_atomic_continue(struct mm_struct *dst_mm, unsigned long start, + unsigned long len, atomic_t *mmap_changing) { - return __mcopy_atomic(dst_mm, start, 0, len, MCOPY_ATOMIC_CONTINUE, - mmap_changing, 0); + return mfill_atomic(dst_mm, start, 0, len, MCOPY_ATOMIC_CONTINUE, + mmap_changing, 0); } long uffd_wp_range(struct mm_struct *dst_mm, struct vm_area_struct *dst_vma,