From patchwork Fri Mar 31 09:39:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Peng Zhang X-Patchwork-Id: 13195698 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8DB54C77B62 for ; Fri, 31 Mar 2023 09:42:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 58B5A6B0078; Fri, 31 Mar 2023 05:41:59 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4EFA66B007E; Fri, 31 Mar 2023 05:41:59 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 2A2FF6B0080; Fri, 31 Mar 2023 05:41:59 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 0E8466B0078 for ; Fri, 31 Mar 2023 05:41:59 -0400 (EDT) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id DF8FE1A0BF9 for ; Fri, 31 Mar 2023 09:41:58 +0000 (UTC) X-FDA: 80628701916.15.0735D5A Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf01.hostedemail.com (Postfix) with ESMTP id 6E7664000D for ; Fri, 31 Mar 2023 09:41:56 +0000 (UTC) Authentication-Results: imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of zhangpeng362@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=zhangpeng362@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1680255717; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=qy4t3Bl+06JFzUuNl0JyheFOKRrvnzWEBhKrgFbkbZk=; b=J1Eig0cG78XGU+Z3T89I171xv60LImZumR+qx3QXhb1iKjOnsE85+NZRJyRsJjrcACiUOG bM8cmakEtSuMZCop8bjJhtW+7254st6cFthw4vBpyKZVSrySKb6vjWHURjy/+ElEK1HjXe /XcvPNZRT4Nn/TTw7NkzPACTIK4C8ok= ARC-Authentication-Results: i=1; imf01.hostedemail.com; dkim=none; spf=pass (imf01.hostedemail.com: domain of zhangpeng362@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=zhangpeng362@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1680255717; a=rsa-sha256; cv=none; b=IMdJlOncQ12WZSWyk/C965SBnqjjQh5WUI/y372i3KYU1j4lFTEmAyfbixsbvAF7KQZBzZ cCraxir6hBSpkFyet8X2ui6MVYuDpJpJ4sHVs7ZCi/M4YyrkAIML/qXzhhjVRwCRlvqXxI VfhnF2znpSCp+3YPpOvhjufCGx1A7T4= Received: from kwepemm600020.china.huawei.com (unknown [172.30.72.54]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4PnwG23pwfz17QVK; Fri, 31 Mar 2023 17:38:34 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by kwepemm600020.china.huawei.com (7.193.23.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.21; Fri, 31 Mar 2023 17:41:51 +0800 From: Peng Zhang To: , , , , , CC: , , , , ZhangPeng Subject: [PATCH v5 5/6] mm: convert copy_user_huge_page() to copy_user_folio() Date: Fri, 31 Mar 2023 17:39:36 +0800 Message-ID: <20230331093937.945725-6-zhangpeng362@huawei.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230331093937.945725-1-zhangpeng362@huawei.com> References: <20230331093937.945725-1-zhangpeng362@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To kwepemm600020.china.huawei.com (7.193.23.147) X-CFilter-Loop: Reflected X-Rspam-User: X-Rspamd-Server: rspam03 X-Stat-Signature: iii6ods5bdgkt167o9z1kea6mqwhx6k7 X-Rspamd-Queue-Id: 6E7664000D X-HE-Tag: 1680255716-784278 X-HE-Meta: U2FsdGVkX19hv0BoJwCqFh4IgBM9/YTCFo5XmxF8J9c6NB5Q1AuNWqZ4LNtah5VSG8lE5yZDi9F0PmvLec5fQG4sX1axQ2BLNEsDP9Xxn/ZnGqL4zx2I4Xd2smeETpwx3H+GFnRgLh1boBC6KlfAOUU+gNwgGh1hfWHRx9XOwEGBpP2eSWU/g9QwJLgrbRRWh4Pb1VXOPS8LYH/SSZo6L8A9oLVUEDnBWJrsTk8vnW2hVR7+pHxQCzencAfYVbIY9Hr9TXdvRD6vfS/Nrvf7ACLOwkJqCs7h+H28DGxIy9j6v47kpE3JeU5mUNUigwU49dQ9m0JHpp6zmHQE9qqUpuYmtjioNkF0yW6hlJoVlCSPHGs2qviTqyYZM4USqg7WzB34v7UGshDhgIxwMh3M8CuN9v55S9jzLE/sO+I30NEv9EX27Tt5GO0JfeUEOd4iJ9dB4PPtwjmppnIAjP1H5HLcWupR8Gzt8KqfIbqIAyNfsLZTpvz9PLMN3b3JpSGPH2Uk5WnFsLXQPr0JHQz2CIEBBaz1s5tX1ppQs6vA1ulsvK48cyO4N/uEifbV0rRXDNNgiq7WtnsCsqBISra91YJYQrdY2GrBFaB4AzR1RjpvIsHFbdwxJ1WJj5iHPDrxchd+J1lxWHkfjpaERH4YLFLN9QOiLvB7j16hTAdmiO6UIkKkmPAESkbPN0v8y7iX6yHI+1ilJXhc+KjXBOzVxZGZ9EWZQda+yygFCuyBtOJiOlXT787tIKnVn3Slx+DUdMeGqg0Fxgb9B8XmNWFvh43eA06CFrPtWr5RnqfP31TJuzykjh7XwgDS2e3T64S5mbtLuyD9tad79jv7b4dxx+kucpCIJIzbbKM8REBMPSDYeUm1v8cXm4KHGHmSx7+NjV3IYKks6tpt01TpMqQ2qERbcBjuzDhDgZQkaQE6Nc8TWBRfnlKLjBmWZEt8gzK5SGg9+c5JEd9JGUGof0y d0LoiuRD W2lVC2bN7+G7dwGgTZEBfRjtC1QrKNbYMiaJTCOP72ISN1a7uaoFuAF9afh6Majs5ILLyABQYyF3+oILDMfSx4kfncKxSf4HHWKbfFDZvUbuqdwOONoSP46sjNJXVrcff05rFLRHimqghwNUFZct42HRFUSVTxMiZEtlnvEP3xFcpicAY5SvLc/H4ig05P64B+A4HJxHNDAm+/sHyIS3cQLUINOJ6UckgIZ3hWq0VArCi3xRKmLtaTNIh/Hii0mODDyBeHtJ6XOOo0h/k25+deYEs4Q== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: ZhangPeng Replace copy_user_huge_page() with copy_user_folio(). copy_user_folio() does the same as copy_user_huge_page(), but takes in folios instead of pages. Convert copy_user_gigantic_page() to take in folios. Remove pages_per_huge_page from copy_user_folio(), because we can get that from folio_nr_pages(dst). Signed-off-by: ZhangPeng --- include/linux/mm.h | 7 +++---- mm/hugetlb.c | 10 ++++------ mm/memory.c | 28 ++++++++++++++-------------- 3 files changed, 21 insertions(+), 24 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index cf4d773ca506..898ece0a3802 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3678,10 +3678,9 @@ extern const struct attribute_group memory_failure_attr_group; extern void clear_huge_page(struct page *page, unsigned long addr_hint, unsigned int pages_per_huge_page); -extern void copy_user_huge_page(struct page *dst, struct page *src, - unsigned long addr_hint, - struct vm_area_struct *vma, - unsigned int pages_per_huge_page); +void copy_user_folio(struct folio *dst, struct folio *src, + unsigned long addr_hint, + struct vm_area_struct *vma); long copy_folio_from_user(struct folio *dst_folio, const void __user *usr_src, bool allow_pagefault); diff --git a/mm/hugetlb.c b/mm/hugetlb.c index c88f856ec2e2..a7ed17cbc84e 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -5097,8 +5097,8 @@ int copy_hugetlb_page_range(struct mm_struct *dst, struct mm_struct *src, ret = PTR_ERR(new_folio); break; } - copy_user_huge_page(&new_folio->page, ptepage, addr, dst_vma, - npages); + copy_user_folio(new_folio, page_folio(ptepage), + addr, dst_vma); put_page(ptepage); /* Install the new hugetlb folio if src pte stable */ @@ -5616,8 +5616,7 @@ static vm_fault_t hugetlb_wp(struct mm_struct *mm, struct vm_area_struct *vma, goto out_release_all; } - copy_user_huge_page(&new_folio->page, old_page, address, vma, - pages_per_huge_page(h)); + copy_user_folio(new_folio, page_folio(old_page), address, vma); __folio_mark_uptodate(new_folio); mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, mm, haddr, @@ -6260,8 +6259,7 @@ int hugetlb_mfill_atomic_pte(pte_t *dst_pte, *foliop = NULL; goto out; } - copy_user_huge_page(&folio->page, &(*foliop)->page, dst_addr, dst_vma, - pages_per_huge_page(h)); + copy_user_folio(folio, *foliop, dst_addr, dst_vma); folio_put(*foliop); *foliop = NULL; } diff --git a/mm/memory.c b/mm/memory.c index 4976422b6979..0a5cefea9774 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5815,21 +5815,21 @@ void clear_huge_page(struct page *page, process_huge_page(addr_hint, pages_per_huge_page, clear_subpage, page); } -static void copy_user_gigantic_page(struct page *dst, struct page *src, - unsigned long addr, - struct vm_area_struct *vma, - unsigned int pages_per_huge_page) +static void copy_user_gigantic_page(struct folio *dst, struct folio *src, + unsigned long addr, + struct vm_area_struct *vma, + unsigned int pages_per_huge_page) { int i; - struct page *dst_base = dst; - struct page *src_base = src; + struct page *dst_page; + struct page *src_page; for (i = 0; i < pages_per_huge_page; i++) { - dst = nth_page(dst_base, i); - src = nth_page(src_base, i); + dst_page = folio_page(dst, i); + src_page = folio_page(src, i); cond_resched(); - copy_user_highpage(dst, src, addr + i*PAGE_SIZE, vma); + copy_user_highpage(dst_page, src_page, addr + i*PAGE_SIZE, vma); } } @@ -5847,15 +5847,15 @@ static void copy_subpage(unsigned long addr, int idx, void *arg) addr, copy_arg->vma); } -void copy_user_huge_page(struct page *dst, struct page *src, - unsigned long addr_hint, struct vm_area_struct *vma, - unsigned int pages_per_huge_page) +void copy_user_folio(struct folio *dst, struct folio *src, + unsigned long addr_hint, struct vm_area_struct *vma) { + unsigned int pages_per_huge_page = folio_nr_pages(dst); unsigned long addr = addr_hint & ~(((unsigned long)pages_per_huge_page << PAGE_SHIFT) - 1); struct copy_subpage_arg arg = { - .dst = dst, - .src = src, + .dst = &dst->page, + .src = &src->page, .vma = vma, };