From patchwork Thu Apr 11 13:09:49 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13625902 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D8A44CD128A for ; Thu, 11 Apr 2024 12:42:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6D3B66B009F; Thu, 11 Apr 2024 08:42:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 65B4A6B00A1; Thu, 11 Apr 2024 08:42:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FC7B6B00A0; Thu, 11 Apr 2024 08:42:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 328FC6B009E for ; Thu, 11 Apr 2024 08:42:45 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id DFF3DA09F1 for ; Thu, 11 Apr 2024 12:42:44 +0000 (UTC) X-FDA: 81997215048.01.BA1AF46 Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) by imf07.hostedemail.com (Postfix) with ESMTP id 7ECAC40017 for ; Thu, 11 Apr 2024 12:42:42 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712839363; a=rsa-sha256; cv=none; b=n87QOuEvHxcpGveca2/UxgWpmGm1iga0FwMBtIyjydssFDtNGHhVp+j2GHoKvC4CU+e7vk DdmK0qjwJRnRlv82f7R7gkZ+Qoi6DucoWrjV0y0zjMhTXNgisjhPNA1pPYjQzQ6fxMfhMf GkROfeBXoSF2hXqGaTHcQnTo5sdjHqI= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=none; spf=pass (imf07.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712839363; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=IEJmb/kdNeQ657RH30YmHLEYuRyLliPn1+PXFXffrvo=; b=jeNXjPDRyuUjWwS737q1E9M6EvxUceell62mWz3DY1V4VCJxlRdKaLHhgnuw4MgKxSvSdv JnyeKUJIQDFqY7JmeoN1uPrf4i1okNX8X1rXI968DA5yCsyLQYpkl6qXj6XA03LiMcFRoW In9cJXX6c42i1jIx4azbfFhPShPn698= Received: from mail.maildlp.com (unknown [172.19.162.112]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4VFfRp4M65z1yn4d; Thu, 11 Apr 2024 20:40:22 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 768A71400D9; Thu, 11 Apr 2024 20:42:39 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Thu, 11 Apr 2024 20:42:39 +0800 From: Kefeng Wang To: Andrew Morton CC: "Matthew Wilcox (Oracle)" , , , Kefeng Wang Subject: [PATCH 1/2] mm: move mm counter updating out of set_pte_range() Date: Thu, 11 Apr 2024 21:09:49 +0800 Message-ID: <20240411130950.73512-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240411130950.73512-1-wangkefeng.wang@huawei.com> References: <20240411130950.73512-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems703-chm.china.huawei.com (10.3.19.180) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: 7ECAC40017 X-Stat-Signature: 3aygbbhzx6ys4obh3fectiyx8krj734e X-HE-Tag: 1712839362-410163 X-HE-Meta: U2FsdGVkX1+Lmi3vX1u5DLlKAXSPlXAO3BIfuVpzr6KP8rYFRcXUEylWC9JHFBDxKAdckap9Pkvjq6fjd0YzyTcAXwlesDEmjEeKeyK8IMnK1Vb17zizHU0dnLOg5kZfeAMgwbfdad1zZpzwlAIeZhR8kL3p3uYQmbMgz2sNE/F/+PHaVMI041fkkT0ylkACMFMnUWKvVntGGzrf7G8RWn3Gt1N8Pnm68GuQI5UdqYeAeYJTK1pxwQlB6xaUH4nQXNCem8785ttK3sjSbqGQM15bbOh0L+O2eDaYoWGj9cWzyOnn200fRX4qm/rTIpUWbVEfR/uPy/DJbl8Toh+3kbnyXdLz+usW5VPVRc+rep5X1Xk+CMei97qh7rr0g42PLlWHk8n0Yw+dzVfWjOTW59+vmIEDtBX6A4R3B6TBg6m8VAERmiLbblA0CxNj7bbxARNfM9qld9fGdvtWcQXodFy11ZO+eS3DIQAgQm6/vF/iJOMBEECQthvDLSts1mAW1AgkTZT4sfmkGSBXagKXd1qPj86RW8HLyysiZzGARt8VItI4raMZmMTrbHu05YOXRglx3wiSBuCB8pqX61nd490YJgqU3Qhyuul+jxUE7vYsABRPNhB+gkjp8NNQBfD95SbE1v0GQ6TEmEkVeUv/NDqOxiEeAjZkHZowgUy67xBOfcZHJQFkx/iIP9lZzLFGkNZPW2TvTarvoGKPfopME3R6jTO425ImK/b1ivDO7U/8gffFiLwAT4W8kl5f1TtPYB40Pszcdre7WfCHRTEP/6oSmv8HkSdLvtCLii7ePONSvxI64gff6/NsxbwwmtweTWmNx9YnV4uVnqRP343V4mjRtx2yM4ielSeIHKhvyCWZSyYe9ApkrQOjCNNYWtSWPJJrmcU2NPO/Kr//TpJknufkwc2jIIJ50woD+qPYd6DTlZ91nLPrlnhfrIiIJ9+e5TroGzuPMmEvOsS852m 3cLEV6CQ a8MGYx/PIrrAmHYC6atSEkSGfK8eHTB9SG7as3z96cCGxb6hXpZW+3SqGr5SGFEtPLi3Q0voQrzNTKdOmiNY2yxAjOfl6vI/qBc0A7IICqgV8+0hM9hKA6EwJp3UqhpDxVa/PFcSwU/9dtuf6yMu6SJKD3wqWqKhSzmKWEswACVDeJOHp+5afAGfiLioZArxdu/nm X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In order to support batch mm counter updating in filemap_map_pages(), make set_pte_range() return the type of MM_COUNTERS and move mm counter updating out of set_pte_range(). Signed-off-by: Kefeng Wang --- include/linux/mm.h | 4 ++-- mm/filemap.c | 10 +++++++--- mm/memory.c | 16 +++++++++++----- 3 files changed, 20 insertions(+), 10 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 0b4046b1e63d..6ad440ac3706 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1366,8 +1366,8 @@ static inline pte_t maybe_mkwrite(pte_t pte, struct vm_area_struct *vma) } vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page); -void set_pte_range(struct vm_fault *vmf, struct folio *folio, - struct page *page, unsigned int nr, unsigned long addr); +int set_pte_range(struct vm_fault *vmf, struct folio *folio, + struct page *page, unsigned int nr, unsigned long addr); vm_fault_t finish_fault(struct vm_fault *vmf); #endif diff --git a/mm/filemap.c b/mm/filemap.c index 92e2d43e4c9d..2274e590bab4 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3512,6 +3512,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, struct page *page = folio_page(folio, start); unsigned int count = 0; pte_t *old_ptep = vmf->pte; + int type; do { if (PageHWPoison(page + count)) @@ -3539,7 +3540,8 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, continue; skip: if (count) { - set_pte_range(vmf, folio, page, count, addr); + type = set_pte_range(vmf, folio, page, count, addr); + add_mm_counter(vmf->vma->vm_mm, type, count); folio_ref_add(folio, count); if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; @@ -3553,7 +3555,8 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, } while (--nr_pages > 0); if (count) { - set_pte_range(vmf, folio, page, count, addr); + type = set_pte_range(vmf, folio, page, count, addr); + add_mm_counter(vmf->vma->vm_mm, type, count); folio_ref_add(folio, count); if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; @@ -3589,7 +3592,8 @@ static vm_fault_t filemap_map_order0_folio(struct vm_fault *vmf, if (vmf->address == addr) ret = VM_FAULT_NOPAGE; - set_pte_range(vmf, folio, page, 1, addr); + add_mm_counter(vmf->vma->vm_mm, + set_pte_range(vmf, folio, page, 1, addr), 1); folio_ref_inc(folio); return ret; diff --git a/mm/memory.c b/mm/memory.c index 78422d1c7381..485ffec9d4c7 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4661,15 +4661,18 @@ vm_fault_t do_set_pmd(struct vm_fault *vmf, struct page *page) * @page: The first page to create a PTE for. * @nr: The number of PTEs to create. * @addr: The first address to create a PTE for. + * + * Return: type of MM_COUNTERS to be updated */ -void set_pte_range(struct vm_fault *vmf, struct folio *folio, - struct page *page, unsigned int nr, unsigned long addr) +int set_pte_range(struct vm_fault *vmf, struct folio *folio, + struct page *page, unsigned int nr, unsigned long addr) { struct vm_area_struct *vma = vmf->vma; bool uffd_wp = vmf_orig_pte_uffd_wp(vmf); bool write = vmf->flags & FAULT_FLAG_WRITE; bool prefault = in_range(vmf->address, addr, nr * PAGE_SIZE); pte_t entry; + int type; flush_icache_pages(vma, page, nr); entry = mk_pte(page, vma->vm_page_prot); @@ -4685,18 +4688,20 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio, entry = pte_mkuffd_wp(entry); /* copy-on-write page */ if (write && !(vma->vm_flags & VM_SHARED)) { - add_mm_counter(vma->vm_mm, MM_ANONPAGES, nr); + type = MM_ANONPAGES; VM_BUG_ON_FOLIO(nr != 1, folio); folio_add_new_anon_rmap(folio, vma, addr); folio_add_lru_vma(folio, vma); } else { - add_mm_counter(vma->vm_mm, mm_counter_file(folio), nr); + type = mm_counter_file(folio); folio_add_file_rmap_ptes(folio, page, nr, vma); } set_ptes(vma->vm_mm, addr, vmf->pte, entry, nr); /* no need to invalidate: a not-present page won't be cached */ update_mmu_cache_range(vmf, vma, addr, vmf->pte, nr); + + return type; } static bool vmf_pte_changed(struct vm_fault *vmf) @@ -4765,8 +4770,9 @@ vm_fault_t finish_fault(struct vm_fault *vmf) /* Re-check under ptl */ if (likely(!vmf_pte_changed(vmf))) { struct folio *folio = page_folio(page); + int type = set_pte_range(vmf, folio, page, 1, vmf->address); - set_pte_range(vmf, folio, page, 1, vmf->address); + add_mm_counter(vmf->vma->vm_mm, type, 1); ret = 0; } else { update_mmu_tlb(vma, vmf->address, vmf->pte);