From patchwork Fri Apr 12 02:57:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13626703 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5ADA2C001CC for ; Fri, 12 Apr 2024 02:30:06 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C093D6B009C; Thu, 11 Apr 2024 22:30:04 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B11F46B009B; Thu, 11 Apr 2024 22:30:04 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7BFE56B009C; Thu, 11 Apr 2024 22:30:04 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 50E4D6B009A for ; Thu, 11 Apr 2024 22:30:04 -0400 (EDT) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id E195C1C0F9C for ; Fri, 12 Apr 2024 02:30:03 +0000 (UTC) X-FDA: 81999299886.03.F9B6F41 Received: from szxga04-in.huawei.com (szxga04-in.huawei.com [45.249.212.190]) by imf05.hostedemail.com (Postfix) with ESMTP id C92C3100007 for ; Fri, 12 Apr 2024 02:30:00 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712889002; a=rsa-sha256; cv=none; b=RY2F2w6PdomrQRiEPfcGJzuS+yfXKyLfy/uTTOmd3rl8FHhC/svohUDVRP4ddDTWTTkKMN WjZeZ+YXMuxrmb2JLILy6wArxNBC23QPbpcKkhD6lfv2dhF1iKt+nxmhtPTs/pg3+dhodI AJIj+hia5hrv1uiU3PD8mcTQJpmR7n8= ARC-Authentication-Results: i=1; imf05.hostedemail.com; dkim=none; spf=pass (imf05.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.190 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712889002; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=9KbkjvmuALWONra0TJwyRfZFDCz2BiUAn7df+x0JPP4=; b=xJDjSorF0gNIRxVxF9z5LunUCFIE1p8bn8wS7mZ5gt6VQZ/2f3oCPjd0PMKHctO200Sjie EWQQveRpP7vbv5ehhiMH25Xy7e28IhS7LvDGxNilZVoRbUau2epYOzZdWLatkfh5AR1Woi nidxaVdqtU9L8fApUbRYKAVoZoCPNN8= Received: from mail.maildlp.com (unknown [172.19.163.44]) by szxga04-in.huawei.com (SkyGuard) with ESMTP id 4VG0pM4JXZz1ynHs; Fri, 12 Apr 2024 10:27:39 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 19E5B140410; Fri, 12 Apr 2024 10:29:57 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Fri, 12 Apr 2024 10:29:56 +0800 From: Kefeng Wang To: Andrew Morton CC: "Matthew Wilcox (Oracle)" , , , Kefeng Wang Subject: [PATCH 1/2] mm: move mm counter updating out of set_pte_range() Date: Fri, 12 Apr 2024 10:57:03 +0800 Message-ID: <20240412025704.53245-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240412025704.53245-1-wangkefeng.wang@huawei.com> References: <20240412025704.53245-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems704-chm.china.huawei.com (10.3.19.181) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspam-User: X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: C92C3100007 X-Stat-Signature: 4fppbhm9rt8dwn5zq9576awofk6f8kh4 X-HE-Tag: 1712889000-406820 X-HE-Meta: U2FsdGVkX1/YoqW++ugU7tUHOo4T+Yt9Xh1yJ8wecMhhfBf/BvEppnY6q6/zP/REjM2jUr5CxteWtI75R4HhvRK10UzGEYAUcxA9Rp71E513DezR7MkM0GddiZL8xSSueUFin0VzozkfN9JV3ttCK7W6k0zGuonYg9q/QN08eLvVbZyr3UBeRx0H4WcLJKHJ7g+vsAqONE3KI6kHSy8u3SZZaGqQu6HXKg8Z5Dxr3cBliHDzDb9C/wfPQ5qXeoK9JGfmM1Kt5RLx35KKqklpCYtkQ7EAUQgTWdQcqK088Uj0HWeJjVlQZ1VMfgZO55Tw1okqCZ3t6BbvnOFiZsMWMmOwdDNiXNbLiYleH1iYG2YWf0NAcXXdu4o1S04wd+Bbyf4YZf3C71RUK2RTjDGBtVu6SrIR0jJPMCrlhQ0doFtV3O8sZ4Q3TlHMvg/mwtMHTPA68t8hL8qORiY40/tHYDUve8z2F+DQ9dSFLmE4VXXBOkZYT/qavyvkwjpDMo3c4JsWaYCx1PMINdlm0GkUDE3F/4Y3Sx6vedIYYd79qcYgOYL2IuhenvmVqhYhC8K9t+iJ1ab4vlpYr6SKRdQ2+WnDDmrBD3CisQ0zbTB4mzBmyuoa7dMtu5kBeXpSIJG7FM/Zz6mEFcqRLa5WLUi3bBbzokeNYVUmLia2yoJpYva8nFYsi4G2MPAJuJqyDfDkpKaBLAexTVfZGTRhjMvAbTSwERd2LiHmST6zjoEDnjI8lTTnV/MX+H0HugUnLBEUwmKB1KeTc1ge2Es+ACdFpsvhQBAW9S6/SwsbZoARKiirts/dRd5lbCVFxxapxCPO75lg29sEYkXj0vhcwKqGR4DfCJAA6MoMS96RTMJeyckzxeRSl4oUnIVC1IfqxIryu7/ZaiGD2P8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In order to support batch mm counter updating in filemap_map_pages(), move mm counter updating out of set_pte_range(), the folios are file from filemap, and distinguish folios type by vmf->flags and vma->vm_flags from another caller finish_fault(). Signed-off-by: Kefeng Wang --- mm/filemap.c | 4 ++++ mm/memory.c | 8 +++++--- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 92e2d43e4c9d..04b813f0146c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3540,6 +3540,8 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, skip: if (count) { set_pte_range(vmf, folio, page, count, addr); + add_mm_counter(vmf->vma->vm_mm, mm_counter_file(folio), + count); folio_ref_add(folio, count); if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; @@ -3554,6 +3556,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, if (count) { set_pte_range(vmf, folio, page, count, addr); + add_mm_counter(vmf->vma->vm_mm, mm_counter_file(folio), count); folio_ref_add(folio, count); if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; @@ -3590,6 +3593,7 @@ static vm_fault_t filemap_map_order0_folio(struct vm_fault *vmf, ret = VM_FAULT_NOPAGE; set_pte_range(vmf, folio, page, 1, addr); + add_mm_counter(vmf->vma->vm_mm, mm_counter_file(folio), 1); folio_ref_inc(folio); return ret; diff --git a/mm/memory.c b/mm/memory.c index 78422d1c7381..69bc63a5d6c8 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4685,12 +4685,10 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio, entry = pte_mkuffd_wp(entry); /* copy-on-write page */ if (write && !(vma->vm_flags & VM_SHARED)) { - add_mm_counter(vma->vm_mm, MM_ANONPAGES, nr); VM_BUG_ON_FOLIO(nr != 1, folio); folio_add_new_anon_rmap(folio, vma, addr); folio_add_lru_vma(folio, vma); } else { - add_mm_counter(vma->vm_mm, mm_counter_file(folio), nr); folio_add_file_rmap_ptes(folio, page, nr, vma); } set_ptes(vma->vm_mm, addr, vmf->pte, entry, nr); @@ -4727,9 +4725,11 @@ vm_fault_t finish_fault(struct vm_fault *vmf) struct vm_area_struct *vma = vmf->vma; struct page *page; vm_fault_t ret; + int is_cow = (vmf->flags & FAULT_FLAG_WRITE) && + !(vma->vm_flags & VM_SHARED); /* Did we COW the page? */ - if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) + if (is_cow) page = vmf->cow_page; else page = vmf->page; @@ -4765,8 +4765,10 @@ vm_fault_t finish_fault(struct vm_fault *vmf) /* Re-check under ptl */ if (likely(!vmf_pte_changed(vmf))) { struct folio *folio = page_folio(page); + int type = is_cow ? MM_ANONPAGES : mm_counter_file(folio); set_pte_range(vmf, folio, page, 1, vmf->address); + add_mm_counter(vma->vm_mm, type, 1); ret = 0; } else { update_mmu_tlb(vma, vmf->address, vmf->pte);