From patchwork Fri Apr 12 06:47:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13626907 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48DB2C4345F for ; Fri, 12 Apr 2024 06:20:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC90D6B0093; Fri, 12 Apr 2024 02:20:48 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A0F996B0092; Fri, 12 Apr 2024 02:20:48 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8AFD56B008C; Fri, 12 Apr 2024 02:20:48 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 6AC7E6B0087 for ; Fri, 12 Apr 2024 02:20:48 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id ED48440D58 for ; Fri, 12 Apr 2024 06:20:47 +0000 (UTC) X-FDA: 81999881334.16.471E0D1 Received: from szxga06-in.huawei.com (szxga06-in.huawei.com [45.249.212.32]) by imf11.hostedemail.com (Postfix) with ESMTP id 7A9C240004 for ; Fri, 12 Apr 2024 06:20:45 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1712902846; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=cLApvnGYBOs88sL8a6WLx9DbC8xedTyD6oAc98vuVF8=; b=678a2dzhl6ZrLZ109xlM+uU/rNJFjdBXibU33j6uDJeY+nCytEi0kZ+Zxz/Vl7itPvHBiq gd9LmJsaBT8lBfhCPATJPNg9u53bLFFSh5c1zk6ncvPZs/PWsmLYn92wSzqQ+wIvQDGNvI ItETgot7bNZ0D0dUdiF8nOpp1d4UreI= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1712902846; a=rsa-sha256; cv=none; b=5nvhNsEr1x3TajTZdfA+meMmGwwEWFOph0QDm4R79y6ooRT2ispnpo0BCwewkLQP01/jUh lAnC85RfLqJ5jCN5P2FnPVj1sXQKi/MRnHo4azvk96IyCKCrdXzSvtLbSP+QQaSiRokcGH Q04MJEuFoFfsgKRCjb+RUIMb/Nc2EbU= ARC-Authentication-Results: i=1; imf11.hostedemail.com; dkim=none; spf=pass (imf11.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.32 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com Received: from mail.maildlp.com (unknown [172.19.88.163]) by szxga06-in.huawei.com (SkyGuard) with ESMTP id 4VG5y96MQFz1wrMk; Fri, 12 Apr 2024 14:19:45 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 23B0318001A; Fri, 12 Apr 2024 14:20:42 +0800 (CST) Received: from localhost.localdomain.localdomain (10.175.113.25) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Fri, 12 Apr 2024 14:20:41 +0800 From: Kefeng Wang To: Andrew Morton CC: "Matthew Wilcox (Oracle)" , , , Kefeng Wang Subject: [PATCH v3 1/2] mm: move mm counter updating out of set_pte_range() Date: Fri, 12 Apr 2024 14:47:50 +0800 Message-ID: <20240412064751.119015-2-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20240412064751.119015-1-wangkefeng.wang@huawei.com> References: <20240412064751.119015-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.113.25] X-ClientProxiedBy: dggems701-chm.china.huawei.com (10.3.19.178) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspamd-Queue-Id: 7A9C240004 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: 3ucgugaiw9o61psnxb8iw8kssaiebrj9 X-HE-Tag: 1712902845-17903 X-HE-Meta: U2FsdGVkX1/Isq7/zMKIkrk2OruUrgJvgy6jsngrVKxPFIRPrPWJFEuwksDjCPgZj/Zg+jDXtGjisbY0sk1CNd83d5xqI6RypZBLMfRgZkb+u9UGMBxsW5WR+0vxy+hYA6Xwc4BmJ7hvds5iIaCZT3X736h8XHRU9KuDY2tXvGDy9HI2tg0dsuOzSyaLzSJEJ2shxZsZAzWaO9jnXfWEZ3wiRl9g2fAfg9krJc87iWgIa52yFW67uD1kGSlOVBHb9F+2bsLToNI6RVoKJTUuxadwZcwtAvUrLXRc78/vN7Yj4jrb419CWHSV0hvKB6NJIZtl+yyKFEXQg6p900cKPpI3jX9AtL0H2uNU7CVc0a21zNzosLZjZgs+hVCKThZfJQaJBw2rr26eKtK6HVKBwTUg2zebH6/BOwiHl+MQH7y2oo40vP5xXJftoxBu+3hMEWp4L/bKAap4ug+izMYkJxS6sCWc46UCeyEL31tKaaeaxZkV4LQ1yZ5Z/rPSOY6Dvr5ujR1GTaGxw+cHx05vzQceaP9/cEMCEWsmpruc7EeNyTUzlBkPy+IAV9rGIdB0nxkOVyY+k2IDLqFG571WQODnUH6hxt8cMk76O0o5Jv9TYm0dgi2/iqmxvtKlIrE6ztSopTVKR/te1IOwxs0U+gihgNYA7tM9YuTX5abhJEeStRqwScMOuZctuT0TgLsc4kbJcpaOGLKYJ1ELS4wsA4pV3ioHKeWPp5+kVBdukHaDM9XtBtaNo7tC4IzMlailYoZSeVvaUW/phYNmbZphH3uT/NgB3IqkWAsygYFAuJOO5N0fLfHT5bPJmIGoGdA2GslVgjCC6SzngZfW9LbFBxV0lTBeJOdsfKHg+htjOuX/X+Qa7hGQWGSRAknPV6QHgkaziT6vEuqr/i4k5BQxlKXOkr5vJh3RBB2MrfZbE6ZCqyMegXjdFWHL1ukd0d1hZ+PHOT0lJSGKLFeHg88 yjwlG4iV X9SQ5P0rIVskvSPo4/Jxj3LvRbLGopWAdZXTtO7GuNp4EBqzeXMcLrTmn1mQMvbOUrBYMx9GbL+8q8gf0gvjZ5gCdAhQGXi1KgFdNy2rb690NkJS4y9zMEG9nJjqXabE6THrdI3i6MZ7F4S4xmcnKBO/yqFAVqDLBQfvjQSiBWWcHlf8vGZ8Ptf5LgQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In order to support batch mm counter updating in filemap_map_pages(), move mm counter updating out of set_pte_range(), the folios are file from filemap, and distinguish folios by vmf->flags and vma->vm_flags from another caller finish_fault(). Signed-off-by: Kefeng Wang --- mm/filemap.c | 4 ++++ mm/memory.c | 8 +++++--- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/mm/filemap.c b/mm/filemap.c index 92e2d43e4c9d..04b813f0146c 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3540,6 +3540,8 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, skip: if (count) { set_pte_range(vmf, folio, page, count, addr); + add_mm_counter(vmf->vma->vm_mm, mm_counter_file(folio), + count); folio_ref_add(folio, count); if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; @@ -3554,6 +3556,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, if (count) { set_pte_range(vmf, folio, page, count, addr); + add_mm_counter(vmf->vma->vm_mm, mm_counter_file(folio), count); folio_ref_add(folio, count); if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; @@ -3590,6 +3593,7 @@ static vm_fault_t filemap_map_order0_folio(struct vm_fault *vmf, ret = VM_FAULT_NOPAGE; set_pte_range(vmf, folio, page, 1, addr); + add_mm_counter(vmf->vma->vm_mm, mm_counter_file(folio), 1); folio_ref_inc(folio); return ret; diff --git a/mm/memory.c b/mm/memory.c index 78422d1c7381..fdfe965f32e4 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4685,12 +4685,10 @@ void set_pte_range(struct vm_fault *vmf, struct folio *folio, entry = pte_mkuffd_wp(entry); /* copy-on-write page */ if (write && !(vma->vm_flags & VM_SHARED)) { - add_mm_counter(vma->vm_mm, MM_ANONPAGES, nr); VM_BUG_ON_FOLIO(nr != 1, folio); folio_add_new_anon_rmap(folio, vma, addr); folio_add_lru_vma(folio, vma); } else { - add_mm_counter(vma->vm_mm, mm_counter_file(folio), nr); folio_add_file_rmap_ptes(folio, page, nr, vma); } set_ptes(vma->vm_mm, addr, vmf->pte, entry, nr); @@ -4727,9 +4725,11 @@ vm_fault_t finish_fault(struct vm_fault *vmf) struct vm_area_struct *vma = vmf->vma; struct page *page; vm_fault_t ret; + bool is_cow = (vmf->flags & FAULT_FLAG_WRITE) && + !(vma->vm_flags & VM_SHARED); /* Did we COW the page? */ - if ((vmf->flags & FAULT_FLAG_WRITE) && !(vma->vm_flags & VM_SHARED)) + if (is_cow) page = vmf->cow_page; else page = vmf->page; @@ -4765,8 +4765,10 @@ vm_fault_t finish_fault(struct vm_fault *vmf) /* Re-check under ptl */ if (likely(!vmf_pte_changed(vmf))) { struct folio *folio = page_folio(page); + int type = is_cow ? MM_ANONPAGES : mm_counter_file(folio); set_pte_range(vmf, folio, page, 1, vmf->address); + add_mm_counter(vma->vm_mm, type, 1); ret = 0; } else { update_mmu_tlb(vma, vmf->address, vmf->pte);