From patchwork Mon Apr 29 07:24:16 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Kefeng Wang X-Patchwork-Id: 13646409 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 43C88C19F53 for ; Mon, 29 Apr 2024 07:24:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 80F756B0087; Mon, 29 Apr 2024 03:24:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 75E3C6B0092; Mon, 29 Apr 2024 03:24:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4C6646B008C; Mon, 29 Apr 2024 03:24:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 26CEF6B0087 for ; Mon, 29 Apr 2024 03:24:36 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id BDCA2418E6 for ; Mon, 29 Apr 2024 07:24:35 +0000 (UTC) X-FDA: 82061731710.21.07EA869 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf18.hostedemail.com (Postfix) with ESMTP id 522FA1C000F for ; Mon, 29 Apr 2024 07:24:32 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf18.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1714375474; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ziwqm9Ta6Oac20CoDTsmUctDiFdn1EuMPpT2nbUkDZg=; b=3WznfWG2CYZTpkFcag4tPBUtqOb/KA9bg8ItLGi/E1DRZdRZ0hiK3UQ/wRuqe1qZaCO6dn sA/rt/dJu6K+OUfIjLl2NllEuVl5oAvJgaA+0A8sakh1PqI4yAXxDQpeC2UKuWlYUssCQG mTtuKbGX8fQcYplV5eboQ79kcC287Q4= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1714375474; a=rsa-sha256; cv=none; b=X89fPLCk8nzo4PLwOgjyQqKUrduaiFLdO8cf7jRG8dxs19MoR/reE/mah0vE1/6fcxsoGL gCTrqyQwKACYY4JuJ+LO+acRfoe2mM6JcU6ranCJEp6HohlOfDq2GxCAA/d8Yz8xggVO7n U4IVIfuWWfmfM8TMJ3SJh45tFJSugEM= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf18.hostedemail.com: domain of wangkefeng.wang@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=wangkefeng.wang@huawei.com Received: from mail.maildlp.com (unknown [172.19.163.252]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4VSZWP3rhQzvQsr; Mon, 29 Apr 2024 15:21:21 +0800 (CST) Received: from dggpemm100001.china.huawei.com (unknown [7.185.36.93]) by mail.maildlp.com (Postfix) with ESMTPS id 93C4918007D; Mon, 29 Apr 2024 15:24:29 +0800 (CST) Received: from localhost.localdomain (10.175.112.125) by dggpemm100001.china.huawei.com (7.185.36.93) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.1.2507.35; Mon, 29 Apr 2024 15:24:29 +0800 From: Kefeng Wang To: Andrew Morton CC: "Matthew Wilcox (Oracle)" , , , Kefeng Wang Subject: [PATCH rfc 3/4] mm: filemap: move __lruvec_stat_mod_folio() out of filemap_set_pte_range() Date: Mon, 29 Apr 2024 15:24:16 +0800 Message-ID: <20240429072417.2146732-4-wangkefeng.wang@huawei.com> X-Mailer: git-send-email 2.27.0 In-Reply-To: <20240429072417.2146732-1-wangkefeng.wang@huawei.com> References: <20240429072417.2146732-1-wangkefeng.wang@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.112.125] X-ClientProxiedBy: dggems706-chm.china.huawei.com (10.3.19.183) To dggpemm100001.china.huawei.com (7.185.36.93) X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 522FA1C000F X-Stat-Signature: hie7ymoptkso1tepw1fsf3kyk87xiaxf X-Rspam-User: X-HE-Tag: 1714375472-399581 X-HE-Meta: U2FsdGVkX1+CzAWw89AZZDWuaFwXxb+Pmfzlc2rTGc0HEtItRd8XGz/fR7FyrjTB38KwItYT61kziJQNwZBSM9RwCucaUm3TGNK6ch/9cqL1s+MsZ3lDEA3JuS2pXTgUURLazIzCSr/cRpWfbHaZYXSkv54WBtGcG8SiSL+4YehFJ+IiXPBXrgDCkeanZKt9fQFdbpt9I86eNWvxS10G5PTotRJapEWBs3njq/s3k346WjSOAe2QDVwHRVNnXgoDtCAzO3XkIxM05exVO85zYDM7kDAAaFmdOTtj9hL7hugG4c9jZS1Z3tUafrV0mo/+m8HQXu+2A17ElnFD2Wo8OUSH8ZMJso2ph/ORWRd5HOZjWHUdkBFStqLMtsmfbhB4TZbBhoBL0bPqrynhyPt76iMWf2ZmX/sTDyqotARhQhFDw6WL29o7wdUSbgbnlLbABQUlJ9r72l1sgh8+2u1oJjaULihDemf8VU61E2nWJaQnRoidXqgISzSZKBW/t46uBXVviB3FFsohxDBs4QfbnzYC843vDnRa1DtMgRd+1r3iPKEAie3Vrqi6ZxbzdX9GQHymE502tXV4uvN9QC29/f4TyIEgrH7Hopk7XI0qsW25GqVJxb6ccmdTWZcr9V2GPFrLhJGinKr8sy7y1bECv56wVB7JQ0K6Ym5fSEr9RTV47e4/7bnGwLYM923nfQSmL7FFfIJCKEkXamPe+HjoyoFL8mvSeXu7nvq/u3EIXZR9sJBsZexWm5uKZQ4WnosMIfubhtM1BLS8hK4iy+eBTN88rOXhgwfDWqPXCUqG+RSOfI0OQB+B2bPkp9YaoK8tbxxVJ53deY1pSZFTl1xgrzCaFGDnE1Xr7icZZJ0zulcTJ35iALnmQXc5GKGKwhLk9CpIrN+7n2m5H6DnXYSFlSWA+1AX7XzPufIJlep0pbkktT8OLWe/DLC2lE8AKmUJ5gZLIqBDQM2MwF+ABvS LDUCnKuY Yh4INJc0OyPYJjf+fi6DDxyupR1cat2imhcouTkpOiYNsmPD4XhZqM5HvJuLZRdNWOLFFN08+O8Xv30M6qr7IkC/6/lGZ6Drx1qoJEHohU23sLYshpiOYKYolcwUEVzgvb/ej3G8dFD2juq+fwybbaeYjf5TbD+zy62TUd3kzgmriWs8hfT9KH0jDdKIeHaHdSc3E X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Adding __folio_add_file_rmap_ptes() which don't update lruvec stat, it is used in filemap_set_pte_range(), with it, lruvec stat updating is moved into the caller, no functional changes. Signed-off-by: Kefeng Wang --- include/linux/rmap.h | 2 ++ mm/filemap.c | 27 ++++++++++++++++++--------- mm/rmap.c | 16 ++++++++++++++++ 3 files changed, 36 insertions(+), 9 deletions(-) diff --git a/include/linux/rmap.h b/include/linux/rmap.h index 7229b9baf20d..43014ddd06f9 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -242,6 +242,8 @@ void folio_add_anon_rmap_pmd(struct folio *, struct page *, struct vm_area_struct *, unsigned long address, rmap_t flags); void folio_add_new_anon_rmap(struct folio *, struct vm_area_struct *, unsigned long address); +int __folio_add_file_rmap_ptes(struct folio *, struct page *, int nr_pages, + struct vm_area_struct *); void folio_add_file_rmap_ptes(struct folio *, struct page *, int nr_pages, struct vm_area_struct *); #define folio_add_file_rmap_pte(folio, page, vma) \ diff --git a/mm/filemap.c b/mm/filemap.c index 7019692daddd..3966b6616d02 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -3501,14 +3501,15 @@ static struct folio *next_uptodate_folio(struct xa_state *xas, static void filemap_set_pte_range(struct vm_fault *vmf, struct folio *folio, struct page *page, unsigned int nr, unsigned long addr, - unsigned long *rss) + unsigned long *rss, int *nr_mapped) { struct vm_area_struct *vma = vmf->vma; pte_t entry; entry = prepare_range_pte_entry(vmf, false, folio, page, nr, addr); - folio_add_file_rmap_ptes(folio, page, nr, vma); + *nr_mapped += __folio_add_file_rmap_ptes(folio, page, nr, vma); + set_ptes(vma->vm_mm, addr, vmf->pte, entry, nr); /* no need to invalidate: a not-present page won't be cached */ @@ -3525,7 +3526,8 @@ static void filemap_set_pte_range(struct vm_fault *vmf, struct folio *folio, static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, struct folio *folio, unsigned long start, unsigned long addr, unsigned int nr_pages, - unsigned long *rss, unsigned int *mmap_miss) + unsigned long *rss, int *nr_mapped, + unsigned int *mmap_miss) { vm_fault_t ret = 0; struct page *page = folio_page(folio, start); @@ -3558,7 +3560,8 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, continue; skip: if (count) { - filemap_set_pte_range(vmf, folio, page, count, addr, rss); + filemap_set_pte_range(vmf, folio, page, count, addr, + rss, nr_mapped); if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; } @@ -3571,7 +3574,8 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, } while (--nr_pages > 0); if (count) { - filemap_set_pte_range(vmf, folio, page, count, addr, rss); + filemap_set_pte_range(vmf, folio, page, count, addr, rss, + nr_mapped); if (in_range(vmf->address, addr, count * PAGE_SIZE)) ret = VM_FAULT_NOPAGE; } @@ -3583,7 +3587,7 @@ static vm_fault_t filemap_map_folio_range(struct vm_fault *vmf, static vm_fault_t filemap_map_order0_folio(struct vm_fault *vmf, struct folio *folio, unsigned long addr, - unsigned long *rss, unsigned int *mmap_miss) + unsigned long *rss, int *nr_mapped, unsigned int *mmap_miss) { vm_fault_t ret = 0; struct page *page = &folio->page; @@ -3606,7 +3610,7 @@ static vm_fault_t filemap_map_order0_folio(struct vm_fault *vmf, if (vmf->address == addr) ret = VM_FAULT_NOPAGE; - filemap_set_pte_range(vmf, folio, page, 1, addr, rss); + filemap_set_pte_range(vmf, folio, page, 1, addr, rss, nr_mapped); return ret; } @@ -3646,6 +3650,7 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, folio_type = mm_counter_file(folio); do { unsigned long end; + int nr_mapped = 0; addr += (xas.xa_index - last_pgoff) << PAGE_SHIFT; vmf->pte += xas.xa_index - last_pgoff; @@ -3655,11 +3660,15 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, if (!folio_test_large(folio)) ret |= filemap_map_order0_folio(vmf, - folio, addr, &rss, &mmap_miss); + folio, addr, &rss, &nr_mapped, + &mmap_miss); else ret |= filemap_map_folio_range(vmf, folio, xas.xa_index - folio->index, addr, - nr_pages, &rss, &mmap_miss); + nr_pages, &rss, &nr_mapped, + &mmap_miss); + + __lruvec_stat_mod_folio(folio, NR_FILE_MAPPED, nr_mapped); folio_unlock(folio); folio_put(folio); diff --git a/mm/rmap.c b/mm/rmap.c index 2608c40dffad..55face4024f2 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1452,6 +1452,22 @@ static __always_inline void __folio_add_file_rmap(struct folio *folio, mlock_vma_folio(folio, vma); } +int __folio_add_file_rmap_ptes(struct folio *folio, struct page *page, + int nr_pages, struct vm_area_struct *vma) +{ + int nr, nr_pmdmapped = 0; + + VM_WARN_ON_FOLIO(folio_test_anon(folio), folio); + + nr = __folio_add_rmap(folio, page, nr_pages, RMAP_LEVEL_PTE, + &nr_pmdmapped); + + /* See comments in folio_add_anon_rmap_*() */ + if (!folio_test_large(folio)) + mlock_vma_folio(folio, vma); + + return nr; +} /** * folio_add_file_rmap_ptes - add PTE mappings to a page range of a folio * @folio: The folio to add the mappings to