From patchwork Fri Apr 14 13:02:54 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13211460 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 10CC6C77B76 for ; Fri, 14 Apr 2023 13:03:42 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BF60A900004; Fri, 14 Apr 2023 09:03:31 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B2C21900002; Fri, 14 Apr 2023 09:03:31 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9A5B6900004; Fri, 14 Apr 2023 09:03:31 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 83052900002 for ; Fri, 14 Apr 2023 09:03:31 -0400 (EDT) Received: from smtpin11.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id EBD731A011C for ; Fri, 14 Apr 2023 13:03:30 +0000 (UTC) X-FDA: 80680012980.11.B035BC8 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf18.hostedemail.com (Postfix) with ESMTP id 32BC21C003B for ; Fri, 14 Apr 2023 13:03:29 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1681477409; a=rsa-sha256; cv=none; b=vQcSeH6Ovt3JvVXCD8eNfax80OjAQNChBhaV8pHb9KpASO1xruyqCxMnxdFFaNbMu+VbU/ 5LegGnJ7zvsczVFKqcnz+XWycJn5gOa+3iycMHmPVGIG8S5etk3SEO5LSp/+POsgJhJCOs rHFl5VdG1sttk8X+qdRK6uZAnfvKVIE= ARC-Authentication-Results: i=1; imf18.hostedemail.com; dkim=none; spf=pass (imf18.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1681477409; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=VzYKZPUN+KxsXR3DLB7tLFMomsKlrH2nSlt7jFeq568=; b=kv2VwYYjGWK5zqBWwnA/UgoTQMGQQ19aO4RrgfuxdaQOsWzN54XqAp+ICGFfTjM8/MyRFF 8EIHfz48eYnjegPxns2jO0EdWCtxCFMdyeRREJtOPguEnO1AWzVhnWt6VSWgij4i21V/2W gVKN4LvsshCHdakVVU74RU8emUX6vcs= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 47F6A1762; Fri, 14 Apr 2023 06:04:13 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C33FF3F6C4; Fri, 14 Apr 2023 06:03:27 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , "Matthew Wilcox (Oracle)" , Yu Zhao , "Yin, Fengwei" Cc: Ryan Roberts , linux-mm@kvack.org, linux-arm-kernel@lists.infradead.org Subject: [RFC v2 PATCH 08/17] mm: Implement folio_move_anon_rmap_range() Date: Fri, 14 Apr 2023 14:02:54 +0100 Message-Id: <20230414130303.2345383-9-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230414130303.2345383-1-ryan.roberts@arm.com> References: <20230414130303.2345383-1-ryan.roberts@arm.com> MIME-Version: 1.0 X-Rspam-User: X-Rspamd-Queue-Id: 32BC21C003B X-Rspamd-Server: rspam01 X-Stat-Signature: wwqohtkacsdf8ab1mop4ngekfbxhmkw7 X-HE-Tag: 1681477409-598271 X-HE-Meta: U2FsdGVkX1/AiWvmBhUWb0fQav2HL9ITHBf8dKlDnzi05OHPgK5SOyj3pS6MMrIPYj6aR6u06Y3Gnck2lGO7m5xFq4e6Z2rkR9AfeL+unpxJPYWY7JZ/Td6QFHFu56r0L/2Ii12xu3HOODLygEoO/HDzD9FAQc7cfr1VNM9PDuszttJJfBjI7fkKB/SedmoXIR9rgoxJrRPc9SgWEMEFuUgrahp4V+VaAeJWIGq0bTybc6Up1DDmr89EpDKCLGCAAJt/7sax88ggfB4v3KUuakesdaL0S2Kw1L/thNPUDHZPNRvfeFCqGaBw22zd1chVY8W2xUcvzC7033Go4BbjU1QTKqYfL5LWaC4WnpFWjbC5wpcILD5CwLSTYDiB3+lPGF7hx9ZiW17o3k/1yd1C0o1UhntIclCin5QosoKxpz/VDS1ssYjdg6af7AnGMoT+iMIyZGesVYFq0PWzhn2CKuLJOWuzbLwJOI8hGOmRzapejShzQhHlJrz5C50VuE69he8tOe8wy6NPCOybr98FBhFr451mty9n7+w3yJmtF2teyFTYM7PrFswyOxlVKSUrt76gLkgyu2LzBwJ/nAUG4X791mPfpdcDJ70Hb+JLucGZFtW6/cEQr3i0gBQ5F5C4tsJ6DGWpi2S5DHim3Kr7pS4+ExuZGwBsC7usVdVCDeOe17QnP8xSclRNtvCOPrwC++fNv39Yzo2L3yuh3CSzmiL/WvS/JLP2Ylrj3OAm1ZAEhuISYHs2twqSCorsfHageJPXq3dxWcpP5zIAUH8VY14CaP+DNIMQjhO1jWldiUqOg6Z/780aUFdeBbQXfuQPgei3DeMjIdM/SGPx9/PG0aGlhcubDZYM7oTYrxns6jjLpJruzIs2lYvKLaoqzFr+V5jKF2+cs+1/FpSzVkH81duqWpF/Q7Tjwi6i+yD70nuNuc+aE5tORV/PrHq9KGLozNgo278xKe/varDFbhJ ElvNqDJ+ NPjwRBwwYKy33YOmc6xRQjQFeIG/tGTfPdrbnyYefyU0DL1PiisOp0RsUzAgXemB9OwTVOBT8mkk1twJwNuqotr3sNofW8+MgYkr7R5vgR9rs8mFXTVpqyhRIMT685Axn/hsB6oQ/uQ4bdECwHPv6QQnOHAcy9SxCE71znNATW3gtxAOt2s6hcuMjLla+DYu2g9Gi4cVYV5Dj+vBeXddupGHWRNLa87IUcXRZ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Similar to page_move_anon_rmap() except it can batch-move a range of pages within a folio for increased efficiency. Will be used to enable reusing multiple pages from a large anonymous folio in one go. Signed-off-by: Ryan Roberts --- include/linux/rmap.h | 2 ++ mm/rmap.c | 40 ++++++++++++++++++++++++++++++---------- 2 files changed, 32 insertions(+), 10 deletions(-) -- 2.25.1 diff --git a/include/linux/rmap.h b/include/linux/rmap.h index 5c707f53d7b5..8cb0ba48d58f 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -190,6 +190,8 @@ typedef int __bitwise rmap_t; * rmap interfaces called when adding or removing pte of page */ void page_move_anon_rmap(struct page *, struct vm_area_struct *); +void folio_move_anon_rmap_range(struct folio *folio, struct page *page, + int nr, struct vm_area_struct *vma); void page_add_anon_rmap(struct page *, struct vm_area_struct *, unsigned long address, rmap_t flags); void page_add_new_anon_rmap(struct page *, struct vm_area_struct *, diff --git a/mm/rmap.c b/mm/rmap.c index 5148a484f915..1cd8fb0b929f 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1103,19 +1103,22 @@ int folio_total_mapcount(struct folio *folio) } /** - * page_move_anon_rmap - move a page to our anon_vma - * @page: the page to move to our anon_vma - * @vma: the vma the page belongs to + * folio_move_anon_rmap_range - batch-move a range of pages within a folio to + * our anon_vma; a more efficient version of page_move_anon_rmap(). + * @folio: folio that owns the range of pages + * @page: the first page to move to our anon_vma + * @nr: number of pages to move to our anon_vma + * @vma: the vma the page belongs to * - * When a page belongs exclusively to one process after a COW event, - * that page can be moved into the anon_vma that belongs to just that - * process, so the rmap code will not search the parent or sibling - * processes. + * When a range of pages belongs exclusively to one process after a COW event, + * those pages can be moved into the anon_vma that belongs to just that process, + * so the rmap code will not search the parent or sibling processes. */ -void page_move_anon_rmap(struct page *page, struct vm_area_struct *vma) +void folio_move_anon_rmap_range(struct folio *folio, struct page *page, + int nr, struct vm_area_struct *vma) { void *anon_vma = vma->anon_vma; - struct folio *folio = page_folio(page); + int i; VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_VMA(!anon_vma, vma); @@ -1127,7 +1130,24 @@ void page_move_anon_rmap(struct page *page, struct vm_area_struct *vma) * folio_test_anon()) will not see one without the other. */ WRITE_ONCE(folio->mapping, anon_vma); - SetPageAnonExclusive(page); + + for (i = 0; i < nr; i++) + SetPageAnonExclusive(page++); +} + +/** + * page_move_anon_rmap - move a page to our anon_vma + * @page: the page to move to our anon_vma + * @vma: the vma the page belongs to + * + * When a page belongs exclusively to one process after a COW event, + * that page can be moved into the anon_vma that belongs to just that + * process, so the rmap code will not search the parent or sibling + * processes. + */ +void page_move_anon_rmap(struct page *page, struct vm_area_struct *vma) +{ + folio_move_anon_rmap_range(page_folio(page), page, 1, vma); } /**