From patchwork Fri Feb 3 13:16:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yin Fengwei X-Patchwork-Id: 13127501 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5E34BC61DA4 for ; Fri, 3 Feb 2023 13:14:50 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9E47B6B0074; Fri, 3 Feb 2023 08:14:49 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 994776B0075; Fri, 3 Feb 2023 08:14:49 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 85CAA6B0078; Fri, 3 Feb 2023 08:14:49 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 752AE6B0074 for ; Fri, 3 Feb 2023 08:14:49 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 4780CAB7EB for ; Fri, 3 Feb 2023 13:14:49 +0000 (UTC) X-FDA: 80426025498.08.5E9F56B Received: from mga11.intel.com (mga11.intel.com [192.55.52.93]) by imf23.hostedemail.com (Postfix) with ESMTP id 6740E14001F for ; Fri, 3 Feb 2023 13:14:46 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Wrlzx7nk; spf=pass (imf23.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675430087; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references:dkim-signature; bh=gX/J5NuLo9wem/Xgh4JkX1ApMa5uGo0Vb2sBO6rRMq8=; b=NjtHpf0tHw7UGmBan+p2mzbEnO9JnYUwMrveftlZO95M3w/M5ouVsiim3gkk0PYPy65uNj 0kRjz6gbXzK6D+bv+awPksgV/+CDaJqg27dn4JUZ/uQZEmfAzAmn8ba4hRJ/tQ+tlCWx7Y OR/JOcoHU+t18gVagvKKp3RH/bF4olY= ARC-Authentication-Results: i=1; imf23.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=Wrlzx7nk; spf=pass (imf23.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.93 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1675430087; a=rsa-sha256; cv=none; b=ZvaXnIcDGb2NaU/Z/CgXhRL6VPUWAcLasB5mNfhpp78HGlcce4wvH+2FpKUoAMIJxf6jPf q9Oittg2E2hndB2tLk0EcKvq2U2Pc2M4JLOpq7ctjPaZCtURaiIKpju9tVwLzlUJPfO21h QVXqNmJ2NN5NYrlmIMA5YMm7b/pRves= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1675430086; x=1706966086; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=gMJzuSPn6L+8O346eopwVKZSAkxvg09xJIs0x4SRU0I=; b=Wrlzx7nk6Ll801DxouMJY1SowbPBW1FqanlVnsJdCLxntq0Ejng22WQL EwfpNIDHABfK05396eb7/evWYVf24doqZw7kzd8yy+SF5X/7MLkeerCPG YxEdccN7jlr9XiK1n17e0OGc+6lGpYqUKJpV9jRx/w/A4czam75pyv0UB uLOENAMdyabsdTNiPwrcRLr02Cz/c3u9grb1iabOdvZvU9g6RsRZC+1kF Ns3vkYyohg1MtrgLwUU9Aw/xu/rvw+VsNycSjmJuMpmLqscO7L8vqqE35 wteXPA02sHEnDhJKmFPfJnKfLPsg0UZ9V4bg9I0+Da78hkAsi+qwat7uB w==; X-IronPort-AV: E=McAfee;i="6500,9779,10609"; a="326441879" X-IronPort-AV: E=Sophos;i="5.97,270,1669104000"; d="scan'208";a="326441879" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by fmsmga102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 03 Feb 2023 05:14:45 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10609"; a="667641940" X-IronPort-AV: E=Sophos;i="5.97,270,1669104000"; d="scan'208";a="667641940" Received: from fyin-dev.sh.intel.com ([10.239.159.32]) by fmsmga007.fm.intel.com with ESMTP; 03 Feb 2023 05:14:42 -0800 From: Yin Fengwei To: willy@infradead.org, david@redhat.com, linux-mm@kvack.org Cc: dave.hansen@intel.com, tim.c.chen@intel.com, ying.huang@intel.com, fengwei.yin@intel.com Subject: [RFC PATCH v3 0/4] folio based filemap_map_pages() Date: Fri, 3 Feb 2023 21:16:32 +0800 Message-Id: <20230203131636.1648662-1-fengwei.yin@intel.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 X-Stat-Signature: n5io86q3ozguah4ocrn9jcpqjkpsk8zg X-Rspam-User: X-Rspamd-Queue-Id: 6740E14001F X-Rspamd-Server: rspam06 X-HE-Tag: 1675430086-199525 X-HE-Meta: U2FsdGVkX1+2ezXWtZ/nMtK4oenOqOBp5UZfHsLHAeETKN64zi/9T/6qE/x1+ThgKy7X84+qMgP7HLxqvwnmBMuUQ8ObIcZnWQZCrhhlrFXcA4+L9CbPYW+XfUdbIVP6XiAfCVnJ9H3kt5w6zGaBGGeGZGJEaumr+jTaoaY8Pq/jGG42oKlwjWWQNj0KWCq6vnNJYfllh5stm4eys5QlpiLqJSBIVpAPrlBFQiCNtR+nLTg49wCOkBk2AV7YS4VHnp6JpBG7+NfJMPC49k1Ad4igGzB3f5nC2ESLgBN2FjqUctFh5UbKTPCmrFQzbxDCd8LjFz+2vc8biPth7XNko9KXbz2Zwx/I7pLcVWuOJqw/3ORfQdiaAulmPc0SmWG74grRGXKzQeeGupliDVTARNmCyMHl9wI5KJHnA8ZF8ursBRqFt67o/yS6MMZBowj2IiLWf0LubdY9BlOnyQQqrRbSs1vMyS2H1+ZlqwNIyHP/1/IKz0O/nSt+jCNNN3RtSGGHGy3opU0mWCo40Cxg6MhyNscLNAmN1XEE7U2l0AuG7XsPw5pkBP44p/BwFzOJOvT7wTg7GZ4KPeLYqBh74RqkumW/LH8J29/V0H2A6A8xSsXDWnvKNMbRBypz+XmoI/XhADFIO+A7K2TNAvlVWA3Y+DoP63s09W6DCJK/Yt53voqkbE6qAxSNogeHMilgbcXQr6gF989pibnnHpKeVbCinHhd+Pl4stlBszsl5xkhdT2Es1vMU04apvElJEK8K3jElf0MPaVq5XPFgdoMXm2jR2y8nJEoivWf8VbFB0C6+PUFQVSXoRpDGARP/5f/55/31QpRaCsGYTI1cIscDuYAxWO13+7G93E86rx7Ilc2emnJVm0PFslxLPZfairgAKkck1OtSeEUnRiCNV/Da31dvkB3LsoszGGMLACKeVcMp8VYXTIamygDlglFfrUEeWak1Hl9UEX3bZ67VnN 83WxSay7 vwiYEmGPPMsqXHCpaTp2e44sQnzaTFsBVXfsNRQjI/UPXbc0yOdAP7XXU6xKGPWfrESGG8gT4LGawUv8bi6aXmACHSGMtIdCJi5WaFAtt9cO0XCqeJ4IL0+174e6RS7IJrV+BNt9UWyu6lwfUsplzINLqTJRCj4SKpiqqrrt8XMfdx4n18dy18C4Qao8FoOWXE29l X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Current filemap_map_pages() uses page granularity even when underneath folio is large folio. Making it use folio based granularity allows batched refcount, rmap and mm counter update. Which brings performance gain. This sereis tries to bring batched refcount, rmap and mm counter for filemap_map_pages(). Testing with a micro benchmark like will-it-scale.pagefault on a 48C/96T IceLake tbox showed: - batched rmap brings around 15% performance gain - batched refcount brings around 2% performance gain Patch 1 update filemap_map_pages() to do map based on folio granularity and batched refcount update Patch 3,4,5 enable batched rmap and mm counter Change from v2: - Drop patch 1 because it misses ->page_mkwrite() as Kirill pointed out. Patch 2: - Change page_add_file_rmap_range() to folio_add_file_rmap_range() as Matthew suggested - Remove "sub" as Matthew suggested Patch 3: - Only handle !cow case in do_set_pte_range() as David pointed out - Add a parameter of pte and avoid change vmf->pte in do_set_pte_range() - Drop do_set_pte_entry() Patch 4: - adopt the suggestion from Matthew to avoid subtracted/add vmf->pte. filemap_map_folio_range() doesn't update vmf->pte now. Make it easy to fit the filemap_map_pages() and no possible point to wrong page table. Change from v1: - Update the struct page * parameter of *_range() to index in the folio as Matthew suggested - Fix indentations problem as Matthew pointed out - Add back the function comment as Matthew pointed out - Use nr_pages than len as Matthew pointed out - Add do_set_pte_range() as Matthew suggested - Add function comment as Ying suggested - Add performance test result to patch 1/2/5 commit message Patch 1: - Adapt commit message as Matthew suggested - Add Reviweed-by from Matthew Patch 3: - Restore general logic of page_add_file_rmap_range() to make patch review easier as Matthew suggested Patch 5: - Add perf data collected to understand the reason of performance gain Yin Fengwei (4): filemap: add function filemap_map_folio_range() rmap: add folio_add_file_rmap_range() mm: add do_set_pte_range() filemap: batched update mm counter,rmap when map file folio include/linux/mm.h | 3 ++ include/linux/rmap.h | 2 + mm/filemap.c | 101 ++++++++++++++++++++++++++++--------------- mm/memory.c | 59 ++++++++++++++++--------- mm/rmap.c | 66 ++++++++++++++++++++++------ 5 files changed, 163 insertions(+), 68 deletions(-)