From patchwork Fri Jul 14 16:04:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13313824 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5FBCEEB64DC for ; Fri, 14 Jul 2023 16:05:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A235A900003; Fri, 14 Jul 2023 12:05:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D3E88E000A; Fri, 14 Jul 2023 12:05:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 87534900003; Fri, 14 Jul 2023 12:05:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 745238E000A for ; Fri, 14 Jul 2023 12:05:40 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 36F84A0332 for ; Fri, 14 Jul 2023 16:05:40 +0000 (UTC) X-FDA: 81010692840.13.3DBE638 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf09.hostedemail.com (Postfix) with ESMTP id C3CFC14006A for ; Fri, 14 Jul 2023 16:04:24 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689350665; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:references; bh=wHPAsnZCWorHCFsepPn9uYKmhk1aEb4Zb8u9M9tawwE=; b=WrViM6lLe/S8zg3nSwZfdyKwgavJ7hMrEVWmCs66GZC/v9PoX/uA0G5Qnv8zRYXLCNTYEr AbnAOB8MUzpZAw8Eyh9NvVyt+/feipheTofBNVZ2UXhCdPV+Fp7sw6qCkTHl3L4rcA+DYI +mRdHcLcFjQFuq9wcKG8AJEW4ds59Tk= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1689350665; a=rsa-sha256; cv=none; b=UGIW6S11BJzqNRPNYEoyEEWB1WKFjUVLhQkwIwlWKUqDgztx3S9xOpVFe1MFMJY2BJ8zfe aha4vXprrsQvT4kqhDe5EpGUW2Q/jCwynwfQbxg8cyPWmouCbsBycpED2so7OS5s/K0iNW A7DDdjh95IvSFZEFEh0YrW35E/yDL1o= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=none; spf=pass (imf09.hostedemail.com: domain of ryan.roberts@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=ryan.roberts@arm.com; dmarc=pass (policy=none) header.from=arm.com Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 54ACA1576; Fri, 14 Jul 2023 09:05:05 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 06F163F740; Fri, 14 Jul 2023 09:04:20 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Yu Zhao , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , "Huang, Ying" , Zi Yan , Luis Chamberlain Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 0/4] variable-order, large folios for anonymous memory Date: Fri, 14 Jul 2023 17:04:03 +0100 Message-Id: <20230714160407.4142030-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-Rspamd-Queue-Id: C3CFC14006A X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: cditrr9j9tucbfrjfmxumxk1din386eq X-HE-Tag: 1689350664-587873 X-HE-Meta: U2FsdGVkX1/3WQNr15Gop8CgsTvYkKtz+pbhBL6faXPd35k/mcDMSkT9sqDCMn63QYpmVpOGatCh10I7sg32BvN8HJ3tFnubMnw/BhtnrNdYo+OFXFXF8jSg7VhaW+OyVAnFfsRo4PBp712mpNUiwXaVKb78HHIoTmtCn8XjMREg4wSY0tfSAwBvpq++4lnRXLdPwEWrPp/vqN4h5TN7P0rl6aPAd2Ho8mjZ6Rzbe5IlfbFB7E8OE+QwO9GzGn0ptJxrIK28qPGiJoPaS2XXJAl/93YCSzrMl932FSRzrpItsGq9s+eQIp2zJ9uJ1t7dzBcRn4K02HTYePyZj+yC7zKEWczHJVKJQUvo4FkeJlO17fJzKMxVPo+bMt/Ygx8yRyTO5jTBqlnEXTEyNSeDtK/tqkQXuhFn9efzyTb5P7KrtDAQXhykc6xcFyvARxfQtTrNJnMOZNRfvVWBeNUmQCRzKGcckkNzof1hhiGDj9e+cPTunRROrMOO8tfaseaFLzFTSAb0ZH2EgwvxXD3LEhJaKmSHDePCS8ad5UpkFFdRvh3td53nln3/32WKmh/HaLVGOAtLzndkLLFn1m9OB24fXmhCC5xkCzNcX9j+ObCrl8SLurgOlLXynbJB+tsNSRjOUmsQmaHOI8sdBvGzdVQWTB9Ey0BsxVNm1HNyM0cukYYaOQidaf5aSSFQeoYyNJ3SSFQmlbAWm3TIIeL5MX756JbIam0n3Q9cuA4T/Oo9MZzuSSwHMJC/m7YBh5I6HbqdL8tbriOiyMSZLcgC6u5/R70iQ5RttTKyfo0IM0AYekYiaatXU9ZbFZiS7WOwLmrdG7QsXvSqfAc6Ropqe4JCAfh6cFakFD9dApyi9GVWNI9+Hs/DeojaISdk0CbaAH6M5A7tR06WmLztBoktD8sg8+ZY486IJPTqNJ7dSc1VdRjBzp1YvOjQBSqCYP9uOhamXW3HQ9APiiwy+Lo qG3DSXHR 5S/FPjPXDDVuHctLvl5X+5YBp3nDmULDhu4y8elWWkGatrIBS4fG2FJXJMfmL+Nn/JNW7wkB4K00rfdVsJpDilbFMhxxnZyP8o6Aey4aCVClC7gvN9Nji4XF5dT+JXFChwj+8B9US4PGBxPvg0idvsGf+P7uuVeI2Odkal/iEwgGvSvXdauy7bqnXxdw9WZtqIGnt5dvQF6AocfnkThXKC82iSm6FIU6376Nw X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Hi All, This is v3 of a series to implement variable order, large folios for anonymous memory. (currently called "FLEXIBLE_THP") The objective of this is to improve performance by allocating larger chunks of memory during anonymous page faults. See [1] and [2] for background. There has been quite a bit more rework and simplification, mainly based on feedback from Yu Zhao. Additionally, I've added a command line parameter, flexthp_unhinted_max, the idea for which came from discussion with David Hildenbrand (thanks for all your feedback!). The last patch is for arm64 to explicitly override the default arch_wants_pte_order() and is intended as an example. If this series is accepted I suggest taking the first 3 patches through the mm tree and the arm64 change could be handled through the arm64 tree separately. Neither has any build dependency on the other. The patches are based on top of v6.5-rc1. I have a branch at [3]. Changes since v2 [2] -------------------- - Dropped commit "Allow deferred splitting of arbitrary large anon folios" - Huang, Ying suggested the "batch zap" work (which I dropped from this series after v1) is a prerequisite for merging FLXEIBLE_THP, so I've moved the deferred split patch to a separate series along with the batch zap changes. I plan to submit this series early next week. - Changed folio order fallback policy - We no longer iterate from preferred to 0 looking for acceptable policy - Instead we iterate through preferred, PAGE_ALLOC_COSTLY_ORDER and 0 only - Removed vma parameter from arch_wants_pte_order() - Added command line parameter `flexthp_unhinted_max` - clamps preferred order when vma hasn't explicitly opted-in to THP - Never allocate large folio for MADV_NOHUGEPAGE vma (or when THP is disabled for process or system). - Simplified implementation and integration with do_anonymous_page() - Removed dependency on set_ptes() Performance ----------- Performance is still similar to v2; see cover letter at [2]. Opens ----- - Feature name: FLEXIBLE_THP or LARGE_ANON_FOLIO? - Given the closer policy ties to THP, I prefer FLEXIBLE_THP - Prerequisits for merging - Sounds like there is a concensus that we should wait until exisitng features are improved to place nicely with large folios. [1] https://lore.kernel.org/linux-mm/20230626171430.3167004-1-ryan.roberts@arm.com/ [2] https://lore.kernel.org/linux-mm/20230703135330.1865927-1-ryan.roberts@arm.com/ [3] https://gitlab.arm.com/linux-arm/linux-rr/-/tree/features/granule_perf/anonfolio-lkml_v3 Thanks, Ryan Ryan Roberts (4): mm: Non-pmd-mappable, large folios for folio_add_new_anon_rmap() mm: Default implementation of arch_wants_pte_order() mm: FLEXIBLE_THP for improved performance arm64: mm: Override arch_wants_pte_order() .../admin-guide/kernel-parameters.txt | 10 + arch/arm64/include/asm/pgtable.h | 6 + include/linux/pgtable.h | 13 ++ mm/Kconfig | 10 + mm/memory.c | 187 ++++++++++++++++-- mm/rmap.c | 28 ++- 6 files changed, 230 insertions(+), 24 deletions(-) --- 2.25.1