From patchwork Fri Jul 14 16:04:03 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Ryan Roberts X-Patchwork-Id: 13313826 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C6C71EB64DC for ; Fri, 14 Jul 2023 16:05:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=sAd5y+kORe9mUe/dJIzagZ+clAT4G1+8Ss2Vhu5f4zo=; b=0XZFLcmQibEoyp j5YaR/E+IZb84ntgkODk6wic2s/+coaJnEEFNGneLctI9faenR52acvj8DQet+UEMoVrcakvT9H2R NuQxsOL8nxz1mapmq9cHcCxL2VMx0zHff/+X33gfGB12il2G1qMz1ACefVwclmNnrrJYGlH+pYfVX l95AP0gy4kmD7dhptvfNfORlss9fBBzkIXALTJ+j3ilKAIwxXv7SzNfNCHm0C+xh86LQQFfjrdw+G jJwvVpzqRWo0jQcrKgwcHVxYiQ2RmEqKfDT5tJg4+O+p/rWywSWTk9hR0LiDnDLffPo858z7BBkwX 1XUEFUyE7nsoShFADGsA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qKLHV-006bGg-2R; Fri, 14 Jul 2023 16:04:33 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qKLHR-006bEX-2R for linux-arm-kernel@lists.infradead.org; Fri, 14 Jul 2023 16:04:31 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 54ACA1576; Fri, 14 Jul 2023 09:05:05 -0700 (PDT) Received: from e125769.cambridge.arm.com (e125769.cambridge.arm.com [10.1.196.26]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 06F163F740; Fri, 14 Jul 2023 09:04:20 -0700 (PDT) From: Ryan Roberts To: Andrew Morton , Matthew Wilcox , "Kirill A. Shutemov" , Yin Fengwei , David Hildenbrand , Yu Zhao , Catalin Marinas , Will Deacon , Anshuman Khandual , Yang Shi , "Huang, Ying" , Zi Yan , Luis Chamberlain Cc: Ryan Roberts , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH v3 0/4] variable-order, large folios for anonymous memory Date: Fri, 14 Jul 2023 17:04:03 +0100 Message-Id: <20230714160407.4142030-1-ryan.roberts@arm.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230714_090429_904775_4CC8FC41 X-CRM114-Status: GOOD ( 17.44 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi All, This is v3 of a series to implement variable order, large folios for anonymous memory. (currently called "FLEXIBLE_THP") The objective of this is to improve performance by allocating larger chunks of memory during anonymous page faults. See [1] and [2] for background. There has been quite a bit more rework and simplification, mainly based on feedback from Yu Zhao. Additionally, I've added a command line parameter, flexthp_unhinted_max, the idea for which came from discussion with David Hildenbrand (thanks for all your feedback!). The last patch is for arm64 to explicitly override the default arch_wants_pte_order() and is intended as an example. If this series is accepted I suggest taking the first 3 patches through the mm tree and the arm64 change could be handled through the arm64 tree separately. Neither has any build dependency on the other. The patches are based on top of v6.5-rc1. I have a branch at [3]. Changes since v2 [2] -------------------- - Dropped commit "Allow deferred splitting of arbitrary large anon folios" - Huang, Ying suggested the "batch zap" work (which I dropped from this series after v1) is a prerequisite for merging FLXEIBLE_THP, so I've moved the deferred split patch to a separate series along with the batch zap changes. I plan to submit this series early next week. - Changed folio order fallback policy - We no longer iterate from preferred to 0 looking for acceptable policy - Instead we iterate through preferred, PAGE_ALLOC_COSTLY_ORDER and 0 only - Removed vma parameter from arch_wants_pte_order() - Added command line parameter `flexthp_unhinted_max` - clamps preferred order when vma hasn't explicitly opted-in to THP - Never allocate large folio for MADV_NOHUGEPAGE vma (or when THP is disabled for process or system). - Simplified implementation and integration with do_anonymous_page() - Removed dependency on set_ptes() Performance ----------- Performance is still similar to v2; see cover letter at [2]. Opens ----- - Feature name: FLEXIBLE_THP or LARGE_ANON_FOLIO? - Given the closer policy ties to THP, I prefer FLEXIBLE_THP - Prerequisits for merging - Sounds like there is a concensus that we should wait until exisitng features are improved to place nicely with large folios. [1] https://lore.kernel.org/linux-mm/20230626171430.3167004-1-ryan.roberts@arm.com/ [2] https://lore.kernel.org/linux-mm/20230703135330.1865927-1-ryan.roberts@arm.com/ [3] https://gitlab.arm.com/linux-arm/linux-rr/-/tree/features/granule_perf/anonfolio-lkml_v3 Thanks, Ryan Ryan Roberts (4): mm: Non-pmd-mappable, large folios for folio_add_new_anon_rmap() mm: Default implementation of arch_wants_pte_order() mm: FLEXIBLE_THP for improved performance arm64: mm: Override arch_wants_pte_order() .../admin-guide/kernel-parameters.txt | 10 + arch/arm64/include/asm/pgtable.h | 6 + include/linux/pgtable.h | 13 ++ mm/Kconfig | 10 + mm/memory.c | 187 ++++++++++++++++-- mm/rmap.c | 28 ++- 6 files changed, 230 insertions(+), 24 deletions(-) --- 2.25.1