From patchwork Wed Aug 23 13:13:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandru Elisei X-Patchwork-Id: 13362343 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9975EEE49A0 for ; Wed, 23 Aug 2023 13:16:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=sBIFspmQVu9oKbpktqPAIjxVYVGUSBmq4XPNX062aRI=; b=lqCH9UYqjH5IKy AE5LNkqRVjtkrlsGtrp6E7J7agVAyOTrDazgLVC6jDIU8nep6tgsO2j14xB6Mhl20xi3MMzijGuTC n5cpL9h0oD1sxBUAtTP83yiA54iR5aTEvWYqOPockUNox1xJgxTOuUmNi7wf+HXCUWdhmYaI8DGn5 KdC4mq3u/G5FDY9sRDP59G6mTDlI+1ijD2IROOzghWvi67stLcdVm0D5B3rXIlQCXmplXJLPHMg2x 0U1PA4oh/aRpcB2reAOqlCoXKiucCRMOyjrmiCHVaQ40FC5F5Ef30kXkFZuooxaKKQ9H+1uX5W5ac lETBfTKjup9nIgH0sI9w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYnhy-000csK-08; Wed, 23 Aug 2023 13:15:38 +0000 Received: from foss.arm.com ([217.140.110.172]) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1qYnhf-000ceD-07 for linux-arm-kernel@lists.infradead.org; Wed, 23 Aug 2023 13:15:22 +0000 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9081C15DB; Wed, 23 Aug 2023 06:15:58 -0700 (PDT) Received: from e121798.cable.virginm.net (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2C7E63F740; Wed, 23 Aug 2023 06:15:12 -0700 (PDT) From: Alexandru Elisei To: catalin.marinas@arm.com, will@kernel.org, oliver.upton@linux.dev, maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, arnd@arndb.de, akpm@linux-foundation.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mhiramat@kernel.org, rppt@kernel.org, hughd@google.com Cc: pcc@google.com, steven.price@arm.com, anshuman.khandual@arm.com, vincenzo.frascino@arm.com, david@redhat.com, eugenis@google.com, kcc@google.com, hyesoo.yu@samsung.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH RFC 11/37] mm: migrate/mempolicy: Allocate metadata-enabled destination page Date: Wed, 23 Aug 2023 14:13:24 +0100 Message-Id: <20230823131350.114942-12-alexandru.elisei@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230823131350.114942-1-alexandru.elisei@arm.com> References: <20230823131350.114942-1-alexandru.elisei@arm.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230823_061519_209996_DA6B2B63 X-CRM114-Status: GOOD ( 13.29 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org With explicit metadata page management support, it's important to know if the source page for migration has metadata associated with it for two reasons: - The page allocator knows to skip metadata pages (which cannot have metadata) when allocating the destination page. - The associated metadata page is correctly reserved when fulfilling the allocation for the destination page. When choosing the destination during migration, keep track if the source page has metadata. The mbind() system call changes the NUMA allocation policy for the specified memory range and nodemask. If the MPOL_MF_MOVE or MPOL_MF_MOVE_ALL flags are set, then any existing allocations that fall within the range which don't conform to the specified policy will be migrated. The function that allocates the destination page for migration is new_page(), teach it too about source pages with metadata. Signed-off-by: Alexandru Elisei --- arch/arm64/include/asm/memory_metadata.h | 4 ++++ include/asm-generic/memory_metadata.h | 4 ++++ mm/mempolicy.c | 4 ++++ mm/migrate.c | 6 ++++++ 4 files changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/memory_metadata.h b/arch/arm64/include/asm/memory_metadata.h index c57c435c8ba3..132707fce9ab 100644 --- a/arch/arm64/include/asm/memory_metadata.h +++ b/arch/arm64/include/asm/memory_metadata.h @@ -21,6 +21,10 @@ static inline bool alloc_can_use_metadata_pages(gfp_t gfp_mask) #define page_has_metadata(page) page_mte_tagged(page) +static inline bool folio_has_metadata(struct folio *folio) +{ + return page_has_metadata(&folio->page); +} #endif /* CONFIG_MEMORY_METADATA */ #endif /* __ASM_MEMORY_METADATA_H */ diff --git a/include/asm-generic/memory_metadata.h b/include/asm-generic/memory_metadata.h index 02b279823920..8f4e2fba222f 100644 --- a/include/asm-generic/memory_metadata.h +++ b/include/asm-generic/memory_metadata.h @@ -20,6 +20,10 @@ static inline bool page_has_metadata(struct page *page) { return false; } +static inline bool folio_has_metadata(struct folio *folio) +{ + return false; +} #endif /* !CONFIG_MEMORY_METADATA */ #endif /* __ASM_GENERIC_MEMORY_METADATA_H */ diff --git a/mm/mempolicy.c b/mm/mempolicy.c index edc25195f5bd..d164b5c50243 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -103,6 +103,7 @@ #include #include +#include #include #include #include @@ -1219,6 +1220,9 @@ static struct folio *new_folio(struct folio *src, unsigned long start) if (folio_test_large(src)) gfp = GFP_TRANSHUGE; + if (folio_has_metadata(src)) + gfp |= __GFP_TAGGED; + /* * if !vma, vma_alloc_folio() will use task or system default policy */ diff --git a/mm/migrate.c b/mm/migrate.c index 24baad2571e3..c6826562220a 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -51,6 +51,7 @@ #include #include +#include #include #include @@ -1990,6 +1991,9 @@ struct folio *alloc_migration_target(struct folio *src, unsigned long private) if (nid == NUMA_NO_NODE) nid = folio_nid(src); + if (folio_has_metadata(src)) + gfp_mask |= __GFP_TAGGED; + if (folio_test_hugetlb(src)) { struct hstate *h = folio_hstate(src); @@ -2476,6 +2480,8 @@ static struct folio *alloc_misplaced_dst_folio(struct folio *src, __GFP_NOWARN; gfp &= ~__GFP_RECLAIM; } + if (folio_has_metadata(src)) + gfp |= __GFP_TAGGED; return __folio_alloc_node(gfp, order, nid); }