From patchwork Wed Aug 23 13:13:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexandru Elisei X-Patchwork-Id: 13362289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7C186EE49A0 for ; Wed, 23 Aug 2023 13:15:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 135838E0011; Wed, 23 Aug 2023 09:15:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 043D190002C; Wed, 23 Aug 2023 09:15:20 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D8A9F8E0015; Wed, 23 Aug 2023 09:15:20 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id C76E28E0011 for ; Wed, 23 Aug 2023 09:15:20 -0400 (EDT) Received: from smtpin24.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id A85DFB11B2 for ; Wed, 23 Aug 2023 13:15:20 +0000 (UTC) X-FDA: 81155415600.24.4DD3B4F Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf03.hostedemail.com (Postfix) with ESMTP id E2BEA2002A for ; Wed, 23 Aug 2023 13:15:18 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of alexandru.elisei@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=alexandru.elisei@arm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1692796519; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=Ys9H5PnldVXevxD/YSg/rx5YKtIBi2TNqs30lPiZcP4=; b=oVF85SCdLoWc+Ec6tiRmmzyjBjiwqqZ4VXpn4rn9Q9ocBbb6A+ujjykV3++mSM7gSD3u/y xjsJ/EdznKAFjDqmFb4r6SqDjdeAL/+yWij6Ux1j7Va3jrQUUUxkSJ2Quq9F5DIifC7KGN 8yDp/is453MiTQrNQ+c0ozQvpc2HZm4= ARC-Authentication-Results: i=1; imf03.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf03.hostedemail.com: domain of alexandru.elisei@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=alexandru.elisei@arm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1692796519; a=rsa-sha256; cv=none; b=zd+ZnxAGPy+vW1GAC28331YOvekTQdgZt8yA0IauOHnW4m2TtZC3/8G9FOMwELkt20/CDa 8MZ6JflRJG+J2b0UD+nadnZpf6KFZGD88Q6Swu16vfq/GfsHlXVQDQr7cG/GLfYFENJgjW rW+h85ehjS2WkH7ueEHX54xgG9Qr0TM= Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 9081C15DB; Wed, 23 Aug 2023 06:15:58 -0700 (PDT) Received: from e121798.cable.virginm.net (unknown [172.31.20.19]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 2C7E63F740; Wed, 23 Aug 2023 06:15:12 -0700 (PDT) From: Alexandru Elisei To: catalin.marinas@arm.com, will@kernel.org, oliver.upton@linux.dev, maz@kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, arnd@arndb.de, akpm@linux-foundation.org, mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com, mhiramat@kernel.org, rppt@kernel.org, hughd@google.com Cc: pcc@google.com, steven.price@arm.com, anshuman.khandual@arm.com, vincenzo.frascino@arm.com, david@redhat.com, eugenis@google.com, kcc@google.com, hyesoo.yu@samsung.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, linux-fsdevel@vger.kernel.org, linux-arch@vger.kernel.org, linux-mm@kvack.org, linux-trace-kernel@vger.kernel.org Subject: [PATCH RFC 11/37] mm: migrate/mempolicy: Allocate metadata-enabled destination page Date: Wed, 23 Aug 2023 14:13:24 +0100 Message-Id: <20230823131350.114942-12-alexandru.elisei@arm.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20230823131350.114942-1-alexandru.elisei@arm.com> References: <20230823131350.114942-1-alexandru.elisei@arm.com> MIME-Version: 1.0 X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: E2BEA2002A X-Stat-Signature: 76r8i9fhjky6z9rjbbfiodbz1rj4dkgu X-Rspam-User: X-HE-Tag: 1692796518-65387 X-HE-Meta: U2FsdGVkX187iPphQE/w3ESbP2g/lEzqvOu7LKDu5ytT7ZCwkhsAFnBAw92j08QDaGNmthZNSjBJIoTaZDhF3SGMhxpp4D1mGsQoEilmW/yvD3+QNpzs2TqwAb+Z6pzxMwW4D+KtdDRsTU46iXz4KsPSLDy+PrWC8rNEw3LXLKZQw3OwknVZDlxp3t0UBtwvj4P9QxDEg+tP0hIqh1ihXZJRlGDPyMwQ2zXIqdlsQndulqvgjAiClKVNCPKB1HQ7sQYcXknPYt+jl1w6+cuLYb08/MmljNHm0YmlEroUkJXmThxQD/ho9NwrohqRX4vSv9o4oKhrUZgxv48qb+cS5d4VELW0EhMDqlsG9PwY96MYPLunNDINaOvFcs+mKjWuqSVzv5V5lUcEJGMutPXATiT6nXoAHs6GSyznc+rQssOC3mEkN4/P75JfUsPp7PR9fmnHUFYSqPndH3OpYR4iBogo3TH/eJttEvCEWRnLte84jmxqNSZmMcLYz2cEg57TSWoFEVZ8qDACw33Xl93vGeH/ilrvezET8AOxnGO1XCR42L/reWOTJFH+v/epPOPZgQSMY1g8mDsN8uJfsFDVtG6FyBcuRdzdw4kLz15i3aJfoLw2yfktMdKUCYXjCm9I2RGAytpx8+bsQJzT+oqp0ZIAVuuMGmzyNsr8xGKq/bDF9O7OlwVFeBVByLDJnMj4NNx9ZlENukYM5MbaqjLReMQnQ2yMnmOMhStYNHopwzfn6g5L7QlS2cfOE1LL6fDvvGgpQZpc+3hvgBtT03VRD0tf6MtWur3F0RAMITYfxC0LBJyG3HPJRCCzWPDAVGNhx/m19FCBjv3TWx/Vls8pUNNQ3Q4l/VRQBeCKDHVNA0h385jfD3OD1zQHXRMuEPH6pkEvdBHmCA6yuPa5bNkch2FW46Gv4aEWGND9Bx93Ee9oIZ/PUahpDhi8p1+NzUU3Z8kh3CJ1XTE4YUIr1nS AAZjA3VK 5P0eCRWWqvbC9Rif005wKHnVv1PyU3sjxoMogVSae7hXPeBWRUfrRCeYlcGjgki/yY8uYz9fr6gLAu455dHEgY6HhMOy/G97d9K3TvGiTDZkmy9zpT3o0hWJ9EM1PCksp6/wa21464e2LAAhJdz26RSBYBhY4vUs8lzfJ X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: With explicit metadata page management support, it's important to know if the source page for migration has metadata associated with it for two reasons: - The page allocator knows to skip metadata pages (which cannot have metadata) when allocating the destination page. - The associated metadata page is correctly reserved when fulfilling the allocation for the destination page. When choosing the destination during migration, keep track if the source page has metadata. The mbind() system call changes the NUMA allocation policy for the specified memory range and nodemask. If the MPOL_MF_MOVE or MPOL_MF_MOVE_ALL flags are set, then any existing allocations that fall within the range which don't conform to the specified policy will be migrated. The function that allocates the destination page for migration is new_page(), teach it too about source pages with metadata. Signed-off-by: Alexandru Elisei --- arch/arm64/include/asm/memory_metadata.h | 4 ++++ include/asm-generic/memory_metadata.h | 4 ++++ mm/mempolicy.c | 4 ++++ mm/migrate.c | 6 ++++++ 4 files changed, 18 insertions(+) diff --git a/arch/arm64/include/asm/memory_metadata.h b/arch/arm64/include/asm/memory_metadata.h index c57c435c8ba3..132707fce9ab 100644 --- a/arch/arm64/include/asm/memory_metadata.h +++ b/arch/arm64/include/asm/memory_metadata.h @@ -21,6 +21,10 @@ static inline bool alloc_can_use_metadata_pages(gfp_t gfp_mask) #define page_has_metadata(page) page_mte_tagged(page) +static inline bool folio_has_metadata(struct folio *folio) +{ + return page_has_metadata(&folio->page); +} #endif /* CONFIG_MEMORY_METADATA */ #endif /* __ASM_MEMORY_METADATA_H */ diff --git a/include/asm-generic/memory_metadata.h b/include/asm-generic/memory_metadata.h index 02b279823920..8f4e2fba222f 100644 --- a/include/asm-generic/memory_metadata.h +++ b/include/asm-generic/memory_metadata.h @@ -20,6 +20,10 @@ static inline bool page_has_metadata(struct page *page) { return false; } +static inline bool folio_has_metadata(struct folio *folio) +{ + return false; +} #endif /* !CONFIG_MEMORY_METADATA */ #endif /* __ASM_GENERIC_MEMORY_METADATA_H */ diff --git a/mm/mempolicy.c b/mm/mempolicy.c index edc25195f5bd..d164b5c50243 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -103,6 +103,7 @@ #include #include +#include #include #include #include @@ -1219,6 +1220,9 @@ static struct folio *new_folio(struct folio *src, unsigned long start) if (folio_test_large(src)) gfp = GFP_TRANSHUGE; + if (folio_has_metadata(src)) + gfp |= __GFP_TAGGED; + /* * if !vma, vma_alloc_folio() will use task or system default policy */ diff --git a/mm/migrate.c b/mm/migrate.c index 24baad2571e3..c6826562220a 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -51,6 +51,7 @@ #include #include +#include #include #include @@ -1990,6 +1991,9 @@ struct folio *alloc_migration_target(struct folio *src, unsigned long private) if (nid == NUMA_NO_NODE) nid = folio_nid(src); + if (folio_has_metadata(src)) + gfp_mask |= __GFP_TAGGED; + if (folio_test_hugetlb(src)) { struct hstate *h = folio_hstate(src); @@ -2476,6 +2480,8 @@ static struct folio *alloc_misplaced_dst_folio(struct folio *src, __GFP_NOWARN; gfp &= ~__GFP_RECLAIM; } + if (folio_has_metadata(src)) + gfp |= __GFP_TAGGED; return __folio_alloc_node(gfp, order, nid); }