From patchwork Fri Jan 28 11:09:41 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 12728225 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 78BFEC4332F for ; Fri, 28 Jan 2022 11:09:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id F2F0D6B0073; Fri, 28 Jan 2022 06:09:50 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E8F0E6B0074; Fri, 28 Jan 2022 06:09:50 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id CBA086B0075; Fri, 28 Jan 2022 06:09:50 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0251.hostedemail.com [216.40.44.251]) by kanga.kvack.org (Postfix) with ESMTP id AE7C06B0073 for ; Fri, 28 Jan 2022 06:09:50 -0500 (EST) Received: from smtpin05.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id 71648181C1921 for ; Fri, 28 Jan 2022 11:09:50 +0000 (UTC) X-FDA: 79079425740.05.3DA8328 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf16.hostedemail.com (Postfix) with ESMTP id 73A9618006D for ; Fri, 28 Jan 2022 11:09:48 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id B6554113E; Fri, 28 Jan 2022 03:09:47 -0800 (PST) Received: from p8cg001049571a15.arm.com (unknown [10.163.45.35]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id F246B3F766; Fri, 28 Jan 2022 03:09:41 -0800 (PST) From: Anshuman Khandual To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: Anshuman Khandual , Steven Rostedt , Ingo Molnar , Zi Yan , Naoya Horiguchi , John Hubbard , Matthew Wilcox , Michael Ellerman , Paul Mackerras , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: [PATCH V3 1/2] mm/migration: Add trace events for THP migrations Date: Fri, 28 Jan 2022 16:39:41 +0530 Message-Id: <1643368182-9588-2-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643368182-9588-1-git-send-email-anshuman.khandual@arm.com> References: <1643368182-9588-1-git-send-email-anshuman.khandual@arm.com> X-Rspamd-Queue-Id: 73A9618006D X-Rspam-User: nil Authentication-Results: imf16.hostedemail.com; dkim=none; spf=pass (imf16.hostedemail.com: domain of anshuman.khandual@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com; dmarc=pass (policy=none) header.from=arm.com X-Stat-Signature: c4m1x9h9kdg7zq3h4w5ke3qrtanj1z8c X-Rspamd-Server: rspam08 X-HE-Tag: 1643368188-546932 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds two trace events for PMD based THP migration without split. These events closely follow the implementation details like setting and removing of PMD migration entries, which are essential operations for THP migration. This moves CREATE_TRACE_POINTS into generic THP from powerpc for these new trace events to be available on other platforms as well. Cc: Steven Rostedt Cc: Ingo Molnar Cc: Andrew Morton Cc: Zi Yan Cc: Naoya Horiguchi Cc: John Hubbard Cc: Matthew Wilcox Cc: Michael Ellerman Cc: Paul Mackerras Cc: linuxppc-dev@lists.ozlabs.org Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Anshuman Khandual --- arch/powerpc/mm/book3s64/trace.c | 1 - include/trace/events/thp.h | 27 +++++++++++++++++++++++++++ mm/huge_memory.c | 5 +++++ 3 files changed, 32 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/mm/book3s64/trace.c b/arch/powerpc/mm/book3s64/trace.c index b86e7b906257..ccd64b5e6cac 100644 --- a/arch/powerpc/mm/book3s64/trace.c +++ b/arch/powerpc/mm/book3s64/trace.c @@ -3,6 +3,5 @@ * This file is for defining trace points and trace related helpers. */ #ifdef CONFIG_TRANSPARENT_HUGEPAGE -#define CREATE_TRACE_POINTS #include #endif diff --git a/include/trace/events/thp.h b/include/trace/events/thp.h index ca3f2767828a..202b3e3e67ff 100644 --- a/include/trace/events/thp.h +++ b/include/trace/events/thp.h @@ -48,6 +48,33 @@ TRACE_EVENT(hugepage_update, TP_printk("hugepage update at addr 0x%lx and pte = 0x%lx clr = 0x%lx, set = 0x%lx", __entry->addr, __entry->pte, __entry->clr, __entry->set) ); +DECLARE_EVENT_CLASS(migration_pmd, + + TP_PROTO(unsigned long addr, unsigned long pmd), + + TP_ARGS(addr, pmd), + + TP_STRUCT__entry( + __field(unsigned long, addr) + __field(unsigned long, pmd) + ), + + TP_fast_assign( + __entry->addr = addr; + __entry->pmd = pmd; + ), + TP_printk("addr=%lx, pmd=%lx", __entry->addr, __entry->pmd) +); + +DEFINE_EVENT(migration_pmd, set_migration_pmd, + TP_PROTO(unsigned long addr, unsigned long pmd), + TP_ARGS(addr, pmd) +); + +DEFINE_EVENT(migration_pmd, remove_migration_pmd, + TP_PROTO(unsigned long addr, unsigned long pmd), + TP_ARGS(addr, pmd) +); #endif /* _TRACE_THP_H */ /* This part must be outside protection */ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 406a3c28c026..ab49f9a3e420 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -39,6 +39,9 @@ #include #include "internal.h" +#define CREATE_TRACE_POINTS +#include + /* * By default, transparent hugepage support is disabled in order to avoid * risking an increased memory footprint for applications that are not @@ -3173,6 +3176,7 @@ void set_pmd_migration_entry(struct page_vma_mapped_walk *pvmw, set_pmd_at(mm, address, pvmw->pmd, pmdswp); page_remove_rmap(page, true); put_page(page); + trace_set_migration_pmd(address, pmd_val(pmdswp)); } void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) @@ -3206,5 +3210,6 @@ void remove_migration_pmd(struct page_vma_mapped_walk *pvmw, struct page *new) if ((vma->vm_flags & VM_LOCKED) && !PageDoubleMap(new)) mlock_vma_page(new); update_mmu_cache_pmd(vma, address, pvmw->pmd); + trace_remove_migration_pmd(address, pmd_val(pmde)); } #endif From patchwork Fri Jan 28 11:09:42 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Anshuman Khandual X-Patchwork-Id: 12728226 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 67FAFC433F5 for ; Fri, 28 Jan 2022 11:09:55 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 00FBC6B0074; Fri, 28 Jan 2022 06:09:55 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id F01706B0075; Fri, 28 Jan 2022 06:09:54 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DF26D6B0078; Fri, 28 Jan 2022 06:09:54 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.a.hostedemail.com [64.99.140.24]) by kanga.kvack.org (Postfix) with ESMTP id D227F6B0074 for ; Fri, 28 Jan 2022 06:09:54 -0500 (EST) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 8F34D22C28 for ; Fri, 28 Jan 2022 11:09:54 +0000 (UTC) X-FDA: 79079425908.01.17A95FC Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by imf08.hostedemail.com (Postfix) with ESMTP id 0A77116003B for ; Fri, 28 Jan 2022 11:09:53 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 5B2F912FC; Fri, 28 Jan 2022 03:09:53 -0800 (PST) Received: from p8cg001049571a15.arm.com (unknown [10.163.45.35]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPA id 85AC03F766; Fri, 28 Jan 2022 03:09:48 -0800 (PST) From: Anshuman Khandual To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: Anshuman Khandual , Steven Rostedt , Ingo Molnar , Zi Yan , Naoya Horiguchi , John Hubbard , Matthew Wilcox , linux-kernel@vger.kernel.org Subject: [PATCH V3 2/2] mm/migration: Add trace events for base page and HugeTLB migrations Date: Fri, 28 Jan 2022 16:39:42 +0530 Message-Id: <1643368182-9588-3-git-send-email-anshuman.khandual@arm.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1643368182-9588-1-git-send-email-anshuman.khandual@arm.com> References: <1643368182-9588-1-git-send-email-anshuman.khandual@arm.com> X-Stat-Signature: sjzqgc1cq5ckiwugccgd7p74ygh6o93f X-Rspam-User: nil Authentication-Results: imf08.hostedemail.com; dkim=none; dmarc=pass (policy=none) header.from=arm.com; spf=pass (imf08.hostedemail.com: domain of anshuman.khandual@arm.com designates 217.140.110.172 as permitted sender) smtp.mailfrom=anshuman.khandual@arm.com X-Rspamd-Server: rspam01 X-Rspamd-Queue-Id: 0A77116003B X-HE-Tag: 1643368193-900259 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: This adds two trace events for base page and HugeTLB page migrations. These events, closely follow the implementation details like setting and removing of PTE migration entries, which are essential operations for migration. The new CREATE_TRACE_POINTS in covers both and based trace events. Hence drop redundant CREATE_TRACE_POINTS from other places which could have otherwise conflicted during build. Cc: Steven Rostedt Cc: Ingo Molnar Cc: Andrew Morton Cc: Zi Yan Cc: Naoya Horiguchi Cc: John Hubbard Cc: Matthew Wilcox Cc: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org Reported-by: kernel test robot Signed-off-by: Anshuman Khandual --- arch/x86/mm/init.c | 1 - include/trace/events/migrate.h | 31 +++++++++++++++++++++++++++++++ mm/migrate.c | 4 +++- mm/rmap.c | 6 ++++++ 4 files changed, 40 insertions(+), 2 deletions(-) diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c index 4ba024d5b63a..d8cfce221275 100644 --- a/arch/x86/mm/init.c +++ b/arch/x86/mm/init.c @@ -31,7 +31,6 @@ * We need to define the tracepoints somewhere, and tlb.c * is only compiled when SMP=y. */ -#define CREATE_TRACE_POINTS #include #include "mm_internal.h" diff --git a/include/trace/events/migrate.h b/include/trace/events/migrate.h index 779f3fad9ecd..061b5128f335 100644 --- a/include/trace/events/migrate.h +++ b/include/trace/events/migrate.h @@ -105,6 +105,37 @@ TRACE_EVENT(mm_migrate_pages_start, __print_symbolic(__entry->reason, MIGRATE_REASON)) ); +DECLARE_EVENT_CLASS(migration_pte, + + TP_PROTO(unsigned long addr, unsigned long pte, int order), + + TP_ARGS(addr, pte, order), + + TP_STRUCT__entry( + __field(unsigned long, addr) + __field(unsigned long, pte) + __field(int, order) + ), + + TP_fast_assign( + __entry->addr = addr; + __entry->pte = pte; + __entry->order = order; + ), + + TP_printk("addr=%lx, pte=%lx order=%d", __entry->addr, __entry->pte, __entry->order) +); + +DEFINE_EVENT(migration_pte, set_migration_pte, + TP_PROTO(unsigned long addr, unsigned long pte, int order), + TP_ARGS(addr, pte, order) +); + +DEFINE_EVENT(migration_pte, remove_migration_pte, + TP_PROTO(unsigned long addr, unsigned long pte, int order), + TP_ARGS(addr, pte, order) +); + #endif /* _TRACE_MIGRATE_H */ /* This part must be outside protection */ diff --git a/mm/migrate.c b/mm/migrate.c index c7da064b4781..79e3a553923a 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -54,7 +54,6 @@ #include -#define CREATE_TRACE_POINTS #include #include "internal.h" @@ -257,6 +256,9 @@ static bool remove_migration_pte(struct page *page, struct vm_area_struct *vma, if (PageTransHuge(page) && PageMlocked(page)) clear_page_mlock(page); + trace_remove_migration_pte(pvmw.address, pte_val(pte), + compound_order(new)); + /* No need to invalidate - it was non-present before */ update_mmu_cache(vma, pvmw.address, pvmw.pte); } diff --git a/mm/rmap.c b/mm/rmap.c index 6a1e8c7f6213..cae1c46440d6 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -76,7 +76,9 @@ #include +#define CREATE_TRACE_POINTS #include +#include #include "internal.h" @@ -1861,6 +1863,8 @@ static bool try_to_migrate_one(struct page *page, struct vm_area_struct *vma, if (pte_swp_uffd_wp(pteval)) swp_pte = pte_swp_mkuffd_wp(swp_pte); set_pte_at(mm, pvmw.address, pvmw.pte, swp_pte); + trace_set_migration_pte(pvmw.address, pte_val(swp_pte), + compound_order(page)); /* * No need to invalidate here it will synchronize on * against the special swap migration pte. @@ -1929,6 +1933,8 @@ static bool try_to_migrate_one(struct page *page, struct vm_area_struct *vma, if (pte_uffd_wp(pteval)) swp_pte = pte_swp_mkuffd_wp(swp_pte); set_pte_at(mm, address, pvmw.pte, swp_pte); + trace_set_migration_pte(address, pte_val(swp_pte), + compound_order(page)); /* * No need to invalidate here it will synchronize on * against the special swap migration pte.