From patchwork Wed Dec 16 09:42:14 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: SeongJae Park X-Patchwork-Id: 11977133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-18.8 required=3.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F340EC4361B for ; Wed, 16 Dec 2020 09:45:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 64D4F2312E for ; Wed, 16 Dec 2020 09:45:13 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 64D4F2312E Authentication-Results: mail.kernel.org; dmarc=fail (p=quarantine dis=none) header.from=amazon.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D95D46B006E; Wed, 16 Dec 2020 04:45:12 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id D44FC6B0070; Wed, 16 Dec 2020 04:45:12 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C0CD96B0073; Wed, 16 Dec 2020 04:45:12 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0116.hostedemail.com [216.40.44.116]) by kanga.kvack.org (Postfix) with ESMTP id A52146B006E for ; Wed, 16 Dec 2020 04:45:12 -0500 (EST) Received: from smtpin30.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 7662A1AD91 for ; Wed, 16 Dec 2020 09:45:12 +0000 (UTC) X-FDA: 77598662064.30.fly40_2c070602742b Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin30.hostedemail.com (Postfix) with ESMTP id 5365B180D6064 for ; Wed, 16 Dec 2020 09:45:12 +0000 (UTC) X-HE-Tag: fly40_2c070602742b X-Filterd-Recvd-Size: 10860 Received: from smtp-fw-2101.amazon.com (smtp-fw-2101.amazon.com [72.21.196.25]) by imf36.hostedemail.com (Postfix) with ESMTP for ; Wed, 16 Dec 2020 09:45:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1608111912; x=1639647912; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version; bh=Lj6+I9XWmZiqjVkd1Bor2fLoWLXqVAIP/n4Y2agFqYw=; b=KEBPieXdYkinAbSA2clUIAqnoIqjYXMYwUfPDti23oPzAzyePTb2jCxi FjFUcnuKCOvbvQf1Tbe9Z4SrR4akz889WG9ExETlf7Ym1jR03o1ZkL1Vc H5qUhYyPZ+e+j3krLXHhKe9Sv1/iEmVUJLQxA00sDRDWUlBIYTRbnvVPZ E=; X-IronPort-AV: E=Sophos;i="5.78,424,1599523200"; d="scan'208";a="69443702" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2c-87a10be6.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-2101.iad2.amazon.com with ESMTP; 16 Dec 2020 09:45:07 +0000 Received: from EX13D31EUA001.ant.amazon.com (pdx1-ws-svc-p6-lb9-vlan2.pdx.amazon.com [10.236.137.194]) by email-inbound-relay-2c-87a10be6.us-west-2.amazon.com (Postfix) with ESMTPS id DF141A1EBF; Wed, 16 Dec 2020 09:44:55 +0000 (UTC) Received: from u3f2cd687b01c55.ant.amazon.com (10.43.161.31) by EX13D31EUA001.ant.amazon.com (10.43.165.15) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Wed, 16 Dec 2020 09:44:36 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC v10 06/13] mm/damon/vaddr: Separate commonly usable functions Date: Wed, 16 Dec 2020 10:42:14 +0100 Message-ID: <20201216094221.11898-7-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20201216094221.11898-1-sjpark@amazon.com> References: <20201216094221.11898-1-sjpark@amazon.com> MIME-Version: 1.0 X-Originating-IP: [10.43.161.31] X-ClientProxiedBy: EX13D16UWB001.ant.amazon.com (10.43.161.17) To EX13D31EUA001.ant.amazon.com (10.43.165.15) X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: SeongJae Park This commit moves functions in the default virtual address spaces monitoring primitives that commonly usable from other address spaces like physical address space into a header file. Those will be reused by the physical address space monitoring primitives in the following commit. Signed-off-by: SeongJae Park --- mm/damon/Makefile | 2 +- mm/damon/prmtv-common.c | 104 ++++++++++++++++++++++++++++++++++++++ mm/damon/prmtv-common.h | 21 ++++++++ mm/damon/vaddr.c | 108 +--------------------------------------- 4 files changed, 128 insertions(+), 107 deletions(-) create mode 100644 mm/damon/prmtv-common.c create mode 100644 mm/damon/prmtv-common.h diff --git a/mm/damon/Makefile b/mm/damon/Makefile index fed4be3bace3..99b1bfe01ff5 100644 --- a/mm/damon/Makefile +++ b/mm/damon/Makefile @@ -1,5 +1,5 @@ # SPDX-License-Identifier: GPL-2.0 obj-$(CONFIG_DAMON) := core.o -obj-$(CONFIG_DAMON_VADDR) += vaddr.o +obj-$(CONFIG_DAMON_VADDR) += prmtv-common.o vaddr.o obj-$(CONFIG_DAMON_DBGFS) += dbgfs.o diff --git a/mm/damon/prmtv-common.c b/mm/damon/prmtv-common.c new file mode 100644 index 000000000000..6cdb96cbc9ef --- /dev/null +++ b/mm/damon/prmtv-common.c @@ -0,0 +1,104 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Common Primitives for Data Access Monitoring + * + * Author: SeongJae Park + */ + +#include "prmtv-common.h" + +static void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm, + unsigned long addr) +{ + bool referenced = false; + struct page *page = pte_page(*pte); + + if (pte_young(*pte)) { + referenced = true; + *pte = pte_mkold(*pte); + } + +#ifdef CONFIG_MMU_NOTIFIER + if (mmu_notifier_clear_young(mm, addr, addr + PAGE_SIZE)) + referenced = true; +#endif /* CONFIG_MMU_NOTIFIER */ + + if (referenced) + set_page_young(page); + + set_page_idle(page); +} + +static void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm, + unsigned long addr) +{ +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + bool referenced = false; + struct page *page = pmd_page(*pmd); + + if (pmd_young(*pmd)) { + referenced = true; + *pmd = pmd_mkold(*pmd); + } + +#ifdef CONFIG_MMU_NOTIFIER + if (mmu_notifier_clear_young(mm, addr, + addr + ((1UL) << HPAGE_PMD_SHIFT))) + referenced = true; +#endif /* CONFIG_MMU_NOTIFIER */ + + if (referenced) + set_page_young(page); + + set_page_idle(page); +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ +} + +void damon_va_mkold(struct mm_struct *mm, unsigned long addr) +{ + pte_t *pte = NULL; + pmd_t *pmd = NULL; + spinlock_t *ptl; + + if (follow_pte_pmd(mm, addr, NULL, &pte, &pmd, &ptl)) + return; + + if (pte) { + damon_ptep_mkold(pte, mm, addr); + pte_unmap_unlock(pte, ptl); + } else { + damon_pmdp_mkold(pmd, mm, addr); + spin_unlock(ptl); + } +} + +bool damon_va_young(struct mm_struct *mm, unsigned long addr, + unsigned long *page_sz) +{ + pte_t *pte = NULL; + pmd_t *pmd = NULL; + spinlock_t *ptl; + bool young = false; + + if (follow_pte_pmd(mm, addr, NULL, &pte, &pmd, &ptl)) + return false; + + *page_sz = PAGE_SIZE; + if (pte) { + young = pte_young(*pte); + if (!young) + young = !page_is_idle(pte_page(*pte)); + pte_unmap_unlock(pte, ptl); + return young; + } + +#ifdef CONFIG_TRANSPARENT_HUGEPAGE + young = pmd_young(*pmd); + if (!young) + young = !page_is_idle(pmd_page(*pmd)); + spin_unlock(ptl); + *page_sz = ((1UL) << HPAGE_PMD_SHIFT); +#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ + + return young; +} diff --git a/mm/damon/prmtv-common.h b/mm/damon/prmtv-common.h new file mode 100644 index 000000000000..a66a6139b4fc --- /dev/null +++ b/mm/damon/prmtv-common.h @@ -0,0 +1,21 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Common Primitives for Data Access Monitoring + * + * Author: SeongJae Park + */ + +#include +#include +#include +#include +#include +#include +#include + +/* Get a random number in [l, r) */ +#define damon_rand(l, r) (l + prandom_u32_max(r - l)) + +void damon_va_mkold(struct mm_struct *mm, unsigned long addr); +bool damon_va_young(struct mm_struct *mm, unsigned long addr, + unsigned long *page_sz); diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index 2075f07f728b..915b12329c6e 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -8,22 +8,14 @@ #define pr_fmt(fmt) "damon-va: " fmt #include -#include -#include -#include -#include -#include -#include -#include + +#include "prmtv-common.h" #ifdef CONFIG_DAMON_VADDR_KUNIT_TEST #undef DAMON_MIN_REGION #define DAMON_MIN_REGION 1 #endif -/* Get a random number in [l, r) */ -#define damon_rand(l, r) (l + prandom_u32_max(r - l)) - /* * 't->id' should be the pointer to the relevant 'struct pid' having reference * count. Caller must put the returned task, unless it is NULL. @@ -370,71 +362,6 @@ void damon_va_update_regions(struct damon_ctx *ctx) } } -static void damon_ptep_mkold(pte_t *pte, struct mm_struct *mm, - unsigned long addr) -{ - bool referenced = false; - struct page *page = pte_page(*pte); - - if (pte_young(*pte)) { - referenced = true; - *pte = pte_mkold(*pte); - } - -#ifdef CONFIG_MMU_NOTIFIER - if (mmu_notifier_clear_young(mm, addr, addr + PAGE_SIZE)) - referenced = true; -#endif /* CONFIG_MMU_NOTIFIER */ - - if (referenced) - set_page_young(page); - - set_page_idle(page); -} - -static void damon_pmdp_mkold(pmd_t *pmd, struct mm_struct *mm, - unsigned long addr) -{ -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - bool referenced = false; - struct page *page = pmd_page(*pmd); - - if (pmd_young(*pmd)) { - referenced = true; - *pmd = pmd_mkold(*pmd); - } - -#ifdef CONFIG_MMU_NOTIFIER - if (mmu_notifier_clear_young(mm, addr, - addr + ((1UL) << HPAGE_PMD_SHIFT))) - referenced = true; -#endif /* CONFIG_MMU_NOTIFIER */ - - if (referenced) - set_page_young(page); - - set_page_idle(page); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ -} - -static void damon_va_mkold(struct mm_struct *mm, unsigned long addr) -{ - pte_t *pte = NULL; - pmd_t *pmd = NULL; - spinlock_t *ptl; - - if (follow_pte_pmd(mm, addr, NULL, &pte, &pmd, &ptl)) - return; - - if (pte) { - damon_ptep_mkold(pte, mm, addr); - pte_unmap_unlock(pte, ptl); - } else { - damon_pmdp_mkold(pmd, mm, addr); - spin_unlock(ptl); - } -} - /* * Functions for the access checking of the regions */ @@ -463,37 +390,6 @@ void damon_va_prepare_access_checks(struct damon_ctx *ctx) } } -static bool damon_va_young(struct mm_struct *mm, unsigned long addr, - unsigned long *page_sz) -{ - pte_t *pte = NULL; - pmd_t *pmd = NULL; - spinlock_t *ptl; - bool young = false; - - if (follow_pte_pmd(mm, addr, NULL, &pte, &pmd, &ptl)) - return false; - - *page_sz = PAGE_SIZE; - if (pte) { - young = pte_young(*pte); - if (!young) - young = !page_is_idle(pte_page(*pte)); - pte_unmap_unlock(pte, ptl); - return young; - } - -#ifdef CONFIG_TRANSPARENT_HUGEPAGE - young = pmd_young(*pmd); - if (!young) - young = !page_is_idle(pmd_page(*pmd)); - spin_unlock(ptl); - *page_sz = ((1UL) << HPAGE_PMD_SHIFT); -#endif /* CONFIG_TRANSPARENT_HUGEPAGE */ - - return young; -} - /* * Check whether the region was accessed after the last preparation *