From patchwork Sun May 8 08:58:52 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 12842276 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B739AC433EF for ; Sun, 8 May 2022 09:01:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=UqjHjesQ9xXQn5FTMX0he7Nk2JHaJAJETS0Svmtx+QU=; b=n9WtAGnuNt2F75 YPXmHjWPv/vFlbxkTMGbmMRZVjtN4tsDUg0Y4C/83kCp2r3h67oz5PBRVMN4kJrJKyAMkYCddtwY7 mtXr0r2tqV24sbMt6EPpqjiRgcFhS6v8GlgCA5ZbblrUOUai9ClOBXzmroHgv6DO+v47lCR8DSU0O ox8Bo5xeRstwMtoA2KfHiDnGJwfXNgR06CbVrtvfwIZcBqF6IHBQTPENXcCMVglODZNbkzc76Ljc0 HlqXJdn5YqGc8ADvD1noLc/QfWXQkKxezcRyyRo/wPLDJiqo3r4z6VdMnKp1pDFVSZfGhKyQEBW3Q TqNIjMa2D/oOlUwMY0YQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nnclI-009YWy-Ml; Sun, 08 May 2022 08:59:32 +0000 Received: from out30-45.freemail.mail.aliyun.com ([115.124.30.45]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nncky-009YMT-OV for linux-arm-kernel@lists.infradead.org; Sun, 08 May 2022 08:59:16 +0000 X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R201e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=e01e04394; MF=baolin.wang@linux.alibaba.com; NM=1; PH=DS; RN=12; SR=0; TI=SMTPD_---0VCZrFAl_1652000344; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VCZrFAl_1652000344) by smtp.aliyun-inc.com(127.0.0.1); Sun, 08 May 2022 16:59:05 +0800 From: Baolin Wang To: catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, mike.kravetz@oracle.com, akpm@linux-foundation.org, sj@kernel.org Cc: baolin.wang@linux.alibaba.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 1/3] arm64/hugetlb: Introduce new huge_ptep_get_access_flags() interface Date: Sun, 8 May 2022 16:58:52 +0800 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220508_015913_048002_A3B18C38 X-CRM114-Status: GOOD ( 13.65 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Now we use huge_ptep_get() to get the pte value of a hugetlb page, however it will only return one specific pte value for the CONT-PTE or CONT-PMD size hugetlb on ARM64 system, which can contain seravel continuous pte or pmd entries with same page table attributes. And it will not take into account the subpages' dirty or young bits of a CONT-PTE/PMD size hugetlb page. So the huge_ptep_get() is inconsistent with huge_ptep_get_and_clear(), which already takes account the dirty or young bits for any subpages in this CONT-PTE/PMD size hugetlb [1]. Meanwhile we can miss dirty or young flags statistics for hugetlb pages with current huge_ptep_get(), such as the gather_hugetlb_stats() function. Thus introduce a new huge_ptep_get_access_flags() interface and define an ARM64 specific implementation, that will take into account any subpages' dirty or young bits for CONT-PTE/PMD size hugetlb page, for those functions that want to check the dirty and young flags of a hugetlb page. [1] https://lore.kernel.org/linux-mm/85bd80b4-b4fd-0d3f-a2e5-149559f2f387@oracle.com/ Signed-off-by: Baolin Wang --- arch/arm64/include/asm/hugetlb.h | 2 ++ arch/arm64/mm/hugetlbpage.c | 24 ++++++++++++++++++++++++ include/asm-generic/hugetlb.h | 7 +++++++ 3 files changed, 33 insertions(+) diff --git a/arch/arm64/include/asm/hugetlb.h b/arch/arm64/include/asm/hugetlb.h index 616b2ca..a473544 100644 --- a/arch/arm64/include/asm/hugetlb.h +++ b/arch/arm64/include/asm/hugetlb.h @@ -44,6 +44,8 @@ extern pte_t huge_ptep_clear_flush(struct vm_area_struct *vma, #define __HAVE_ARCH_HUGE_PTE_CLEAR extern void huge_pte_clear(struct mm_struct *mm, unsigned long addr, pte_t *ptep, unsigned long sz); +#define __HAVE_ARCH_HUGE_PTEP_GET_ACCESS_FLAGS +extern pte_t huge_ptep_get_access_flags(pte_t *ptep, unsigned long sz); extern void set_huge_swap_pte_at(struct mm_struct *mm, unsigned long addr, pte_t *ptep, pte_t pte, unsigned long sz); #define set_huge_swap_pte_at set_huge_swap_pte_at diff --git a/arch/arm64/mm/hugetlbpage.c b/arch/arm64/mm/hugetlbpage.c index ca8e65c..ce39699 100644 --- a/arch/arm64/mm/hugetlbpage.c +++ b/arch/arm64/mm/hugetlbpage.c @@ -158,6 +158,30 @@ static inline int num_contig_ptes(unsigned long size, size_t *pgsize) return contig_ptes; } +pte_t huge_ptep_get_access_flags(pte_t *ptep, unsigned long sz) +{ + int ncontig, i; + size_t pgsize; + pte_t orig_pte = ptep_get(ptep); + + if (!pte_cont(orig_pte)) + return orig_pte; + + ncontig = num_contig_ptes(sz, &pgsize); + + for (i = 0; i < ncontig; i++, ptep++) { + pte_t pte = ptep_get(ptep); + + if (pte_dirty(pte)) + orig_pte = pte_mkdirty(orig_pte); + + if (pte_young(pte)) + orig_pte = pte_mkyoung(orig_pte); + } + + return orig_pte; +} + /* * Changing some bits of contiguous entries requires us to follow a * Break-Before-Make approach, breaking the whole contiguous set diff --git a/include/asm-generic/hugetlb.h b/include/asm-generic/hugetlb.h index a57d667..bb77fb0 100644 --- a/include/asm-generic/hugetlb.h +++ b/include/asm-generic/hugetlb.h @@ -150,6 +150,13 @@ static inline pte_t huge_ptep_get(pte_t *ptep) } #endif +#ifndef __HAVE_ARCH_HUGE_PTEP_GET_ACCESS_FLAGS +static inline pte_t huge_ptep_get_access_flags(pte_t *ptep, unsigned long sz) +{ + return ptep_get(ptep); +} +#endif + #ifndef __HAVE_ARCH_GIGANTIC_PAGE_RUNTIME_SUPPORTED static inline bool gigantic_page_runtime_supported(void) { From patchwork Sun May 8 08:58:53 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 12842273 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7DBACC433EF for ; Sun, 8 May 2022 09:00:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=zJtUe98CHOnFAYRDlWRES9QZvvZBSMsZOSonLnpSV5k=; b=Rv3znYnn5k++yq G18wqRHLJe7XVbfgnLqd9N2zrdb7CCUnrj0YTRRgPeVcd8dd03CF8sGwOoUyiHA+brmirbajzn/3S VJaeeyBV+bGoryHeK7kCiuwoyq7KNonL5a2NNwybotUC1ivvYxh9wLIbs9ydYldC6HQMOo9IV/C5w Imo/cHDQSg9MB6hjMm86SiBnMveVxKJk3IoTMi5hnI5K5yOnCRDesF6xaoXu0vDESmlrz9HOo+A2+ KoUlYZyHWg+4vAZxuf89SVJLSNGrpYjqYJhI8ZT3Jj8SHZzpioZTDTxENYuz3WinBEBP0emI1uxjI GS2XHTKn8URTfrho+Bsg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nnclQ-009YcB-UQ; Sun, 08 May 2022 08:59:41 +0000 Received: from out30-131.freemail.mail.aliyun.com ([115.124.30.131]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nncl0-009YMV-Hv for linux-arm-kernel@lists.infradead.org; Sun, 08 May 2022 08:59:17 +0000 X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R121e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=e01e01424; MF=baolin.wang@linux.alibaba.com; NM=1; PH=DS; RN=12; SR=0; TI=SMTPD_---0VCZrFB4_1652000345; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VCZrFB4_1652000345) by smtp.aliyun-inc.com(127.0.0.1); Sun, 08 May 2022 16:59:06 +0800 From: Baolin Wang To: catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, mike.kravetz@oracle.com, akpm@linux-foundation.org, sj@kernel.org Cc: baolin.wang@linux.alibaba.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 2/3] fs/proc/task_mmu: Change to use huge_ptep_get_access_flags() Date: Sun, 8 May 2022 16:58:53 +0800 Message-Id: <62de656111dcdb8d189698316f1c2721753d7c7a.1651998586.git.baolin.wang@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220508_015914_809300_B1625A33 X-CRM114-Status: GOOD ( 10.18 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The ARM64 platform can support CONT-PTE/PMD size hugetlb, which can contain seravel continuous pte or pmd entries. However current huge_ptep_get() only return one specific pte value for the CONT-PTE or CONT-PMD size hugetlb, which did not take into accounts the subpages' dirty or young flags. So the gather_hugetlb_stats() will miss some dirty hugetlb statistics. Thus change to use huge_ptep_get_access_flags() taking into accounts the subpages' dirty or young flags of a CONT-PTE/PMD size hugetlb, to make the hugetlb statistics more accurate. Signed-off-by: Baolin Wang --- fs/proc/task_mmu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c index f9c9abb..3f224a7 100644 --- a/fs/proc/task_mmu.c +++ b/fs/proc/task_mmu.c @@ -1880,7 +1880,8 @@ static int gather_pte_stats(pmd_t *pmd, unsigned long addr, static int gather_hugetlb_stats(pte_t *pte, unsigned long hmask, unsigned long addr, unsigned long end, struct mm_walk *walk) { - pte_t huge_pte = huge_ptep_get(pte); + pte_t huge_pte = huge_ptep_get_access_flags(pte, + huge_page_size(hstate_vma(walk->vma))); struct numa_maps *md; struct page *page; From patchwork Sun May 8 08:58:54 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Baolin Wang X-Patchwork-Id: 12842275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A343BC433F5 for ; Sun, 8 May 2022 09:00:56 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RzvhQouLr2f1qVeoHAjaZW1po9hMpoXwOsJ13EDbthk=; b=Gr6Pn54zixVV0K T09gJNL4Qf8CgEmRiWAUNWEw6WOejm9VI2PzjOs1rnMNgvjOblK/9a6v+QXS2qVC7vuVJyrvLDLfS 3qToR8G91tTRDt18hj4cIMguR2RwSXIKofl7gHvte7KwDfOiwY/oH7h3QSH80zD9ejzyKMQhYdzPr pNuC3WXXl8oCnLluzoQhZTsrWS1fxjIws6LyIyk5dUC7gh1acMeFSLipVUZrtnlBDQRtt61wbEQG5 D60t2NvUGNn1ElFetpmSXpFk15CYCmd+mRmkNkfSTpMqfV8pU1cskrEZxLCi3+wObmQ5nL9XTdxK8 ThabA91WUmpeCy79nnhw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nnclA-009YSk-AO; Sun, 08 May 2022 08:59:24 +0000 Received: from out30-130.freemail.mail.aliyun.com ([115.124.30.130]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nncl0-009YMU-06 for linux-arm-kernel@lists.infradead.org; Sun, 08 May 2022 08:59:16 +0000 X-Alimail-AntiSpam: AC=PASS; BC=-1|-1; BR=01201311R461e4; CH=green; DM=||false|; DS=||; FP=0|-1|-1|-1|0|-1|-1|-1; HT=e01e04395; MF=baolin.wang@linux.alibaba.com; NM=1; PH=DS; RN=12; SR=0; TI=SMTPD_---0VCZrFBZ_1652000346; Received: from localhost(mailfrom:baolin.wang@linux.alibaba.com fp:SMTPD_---0VCZrFBZ_1652000346) by smtp.aliyun-inc.com(127.0.0.1); Sun, 08 May 2022 16:59:07 +0800 From: Baolin Wang To: catalin.marinas@arm.com, will@kernel.org, arnd@arndb.de, mike.kravetz@oracle.com, akpm@linux-foundation.org, sj@kernel.org Cc: baolin.wang@linux.alibaba.com, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-mm@kvack.org Subject: [RFC PATCH 3/3] mm/damon/vaddr: Change to use huge_ptep_get_access_flags() Date: Sun, 8 May 2022 16:58:54 +0800 Message-Id: X-Mailer: git-send-email 1.8.3.1 In-Reply-To: References: In-Reply-To: References: X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220508_015914_251781_B4A15CBE X-CRM114-Status: GOOD ( 11.38 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org The ARM64 platform can support CONT-PTE/PMD size hugetlb, which can contain seravel continuous pte or pmd entries. However current huge_ptep_get() only return one specific pte value for the CONT-PTE or CONT-PMD size hugetlb, which did not take into accounts the subpages' dirty or young flags. That will make the hugetlb pages monitoring inaccurate with missing young flags. Thus change to use huge_ptep_get_access_flags() taking into accounts the subpages' dirty or young flags of a CONT-PTE/PMD size hugetlb. Signed-off-by: Baolin Wang --- mm/damon/vaddr.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index d6abf76..29459ed 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -400,7 +400,8 @@ static void damon_hugetlb_mkold(pte_t *pte, struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr) { bool referenced = false; - pte_t entry = huge_ptep_get(pte); + pte_t entry = huge_ptep_get_access_flags(pte, + huge_page_size(hstate_vma(vma))); struct page *page = pte_page(entry); get_page(page); @@ -557,7 +558,7 @@ static int damon_young_hugetlb_entry(pte_t *pte, unsigned long hmask, pte_t entry; ptl = huge_pte_lock(h, walk->mm, pte); - entry = huge_ptep_get(pte); + entry = huge_ptep_get_access_flags(pte, huge_page_size(h)); if (!pte_present(entry)) goto out;