From patchwork Thu Feb 23 08:31:58 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yin Fengwei X-Patchwork-Id: 13149973 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 42470C636D7 for ; Thu, 23 Feb 2023 08:30:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A3F056B0075; Thu, 23 Feb 2023 03:30:35 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9DB7B6B007D; Thu, 23 Feb 2023 03:30:35 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7C8466B007E; Thu, 23 Feb 2023 03:30:35 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 4E3606B0075 for ; Thu, 23 Feb 2023 03:30:35 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 26AEC1C6B6E for ; Thu, 23 Feb 2023 08:30:35 +0000 (UTC) X-FDA: 80497885230.03.D2C8F97 Received: from mga02.intel.com (mga02.intel.com [134.134.136.20]) by imf26.hostedemail.com (Postfix) with ESMTP id F2139140011 for ; Thu, 23 Feb 2023 08:30:32 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=O1+B8x3R; spf=pass (imf26.hostedemail.com: domain of fengwei.yin@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1677141033; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=scTs1tSNlZ/Q2C/i6LH4vDzWHd+TD1Qjw97tbUPFr8Q=; b=NUxAKimqxoRt4bA1tTniiPFdwuA3v5P8coS1+QDGTv/HkqhmRCwoTbuXWcS/j+p4UWTkTj J8IAl6o5XDYbdnSvGwCKgy7WRTwU9Xe/tyFroww/JhgkMWsjjAAqJI7LRH7apVkyMvszZT U69UfhqGowVL3JAo1YpjPzLWe9uHd6w= ARC-Authentication-Results: i=1; imf26.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=O1+B8x3R; spf=pass (imf26.hostedemail.com: domain of fengwei.yin@intel.com designates 134.134.136.20 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1677141033; a=rsa-sha256; cv=none; b=dJ8ufiYelAOcmMzl0ko6+IA0zu3kgLlCsLNpTCbWTrtj/pQmZ5VJ37VFZzriel5jiVxxiT mTnD2mhmN0sO1NP0c2p0cD1W/oiO9K431xDTGxKZr+vyBZNS0sYdV4mhKjgdicunptrnsb DigyvWEUuYr8OwHtuStCxoON1TIahak= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1677141033; x=1708677033; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=pn1rQHlQmuF6JxGWRGKiya8QX8Fx+EyQOThE5TbPCIc=; b=O1+B8x3RHjLwSz8/w7GBx4uQwohJ42p4/E4fae0O927sPi0rSOzr6Q8m l9965xTqx6CxcvD4eBoHn3FRW6nz6y7fzuoyeStjPK2Xly6ixUFRlSxmn +zljUPgPdB2sH+jSg9kzCtUBN6IspAFSMkRGkIC/7B1rRKCVLCOd43nA0 aOuIaeJb2YTX3sTXgsVcWIugndvTiAZQmb2fZqld3OuzYbosFS9CciE3D L1ZfOcNp133VzsfQNNH2RmFMY+mB7yL+nHLSIrz51en89k3rL0bVveTid Kj4iMqVoYpNOVGmEKCFAQmsl0cBNub5w4piRNwtz4CDNZXS18PeUyHczU w==; X-IronPort-AV: E=McAfee;i="6500,9779,10629"; a="321298492" X-IronPort-AV: E=Sophos;i="5.97,320,1669104000"; d="scan'208";a="321298492" Received: from fmsmga007.fm.intel.com ([10.253.24.52]) by orsmga101.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 Feb 2023 00:30:31 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10629"; a="674437903" X-IronPort-AV: E=Sophos;i="5.97,320,1669104000"; d="scan'208";a="674437903" Received: from fyin-dev.sh.intel.com ([10.239.159.32]) by fmsmga007.fm.intel.com with ESMTP; 23 Feb 2023 00:30:30 -0800 From: Yin Fengwei To: linux-mm@kvack.org, akpm@linux-foundation.org, willy@infradead.org Cc: fengwei.yin@intel.com Subject: [PATCH 3/5] rmap: cleanup exit path of try_to_unmap_one_page() Date: Thu, 23 Feb 2023 16:31:58 +0800 Message-Id: <20230223083200.3149015-4-fengwei.yin@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230223083200.3149015-1-fengwei.yin@intel.com> References: <20230223083200.3149015-1-fengwei.yin@intel.com> MIME-Version: 1.0 X-Stat-Signature: ya68a8fuwwqjqyfyti4nxeqbaq7o8mpx X-Rspam-User: X-Rspamd-Queue-Id: F2139140011 X-Rspamd-Server: rspam06 X-HE-Tag: 1677141032-12111 X-HE-Meta: U2FsdGVkX1/sYa1XWmNiQjcrYK2dbpxSVgJaj7FQIdLUZGF5e0aCSN+n6JcRLiRFK6hAvVSOElVN9aGg4LcgkcfaF+kTY6TYgO9ZQeLek+7upCEFCMgsnuLO6w83puPD3XU51Xh50od8AqYUIZZLBU85Si4pDoXPxeoN53gDfXqER8msOSlfheIG4NQadI3rDlx5JLqGRmVkPcXY5HGsOEkT6dyHzVCWi5p8UNxsFti3DsIEewwtcXlno77NBhM3UbL7U8BD15MUL4berQanl2eebGUkKzQgaP/lZBqprhGoYdecnQsULLO3PkKz3gSFDRHFkfY4dlUTRpJXRPOrc/ENU0WL3HB8V3ID9GY2IrUiJ9TG/d+JqUFsxq0nDIzta3GAuK0t5zRqqMmrPMqHRP9g87raxXEMNvVOJAENYuY/eAHDGj4TmVa07m1TRgwcBKNmMwcDrqvXvN/uAWq9msaVV53QOawbGZEFfI2Jog1Omr6BLVRa+QexfstL7OHbO8COjpxH/T1EOOsuGqKIXTyodzUPYg/vaQ2x8teN2tmWedkXPaY0Zi2TQtxeQu9CldIiK+XwspRguMhderIjkLM4vsp0Dzd+tNi6OZeiogLdJwMNUMczKhB0kr6s7KBan3lGerqDG0fzvHdQsS1Kiw5f0jQZrZUrjcJmitNJ+zWj94khJ5vy8VJQ/HuhxgcD/vpYu3RZFF25KSF2MQFCsWz7TjccmawIP0F4+CXeh+IEioxbt/7oa1TCJJjK89Hn6YXD0D7ooJtny2UnwoxPJGfDYNUKIOZ7FM+gFoxaa3zdZYiTpH1x7loxSShFH/9hBaSLSYYk68CON3Lwe+90hzN6oz3lez01si0HmM01Cty3AZDeVGsJ18Pjd1XIcWWPTmICYLkNMYypT5xOjgYMWp9oD61pzvIsFBq98iogaX07jzKHMkl2QJXiuOfov4uGCDqOnUooJBvh7o28cXS 4cxe6W9X 29lgIi5NAcMS7ACph0FrVcalHV3YdWBrCLlUBgYH5TS5bUWQPiPOjaJh52DILUv2QXu0rgGHAgrmbSl5ZAO+iGPTjdHlPy5ATrZgNg4ePujGARefOXDdOvgcRFY5pqToXc7RBD6owFwCO/MhK+EegkAUtcJGN322Z5My1LWAk71XS8mS/EIlybBJj0CKDa0rmkGBsVe8tcu2CQhipB3nujCwJUQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Cleanup exit path of try_to_unmap_one_page() by removing some duplicated code. Move page_vma_mapped_walk_done() back to try_to_unmap_one(). Change subpage to page as folio has no concept of subpage. Signed-off-by: Yin Fengwei --- mm/rmap.c | 74 ++++++++++++++++++++++--------------------------------- 1 file changed, 30 insertions(+), 44 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 879e90bbf6aa..097774c809a0 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1536,7 +1536,7 @@ static bool try_to_unmap_one_hugetlb(struct folio *folio, * * See Documentation/mm/mmu_notifier.rst */ - page_remove_rmap(&folio->page, vma, folio_test_hugetlb(folio)); + page_remove_rmap(&folio->page, vma, true); if (vma->vm_flags & VM_LOCKED) mlock_drain_local(); folio_put(folio); @@ -1550,15 +1550,13 @@ static bool try_to_unmap_one_page(struct folio *folio, struct page_vma_mapped_walk pvmw, unsigned long address, enum ttu_flags flags) { - bool anon_exclusive, ret = true; - struct page *subpage; + bool anon_exclusive; + struct page *page; struct mm_struct *mm = vma->vm_mm; pte_t pteval; - subpage = folio_page(folio, - pte_pfn(*pvmw.pte) - folio_pfn(folio)); - anon_exclusive = folio_test_anon(folio) && - PageAnonExclusive(subpage); + page = folio_page(folio, pte_pfn(*pvmw.pte) - folio_pfn(folio)); + anon_exclusive = folio_test_anon(folio) && PageAnonExclusive(page); flush_cache_page(vma, address, pte_pfn(*pvmw.pte)); /* Nuke the page table entry. */ @@ -1586,15 +1584,14 @@ static bool try_to_unmap_one_page(struct folio *folio, pte_install_uffd_wp_if_needed(vma, address, pvmw.pte, pteval); /* Set the dirty flag on the folio now the pte is gone. */ - if (pte_dirty(pteval)) + if (pte_dirty(pteval) && !folio_test_dirty(folio)) folio_mark_dirty(folio); /* Update high watermark before we lower rss */ update_hiwater_rss(mm); - if (PageHWPoison(subpage) && !(flags & TTU_IGNORE_HWPOISON)) { - pteval = swp_entry_to_pte(make_hwpoison_entry(subpage)); - dec_mm_counter(mm, mm_counter(&folio->page)); + if (PageHWPoison(page) && !(flags & TTU_IGNORE_HWPOISON)) { + pteval = swp_entry_to_pte(make_hwpoison_entry(page)); set_pte_at(mm, address, pvmw.pte, pteval); } else if (pte_unused(pteval) && !userfaultfd_armed(vma)) { /* @@ -1607,12 +1604,11 @@ static bool try_to_unmap_one_page(struct folio *folio, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); /* We have to invalidate as we cleared the pte */ mmu_notifier_invalidate_range(mm, address, address + PAGE_SIZE); } else if (folio_test_anon(folio)) { - swp_entry_t entry = { .val = page_private(subpage) }; + swp_entry_t entry = { .val = page_private(page) }; pte_t swp_pte; /* * Store the swap location in the pte. @@ -1621,12 +1617,10 @@ static bool try_to_unmap_one_page(struct folio *folio, if (unlikely(folio_test_swapbacked(folio) != folio_test_swapcache(folio))) { WARN_ON_ONCE(1); - ret = false; /* We have to invalidate as we cleared the pte */ mmu_notifier_invalidate_range(mm, address, address + PAGE_SIZE); - page_vma_mapped_walk_done(&pvmw); - goto discard; + goto exit; } /* MADV_FREE page check */ @@ -1658,7 +1652,6 @@ static bool try_to_unmap_one_page(struct folio *folio, /* Invalidate as we cleared the pte */ mmu_notifier_invalidate_range(mm, address, address + PAGE_SIZE); - dec_mm_counter(mm, MM_ANONPAGES); goto discard; } @@ -1666,43 +1659,30 @@ static bool try_to_unmap_one_page(struct folio *folio, * If the folio was redirtied, it cannot be * discarded. Remap the page to page table. */ - set_pte_at(mm, address, pvmw.pte, pteval); folio_set_swapbacked(folio); - ret = false; - page_vma_mapped_walk_done(&pvmw); - goto discard; + goto exit_restore_pte; } - if (swap_duplicate(entry) < 0) { - set_pte_at(mm, address, pvmw.pte, pteval); - ret = false; - page_vma_mapped_walk_done(&pvmw); - goto discard; - } + if (swap_duplicate(entry) < 0) + goto exit_restore_pte; + if (arch_unmap_one(mm, vma, address, pteval) < 0) { swap_free(entry); - set_pte_at(mm, address, pvmw.pte, pteval); - ret = false; - page_vma_mapped_walk_done(&pvmw); - goto discard; + goto exit_restore_pte; } /* See page_try_share_anon_rmap(): clear PTE first. */ - if (anon_exclusive && - page_try_share_anon_rmap(subpage)) { + if (anon_exclusive && page_try_share_anon_rmap(page)) { swap_free(entry); - set_pte_at(mm, address, pvmw.pte, pteval); - ret = false; - page_vma_mapped_walk_done(&pvmw); - goto discard; + goto exit_restore_pte; } + if (list_empty(&mm->mmlist)) { spin_lock(&mmlist_lock); if (list_empty(&mm->mmlist)) list_add(&mm->mmlist, &init_mm.mmlist); spin_unlock(&mmlist_lock); } - dec_mm_counter(mm, MM_ANONPAGES); inc_mm_counter(mm, MM_SWAPENTS); swp_pte = swp_entry_to_pte(entry); if (anon_exclusive) @@ -1713,8 +1693,7 @@ static bool try_to_unmap_one_page(struct folio *folio, swp_pte = pte_swp_mkuffd_wp(swp_pte); set_pte_at(mm, address, pvmw.pte, swp_pte); /* Invalidate as we cleared the pte */ - mmu_notifier_invalidate_range(mm, address, - address + PAGE_SIZE); + mmu_notifier_invalidate_range(mm, address, address + PAGE_SIZE); } else { /* * This is a locked file-backed folio, @@ -1727,11 +1706,16 @@ static bool try_to_unmap_one_page(struct folio *folio, * * See Documentation/mm/mmu_notifier.rst */ - dec_mm_counter(mm, mm_counter_file(&folio->page)); } discard: - return ret; + dec_mm_counter(vma->vm_mm, mm_counter(&folio->page)); + return true; + +exit_restore_pte: + set_pte_at(mm, address, pvmw.pte, pteval); +exit: + return false; } /* @@ -1809,8 +1793,10 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, pte_pfn(*pvmw.pte) - folio_pfn(folio)); ret = try_to_unmap_one_page(folio, vma, range, pvmw, address, flags); - if (!ret) + if (!ret) { + page_vma_mapped_walk_done(&pvmw); break; + } /* * No need to call mmu_notifier_invalidate_range() it has be @@ -1819,7 +1805,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, * * See Documentation/mm/mmu_notifier.rst */ - page_remove_rmap(subpage, vma, folio_test_hugetlb(folio)); + page_remove_rmap(subpage, vma, false); if (vma->vm_flags & VM_LOCKED) mlock_drain_local(); folio_put(folio);