From patchwork Mon Mar 13 12:45:24 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yin Fengwei X-Patchwork-Id: 13172442 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id CAAA9C6FD19 for ; Mon, 13 Mar 2023 12:44:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 67AEA6B0078; Mon, 13 Mar 2023 08:44:44 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 62B076B0075; Mon, 13 Mar 2023 08:44:44 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 51A506B0078; Mon, 13 Mar 2023 08:44:44 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 417C26B0071 for ; Mon, 13 Mar 2023 08:44:44 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 1900F160A78 for ; Mon, 13 Mar 2023 12:44:44 +0000 (UTC) X-FDA: 80563844088.16.3828BBA Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf09.hostedemail.com (Postfix) with ESMTP id D70E4140028 for ; Mon, 13 Mar 2023 12:44:41 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ZtFxWWIU; spf=pass (imf09.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1678711482; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=aBgyHS9VdZiCBplye4znKQcf2Ia4igUspSMdoM6iekU=; b=kdKK/8qkNb6LqmaW0/+BIdHg52h1XxLtQK5tsZHLMYXiAcLaGA033j9muKZym19M0gfO+G Tp6SQ4VKVfnpxjX75ijOFqZQ2/FzMn2hQyICKFmsBwn3sy82I4wVq5jlkzMZaJu8F3F0JA ec4TSxiT0duJZrgiCBGrj8IdnTN6rEs= ARC-Authentication-Results: i=1; imf09.hostedemail.com; dkim=pass header.d=intel.com header.s=Intel header.b=ZtFxWWIU; spf=pass (imf09.hostedemail.com: domain of fengwei.yin@intel.com designates 192.55.52.43 as permitted sender) smtp.mailfrom=fengwei.yin@intel.com; dmarc=pass (policy=none) header.from=intel.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1678711482; a=rsa-sha256; cv=none; b=GlTLOWmjHeQXp5UnHBB+GwXh4AmbnHqH/IahbEtOTQfVmGPBZnQL6sH28506jFZOq8s7Ys MTeHcoJuLlOy/chzOHJURHfWCrpLsz+9bHoE4l2i44yngCI40QsZVh9/cAf2f13y6mV3SR a2eXCwN38rdD3nXyl8GpGS9GER0kD/w= DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1678711481; x=1710247481; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=lUNfaXC0kDCBuJ3XPQm6R4hq0DCdot7KNdmyPQuE5BY=; b=ZtFxWWIUd46HeokxifxnVFMj+kShFJ6DZD0KESkA8vAvKqljDcT4+Amb 2pyA/TZIVV3+QGBqCnckGEYFtoB5lGOhZ/Wc00p9sNEF9YNVvhn5MQOxV UeBR67uo07zirlql26MDeBNaOP0J8DRqgHkHdzehlPExQZ8du8RLygjy0 KlpxlfYFfy/OZPMO+HEywZQw7JdcxWGI+8KFvhwHj2A4x0NBSMJrm/ncz bh//crjMoLCTssqwxEf0dx7SN2TDfbfM8BbSvOdFLpCCiQjWr70s4X0VY tHWmXyvLWBi5mQ1N8CBxE5CtYvKwEPsuC35fq5YyLm9ZfA/0UbXlMH6Vr Q==; X-IronPort-AV: E=McAfee;i="6500,9779,10647"; a="423400767" X-IronPort-AV: E=Sophos;i="5.98,256,1673942400"; d="scan'208";a="423400767" Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 13 Mar 2023 05:44:40 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6500,9779,10647"; a="747597783" X-IronPort-AV: E=Sophos;i="5.98,256,1673942400"; d="scan'208";a="747597783" Received: from fyin-dev.sh.intel.com ([10.239.159.32]) by fmsmga004.fm.intel.com with ESMTP; 13 Mar 2023 05:44:37 -0700 From: Yin Fengwei To: linux-mm@kvack.org, akpm@linux-foundation.org, willy@infradead.org, mike.kravetz@oracle.com, sidhartha.kumar@oracle.com, naoya.horiguchi@nec.com, jane.chu@oracle.com, david@redhat.com Cc: fengwei.yin@intel.com Subject: [PATCH v4 3/5] rmap: cleanup exit path of try_to_unmap_one_page() Date: Mon, 13 Mar 2023 20:45:24 +0800 Message-Id: <20230313124526.1207490-4-fengwei.yin@intel.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230313124526.1207490-1-fengwei.yin@intel.com> References: <20230313124526.1207490-1-fengwei.yin@intel.com> MIME-Version: 1.0 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: D70E4140028 X-Rspam-User: X-Stat-Signature: bj6a6rz9jxei88i86n8rdiktu8s4ai7s X-HE-Tag: 1678711481-486656 X-HE-Meta: U2FsdGVkX1+2X9OZDYW5BTcAZMnB7c9u46wTyGp0feHeWJ4GvhXdq4btvi/izmO9pUTGgMteG5Wa5N4kbz8ftA2Ssu3MuATEMAbhBflaGDCcErL2wiv1DPBlmtS2roafvheptPVtQ1OeE1WQYbgcZDRznlxYUY49CwaTlBw0Td5u9dChBQMWw7OX05nxHIhbaYlsKuLtwqUHHzbMlrYQ3C9UCJN1+A/kyCNj9vj9ia46VSLq+TdfqjYFVQJP1Zko799S5k300vE+Og90pbuQXFWiCiq1XChYyJ9nCFemP8+ji3E6RtGuvrYz3ioJ+0aEB4gz1QRxRp6np9S5ba4OF5Ef8iSi12O4lhFWbbQLQ/YMTCHpxd1NCqF5pB7dtVhzDoLXy4ASNJjRJynUkuyUwCQFT3JOqcxi1OM4YpURZaop5zzRUFKdfUI8aMeOHQWxkKfszUDrGTioTfPYtTGfbQiZhxukTCYhCNVoWqKLkyFZZQCdJHsZrW6Hiu6htcQlNbI2yhG7NdveHz6bd3VVhYjdhcpIBAZkV2Wbn1NxYzTiF+jq7UzCqXjiBjzT923xKoOj2THSQPTGCTZvyRc+Nc0LIngx71ZUQN33496BE0MREqhq2mzR3C6UOwwZFJQgsCeKaY4ccrYUgmzzV6lMdZQUBI3DuCH+472gb2ZwhfxUgnUPzs6XK4gTulA+4FW3lCR7c/8YeY45zVko7/CWlD+z4OiFWuHgn9Yf5tyJnzm1P67Nq1aVsk40wAqAFNjrx9lzGaGu92hOvNOWjgNUelOnb8rg0HI8yLrz26inIMa3sCPEMr5YThKAz0S1sivZ2k5P4pSA8TxE0CyOwdsJzx0gfpdMCIUKeI4rZAmaB7Zs2qAsCkrHmHCSkENOEu9HBGyhuceT7/HxTg46AYD+T7Y3TipQc2vcfjjwcyiuc9D27M+TvptksbAM+0mARHSVN4F5wooPV4rUraJv2qS d0mwERhJ 5awcvAyF48StgZB0qiH13yYmCDdL3oddEyBPZNJ3TCe5iX1in8rHW0i3jCrvfkU68gK/w9tIYfQIyplwpdFBAggNq2vNRmaz59H2G24iknflKxTs9+l1sbzg/2cza+D05JURMjGdZyjbDZOtd/kCJ/UcxWa0Xr20grQeFApoUo0RJpmNJTe9jCU//SyYqzphdkiOn70aOt/05n58azDXJh0H7aw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Cleanup exit path of try_to_unmap_one_page() by removing some duplicated code. Move page_vma_mapped_walk_done() back to try_to_unmap_one(). Change subpage to page as folio has no concept of subpage. Signed-off-by: Yin Fengwei --- mm/rmap.c | 72 ++++++++++++++++++++++--------------------------------- 1 file changed, 29 insertions(+), 43 deletions(-) diff --git a/mm/rmap.c b/mm/rmap.c index 23eda671447a..72fc8c559cd9 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1543,15 +1543,13 @@ static bool try_to_unmap_one_page(struct folio *folio, struct page_vma_mapped_walk pvmw, unsigned long address, enum ttu_flags flags) { - bool anon_exclusive, ret = true; - struct page *subpage; + bool anon_exclusive; + struct page *page; struct mm_struct *mm = vma->vm_mm; pte_t pteval; - subpage = folio_page(folio, - pte_pfn(*pvmw.pte) - folio_pfn(folio)); - anon_exclusive = folio_test_anon(folio) && - PageAnonExclusive(subpage); + page = folio_page(folio, pte_pfn(*pvmw.pte) - folio_pfn(folio)); + anon_exclusive = folio_test_anon(folio) && PageAnonExclusive(page); flush_cache_page(vma, address, pte_pfn(*pvmw.pte)); /* Nuke the page table entry. */ @@ -1579,15 +1577,14 @@ static bool try_to_unmap_one_page(struct folio *folio, pte_install_uffd_wp_if_needed(vma, address, pvmw.pte, pteval); /* Set the dirty flag on the folio now the pte is gone. */ - if (pte_dirty(pteval)) + if (pte_dirty(pteval) && !folio_test_dirty(folio)) folio_mark_dirty(folio); /* Update high watermark before we lower rss */ update_hiwater_rss(mm); - if (PageHWPoison(subpage) && !(flags & TTU_HWPOISON)) { - pteval = swp_entry_to_pte(make_hwpoison_entry(subpage)); - dec_mm_counter(mm, mm_counter(&folio->page)); + if (PageHWPoison(page) && !(flags & TTU_HWPOISON)) { + pteval = swp_entry_to_pte(make_hwpoison_entry(page)); set_pte_at(mm, address, pvmw.pte, pteval); } else if (pte_unused(pteval) && !userfaultfd_armed(vma)) { /* @@ -1600,12 +1597,11 @@ static bool try_to_unmap_one_page(struct folio *folio, * migration) will not expect userfaults on already * copied pages. */ - dec_mm_counter(mm, mm_counter(&folio->page)); /* We have to invalidate as we cleared the pte */ mmu_notifier_invalidate_range(mm, address, address + PAGE_SIZE); } else if (folio_test_anon(folio)) { - swp_entry_t entry = { .val = page_private(subpage) }; + swp_entry_t entry = { .val = page_private(page) }; pte_t swp_pte; /* * Store the swap location in the pte. @@ -1614,12 +1610,10 @@ static bool try_to_unmap_one_page(struct folio *folio, if (unlikely(folio_test_swapbacked(folio) != folio_test_swapcache(folio))) { WARN_ON_ONCE(1); - ret = false; /* We have to invalidate as we cleared the pte */ mmu_notifier_invalidate_range(mm, address, address + PAGE_SIZE); - page_vma_mapped_walk_done(&pvmw); - goto discard; + goto exit; } /* MADV_FREE page check */ @@ -1651,7 +1645,6 @@ static bool try_to_unmap_one_page(struct folio *folio, /* Invalidate as we cleared the pte */ mmu_notifier_invalidate_range(mm, address, address + PAGE_SIZE); - dec_mm_counter(mm, MM_ANONPAGES); goto discard; } @@ -1659,43 +1652,30 @@ static bool try_to_unmap_one_page(struct folio *folio, * If the folio was redirtied, it cannot be * discarded. Remap the page to page table. */ - set_pte_at(mm, address, pvmw.pte, pteval); folio_set_swapbacked(folio); - ret = false; - page_vma_mapped_walk_done(&pvmw); - goto discard; + goto exit_restore_pte; } - if (swap_duplicate(entry) < 0) { - set_pte_at(mm, address, pvmw.pte, pteval); - ret = false; - page_vma_mapped_walk_done(&pvmw); - goto discard; - } + if (swap_duplicate(entry) < 0) + goto exit_restore_pte; + if (arch_unmap_one(mm, vma, address, pteval) < 0) { swap_free(entry); - set_pte_at(mm, address, pvmw.pte, pteval); - ret = false; - page_vma_mapped_walk_done(&pvmw); - goto discard; + goto exit_restore_pte; } /* See page_try_share_anon_rmap(): clear PTE first. */ - if (anon_exclusive && - page_try_share_anon_rmap(subpage)) { + if (anon_exclusive && page_try_share_anon_rmap(page)) { swap_free(entry); - set_pte_at(mm, address, pvmw.pte, pteval); - ret = false; - page_vma_mapped_walk_done(&pvmw); - goto discard; + goto exit_restore_pte; } + if (list_empty(&mm->mmlist)) { spin_lock(&mmlist_lock); if (list_empty(&mm->mmlist)) list_add(&mm->mmlist, &init_mm.mmlist); spin_unlock(&mmlist_lock); } - dec_mm_counter(mm, MM_ANONPAGES); inc_mm_counter(mm, MM_SWAPENTS); swp_pte = swp_entry_to_pte(entry); if (anon_exclusive) @@ -1706,8 +1686,7 @@ static bool try_to_unmap_one_page(struct folio *folio, swp_pte = pte_swp_mkuffd_wp(swp_pte); set_pte_at(mm, address, pvmw.pte, swp_pte); /* Invalidate as we cleared the pte */ - mmu_notifier_invalidate_range(mm, address, - address + PAGE_SIZE); + mmu_notifier_invalidate_range(mm, address, address + PAGE_SIZE); } else { /* * This is a locked file-backed folio, @@ -1720,11 +1699,16 @@ static bool try_to_unmap_one_page(struct folio *folio, * * See Documentation/mm/mmu_notifier.rst */ - dec_mm_counter(mm, mm_counter_file(&folio->page)); } discard: - return ret; + dec_mm_counter(vma->vm_mm, mm_counter(&folio->page)); + return true; + +exit_restore_pte: + set_pte_at(mm, address, pvmw.pte, pteval); +exit: + return false; } /* @@ -1802,8 +1786,10 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, pte_pfn(*pvmw.pte) - folio_pfn(folio)); ret = try_to_unmap_one_page(folio, vma, range, pvmw, address, flags); - if (!ret) + if (!ret) { + page_vma_mapped_walk_done(&pvmw); break; + } /* * No need to call mmu_notifier_invalidate_range() it has be @@ -1812,7 +1798,7 @@ static bool try_to_unmap_one(struct folio *folio, struct vm_area_struct *vma, * * See Documentation/mm/mmu_notifier.rst */ - page_remove_rmap(subpage, vma, folio_test_hugetlb(folio)); + page_remove_rmap(subpage, vma, false); if (vma->vm_flags & VM_LOCKED) mlock_drain_local(); folio_put(folio);