From patchwork Mon May 22 04:51:00 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Hugh Dickins X-Patchwork-Id: 13249752 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 11EA2C7EE23 for ; Mon, 22 May 2023 04:51:08 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A2894900005; Mon, 22 May 2023 00:51:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9D964900002; Mon, 22 May 2023 00:51:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8A16C900005; Mon, 22 May 2023 00:51:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 79E47900002 for ; Mon, 22 May 2023 00:51:07 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 420FA4043D for ; Mon, 22 May 2023 04:51:07 +0000 (UTC) X-FDA: 80816666574.16.3501D4E Received: from mail-yb1-f174.google.com (mail-yb1-f174.google.com [209.85.219.174]) by imf07.hostedemail.com (Postfix) with ESMTP id 5A8A44000C for ; Mon, 22 May 2023 04:51:05 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="suZ7/fWy"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of hughd@google.com designates 209.85.219.174 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1684731065; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ANcMmc9BeTTDRJseGPHLnYq8DMAivc7OJ7zKS5AkOEU=; b=367zwI9YXnirYuV6xh70NEscmeezPHUwhMFRM6jMdN8KEyxZAarGifAPPj4zH/J+ub4Ful uNJE6F8ZiYnOhdXUqDTnrK/n+m/0FyTp6mw76Una36RWwIbfhWlAlJ+Jkyz2E7xQ2hhgOE pXvCKpmXYxprzWyRQFfB8K4AV/lmPd0= ARC-Authentication-Results: i=1; imf07.hostedemail.com; dkim=pass header.d=google.com header.s=20221208 header.b="suZ7/fWy"; dmarc=pass (policy=reject) header.from=google.com; spf=pass (imf07.hostedemail.com: domain of hughd@google.com designates 209.85.219.174 as permitted sender) smtp.mailfrom=hughd@google.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1684731065; a=rsa-sha256; cv=none; b=ocl5QYDeioc9ftkcMMYuUZg0oFggmsjW9+DgcyGzf3bDQl3ePnfbqWijEHaTZZT6unfW6E WsygjmnqC+ZwSPgvayoueBOLwak+woUhfgW0e7YfGFo28En6nbUeAgk3ut0UORPwXnGKB4 zcSZxBArQI2mkpWRFJBk/ScB+VHBh94= Received: by mail-yb1-f174.google.com with SMTP id 3f1490d57ef6-ba8afcc82c0so8138705276.2 for ; Sun, 21 May 2023 21:51:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20221208; t=1684731064; x=1687323064; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to; bh=ANcMmc9BeTTDRJseGPHLnYq8DMAivc7OJ7zKS5AkOEU=; b=suZ7/fWyjgtDmmUUG368PhOQSIoqVmOnxBEGz1k9zCkGOxssorwZrFR60X+8a4DQqr 7u71fp8c7x68EiyyW8ZSDwzHkHiPONOn7yDOgmy63jiQBTyUNcqNciC9w+0U347zF4Gf qZEb5FQPLn1+cE60AE8Q2hbQucuksdSO++m5CDyyBTN/iGnKQo4gRrU//els8iIiBdS7 ovYLm7Yj3qMDE0Qoy1tWZGPaThqr4eXUwvzaDxInjjwEGGfg2sUxlTqNWk6AaZWuz+i8 FTUjVygqWWxTs+smOosRSftvUI/03MEA0EZ3L2DHmfVjXa6bRJ1mwd0Lc0WFnRQmYQLX GTlA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684731064; x=1687323064; h=mime-version:references:message-id:in-reply-to:subject:cc:to:from :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ANcMmc9BeTTDRJseGPHLnYq8DMAivc7OJ7zKS5AkOEU=; b=Cl8s/txM+yn7DYmkGxSs/7t+BpEoi6uww1ak7B8SN7r0vb0cUEK/jcrOBIGGl/bAwd zFK1pHGCB3/mcpOZcczIxDVERE0hMenrW4ZNoVJpQrM6WWoRkLy2cCMZjKenydSljZ1a fGBnfvmpD53gO1pBo48bLeoJSeWZhJwAinSgesThmScDy55e4S8hHMcsN+ZUY+Hd60RK muEos2kfpka/uxHJBdl7su7qMGvYOrGLI2xF6tdaSpoKCnJg6fvKdJdrCm3OFCNeo51w 1v3P75CwHHfJMWoS5qwwAOXOZXuh7brdIuK33iGwJeRCQDv4hMa5v2ZX2nhICiMoDGTd HQfw== X-Gm-Message-State: AC+VfDwRvPk7jq6JJV6WnmG/9EQ0tCD8UUjMHfQxPDPjZk8UkEw6xeq1 oeEBbgdUJO3AbEL9swRp7cqH/A== X-Google-Smtp-Source: ACHHUZ6jO0XEKla2wlyWHkLbd6267BBIAu3WYemOi2dKQNM0AlHnDzPyGpw7fyb0X8v56kZQVrFU2A== X-Received: by 2002:a25:1042:0:b0:ba8:1807:9d7f with SMTP id 63-20020a251042000000b00ba818079d7fmr9116239ybq.58.1684731064262; Sun, 21 May 2023 21:51:04 -0700 (PDT) Received: from ripple.attlocal.net (172-10-233-147.lightspeed.sntcca.sbcglobal.net. [172.10.233.147]) by smtp.gmail.com with ESMTPSA id a11-20020a81bb4b000000b0054e82b6a95esm1808098ywl.42.2023.05.21.21.51.01 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 21 May 2023 21:51:04 -0700 (PDT) Date: Sun, 21 May 2023 21:51:00 -0700 (PDT) From: Hugh Dickins X-X-Sender: hugh@ripple.attlocal.net To: Andrew Morton cc: Mike Kravetz , Mike Rapoport , "Kirill A. Shutemov" , Matthew Wilcox , David Hildenbrand , Suren Baghdasaryan , Qi Zheng , Yang Shi , Mel Gorman , Peter Xu , Peter Zijlstra , Will Deacon , Yu Zhao , Alistair Popple , Ralph Campbell , Ira Weiny , Steven Price , SeongJae Park , Naoya Horiguchi , Christophe Leroy , Zack Rusin , Jason Gunthorpe , Axel Rasmussen , Anshuman Khandual , Pasha Tatashin , Miaohe Lin , Minchan Kim , Christoph Hellwig , Song Liu , Thomas Hellstrom , linux-kernel@vger.kernel.org, linux-mm@kvack.org Subject: [PATCH 02/31] mm/migrate: remove cruft from migration_entry_wait()s In-Reply-To: <68a97fbe-5c1e-7ac6-72c-7b9c6290b370@google.com> Message-ID: <1659568-468a-6d36-c26-6a52a335ab59@google.com> References: <68a97fbe-5c1e-7ac6-72c-7b9c6290b370@google.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 5A8A44000C X-Rspam-User: X-Rspamd-Server: rspam05 X-Stat-Signature: j41h9ymcrtyih686in1umnb9mjjfh4qz X-HE-Tag: 1684731065-645011 X-HE-Meta: U2FsdGVkX1+S7AYFX9vBtFbDkQMvZ9lyNJ2IQxxIdUNGlEQbkwZOlf2ps2Czj8xb9BT1xWHRnt1pxTZ0+ez0JZSuHBLVBq0cQNC8702Tjl3EHJUEs/IdwW7lN97kO4r6t08HcXGsnC2m/GnsCUm/o/O6EPjwj1ROrz/N0ztjIloACy7IH8bLRCXO0JHYRBRFsjSQJVimGbQmv6Rd0jBMwIdsKyFWkoNEmy3CAtYOf95R+HZNXb/8cxOybgynbM5+S0h8oVIWxFDWeAqPVxgmL3jdNtq0K+BqIRR3TZor98d5wywDutoawD+XlzGB+67zMTI19da0wmGeaDbX+DDf6EpO66yDFPPkbLkgu91G8VE6YDVbSXLAonjb0dX2CffkqztFdZrsIDOLZFhRNTVz6VcsMmmgZU5TvVm9RT34CQ+8Gvkyg2igWNKwD1MutAcPi5BaMyPyGWff2JpeziRcTN3pFuM/nD4tCgWun2U+sd9zOP4N7UtEC7VvIrJFrxdEiM0GwY3F4qntpHBR8Oi/40P/LuzID/3xiI0xwC+E6ZLHhusQZM976XU5gPsFKM0QkkqJh6TT9GBGT38yyGNyz/JyxL7i1/DJmumCrHMeIBQkp9qJUUUy7sg2zA7/eW+ZM4zWM/cS1a/HHmXxrAaei+7rIKWJv2ZUHncaUfgOLL1GiXh8qkyo6gK4Rc45NiKKFrg3jv4eNGKRC0ljlWAT/a9yMcuiMLeGP9yB6wrygi0HqZK5X09RdrIonE9SCKho116ZS5Nh+mJ3NG9+3LlQ8OFMYzVxx8MeroQ7ETwnRpj6bg6m/EfRtAQBc6pUivJ+W9u0mKdyv16avbp6W9BfWLQzrZYiXlSQr+E5wJbD3S6hrc6yhqDgAFHuH+KTlANIBukro3MLANF/RS0XnuJKH2VxBgjtDXpOVRWRTZ9sHirpKMAPVJR3VQ0FtJXAeYVfTjcnl/XvB36TDKBkn9D I7ixwViA bXB7RLoIitPmW3aZKTPgT6MrDVk4gB4CUmsweHl+9W03YE4HFle5SsCLAGaPaxhvbpeUEu/aW2iwkdr6d0PO+uZ+vxX/PhY3aTzROhuVH23kdknJ2ZrFtUlzsJusHoCx5ieMTf6JAFhOatrTaChC0J6N5w6AlOUGI0NNY03f3sI5AdOVQBQxZ4VUX10xyL9Lba6f6EEuflOzfXG92a8j3TnfmRTZ6Sf3HG2arkxUmYuaj1KDdUw00+YeI8YG53LJJhXmYTiIK/lC+oNrMktSqXWiCz10rQDTi8pdK7f5uFfm7aMV7vFGoGxD7+CTW+p3bwF3765K7CimC1mLcJRcH1EBSFDaFs+ikSbknVXvRKIzSrdguchC2TZy0jKv4f62is+8UcEm7xZy3OzAhrxs7JQWbWiU50qWDB1d3skr/XPiTdXdBPBM61wGxqg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: migration_entry_wait_on_locked() does not need to take a mapped pte pointer, its callers can do the unmap first. Annotate it with __releases(ptl) to reduce sparse warnings. Fold __migration_entry_wait_huge() into migration_entry_wait_huge(). Fold __migration_entry_wait() into migration_entry_wait(), preferring the tighter pte_offset_map_lock() to pte_offset_map() and pte_lockptr(). Signed-off-by: Hugh Dickins Reviewed-by: Alistair Popple --- include/linux/migrate.h | 4 ++-- include/linux/swapops.h | 17 +++-------------- mm/filemap.c | 13 ++++--------- mm/migrate.c | 37 +++++++++++++------------------------ 4 files changed, 22 insertions(+), 49 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 6241a1596a75..affea3063473 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -75,8 +75,8 @@ bool isolate_movable_page(struct page *page, isolate_mode_t mode); int migrate_huge_page_move_mapping(struct address_space *mapping, struct folio *dst, struct folio *src); -void migration_entry_wait_on_locked(swp_entry_t entry, pte_t *ptep, - spinlock_t *ptl); +void migration_entry_wait_on_locked(swp_entry_t entry, spinlock_t *ptl) + __releases(ptl); void folio_migrate_flags(struct folio *newfolio, struct folio *folio); void folio_migrate_copy(struct folio *newfolio, struct folio *folio); int folio_migrate_mapping(struct address_space *mapping, diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 3a451b7afcb3..4c932cb45e0b 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -332,15 +332,9 @@ static inline bool is_migration_entry_dirty(swp_entry_t entry) return false; } -extern void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, - spinlock_t *ptl); extern void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address); -#ifdef CONFIG_HUGETLB_PAGE -extern void __migration_entry_wait_huge(struct vm_area_struct *vma, - pte_t *ptep, spinlock_t *ptl); extern void migration_entry_wait_huge(struct vm_area_struct *vma, pte_t *pte); -#endif /* CONFIG_HUGETLB_PAGE */ #else /* CONFIG_MIGRATION */ static inline swp_entry_t make_readable_migration_entry(pgoff_t offset) { @@ -362,15 +356,10 @@ static inline int is_migration_entry(swp_entry_t swp) return 0; } -static inline void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, - spinlock_t *ptl) { } static inline void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, - unsigned long address) { } -#ifdef CONFIG_HUGETLB_PAGE -static inline void __migration_entry_wait_huge(struct vm_area_struct *vma, - pte_t *ptep, spinlock_t *ptl) { } -static inline void migration_entry_wait_huge(struct vm_area_struct *vma, pte_t *pte) { } -#endif /* CONFIG_HUGETLB_PAGE */ + unsigned long address) { } +static inline void migration_entry_wait_huge(struct vm_area_struct *vma, + pte_t *pte) { } static inline int is_writable_migration_entry(swp_entry_t entry) { return 0; diff --git a/mm/filemap.c b/mm/filemap.c index b4c9bd368b7e..28b42ee848a4 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1359,8 +1359,6 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, /** * migration_entry_wait_on_locked - Wait for a migration entry to be removed * @entry: migration swap entry. - * @ptep: mapped pte pointer. Will return with the ptep unmapped. Only required - * for pte entries, pass NULL for pmd entries. * @ptl: already locked ptl. This function will drop the lock. * * Wait for a migration entry referencing the given page to be removed. This is @@ -1369,13 +1367,13 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, * should be called while holding the ptl for the migration entry referencing * the page. * - * Returns after unmapping and unlocking the pte/ptl with pte_unmap_unlock(). + * Returns after unlocking the ptl. * * This follows the same logic as folio_wait_bit_common() so see the comments * there. */ -void migration_entry_wait_on_locked(swp_entry_t entry, pte_t *ptep, - spinlock_t *ptl) +void migration_entry_wait_on_locked(swp_entry_t entry, spinlock_t *ptl) + __releases(ptl) { struct wait_page_queue wait_page; wait_queue_entry_t *wait = &wait_page.wait; @@ -1409,10 +1407,7 @@ void migration_entry_wait_on_locked(swp_entry_t entry, pte_t *ptep, * a valid reference to the page, and it must take the ptl to remove the * migration entry. So the page is valid until the ptl is dropped. */ - if (ptep) - pte_unmap_unlock(ptep, ptl); - else - spin_unlock(ptl); + spin_unlock(ptl); for (;;) { unsigned int flags; diff --git a/mm/migrate.c b/mm/migrate.c index 01cac26a3127..3ecb7a40075f 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -296,14 +296,18 @@ void remove_migration_ptes(struct folio *src, struct folio *dst, bool locked) * get to the page and wait until migration is finished. * When we return from this function the fault will be retried. */ -void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, - spinlock_t *ptl) +void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, + unsigned long address) { + spinlock_t *ptl; + pte_t *ptep; pte_t pte; swp_entry_t entry; - spin_lock(ptl); + ptep = pte_offset_map_lock(mm, pmd, address, &ptl); pte = *ptep; + pte_unmap(ptep); + if (!is_swap_pte(pte)) goto out; @@ -311,18 +315,10 @@ void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, if (!is_migration_entry(entry)) goto out; - migration_entry_wait_on_locked(entry, ptep, ptl); + migration_entry_wait_on_locked(entry, ptl); return; out: - pte_unmap_unlock(ptep, ptl); -} - -void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, - unsigned long address) -{ - spinlock_t *ptl = pte_lockptr(mm, pmd); - pte_t *ptep = pte_offset_map(pmd, address); - __migration_entry_wait(mm, ptep, ptl); + spin_unlock(ptl); } #ifdef CONFIG_HUGETLB_PAGE @@ -332,9 +328,9 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, * * This function will release the vma lock before returning. */ -void __migration_entry_wait_huge(struct vm_area_struct *vma, - pte_t *ptep, spinlock_t *ptl) +void migration_entry_wait_huge(struct vm_area_struct *vma, pte_t *ptep) { + spinlock_t *ptl = huge_pte_lockptr(hstate_vma(vma), vma->vm_mm, ptep); pte_t pte; hugetlb_vma_assert_locked(vma); @@ -352,16 +348,9 @@ void __migration_entry_wait_huge(struct vm_area_struct *vma, * lock release in migration_entry_wait_on_locked(). */ hugetlb_vma_unlock_read(vma); - migration_entry_wait_on_locked(pte_to_swp_entry(pte), NULL, ptl); + migration_entry_wait_on_locked(pte_to_swp_entry(pte), ptl); } } - -void migration_entry_wait_huge(struct vm_area_struct *vma, pte_t *pte) -{ - spinlock_t *ptl = huge_pte_lockptr(hstate_vma(vma), vma->vm_mm, pte); - - __migration_entry_wait_huge(vma, pte, ptl); -} #endif #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION @@ -372,7 +361,7 @@ void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd) ptl = pmd_lock(mm, pmd); if (!is_pmd_migration_entry(*pmd)) goto unlock; - migration_entry_wait_on_locked(pmd_to_swp_entry(*pmd), NULL, ptl); + migration_entry_wait_on_locked(pmd_to_swp_entry(*pmd), ptl); return; unlock: spin_unlock(ptl);