From patchwork Sat Apr 9 07:38:43 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Miaohe Lin X-Patchwork-Id: 12807725 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3A02C433F5 for ; Sat, 9 Apr 2022 07:39:03 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3F7E68D0001; Sat, 9 Apr 2022 03:39:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3A6A66B0074; Sat, 9 Apr 2022 03:39:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 295F18D0001; Sat, 9 Apr 2022 03:39:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.28]) by kanga.kvack.org (Postfix) with ESMTP id 1B4BD6B0073 for ; Sat, 9 Apr 2022 03:39:03 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id F06EC6042B for ; Sat, 9 Apr 2022 07:39:02 +0000 (UTC) X-FDA: 79336539324.07.CF1B689 Received: from szxga02-in.huawei.com (szxga02-in.huawei.com [45.249.212.188]) by imf10.hostedemail.com (Postfix) with ESMTP id 40BE3C0002 for ; Sat, 9 Apr 2022 07:39:02 +0000 (UTC) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.55]) by szxga02-in.huawei.com (SkyGuard) with ESMTP id 4Kb6P45dZyzDq5x; Sat, 9 Apr 2022 15:36:08 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Sat, 9 Apr 2022 15:38:29 +0800 From: Miaohe Lin To: CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH 1/4] mm/migration: reduce the rcu lock duration Date: Sat, 9 Apr 2022 15:38:43 +0800 Message-ID: <20220409073846.22286-2-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220409073846.22286-1-linmiaohe@huawei.com> References: <20220409073846.22286-1-linmiaohe@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected X-Stat-Signature: 77fj1ekdfk5fh9moibbqr9w5wu5ijypo Authentication-Results: imf10.hostedemail.com; dkim=none; spf=pass (imf10.hostedemail.com: domain of linmiaohe@huawei.com designates 45.249.212.188 as permitted sender) smtp.mailfrom=linmiaohe@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspam-User: X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 40BE3C0002 X-HE-Tag: 1649489942-951021 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: rcu_read_lock is required by grabbing the task refcount but it's not needed for ptrace_may_access. So we could release the rcu lock after task refcount is successfully grabbed to reduce the rcu holding time. Reviewed-by: Muchun Song Signed-off-by: Miaohe Lin Reviewed-by: Christoph Hellwig --- mm/migrate.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index a3d8c2be2631..d8aae6c75990 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1907,17 +1907,16 @@ static struct mm_struct *find_mm_struct(pid_t pid, nodemask_t *mem_nodes) return ERR_PTR(-ESRCH); } get_task_struct(task); + rcu_read_unlock(); /* * Check if this process has the right to modify the specified * process. Use the regular "ptrace_may_access()" checks. */ if (!ptrace_may_access(task, PTRACE_MODE_READ_REALCREDS)) { - rcu_read_unlock(); mm = ERR_PTR(-EPERM); goto out; } - rcu_read_unlock(); mm = ERR_PTR(security_task_movememory(task)); if (IS_ERR(mm)) From patchwork Sat Apr 9 07:38:44 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Miaohe Lin X-Patchwork-Id: 12807728 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57321C433F5 for ; Sat, 9 Apr 2022 07:55:37 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B37146B0071; Sat, 9 Apr 2022 03:55:36 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AE6F16B0073; Sat, 9 Apr 2022 03:55:36 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FC756B0074; Sat, 9 Apr 2022 03:55:36 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0041.hostedemail.com [216.40.44.41]) by kanga.kvack.org (Postfix) with ESMTP id 9153C6B0071 for ; Sat, 9 Apr 2022 03:55:36 -0400 (EDT) Received: from smtpin26.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 381ECA7C30 for ; Sat, 9 Apr 2022 07:55:36 +0000 (UTC) X-FDA: 79336581072.26.472BAB1 Received: from szxga08-in.huawei.com (szxga08-in.huawei.com [45.249.212.255]) by imf11.hostedemail.com (Postfix) with ESMTP id 2B50B40005 for ; Sat, 9 Apr 2022 07:55:34 +0000 (UTC) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.55]) by szxga08-in.huawei.com (SkyGuard) with ESMTP id 4Kb6RD69sZz1HBW1; Sat, 9 Apr 2022 15:38:00 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Sat, 9 Apr 2022 15:38:30 +0800 From: Miaohe Lin To: CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH 2/4] mm/migration: remove unneeded lock page and PageMovable check Date: Sat, 9 Apr 2022 15:38:44 +0800 Message-ID: <20220409073846.22286-3-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220409073846.22286-1-linmiaohe@huawei.com> References: <20220409073846.22286-1-linmiaohe@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Authentication-Results: imf11.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf11.hostedemail.com: domain of linmiaohe@huawei.com designates 45.249.212.255 as permitted sender) smtp.mailfrom=linmiaohe@huawei.com X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: 2B50B40005 X-Stat-Signature: nxb1raee1jyqig5iyfzmdxggxohqsbyy X-HE-Tag: 1649490934-612627 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: When non-lru movable page was freed from under us, __ClearPageMovable must have been done. Even if it's not done, ClearPageIsolated here won't hurt as page will be freed anyway. So we can thus remove unneeded lock page and PageMovable check here. Signed-off-by: Miaohe Lin Reviewed-by: Christoph Hellwig --- mm/migrate.c | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index d8aae6c75990..381963231a62 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1098,12 +1098,8 @@ static int unmap_and_move(new_page_t get_new_page, /* page was freed from under us. So we are done. */ ClearPageActive(page); ClearPageUnevictable(page); - if (unlikely(__PageMovable(page))) { - lock_page(page); - if (!PageMovable(page)) - ClearPageIsolated(page); - unlock_page(page); - } + if (unlikely(__PageMovable(page))) + ClearPageIsolated(page); goto out; } From patchwork Sat Apr 9 07:38:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Miaohe Lin X-Patchwork-Id: 12807729 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 57923C433F5 for ; Sat, 9 Apr 2022 07:56:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BAB568D0001; Sat, 9 Apr 2022 03:56:09 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B5BF56B0074; Sat, 9 Apr 2022 03:56:09 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A4BB98D0001; Sat, 9 Apr 2022 03:56:09 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (relay.hostedemail.com [64.99.140.26]) by kanga.kvack.org (Postfix) with ESMTP id 981666B0073 for ; Sat, 9 Apr 2022 03:56:09 -0400 (EDT) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay13.hostedemail.com (Postfix) with ESMTP id 665046029F for ; Sat, 9 Apr 2022 07:56:09 +0000 (UTC) X-FDA: 79336582458.19.1EF3C00 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf28.hostedemail.com (Postfix) with ESMTP id 95019C0004 for ; Sat, 9 Apr 2022 07:56:08 +0000 (UTC) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.54]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Kb6Pn6S06zgYWT; Sat, 9 Apr 2022 15:36:45 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Sat, 9 Apr 2022 15:38:31 +0800 From: Miaohe Lin To: CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH 3/4] mm/migration: return errno when isolate_huge_page failed Date: Sat, 9 Apr 2022 15:38:45 +0800 Message-ID: <20220409073846.22286-4-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220409073846.22286-1-linmiaohe@huawei.com> References: <20220409073846.22286-1-linmiaohe@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected X-Rspam-User: Authentication-Results: imf28.hostedemail.com; dkim=none; spf=pass (imf28.hostedemail.com: domain of linmiaohe@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=linmiaohe@huawei.com; dmarc=pass (policy=quarantine) header.from=huawei.com X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 95019C0004 X-Stat-Signature: 3mrga7955h5wcz5pdnwrphctek9781ia X-HE-Tag: 1649490968-375871 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: We might fail to isolate huge page due to e.g. the page is under migration which cleared HPageMigratable. So we should return -EBUSY in this case rather than always return 1 which could confuse the user. Fixes: e8db67eb0ded ("mm: migrate: move_pages() supports thp migration") Reviewed-by: Muchun Song Reviewed-by: Baolin Wang Signed-off-by: Miaohe Lin --- mm/migrate.c | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 381963231a62..044656a14ae2 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1632,10 +1632,8 @@ static int add_page_for_migration(struct mm_struct *mm, unsigned long addr, goto out_putpage; if (PageHuge(page)) { - if (PageHead(page)) { - isolate_huge_page(page, pagelist); - err = 1; - } + if (PageHead(page)) + err = isolate_huge_page(page, pagelist) ? 1 : -EBUSY; } else { struct page *head; From patchwork Sat Apr 9 07:38:46 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Miaohe Lin X-Patchwork-Id: 12807726 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8EF8C433F5 for ; Sat, 9 Apr 2022 07:39:07 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 333528D0002; Sat, 9 Apr 2022 03:39:07 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2E32D6B0074; Sat, 9 Apr 2022 03:39:07 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1ABAA8D0002; Sat, 9 Apr 2022 03:39:07 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0180.hostedemail.com [216.40.44.180]) by kanga.kvack.org (Postfix) with ESMTP id 0DE076B0073 for ; Sat, 9 Apr 2022 03:39:07 -0400 (EDT) Received: from smtpin22.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id C5EE28249980 for ; Sat, 9 Apr 2022 07:39:06 +0000 (UTC) X-FDA: 79336539492.22.883BA29 Received: from szxga01-in.huawei.com (szxga01-in.huawei.com [45.249.212.187]) by imf27.hostedemail.com (Postfix) with ESMTP id A4ACC40007 for ; Sat, 9 Apr 2022 07:39:04 +0000 (UTC) Received: from canpemm500002.china.huawei.com (unknown [172.30.72.55]) by szxga01-in.huawei.com (SkyGuard) with ESMTP id 4Kb6RG2zn8zdZj3; Sat, 9 Apr 2022 15:38:02 +0800 (CST) Received: from huawei.com (10.175.124.27) by canpemm500002.china.huawei.com (7.192.104.244) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.24; Sat, 9 Apr 2022 15:38:32 +0800 From: Miaohe Lin To: CC: , , , , , , , , , , , , , , , , , , , Subject: [PATCH 4/4] mm/migration: fix potential pte_unmap on an not mapped pte Date: Sat, 9 Apr 2022 15:38:46 +0800 Message-ID: <20220409073846.22286-5-linmiaohe@huawei.com> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20220409073846.22286-1-linmiaohe@huawei.com> References: <20220409073846.22286-1-linmiaohe@huawei.com> MIME-Version: 1.0 X-Originating-IP: [10.175.124.27] X-ClientProxiedBy: dggems705-chm.china.huawei.com (10.3.19.182) To canpemm500002.china.huawei.com (7.192.104.244) X-CFilter-Loop: Reflected Authentication-Results: imf27.hostedemail.com; dkim=none; dmarc=pass (policy=quarantine) header.from=huawei.com; spf=pass (imf27.hostedemail.com: domain of linmiaohe@huawei.com designates 45.249.212.187 as permitted sender) smtp.mailfrom=linmiaohe@huawei.com X-Rspam-User: X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: A4ACC40007 X-Stat-Signature: i49dnhfo4kgoiyc9ujbidaxq55qdsx46 X-HE-Tag: 1649489944-972165 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: __migration_entry_wait and migration_entry_wait_on_locked assume pte is always mapped from caller. But this is not the case when it's called from migration_entry_wait_huge and follow_huge_pmd. And a parameter unmap to indicate whether pte needs to be unmapped to fix this issue. Fixes: 30dad30922cc ("mm: migration: add migrate_entry_wait_huge()") Signed-off-by: Miaohe Lin Reported-by: kernel test robot --- include/linux/migrate.h | 2 +- include/linux/swapops.h | 4 ++-- mm/filemap.c | 10 +++++----- mm/hugetlb.c | 2 +- mm/migrate.c | 14 ++++++++------ 5 files changed, 17 insertions(+), 15 deletions(-) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 2707bfd43a0d..2c4de1972f99 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -41,7 +41,7 @@ extern int migrate_huge_page_move_mapping(struct address_space *mapping, extern int migrate_page_move_mapping(struct address_space *mapping, struct page *newpage, struct page *page, int extra_count); void migration_entry_wait_on_locked(swp_entry_t entry, pte_t *ptep, - spinlock_t *ptl); + spinlock_t *ptl, bool unmap); void folio_migrate_flags(struct folio *newfolio, struct folio *folio); void folio_migrate_copy(struct folio *newfolio, struct folio *folio); int folio_migrate_mapping(struct address_space *mapping, diff --git a/include/linux/swapops.h b/include/linux/swapops.h index 32d517a28969..3e6a293f88e0 100644 --- a/include/linux/swapops.h +++ b/include/linux/swapops.h @@ -231,7 +231,7 @@ static inline swp_entry_t make_writable_migration_entry(pgoff_t offset) } extern void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, - spinlock_t *ptl); + spinlock_t *ptl, bool unmap); extern void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address); extern void migration_entry_wait_huge(struct vm_area_struct *vma, @@ -258,7 +258,7 @@ static inline int is_migration_entry(swp_entry_t swp) } static inline void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, - spinlock_t *ptl) { } + spinlock_t *ptl, bool unmap) { } static inline void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, unsigned long address) { } static inline void migration_entry_wait_huge(struct vm_area_struct *vma, diff --git a/mm/filemap.c b/mm/filemap.c index 3a5ffb5587cd..02f2d920c8cf 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -1392,6 +1392,7 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, * @ptep: mapped pte pointer. Will return with the ptep unmapped. Only required * for pte entries, pass NULL for pmd entries. * @ptl: already locked ptl. This function will drop the lock. + * @unmap: indicating whether ptep need to be unmapped. * * Wait for a migration entry referencing the given page to be removed. This is * equivalent to put_and_wait_on_page_locked(page, TASK_UNINTERRUPTIBLE) except @@ -1405,7 +1406,7 @@ static inline int folio_wait_bit_common(struct folio *folio, int bit_nr, * there. */ void migration_entry_wait_on_locked(swp_entry_t entry, pte_t *ptep, - spinlock_t *ptl) + spinlock_t *ptl, bool unmap) { struct wait_page_queue wait_page; wait_queue_entry_t *wait = &wait_page.wait; @@ -1442,10 +1443,9 @@ void migration_entry_wait_on_locked(swp_entry_t entry, pte_t *ptep, * a valid reference to the page, and it must take the ptl to remove the * migration entry. So the page is valid until the ptl is dropped. */ - if (ptep) - pte_unmap_unlock(ptep, ptl); - else - spin_unlock(ptl); + spin_unlock(ptl); + if (unmap && ptep) + pte_unmap(ptep); for (;;) { unsigned int flags; diff --git a/mm/hugetlb.c b/mm/hugetlb.c index fb5a549169ce..3fc61a437c2a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -6778,7 +6778,7 @@ follow_huge_pmd(struct mm_struct *mm, unsigned long address, } else { if (is_hugetlb_entry_migration(pte)) { spin_unlock(ptl); - __migration_entry_wait(mm, (pte_t *)pmd, ptl); + __migration_entry_wait(mm, (pte_t *)pmd, ptl, false); goto retry; } /* diff --git a/mm/migrate.c b/mm/migrate.c index 044656a14ae2..0bdf27fbc45b 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -287,7 +287,7 @@ void remove_migration_ptes(struct folio *src, struct folio *dst, bool locked) * When we return from this function the fault will be retried. */ void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, - spinlock_t *ptl) + spinlock_t *ptl, bool unmap) { pte_t pte; swp_entry_t entry; @@ -301,10 +301,12 @@ void __migration_entry_wait(struct mm_struct *mm, pte_t *ptep, if (!is_migration_entry(entry)) goto out; - migration_entry_wait_on_locked(entry, ptep, ptl); + migration_entry_wait_on_locked(entry, ptep, ptl, unmap); return; out: - pte_unmap_unlock(ptep, ptl); + spin_unlock(ptl); + if (unmap) + pte_unmap(ptep); } void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, @@ -312,14 +314,14 @@ void migration_entry_wait(struct mm_struct *mm, pmd_t *pmd, { spinlock_t *ptl = pte_lockptr(mm, pmd); pte_t *ptep = pte_offset_map(pmd, address); - __migration_entry_wait(mm, ptep, ptl); + __migration_entry_wait(mm, ptep, ptl, true); } void migration_entry_wait_huge(struct vm_area_struct *vma, struct mm_struct *mm, pte_t *pte) { spinlock_t *ptl = huge_pte_lockptr(hstate_vma(vma), mm, pte); - __migration_entry_wait(mm, pte, ptl); + __migration_entry_wait(mm, pte, ptl, false); } #ifdef CONFIG_ARCH_ENABLE_THP_MIGRATION @@ -330,7 +332,7 @@ void pmd_migration_entry_wait(struct mm_struct *mm, pmd_t *pmd) ptl = pmd_lock(mm, pmd); if (!is_pmd_migration_entry(*pmd)) goto unlock; - migration_entry_wait_on_locked(pmd_to_swp_entry(*pmd), NULL, ptl); + migration_entry_wait_on_locked(pmd_to_swp_entry(*pmd), NULL, ptl, false); return; unlock: spin_unlock(ptl);