From patchwork Tue Apr 13 21:24:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12201451 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B1CCBC43460 for ; Tue, 13 Apr 2021 21:24:33 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 453C9611F2 for ; Tue, 13 Apr 2021 21:24:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 453C9611F2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id D14586B0073; Tue, 13 Apr 2021 17:24:32 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C286C6B0074; Tue, 13 Apr 2021 17:24:32 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A77676B0075; Tue, 13 Apr 2021 17:24:32 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8A7826B0073 for ; Tue, 13 Apr 2021 17:24:32 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 504A335A1346 for ; Tue, 13 Apr 2021 21:24:32 +0000 (UTC) X-FDA: 78028622784.07.9FDE59A Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf16.hostedemail.com (Postfix) with ESMTP id 49C1880192E7 for ; Tue, 13 Apr 2021 21:24:31 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id lt13so304148pjb.1 for ; Tue, 13 Apr 2021 14:24:31 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=nuh9Sc/RYtjXzBdNTFFbp+DaukNGNDhP3A6BK4PLHbk=; b=dpLkJ6XbI5rtIYwpPZ5jv/8lwJfdyhUYeCwj2FZJPh8vQWTUyx1Cj0VkUxQqhmyKvC 2LrwJ/pMyonHoxs0eUl6RhEwpI8uC7CtSIJQv8YxYdfoCMLDMPX2kh8f5NroHFFhaT1U ZgQ0k16qMgrIZsgyTtcCaG7Y4ghw5qqboqx3JkV+adetlRoig8h5Qf4dfAZ9LX+m8/h+ Ura50H4N3vsvrjycz6vAA/IuYPMGl/lNnNk3MkNEkw+uODxvzK1fXMtKDxwUUzCoAywW xP/ENH3qoKHGvP5/+2Tr97/Qwa6v3d6VgAZHDRvOIYjEWcnWmtmaa1WQFDD7awn5pcuf ymeQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=nuh9Sc/RYtjXzBdNTFFbp+DaukNGNDhP3A6BK4PLHbk=; b=e7LShpX+NphMTlkZjNvV/J1CNxpvsiCwSPFzNxiX+MKoYYvQ7Xtgpy/PKaEnBnTORr 4zEc/IcdtzSUkDnUNF2CQXfgKGWOHQ0esse1mO9rpK4veccU7TE5vTfeHNXdtpKiiOTC PtYjRTAj8X9Tec0pc3hbYUaZvJ+NTXIdvBWwryWdQS9x6hw+kmVd6+iiec7RSVTkGMVg +NTIeXp6taEzcfYo/yVQDLfULFliBZXn4HP2oAPNsixjo6qSlZRJQ6jkvhebQb1BgGR4 Ovw/pKKg1Kg5uw7EJPFULnrXsOGXeTA4d5iczahSU+7ZRc8MF10j8GNJszI6DmblC3aC Ao4w== X-Gm-Message-State: AOAM532zL7Wx2o8M3PcNGR57HQKx+itSj/KMMWP0vzxF2G8CmSJxHAZH pupXBBeUw5VT6TeT6CUstK0= X-Google-Smtp-Source: ABdhPJx8OUOYf9GPNdWTjI/X/v0JsVtCMatdMAanmAaAygiQFqXEXQgpSt7VOkgjdOY3WSiVDAbjFA== X-Received: by 2002:a17:90a:c203:: with SMTP id e3mr2035635pjt.173.1618349070963; Tue, 13 Apr 2021 14:24:30 -0700 (PDT) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id fw24sm3069345pjb.21.2021.04.13.14.24.29 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Apr 2021 14:24:30 -0700 (PDT) From: Yang Shi To: mgorman@suse.de, kirill.shutemov@linux.intel.com, ziy@nvidia.com, mhocko@suse.com, ying.huang@intel.com, hughd@google.com, gerald.schaefer@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 1/7] mm: memory: add orig_pmd to struct vm_fault Date: Tue, 13 Apr 2021 14:24:10 -0700 Message-Id: <20210413212416.3273-2-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210413212416.3273-1-shy828301@gmail.com> References: <20210413212416.3273-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: 49C1880192E7 X-Stat-Signature: dfp63ykqidsr8huhheo4okgtc3azp869 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf16; identity=mailfrom; envelope-from=""; helo=mail-pj1-f47.google.com; client-ip=209.85.216.47 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618349071-796214 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Add orig_pmd to struct vm_fault so the "orig_pmd" parameter used by huge page fault could be removed, just like its PTE counterpart does. Signed-off-by: Yang Shi --- include/linux/huge_mm.h | 9 ++++----- include/linux/mm.h | 3 +++ mm/huge_memory.c | 9 ++++++--- mm/memory.c | 26 +++++++++++++------------- 4 files changed, 26 insertions(+), 21 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 9626fda5efce..3b38070f4337 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -11,7 +11,7 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf); int copy_huge_pmd(struct mm_struct *dst_mm, struct mm_struct *src_mm, pmd_t *dst_pmd, pmd_t *src_pmd, unsigned long addr, struct vm_area_struct *vma); -void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd); +void huge_pmd_set_accessed(struct vm_fault *vmf); int copy_huge_pud(struct mm_struct *dst_mm, struct mm_struct *src_mm, pud_t *dst_pud, pud_t *src_pud, unsigned long addr, struct vm_area_struct *vma); @@ -24,7 +24,7 @@ static inline void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud) } #endif -vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd); +vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf); struct page *follow_trans_huge_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmd, unsigned int flags); @@ -283,7 +283,7 @@ struct page *follow_devmap_pmd(struct vm_area_struct *vma, unsigned long addr, struct page *follow_devmap_pud(struct vm_area_struct *vma, unsigned long addr, pud_t *pud, int flags, struct dev_pagemap **pgmap); -vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t orig_pmd); +vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf); extern struct page *huge_zero_page; @@ -429,8 +429,7 @@ static inline spinlock_t *pud_trans_huge_lock(pud_t *pud, return NULL; } -static inline vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, - pmd_t orig_pmd) +static inline vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) { return 0; } diff --git a/include/linux/mm.h b/include/linux/mm.h index 25b9041f9925..9c5856f8cc81 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -547,6 +547,9 @@ struct vm_fault { * the 'address' */ pte_t orig_pte; /* Value of PTE at the time of fault */ + pmd_t orig_pmd; /* Value of PMD at the time of fault, + * used by PMD fault only. + */ struct page *cow_page; /* Page handler may use for COW fault */ struct page *page; /* ->fault handlers should return a diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 63ed6b25deaa..35cac4aeaf68 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1251,11 +1251,12 @@ void huge_pud_set_accessed(struct vm_fault *vmf, pud_t orig_pud) } #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ -void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd) +void huge_pmd_set_accessed(struct vm_fault *vmf) { pmd_t entry; unsigned long haddr; bool write = vmf->flags & FAULT_FLAG_WRITE; + pmd_t orig_pmd = vmf->orig_pmd; vmf->ptl = pmd_lock(vmf->vma->vm_mm, vmf->pmd); if (unlikely(!pmd_same(*vmf->pmd, orig_pmd))) @@ -1272,11 +1273,12 @@ void huge_pmd_set_accessed(struct vm_fault *vmf, pmd_t orig_pmd) spin_unlock(vmf->ptl); } -vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf, pmd_t orig_pmd) +vm_fault_t do_huge_pmd_wp_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; struct page *page; unsigned long haddr = vmf->address & HPAGE_PMD_MASK; + pmd_t orig_pmd = vmf->orig_pmd; vmf->ptl = pmd_lockptr(vma->vm_mm, vmf->pmd); VM_BUG_ON_VMA(!vma->anon_vma, vma); @@ -1412,9 +1414,10 @@ struct page *follow_trans_huge_pmd(struct vm_area_struct *vma, } /* NUMA hinting page fault entry point for trans huge pmds */ -vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf, pmd_t pmd) +vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; + pmd_t pmd = vmf->orig_pmd; struct anon_vma *anon_vma = NULL; struct page *page; unsigned long haddr = vmf->address & HPAGE_PMD_MASK; diff --git a/mm/memory.c b/mm/memory.c index 4e358601c5d6..e0cbffeceb0b 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4232,12 +4232,12 @@ static inline vm_fault_t create_huge_pmd(struct vm_fault *vmf) } /* `inline' is required to avoid gcc 4.1.2 build error */ -static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf, pmd_t orig_pmd) +static inline vm_fault_t wp_huge_pmd(struct vm_fault *vmf) { if (vma_is_anonymous(vmf->vma)) { - if (userfaultfd_huge_pmd_wp(vmf->vma, orig_pmd)) + if (userfaultfd_huge_pmd_wp(vmf->vma, vmf->orig_pmd)) return handle_userfault(vmf, VM_UFFD_WP); - return do_huge_pmd_wp_page(vmf, orig_pmd); + return do_huge_pmd_wp_page(vmf); } if (vmf->vma->vm_ops->huge_fault) { vm_fault_t ret = vmf->vma->vm_ops->huge_fault(vmf, PE_SIZE_PMD); @@ -4464,26 +4464,26 @@ static vm_fault_t __handle_mm_fault(struct vm_area_struct *vma, if (!(ret & VM_FAULT_FALLBACK)) return ret; } else { - pmd_t orig_pmd = *vmf.pmd; + vmf.orig_pmd = *vmf.pmd; barrier(); - if (unlikely(is_swap_pmd(orig_pmd))) { + if (unlikely(is_swap_pmd(vmf.orig_pmd))) { VM_BUG_ON(thp_migration_supported() && - !is_pmd_migration_entry(orig_pmd)); - if (is_pmd_migration_entry(orig_pmd)) + !is_pmd_migration_entry(vmf.orig_pmd)); + if (is_pmd_migration_entry(vmf.orig_pmd)) pmd_migration_entry_wait(mm, vmf.pmd); return 0; } - if (pmd_trans_huge(orig_pmd) || pmd_devmap(orig_pmd)) { - if (pmd_protnone(orig_pmd) && vma_is_accessible(vma)) - return do_huge_pmd_numa_page(&vmf, orig_pmd); + if (pmd_trans_huge(vmf.orig_pmd) || pmd_devmap(vmf.orig_pmd)) { + if (pmd_protnone(vmf.orig_pmd) && vma_is_accessible(vma)) + return do_huge_pmd_numa_page(&vmf); - if (dirty && !pmd_write(orig_pmd)) { - ret = wp_huge_pmd(&vmf, orig_pmd); + if (dirty && !pmd_write(vmf.orig_pmd)) { + ret = wp_huge_pmd(&vmf); if (!(ret & VM_FAULT_FALLBACK)) return ret; } else { - huge_pmd_set_accessed(&vmf, orig_pmd); + huge_pmd_set_accessed(&vmf); return 0; } } From patchwork Tue Apr 13 21:24:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12201453 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 77F72C433ED for ; Tue, 13 Apr 2021 21:24:35 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 070BB61242 for ; Tue, 13 Apr 2021 21:24:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 070BB61242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 9DE1F6B0074; Tue, 13 Apr 2021 17:24:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 93FF56B0075; Tue, 13 Apr 2021 17:24:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6F5406B0078; Tue, 13 Apr 2021 17:24:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0148.hostedemail.com [216.40.44.148]) by kanga.kvack.org (Postfix) with ESMTP id 5407E6B0074 for ; Tue, 13 Apr 2021 17:24:34 -0400 (EDT) Received: from smtpin33.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay03.hostedemail.com (Postfix) with ESMTP id 1A13082499A8 for ; Tue, 13 Apr 2021 21:24:34 +0000 (UTC) X-FDA: 78028622868.33.78FA004 Received: from mail-pl1-f179.google.com (mail-pl1-f179.google.com [209.85.214.179]) by imf19.hostedemail.com (Postfix) with ESMTP id D0EA690009F2 for ; Tue, 13 Apr 2021 21:24:18 +0000 (UTC) Received: by mail-pl1-f179.google.com with SMTP id d8so8931274plh.11 for ; Tue, 13 Apr 2021 14:24:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=q1ZBdEMq+3MjnBJy65dpAKj+xf+cUYHcWFOoNEXmf+M=; b=SvB9X8lBvtivut2BA5VI0UZ2NrD4YaT93CUQv7Q6uXHLcuHfXSTe5LcXe9DiMnEe0Z eVK6PupcUxdh0EFav96gan+YlGB2jFDZlBnVl0XlwTF4DAkE86Rvnl7DdRN12R2fItFL OFOZ+FyJBh0D875PdCB1VSZDiCk/D/oIvYU0W3FHQJmRctIEuSeKjqkLJ6fDEUW9sHJi Q3iA5gW5QEHByQ2mI85/5Q8wfINMRHx2JcMCPbesUHLRhDBDQ35rSKmxO/huerggbdPs T+VzKzVCiFxXIYn2xky/SNeVyWdA6wfjLaW/eFjOP8Kf5nlVae6e510LWUlOqONan4E4 IlYg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=q1ZBdEMq+3MjnBJy65dpAKj+xf+cUYHcWFOoNEXmf+M=; b=d0rqeat4tAWdYYFOwMAGsWaoesne68N+UQBPEz4RaS9iZHcuZa+eigp3Is2lwZBB6b xVDg1yq2+iu3HGlPkBOg/42DKsj6FAu+1iMxgYUbUeBE6nro9RjWe3ldxgN9njlIvMtm OUmqdSQLGaSdwmLqhIAMRm6Piwvqu7aN8aB8fPLA3o92s0g3gOgyQd+XQftXYmPvHEft kzjkhk+FkO1op9AHJhfoVxz5C8JjxEoqdQIBTfH6WfdLPa94uLAh8qT5cz51FrUbWqlc cufd4uo/EJl+//Sn4+wDDYR3DQ5d3vOeUgDe82m2Y7DMMH1IrOSG8Lu1c/azhrnVJSeJ bmUQ== X-Gm-Message-State: AOAM532U9Bmb2JDmVjymOXcvOiGOhL97aYQh+gAVz3kW0DbIG8mgfc4X nFkQsDJRSyJ5M1GUW4cGMlo= X-Google-Smtp-Source: ABdhPJw3JAQ5VVCViHQ3J/0GRFEvDaPxbEcy3o1iaSfzjbRncHPT6v4ClCmjTBDlc/t0lyCX1Iqn9g== X-Received: by 2002:a17:90a:1188:: with SMTP id e8mr2144226pja.89.1618349073080; Tue, 13 Apr 2021 14:24:33 -0700 (PDT) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id fw24sm3069345pjb.21.2021.04.13.14.24.31 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Apr 2021 14:24:32 -0700 (PDT) From: Yang Shi To: mgorman@suse.de, kirill.shutemov@linux.intel.com, ziy@nvidia.com, mhocko@suse.com, ying.huang@intel.com, hughd@google.com, gerald.schaefer@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 2/7] mm: memory: make numa_migrate_prep() non-static Date: Tue, 13 Apr 2021 14:24:11 -0700 Message-Id: <20210413212416.3273-3-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210413212416.3273-1-shy828301@gmail.com> References: <20210413212416.3273-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: D0EA690009F2 X-Stat-Signature: r6anywcjhirti5bmz75seyggssakp7z8 X-Rspamd-Server: rspam02 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf19; identity=mailfrom; envelope-from=""; helo=mail-pl1-f179.google.com; client-ip=209.85.214.179 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618349058-737962 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The numa_migrate_prep() will be used by huge NUMA fault as well in the following patch, make it non-static. Signed-off-by: Yang Shi Acked-by: Mel Gorman --- mm/internal.h | 3 +++ mm/memory.c | 5 ++--- 2 files changed, 5 insertions(+), 3 deletions(-) diff --git a/mm/internal.h b/mm/internal.h index f469f69309de..b6f889d9c22d 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -659,4 +659,7 @@ int vmap_pages_range_noflush(unsigned long addr, unsigned long end, void vunmap_range_noflush(unsigned long start, unsigned long end); +int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, + unsigned long addr, int page_nid, int *flags); + #endif /* __MM_INTERNAL_H */ diff --git a/mm/memory.c b/mm/memory.c index e0cbffeceb0b..bbb38066021a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4119,9 +4119,8 @@ static vm_fault_t do_fault(struct vm_fault *vmf) return ret; } -static int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, - unsigned long addr, int page_nid, - int *flags) +int numa_migrate_prep(struct page *page, struct vm_area_struct *vma, + unsigned long addr, int page_nid, int *flags) { get_page(page); From patchwork Tue Apr 13 21:24:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12201455 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB5C4C433B4 for ; Tue, 13 Apr 2021 21:24:39 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 90BD461246 for ; Tue, 13 Apr 2021 21:24:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 90BD461246 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 2FF9A6B0075; Tue, 13 Apr 2021 17:24:39 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 2B0946B0078; Tue, 13 Apr 2021 17:24:39 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1507D6B007B; Tue, 13 Apr 2021 17:24:39 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0089.hostedemail.com [216.40.44.89]) by kanga.kvack.org (Postfix) with ESMTP id EE25F6B0075 for ; Tue, 13 Apr 2021 17:24:38 -0400 (EDT) Received: from smtpin24.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay02.hostedemail.com (Postfix) with ESMTP id 9D54235A1350 for ; Tue, 13 Apr 2021 21:24:38 +0000 (UTC) X-FDA: 78028623036.24.DBD60BC Received: from mail-pj1-f47.google.com (mail-pj1-f47.google.com [209.85.216.47]) by imf29.hostedemail.com (Postfix) with ESMTP id 73936E6 for ; Tue, 13 Apr 2021 21:24:36 +0000 (UTC) Received: by mail-pj1-f47.google.com with SMTP id e8-20020a17090a7288b029014e51f5a6baso4358720pjg.2 for ; Tue, 13 Apr 2021 14:24:38 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=AmtfeB5twZZ4rQ0Q6SWlOUiWCuvN4QYBszoXlhLmsSk=; b=iVr3rN5OZs9UQnNoWS5iJxIk8vfUyYrz+b1F8GvRCkC0gTKt40QJl9BcrF0UdZKSl0 6COTC0Pktx90r15Zy7koDF9/Dn7U6cIvBLPkygAFyAoUOnOlVvGowkPqgG6oP2fcL0Cj gvvEr2nlre03l6pvwggVW7Gvl3x1WE7ykWutoyWLJArbqlXjj7GHYQ55j4IEJFm5bwOG GtyBoUfYkwuuchv88jJ3xQBaWx/v7PgWe18UVhrAv1Pxx8EatORoi6ZI5VS3UE/QHg6F 40fda+SU9So2hGVDJTPjRvdF/ZmqsYs9hZqB16HIJvfQ6B264VAwUzsAqDtzE9tH7iau R0Bw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=AmtfeB5twZZ4rQ0Q6SWlOUiWCuvN4QYBszoXlhLmsSk=; b=FbgW8ri9R4bk4zwDY5PWab5jGOswBmJtd9rg3wATphScLYj9m/iooRiInDCy7LROCz OBpqlxukHYmhDXIvm0ebRjCzI/B8w0LtXYFo8V484bEFKSuJM3mfOUBX4fr5kStasAs8 LGwh4E+0rj10zU6B5jie9PBGqyp3HNGgF5Dz4oo4aVlFwa9ExNltqHCZ2GTKCXOER7RV gokXkb2Ki8P6CLORYzA3+vI2ZYEX1DOfjBGbRWOO8Maw01h5gw7HN3ZoC3+LrqzGHouV AE+18dVDh7VQmXZIIrpLqeuOPt9OEv/YTqkoT1CjknY7TvJRilQeWnuUSf9ouKFriHdU gy4A== X-Gm-Message-State: AOAM530d6g+sA/I/ZN9RBjmKEqkVXq6+7T1nXZTXZMqKurchkcV0M0eS +2DM+in+BNfnTAQwDU128KU= X-Google-Smtp-Source: ABdhPJwYiPy8wHeDk26h6rj5vIUXFbUQ74snf6AFlp+1DmZ150lwhBeMHSj2yODGu2UXHK9FslfXkA== X-Received: by 2002:a17:90a:ce0a:: with SMTP id f10mr2001141pju.98.1618349077593; Tue, 13 Apr 2021 14:24:37 -0700 (PDT) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id fw24sm3069345pjb.21.2021.04.13.14.24.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Apr 2021 14:24:36 -0700 (PDT) From: Yang Shi To: mgorman@suse.de, kirill.shutemov@linux.intel.com, ziy@nvidia.com, mhocko@suse.com, ying.huang@intel.com, hughd@google.com, gerald.schaefer@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 4/7] mm: migrate: account THP NUMA migration counters correctly Date: Tue, 13 Apr 2021 14:24:13 -0700 Message-Id: <20210413212416.3273-5-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210413212416.3273-1-shy828301@gmail.com> References: <20210413212416.3273-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 73936E6 X-Stat-Signature: 11qncubt9xwphafhfck4qyck98886mg8 X-Rspamd-Server: rspam02 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf29; identity=mailfrom; envelope-from=""; helo=mail-pj1-f47.google.com; client-ip=209.85.216.47 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618349076-865397 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now both base page and THP NUMA migration is done via migrate_misplaced_page(), keep the counters correctly for THP. Signed-off-by: Yang Shi Acked-by: Mel Gorman --- mm/migrate.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index 333448aa53f1..a473f25fbd01 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2111,6 +2111,7 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, LIST_HEAD(migratepages); new_page_t *new; bool compound; + unsigned int nr_pages = thp_nr_pages(page); /* * PTE mapped THP or HugeTLB page can't reach here so the page could @@ -2149,13 +2150,13 @@ int migrate_misplaced_page(struct page *page, struct vm_area_struct *vma, if (nr_remaining) { if (!list_empty(&migratepages)) { list_del(&page->lru); - dec_node_page_state(page, NR_ISOLATED_ANON + - page_is_file_lru(page)); + mod_node_page_state(page_pgdat(page), NR_ISOLATED_ANON + + page_is_file_lru(page), -nr_pages); putback_lru_page(page); } isolated = 0; } else - count_vm_numa_event(NUMA_PAGE_MIGRATE); + count_vm_numa_events(NUMA_PAGE_MIGRATE, nr_pages); BUG_ON(!list_empty(&migratepages)); return isolated; From patchwork Tue Apr 13 21:24:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12201457 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8971C43462 for ; Tue, 13 Apr 2021 21:24:41 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 67F8961242 for ; Tue, 13 Apr 2021 21:24:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 67F8961242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 0434B6B0078; Tue, 13 Apr 2021 17:24:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F0B206B007B; Tue, 13 Apr 2021 17:24:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D5DD66B007D; Tue, 13 Apr 2021 17:24:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0120.hostedemail.com [216.40.44.120]) by kanga.kvack.org (Postfix) with ESMTP id B14216B0078 for ; Tue, 13 Apr 2021 17:24:40 -0400 (EDT) Received: from smtpin21.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id 6AC3135A1353 for ; Tue, 13 Apr 2021 21:24:40 +0000 (UTC) X-FDA: 78028623120.21.9483BB5 Received: from mail-pj1-f49.google.com (mail-pj1-f49.google.com [209.85.216.49]) by imf04.hostedemail.com (Postfix) with ESMTP id 77A0513A for ; Tue, 13 Apr 2021 21:24:38 +0000 (UTC) Received: by mail-pj1-f49.google.com with SMTP id z22-20020a17090a0156b029014d4056663fso9701741pje.0 for ; Tue, 13 Apr 2021 14:24:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=w6LlC7e6qPBv/DFANfawt2CvXuDZCeYDosf1+Pz2+ts=; b=iE9WM8EBKV5aqe7bVrS2wJeFBV1IT4YtJyiImW0a6iK5SLKlcjpXJyB4HbZU6uTNAh +Zg1V+1OsAruwtxi3bUb7wvRNYJkSSu0K3Fe2/D+D6SX09BPrsZxu1y3gmQwWmiaqRZo rpEEtoNIHZEbDUTv2KOSim2R4Hbe3c2sGbr/7ArtBmWf2ePJPu4za1U7JXB0muQ+ZXaV NEpSPouCHt6XY1qNmzMYD0Kny3niGAKCoDqG2F2VGGIKTQG9nSjZl9xgkWWLT8xg/Fds htFwiTIvNWesTRIHp/BEPmyg1a9DmRR3jR9+En280LgLIjCHUQDeX8Y6ASS45SuON+4N Zybg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=w6LlC7e6qPBv/DFANfawt2CvXuDZCeYDosf1+Pz2+ts=; b=GsinKcxmYMrlrLTgLUEE1XAHjj36CzllGxhUFN01dWkp5AMabVH48AdfMLF9u63QvJ DmKO9rlh4We1spg5pdkXOGkJbfWfgLz49mJhWqYmlEHeoskG6fDzVR5AvuTnIJEGhvFG PrxYEB48atW9nSyhht2C3LxsqT5Uvb/muvJ+3r5Y39nYeiP3Idl5HjSNqUKtCwvgXYGR TzQ+SKfkXN4m8fQqCv+hgrM52NUrURFoQa9bF67vF0/xj+zedGzsOwuEBGt+BeXGC2vc tb0BGrN5OdNsChz/3AzYQBGeg2nrN/8TzmZDOYT/Ca/J90n7rTXSC07kdNv+wzFJAiue 7i1g== X-Gm-Message-State: AOAM533ja51fo1CcFvVIUtqwX4EInOqOXsoGmeCIIDgYw9OYxyDKznYY bhJ9Ee2Q3MXW2+HEIC2wIAA= X-Google-Smtp-Source: ABdhPJxo7MBByycqzfC3pAyOVupxOhelSDT7jOQ4KiIdzVmez62LtHArR8gJdQ7X0jtuHVX3vGl75Q== X-Received: by 2002:a17:90a:5418:: with SMTP id z24mr2117305pjh.189.1618349079422; Tue, 13 Apr 2021 14:24:39 -0700 (PDT) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id fw24sm3069345pjb.21.2021.04.13.14.24.37 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Apr 2021 14:24:38 -0700 (PDT) From: Yang Shi To: mgorman@suse.de, kirill.shutemov@linux.intel.com, ziy@nvidia.com, mhocko@suse.com, ying.huang@intel.com, hughd@google.com, gerald.schaefer@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 5/7] mm: migrate: don't split THP for misplaced NUMA page Date: Tue, 13 Apr 2021 14:24:14 -0700 Message-Id: <20210413212416.3273-6-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210413212416.3273-1-shy828301@gmail.com> References: <20210413212416.3273-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Queue-Id: 77A0513A X-Stat-Signature: pb5wukwcj7ibebjycdakk1yph3hic8u9 X-Rspamd-Server: rspam02 Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf04; identity=mailfrom; envelope-from=""; helo=mail-pj1-f49.google.com; client-ip=209.85.216.49 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618349078-394937 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The old behavior didn't split THP if migration is failed due to lack of memory on the target node. But the THP migration does split THP, so keep the old behavior for misplaced NUMA page migration. Signed-off-by: Yang Shi Acked-by: Mel Gorman --- mm/migrate.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/mm/migrate.c b/mm/migrate.c index a473f25fbd01..a72994c68ec6 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -1417,6 +1417,7 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, int swapwrite = current->flags & PF_SWAPWRITE; int rc, nr_subpages; LIST_HEAD(ret_pages); + bool nosplit = (reason == MR_NUMA_MISPLACED); trace_mm_migrate_pages_start(mode, reason); @@ -1488,8 +1489,9 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page, /* * When memory is low, don't bother to try to migrate * other pages, just exit. + * THP NUMA faulting doesn't split THP to retry. */ - if (is_thp) { + if (is_thp && !nosplit) { if (!try_split_thp(page, &page2, from)) { nr_thp_split++; goto retry; From patchwork Tue Apr 13 21:24:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12201459 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03CA8C43470 for ; Tue, 13 Apr 2021 21:24:44 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id AA3DB61242 for ; Tue, 13 Apr 2021 21:24:43 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AA3DB61242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 45F4A6B007B; Tue, 13 Apr 2021 17:24:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 3C3C36B007D; Tue, 13 Apr 2021 17:24:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 1C70D6B007E; Tue, 13 Apr 2021 17:24:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0192.hostedemail.com [216.40.44.192]) by kanga.kvack.org (Postfix) with ESMTP id 020D96B007B for ; Tue, 13 Apr 2021 17:24:42 -0400 (EDT) Received: from smtpin07.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id BD9A0180257D3 for ; Tue, 13 Apr 2021 21:24:42 +0000 (UTC) X-FDA: 78028623204.07.2752B7A Received: from mail-pg1-f179.google.com (mail-pg1-f179.google.com [209.85.215.179]) by imf22.hostedemail.com (Postfix) with ESMTP id D37BFC0007C1 for ; Tue, 13 Apr 2021 21:24:38 +0000 (UTC) Received: by mail-pg1-f179.google.com with SMTP id k8so12870229pgf.4 for ; Tue, 13 Apr 2021 14:24:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=pRfd+wOd4qoxNAST21yPZCThPxUu/7bAGiKFZOKY6d8=; b=lJITJut37aBTK+f9dTgHu1UjML1K+sz8l3RIQxMTDn07BEPrnJ02I5I/bwqAHPhIn5 S8tiU2O+awovLmX4VVToKSks1+jKTAinxrg9fNCaZhIKwQrgKQKT6YmsjataVgO+KN+s WC1NYW1pTKOYHTIYPYD080AF6kOsZvrjZj+grK6E1WkxAbZuVOP2V/irQEyqxdp4va8U 5XREEOrhTQwmK19x8KewOYi0qofybk5PGLbYU1bSCZtMhhc6SfmXUZjygzLhq5IjTcqL 8frdihCWj1rzB8mYWv7kn0UwefCShpU571suGRgv6KLXt9pGzjNJiaVRXpOVs4ZfoGL9 j+8Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=pRfd+wOd4qoxNAST21yPZCThPxUu/7bAGiKFZOKY6d8=; b=fK3iLe8bGZhI8Ej6KCPuH9zmeYgVHOmmeFqri+KGA5CAjVGGBI//faxVshlrbalnG6 P6DnVRuy5OMRsvUiuMEOVJd2wBI5dOn6Jdd/XhFvCYXoihldkdPFp596FDRK2jtDFqqJ oS8vc3xSpxds1eC6qCI7tu9qPJMk3IRmotyLnY1YLPLTg/wL4OIU7OCfb+EPaAZk5n26 U82A5+xzHFnqvIeuCNPW1YjPipPpCaBK5BWsJuY/DJ6yQlkq/W3YNQUPpic/5ZIZT4T5 x5wo8fI3x9bJDcTS/Pve5agALpxMCQswzHYH5fuMIGWh3zE1dnRsHIGaRgJx407J7Y4f aqnw== X-Gm-Message-State: AOAM533ENIhUdL1lBSKOk0PM4kdmw/eQ7XCoSQh4Kpd/rjany7A1mYq/ AKrRhryEE+/Hwclwk4BkHmI= X-Google-Smtp-Source: ABdhPJwgpAER6R3GgfwWJo6MkscSPav2zxnCK/y1FAkldOUG8y4v39OSNPDocMiAl+h9muoqcInf6w== X-Received: by 2002:a62:e315:0:b029:243:e499:d5ba with SMTP id g21-20020a62e3150000b0290243e499d5bamr28410928pfh.75.1618349081432; Tue, 13 Apr 2021 14:24:41 -0700 (PDT) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id fw24sm3069345pjb.21.2021.04.13.14.24.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Apr 2021 14:24:40 -0700 (PDT) From: Yang Shi To: mgorman@suse.de, kirill.shutemov@linux.intel.com, ziy@nvidia.com, mhocko@suse.com, ying.huang@intel.com, hughd@google.com, gerald.schaefer@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 6/7] mm: migrate: check mapcount for THP instead of ref count Date: Tue, 13 Apr 2021 14:24:15 -0700 Message-Id: <20210413212416.3273-7-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210413212416.3273-1-shy828301@gmail.com> References: <20210413212416.3273-1-shy828301@gmail.com> MIME-Version: 1.0 X-Rspamd-Server: rspam03 X-Rspamd-Queue-Id: D37BFC0007C1 X-Stat-Signature: hcmq5uzu7ssi6yqzcs357q9kbmiwxb3m Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf22; identity=mailfrom; envelope-from=""; helo=mail-pg1-f179.google.com; client-ip=209.85.215.179 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618349078-276130 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: The generic migration path will check refcount, so no need check refcount here. But the old code actually prevents from migrating shared THP (mapped by multiple processes), so bail out early if mapcount is > 1 to keep the behavior. Signed-off-by: Yang Shi --- mm/migrate.c | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/mm/migrate.c b/mm/migrate.c index a72994c68ec6..dc7cc7f3a124 100644 --- a/mm/migrate.c +++ b/mm/migrate.c @@ -2067,6 +2067,10 @@ static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page) VM_BUG_ON_PAGE(compound_order(page) && !PageTransHuge(page), page); + /* Do not migrate THP mapped by multiple processes */ + if (PageTransHuge(page) && page_mapcount(page) > 1) + return 0; + /* Avoid migrating to a node that is nearly full */ if (!migrate_balanced_pgdat(pgdat, compound_nr(page))) return 0; @@ -2074,18 +2078,6 @@ static int numamigrate_isolate_page(pg_data_t *pgdat, struct page *page) if (isolate_lru_page(page)) return 0; - /* - * migrate_misplaced_transhuge_page() skips page migration's usual - * check on page_count(), so we must do it here, now that the page - * has been isolated: a GUP pin, or any other pin, prevents migration. - * The expected page count is 3: 1 for page's mapcount and 1 for the - * caller's pin and 1 for the reference taken by isolate_lru_page(). - */ - if (PageTransHuge(page) && page_count(page) != 3) { - putback_lru_page(page); - return 0; - } - page_lru = page_is_file_lru(page); mod_node_page_state(page_pgdat(page), NR_ISOLATED_ANON + page_lru, thp_nr_pages(page)); From patchwork Tue Apr 13 21:24:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Yang Shi X-Patchwork-Id: 12201461 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.5 required=3.0 tests=BAYES_00, DKIM_ADSP_CUSTOM_MED,DKIM_INVALID,DKIM_SIGNED,FREEMAIL_FORGED_FROMDOMAIN, FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER, INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1FCADC433B4 for ; Tue, 13 Apr 2021 21:24:46 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id C00FB61242 for ; Tue, 13 Apr 2021 21:24:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C00FB61242 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 610066B007D; Tue, 13 Apr 2021 17:24:45 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 56ECC6B007E; Tue, 13 Apr 2021 17:24:45 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 3C4126B0080; Tue, 13 Apr 2021 17:24:45 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0190.hostedemail.com [216.40.44.190]) by kanga.kvack.org (Postfix) with ESMTP id 208326B007D for ; Tue, 13 Apr 2021 17:24:45 -0400 (EDT) Received: from smtpin23.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay05.hostedemail.com (Postfix) with ESMTP id CE9CA18036D77 for ; Tue, 13 Apr 2021 21:24:44 +0000 (UTC) X-FDA: 78028623288.23.E85B9FC Received: from mail-pl1-f169.google.com (mail-pl1-f169.google.com [209.85.214.169]) by imf24.hostedemail.com (Postfix) with ESMTP id AA599A0003AF for ; Tue, 13 Apr 2021 21:24:38 +0000 (UTC) Received: by mail-pl1-f169.google.com with SMTP id h20so8948625plr.4 for ; Tue, 13 Apr 2021 14:24:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=cvqUdonBPmk2kB4YQmzOfQMs+Xj3C2NnGl5E7POeuCY=; b=a7HJlLdPXf1Ie2ABAbT1fZb01nutSrYXrK/qw9DCVz8ySPMt/orx2o9v45UX+/GsGJ 9+c1FrvE21rBLQDcbm2CsgF95BEG3vUAoMTFqeFd76lG9GwE0je/Bt232Ci81FclqkUS O/mt92nq3yLV5l9asSY+bwhjw3VCRRL9tXBO+VhscJdAZLrGxHOc2d3n82CLAU3+NKKa 2klPeH4N/FbCZ+ve4uSQCzKDPO2X+Bn0hQTK+vOCcm9gtZKjiexMKbP5IXCkQBYlSk8q KW7Ckt2010QZN2LgFkRIWQSm/LtT8kUfLosckW6yz0YLhkbxOJpnsJK5LuQWvMrQRxC8 DASA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=cvqUdonBPmk2kB4YQmzOfQMs+Xj3C2NnGl5E7POeuCY=; b=UyU634nAHWd9+BGmqUYjRkyNOf341BKXloErnzY/LnDI9PavCPHMaR3005YvXmoT9t 0y1sTdo3En2mbrtq3e2vlMan/8W13XNOX/7L7qLlG3KNBpiKNI2PcsjYpYH6qn3HIG8Q LI0IVfgOzUrg277GN2su988uh+WDccgwk1t51D3/r6HqFzrLuEARDat41u3cGXkhuep6 wx7+wKFf+ZeZJKTA8B9M/Rw5nsQKGQdqQQs3DlHXcDuETOTvmS5CrZurxw7UeLdDx5Dc TawZl/YuxWGcfcTC3FDK3tcNSynbglw36Lbf4aTsy9IIKWKAkBCErZhaAo4Z7TGbyuoe jYyg== X-Gm-Message-State: AOAM5316tYDe+5vxWjCalEpBWeAkH6e36GtdsFSb5c53sA87oqEM/Fjl s03kPSsRmOkRsePntV5q55s= X-Google-Smtp-Source: ABdhPJx5XohOwflczqaouWLBS/EIq7j+NpJVG76BrGTlOzkAJUP2+sOE/hBZBuiYoDLI2FUEsUe7Uw== X-Received: by 2002:a17:90a:bb0b:: with SMTP id u11mr2115705pjr.159.1618349083713; Tue, 13 Apr 2021 14:24:43 -0700 (PDT) Received: from localhost.localdomain (c-73-93-239-127.hsd1.ca.comcast.net. [73.93.239.127]) by smtp.gmail.com with ESMTPSA id fw24sm3069345pjb.21.2021.04.13.14.24.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 13 Apr 2021 14:24:42 -0700 (PDT) From: Yang Shi To: mgorman@suse.de, kirill.shutemov@linux.intel.com, ziy@nvidia.com, mhocko@suse.com, ying.huang@intel.com, hughd@google.com, gerald.schaefer@linux.ibm.com, hca@linux.ibm.com, gor@linux.ibm.com, borntraeger@de.ibm.com, akpm@linux-foundation.org Cc: shy828301@gmail.com, linux-mm@kvack.org, linux-s390@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [v2 PATCH 7/7] mm: thp: skip make PMD PROT_NONE if THP migration is not supported Date: Tue, 13 Apr 2021 14:24:16 -0700 Message-Id: <20210413212416.3273-8-shy828301@gmail.com> X-Mailer: git-send-email 2.26.2 In-Reply-To: <20210413212416.3273-1-shy828301@gmail.com> References: <20210413212416.3273-1-shy828301@gmail.com> MIME-Version: 1.0 X-Stat-Signature: wz43jzu38nmny6sixm53gz88otxr5t5u X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: AA599A0003AF Received-SPF: none (gmail.com>: No applicable sender policy available) receiver=imf24; identity=mailfrom; envelope-from=""; helo=mail-pl1-f169.google.com; client-ip=209.85.214.169 X-HE-DKIM-Result: pass/pass X-HE-Tag: 1618349078-76217 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: A quick grep shows x86_64, PowerPC (book3s), ARM64 and S390 support both NUMA balancing and THP. But S390 doesn't support THP migration so NUMA balancing actually can't migrate any misplaced pages. Skip make PMD PROT_NONE for such case otherwise CPU cycles may be wasted by pointless NUMA hinting faults on S390. Signed-off-by: Yang Shi Acked-by: Mel Gorman --- mm/huge_memory.c | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 94981907fd4c..f63445f3a17d 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1741,6 +1741,7 @@ bool move_huge_pmd(struct vm_area_struct *vma, unsigned long old_addr, * Returns * - 0 if PMD could not be locked * - 1 if PMD was locked but protections unchanged and TLB flush unnecessary + * or if prot_numa but THP migration is not supported * - HPAGE_PMD_NR if protections changed and TLB flush necessary */ int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, @@ -1755,6 +1756,9 @@ int change_huge_pmd(struct vm_area_struct *vma, pmd_t *pmd, bool uffd_wp = cp_flags & MM_CP_UFFD_WP; bool uffd_wp_resolve = cp_flags & MM_CP_UFFD_WP_RESOLVE; + if (prot_numa && !thp_migration_supported()) + return 1; + ptl = __pmd_trans_huge_lock(pmd, vma); if (!ptl) return 0;