From patchwork Wed Sep 16 00:59:36 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Huang, Ying" X-Patchwork-Id: 11778651 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id C8BBA59D for ; Wed, 16 Sep 2020 01:00:15 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id 79500208E4 for ; Wed, 16 Sep 2020 01:00:15 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 79500208E4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 656BD8E0007; Tue, 15 Sep 2020 21:00:14 -0400 (EDT) Delivered-To: linux-mm-outgoing@kvack.org Received: by kanga.kvack.org (Postfix, from userid 40) id 5E0E38E0006; Tue, 15 Sep 2020 21:00:14 -0400 (EDT) X-Original-To: int-list-linux-mm@kvack.org X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 481C08E0007; Tue, 15 Sep 2020 21:00:14 -0400 (EDT) X-Original-To: linux-mm@kvack.org X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0053.hostedemail.com [216.40.44.53]) by kanga.kvack.org (Postfix) with ESMTP id 231528E0006 for ; Tue, 15 Sep 2020 21:00:14 -0400 (EDT) Received: from smtpin09.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay01.hostedemail.com (Postfix) with ESMTP id D4B75180AD815 for ; Wed, 16 Sep 2020 01:00:13 +0000 (UTC) X-FDA: 77267118306.09.bean88_2a05c9927115 Received: from filter.hostedemail.com (10.5.16.251.rfc1918.com [10.5.16.251]) by smtpin09.hostedemail.com (Postfix) with ESMTP id ACF47180AD81A for ; Wed, 16 Sep 2020 01:00:13 +0000 (UTC) X-Spam-Summary: 1,0,0,,d41d8cd98f00b204,ying.huang@intel.com,,RULES_HIT:30041:30054:30064:30090,0,RBL:134.134.136.65:@intel.com:.lbl8.mailshell.net-62.18.0.100 64.95.201.95;04y8eiec6wc495gdt5s9uhqd3p1j5op7f5pxpkdt11ebmensmyrdxjw74zp8k7h.1njbwy586dne4k7386199gya7anyuain78sej6cwthwxiprqbowhh8wqgjkxdha.1-lbl8.mailshell.net-223.238.255.100,CacheIP:none,Bayesian:0.5,0.5,0.5,Netcheck:none,DomainCache:0,MSF:not bulk,SPF:ft,MSBL:0,DNSBL:none,Custom_rules:0:0:0,LFtime:25,LUA_SUMMARY:none X-HE-Tag: bean88_2a05c9927115 X-Filterd-Recvd-Size: 7109 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by imf30.hostedemail.com (Postfix) with ESMTP for ; Wed, 16 Sep 2020 01:00:12 +0000 (UTC) IronPort-SDR: d058nL8iVuCsGbVGoS50ZRi6DsAELlwMKHRKVTi78t9oqGnNaSY7UlGaOkK6op9G/bbmP3aCgP bHkfuyAwPG4w== X-IronPort-AV: E=McAfee;i="6000,8403,9745"; a="159429170" X-IronPort-AV: E=Sophos;i="5.76,430,1592895600"; d="scan'208";a="159429170" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga005.jf.intel.com ([10.7.209.41]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2020 18:00:05 -0700 IronPort-SDR: pWpP/LcYj+LfufZPRmoOf2HytaUADuBxdt8M6ExRNFCD3nu5SBqSM20CB10lKxM5SUlFmtdKcA C29gKJKumzhQ== X-IronPort-AV: E=Sophos;i="5.76,430,1592895600"; d="scan'208";a="483078065" Received: from yhuang-mobile.sh.intel.com ([10.238.4.22]) by orsmga005-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 15 Sep 2020 18:00:01 -0700 From: Huang Ying To: Peter Zijlstra Cc: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Huang Ying , Andrew Morton , Ingo Molnar , Mel Gorman , Rik van Riel , Johannes Weiner , "Matthew Wilcox (Oracle)" , Dave Hansen , Andi Kleen , Michal Hocko , David Rientjes Subject: [RFC] autonuma: Migrate on fault among multiple bound nodes Date: Wed, 16 Sep 2020 08:59:36 +0800 Message-Id: <20200916005936.232788-1-ying.huang@intel.com> X-Mailer: git-send-email 2.28.0 MIME-Version: 1.0 X-Rspamd-Queue-Id: ACF47180AD81A X-Spamd-Result: default: False [0.00 / 100.00] X-Rspamd-Server: rspam02 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now, AutoNUMA can only optimize the page placement among the NUMA nodes if the default memory policy is used. Because the memory policy specified explicitly should take precedence. But this seems too strict in some situations. For example, on a system with 4 NUMA nodes, if the memory of an application is bound to the node 0 and 1, AutoNUMA can potentially migrate the pages between the node 0 and 1 to reduce cross-node accessing without breaking the explicit memory binding policy. So in this patch, if MPOL_BIND is used to bind the memory of the application to multiple nodes, and in the hint page fault handler both the faulting page node and the accessing node are in the policy nodemask, the page will be tried to be migrated to the accessing node to reduce the cross-node accessing. Questions: Sysctl knob kernel.numa_balancing can enable/disable AutoNUMA optimizing globally. And now, it appears that the explicit NUMA memory policy specifying (e.g. via numactl, mbind(), etc.) acts like an implicit per-thread/VMA knob to enable/disable the AutoNUMA optimizing for the thread/VMA. Although this looks like a side effect instead of an API, from commit fc3147245d19 ("mm: numa: Limit NUMA scanning to migrate-on-fault VMAs"), this is used by some users? So the question is, do we need an explicit per-thread/VMA knob to enable/disable AutoNUMA optimizing for the thread/VMA? Or just use the global knob, either optimize all thread/VMAs as long as the explicitly specified memory policies are respected, or don't optimize at all. Signed-off-by: "Huang, Ying" Cc: Andrew Morton Cc: Ingo Molnar Cc: Mel Gorman Cc: Rik van Riel Cc: Johannes Weiner Cc: "Matthew Wilcox (Oracle)" Cc: Dave Hansen Cc: Andi Kleen Cc: Michal Hocko Cc: David Rientjes --- mm/mempolicy.c | 43 +++++++++++++++++++++++++++++++------------ 1 file changed, 31 insertions(+), 12 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index eddbe4e56c73..a941eab2de24 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -1827,6 +1827,13 @@ static struct mempolicy *get_vma_policy(struct vm_area_struct *vma, return pol; } +static bool mpol_may_mof(struct mempolicy *pol) +{ + /* May migrate among bound nodes for MPOL_BIND */ + return pol->flags & MPOL_F_MOF || + (pol->mode == MPOL_BIND && nodes_weight(pol->v.nodes) > 1); +} + bool vma_policy_mof(struct vm_area_struct *vma) { struct mempolicy *pol; @@ -1835,7 +1842,7 @@ bool vma_policy_mof(struct vm_area_struct *vma) bool ret = false; pol = vma->vm_ops->get_policy(vma, vma->vm_start); - if (pol && (pol->flags & MPOL_F_MOF)) + if (pol && mpol_may_mof(pol)) ret = true; mpol_cond_put(pol); @@ -1846,7 +1853,7 @@ bool vma_policy_mof(struct vm_area_struct *vma) if (!pol) pol = get_task_policy(current); - return pol->flags & MPOL_F_MOF; + return mpol_may_mof(pol); } static int apply_policy_zone(struct mempolicy *policy, enum zone_type zone) @@ -2474,11 +2481,13 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long int thisnid = cpu_to_node(thiscpu); int polnid = NUMA_NO_NODE; int ret = -1; + bool moron; pol = get_vma_policy(vma, addr); - if (!(pol->flags & MPOL_F_MOF)) + if (!mpol_may_mof(pol)) goto out; + moron = pol->flags & MPOL_F_MORON; switch (pol->mode) { case MPOL_INTERLEAVE: pgoff = vma->vm_pgoff; @@ -2494,20 +2503,30 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long break; case MPOL_BIND: - /* - * allows binding to multiple nodes. - * use current page if in policy nodemask, - * else select nearest allowed node, if any. - * If no allowed nodes, use current [!misplaced]. + * Allows binding to multiple nodes. If both current and + * accessing nodes are in policy nodemask, migrate to + * accessing node to optimize page placement. Otherwise, + * use current page if in policy nodemask or MPOL_F_MOF not + * set, else select nearest allowed node, if any. If no + * allowed nodes, use current [!misplaced]. */ - if (node_isset(curnid, pol->v.nodes)) + if (node_isset(curnid, pol->v.nodes)) { + if (node_isset(thisnid, pol->v.nodes)) { + moron = true; + polnid = thisnid; + } else { + goto out; + } + } else if (!(pol->flags & MPOL_F_MOF)) { goto out; - z = first_zones_zonelist( + } else { + z = first_zones_zonelist( node_zonelist(numa_node_id(), GFP_HIGHUSER), gfp_zone(GFP_HIGHUSER), &pol->v.nodes); - polnid = zone_to_nid(z->zone); + polnid = zone_to_nid(z->zone); + } break; default: @@ -2515,7 +2534,7 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long } /* Migrate the page towards the node whose CPU is referencing it */ - if (pol->flags & MPOL_F_MORON) { + if (moron) { polnid = thisnid; if (!should_numa_migrate_memory(current, page, curnid, thiscpu))