From patchwork Thu Mar 21 11:29:50 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Donet Tom X-Patchwork-Id: 13598666 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84FE9C54E58 for ; Thu, 21 Mar 2024 11:30:51 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 17D706B008C; Thu, 21 Mar 2024 07:30:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 12D3A6B0092; Thu, 21 Mar 2024 07:30:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id F37796B0093; Thu, 21 Mar 2024 07:30:50 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id E3ECE6B008C for ; Thu, 21 Mar 2024 07:30:50 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 83B2AC1563 for ; Thu, 21 Mar 2024 11:30:50 +0000 (UTC) X-FDA: 81920829060.16.1997D75 Received: from mx0b-001b2d01.pphosted.com (mx0b-001b2d01.pphosted.com [148.163.158.5]) by imf10.hostedemail.com (Postfix) with ESMTP id 3A2E8C0031 for ; Thu, 21 Mar 2024 11:30:48 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=UA3E++nh; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf10.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711020648; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=GqNwmjjGnKkyJCZ+En/YSrGZg3Tm76YRIoUTja7BmQk=; b=ieJOnaBmDZN4azCFHKNBr7sADtKBaiT9/NEoCbgHZLh1NfQ0x54K+06j4dk6GtnODEJLD2 smcQCbdzJMpYXmcebuQJL4RVaHQ6RBTjQMN+dTg+93rY89sXTdCP1+f4EUbgnL3OKvcnPO D1x5FEwwcP2R60trGDEM8WnKdRRmCJc= ARC-Authentication-Results: i=1; imf10.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=UA3E++nh; dmarc=pass (policy=none) header.from=ibm.com; spf=pass (imf10.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.158.5 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711020648; a=rsa-sha256; cv=none; b=pWLKElzaE2RgTzc1xrYqfIjhv4RcjU9mGqwjiSUJrqkyJoa7dlwPBWpqIeiT/kAsdLItgd 7ssOGy9gCHOkmHa5lIpFDIDZCmh0QVmGUMIummxWjahKtF7gERJWS5KIUEzCksnKtMFDB/ 41zXD0BesbXt5ZcJPFnM24TJnKqimXs= Received: from pps.filterd (m0353723.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 42LBU38s018622; Thu, 21 Mar 2024 11:30:29 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=GqNwmjjGnKkyJCZ+En/YSrGZg3Tm76YRIoUTja7BmQk=; b=UA3E++nhHOlFtWvtnxJTgdW3PsZRQ8o+9z1W5Z7zoaby5Vl70/l4uzp7nsw5PbSQaOYw 6+6AGHYA9LtCfJWhsAkLzgbFE+7YUdIUhpMAJoFe+9NYQNQ1GLrxYAB1nfK7JjVeHWOL GukFWeEGdsMJ6iAD4UAPn2q11wqtnmdlC29Ssj2AmhOHluKkJqfBwkz0bK9Go6WhmGri 0NQWRC/zXq5Stu5MW9SFZYTtJMPdadE5gY41MVWqic8gvqw/3S7pZ+lu3SRsHJN6eLQq JBhbbVvR500/x2KwkbLmsNkHLv/kpe03H5cpiNTPETVdhFwfyiOgG8p74Db/+Pd6x/gQ 0w== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3x0m2s001g-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Mar 2024 11:30:29 +0000 Received: from m0353723.ppops.net (m0353723.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 42LBUStv019341; Thu, 21 Mar 2024 11:30:28 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3x0m2s001a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Mar 2024 11:30:28 +0000 Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 42L8t9PJ015781; Thu, 21 Mar 2024 11:30:27 GMT Received: from smtprelay05.fra02v.mail.ibm.com ([9.218.2.225]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3wwp50cvjm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Mar 2024 11:30:27 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay05.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 42LBUNi644892526 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Mar 2024 11:30:25 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id B491F2004F; Thu, 21 Mar 2024 11:30:23 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8D1C220043; Thu, 21 Mar 2024 11:30:20 +0000 (GMT) Received: from ltczz402-lp1.aus.stglabs.ibm.com (unknown [9.53.171.174]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 21 Mar 2024 11:30:20 +0000 (GMT) From: Donet Tom To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Aneesh Kumar , Huang Ying , Michal Hocko , Dave Hansen , Mel Gorman , Feng Tang , Andrea Arcangeli , Peter Zijlstra , Ingo Molnar , Rik van Riel , Johannes Weiner , Matthew Wilcox , Vlastimil Babka , Dan Williams , Hugh Dickins , Kefeng Wang , Suren Baghdasaryan , Donet Tom Subject: [PATCH v3 1/2] mm/mempolicy: Use numa_node_id() instead of cpu_to_node() Date: Thu, 21 Mar 2024 06:29:50 -0500 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: 6KIsTU1Xt9z7KQC3vi2YLNXVJ5u8i2oL X-Proofpoint-ORIG-GUID: npUxFYHjrT0XFx-yQj_V238uNe-7JiOO X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-03-21_08,2024-03-18_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 bulkscore=0 phishscore=0 mlxlogscore=863 spamscore=0 lowpriorityscore=0 adultscore=0 clxscore=1015 impostorscore=0 mlxscore=0 malwarescore=0 priorityscore=1501 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2403140000 definitions=main-2403210080 X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 3A2E8C0031 X-Stat-Signature: o1u68hc1uto9cr7d651ofoksdam1sucr X-HE-Tag: 1711020647-877795 X-HE-Meta: U2FsdGVkX1+dIY9KzR+WfCM2vlaL51SOu+B+U4Lwxhw987FiSA4zcWCJmbfa0rhCOLwg5aDcOIOdvjST5fTarLW2zS7OzozEicgI0Rpi2n7JuGelpxbSycPu4QEF7R+XTuvL//UQKSrgOTmFLvaD1EhPx8k6z57XW/yEU6TCFJ9iOFvd3BY970YtRb0rqn4BlDx4kWdaKkkLVz2lrDTpgr1CPeJbdF54PhdA1QRiDr/Ees/7ExBqSl8Uhyy6ud0vnVkRJj4CBiDdv6WU4dqNImGd07GChPCgZFUMb5w9ELR/mOviIZZ43TAx4VdqvivFxlhodUqcqDDcyI73ey37yVm4G4GN5H/3bhGZbUcENVxhAdhqtmHz9nluciZ8si1GuQDh65unYnHuZOsIxb+5j/e+J2rMrckT/I8lC07XQQwKVl/Rq7eY16zIbsakARHLiro6SHB8kQRmN+MPrCNVNzQuroLz5t4XbBi3fpUUfOGc+OC43Zi8JRE3WhNQG6qM9gZyXA6QzGxnCe6sDVq5Ym6J1tUcv19PDu538CVFrE6aO8GWNtFbeMiwvm4pEHrMeex1fTJkG9uFCIIlaiUAWT9HI2QGGZkEoTiO78D2xrENpj6AZtZxkViLEik3fhT81CzvKS3f2+nTv3lDaPMLTyEdtNSik210JW2cFyjN55v29DdOZKht3Wq+FRB2wbSgALod8owHxOdw/QCrnBfV+7Tb0bHjCiYDlGU/ex9rwfflg/yqQSaErmgzdpJeSVPiHZ09t2M5x3cqAPLcm/wT4oknhIxCrt1ZR8qMAGExNq+FTncgmPjIqds5/A4MpOdLzQrmUpwNhs14TiYoTFS8obipwnkbs03dHyivCH7iJsTGXs3DeYOqITXDgrM407xp4YjMsp/434kDxmFwA5Imq+O5xA0GnayGOVaYyVMWz1wcRLTXFJ7PXAsFKEXMF64Jj9EE+pEiuwtxBnJU8U9 sdCw8WuP rrX8r/Zze2onJ2bmvruHoJztLYhwF4ObuAF9FsAkdPxwP6JWM4+HJ7nH5iks9oA3WbjiZfgCc+UnNGijXgkFYuvsQbax9ZhcCDELTNCf0yqa8AJSmEdOz1mRfCYExsNbO+UFUTJcIDOQ+TFWdcUNVm9RQOgyQTTivp9DMj3g8x4NbcM+tBepK9Tg6RXjoVx9Ro6j8xp8IL6PA+mACORDNnuhPxPlBclOUgkW0fCjB1/YeLvE7mHHHGOrnCA0AnZCvmOWO X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Instead of using 'cpu_to_node()', we use 'numa_node_id()', which is quicker. smp_processor_id is guaranteed to be stable in the 'mpol_misplaced()' function because it is called with ptl held. lockdep_assert_held was added to ensure that. No functional change in this patch. Signed-off-by: Aneesh Kumar K.V (IBM) Signed-off-by: Donet Tom --- include/linux/mempolicy.h | 5 +++-- mm/huge_memory.c | 2 +- mm/internal.h | 2 +- mm/memory.c | 8 +++++--- mm/mempolicy.c | 14 ++++++++++---- 5 files changed, 20 insertions(+), 11 deletions(-) diff --git a/include/linux/mempolicy.h b/include/linux/mempolicy.h index 931b118336f4..1add16f21612 100644 --- a/include/linux/mempolicy.h +++ b/include/linux/mempolicy.h @@ -167,7 +167,8 @@ extern void mpol_to_str(char *buffer, int maxlen, struct mempolicy *pol); /* Check if a vma is migratable */ extern bool vma_migratable(struct vm_area_struct *vma); -int mpol_misplaced(struct folio *, struct vm_area_struct *, unsigned long); +int mpol_misplaced(struct folio *folio, struct vm_fault *vmf, + unsigned long addr); extern void mpol_put_task_policy(struct task_struct *); static inline bool mpol_is_preferred_many(struct mempolicy *pol) @@ -282,7 +283,7 @@ static inline int mpol_parse_str(char *str, struct mempolicy **mpol) #endif static inline int mpol_misplaced(struct folio *folio, - struct vm_area_struct *vma, + struct vm_fault *vmf, unsigned long address) { return -1; /* no node preference */ diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 9859aa4f7553..b40bd9f3ead5 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1754,7 +1754,7 @@ vm_fault_t do_huge_pmd_numa_page(struct vm_fault *vmf) */ if (node_is_toptier(nid)) last_cpupid = folio_last_cpupid(folio); - target_nid = numa_migrate_prep(folio, vma, haddr, nid, &flags); + target_nid = numa_migrate_prep(folio, vmf, haddr, nid, &flags); if (target_nid == NUMA_NO_NODE) { folio_put(folio); goto out_map; diff --git a/mm/internal.h b/mm/internal.h index 7e486f2c502c..e0001c681c56 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -1088,7 +1088,7 @@ void vunmap_range_noflush(unsigned long start, unsigned long end); void __vunmap_range_noflush(unsigned long start, unsigned long end); -int numa_migrate_prep(struct folio *folio, struct vm_area_struct *vma, +int numa_migrate_prep(struct folio *folio, struct vm_fault *vmf, unsigned long addr, int page_nid, int *flags); void free_zone_device_page(struct page *page); diff --git a/mm/memory.c b/mm/memory.c index f2bc6dd15eb8..29e240978f45 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -5033,9 +5033,11 @@ static vm_fault_t do_fault(struct vm_fault *vmf) return ret; } -int numa_migrate_prep(struct folio *folio, struct vm_area_struct *vma, +int numa_migrate_prep(struct folio *folio, struct vm_fault *vmf, unsigned long addr, int page_nid, int *flags) { + struct vm_area_struct *vma = vmf->vma; + folio_get(folio); /* Record the current PID acceesing VMA */ @@ -5047,7 +5049,7 @@ int numa_migrate_prep(struct folio *folio, struct vm_area_struct *vma, *flags |= TNF_FAULT_LOCAL; } - return mpol_misplaced(folio, vma, addr); + return mpol_misplaced(folio, vmf, addr); } static vm_fault_t do_numa_page(struct vm_fault *vmf) @@ -5121,7 +5123,7 @@ static vm_fault_t do_numa_page(struct vm_fault *vmf) last_cpupid = (-1 & LAST_CPUPID_MASK); else last_cpupid = folio_last_cpupid(folio); - target_nid = numa_migrate_prep(folio, vma, vmf->address, nid, &flags); + target_nid = numa_migrate_prep(folio, vmf, vmf->address, nid, &flags); if (target_nid == NUMA_NO_NODE) { folio_put(folio); goto out_map; diff --git a/mm/mempolicy.c b/mm/mempolicy.c index 0fe77738d971..aa48376e2d34 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -2718,7 +2718,7 @@ static void sp_free(struct sp_node *n) * mpol_misplaced - check whether current folio node is valid in policy * * @folio: folio to be checked - * @vma: vm area where folio mapped + * @vmf: structure describing the fault * @addr: virtual address in @vma for shared policy lookup and interleave policy * * Lookup current policy node id for vma,addr and "compare to" folio's @@ -2728,18 +2728,24 @@ static void sp_free(struct sp_node *n) * Return: NUMA_NO_NODE if the page is in a node that is valid for this * policy, or a suitable node ID to allocate a replacement folio from. */ -int mpol_misplaced(struct folio *folio, struct vm_area_struct *vma, +int mpol_misplaced(struct folio *folio, struct vm_fault *vmf, unsigned long addr) { struct mempolicy *pol; pgoff_t ilx; struct zoneref *z; int curnid = folio_nid(folio); + struct vm_area_struct *vma = vmf->vma; int thiscpu = raw_smp_processor_id(); - int thisnid = cpu_to_node(thiscpu); + int thisnid = numa_node_id(); int polnid = NUMA_NO_NODE; int ret = NUMA_NO_NODE; + /* + * Make sure ptl is held so that we don't preempt and we + * have a stable smp processor id + */ + lockdep_assert_held(vmf->ptl); pol = get_vma_policy(vma, addr, folio_order(folio), &ilx); if (!(pol->flags & MPOL_F_MOF)) goto out; @@ -2781,7 +2787,7 @@ int mpol_misplaced(struct folio *folio, struct vm_area_struct *vma, if (node_isset(curnid, pol->nodes)) goto out; z = first_zones_zonelist( - node_zonelist(numa_node_id(), GFP_HIGHUSER), + node_zonelist(thisnid, GFP_HIGHUSER), gfp_zone(GFP_HIGHUSER), &pol->nodes); polnid = zone_to_nid(z->zone); From patchwork Thu Mar 21 11:29:51 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Donet Tom X-Patchwork-Id: 13598667 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id AC17AC54E68 for ; Thu, 21 Mar 2024 11:30:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 06DA16B0092; Thu, 21 Mar 2024 07:30:52 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id F38716B0093; Thu, 21 Mar 2024 07:30:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id D88E06B0095; Thu, 21 Mar 2024 07:30:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id CA5506B0092 for ; Thu, 21 Mar 2024 07:30:51 -0400 (EDT) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 98D031A0A49 for ; Thu, 21 Mar 2024 11:30:51 +0000 (UTC) X-FDA: 81920829102.02.5CB54BE Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf12.hostedemail.com (Postfix) with ESMTP id 3E9EE40005 for ; Thu, 21 Mar 2024 11:30:48 +0000 (UTC) Authentication-Results: imf12.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=UGM3RtES; spf=pass (imf12.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1711020649; a=rsa-sha256; cv=none; b=i3+khB0xpeV3Hr1DTrUHXiZdz5aM4MYj1fbvj5TnbxvGmMwciZIg7kJw6LZxxeztnEZx9V 6qHJx5zDDZQIGXkWD8wMVKgHXnuHRAJ/YCotu6cK9WF2m+Ucfs3UTr49o0RWqif6gstwTf mx0lRp5CdJAbRgRFqHShO6ths12euCQ= ARC-Authentication-Results: i=1; imf12.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=UGM3RtES; spf=pass (imf12.hostedemail.com: domain of donettom@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=donettom@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1711020649; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=X3hz2VPzVZOEkd8yWvn5oS5lDp/KJJL1DQQq8W3Frm4=; b=e8QfyRzJbKSrlXF5U26m7tCpej6IUmQHnaho8eT+bVNS4TP8gVs9ZawNHfz/qsyUm2N4CE A2HaVNWphaLRsi0NBpkfUvuBSKnphKH0iTI7U2MzkIcV8isntMwIN1AV8sYrzUYsY45Zkc oIT+ORPIjnM1w2t7FaDwzqgghYyv610= Received: from pps.filterd (m0353726.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 42LBCqGt001660; Thu, 21 Mar 2024 11:30:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=X3hz2VPzVZOEkd8yWvn5oS5lDp/KJJL1DQQq8W3Frm4=; b=UGM3RtESnozu5OiOcI3+fihab4r9+/RXiL01seVABSbuemgeCUbPkN6nOGK5VEy5+/QQ BgONPZWPXOw3myg+9nQ7BLocKn+AnsfsSl4FV2E1tCNp5Kn1ckphV8+4C9SY5tBpLjjt IasiT5E1khMuxX2o9fqFO6jvYKr19XXVUJftmy9lDk6Hmf40MjpKWMHA8upEUSTrpSqc H0MqiuMK/D+3Nkj0Tm9I1JAywQLPSR0S//04QAIHiU4CKixwx87pSOK2jtZXAOFnj60T bIJPQ0HEXXJ3JirrpNzxKFuH0o7+cnoYvq4DZHHDiVVg8PBcIKBLafKqzkXq8G8mgSEs zQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3x0ktd81hw-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Mar 2024 11:30:37 +0000 Received: from m0353726.ppops.net (m0353726.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 42LBUapA000324; Thu, 21 Mar 2024 11:30:36 GMT Received: from ppma22.wdc07v.mail.ibm.com (5c.69.3da9.ip4.static.sl-reverse.com [169.61.105.92]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3x0ktd81hs-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Mar 2024 11:30:36 +0000 Received: from pps.filterd (ppma22.wdc07v.mail.ibm.com [127.0.0.1]) by ppma22.wdc07v.mail.ibm.com (8.17.1.19/8.17.1.19) with ESMTP id 42L8qijm015792; Thu, 21 Mar 2024 11:30:35 GMT Received: from smtprelay06.fra02v.mail.ibm.com ([9.218.2.230]) by ppma22.wdc07v.mail.ibm.com (PPS) with ESMTPS id 3wwp50cvkm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 21 Mar 2024 11:30:35 +0000 Received: from smtpav02.fra02v.mail.ibm.com (smtpav02.fra02v.mail.ibm.com [10.20.54.101]) by smtprelay06.fra02v.mail.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 42LBUVr716318820 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Mar 2024 11:30:33 GMT Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 3EE342006A; Thu, 21 Mar 2024 11:30:31 +0000 (GMT) Received: from smtpav02.fra02v.mail.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 1A8752005A; Thu, 21 Mar 2024 11:30:28 +0000 (GMT) Received: from ltczz402-lp1.aus.stglabs.ibm.com (unknown [9.53.171.174]) by smtpav02.fra02v.mail.ibm.com (Postfix) with ESMTP; Thu, 21 Mar 2024 11:30:27 +0000 (GMT) From: Donet Tom To: Andrew Morton , linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Aneesh Kumar , Huang Ying , Michal Hocko , Dave Hansen , Mel Gorman , Feng Tang , Andrea Arcangeli , Peter Zijlstra , Ingo Molnar , Rik van Riel , Johannes Weiner , Matthew Wilcox , Vlastimil Babka , Dan Williams , Hugh Dickins , Kefeng Wang , Suren Baghdasaryan , Donet Tom Subject: [PATCH v3 2/2] mm/numa_balancing:Allow migrate on protnone reference with MPOL_PREFERRED_MANY policy Date: Thu, 21 Mar 2024 06:29:51 -0500 Message-Id: X-Mailer: git-send-email 2.39.3 In-Reply-To: References: MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: 6tqmExyYJedqdWzycIOEnWNwKfC9t0vE X-Proofpoint-GUID: i0C2nB4_Vyn4lJwcM9z3m0RrA2GKLa-C X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26 definitions=2024-03-21_08,2024-03-18_03,2023-05-22_02 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 clxscore=1015 mlxscore=0 adultscore=0 impostorscore=0 malwarescore=0 mlxlogscore=999 phishscore=0 bulkscore=0 priorityscore=1501 suspectscore=0 lowpriorityscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2403140000 definitions=main-2403210080 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 3E9EE40005 X-Stat-Signature: p85oezzc8kwy39rkcn7gobqgytaee49o X-Rspam-User: X-HE-Tag: 1711020648-810923 X-HE-Meta: U2FsdGVkX18SAzmwVBZjRIqxcPWY6+XaIutcZr9H4aF9Ba90cpjuEWlJaOeeLF9mntbfoui04o8FisM8lGUpscUF4f5D7y8crRA1M/+McWv9oV5DxEdha3wjqu2qlUaj2pxypyEBJtvgDdXMdkeyqifiRMHnOMMWb4+50R5Kj1b9vIax4Yg/tzD1PKy8PzY9lv+gtmJfgUR8xJ4sF9jwYLKMZYmJmH2wR2djmY+6b/YGru+pZ1nw1ATzUTlFlqTkdJJGBBTMuHfFKmGkfu8PbHvljiHn5giA6WoGshPOy2jPNkpYBLYexTOJAcbbU86yggzaKKy+eFZUzgNoP0NwzA/YmGkrP4Lxgrtq0o4gykbVRRr/aLzzOVG3T93CahcaCHv9x15ly2LvvOlAbYFI5JOKHWIE8S3y6ma/7BiF+rR91KXJEvPMtitgNWVjMUF9JLJD5MUi2HwnDY2w2/Pl+tHUOBv0BKGMtMe7S/PBp028fr60j+ZoHvHWk0rn3n27cKHrHd0t02lMq6GUBEe7Lq5M7P6ZtAD0pgZd3mlSQIBb9txhcIf3YoccgEdmzW1OzKyqqadmBnnnS9pJbjDAD2nWurrhoWwo4yg92O+92Zd2PuDSNQ7AFUq2GaKVPFAgRkPrQblJOOyoC6g3YTjTutqFQMNpqy+PtzHT2v/K2BOSbGP+Wl0MHyczDu6eP+b2EgU7sls0qnO3zTIGcX8S9Te6J8+PndEJA8DdPbcBY3CEIUxh5Cv8G77X1YsjU8vwwP232gLcQjaftE2/VXYFeKcpmSxC7duLwr9PeyouDBOYyf2RLFX7EDJFD8K9hlTqZTNkI5qww+yfl3vKmHAtnqJsLCXxLwgeh0g7290kpuFjEMpr4t9/4Yvxfw7WwMhdCv4ykp6FH9cGs8RdqFcfK+Ve4rsZBO55lC36TGCfZvURW5kWFQDoqS8f4zulS2chucB0/QCEL4hUiKb5QfD k+NUqH/G r7eK4udXfo8C101K+AlX7Dz42HE2+l+wNFH98Xo4hCL08QHL+ohGpulEnhA+Olt5ptVrzgJSP7GXyOAUV9vZP8nNrSgrtrGYJY54DB0OEoJ7I5xiSwsRUPdp2NZz6llwPVZbJh+o0W4LZF4gGxpxDuz08ysDNEZeaHEZ5VxpDD3IQ37/31iRhBb7Z758lQIAp3ML57zAw/C+nufy4a2qwfn94fODuCKJSGUcpYTmolnAHAS0yHmc+5St5wlSLu4sM6S8U X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: commit bda420b98505 ("numa balancing: migrate on fault among multiple bound nodes") added support for migrate on protnone reference with MPOL_BIND memory policy. This allowed numa fault migration when the executing node is part of the policy mask for MPOL_BIND. This patch extends migration support to MPOL_PREFERRED_MANY policy. Currently, we cannot specify MPOL_PREFERRED_MANY with the mempolicy flag MPOL_F_NUMA_BALANCING. This causes issues when we want to use NUMA_BALANCING_MEMORY_TIERING. To effectively use the slow memory tier, the kernel should not allocate pages from the slower memory tier via allocation control zonelist fallback. Instead, we should move cold pages from the faster memory node via memory demotion. For a page allocation, kswapd is only woken up after we try to allocate pages from all nodes in the allocation zone list. This implies that, without using memory policies, we will end up allocating hot pages in the slower memory tier. MPOL_PREFERRED_MANY was added by commit b27abaccf8e8 ("mm/mempolicy: add MPOL_PREFERRED_MANY for multiple preferred nodes") to allow better allocation control when we have memory tiers in the system. With MPOL_PREFERRED_MANY, the user can use a policy node mask consisting only of faster memory nodes. When we fail to allocate pages from the faster memory node, kswapd would be woken up, allowing demotion of cold pages to slower memory nodes. With the current kernel, such usage of memory policies implies we can't do page promotion from a slower memory tier to a faster memory tier using numa fault. This patch fixes this issue. For MPOL_PREFERRED_MANY, if the executing node is in the policy node mask, we allow numa migration to the executing nodes. If the executing node is not in the policy node mask, we do not allow numa migration. Signed-off-by: Aneesh Kumar K.V (IBM) Signed-off-by: Donet Tom --- mm/mempolicy.c | 22 +++++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/mm/mempolicy.c b/mm/mempolicy.c index aa48376e2d34..13100a290918 100644 --- a/mm/mempolicy.c +++ b/mm/mempolicy.c @@ -1504,9 +1504,10 @@ static inline int sanitize_mpol_flags(int *mode, unsigned short *flags) if ((*flags & MPOL_F_STATIC_NODES) && (*flags & MPOL_F_RELATIVE_NODES)) return -EINVAL; if (*flags & MPOL_F_NUMA_BALANCING) { - if (*mode != MPOL_BIND) + if (*mode == MPOL_BIND || *mode == MPOL_PREFERRED_MANY) + *flags |= (MPOL_F_MOF | MPOL_F_MORON); + else return -EINVAL; - *flags |= (MPOL_F_MOF | MPOL_F_MORON); } return 0; } @@ -2770,15 +2771,26 @@ int mpol_misplaced(struct folio *folio, struct vm_fault *vmf, break; case MPOL_BIND: - /* Optimize placement among multiple nodes via NUMA balancing */ + case MPOL_PREFERRED_MANY: + /* + * Even though MPOL_PREFERRED_MANY can allocate pages outside + * policy nodemask we don't allow numa migration to nodes + * outside policy nodemask for now. This is done so that if we + * want demotion to slow memory to happen, before allocating + * from some DRAM node say 'x', we will end up using a + * MPOL_PREFERRED_MANY mask excluding node 'x'. In such scenario + * we should not promote to node 'x' from slow memory node. + */ if (pol->flags & MPOL_F_MORON) { + /* + * Optimize placement among multiple nodes + * via NUMA balancing + */ if (node_isset(thisnid, pol->nodes)) break; goto out; } - fallthrough; - case MPOL_PREFERRED_MANY: /* * use current page if in policy nodemask, * else select nearest allowed node, if any.