From patchwork Fri Jul 29 06:13:45 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Aneesh Kumar K.V" X-Patchwork-Id: 12931951 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 23EB6C00144 for ; Fri, 29 Jul 2022 06:14:35 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B19B8940007; Fri, 29 Jul 2022 02:14:34 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A54276B0075; Fri, 29 Jul 2022 02:14:34 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 8CD1F940007; Fri, 29 Jul 2022 02:14:34 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8004E6B0074 for ; Fri, 29 Jul 2022 02:14:34 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 5D6C81C70AE for ; Fri, 29 Jul 2022 06:14:34 +0000 (UTC) X-FDA: 79739123268.04.18AD4FE Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) by imf16.hostedemail.com (Postfix) with ESMTP id B21211800DD for ; Fri, 29 Jul 2022 06:14:33 +0000 (UTC) Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 26T6Bqr4019746; Fri, 29 Jul 2022 06:14:22 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : mime-version : content-transfer-encoding; s=pp1; bh=02Eyb4zC1pVNeQEbmkNdkdisYnA/zoMQr6Y9Hk5QTQM=; b=rkMvZdQkrL3LUkKzb0BpHoYZIl/bHBMmmYfae/2IG6nJLq30G367boWs+ColH+M1uL6n KX6JTWetBbfVzmivX3GLYELNLS1tjlZB6xu/d0rxJeSLjma/gexWpksLXS+PYsoyoJ/a rtuwJUI3oyoY4dq/3cSWBjQemE4GRoHxuDaaix9xO7P5dEfzHzNjDuzXjponSo9bMaxS dnHpYerrEHb4yq7+kZ/oULKg9cdl6xuk/NWfugOrVgnSzolCgpRNwclO1HAznOvCssaR S6A54h3Yh7f2jXABeN8w/Y0e6OW4jup4Q2wVDrc9k1JC6UA41jPgIrf8n5mQ4jBAl9+D zQ== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3hma2m01yx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 29 Jul 2022 06:14:22 +0000 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.17.1.5/8.17.1.5) with ESMTP id 26T6Dcet030574; Fri, 29 Jul 2022 06:14:21 GMT Received: from ppma02dal.us.ibm.com (a.bd.3ea9.ip4.static.sl-reverse.com [169.62.189.10]) by mx0a-001b2d01.pphosted.com (PPS) with ESMTPS id 3hma2m01y0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 29 Jul 2022 06:14:21 +0000 Received: from pps.filterd (ppma02dal.us.ibm.com [127.0.0.1]) by ppma02dal.us.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 26T65oDB026532; Fri, 29 Jul 2022 06:14:20 GMT Received: from b03cxnp08028.gho.boulder.ibm.com (b03cxnp08028.gho.boulder.ibm.com [9.17.130.20]) by ppma02dal.us.ibm.com with ESMTP id 3hhfpj9dnu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 29 Jul 2022 06:14:20 +0000 Received: from b03ledav004.gho.boulder.ibm.com (b03ledav004.gho.boulder.ibm.com [9.17.130.235]) by b03cxnp08028.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 26T6EJ2p36372880 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 29 Jul 2022 06:14:19 GMT Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 63DE07805E; Fri, 29 Jul 2022 06:14:19 +0000 (GMT) Received: from b03ledav004.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 8C9767805C; Fri, 29 Jul 2022 06:14:14 +0000 (GMT) Received: from skywalker.ibmuc.com (unknown [9.43.86.244]) by b03ledav004.gho.boulder.ibm.com (Postfix) with ESMTP; Fri, 29 Jul 2022 06:14:14 +0000 (GMT) From: "Aneesh Kumar K.V" To: linux-mm@kvack.org, akpm@linux-foundation.org Cc: Wei Xu , Huang Ying , Yang Shi , Davidlohr Bueso , Tim C Chen , Michal Hocko , Linux Kernel Mailing List , Hesham Almatary , Dave Hansen , Jonathan Cameron , Alistair Popple , Dan Williams , Johannes Weiner , jvgediya.oss@gmail.com, "Aneesh Kumar K.V" Subject: [PATCH v12 4/8] mm/demotion/dax/kmem: Set node's abstract distance to MEMTIER_ADISTANCE_PMEM Date: Fri, 29 Jul 2022 11:43:45 +0530 Message-Id: <20220729061349.968148-5-aneesh.kumar@linux.ibm.com> X-Mailer: git-send-email 2.37.1 In-Reply-To: <20220729061349.968148-1-aneesh.kumar@linux.ibm.com> References: <20220729061349.968148-1-aneesh.kumar@linux.ibm.com> MIME-Version: 1.0 X-TM-AS-GCONF: 00 X-Proofpoint-GUID: S54XRBUG3nolwtZxUGjiDjNOXeUQNSLa X-Proofpoint-ORIG-GUID: Mxz7pcW5NKGeFFBti6nnZaJmV-zOgp4q X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.883,Hydra:6.0.517,FMLib:17.11.122.1 definitions=2022-07-28_06,2022-07-28_02,2022-06-22_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1015 phishscore=0 priorityscore=1501 mlxscore=0 adultscore=0 bulkscore=0 lowpriorityscore=0 impostorscore=0 spamscore=0 suspectscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2206140000 definitions=main-2207290023 ARC-Authentication-Results: i=1; imf16.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=rkMvZdQk; spf=pass (imf16.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1659075273; a=rsa-sha256; cv=none; b=v4GrtkUkWJExwZ6njhqlB/0cunmntiUXfk8PedVNlQ9meClbYCVymP9WdRNRF/793MJf1q 6QDPph/MkWQce9KO8Vxs1XYYywf74fNlPu3+Br6Pw6Dss94QNiRCibFXnW07LhTHwTBMkY R//I5BWUBAkoRdsLkfttXg8wmxKXVoA= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1659075273; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=02Eyb4zC1pVNeQEbmkNdkdisYnA/zoMQr6Y9Hk5QTQM=; b=FPaV2EcsWahTnbBpaj2h6ckzK8K1tholgYKhiYM59Z1ZPJliIvoogzYrv4RwSgbLx4REIR JKi9QFxgTTuwEfvfa4oWV8t9BDy2SDD8Y0KLEdtxO8QYlMR+4FVyCnvC9a5RA4P5t0uYYO g186EIUL50WXxFF6myS3uGgcJqT/ILo= Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=ibm.com header.s=pp1 header.b=rkMvZdQk; spf=pass (imf16.hostedemail.com: domain of aneesh.kumar@linux.ibm.com designates 148.163.156.1 as permitted sender) smtp.mailfrom=aneesh.kumar@linux.ibm.com; dmarc=pass (policy=none) header.from=ibm.com X-Rspam-User: X-Rspamd-Server: rspam05 X-Rspamd-Queue-Id: B21211800DD X-Stat-Signature: 418s7rssere6iyed1nxg4uod3bg3hfb1 X-HE-Tag: 1659075273-228285 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: By default, all nodes are assigned to the default memory tier which is the memory tier designated for nodes with DRAM Set dax kmem device node's tier to slower memory tier by assigning abstract distance to MEMTIER_ADISTANCE_PMEM. PMEM tier appears below the default memory tier in demotion order. Signed-off-by: Aneesh Kumar K.V --- drivers/dax/kmem.c | 9 +++++++++ include/linux/memory-tiers.h | 19 ++++++++++++++++++- mm/memory-tiers.c | 28 ++++++++++++++++------------ 3 files changed, 43 insertions(+), 13 deletions(-) diff --git a/drivers/dax/kmem.c b/drivers/dax/kmem.c index a37622060fff..6b0d5de9a3e9 100644 --- a/drivers/dax/kmem.c +++ b/drivers/dax/kmem.c @@ -11,6 +11,7 @@ #include #include #include +#include #include "dax-private.h" #include "bus.h" @@ -41,6 +42,12 @@ struct dax_kmem_data { struct resource *res[]; }; +static struct memory_dev_type default_pmem_type = { + .adistance = MEMTIER_ADISTANCE_PMEM, + .tier_sibiling = LIST_HEAD_INIT(default_pmem_type.tier_sibiling), + .nodes = NODE_MASK_NONE, +}; + static int dev_dax_kmem_probe(struct dev_dax *dev_dax) { struct device *dev = &dev_dax->dev; @@ -62,6 +69,8 @@ static int dev_dax_kmem_probe(struct dev_dax *dev_dax) return -EINVAL; } + init_node_memory_type(numa_node, &default_pmem_type); + for (i = 0; i < dev_dax->nr_range; i++) { struct range range; diff --git a/include/linux/memory-tiers.h b/include/linux/memory-tiers.h index eeb4b045e631..c25f385ff9a8 100644 --- a/include/linux/memory-tiers.h +++ b/include/linux/memory-tiers.h @@ -2,6 +2,8 @@ #ifndef _LINUX_MEMORY_TIERS_H #define _LINUX_MEMORY_TIERS_H +#include +#include /* * Each tier cover a abstrace distance chunk size of 128 */ @@ -14,12 +16,27 @@ #define MEMTIER_ADISTANCE_PMEM (1 << (MEMTIER_CHUNK_BITS + 3)) #define MEMTIER_HOTPLUG_PRIO 100 +struct memory_tier; +struct memory_dev_type { + /* list of memory types that are are part of same tier as this type */ + struct list_head tier_sibiling; + /* abstract distance for this specific memory type */ + int adistance; + /* Nodes of same abstract distance */ + nodemask_t nodes; + struct memory_tier *memtier; +}; + #ifdef CONFIG_NUMA -#include extern bool numa_demotion_enabled; +struct memory_dev_type *init_node_memory_type(int node, struct memory_dev_type *default_type); #else #define numa_demotion_enabled false +static inline struct memory_dev_type *init_node_memory_type(int node, struct memory_dev_type *default_type) +{ + return ERR_PTR(-EINVAL); +} #endif /* CONFIG_NUMA */ #endif /* _LINUX_MEMORY_TIERS_H */ diff --git a/mm/memory-tiers.c b/mm/memory-tiers.c index 0a7a79cad2a0..0c4394975d1d 100644 --- a/mm/memory-tiers.c +++ b/mm/memory-tiers.c @@ -1,6 +1,4 @@ // SPDX-License-Identifier: GPL-2.0 -#include -#include #include #include #include @@ -21,16 +19,6 @@ struct memory_tier { int adistance_start; }; -struct memory_dev_type { - /* list of memory types that are are part of same tier as this type */ - struct list_head tier_sibiling; - /* abstract distance for this specific memory type */ - int adistance; - /* Nodes of same abstract distance */ - nodemask_t nodes; - struct memory_tier *memtier; -}; - static DEFINE_MUTEX(memory_tier_lock); static LIST_HEAD(memory_tiers); struct memory_dev_type *node_memory_types[MAX_NUMNODES]; @@ -143,6 +131,22 @@ static void clear_node_memory_tier(int node) mutex_unlock(&memory_tier_lock); } +struct memory_dev_type *init_node_memory_type(int node, struct memory_dev_type *default_type) +{ + struct memory_dev_type *mem_type; + + mutex_lock(&memory_tier_lock); + if (node_memory_types[node]) { + mem_type = node_memory_types[node]; + } else { + node_memory_types[node] = default_type; + node_set(node, default_type->nodes); + mem_type = default_type; + } + mutex_unlock(&memory_tier_lock); + return mem_type; +} + static int __meminit memtier_hotplug_callback(struct notifier_block *self, unsigned long action, void *_arg) {