From patchwork Wed Feb 14 03:53:55 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Byungchul Park X-Patchwork-Id: 13555908 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 26ACBC4829A for ; Wed, 14 Feb 2024 03:54:16 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 5E45D6B0093; Tue, 13 Feb 2024 22:54:15 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 591606B0095; Tue, 13 Feb 2024 22:54:15 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 40A746B0096; Tue, 13 Feb 2024 22:54:15 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 2CF2E6B0093 for ; Tue, 13 Feb 2024 22:54:15 -0500 (EST) Received: from smtpin08.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id C6A0EA21C1 for ; Wed, 14 Feb 2024 03:54:14 +0000 (UTC) X-FDA: 81789041628.08.B278645 Received: from invmail4.hynix.com (exvmail4.skhynix.com [166.125.252.92]) by imf20.hostedemail.com (Postfix) with ESMTP id 803C01C000C for ; Wed, 14 Feb 2024 03:54:10 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1707882853; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:content-type: content-transfer-encoding:in-reply-to:references; bh=8eVGciwhU8lCi+euJCCTtr4v1C2N5uLww2/wJOliNk0=; b=0r6k+T4UeYP+VwMQAebBBWv4RDHgYn4/9uTqSynIwJ98d4ASBE0cjfURu+ROdD571lH7e7 m17k25Uken9ut+JFsi+WKa8TbM09G3qDV7MBgpHZadcBtsxJ+iE4eL3P66jXPjhcSNKg1w ncL91DoJCFFcbNl0/E6kikgU6SKUzAM= ARC-Seal: i=1; s=arc-20220608; d=hostedemail.com; t=1707882853; a=rsa-sha256; cv=none; b=IbRTI0lEOeV0qid24sPsflQjTID/ccGtD8O6Y6+Tt2l0A+CFoE1yPpCj87bmqaTNJU2ck6 0LNClL8E/JjjvK5JTGdBGgeByDja+BvcZTHH+eQws5kOArKJVhWoS5j8KBO8bqmNhpBJgH haoaPiGzOzAjP9Iq3jWFtseF6gd9gAI= ARC-Authentication-Results: i=1; imf20.hostedemail.com; dkim=none; spf=pass (imf20.hostedemail.com: domain of byungchul@sk.com designates 166.125.252.92 as permitted sender) smtp.mailfrom=byungchul@sk.com; dmarc=none X-AuditID: a67dfc5b-d85ff70000001748-ae-65cc395d8c12 From: Byungchul Park To: mingo@redhat.com, peterz@infradead.org, juri.lelli@redhat.com, vincent.guittot@linaro.org, dietmar.eggemann@arm.com, rostedt@goodmis.org, bsegall@google.com, mgorman@suse.de, bristot@redhat.com, vschneid@redhat.com Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, kernel_team@skhynix.com, akpm@linux-foundation.org Subject: [PATCH] sched/numa, mm: do not promote folios to nodes not set N_MEMORY Date: Wed, 14 Feb 2024 12:53:55 +0900 Message-Id: <20240214035355.18335-1-byungchul@sk.com> X-Mailer: git-send-email 2.17.1 X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFlrNLMWRmVeSWpSXmKPExsXC9ZZnoW6c5ZlUg3N/WCzmrF/DZnHp8VU2 i+kvG1ksnk7Yymxxt38qi8XlXXPYLO6t+c9qMfndM0aLSwcWMFkc7z3AZLGv4wGTRceRb8wW W49+Z3fg9Vgzbw2jR8u+W+weCzaVemxeoeWx6dMkdo871/aweZyY8ZvF4/2+q2wem09Xe3ze JBfAFcVlk5Kak1mWWqRvl8CV8fbGOtaCE0oVp1riGxj/yHQxcnBICJhILF6Q2MXICWbumLee BcRmE1CXuHHjJ3MXIxeHiMAbRonOZWfYQBLMAnkSrf/7mEBsYYEAic33r4A1sAioSpzcvJwR xOYVMJWY0bKKFWKovMTqDQfABkkIrGGTOPz2B1RCUuLgihssExi5FzAyrGIUyswry03MzDHR y6jMy6zQS87P3cQIDMJltX+idzB+uhB8iFGAg1GJh/dEwulUIdbEsuLK3EOMEhzMSiK8l2ac SBXiTUmsrEotyo8vKs1JLT7EKM3BoiTOa/StPEVIID2xJDU7NbUgtQgmy8TBKdXAGNP3P+gG I3tl6mJpGcM9h2YvDruk9zlkhrL2or/79BtiL6RoVekd/sX/PrCxyvHEwcbTX3zY8uYJljJO XJLUf6OlI3nu/WzZnoXLLtY84vl858db/jV6bXmZuZsEBG+EP0iNKp/DO8FnmqG0D//9zCbn kwt3T2Aocjun3vtulUWiZ858viUVSizFGYmGWsxFxYkA/IN2JD4CAAA= X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprMLMWRmVeSWpSXmKPExsXC5WfdrBtreSbVoPOGicWc9WvYLC49vspm Mf1lI4vF0wlbmS3u9k9lsTg89ySrxeVdc9gs7q35z2ox+d0zRotLBxYwWRzvPcBksa/jAZNF x5FvzBZbj35nd+DzWDNvDaNHy75b7B4LNpV6bF6h5bHp0yR2jzvX9rB5nJjxm8Xj/b6rbB6L X3xg8th8utrj8ya5AO4oLpuU1JzMstQifbsEroy3N9axFpxQqjjVEt/A+Eemi5GTQ0LARGLH vPUsIDabgLrEjRs/mbsYuThEBN4wSnQuO8MGkmAWyJNo/d/HBGILCwRIbL5/BayBRUBV4uTm 5YwgNq+AqcSMllWsEEPlJVZvOMA8gZFjASPDKkaRzLyy3MTMHFO94uyMyrzMCr3k/NxNjMCg Wlb7Z+IOxi+X3Q8xCnAwKvHwnkg4nSrEmlhWXJl7iFGCg1lJhPfSjBOpQrwpiZVVqUX58UWl OanFhxilOViUxHm9wlMThATSE0tSs1NTC1KLYLJMHJxSDYz+B34vkI58+f+zIzP7jbsTlO4b bT16cbs/e3mwvGZKbIrAVamm6QsZ+jiXvpPmXmKcfePdgctVBRd0Bby6Jxcc/fTOk0XyhInM Thf/j3tvCfkEbD8de5nXUrhOfvnqtarTQvc5qFsEKD3WClh+62nWnbmOTHatYf8/CJz+xv7v 67YTJmf01J8qsRRnJBpqMRcVJwIA2sikGiYCAAA= X-CFilter-Loop: Reflected X-Rspamd-Queue-Id: 803C01C000C X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: w56yigws5kfdigj86cs8b6egt9i4mqgz X-HE-Tag: 1707882850-423918 X-HE-Meta: U2FsdGVkX19U4PKcu0LF1IEzu6jbg3OGUKXM7ZfYfUjO/Y+9MEfIjFXpEusFOTjAbqcjlv6FFHp4bO0748NqtnfjgDxPAnHUFNR+QReRuljTMPNCyBw2oJDkcRsFUGTH2YYgm3+xyK+G5hj6yEOGq2NfbCZG7+1z8JhfM5BnWx3knsrofmAnZBYzYf39ogp1w/VwFJBbL9tsCMP4zui0ywVyU92WXGpvge22NeFeA02nI7qi182L1K1WY3rzRDBlCl7jvnzzR/L99DU1LFet9Pe1aRw3DFrY3eVUwShoxsI5ix2j5NvamGbGPzmEusxdEtUpnGJJr3w4tdGh6MI2hCDuJPJeIvyFaLekGxxwmbsGAd4bZ7Nyhs3ra8uvJ9a3Jnb+zmdC3kfho19i69/+7WTEYB1iBDzA+4vfOiD8yZivzNf6fnjoSBGTWZ8IOkbiP8BoGJQ+KUfQaahY2yt+Pdxp1fwR4HeZjxDJqNOtXks9H3WvvWKRVaOeckpYdp95X0gLlwDqpyXkXP86XokHyoxnx4g77HrE+JhCLDTKzXzoRpQxnkXiBj8/5j7PAlfURtryLfBUFpyLGk3QeRrtt/C+dQ6PQblTFjEhYXSgo2MKeWFwrnklM1TFVwmbxhZAytPP46r1EMKsCc8Oc0xI8qI+6UDYfXItdspengnOGW4KdENxLBLOGJUK/SjAN8hAVEQSR1Zk58yDoJ3kAX2shbHlGX8ukgMMRtD5IbIzC6xTetmASF+FwKzKWP/ehy3W X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: While running qemu with a configuration where some CPUs don't have their local memory and with a kernel numa balancing on, the following oops has been observed. It's because of null pointers of ->zone_pgdat of zones of those nodes that are not initialized at booting time. So should avoid nodes not set N_MEMORY from getting promoted. > BUG: unable to handle page fault for address: 00000000000033f3 > #PF: supervisor read access in kernel mode > #PF: error_code(0x0000) - not-present page > PGD 0 P4D 0 > Oops: 0000 [#1] PREEMPT SMP NOPTI > CPU: 2 PID: 895 Comm: masim Not tainted 6.6.0-dirty #255 > Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS > rel-1.16.0-0-gd239552ce722-prebuilt.qemu.org 04/01/2014 > RIP: 0010:wakeup_kswapd (./linux/mm/vmscan.c:7812) > Code: (omitted) > RSP: 0000:ffffc90004257d58 EFLAGS: 00010286 > RAX: ffffffffffffffff RBX: ffff88883fff0480 RCX: 0000000000000003 > RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88883fff0480 > RBP: ffffffffffffffff R08: ff0003ffffffffff R09: ffffffffffffffff > R10: ffff888106c95540 R11: 0000000055555554 R12: 0000000000000003 > R13: 0000000000000000 R14: 0000000000000000 R15: ffff88883fff0940 > FS: 00007fc4b8124740(0000) GS:ffff888827c00000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 00000000000033f3 CR3: 000000026cc08004 CR4: 0000000000770ee0 > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 > PKRU: 55555554 > Call Trace: > > ? __die > ? page_fault_oops > ? __pte_offset_map_lock > ? exc_page_fault > ? asm_exc_page_fault > ? wakeup_kswapd > migrate_misplaced_page > __handle_mm_fault > handle_mm_fault > do_user_addr_fault > exc_page_fault > asm_exc_page_fault > RIP: 0033:0x55b897ba0808 > Code: (omitted) > RSP: 002b:00007ffeefa821a0 EFLAGS: 00010287 > RAX: 000055b89983acd0 RBX: 00007ffeefa823f8 RCX: 000055b89983acd0 > RDX: 00007fc2f8122010 RSI: 0000000000020000 RDI: 000055b89983acd0 > RBP: 00007ffeefa821a0 R08: 0000000000000037 R09: 0000000000000075 > R10: 0000000000000000 R11: 0000000000000202 R12: 0000000000000000 > R13: 00007ffeefa82410 R14: 000055b897ba5dd8 R15: 00007fc4b8340000 > > Modules linked in: > CR2: 00000000000033f3 > ---[ end trace 0000000000000000 ]--- > RIP: 0010:wakeup_kswapd (./linux/mm/vmscan.c:7812) > Code: (omitted) > RSP: 0000:ffffc90004257d58 EFLAGS: 00010286 > RAX: ffffffffffffffff RBX: ffff88883fff0480 RCX: 0000000000000003 > RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff88883fff0480 > RBP: ffffffffffffffff R08: ff0003ffffffffff R09: ffffffffffffffff > R10: ffff888106c95540 R11: 0000000055555554 R12: 0000000000000003 > R13: 0000000000000000 R14: 0000000000000000 R15: ffff88883fff0940 > FS: 00007fc4b8124740(0000) GS:ffff888827c00000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 00000000000033f3 CR3: 000000026cc08004 CR4: 0000000000770ee0 > DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 > DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 > PKRU: 55555554 > note: masim[895] exited with irqs disabled Signed-off-by: Byungchul Park Reported-by: hyeongtak.ji@sk.com --- kernel/sched/fair.c | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index d7a3c63a2171..6d215cc85f14 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -1828,6 +1828,23 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, int dst_nid = cpu_to_node(dst_cpu); int last_cpupid, this_cpupid; + /* + * A node of dst_nid might not have its local memory. Promoting + * a folio to the node is meaningless. What's even worse, oops + * can be observed by the null pointer of ->zone_pgdat in + * various points of the code during migration. + * + * For instance, oops has been observed at CPU2 while qemu'ing: + * + * {qemu} \ + * -numa node,nodeid=0,mem=1G,cpus=0-1 \ + * -numa node,nodeid=1,cpus=2-3 \ + * -numa node,nodeid=2,mem=8G \ + * ... + */ + if (!node_state(dst_nid, N_MEMORY)) + return false; + /* * The pages in slow memory node should be migrated according * to hot/cold instead of private/shared.