From patchwork Wed Mar 19 19:30:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Raghavendra K T X-Patchwork-Id: 14023060 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4047C35FFA for ; Wed, 19 Mar 2025 19:31:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0D023280002; Wed, 19 Mar 2025 15:31:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 05C4D280001; Wed, 19 Mar 2025 15:31:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEEF8280002; Wed, 19 Mar 2025 15:31:13 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id B8769280001 for ; Wed, 19 Mar 2025 15:31:13 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id B072E16033F for ; Wed, 19 Mar 2025 19:31:14 +0000 (UTC) X-FDA: 83239294068.01.7CA180F Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2071.outbound.protection.outlook.com [40.107.244.71]) by imf02.hostedemail.com (Postfix) with ESMTP id B565780005 for ; Wed, 19 Mar 2025 19:31:11 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=ivQI55qx; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf02.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.244.71 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1742412672; a=rsa-sha256; cv=pass; b=LMa8v7dXLq/i/t62p20lEgrYDHZTueZ2w1XPWmJruQS+lQEM8qq/pNrGbCbpsMJ9UqUWSi 0yQUbpDuol5d/DdQHmBu9OUWc8JVP57WlvwCpkLPJ0ZDPjfD/MkXYlZHxrjOMv8PD5gp3F ppQAKsglzjPg7W6B6SyZPmnTAA0tKqY= ARC-Authentication-Results: i=2; imf02.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=ivQI55qx; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf02.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.244.71 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742412672; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding:in-reply-to: references:dkim-signature; bh=jvikH11XOj5BBJEFuGMi6vlqefuiianXeQD7VPW3FBo=; b=1mBFw/k30ixXoM6We1bKYEuPUMO/6/enlXttsA1OFdvDy+3QA9pdeq5ijAvQOsya9q6pvR X52M3YjTE10kg+t8Xa2ilexIP7SJZ60p11Ih0oYc33ev7mS0LySaIfm7SvbBSaTqx8g5CD 8XBcy0kqan3CfQwXGWmT9PJC/nAj5dE= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Uy/KIsda+vfniXbu6LQvrk24IV57bDIbvWvEp/ef+oY80NKu6OstYWnnWeEGFAuRMbIXbFPUnofq9hteCMn5RegDl8N4S72P8it3WDslksZK+daPSikOfEe4LIBsHtJTCmDGyAJcnwN0gGwOxMCEF6bO+X2WfSpTzCxwAXHv4uw5kYx1TAiorn2OCzXIEyyCY1MNnHD2QYy+o2JSHNhreIHsWYF94IRRk/Y78pqnkHjeKbUVud6ClhYx9KaYjnr/biTrQ8wcClfuMNF7QgiV4xVXsUFNktapjBHkcteVsI5aBhYU/+X4w6O5vrqpmp2zBAYdgbUY3y/jqilnneQPpA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=jvikH11XOj5BBJEFuGMi6vlqefuiianXeQD7VPW3FBo=; b=bv0yPm3hsbBcyAm91+GdAUxPX7TQ2v83+dwSCEvs8y2HJDLb68WKhVMkTNZ4oNmM0BZFUjeYYzB2KtHcOBeCLoyxoDYPVOb3rA11+1SACZmEwQNL8Cuce7MhyhPccat9lbZ19zvOdnpL3BFoECW32m0Vq0Kabhcf8F81zg/XwoNHkGVI70bAGjGJ81n9QQAHqo70ohazuJuWhFlX5ioSkAFEjSwnAi/tmt6MVaQSfZ2D0iolz6ZzvJqMI1La6Jf7BaUoJk550TFwlzvhB3ZQg8Lsn/+F6YyzNxzHXf41yc0B3W2p6FhV3Hb1zEJmDSx8RJxC+k30lmJlreeilCAffQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=jvikH11XOj5BBJEFuGMi6vlqefuiianXeQD7VPW3FBo=; b=ivQI55qxYvmVH19SyICFBIugIq73rDw4XMUOEbgzcf7lfqO7zQFQGcx0GUYkgW1Rfc70Pw6NEMvUVeWWekycKUv5y+DPsksezAFXHXAKLBEYG//PknHcHGKkyT7gPpqbGH/PmJZquazPMTTNOw1KavmWmcyFEYoblka3Uw6a9jA= Received: from SA1P222CA0099.NAMP222.PROD.OUTLOOK.COM (2603:10b6:806:35e::13) by BN5PR12MB9537.namprd12.prod.outlook.com (2603:10b6:408:2a9::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8511.28; Wed, 19 Mar 2025 19:31:02 +0000 Received: from SN1PEPF00026368.namprd02.prod.outlook.com (2603:10b6:806:35e:cafe::52) by SA1P222CA0099.outlook.office365.com (2603:10b6:806:35e::13) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.26 via Frontend Transport; Wed, 19 Mar 2025 19:31:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SN1PEPF00026368.mail.protection.outlook.com (10.167.241.133) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Wed, 19 Mar 2025 19:31:02 +0000 Received: from tunga.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 19 Mar 2025 14:30:52 -0500 From: Raghavendra K T To: CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH V1 00/13] mm: slowtier page promotion based on PTE A bit Date: Wed, 19 Mar 2025 19:30:15 +0000 Message-ID: <20250319193028.29514-1-raghavendra.kt@amd.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF00026368:EE_|BN5PR12MB9537:EE_ X-MS-Office365-Filtering-Correlation-Id: 1c64d886-12fb-402d-a810-08dd671c95b0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|7416014|1800799024|376014|82310400026|13003099007; X-Microsoft-Antispam-Message-Info: SwHzqUGwjf8nsxBKNh7EIaJFog8x36h5FMrleDOpw1QD/qQB+L6A4MXDl0a/u4wIBPOXVugY0C3o9jbtyIToXWmU/jrlCf/fbL6RBIMXckaO59kU+oxdL4tZMs0n1CogZpH27N0P8+5/GS4AoF7GSy0GNIoJ8p189h7O/XCMNtZUGPXZf6oybmP17JfcdDcybTv1aXoehRXZZk6U/bZ1/GhOhETsizSUIIMYwP22PTkr4T7CTwZBeX31Bv9KS0GjTJ6T8KQte8rDuDRQbGtjbuGerWXO/nXa8W+AyHZpCj/sNuuipAVr2tuWfHuVa08dlCzQhndez7xfm5i7cTD+vRJh/tOfFNhJT8nMuxi695mEoX7oPyw0slW8dr13H91Ydd7vkFT8O8KJPn9f2EsR0O1xE8rBGuzgIMvYELRsonNcCSMIJLMBKf9KicDP7jYgmUWyRy5Lbg6323dtgdjlMchea4TlVyTHhWmaGJKSJdkqQLKphZD9BIvT2CR50zQrqyvwag91fLOWBtuoSx7XlMhMt4/nUObduouhOoIfptqmX/M1we8yDFf7rdtsIwUp1hC/W1YaL4Ll2zaUfoIFmc1pkvsa/LBlVqIsQrzC0r60gArUmKF9jUm9cczxH3H+0nsC2bH+s+UE/Atu5ggCeG4kywO3B5kVPAlTr3eqgltBJWirabhut2pw+r95G8IMJpjtOa9PMoEBVh4LRuMWurpbnrI2o7rzNCSaza0iS2HmH0dAw1MYfgeZ2SmsmqFfrj1Pdx8km7YHKtksSQKc1o9Me84Hl5sObLJAimI3ihEnsuRlyhjzgL2a2G2pq9jgZRLfHSDR8+4r23wiRAvuAsY7hvM1lDfd4qrOYYLE/lVW7Y8GHsfMqydIRgx42ZsastjztLn86PiJ7I3xrAxtr1GVXLxRU+gMyuoPuC1cOS+bfpf/78bd/g4+Sg8VAbsqZ5nKVjTCprq4dsNcKrthx/YJLo2Le29jYNQUzSGRHRwfiiyDTtOQcN6uny/sqgDbCYXw/1h5K9nYMi3EX4FnzU93iPXzwnhVKQncNyUmCgMfJ8cQAE++7FTdwRt5kE0MxZunvpvchygUyjAxdfaQCg8XQhSwG1Yk3xlh7JoxFkrTlL8mtT423vdWv7hN3NHUC8T9S5bPDcSY1FLn2BdFu2fwusRPNY6XZwlOOCF3+u3UcIMDOQ3+Ly3ZtCO91Pl4Cro0OnA6HSg+dCKCJd5KdZNPrMA8dnS58DoBRw3u//LiCv+WxBFY/ZNov78K0WB8XYDbBGes8OPT1AIvEvpl6GUxOn1gQ0bwgwz+cyogQZM6XGJ46/tEKoE/UyJhaS8ZA0bbJ5ekCz7W3xJIAgw4jpvRjL2rZlF6RBn55jHQIGjQ18nFrmssCZkEbZC1p23qzv3LIhVatjMuPB98KK1esfQZyJ3q05JMusmlOWUrO825tPA8IbtA+dSQNIweM+T7 X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(36860700013)(7416014)(1800799024)(376014)(82310400026)(13003099007);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2025 19:31:02.3981 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1c64d886-12fb-402d-a810-08dd671c95b0 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF00026368.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN5PR12MB9537 X-Rspam-User: X-Rspamd-Queue-Id: B565780005 X-Stat-Signature: 1nkfiku1c6fdit4k7wwuwhss4sjqt7e4 X-Rspamd-Server: rspam06 X-HE-Tag: 1742412671-711728 X-HE-Meta: U2FsdGVkX181qw6mQxWDjlIt500V1m8P2COXrBOeWCyVtyAD/4op8ffSdne8HEOuxf7MR9itlq3fEvqge+l7//X/94hVRdBoqrTgn/Ddo1RY2TxQqrOd20YqvVH3MW0tSY3UI28/SPf8+DRlzDWUf8W5jyhwkPBwi9a88bDDtH+vfw1avgdIN9A7Y6DsclKPAxEllZ5xJqXBoMsuv/KmWAff0gktQqw9+SOcaxdGUOkWAG/IsVoTncTjjuiDE68LPv+0yX4AABb70BCphdW2v5TwCCJ8LE0J0vLm/mCEsadqrZtlLxOhiH5uz0imUhNzChcQZ7/UAZgSq20Gu98ThenJC+jxhBkazFJp/QnTr/fUvM2JiNEmYim9Kt8XMsKHyprL5Rs7Uu/Wt7QahAZBl9ipVsmHsJ9gPVCsXimiQ/z7kNg5RkT6iQdyo79oOkZU5IsxlVAEcNDFIjzLKS9cYL0hHNy3FH/yT9paE7TqZ3rdkvT3nKc2F3k5g+DI3yUM5e18Jpr3m2YjO+SfvhWtatUDHx9ZSU6nrYvJskrqpUn3fA7c32yYNn0Jif6lRofFoPbb1DrUMVCe6E6uP4IBosv+t6yWCnt5o49WF7+/3UAlvIlhqEsB7m5z+cmIOULuuRff9Yq4XWCt9gV7/50rg3zHGUGwBtGSa47EvgYcm8j/avm64fdKnDcKRcqyHm2g7R2fIhDC/RR0awoBVfPzet+I7H3AYcTdLqHak9NLIRw3F5C9vSMYP/YTKz6fhCxxx1jopK2sf0vCRvLWqH/EEcuhgpfxlWqssK9KWCoWrlhFD4yvCkh1yTEJLyCnkPuD0FMqYP8R+lnPD/F76E1C5BIy17ubD/3uT9mnvMjF3W+6/qMK9Xq8ii86HVR3BlO8/KeuQxwu8s/UyP22FDFBY1syqm6zf/KhkvPx0/mwuoPh5qmkL8VNkvDx2Af2QIb73ZEq9pDX6ojKAbGLpwA vdThOpHj NKoej/wN0+ACMDeY+vGxd2r1FZQG+fxLNEfYrRbYpY2AaC+rNOHNaTMgNL3MsqQKRw/rqfXjI6fQ0RBRBNfGTJm744EzUNRHfBWZCODT3Sc1uSKX25R97TwNeVEbmKFpqF4DemJQhM/ylq6FDlmwWDqi+HPhV0uZk2k2NAVy5Eso7pPkI8TIDncKz1sZzDuU3i/T1DcO9G3+1XOGYpVcy2UHNQ0D7w+kpXy1tVO50Vf9Rcs1Ujt+JAPbv5w5qE8oiobAItyUCUoImnN5BhqL707tUsUzp3LSWQcW+gwI1u9UlZr4otnsf5QvujVkFp8kCgmn2/yhc6X/KGu0Uo7TIZRCH1JEQAI14mIP7NoGcyDGryUgwD137w2dU31SfPTe5ge/8TT2OllD272JvpROCjLQOUdNvGuVOhhomQ3GL0fL0qRCiU9Y2mejekyC8ErKBETnBIai6YirwJk0vPuNWT3X76Urf40qds2hJQqi3vyO8keTCtqTmHqA46OaClzg9GSF6qXfYW9wJiZKcbKa8MCum2PLAvqLO1GSqqVJNkmJNr555fh//Zt8lQ0C7q61SmpH5Z7qqfApIo3cw6iIIXt3VrpwoIRNa07BZvj0u01DQAMUd2HouW5ylG6PYD5HiGd3JqEL/OcsBd+Z5YB8CPX79yN4iAKr8dUfzLs+TJQaM1r6iad3Y4qyqBjYGPkh1CWNl1rQtG+s9XEhvHVwa7Bkf6Ib6VA7zP9wiZDmiQSYVuEkAj2+5CVk9r8DUxg8nrBGcH6jYM8q6f5d0cPhA/kvrdBwcaI5d3kHfDsASp0cjwENUa6aZXQsnEevPdl0D/aNN/8gSJqwK3vE2ri2TUCnocNgtAHfNDDjbUg5M3WVssdUqR/U0kKpHbZTMZ3WTD2bU8pkbBNotjIm2qmZfGWZ6X8TmS3uJSCvg8bNx+BKJXKsi6+CBmD3U46jUzE5xHBDVoxMPdtnsV3jWNE7N0xbJ2A== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Introduction: ============= In the current hot page promotion, all the activities including the process address space scanning, NUMA hint fault handling and page migration is performed in the process context. i.e., scanning overhead is borne by applications. This is RFC V1 patch series to do (slow tier) CXL page promotion. The approach in this patchset assists/addresses the issue by adding PTE Accessed bit scanning. Scanning is done by a global kernel thread which routinely scans all the processes' address spaces and checks for accesses by reading the PTE A bit. A separate migration thread migrates/promotes the pages to the toptier node based on a simple heuristic that uses toptier scan/access information of the mm. Additionally based on the feedback for RFC V0 [4], a prctl knob with a scalar value is provided to control per task scanning. Initial results show promising number on a microbenchmark. Soon will get numbers with real benchmarks and findings (tunings). Experiment: ============ Abench microbenchmark, - Allocates 8GB/16GB/32GB/64GB of memory on CXL node - 64 threads created, and each thread randomly accesses pages in 4K granularity. - 512 iterations with a delay of 1 us between two successive iterations. SUT: 512 CPU, 2 node 256GB, AMD EPYC. 3 runs, command: abench -m 2 -d 1 -i 512 -s Calculates how much time is taken to complete the task, lower is better. Expectation is CXL node memory is expected to be migrated as fast as possible. Base case: 6.14-rc6 w/ numab mode = 2 (hot page promotion is enabled). patched case: 6.14-rc6 w/ numab mode = 1 (numa balancing is enabled). we expect daemon to do page promotion. Result: ======== base NUMAB2 patched NUMAB1 time in sec (%stdev) time in sec (%stdev) %gain 8GB 134.33 ( 0.19 ) 120.52 ( 0.21 ) 10.28 16GB 292.24 ( 0.60 ) 275.97 ( 0.18 ) 5.56 32GB 585.06 ( 0.24 ) 546.49 ( 0.35 ) 6.59 64GB 1278.98 ( 0.27 ) 1205.20 ( 2.29 ) 5.76 Base case: 6.14-rc6 w/ numab mode = 1 (numa balancing is enabled). patched case: 6.14-rc6 w/ numab mode = 1 (numa balancing is enabled). base NUMAB1 patched NUMAB1 time in sec (%stdev) time in sec (%stdev) %gain 8GB 186.71 ( 0.99 ) 120.52 ( 0.21 ) 35.45 16GB 376.09 ( 0.46 ) 275.97 ( 0.18 ) 26.62 32GB 744.37 ( 0.71 ) 546.49 ( 0.35 ) 26.58 64GB 1534.49 ( 0.09 ) 1205.20 ( 2.29 ) 21.45 Major Changes since V0: ====================== - A separate migration thread is used for migration, thus alleviating need for multi-threaded scanning (atleast as per tracing). - A simple heuristic for target node calculation is added. - prctl (David R) interface with scalar value is added to control per task scanning. - Steve's comment on tracing incorporated. - Davidlohr's reported bugfix. - Initial scan delay similar to NUMAB1 mode added. - Got rid of migration lock during mm_walk. PS: Occassionally I do see if scanning is too fast compared to migration, scanning can stall waiting for lock. Should be fixed in next version by using memslot for migration.. Disclaimer, Takeaways and discussion points and future TODOs ============================================================== 1) Source code, patch seggregation still to be improved, current patchset only provides a skeleton. 2) Unification of source of hotness is not easy (as mentioned perhaps by Jonathan) but perhaps all the consumers/producers can work coopertaively. Scanning: 3) Major positive: Current patchset is able to cover all the process address space scanning effectively with simple algorithms to tune scan_size and scan_period. 4) Effective tracking of folio's or address space using / or ideas used in DAMON is yet to be explored fully. 5) Use timestamp information-based migration (Similar to numab mode=2). instead of migrating immediately when PTE A bit set. (cons: - It will not be accurate since it is done outside of process context. - Performance benefit may be lost.) Migration: 6) Currently fast scanner can bombard migration list, need to maintain migration list in a more organized way (for e.g. using memslot, so that it is also helpful in maintaining recency, frequency information (similar to kpromoted posted by Bharata) 7) NUMAB2 throttling is very effective, we would need a common interface to control migration and also exploit batch migration. Thanks to Bharata, Joannes, Gregory, SJ, Chris, David Rientjes, Jonathan, John Hubbard, Davidlohr, Ying, Willy, Hyeonggon Yoo and many of you for your valuable comments and support. Links: [1] https://lore.kernel.org/lkml/20241127082201.1276-1-gourry@gourry.net/ [2] kstaled: https://lore.kernel.org/lkml/1317170947-17074-3-git-send-email-walken@google.com/#r [3] https://lore.kernel.org/lkml/Y+Pj+9bbBbHpf6xM@hirez.programming.kicks-ass.net/ [4] RFC V0: https://lore.kernel.org/all/20241201153818.2633616-1-raghavendra.kt@amd.com/ [5] Recap: https://lore.kernel.org/linux-mm/20241226012833.rmmbkws4wdhzdht6@ed.ac.uk/T/ [6] LSFMM: https://lore.kernel.org/linux-mm/20250123105721.424117-1-raghavendra.kt@amd.com/#r [7] LSFMM: https://lore.kernel.org/linux-mm/20250131130901.00000dd1@huawei.com/ I might have CCed more people or less people than needed unintentionally. Patch organization: patch 1-4 initial skeleton for scanning and migration patch 5: migration patch 6-8: scanning optimizations patch 9: target_node heuristic patch 10-12: sysfs, vmstat and tracing patch 13: A basic prctl implementation. Raghavendra K T (13): mm: Add kmmscand kernel daemon mm: Maintain mm_struct list in the system mm: Scan the mm and create a migration list mm: Create a separate kernel thread for migration mm/migration: Migrate accessed folios to toptier node mm: Add throttling of mm scanning using scan_period mm: Add throttling of mm scanning using scan_size mm: Add initial scan delay mm: Add heuristic to calculate target node sysfs: Add sysfs support to tune scanning vmstat: Add vmstat counters trace/kmmscand: Add tracing of scanning and migration prctl: Introduce new prctl to control scanning Documentation/filesystems/proc.rst | 2 + fs/exec.c | 4 + fs/proc/task_mmu.c | 4 + include/linux/kmmscand.h | 31 + include/linux/migrate.h | 2 + include/linux/mm.h | 11 + include/linux/mm_types.h | 7 + include/linux/vm_event_item.h | 10 + include/trace/events/kmem.h | 90 ++ include/uapi/linux/prctl.h | 7 + kernel/fork.c | 8 + kernel/sys.c | 25 + mm/Kconfig | 8 + mm/Makefile | 1 + mm/kmmscand.c | 1515 ++++++++++++++++++++++++++++ mm/migrate.c | 2 +- mm/vmstat.c | 10 + 17 files changed, 1736 insertions(+), 1 deletion(-) create mode 100644 include/linux/kmmscand.h create mode 100644 mm/kmmscand.c base-commit: b7f94fcf55469ad3ef8a74c35b488dbfa314d1bb