From patchwork Wed Feb 1 08:02:20 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Raghavendra K T X-Patchwork-Id: 13123893 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3F09BC38142 for ; Wed, 1 Feb 2023 08:03:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C576E6B0072; Wed, 1 Feb 2023 03:03:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C06556B0074; Wed, 1 Feb 2023 03:03:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AA7B46B0075; Wed, 1 Feb 2023 03:03:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 9BE436B0072 for ; Wed, 1 Feb 2023 03:03:32 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 7D2FB8091B for ; Wed, 1 Feb 2023 08:03:32 +0000 (UTC) X-FDA: 80417983464.29.8B5D20B Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2079.outbound.protection.outlook.com [40.107.243.79]) by imf24.hostedemail.com (Postfix) with ESMTP id 9E9B2180002 for ; Wed, 1 Feb 2023 08:03:28 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=RxcmXlie; spf=pass (imf24.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.243.79 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1675238608; a=rsa-sha256; cv=pass; b=vtyJ3imBgTVdYwP1a8SSCtTOtgHxdONYsBwYh6FfHpnanLjiND654imfTqHg72wVlZnT7E jlKjHj+YvhIaY0uIVYWokdhhNEMh/t/rqTUn3E5EPxhOa5gBkRyPu9ROnDQzRTzmPVS5Lx QrhsJ/JvqdHZVbV105Yo7EJZe1VQR54= ARC-Authentication-Results: i=2; imf24.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=RxcmXlie; spf=pass (imf24.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.243.79 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com; dmarc=pass (policy=quarantine) header.from=amd.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1675238608; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AhcuMGmyMR6wFmrQd9SM/5hyh6AFl2dn/jX8xfA+fTk=; b=u6QdX65XlyqZzdX6tl1jolIc1EdqXBdmU1xa6kdEnZqJ4zhXIOtpCFNRXB557QmgbCOU94 G6fXxJkrZkDhcZ5j7LReaOauhRWYwQZWl7Ccx91DnE3iJlCC/V7Ey6MnpfAXRfz073ZBoB rR0Zr5aJAZTI38k3lLObSSK6ugMnGZc= ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Mvsd5PIG9G1vJY0uDpjV3o1hPM7aBxusFhRgoWan/BGg8YWM8Ho4fhUvoz470GV/sZ23w1BumzF1K0jYVFYteHuyEpa61GM2pReYrjpNXpcGm3ua0pYjtgfKW90wvYek6rt1eqWhkg+7CvLTC8HXSObkyvkXy3GbCU0tgjJTXufCOmJhllazG6PAjgB62t34dko59qxLJQlBaYWwvw4XZyvFcGchSfAJSYAzsZ8Fhqmv3aARpdvCON2rHvFZEymhtBO6h3Nol7NZI1ug4xCtWPrQF38ncNkLukx1Sbj8barPrvB1g/I5MIr3qe21pQ/VckFauVy9cJ4ClhhgNg+Q4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AhcuMGmyMR6wFmrQd9SM/5hyh6AFl2dn/jX8xfA+fTk=; b=etD+lxv6XdLA/T2Myd7/YFDB4Z1HxELSMpa+/0tLmP2jADZDSYoYqrHC9NwySwXNlr67+gaqCyAO7rOY+OuALoIb8CbdSdi8bebXEnIq/7j5yio6KLMqiYkyBb5unqlc6Zah4yTPTZwpHmJo0HOvSmzPdkVI1E7Hy6oCAYwFz0Fg9W+vna+19KxVzRx5N1xK2BiNdH/4dM+fGCm40d2rIvTAAXR8j1uSqe6T8YaDJzUOpN1bmGg1ISwHGGALZtfSh3a3CH5l6mK/KkXPL/8asck/ieQFJ1Q57OGbkNePVivRN4yDSQ64cpRI0/fnyiwb1l8DbPRhBQmGIIjEwKXBtA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AhcuMGmyMR6wFmrQd9SM/5hyh6AFl2dn/jX8xfA+fTk=; b=RxcmXlieyT4aNlBaMdoa68K87VGD0TBpMjYRJVq2wmpnq7veMTzPq50p82cjoPt10eJ7t+ivKbziPkcANnrfBwUfkXJRpg6uXcFiyju7PAaJxYQvWkbNHxMqvZMSio+04WwOaRbPKHP+aZCbJ+J9x85Cyiua6he5W47yaOAetkY= Received: from BN9P223CA0019.NAMP223.PROD.OUTLOOK.COM (2603:10b6:408:10b::24) by CH3PR12MB8511.namprd12.prod.outlook.com (2603:10b6:610:15c::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6043.38; Wed, 1 Feb 2023 08:03:26 +0000 Received: from BN8NAM11FT082.eop-nam11.prod.protection.outlook.com (2603:10b6:408:10b:cafe::36) by BN9P223CA0019.outlook.office365.com (2603:10b6:408:10b::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6043.38 via Frontend Transport; Wed, 1 Feb 2023 08:03:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by BN8NAM11FT082.mail.protection.outlook.com (10.13.176.94) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6064.22 via Frontend Transport; Wed, 1 Feb 2023 08:03:25 +0000 Received: from BLR-L-RKODSARA.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Wed, 1 Feb 2023 02:03:22 -0600 From: Raghavendra K T To: , CC: Ingo Molnar , Peter Zijlstra , "Mel Gorman" , Andrew Morton , "David Hildenbrand" , , Bharata B Rao , Disha Talreja , Mel Gorman , Raghavendra K T Subject: [PATCH V2 1/3] sched/numa: Apply the scan delay to every vma instead of tasks Date: Wed, 1 Feb 2023 13:32:20 +0530 Message-ID: <1aebc55030925998a3df3cafb79c5cd28b199ea8.1675159422.git.raghavendra.kt@amd.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: References: MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BN8NAM11FT082:EE_|CH3PR12MB8511:EE_ X-MS-Office365-Filtering-Correlation-Id: 510611f5-71e3-49c2-9d40-08db042acbef X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /SRV4htTNbr7JEQTc+CKomAgA6e1uQtQT7+e74B3FoFmk66xOXpU5iqvOZ4kljvy6KwcH/DsdEX0hUnJlt/DgtUq1ZGjWh90uqDpL/BzgD5wjwX1jkwqGfBCa7pf1C30QN7RwAUEytf04cnHYb59cPTS4Revv6OOY1OD8Pb/4dV6oinvQ3TnYWcGC37eIoyHV2FpmYnRd0MaJgFgA3gRNX2GE1h84X0D0IggUOGZSCvMgI92UZVdc+X+D7L0un7r6Y1ggh8NRJ1ZYCJa7xuBT/MeXYvTPCysXLDyCD+EZH0cKZomBK+hT9h68Uyge2d1pPcKtj4jPJhPQfCfM7XqFfgQ9uKA25zH04fScUXebJla/L1F5gbD+BreG1CCburGOKUvHXj0D8p6LrEwHWzu9B01I1D9kJf8TE9EMLjetS1vroXTi9zgxhbhz+OB5WUqawekepNfg0/16czmXpmFmZn5m6TuxIuCSfAHILG32p2VpkRbsVIfRDFhYiVzfi9qaoyfw+NuA6BhRX9Eft6I2D3fAQSN8D54Av3mYkXLuaMK2jPlZ6/9kdVovXq6u49q+p4h8P8XZZPeM8P3BVSn3J1OZhZ82uACNFqMCQdsCKD5C/acJfcahgwprlvkFYxlJpxDzewXUk1rIOJ5OF9deeozAuIcUTKaybDBq928Zps7kdNudPiNLFS3PhTa/34JAIP3NgfCjNqB8SE/ErhPKO1HrJdMijTaUH09qFQ8LTQjkdjUp6o0hhWA/cxZfzmg X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230025)(4636009)(376002)(396003)(136003)(346002)(39860400002)(451199018)(46966006)(36840700001)(40470700004)(54906003)(316002)(8936002)(110136005)(4326008)(70586007)(8676002)(41300700001)(70206006)(356005)(40480700001)(82740400003)(36756003)(36860700001)(81166007)(40460700003)(186003)(26005)(2906002)(82310400005)(7696005)(47076005)(478600001)(426003)(83380400001)(2616005)(5660300002)(16526019)(336012)(2101003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Feb 2023 08:03:25.9204 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 510611f5-71e3-49c2-9d40-08db042acbef X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: BN8NAM11FT082.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8511 X-Rspam-User: X-Rspamd-Queue-Id: 9E9B2180002 X-Rspamd-Server: rspam01 X-Stat-Signature: 68bxgoj9qouftkc7q5nhh38frebusko4 X-HE-Tag: 1675238608-155286 X-HE-Meta: U2FsdGVkX1/W1yZuppUVugxBCw6mtr2vR+T13bGKFKTYL33fBJjTA11G940+xsLHzjuF7dHbANTe2xhD39jAMmmZam6NgwwhlPFHYW4AAt4qS9X8hN+S0aeewnJqY2yPR7BYzo34nPoLcyJA14t+S6hFwPGTZsI4eI349obFbAmMgpqsLrzaIYecChgUT2MZRmZEh52jh2dbQRsHMVPtsJSWDGadlWCKSbLDIeFiAOATdQRu7IiE4ZhMjsPiNbGx5AT8Q0u4Aeuq4BRohUp7/Di66SkRGdN96+9GYkRUae7neDMcC8v8IZgytX1Kiow0HtWy32UjWkjMWImeTBQqXVLpgJWvY95lhhpjPZOS53mNkMlBuGNIslgr4aggrqtvKyFY5YR2hbE/SUSLaG+eZNi5gFWWqVK60uXnvMAwsb+tHaNFBhkmqPJdqV7cQiHHiNplXzC3SNrz219VMUVtO6k/wbSnqRLfUCHYsscop68gxUBuDD6kjK81GagZ4w62dFd2JyhQFIlL0u9BT30HOCm8J0swAIEqTSUV6jOSf9aHyGhsAt1Tk7rgsYD9dOIHQO2GW3f6otUkr3mukg6tAzHEGy6BsZDSDWDahbZOvBSprE3ddU20iVdbbT+iXsNiptO2tCBcp6Gy+RhocRjmJcajqLQHI9JyuV6fy/bbFv5JLni4ffbZrgUUSTBb4ldT73EdI+kJ0CvvtXgNzEyige0KLAnBIIXbzvlKA7xrC55kzpNFTjQ7BtQCwnvboZK55o4Wcb4DGoK88j9YuCThIjSWLtvw2cTiZHNpg1PWNAzgGBvrtkN/Pc8gNKB08A1uBd0/g+M4fIVPSvQr2Fd0/rlX93enA4IipPCj0dZIGmdeTudPV8+zW9rYB2MaMtyKwPv8PBNSmAY4bi4BKepN6VSTftW9vD9L58PYv7jua1ERMmJgrFPhX6CBLftSegyqXslwvF1EHKFX14VPWaW CsPxPh8c D5gnAuqsg9MjI5bQFwvATUe/7wKGJ91CBGGJXFnucm/aShZDDYttnza1nWpIUmQziqJ4y17GNHTNuA7TbbJWbRsaNAts7ui6sUaeDgNyc5gxCnI8onxLuVt3H22hRTVzVXLhBwMpQBut6Hty1Ho2iRHG7lkRoxdSRiIqGsLby81QP3SJn4blG1FIArPIK2KThAMfK0H3LF1c04MZnxHrCQzV0YEsYCCUPIZWL5UHHhsY1AFytb2+KHorZ7AXZzb+74qpQaksOeN1tcH9fdrSxxlVyH0roEZGPEA5fEPfww+i8H4kTEfi39XlxyLiqZuT6IrXOoHHLFJSFAwgxqohXDBNVGgeLnfpfIU5hkCsVDz2t0+T4rImJ++DXGuSIUvQ2JtiJC+zw2MhYJxmRsxfeYjxvroy9C2zxdQ8Gry1aD3URqxTwNOhcRVTCLRgsGspzZ7NaVB9PLkTifxwYZcGqo5H0DIDNkqHluTI6gB8HheLFSjFnJKOesdj4x4kRZBTCxhJAOnVt1tGVKVaut1q7vcQM7/C4X5YCzzHPsGWI+6+uIqAQWOJX86NPi0OKV4lip3d/4wCFEsAWuQcnzxjyVI0ELSL9YSzQ22eW X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Mel Gorman Avoid scanning new or very short-lived VMAs. (Raghavendra: Add initialization in vm_area_dup()) Signed-off-by: Mel Gorman Signed-off-by: Raghavendra K T --- include/linux/mm.h | 9 +++++++++ include/linux/mm_types.h | 7 +++++++ kernel/fork.c | 2 ++ kernel/sched/fair.c | 17 +++++++++++++++++ 4 files changed, 35 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index 974ccca609d2..74d9df1d8982 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -611,6 +611,14 @@ struct vm_operations_struct { unsigned long addr); }; +#ifdef CONFIG_NUMA_BALANCING +#define vma_numab_init(vma) do { (vma)->numab = NULL; } while (0) +#define vma_numab_free(vma) do { kfree((vma)->numab); } while (0) +#else +static inline void vma_numab_init(struct vm_area_struct *vma) {} +static inline void vma_numab_free(struct vm_area_struct *vma) {} +#endif /* CONFIG_NUMA_BALANCING */ + static inline void vma_init(struct vm_area_struct *vma, struct mm_struct *mm) { static const struct vm_operations_struct dummy_vm_ops = {}; @@ -619,6 +627,7 @@ static inline void vma_init(struct vm_area_struct *vma, struct mm_struct *mm) vma->vm_mm = mm; vma->vm_ops = &dummy_vm_ops; INIT_LIST_HEAD(&vma->anon_vma_chain); + vma_numab_init(vma); } static inline void vma_set_anonymous(struct vm_area_struct *vma) diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 500e536796ca..e84f95a77321 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -435,6 +435,10 @@ struct anon_vma_name { char name[]; }; +struct vma_numab { + unsigned long next_scan; +}; + /* * This struct describes a virtual memory area. There is one of these * per VM-area/task. A VM area is any part of the process virtual memory @@ -504,6 +508,9 @@ struct vm_area_struct { #endif #ifdef CONFIG_NUMA struct mempolicy *vm_policy; /* NUMA policy for the VMA */ +#endif +#ifdef CONFIG_NUMA_BALANCING + struct vma_numab *numab; /* NUMA Balancing state */ #endif struct vm_userfaultfd_ctx vm_userfaultfd_ctx; } __randomize_layout; diff --git a/kernel/fork.c b/kernel/fork.c index 08969f5aa38d..ac6f0477cf6e 100644 --- a/kernel/fork.c +++ b/kernel/fork.c @@ -474,6 +474,7 @@ struct vm_area_struct *vm_area_dup(struct vm_area_struct *orig) */ *new = data_race(*orig); INIT_LIST_HEAD(&new->anon_vma_chain); + vma_numab_init(new); dup_anon_vma_name(orig, new); } return new; @@ -481,6 +482,7 @@ struct vm_area_struct *vm_area_dup(struct vm_area_struct *orig) void vm_area_free(struct vm_area_struct *vma) { + vma_numab_free(vma); free_anon_vma_name(vma); kmem_cache_free(vm_area_cachep, vma); } diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c index e4a0b8bd941c..060b241ce3c5 100644 --- a/kernel/sched/fair.c +++ b/kernel/sched/fair.c @@ -3015,6 +3015,23 @@ static void task_numa_work(struct callback_head *work) if (!vma_is_accessible(vma)) continue; + /* Initialise new per-VMA NUMAB state. */ + if (!vma->numab) { + vma->numab = kzalloc(sizeof(struct vma_numab), GFP_KERNEL); + if (!vma->numab) + continue; + + vma->numab->next_scan = now + + msecs_to_jiffies(sysctl_numa_balancing_scan_delay); + } + + /* + * After the first scan is complete, delay the balancing scan + * for new VMAs. + */ + if (mm->numa_scan_seq && time_before(jiffies, vma->numab->next_scan)) + continue; + do { start = max(start, vma->vm_start); end = ALIGN(start + (pages << PAGE_SHIFT), HPAGE_SIZE);