From patchwork Wed Mar 19 19:30:21 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Raghavendra K T X-Patchwork-Id: 14023074 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83BB5C35FFB for ; Wed, 19 Mar 2025 19:32:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A40B3280013; Wed, 19 Mar 2025 15:32:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9C951280010; Wed, 19 Mar 2025 15:32:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7F59B280013; Wed, 19 Mar 2025 15:32:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 61374280010 for ; Wed, 19 Mar 2025 15:32:11 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 396191C88F5 for ; Wed, 19 Mar 2025 19:32:12 +0000 (UTC) X-FDA: 83239296504.29.60E0597 Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2077.outbound.protection.outlook.com [40.107.220.77]) by imf18.hostedemail.com (Postfix) with ESMTP id E8FF31C001D for ; Wed, 19 Mar 2025 19:32:08 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=sa7knc7K; dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf18.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.220.77 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1742412729; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=4wGhb7LYtxUiwXUG0HCYGUwtr/i49pZpHiAdkz0j43g=; b=hsjPSQMQaoXF7jjmg96VceC3uFH/oFzm617ZbjhK5YJQmmV+13d5haf19CdRMWSBj7GkKb Z/fQFZuN8wUVX6ZDrW5ndJCBivzUddv8ojUSLp2+192RiZ/UdER43DZ7MgDjSKKa4jOOU6 6XgmDTQEjFaW862zhPPzWDPLNFz6wEI= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1742412729; a=rsa-sha256; cv=pass; b=6MC3585pBYJoE9JP2pG3Lp4SqpZhCyF0WWB+HvAnL2RyRK7CZug6AH78307mdZDMRQVSZ0 62hODGVar6FWYFbi6cHTAqk7AATizOJuaVJ3FKjxSqV3pnXZy3pd2GGAq1EAvJxZCy8ccU fOvjaveoKIyfu9jE2qwwj5Otp1xOZjo= ARC-Authentication-Results: i=2; imf18.hostedemail.com; dkim=pass header.d=amd.com header.s=selector1 header.b=sa7knc7K; dmarc=pass (policy=quarantine) header.from=amd.com; spf=pass (imf18.hostedemail.com: domain of Raghavendra.KodsaraThimmappa@amd.com designates 40.107.220.77 as permitted sender) smtp.mailfrom=Raghavendra.KodsaraThimmappa@amd.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ec0JQ5oTyYwgXEw+UGkjBJk8LeoTOfpWlqoggoTT9BJdYZyUleAU5GOsjqlnq+q29BGAW7be9IsxfcwsvAzT30rbrasSungoUvFusfzZB3OwQdjJX11cBGuvRUafJ1r19O93eOcflrWqpWNOCcAQSGvrAbz1zrY+qBZ3KQ19hvlTrE1lBMTHBly7ADA+gu/9NRfO7BJkgKCGP/o0W05ymuPPIyKxJers+GynWtNr+iKGKiTpGEedgu/W865YET31Pz+kQkjn+obxkbDBXkIfpsPXjthRD0HsV/LL33CpI8B+0vqIgIYjju/Te2Jud3C49SpBOvlsXoyaF1CGIqtRMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=4wGhb7LYtxUiwXUG0HCYGUwtr/i49pZpHiAdkz0j43g=; b=LKqQSltTEfcAvDtNKc2gmntGYa/Z8RWPKgrEkNhw8ye0SLOjFW7IIORd4n3dPF2xHpfukvH+5OlR5l4DtxUTzISiVSZkarMQ1n8MY3i8Qlsw7uyZKsULUTEXY89DpmcIeEerWJfS/H4C8u8WK7Q2tSADVwij47zoAxhbHcof6talKy+PYbxjgiX4GRmuXebWhkFy4fCfDcQ9OIel1wOwStA1k1wg/AO3lGeNPIcIvAK8UPnTSZ/cBryvJ9parg4rSQNZJALxbnS8dANxFBrz3UGqUJuFWIVfIi/qR6IMjDTBDcYgVltWAbAWK5fZ+9+XKnfL4URteT6JgKX52bOBLA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=arm.com smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=4wGhb7LYtxUiwXUG0HCYGUwtr/i49pZpHiAdkz0j43g=; b=sa7knc7KDjngonBEJXlqBh6UWjTsT87vDOGx9x/GK9LEtwCW3XN4QoGNKjmW062H0cbNtA2WLNFHx0l5CpE783xKbngoVJOIx2H6/KtP5lZDhJWtL+qJGLYO3sB7qUPEEOeIRmqOmPdPXW6wXq/zxBlXYQhMmAJw5Mj9Ex+UEBs= Received: from SA9PR13CA0118.namprd13.prod.outlook.com (2603:10b6:806:24::33) by SN7PR12MB7370.namprd12.prod.outlook.com (2603:10b6:806:299::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8534.33; Wed, 19 Mar 2025 19:32:02 +0000 Received: from SN1PEPF0002636D.namprd02.prod.outlook.com (2603:10b6:806:24:cafe::21) by SA9PR13CA0118.outlook.office365.com (2603:10b6:806:24::33) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.20.8534.33 via Frontend Transport; Wed, 19 Mar 2025 19:32:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by SN1PEPF0002636D.mail.protection.outlook.com (10.167.241.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.8534.20 via Frontend Transport; Wed, 19 Mar 2025 19:32:01 +0000 Received: from tunga.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.39; Wed, 19 Mar 2025 14:31:52 -0500 From: Raghavendra K T To: CC: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC PATCH V1 06/13] mm: Add throttling of mm scanning using scan_period Date: Wed, 19 Mar 2025 19:30:21 +0000 Message-ID: <20250319193028.29514-7-raghavendra.kt@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20250319193028.29514-1-raghavendra.kt@amd.com> References: <20250319193028.29514-1-raghavendra.kt@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: SN1PEPF0002636D:EE_|SN7PR12MB7370:EE_ X-MS-Office365-Filtering-Correlation-Id: 9d38bf76-7eaa-4fd3-927a-08dd671cb923 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|82310400026|36860700013|1800799024|376014|7416014; X-Microsoft-Antispam-Message-Info: EZMw0dXOIE1FJAwUi3ajEr3mFpLDxIn4VgVVCm5R4kyu/DrBTEZyjMekcC2bhQAEauWrNTLX2108tSbLMDDAaMD/AcjFCB2hmRdnctwE9UeyAsaUgOHxq8OG8NAEkJ2UlSbmU3lVFKzD0hv9WqBDX4KRg+VA5LfYiVV3HL5saMlDUHW6+Ir8/boPiGN6NI92qriPAEavNWrYnvZrb6rlSXw86cABtkmxaNKcdugbvcGtuah8GNKz54tyUNvtOcRNk9ha5WgcTc9f1A0Zy2U6Z+CQq+Sm98V5Ks+CQmyRQ9BGZFWUEAiiYSit2oYBJEbQMqH+O3RV0XZVE/pW+B9pPTD9nF15wNodahVpbDAFxEDFG4Be29ir0NJ9VYHgPcS4BaXCxEOWgVAS2zw5wVTsVk2tq6pjALwd7Qpnf8NQVQ41kITpMm5Rd0jx4bG87lo1QToTvR56b82DOxFFhNDCe9WrCbN8qH2D9CuTifXTglOr2/cPalvYswuybkevKY4EoL2U22YWgnmsRxOVj4fSXxPtBDzZaGg5M3GAEimkSwRQCNmui6UlvyFrZ1yU+fGbdAPQl8Grh0iJtwzoE4xKcxmQ0IhMXlMhdC0RMBLLVuiANTvd3i/JncY/8dfS+lwjKgcc0N5aKur5fh3czrxraHN7zE3G+KVFoeI5QH3UltTDd1Zuha1bnziDN/3UAWabbX2fkf/W7/MGRVMf8z/RxPA0s9Faas5+fYyFzY8xhUhsC5mLHg+fZIUIVlpYSyzlrExj3v5SYQqOktpruj7RXclvFfmzmifmiko1mnONk4DSJSOOYtu46jOlYP8ZtXL5Mh60R7dQyqhDLjP9YbUvLzalXv1VjTwuSeuxpGfe62Q+b5pIN7zsLOjosUwEui65ThZaxcXvQY9AZohdePwr71eJL1O1iWOTDYVUb6p5y2Y8i47CcFmq/bUcimXfX9db14aoHo1ByqeSBX32NTh9rTqFO5x8anVp00+heLvh2nSHBP2F/JlBzQc0d9f3cN8tgfxGC37nMIUXrwegakiGRU6ljaBkmBAr5sAyYP5UvxWDQIKLKfauHgyoZXrw6L6rw7W2PygQrQ6fG4ovFqhsnKHtZ3uVXljIfoL+q8/P0SSBZ+cfKM3rIUja3uJtpXoMCzkL3mE67a6blvFu3JwttdMBJs6taw5ki8cVgsdtQMESUVcB82OPvr2VrnbVfwag5y4KqvJ2pxgHkfzjxdPeWWu+AfaII19kkTO/K4LwCgxZgvfD7GYwvAyNF7s23Z/wDzqxRkO+DXRijRiGjikOcE/QgnXifpDHJivBhp2p1TLIq4I2tWXAgoxReMC6C0KqfytzWxMKawRYTll/hcN44oavhwSr3osQXv37mEyI81ZNmq2fQYZuj53zJF7Hnb54d7BSnnwrXTBwTGXZmBEH2r4GiD1Vc++lLTu0MHHFao4aH58AqrBtkZy4IJiPke5NdGGXNTdNiuh2Z2LplMjk+GzjM4GgsrAFvy8KElPsfms= X-Forefront-Antispam-Report: CIP:165.204.84.17;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:SATLEXMB04.amd.com;PTR:InfoDomainNonexistent;CAT:NONE;SFS:(13230040)(82310400026)(36860700013)(1800799024)(376014)(7416014);DIR:OUT;SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Mar 2025 19:32:01.8714 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9d38bf76-7eaa-4fd3-927a-08dd671cb923 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: SN1PEPF0002636D.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR12MB7370 X-Rspam-User: X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: E8FF31C001D X-Stat-Signature: no83bp3h9oifqdw74zi3gni74fjk9zww X-HE-Tag: 1742412728-709658 X-HE-Meta: U2FsdGVkX1+8QqMedhbh6o/IU0adOkGV9C3W+aj+M7rmrQl8GcpubSlKrYJdObZtKSptxGk0TnEqwIbZIBtNCvsSDEOBpufK7AHnQVhQpvwYNhgQhV8Lc75a9pXZfaLhXZGDauSYj883KL4cgvf2A518qljZUUsggR08uiqxz2hH5q/ggSj2AX653KIQJGdI8sb0aKGoahYFYf66dOZe0qKbfl8bA7EiVCiqWfyminpzD2DiW+BpBI16Ji1opCwrieSq2d7IH4UcW3YG47uiU5n0i/cDzZOevCuA2CgedGMN80Il4UE/l/PxzvRKFLgwgAjlsLXd/gRiK+YomqaH5G51K2sIFJBRdCatPKGCVffF8XJNCFrE+j+Qw+vkd0TEPKz3ux+cv7AgUgJyRg0CRKhSV98C6yAYjLDCWOM/HNgA89EVoGo8usYGaItaULOUuzfLOKn6N4jQ7dXYNyZ+AmBu+Fs1/j1jwGFYeC4pp1qQKBF+DK/M9pm82jH7vB8tKDav+5c7VtDa4rnyXkMFgWv7wWZFQzIF6mk8mJ6xdE2WVrkOe72c7fWZOwS4Lu/LIy9MQCOV8LHj3yAwXn0DbY2e2NEnglVedRSrkl4LPOSQapfBDOUHT8vJz9Ra5olUHE+WrR881V+s8s4cjPFzaiaBBFjTlyZycsrpJY6BzP3XeY6Myh5tD3qIkJJwzL3tmm1UZe3FyKE978UNj4QHIPN7YwDXuWqAZAgD4IHRb7R2ejb8VN4sC93rQQc9DRW8tDaLhui2dlg+Olt6hBW/U6/GpiAbczlbfdgS9DAcc0bJvAl8sSNVRsO1w+dwTrOpLp9nMN9ny+lm2orA8u+uVI0uMc4CGjDmy9Z6nptGHI4/bsIiyYzPOC2XwhUfblQhdzmHkTQdiYtB9Vb+FM9WiPvAE6EVY+9nctcpDs1K2o7x85nsKPSi9OOBjq0e51j8u7oR57ZfpNgOCiy5e/T uV5Aam9x gfruM+L+Hf2kHEZotFwPS4q7jIrtB6BXx1eUnI3w6YWmu64svf/uBO7/VjMgVQ/AOxh7gqH0ZOHOHwN4/60fpfD/fOjYD+5Ntn9kd12JPXqY+OBPC0uEK3K1M1a7aYN/PspZEquCjlWAQkn6A0OgkImgvDLVKT69h47A4+wINUrFD1hPHFPopF9/2OAU92isfjQ9evweZiQpjD8Dza+6x0o1G7s4qBiRK0AFmAXIw4VB0gJXaOsrybbO/+Nh9Q4uGggqQpP+9Pjc+I2/Xdn3xCqpkSakKovpgzqvbIsvb3a6U8uiOQVNHkbUH6JRyIBSX+tfhMd5WAAq/daVuzAxkab4N3fkExw8X47F7DSS2nkXCeTfsLfWVuIcYMA6x6e+iibqUhRR7P/WCzCih4p5cNkO0S8sl5eHikxRlXpjg+8CWQbgc1U5bJFPwhoOJQ3I3t0Dol2ypBjg2ssieCW9coQcOT3S+R1uB5U0kf2qJTc0uHnoAbZNO8wmgD3t9Ug1l3+48t1x/vUCNMaTKzbkMl3zc7Zby+wyNSFLnFIyxn133RPMtAD7kLwHh5191VWXBpou+LAd6JiwRguOKV5OJEiVsHc+ZYLm2JODW1//9OopAFdGu2AGw2KPbvAjLXZI1uLD67piooipnqonahLmjmkTTKd4O6ifB4QEkL+Z6ApkQ+LTaI8xkYdlvozlYH/jUl/KZBZY4vn2JNwIQUQ055txD5uG6bBM3WaA0YEnyuvDDRsaFavK9HCB2YfFs7bJg2OIKczrsKCHcRZjz1ahNROrKtkNSb4e1hTB+slCFjS4LQ1rJ2Vg8SiVHkg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Before this patch, scanning of tasks' mm is done continuously and also at the same rate. Improve that by adding a throttling logic: 1) If there were useful pages found during last scan and current scan, decrease the scan_period (to increase scan rate) by TUNE_PERCENT (15%). 2) If there were no useful pages found in last scan, and there are candidate migration pages in the current scan decrease the scan_period aggressively by 2 power SCAN_CHANGE_SCALE (2^3 = 8 now). Vice versa is done for the reverse case. Scan period is clamped between MIN (500ms) and MAX (5sec). Signed-off-by: Raghavendra K T --- mm/kmmscand.c | 110 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 109 insertions(+), 1 deletion(-) diff --git a/mm/kmmscand.c b/mm/kmmscand.c index feca775d0191..cd2215f2e00e 100644 --- a/mm/kmmscand.c +++ b/mm/kmmscand.c @@ -20,6 +20,7 @@ #include #include #include +#include #include #include "internal.h" @@ -33,6 +34,16 @@ static DEFINE_MUTEX(kmmscand_mutex); #define KMMSCAND_SCAN_SIZE (1 * 1024 * 1024 * 1024UL) static unsigned long kmmscand_scan_size __read_mostly = KMMSCAND_SCAN_SIZE; +/* + * Scan period for each mm. + * Min: 500ms default: 2sec Max: 5sec + */ +#define KMMSCAND_SCAN_PERIOD_MAX 5000U +#define KMMSCAND_SCAN_PERIOD_MIN 500U +#define KMMSCAND_SCAN_PERIOD 2000U + +static unsigned int kmmscand_mm_scan_period_ms __read_mostly = KMMSCAND_SCAN_PERIOD; + /* How long to pause between two scan and migration cycle */ static unsigned int kmmscand_scan_sleep_ms __read_mostly = 16; @@ -74,6 +85,11 @@ static struct kmem_cache *kmmscand_slot_cache __read_mostly; /* Per mm information collected to control VMA scanning */ struct kmmscand_mm_slot { struct mm_slot slot; + /* Unit: ms. Determines how aften mm scan should happen. */ + unsigned int scan_period; + unsigned long next_scan; + /* Tracks how many useful pages obtained for migration in the last scan */ + unsigned long scan_delta; long address; bool is_scanned; }; @@ -590,13 +606,92 @@ static void kmmscand_migrate_folio(void) spin_unlock(&kmmscand_migrate_lock); } +/* + * This is the normal change percentage when old and new delta remain same. + * i.e., either both positive or both zero. + */ +#define SCAN_PERIOD_TUNE_PERCENT 15 + +/* This is to change the scan_period aggressively when deltas are different */ +#define SCAN_PERIOD_CHANGE_SCALE 3 +/* + * XXX: Hack to prevent unmigrated pages coming again and again while scanning. + * Actual fix needs to identify the type of unmigrated pages OR consider migration + * failures in next scan. + */ +#define KMMSCAND_IGNORE_SCAN_THR 256 + +/* Maintains stability of scan_period by decaying last time accessed pages */ +#define SCAN_DECAY_SHIFT 4 +/* + * X : Number of useful pages in the last scan. + * Y : Number of useful pages found in current scan. + * Tuning scan_period: + * Initial scan_period is 2s. + * case 1: (X = 0, Y = 0) + * Increase scan_period by SCAN_PERIOD_TUNE_PERCENT. + * case 2: (X = 0, Y > 0) + * Decrease scan_period by (2 << SCAN_PERIOD_CHANGE_SCALE). + * case 3: (X > 0, Y = 0 ) + * Increase scan_period by (2 << SCAN_PERIOD_CHANGE_SCALE). + * case 4: (X > 0, Y > 0) + * Decrease scan_period by SCAN_PERIOD_TUNE_PERCENT. + */ +static inline void kmmscand_update_mmslot_info(struct kmmscand_mm_slot *mm_slot, + unsigned long total) +{ + unsigned int scan_period; + unsigned long now; + unsigned long old_scan_delta; + + scan_period = mm_slot->scan_period; + old_scan_delta = mm_slot->scan_delta; + + /* decay old value */ + total = (old_scan_delta >> SCAN_DECAY_SHIFT) + total; + + /* XXX: Hack to get rid of continuously failing/unmigrateable pages */ + if (total < KMMSCAND_IGNORE_SCAN_THR) + total = 0; + + /* + * case 1: old_scan_delta and new delta are similar, (slow) TUNE_PERCENT used. + * case 2: old_scan_delta and new delta are different. (fast) CHANGE_SCALE used. + * TBD: + * 1. Further tune scan_period based on delta between last and current scan delta. + * 2. Optimize calculation + */ + if (!old_scan_delta && !total) { + scan_period = (100 + SCAN_PERIOD_TUNE_PERCENT) * scan_period; + scan_period /= 100; + } else if (old_scan_delta && total) { + scan_period = (100 - SCAN_PERIOD_TUNE_PERCENT) * scan_period; + scan_period /= 100; + } else if (old_scan_delta && !total) { + scan_period = scan_period << SCAN_PERIOD_CHANGE_SCALE; + } else { + scan_period = scan_period >> SCAN_PERIOD_CHANGE_SCALE; + } + + scan_period = clamp(scan_period, KMMSCAND_SCAN_PERIOD_MIN, KMMSCAND_SCAN_PERIOD_MAX); + + now = jiffies; + mm_slot->next_scan = now + msecs_to_jiffies(scan_period); + mm_slot->scan_period = scan_period; + mm_slot->scan_delta = total; +} + static unsigned long kmmscand_scan_mm_slot(void) { bool next_mm = false; bool update_mmslot_info = false; + unsigned int mm_slot_scan_period; + unsigned long now; + unsigned long mm_slot_next_scan; unsigned long vma_scanned_size = 0; unsigned long address; + unsigned long total = 0; struct mm_slot *slot; struct mm_struct *mm; @@ -620,6 +715,8 @@ static unsigned long kmmscand_scan_mm_slot(void) mm = slot->mm; mm_slot->is_scanned = true; + mm_slot_next_scan = mm_slot->next_scan; + mm_slot_scan_period = mm_slot->scan_period; spin_unlock(&kmmscand_mm_lock); if (unlikely(!mmap_read_trylock(mm))) @@ -630,6 +727,11 @@ static unsigned long kmmscand_scan_mm_slot(void) goto outerloop; } + now = jiffies; + + if (mm_slot_next_scan && time_before(now, mm_slot_next_scan)) + goto outerloop; + VMA_ITERATOR(vmi, mm, address); for_each_vma(vmi, vma) { @@ -658,8 +760,10 @@ static unsigned long kmmscand_scan_mm_slot(void) update_mmslot_info = true; - if (update_mmslot_info) + if (update_mmslot_info) { mm_slot->address = address; + kmmscand_update_mmslot_info(mm_slot, total); + } outerloop: /* exit_mmap will destroy ptes after this */ @@ -759,6 +863,10 @@ void __kmmscand_enter(struct mm_struct *mm) return; kmmscand_slot->address = 0; + kmmscand_slot->scan_period = kmmscand_mm_scan_period_ms; + kmmscand_slot->next_scan = 0; + kmmscand_slot->scan_delta = 0; + slot = &kmmscand_slot->slot; spin_lock(&kmmscand_mm_lock);