From patchwork Wed Aug 28 20:22:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13781799 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2D4F1C71133 for ; Wed, 28 Aug 2024 20:23:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id BD9FC6B0088; Wed, 28 Aug 2024 16:23:25 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B61F06B0089; Wed, 28 Aug 2024 16:23:25 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 98D686B008A; Wed, 28 Aug 2024 16:23:25 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 7D7686B0088 for ; Wed, 28 Aug 2024 16:23:25 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 2BE1F14072A for ; Wed, 28 Aug 2024 20:23:25 +0000 (UTC) X-FDA: 82502779170.26.FE0BB53 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2041.outbound.protection.outlook.com [40.107.244.41]) by imf20.hostedemail.com (Postfix) with ESMTP id 44A201C0016 for ; Wed, 28 Aug 2024 20:23:22 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="i+V/tbRL"; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf20.hostedemail.com: domain of ziy@nvidia.com designates 40.107.244.41 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1724876504; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=JvlvOKiL1uDrY0SOAsMeAPqpU7SQeBVYP8MOTmwgRbQ=; b=J+EkmxhpHiLqvKk+/WLXHDrwUOMelKKDWpQtoZhrkpGkbZed4ERQx3Iez/SqDx13CZsloa a2Hxq7Yjzr8UmjbTfLmFlgonSBuEOOoYYe0AALh5/GcRflZCvDN+zUv7rvd+VaNoHn7ThM Z8mlRGftSeNkV+db5Lhsiqu2gVo/TK4= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1724876504; a=rsa-sha256; cv=pass; b=zWUlmiBT+bjuWlK2U1qNxT4Qk3+nO2t9u1JPuLaFZuyyfcFbiuVdaC2N38AtH4V16RKM7Y nXCtlb2fRPFFLPWnTSj+iXC7eI/aKQxGYmUJM5ReCwCF5iNf/CAu2Xt5Yo9j3YFislQuCp 4ldnLJGr0HXqE1uyGBt0T+jj0o0hKV4= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="i+V/tbRL"; dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf20.hostedemail.com: domain of ziy@nvidia.com designates 40.107.244.41 as permitted sender) smtp.mailfrom=ziy@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LMZcoTgFOM25p+pb/l6g+O4KzdIRq1CEcWpOeQR+4G/NgzsgoqDKvZFkaogyvJM+z2yHO6qNRBJvbgoEXdlCyDh6TKpfuUiIAyxDUxaAPohqf+9yfQ4kTBJ7/4lYe+M04KSoBeZp8E+RfsiclUs2YkOEhRtYLoYn9/Mx4iRcn90rjIOfGjr5JQP5QJXV0Dm8HhWKxEWwDtAzXezPzMjp8PsdhSP8aaaF/9MSYdFhHM9/+vgLebFkKWj5enJBm4L9IvmrRuin7xc3/Yj8DEBKwiF3bYeHYdoJXoXaG0v5LqocGi6NSBu4fcxilOmKqscLWo63jtBklMi/0EnaaUjDQQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=JvlvOKiL1uDrY0SOAsMeAPqpU7SQeBVYP8MOTmwgRbQ=; b=oI781mdNe6mfbpDrqlZAQ3Cic3a6xISaADwq6n0fFMiV8+LUIwpQT1ghMWJ2u1SE0narmeGsYCNyAydqsmcxbdcIut4lKJiiQXAJqrJHoq2kTMKtRwW6Hok7Y/NniCBhia1jAlNa552Y/46QAie/WEgsgXMuIIxctxa19X/sN4cc8aA9pfiljBoSe/2byXK0+twytci5wo1460HNo4FAR/jpDamfzO8Kz71KCG7xiboHF/RGuF+StBEvtDtMYb9wcJjkJpuZTjbtgrEbd7CV/rrOIq1fgaMpiGaCfMrX1tNRzTAuuYjUezhkIRcgIFy3TerCiOpMEMrAXeDdpzE8aw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=JvlvOKiL1uDrY0SOAsMeAPqpU7SQeBVYP8MOTmwgRbQ=; b=i+V/tbRL4+mb8d+yVe2p231T0pYFdYGxTN5sJ0DZflCR4UGKcjzswzc7ktp+D0/Y/2awh3ScdgpLMO9VFDjvWAxHnJcGjXu08LOOlRnCrzP9K4iTLy5c8TOjzCvgpxNFNfFMUJGOjgU4f70lSbq24RrJ1R7NJ602bsYxvNBISOE/kVQTvCFsjPoO8WJRNqZnkF2b0HPLrDIcQP2IDfxqThskE2fgj56j3nlYjf7Z8Cp9LWBnKQlqa8c9O5gb5BoRAeuj3GesNqgKSp+3jT1ETiHK0xzhocM1zHSYonJFimJcmdjGmn8oxk5UdxHLIFv/72nAqktkh/VHOcDru/rxCA== Received: from CYXPR12MB9320.namprd12.prod.outlook.com (2603:10b6:930:e6::9) by CH2PR12MB4134.namprd12.prod.outlook.com (2603:10b6:610:a7::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7897.28; Wed, 28 Aug 2024 20:23:18 +0000 Received: from CYXPR12MB9320.namprd12.prod.outlook.com ([fe80::9347:9720:e1df:bb5f]) by CYXPR12MB9320.namprd12.prod.outlook.com ([fe80::9347:9720:e1df:bb5f%5]) with mapi id 15.20.7897.027; Wed, 28 Aug 2024 20:23:18 +0000 From: Zi Yan To: linux-mm@kvack.org, David Hildenbrand Cc: Oscar Salvador , Vlastimil Babka , Johannes Weiner , Baolin Wang , "Kirill A. Shutemov" , Mel Gorman , linux-kernel@vger.kernel.org, Zi Yan Subject: [RFC PATCH 1/4] mm/page_isolation: make page isolation a standalone bit. Date: Wed, 28 Aug 2024 16:22:37 -0400 Message-ID: <20240828202240.2809740-2-ziy@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: <20240828202240.2809740-1-ziy@nvidia.com> References: <20240828202240.2809740-1-ziy@nvidia.com> X-ClientProxiedBy: BN9PR03CA0978.namprd03.prod.outlook.com (2603:10b6:408:109::23) To CYXPR12MB9320.namprd12.prod.outlook.com (2603:10b6:930:e6::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: CYXPR12MB9320:EE_|CH2PR12MB4134:EE_ X-MS-Office365-Filtering-Correlation-Id: 4a8db155-2a3e-498e-fb1e-08dcc79f40e7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: clqcE2F/YBs8zz5EqWsUzj4SFZeoe2ipdycRcgWakJE3ez1w/rPWugw2FZishDFdz7YoZfe/pIF3M6weiW5T7wKArGs+kXetHZsGt84RtVEdcDBqomk/eb9IDoIVrAHuj103mhc+fJlSYf1DreCYiuQ5U6yH9NetFfG8AkyOWmZiZ97BvxPXZvGCIf7/FOmJv93FhU1smSi1XdZ5JBOMUO47btxKUZaRGve8T6d1NnlL8hG2m+NAjq0XZ8z2qMaUL4w8nf/JR85G6c+tF1agqsB0IXTqf6QcDqUApBjmgnzQDqpuVgN364IIfjPlxFce+pP3Qrp8jcYMpYu5990RE7Dw4WFYAefeGpw4GYdNufAX6jvbk3hgYFofLE1uLbRSRbWlC5zHphFdw2jM67KHlhDJ/r6CoRvjLGypPD+9eJD6y2NBwkj/ChYmM2UPA+HiomgtOIbYxMwEUOTCp4V6pZDc5+YQKZM5dCINEFgKwohOzaSfB6ktdkPO4FnH72MRGIenJoXAx12ro5T8J/ET3Lx50ZSTrzrG9tdXTpS+29ymLoC4c1otym3wB5kpp0VtJdBPBNku1g+PR3EGF/E6ok81r6CYwa06s3wnbekb6sbbU215cBwdcBuebsJxMFpEtwhV90HWNRosaK5yCcwZX80aRm8Hz473aYTYBW4qE218FrohanD2j+nLMKk5SSsYPsHa14KVb2FCYQmB9Lp5md7Xb8UkPUhHePUVSE1vUTKU4DeWNoYjOc1nKrxMB0uHLEi20v9QwHO5PVXbDVGtxptA4vuHfQYHIuUqENC9S/4nqnb0hZuG4gFYbItAFEoI0H2AnJ2gGxeCevbKomP0tEDq6n7zEP39PRejKR7Izz2bXVzTtTHP0Xjq/WmKu6ZfJaWryfciHcBvMCvTNpNJ2da1myr7Af8YeGOe4y1h86YcH3w+AImtl0qh1m2sf7DHKjWxC4kRsNwV6QmpdByIrgkOIHVO+wlrjT00/RO3vrbOzozhf8abLb8GTXHkvvZdB02RMLEMJnkojxuywNSzwDGiL1lI2+eh8mkkDifUS884UMcVeyq7jlDH/cXlSWlMJbIWronh91i9npqSFTqwWAm6uEfR4YoXzn2b1cbC5FCePnDMblrwrJqqQEVg4/8HDyT2gsFwuKmakZeY8R88Rm8PDsa6So7IKODAqQd1qRdMhsoKe8MCB3HxQac7tIqjDW/e5xvnlUqVAu0Ohekikwefvnpa6M8d4cPqrwFqv841mxhIupN2+lxj9GclblEk0U9NZM0lxUbt8jX7pAfn/U2zABg6xGiTz1cMHzYl9+P3qgnPq3X5an3PVtOo6Ly/05One4vkdy2G07//4pjoZQ== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:CYXPR12MB9320.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 6SIQ1oHiNuUr+5BTfbx6SXY/XXp60TUPtf59z/FeBQt6DkpcWOfUVJBX8b7sHegxCw3+dnUe8PbUWGrDuUz8hTCSMiv0Qlrvi6okENpusnC+GcyAVN1mman+BYS3zJ0s8ktw48eYCzyCfNaE8A16N+zgFf9jUubsXuqnTPOT2Q7Ot+4Xv28RRiFcKsQaSwDQExN+2ZMTbRkZpsqooMjBnRKbE05br2nkVKc3UayZiT4TX7lzK2/pPSPA6ayzGpeRiopJRMM9//6nnC4jC+06LhQ2NsysAziQgU3T/B0MDRz8/5Kcm/nIpihK8Rcny2t+GeaIx4xrXy0FOzaSXBH7dHTf4ED5prp+Ew3n8QveOd2Luu5fIqNxh/sUSzfeGv3oiVFJA3oc0BQ9dfpb6F4HHRG3WkPPRlHNrUWHfG63FZXnrRGwIeEqtD17Wp/4MSnk15fNSFxEwB5QnTO0Qr1vf5XVgkPQ5CYCz/EFraYpS0wI7OduScYge+hc5PiufOry7QUOCFgEalColDickF1W9sXxSUAZh8WjrGe4MGSZy+/N1dKqSe3NPLyeKmlrXax3H0fOXl82tIY7BJYeVxATet4WrZn3Nbcp5zlXileYMYcbhyW1RucXffi+JLVyIilXfim9gEkcOC4W5FXf/ChY1yhxHlwzmn9GtmL5V3IS/E0x4oEztI6e224yYCN5iazkH80L8BEcyZzqWLeZ2bYDqGTkuPSev0+k/ZUIfc38m5sygceO1y882wwkXV1+sJ03KPx86oVtwfdWtr2oyIMI7jjDaPxRcXp+KqVP2/cIQF36J3LThKgkJ4k5c9xFBxXCgYeNBiqMJ+j/VzNUCXBi02oaLdKJBToPwmyHhIAncEsYjaqmFDk/Rf0P71A3a3GRB8zXIdDt4QmVEoHppe8Ej/aXDB4vSUs2hUNBLPavY2CfNo0PlmD6pAqcbVWn0k4IBTQ+H/CaBnNLdEbOYO8bdO3yuzH+yUK5wbVwNRJQ1Dqx5Oasxepy+0Elm5IOoab1OBu5tYYvIu8FZUjH6bHb9Ak6llO70W6UJEe/FKyTqV1gN2Q7Pl/hN9TnE0ZcnT1D6ZJJ9d1Wyvrh17QiMejco+UJEbkSjM1eSx4tayUPbnXZBoJto8QIbZRWp+C52swHHh4Uq2b8cTADQf+Asu4BN0IuTbNQ4QoFIaLHqtBZ+zERweb0Gkob/IgyGEKoL1BMIa8c4lJYOOaalxd/2ciNF4yJsx94H82KNEeKlQKjEbimIN3o3UHfQBBGzzt26lX/n7rWWA7OFY7uAdMqlS26hKYd8lXMdd3i03StbZbJRc8HPJm+1JZSoDuo0gpz3oTKTmmVAVTtR+F5bxi3gurFIqtr7rzI3dOo+ChUOlopuNuoJb68Mzw8wGs2Ai4emyJpIOk8neH0m0qc204fCogbs6Nv5MhZ0snlOZ4Ntl08RkGqpCrfJYuIVd1Z3m6VIozyZkpyBSgkx4LijyrjawoMn2JFNc6kihSk9Lwc8yiLOn1MDz0BINMeMd/P6yFF1DnPKZ9zmFcG0rcXbfWhA2/yLdWMFsP4FgzJYeJlgZwcSB80PLdg9BMbP51vOH3atHyD X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4a8db155-2a3e-498e-fb1e-08dcc79f40e7 X-MS-Exchange-CrossTenant-AuthSource: CYXPR12MB9320.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Aug 2024 20:23:18.3486 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mhepnQBCMwD1259WBO95DCSZv6Lk3iS0CkMxy9+7K0UKdPp79dCydTysERZbyW7B X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4134 X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: 44A201C0016 X-Stat-Signature: mbjwz8rytxobhwd31tfp3swjarhan1gr X-Rspam-User: X-HE-Tag: 1724876602-838633 X-HE-Meta: U2FsdGVkX1+TJdtWljcKH9ArVCLhp5tn6VihthAQNnoPwCuG96WQJanyZqUR1RYyqPfsXZUn9zueeOdDDdRBtLicTNPedXR+clw0SZSyIsecb4HAvkaIqWGNRgecQwbHHgy1cu6paZVcVbFtA1zVWuHs3OJ6DWccQE4FYIdW3H7M4NDAe9us9oQcmQLb0Khzjt2nc6l8g/iehQHsrbvfl1lOdaxr0dqtSC1ShweW3gNyXyVAaqqEtin0THaKEwFW09tzNDUolmFd9m3Ta4UTqJKmC5wWPTVODBz4w5/YDd+mW3Ph3JM4U8e0WtSFplwgDa70qlj5ByOOyAV29v6/UKhwgn2BOJ2nOSl+kWWTIwBniictGV/ZLyLPOib84tBKZEAAns5V8Uk9eVVbLfN5EyVUETPo216RMK8+cxQX5cCs4010XNBOch7t1iEY8WsPbum4emOcnyUanfwyOT3Yt8LukqveVFtXABP/EcWWbQfsBLiveERuUtmnyAnzDYd5Lh86la3J4tk3/ekLMgLYtk0+BZMYXisuY12P0zGq/lV0nfN0M4qRbviAowUfqRxSrjaWqpLMYLkETxnL2d+zzn/eOzuoPRrXYvWlK0d/FuNQVeedna9fW6ovCdX66pI4No/tjNKTt4Mj5X+ACEhk70ogBYsAcU44tiSL8ne787I57HXu9TJwy+EqWRpUd6L0BjLj+YKfG6pZmrGyXF8acC9O5SrJyFXwyNSbhjZwK/i5nd4XbqLiFNwfyjLIe9h7T3IHb74DI57Fi7v1Mg9rc507eBNyX5tiHqo16eBCVn07AlT1nzNzT/m4+/QID0K2tCZtfaghcZPrbX801m7HKeuu5C1E0A7y7XvSsgdcLLgM3VznvrXsR+klogesEk7HfSekQ2G5mq5l7D0pD1iO7bL9WuOq8l6QXenQrBEEzB5GloquWhGlY3CMmiEpuDTGIKXA6dWlJCzNmQXT6SS FQu+bSYz zjRzdbwb/P/jrqv62GQErBrVdZrLrT3QopwetCkC9L2l1pphr1wblCmJ9xRO12VmZm/v57/HrljLRanWgKudyCTkIY4Nza7scHEDwivYdfJxOU0mByo3g3uAH4Sd0GeqKzrrB8nSx4lTckZXPKdTeGdLp8pTW+zovxrJsyCv9Um0kUDjNkTeKMRHPZAEHA1YOEla3U7JZCFTCLZNG2aMiuR+pibDes3iov9UGODOFzK2G3shro4POJaTNsNiCVR6pdO+eqzlDeFbKx8Fk7FwF0Oy+w2VfUwoifWUk5gNIKmutdm+0N35dwgdEANPuOmfyZrXLq+gbXF/vnoRkMmACZkOAYQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: During page isolation, the original migratetype is overwritten, since MIGRATE_* are enums. Change MIGRATE_ISOLATE to be a standalone bit like PB_migrate_skip. pageblock bits needs to be word aligned, so expand the number of pageblock bits from 4 to 8 and make migrate isolate bit 7. Signed-off-by: Zi Yan --- include/linux/mmzone.h | 24 ++++++++++++++++++++---- include/linux/page-isolation.h | 2 +- include/linux/pageblock-flags.h | 29 ++++++++++++++++++++++++++++- mm/page_alloc.c | 22 ++++++++++++++++++++-- 4 files changed, 69 insertions(+), 8 deletions(-) diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 17506e4a2835..5191a90b94f9 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -108,12 +108,28 @@ extern int page_group_by_mobility_disabled; #define MIGRATETYPE_MASK ((1UL << PB_migratetype_bits) - 1) -#define get_pageblock_migratetype(page) \ - get_pfnblock_flags_mask(page, page_to_pfn(page), MIGRATETYPE_MASK) +#ifdef CONFIG_MEMORY_ISOLATION +#define get_pageblock_migratetype(page) \ + (get_pageblock_isolate(page) ? \ + MIGRATE_ISOLATE : \ + get_pfnblock_flags_mask(page, page_to_pfn(page), \ + MIGRATETYPE_MASK)) + +#define folio_migratetype(folio) \ + (get_pageblock_isolate(&folio->page) ? \ + MIGRATE_ISOLATE : \ + get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ + MIGRATETYPE_MASK)) +#else +#define get_pageblock_migratetype(page) \ + get_pfnblock_flags_mask(page, page_to_pfn(page), \ + MIGRATETYPE_MASK) -#define folio_migratetype(folio) \ - get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ +#define folio_migratetype(folio) \ + get_pfnblock_flags_mask(&folio->page, folio_pfn(folio), \ MIGRATETYPE_MASK) +#endif + struct free_area { struct list_head free_list[MIGRATE_TYPES]; unsigned long nr_free; diff --git a/include/linux/page-isolation.h b/include/linux/page-isolation.h index c16db0067090..11b8695115ea 100644 --- a/include/linux/page-isolation.h +++ b/include/linux/page-isolation.h @@ -9,7 +9,7 @@ static inline bool has_isolate_pageblock(struct zone *zone) } static inline bool is_migrate_isolate_page(struct page *page) { - return get_pageblock_migratetype(page) == MIGRATE_ISOLATE; + return get_pageblock_isolate(page); } static inline bool is_migrate_isolate(int migratetype) { diff --git a/include/linux/pageblock-flags.h b/include/linux/pageblock-flags.h index fc6b9c87cb0a..a8121cab4b4f 100644 --- a/include/linux/pageblock-flags.h +++ b/include/linux/pageblock-flags.h @@ -20,7 +20,10 @@ enum pageblock_bits { PB_migrate_end = PB_migrate + PB_migratetype_bits - 1, /* 3 bits required for migrate types */ PB_migrate_skip,/* If set the block is skipped by compaction */ - +#ifdef CONFIG_MEMORY_ISOLATION + PB_migrate_isolate = 7, /* If set the block is isolated */ + /* set it to 7 to make pageblock bit word aligned */ +#endif /* * Assume the bits will always align on a word. If this assumption * changes then get/set pageblock needs updating. @@ -99,4 +102,28 @@ static inline void set_pageblock_skip(struct page *page) } #endif /* CONFIG_COMPACTION */ +#ifdef CONFIG_MEMORY_ISOLATION +#define get_pageblock_isolate(page) \ + get_pfnblock_flags_mask(page, page_to_pfn(page), \ + (1 << (PB_migrate_isolate))) +#define clear_pageblock_isolate(page) \ + set_pfnblock_flags_mask(page, 0, page_to_pfn(page), \ + (1 << PB_migrate_isolate)) +#define set_pageblock_isolate(page) \ + set_pfnblock_flags_mask(page, (1 << PB_migrate_isolate), \ + page_to_pfn(page), \ + (1 << PB_migrate_isolate)) +#else +static inline bool get_pageblock_isolate(struct page *page) +{ + return false; +} +static inline void clear_pageblock_isolate(struct page *page) +{ +} +static inline void set_pageblock_isolate(struct page *page) +{ +} +#endif /* CONFIG_MEMORY_ISOLATION */ + #endif /* PAGEBLOCK_FLAGS_H */ diff --git a/mm/page_alloc.c b/mm/page_alloc.c index c2ffccf9d213..4ea5cd1a07e2 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -380,7 +380,12 @@ unsigned long get_pfnblock_flags_mask(const struct page *page, static __always_inline int get_pfnblock_migratetype(const struct page *page, unsigned long pfn) { - return get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); + return +#ifdef CONFIG_MEMORY_ISOLATION + get_pageblock_isolate(page) ? + MIGRATE_ISOLATE : +#endif + get_pfnblock_flags_mask(page, pfn, MIGRATETYPE_MASK); } /** @@ -398,7 +403,11 @@ void set_pfnblock_flags_mask(struct page *page, unsigned long flags, unsigned long bitidx, word_bitidx; unsigned long word; +#ifdef CONFIG_MEMORY_ISOLATION + BUILD_BUG_ON(NR_PAGEBLOCK_BITS != 8); +#else BUILD_BUG_ON(NR_PAGEBLOCK_BITS != 4); +#endif BUILD_BUG_ON(MIGRATE_TYPES > (1 << PB_migratetype_bits)); bitmap = get_pageblock_bitmap(page, pfn); @@ -422,8 +431,17 @@ void set_pageblock_migratetype(struct page *page, int migratetype) migratetype < MIGRATE_PCPTYPES)) migratetype = MIGRATE_UNMOVABLE; - set_pfnblock_flags_mask(page, (unsigned long)migratetype, +#ifdef CONFIG_MEMORY_ISOLATION + if (migratetype == MIGRATE_ISOLATE) + set_pageblock_isolate(page); + else +#endif + { + if (get_pageblock_isolate(page)) + clear_pageblock_isolate(page); + set_pfnblock_flags_mask(page, (unsigned long)migratetype, page_to_pfn(page), MIGRATETYPE_MASK); + } } #ifdef CONFIG_DEBUG_VM