From patchwork Mon Mar 10 20:50:37 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 14010664 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1DCAEC282DE for ; Mon, 10 Mar 2025 20:51:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CF9CB280008; Mon, 10 Mar 2025 16:51:41 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id CA536280004; Mon, 10 Mar 2025 16:51:41 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A84FA280008; Mon, 10 Mar 2025 16:51:41 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 8183A280004 for ; Mon, 10 Mar 2025 16:51:41 -0400 (EDT) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id C52A78150E for ; Mon, 10 Mar 2025 20:51:42 +0000 (UTC) X-FDA: 83206837644.26.DB2D540 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf29.hostedemail.com (Postfix) with ESMTP id C9D3D120009 for ; Mon, 10 Mar 2025 20:51:38 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=NRjcTaTr; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=DXLdL6+0; spf=pass (imf29.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; dmarc=pass (policy=reject) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741639898; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=3IEXdrd+htUza/gi+AInBAhDpe7VVvHgfAnuFGhv4vc=; b=AlN9lJZ7uGax8U8jpbxH9QvUeiNS7ZlUIiKcaoU8zh8JV6CToDDZlrB0ZVSB8Jv38FGzIr FuAYOnl+mxzGSosu6QpoZJ4sfnYNr1Kh2ecYnZQglpBkuyAUsN+lkny5DAulc3/viX1Npm gjD6+5QczPjpXUSQY/l4aLzqx/M8MpM= ARC-Authentication-Results: i=2; imf29.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=NRjcTaTr; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=DXLdL6+0; spf=pass (imf29.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; dmarc=pass (policy=reject) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1741639898; a=rsa-sha256; cv=pass; b=KzVsM/fUjJcENkMkee8W7I+kbLYZOxQYrVHbUxEgrhQWCWpT48pbnJrsadHEugj4ZTp2gV MK/X1gkkXqdym1nRG/W2shDAdh7guU7RZcf62x5FE5Y7PNBNM8aqbC08t23Q18IOXFBO50 owVluS4BBiRQKVadDkoSN9cmGtPXYJo= Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 52AJtwlY002661; Mon, 10 Mar 2025 20:51:35 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=3IEXdrd+htUza/gi+AInBAhDpe7VVvHgfAnuFGhv4vc=; b= NRjcTaTrCLMhvl9M8xNs3cpC94b5sHvqbWQL/yp45xnba6Tsh5FgETDFMTyUXEsm 0CDT5ViW8jpauD0v+ISzYaoqKfzDIyS248UBejGlOYTCOJsFFdtu3p4TXlr09nOk dQk+rAl46y3rtTJe6zgq5pUo7b1FCWg6ubnJnaIvWiPYMRs6DSHEniiLXq+l9Qx+ RWY1L9uTsSDbHKNroiEJW2uxJV4dWOsBOJ1gHO90zmg6dq1bc5tQjWgda8YSHG7P aU5gbqLFRsJ7dSKspDb+dQ9BBN6I4qq+fR9B1T9JyPrNcc4L5CopCQAaVYNyJzq7 ZuCg29P9UASb8VSb0TPgdQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 458dgt3gff-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Mar 2025 20:51:35 +0000 (GMT) Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 52AJupWj021906; Mon, 10 Mar 2025 20:51:34 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2174.outbound.protection.outlook.com [104.47.56.174]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 458wmnm5gu-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Mar 2025 20:51:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=LjbBmK8h/sPUftQg2RG5L/OgZXleEnEfO1ITW4P5a9EJKiX2TqGf80Aaw4+UbpqNPsFwUFF0iVogZKkG932MG54YA592ufIYpyiLSkcYVDZCf9gyw/Yck054vlnmO5VLRIiPvwZAZonLXzQq2XZXWuVp3rpsd15BtcaWoQpByFsQ643sXhbZIHUFT0zDLBojJA54eXkjdSqT+erIs25LQZwKH7I6o9vAJCiqK2GqRiHef0wHmYWJf0pIUJqzUeTnmTyiuvkEOxPuANj/om9MZUWuGm53S4kwBzi1ya8lhr3gKaTlu9lDGBS1KavLv2nA+VWNL2gf8XJ/MsxpXrGcQA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=3IEXdrd+htUza/gi+AInBAhDpe7VVvHgfAnuFGhv4vc=; b=gcqTElkCW1+SQXqvDJzRdQbGJj0mQZdEVJD6BpcHTAsTtA5okUT6qvyRjACKLMmDLLG4H/a8y0DJeXw/BSCUk8cLhPEmCdURWuqPMYH+f2CS+FuDsJzcLmMEFCzcg798U4isFC0gRfvI2xIGqwLRWhObJvIgw3v0r1Ou/qKHctYObcNWv9CuSikgGnNXotT6ldcfm2pXz+GviQDLvSYPo7I5bxR7l9tZCjqEyhL/oxYOQ9ZAbhrh9ro+KBrAsaOGdb2dPGGNT/HY7yQJQLzRNjwvZI5x/xmk/XEzIE3+l9eOUbIyknkXSSDuRc0ABrDHbjybWV+KXLszbJ/nfeyHqw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=3IEXdrd+htUza/gi+AInBAhDpe7VVvHgfAnuFGhv4vc=; b=DXLdL6+09+nyRFAu1rHTyW614sg47hXkM6u+wwFH6lDVq05nFLYJn5pMyZoLyyCLfnkFsOUJGLY3DebnXi/LBgdYZJi/++Iu/CzF5I06vBT11Z9M39isFTFv+sQnS8hZsQPQnieSdahttE9qJOa/1RV5JiJeaUab3NpNGduC9WQ= Received: from MN2PR10MB4112.namprd10.prod.outlook.com (2603:10b6:208:11e::33) by DS7PR10MB7250.namprd10.prod.outlook.com (2603:10b6:8:e0::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8511.27; Mon, 10 Mar 2025 20:51:32 +0000 Received: from MN2PR10MB4112.namprd10.prod.outlook.com ([fe80::3256:3c8c:73a9:5b9c]) by MN2PR10MB4112.namprd10.prod.outlook.com ([fe80::3256:3c8c:73a9:5b9c%7]) with mapi id 15.20.8489.025; Mon, 10 Mar 2025 20:51:31 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Harry Yoo , Yosry Ahmed , Oliver Sang Subject: [PATCH v3 4/7] mm/mremap: initial refactor of move_vma() Date: Mon, 10 Mar 2025 20:50:37 +0000 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: X-ClientProxiedBy: LO4P123CA0463.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1aa::18) To MN2PR10MB4112.namprd10.prod.outlook.com (2603:10b6:208:11e::33) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR10MB4112:EE_|DS7PR10MB7250:EE_ X-MS-Office365-Filtering-Correlation-Id: 9e883ffb-cf55-44b7-e703-08dd60155669 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: MZXNOxVWsS0LOZQV96o0P8JQLCOdz5cVJNLFt02CwE+c7qTNOcODmvIfz7iriE+22Q7Gv4GHpk72SZPWaH4UuR9ERWlMDgW3uwOAyhbyx5Nua4WeWX0Eh8vNOijMgGGFJaC1vtt+6oOImwS5G9EmqiZpG2NF9OLsZpkvP360ieptOsp7T+/SjAzYx1rzL70Bsf5QaNWyI/ZctbcRNt545zlW29EUz4tWn8erBdHTgdmPknI5c1JHG2QLpkNQeWfnXtawoCZf2MPwKi51GpOdIx8ofS4GEsY2MG1nWu9IgZ+V2SfgDsa936TS3+7n0XqjSEZ0+7RUMz+NpLzFhKlp0M7gwfWFJcYc+SeBLaKnCUB+CzDaiU1+QjoprlLF1cb5rb6pRaCg2p/t5KmXQCZISeZYiR8qJAdivG6nJKOMF+Q0Xkla2jJ8ilTEDkxY3OZ/FrcJuwINrIXjtKbMLHBv1Ner8XB7VU4I9AixHBifc0h+4aOVRBARmQk4PXL5JGHz0aKTVOc5yitVT1up0fFRjEzgrBYMjgxnakUc0MfURNGojIEg/f98IWE1liHY7OiIp28b4lvsu2bs4fPeBbaOQsXPIT1Uj1kLQzszj2M4emVNu/Pqul7i4Bw8j/9c5+ZSwRUkFdsiJX28qQrwMvBh0Hw6fFKDUe1+pgf09M4eCissU8CHUYNwHpDyDQn9ymslEx4yKmkwKrMiRd3j5aBCcu7UClXPTbSb8ChlkLpxwBQvEmevcka58qRdSVaOPjf0/jYltSDbnN749tbRVqSjxcFmPK5l2ulStso64gP9it6Cn8ELYxym2Cis36QtshHH8LyHoLeO2zN+Akb2j73HwIPzIKmbfZYqddQH2cDDJYG8m1O4GaokpUcM5qf4Mf2/YwjG1gIybLj1PByitPqQg5TYTPJDlsPDZUeFixCD4EUR4USBcZv2DK0UFAs6pJLPDmi5FMQIiolJfxGxC9FOh5B/zrDAy5Kb0oJpgVjuB6cElMMWT9LYiMwmUkxrevfZFJ6hL4lgstomj/qzx62iZBKmnKzq/3kuAd/VuHJJSFMlY8Hh0Cas7+jWWWYjgfEJRR7KCrZHl3fjrA7L9EWlLlnVvoCpF/zmcE3vXLvf79NtRybhNNEoIH9HPlK0xVH0095zAgEayVMV4mpLfaR3HNnnP2kS/XnQ0BJ923YSeUxAOYkIVXZRCc2RusxMjV83XQN40pbXjbCx7ZodErpGYO878MhbsfnpnsUHTpPS1UKEU7QzWkw278faahInCzsKHvE1on2JgT83KdfIWid8fJ6FmmZrPlfArwhdoun9C37Hat4IFAQmK8UkiDa+rTCw1kswLtr/W30rM23nPgp2ZlEEwN7I/XUMQobqJPhG/GMN5p71Swgg0KKFaRttSN9D X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB4112.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: K1zZTVQuNlMDd5mlri2VsvkwRXbnfsFgRy8ROm6nC1+IlAlhpVWv7Lx7XpS2joNYQqshZ63ogLKMPNAwpnoCGIC/eGOYMVPJAcx5t3qikPUgu3e84iRFUUi0Mid5+ZtK/I7402TvUh5uyaNNZ63vjEAei3zzbpuy0Q5XUbG2IGzXTByo7zk1e1NnbbwLvkCFoJbas6xxLfZ8uGZDltdXW1fSumeyLrhIYb3qCiQIueSVam6o555QauA2ugxq4+3AE2Ba8+i2fnjLV/1XBJfnyVIZZSidWDV/QoIcPuTykJB6Eor27p88VRERJtZ5V2x4Owqgcm7Y61SuA1HsDHCZb0HqX0Lf+5xmh9eLhtAdGXC/j/VaoZfJN4pZxcIzAc9xesnesKnK3X7a9liy/rfcG45FqREKtrLTmWQOCphAtJXKEQLG1AScB28dInP0E7FH96NYPoEaRaVmfaKrdDUnn3RjQ0o9JlvNureV43NxWC7pum+bMk8SRd/G62dWCSlhw8oMytE3hpCzUFL26yFIYDUJSc8VQCmynZ3jX+7BR4CcnE2s18MKE2vvdPxekTqNqe7TGlEvdw+qpMt9vtFMQs92u27ETbBpA6veAWItFysWexPCfrNCqBheY2NloPVs9tCpzMT2DPontGrha+dXMN3axxgfVbH6fxHUQa9wLbzLxYLDMVE3aZ6MwxFMlNm2vSjUxlxnjyjiEDI2ws3257VuXkM12g5NTVxJN74d/D1rdtg3iThwKWiwR4s6fWEzTTpP8VwJk3rYDOEOpF5wypPCOAFG6MoVeykGJCKxQbFozcwXaa7HbSsNx/pCV3Xn6Dogkjlk7PPMIJoQ8MwNHkIdNhHN1XFFEd9q4pkhhQswnMwtWodeSTgtQiTq00YyCcDtKa5AJJy36yoWcPbnnbVNPc2+rdwz5sSE4SGxuBrgFbbgljbbffTZMWKZz7yPmHdCGSJbBgfKibhNiQoiLfKB3orRPs55ioDvFdg7/n+CvhD38S2JWJAQ3ol/qG/Mk55nKJsuhfCjuCzkW36XuHYWIITRAsUmm+3AdhpPK0uZIWa4SSSF5tbWpoYtdHODONlplBiCHcXAbEan7ydMZv5rXxJxwDV4j3ZbtvkEENDLQslM/Rdv6mcA8BfenEYoghrBcbnPgMoatd7+lEcFJP7k5f2DlMK5gXZb7IpycNs1cZTWXtfNA4fJc7LzbrwFo+7UH8lHw3Uu1YL/PGHTTb15e1m1CkjOnTALBA6+X/HMjiQZaaPYaztyH/mrVoFS+JFtxiphKetmhK4lKUe3/C06VwhH7YdQHkwI6zhHgmd1k//IVJ/tGV6PnRi9Es2+5G0XGZc2ILf3BnGWjXGi5KP9oyRME4Zk3aJKgamUuzOpjJyBbiSzADdSMe+C8zCVnHh8dn1rwjkiWQDqoVo/UqjN9iYFN/febA7NS0AK0ULpDfF6Gzl6mFh6vehhll4HRbuH6E381EuBU+SiRfZ0ng9Wo1HUdsLcubD/32gIuBcsreSqRZk9vuWNoLDpeG8eXekbp71Gu4RksC5wf8TRAOsfgiF/RjIP7w/ho4l+L4Qe85Ibh6qOIz8LYOGEi4KHLtFV6NSSyZ66amsdUmSHSg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: mbvSt9Xd29FalN4RkvoVf9jbzN5wZ8N5iQSx6r6XW9q8Q5pAeUJj9hJDHV39mo7THO6s/sdvL5skMjAJsfudLOVjbOFU43RFf+pukUD019glX4yAC9ckBlVhoTBDqqDTjfOA5mwznibXvcAsPDzFXAMOzDkdvDk+uF8e+6xQaa5cPY7Ekk4lN5k0ctvwHHBNu+DRVJjbrmpJB0m9p2MuENu1CkQUdDsNJ7rHcxliKyaFzOrM+HMu8G1OtXKFtRzgkz+uZs5bzYNHpEligoIebTcklAvC0COXQP4TavKW7IzferkQeMpKEoN/Y4XLAnPgcaMDy43MxsXTwQKGA55jNN5W7gHZIdQGWpNaCYA/jHJiT6FxG0dkOLWykXtU6mx9K1fkrEWB76EJ6XQThOfWqEdoUud+syn1kpkSIe2oUFNueN12WdY151netoz97aRwEFM92yrA/mW5CwYupPRjVBPyW0f/N2SXtOxzbCLYauweFB+c39/qJ6Ai9NI/OJI4kQM/L2sGvIyZJYX0R/0PGJrPOsfpYGiOL4sgqfa2zP9bL56S0eCz8N7+blw942lw/oN0EFMOdey86tpumsvpMi6t4LgDpZ2fkvNVsly5OXs= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9e883ffb-cf55-44b7-e703-08dd60155669 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4112.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Mar 2025 20:51:31.9076 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dmXiPmuh6adUxMRH02U1UHvZv6Y48TANT/wfjj3InLKE3ydNt0Z0FJY8sm6z0OEKUuLHUFd1i/5uQTKxJElz7yhbNI7GqT5SPt5q4BMCelM= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB7250 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1093,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-03-10_08,2025-03-07_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 spamscore=0 adultscore=0 malwarescore=0 phishscore=0 mlxlogscore=999 bulkscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2502100000 definitions=main-2503100159 X-Proofpoint-GUID: IHmFU78Knw3zo7l0vE2w-Dmc8cjUIbGO X-Proofpoint-ORIG-GUID: IHmFU78Knw3zo7l0vE2w-Dmc8cjUIbGO X-Rspamd-Server: rspam07 X-Rspamd-Queue-Id: C9D3D120009 X-Stat-Signature: kxmhgexn6t7inkcph8dqpmkfd4dmkzis X-Rspam-User: X-HE-Tag: 1741639898-273390 X-HE-Meta: U2FsdGVkX1+HY1hgcqaTi3ECSOVbhGQWykza4VIDZjGHreg+5gASNOr0EqQaw17pGrI6fNh6fOwgVsRu4E6oCK3GNKIQbxPMs66wp2vbUj1NFH41lZZ2blewD4zqJSwqZCA9zxMFn2c8OE+3phBuHvVCQTv80wOA2+niVCRje8aR18FFT7oT4tWMI50qIk/g/spQghQZL0I7Y4ygxIpLRixQ74hOcHnXhBMRb/MDe3OigV8AU505BVFAiZtbHzAM+wNnAowQtWloK4Kaw0QSUChtCu4+ZgiHWCEGgfvNvShBs2wKz9QwUefRfAy6s6Msdm0lME5HnGU2r18Xtr7JdWllwzEtN+lwTKobaHaDyPoMzObq/ywprVXFmTp3kyvSv7y4zQy6a4AotFEC2d8h+s0PcmqduG4mFZEsZo/ItlSUtydm6x+zS4a60molkxnkniIm+DZiYZwW0VxpTz5cm2pbdoLXTaKMVt0wJ1qLC61Lh87br5ig8of4rh726iTLgXzlkxtVhbJhiZ2yo4GsRG4XpuJA3MidtjFiIMvKxm1NEQcAWo2FFwSZsr+yIUwi6YJvNng2JSt5H+Diw3An1Kk12K5rlwOESdW9CW3zCERHgT1qLd165GOUUorZuHFFdypf3cgIBeVjiR36+qwcb4sWUqOyjKr2WPkBjtZSnvJ0WggFAWr5IPsS5FVInsqYDZQrHD+ua1Pt+cUlEM5UmENx8i487acLi/WfF7c25D0J6kgjv9qyevTbLjVSBQODKbnrQeCbx1nrxdJowyLPaHhl5QQVnDzzMtOXSFQXCz0mUzoQnA8UkGJZGa7ejjst/CgTrmBDLNv8FwWDF/SIYH27YVj9NzL85TszvWDTmAKcTv+wBxPcrWS15v4HfZSO4wANxIEoVmmqcAaApiUlWwSjk1MCDExVt/EyiHQWZA8+PvbfHyuXnuOkft4CwOrfsHLMQq0SiQALNHUd9NH rJrjMR9g Z0c2AULSMzh/7BtU+GwAaCVweO0nBIX5rFtka02DV+/Z8Mjfi4D5+JeEFY570aeZVpr8v62fYByvsOB1jEvp4ThCIFjSGxbTqCSr2MhfIaezbVjdZN+aRbLp9KEcqPKODWxDtwdVKGXDjVprY9ThkKKy9s8sMktg7LIOAQGK9LTtZmVtbEc9+3Ad+OZkOHYOXz5HFCYV/ovPJM2vTcFAf6SrXZI7VxG9GjawCY6cO4FcDW2GtDSG6nPGM3U9TrCIvpmoGwWYVWqnCYpU4gjsCbRcjmvQwiAky1GHfHWp8DxUnbFagGGFuSSVr/wVUVej2/ZXfNexZgiA++Of13ZCp7OsfMDwhEY2XcSEIcZfWj1Qyiq64fUxharsFZ/2vaqTFIEClVNUAm3ERRbXYuTR35Pwpu65eNnrgQ+YPruSR+A3r2d8Y5b8uB5mFUnqG7mZBmaUmJrxebHfekHm9zg2PWhr1PbQcN5AFdbWIwCj/CLmM52LTxxo0xUhFcn5MPIvs39uOYS2ohnHh8RAgZe3QFmfBh1XYE9664oeVB5HR9ET9z9frtFI3jXmcnM5htcEqonvXvyi4hCaholtcbNqAjyJGxcBufY63FBLZabnSaNwdd8XDZEoAM2s7TE1hIgFdh3UlJwsqsqCELkLn6a51nP2SBFpb5PwNAw/V X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Update move_vma() to use the threaded VRM object, de-duplicate code and separate into smaller functions to aid readability and debug-ability. This in turn allows further simplification of expand_vma() as we can simply thread VRM through the function. We also take the opportunity to abstract the account charging page count into the VRM in order that we can correctly thread this through the operation. We additionally do the same for tracking mm statistics - exec_vm, stack_vm, data_vm, and locked_vm. As part of this change, we slightly modify when locked pages statistics are counted for in mm_struct statistics. However this should cause no issues, as there is no chance of underflow, nor will any rlimit failures occur as a result. This is an intermediate step before a further refactoring of move_vma() in order to aid review. Signed-off-by: Lorenzo Stoakes Reviewed-by: Liam R. Howlett Reviewed-by: Vlastimil Babka --- mm/mremap.c | 186 ++++++++++++++++++++++++++++++++++------------------ 1 file changed, 122 insertions(+), 64 deletions(-) diff --git a/mm/mremap.c b/mm/mremap.c index af022e3b89e2..6305cb9a86f6 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -68,6 +68,7 @@ struct vma_remap_struct { bool mlocked; /* Was the VMA mlock()'d? */ enum mremap_type remap_type; /* expand, shrink, etc. */ bool mmap_locked; /* Is mm currently write-locked? */ + unsigned long charged; /* If VM_ACCOUNT, # pages to account. */ }; static pud_t *get_old_pud(struct mm_struct *mm, unsigned long addr) @@ -816,35 +817,88 @@ static unsigned long vrm_set_new_addr(struct vma_remap_struct *vrm) return 0; } -static unsigned long move_vma(struct vm_area_struct *vma, - unsigned long old_addr, unsigned long old_len, - unsigned long new_len, unsigned long new_addr, - bool *mlocked, unsigned long flags, - struct vm_userfaultfd_ctx *uf, struct list_head *uf_unmap) +/* + * Keep track of pages which have been added to the memory mapping. If the VMA + * is accounted, also check to see if there is sufficient memory. + * + * Returns true on success, false if insufficient memory to charge. + */ +static bool vrm_charge(struct vma_remap_struct *vrm) { - long to_account = new_len - old_len; - struct mm_struct *mm = vma->vm_mm; - struct vm_area_struct *new_vma; - unsigned long vm_flags = vma->vm_flags; - unsigned long new_pgoff; - unsigned long moved_len; - bool account_start = false; - bool account_end = false; - unsigned long hiwater_vm; - int err = 0; - bool need_rmap_locks; - struct vma_iterator vmi; + unsigned long charged; + + if (!(vrm->vma->vm_flags & VM_ACCOUNT)) + return true; + + /* + * If we don't unmap the old mapping, then we account the entirety of + * the length of the new one. Otherwise it's just the delta in size. + */ + if (vrm->flags & MREMAP_DONTUNMAP) + charged = vrm->new_len >> PAGE_SHIFT; + else + charged = vrm->delta >> PAGE_SHIFT; + + + /* This accounts 'charged' pages of memory. */ + if (security_vm_enough_memory_mm(current->mm, charged)) + return false; + + vrm->charged = charged; + return true; +} + +/* + * an error has occurred so we will not be using vrm->charged memory. Unaccount + * this memory if the VMA is accounted. + */ +static void vrm_uncharge(struct vma_remap_struct *vrm) +{ + if (!(vrm->vma->vm_flags & VM_ACCOUNT)) + return; + + vm_unacct_memory(vrm->charged); + vrm->charged = 0; +} + +/* + * Update mm exec_vm, stack_vm, data_vm, and locked_vm fields as needed to + * account for 'bytes' memory used, and if locked, indicate this in the VRM so + * we can handle this correctly later. + */ +static void vrm_stat_account(struct vma_remap_struct *vrm, + unsigned long bytes) +{ + unsigned long pages = bytes >> PAGE_SHIFT; + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma = vrm->vma; + + vm_stat_account(mm, vma->vm_flags, pages); + if (vma->vm_flags & VM_LOCKED) { + mm->locked_vm += pages; + vrm->mlocked = true; + } +} + +/* + * Perform checks before attempting to write a VMA prior to it being + * moved. + */ +static unsigned long prep_move_vma(struct vma_remap_struct *vrm, + unsigned long *vm_flags_ptr) +{ + unsigned long err = 0; + struct vm_area_struct *vma = vrm->vma; + unsigned long old_addr = vrm->addr; + unsigned long old_len = vrm->old_len; /* * We'd prefer to avoid failure later on in do_munmap: * which may split one vma into three before unmapping. */ - if (mm->map_count >= sysctl_max_map_count - 3) + if (current->mm->map_count >= sysctl_max_map_count - 3) return -ENOMEM; - if (unlikely(flags & MREMAP_DONTUNMAP)) - to_account = new_len; - if (vma->vm_ops && vma->vm_ops->may_split) { if (vma->vm_start != old_addr) err = vma->vm_ops->may_split(vma, old_addr); @@ -862,22 +916,46 @@ static unsigned long move_vma(struct vm_area_struct *vma, * so KSM can come around to merge on vma and new_vma afterwards. */ err = ksm_madvise(vma, old_addr, old_addr + old_len, - MADV_UNMERGEABLE, &vm_flags); + MADV_UNMERGEABLE, vm_flags_ptr); if (err) return err; - if (vm_flags & VM_ACCOUNT) { - if (security_vm_enough_memory_mm(mm, to_account >> PAGE_SHIFT)) - return -ENOMEM; - } + return 0; +} + +static unsigned long move_vma(struct vma_remap_struct *vrm) +{ + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma = vrm->vma; + struct vm_area_struct *new_vma; + unsigned long vm_flags = vma->vm_flags; + unsigned long old_addr = vrm->addr, new_addr = vrm->new_addr; + unsigned long old_len = vrm->old_len, new_len = vrm->new_len; + unsigned long new_pgoff; + unsigned long moved_len; + unsigned long account_start = false; + unsigned long account_end = false; + unsigned long hiwater_vm; + int err; + bool need_rmap_locks; + struct vma_iterator vmi; + + err = prep_move_vma(vrm, &vm_flags); + if (err) + return err; + + /* If accounted, charge the number of bytes the operation will use. */ + if (!vrm_charge(vrm)) + return -ENOMEM; vma_start_write(vma); new_pgoff = vma->vm_pgoff + ((old_addr - vma->vm_start) >> PAGE_SHIFT); - new_vma = copy_vma(&vma, new_addr, new_len, new_pgoff, + new_vma = copy_vma(&vrm->vma, new_addr, new_len, new_pgoff, &need_rmap_locks); + /* This may have been updated. */ + vma = vrm->vma; if (!new_vma) { - if (vm_flags & VM_ACCOUNT) - vm_unacct_memory(to_account >> PAGE_SHIFT); + vrm_uncharge(vrm); return -ENOMEM; } @@ -902,7 +980,7 @@ static unsigned long move_vma(struct vm_area_struct *vma, old_addr = new_addr; new_addr = err; } else { - mremap_userfaultfd_prep(new_vma, uf); + mremap_userfaultfd_prep(new_vma, vrm->uf); } if (is_vm_hugetlb_page(vma)) { @@ -910,7 +988,7 @@ static unsigned long move_vma(struct vm_area_struct *vma, } /* Conceal VM_ACCOUNT so old reservation is not undone */ - if (vm_flags & VM_ACCOUNT && !(flags & MREMAP_DONTUNMAP)) { + if (vm_flags & VM_ACCOUNT && !(vrm->flags & MREMAP_DONTUNMAP)) { vm_flags_clear(vma, VM_ACCOUNT); if (vma->vm_start < old_addr) account_start = true; @@ -928,13 +1006,12 @@ static unsigned long move_vma(struct vm_area_struct *vma, * If this were a serious issue, we'd add a flag to do_munmap(). */ hiwater_vm = mm->hiwater_vm; - vm_stat_account(mm, vma->vm_flags, new_len >> PAGE_SHIFT); /* Tell pfnmap has moved from this vma */ if (unlikely(vma->vm_flags & VM_PFNMAP)) untrack_pfn_clear(vma); - if (unlikely(!err && (flags & MREMAP_DONTUNMAP))) { + if (unlikely(!err && (vrm->flags & MREMAP_DONTUNMAP))) { /* We always clear VM_LOCKED[ONFAULT] on the old vma */ vm_flags_clear(vma, VM_LOCKED_MASK); @@ -947,22 +1024,20 @@ static unsigned long move_vma(struct vm_area_struct *vma, unlink_anon_vmas(vma); /* Because we won't unmap we don't need to touch locked_vm */ + vrm_stat_account(vrm, new_len); return new_addr; } + vrm_stat_account(vrm, new_len); + vma_iter_init(&vmi, mm, old_addr); - if (do_vmi_munmap(&vmi, mm, old_addr, old_len, uf_unmap, false) < 0) { + if (do_vmi_munmap(&vmi, mm, old_addr, old_len, vrm->uf_unmap, false) < 0) { /* OOM: unable to split vma, just get accounts right */ - if (vm_flags & VM_ACCOUNT && !(flags & MREMAP_DONTUNMAP)) + if (vm_flags & VM_ACCOUNT && !(vrm->flags & MREMAP_DONTUNMAP)) vm_acct_memory(old_len >> PAGE_SHIFT); account_start = account_end = false; } - if (vm_flags & VM_LOCKED) { - mm->locked_vm += new_len >> PAGE_SHIFT; - *mlocked = true; - } - mm->hiwater_vm = hiwater_vm; /* Restore VM_ACCOUNT if one or two pieces of vma left */ @@ -1141,9 +1216,7 @@ static unsigned long mremap_to(struct vma_remap_struct *vrm) if (err) return err; - return move_vma(vrm->vma, vrm->addr, vrm->old_len, vrm->new_len, - vrm->new_addr, &vrm->mlocked, vrm->flags, - vrm->uf, vrm->uf_unmap); + return move_vma(vrm); } static int vma_expandable(struct vm_area_struct *vma, unsigned long delta) @@ -1248,17 +1321,11 @@ static unsigned long check_mremap_params(struct vma_remap_struct *vrm) static unsigned long expand_vma_in_place(struct vma_remap_struct *vrm) { struct mm_struct *mm = current->mm; - long pages = vrm->delta >> PAGE_SHIFT; struct vm_area_struct *vma = vrm->vma; VMA_ITERATOR(vmi, mm, vma->vm_end); - long charged = 0; - - if (vma->vm_flags & VM_ACCOUNT) { - if (security_vm_enough_memory_mm(mm, pages)) - return -ENOMEM; - charged = pages; - } + if (!vrm_charge(vrm)) + return -ENOMEM; /* * Function vma_merge_extend() is called on the @@ -1271,15 +1338,11 @@ static unsigned long expand_vma_in_place(struct vma_remap_struct *vrm) */ vma = vrm->vma = vma_merge_extend(&vmi, vma, vrm->delta); if (!vma) { - vm_unacct_memory(charged); + vrm_uncharge(vrm); return -ENOMEM; } - vm_stat_account(mm, vma->vm_flags, pages); - if (vma->vm_flags & VM_LOCKED) { - mm->locked_vm += pages; - vrm->mlocked = true; - } + vrm_stat_account(vrm, vrm->delta); return 0; } @@ -1319,11 +1382,7 @@ static bool align_hugetlb(struct vma_remap_struct *vrm) static unsigned long expand_vma(struct vma_remap_struct *vrm) { unsigned long err; - struct vm_area_struct *vma = vrm->vma; unsigned long addr = vrm->addr; - unsigned long old_len = vrm->old_len; - unsigned long new_len = vrm->new_len; - unsigned long flags = vrm->flags; err = resize_is_valid(vrm); if (err) @@ -1356,7 +1415,7 @@ static unsigned long expand_vma(struct vma_remap_struct *vrm) */ /* We're not allowed to move the VMA, so error out. */ - if (!(flags & MREMAP_MAYMOVE)) + if (!(vrm->flags & MREMAP_MAYMOVE)) return -ENOMEM; /* Find a new location to move the VMA to. */ @@ -1364,8 +1423,7 @@ static unsigned long expand_vma(struct vma_remap_struct *vrm) if (err) return err; - return move_vma(vma, addr, old_len, new_len, vrm->new_addr, - &vrm->mlocked, flags, vrm->uf, vrm->uf_unmap); + return move_vma(vrm); } /*