From patchwork Mon Mar 3 11:08:35 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13998742 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 330EFC282C5 for ; Mon, 3 Mar 2025 12:26:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id A0B606B0085; Mon, 3 Mar 2025 07:26:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9BAE06B0088; Mon, 3 Mar 2025 07:26:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 810846B0089; Mon, 3 Mar 2025 07:26:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 643666B0085 for ; Mon, 3 Mar 2025 07:26:40 -0500 (EST) Received: from smtpin12.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 13A301C8371 for ; Mon, 3 Mar 2025 12:26:40 +0000 (UTC) X-FDA: 83180163360.12.22FAD52 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf23.hostedemail.com (Postfix) with ESMTP id 81204140007 for ; Mon, 3 Mar 2025 12:26:36 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=Mgt2ekRx; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=nmLdRnED; dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf23.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741004796; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Au38QI6Bec+OFkRDyXv79mMYH5dNLY0/KARSrcoXYSA=; b=zzyKHIyGlG4N43S/r5hmknNIuaLnx/oMFEn4ao8FNipfVNMEW/bIfZcm7obvdg3tF8KugI sFVe9ZFacrDztbA7+zfBhebsILeH5VI0tiT5SxNRcYw8kRETG8myzRFOJN2uABH8VKAiqZ Y41ozUyvU6JcznxdEdLc//hsNbcI6Hc= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1741004796; a=rsa-sha256; cv=pass; b=5AvhxG+od0xUrE8sr8xzGgZ21M0ObtPOv/LlaWBA2Vi2Xztz/KFGHskw0lHbNWCsdCJ9wY OzzmZpWXKipwvTgOenF+7OhUkI4U6VN//9NesXrWpQvsQ6R0yQ/klILOJ3SLC/1r60pol/ 4TzkB7cUyrXRmv1Y0y89jCfVNThhmWk= ARC-Authentication-Results: i=2; imf23.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=Mgt2ekRx; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=nmLdRnED; dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf23.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 5237tiV9026461; Mon, 3 Mar 2025 11:09:00 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=Au38QI6Bec+OFkRDyXv79mMYH5dNLY0/KARSrcoXYSA=; b= Mgt2ekRxUwVThS+ZM4eKes2Drg2u2pGICuVYJrt9gh+5lPcItlf0kXrGab4qsAOF kHiwuc9QgqVWBVntoTfPOJpp28tCO47JKQ9gynOmGsN1ZQIqjDSIk7i7X+rzzfsa Kbjjb3dfQMVik6sGVtJpiKoL3BFRdJ+sqZKSBcmq+p7lYq6WBtAoouoH/ytlQ1UF 4xyduJwTA3eEKeowu/EZ7y/EDHdCFLNPmIpnQeyb86DFcdsvyVXJsobEhwrgmsHI JmbjRw62f3pvpTE9oLHaC3NWnpAxoyzL0X0YtIiXb29eqm1kouxrmjPLve4u69JI OGZeWgNMH42We9bypaCvmQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 453u9qae10-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Mar 2025 11:09:00 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 523AiLnP039768; Mon, 3 Mar 2025 11:08:59 GMT Received: from nam12-mw2-obe.outbound.protection.outlook.com (mail-mw2nam12lp2046.outbound.protection.outlook.com [104.47.66.46]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 453rp82hmx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 03 Mar 2025 11:08:59 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vkrAc9NQ7HxhwOBW4XKEWDIBO+LrfWUEKCU75TWKLMf5hGsvShVDBdsCSdHKOyMv21ZZQY0HTTr1mXOENQWOAb4Prs+dM3dfVVlU34JVMH0KAo59xFIrh1HJvlhsOXX6WKoT6xgseMbdtU7TqZwEsfewBj7YwLmAVwRIVp3wjSrQxUhOko8A5QT5PGVCpURZrSvxEt7X5pTdUHUDp9bHRRoLvNyGo0rKCy0VuUy5gJ8Z/1yTOtdb5xsfJRfeNmILLyu5l+gEgzEcAfwtTtHxri+ypXS/k0C5jyFyKaES3LeSJp7P3AFOGrTOKHaTu/u13Dl+IDk7zyeImkzUzDj80Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Au38QI6Bec+OFkRDyXv79mMYH5dNLY0/KARSrcoXYSA=; b=fCUVZy4WO0g5gMhxuDKd1d/VDFTp8lpJuj7og/tgmf8awrsMqeX9KteteR8oitjSvo01uRV+tNm1tHR5WAlvp3rGawEKOlyRkQa0NwppTfadNHn+YNEBiWPc58NwdUETBq+URm63iky+EN/w4ICOOhec0JEPsUSciTJ10ztHXBNPorbutSbMYt+xoGXEyTmjNXmURAM3H9QmEu+xaWma1pDL21A1MIUZQEeVwv126J8xv3++DvnZlFCKtrQ/JKjNBpKLWSLzQV97eaU6Y7Tx+MbgLpb4WbaRrCUY06afD+Q/MrrOHUnVd+tqdKzg9kZaY8ZSmeJm/LQSniqDo4+RWA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Au38QI6Bec+OFkRDyXv79mMYH5dNLY0/KARSrcoXYSA=; b=nmLdRnEDFtMwvBAWrbYGJ9CVqrihqtW9dk9zg0S+uK24MOf9DQj4WzcR3eorHbmEGWMo/7VWj6x1jdYJOHIeiTe31iQqnsTiKBKUc5oXc6KXX054140z+gilbZioicX+FZ6HZcAMVKjWOBpModdUequ11ofyPL9D8NEvbYF71bs= Received: from MN2PR10MB4112.namprd10.prod.outlook.com (2603:10b6:208:11e::33) by DM4PR10MB7427.namprd10.prod.outlook.com (2603:10b6:8:181::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.27; Mon, 3 Mar 2025 11:08:58 +0000 Received: from MN2PR10MB4112.namprd10.prod.outlook.com ([fe80::3256:3c8c:73a9:5b9c]) by MN2PR10MB4112.namprd10.prod.outlook.com ([fe80::3256:3c8c:73a9:5b9c%7]) with mapi id 15.20.8489.025; Mon, 3 Mar 2025 11:08:57 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-mm@kvack.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/7] mm/mremap: complete refactor of move_vma() Date: Mon, 3 Mar 2025 11:08:35 +0000 Message-ID: <61a8071433adf3815713523a5c1db62cbe1e55a1.1740911247.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.48.1 In-Reply-To: References: X-ClientProxiedBy: LO4P265CA0246.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:350::9) To MN2PR10MB4112.namprd10.prod.outlook.com (2603:10b6:208:11e::33) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR10MB4112:EE_|DM4PR10MB7427:EE_ X-MS-Office365-Filtering-Correlation-Id: f93c710e-318d-4a3a-006e-08dd5a43cb59 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014; X-Microsoft-Antispam-Message-Info: qRQETfuCm8tEHR1Q2btnrPKTHtfRag5V1b0QlkpcWiA27KRuDS3HJsn3vZkyGYxX2JS0+a6tNaGx3EAuL/uwt/sKTZBZD9gAk/ASxdnjIaDXkp9ONxHDBe82meAOKdNhV0DDQtEACtfd4tqQ20aiOrn5QrRPT7uUjni7xLiaJP/8Rp5/OQAqO51hyG1fFLfublnjPZVepghEakf99aIdc2t30teUJnF/j7NCwqhMCtM1Vvo7kYDxVSQwFGZKGkwl5L71qLKQmbAoreYqVFchqfOcBp+mEUXwb8P0CTKMeMFcqaQJUGcq9355OzFd0ZeRa5pUN0d6dHjoB9XXMheKKY2vt0VPNQdHdd6728xMIccIZH2SEZfUaUxKkuokaLMWjOR0lDvR0gHhrMFhJKlEufuNdjawb7BkYRkkM+9ZuPV0z8FhWckZWDCqow3w59oQDScBopphRMrOi70VD8YmLK53p5iqFE9YNNGgr6RYgqHFlGlIE0l6gaHSbCDBuUv5iVSkABTp8ithakk7O30niyL0Yx5vVcNqx3x5ezATzuGgBsElgW74+DLixKO7V7K7fwSQCGBvybwPFpzJ2S+v4grEM/mwIhJMQT+VN+N0yG+flEMPmokrhUjfXHJHN/cYwRkSU+OdJpuPgR1mkVd7ngYG1zVNp9uMQL6J8ZNYCjlbIatOHh7iJ7E40xTkxl6ASLN+fKNn7PWyv/Jg3ziUGhBIeWHvFEzVwkO/qgoO90BUC/lkA95UXlFroanmZBxQgjLI+aHZUj+zSknBlZPoRmp9tsN2FZC/pPZFvh/df7VogRHQ5yQmERImNpFB1tKHfCWvdDG3igAyU28T18A6Ux3fVu6YziH8YMfHjE1bmxwfD4GOVkY2CfXtYgD+ltj890gqwVe5+/ZyDLfVJfZIz18rkFXmxrO3Aif/odBxF0MDHlktGHlLbKXIQ0Tp8zAPOBMbgQM+HzPIiGQmFN/XNi5Cimo+S68/KpCk64vX0Q/U35T0Slr3sspOkTmtiUJJkllHpLdn/d6UGFTmm7kApSFwoSIzaAfXCzPAA9D6PI8X4YlJ/50Qtcr2Ag9Ok0jHEAfSO5TVa0dmPUy7eRHz0R1yIWlDBqe5Lnc7PKzqVZuqrQWDIdLrr3PVOegQrKZUqesdQSbPo7TwJ6+nUYRrYjF7uSzgR6lmVM/UMKpBaxH99SFAsbErrXWFsB31f1zXcOFxfAr8kuTbVJkvbltunagpLyXtwi9Q3NHeCz0nDDkgdKL7TrRuyozsxGr3DyAWQZfts8S1MWzJr55z4t4VTNZRs2b6VxcWOg3fgCNUWc0awrcmjsxyUujvc1qtNkgPSXXBbupw9rE3N+7DcZYHvTyBaMIEJOo6cqmwODwf2MYYZMFp+szoDeFpPgMcxmSU X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB4112.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: MKkuKZ1OqBOeo6QTj2OtGgrblUUbQLb3GrCAXPiF6+eG0AMkf1Y17+SGjal9dBvSzd9g7k8VnO72XaWQXuxBeliAAEuNKjJX0iomIKI+3/yReA8sHd9yrVeYeyDeby/E3i45/B3JXfWkdNg3LAkZ++fsXPMypCz4yo3lChRlPz1g6+sn/ml9hXFZa1XVl5izkxIHQRg9NfcYPXQphQH+5saq9BWo7r3/dVZ4Kp7zByxp612ojLSfRBmQKKhB7UjWMbxVrxnf3eIsx1Pz+I0kvfBkERLReaKEpP6HagU+/jXlTPnJ71pdSNjBpNHUgXkTe7733bKnyomrcswz+w3+LbRVdvGgasUnqGUchhXcUhanvrRpi8jf394AJ9p0wGCA9+hMxnCYEFG9iph2xz6xXkVKt8D8wzE6WZwe54xDNTdlnqSgXlnug64fI78LQtfDnSRB2zXB8BL7erLepwRqMC95dolCi+sTJsrj7OomVUwatpYC+gxCyb5B2Qe+Lpm/5E+YEGUSg005RFiCy5n75MGKbwm4OkXDXu0+bvOCEXNQqxw3zr3OZxBcGPoa1DJn9YaqdDcRhCMa7dyZWB7945vbGeXE4wO/C41htob1Oae4cZy8mTLgkiBwyMbVH8M84V5Yor2EkYnoh3TAyCJkw6JlXpXITQe/x0dcppex9RbPw/cIhdelaC1uyDTZScRVOsmneoSbRnUsxOt6gCsx5ns4sgpyR9YnvIFc91IuVba5jlsYf8LMuVHTfS90u2zatr8nwC6c8+um6u6McFEKPTedj3dMoF2TyhV3eutBMUgytvQYoAp6THzg7W51ATpc4cRandkeoMnVdBkRk2aCAe96pBK5dLYNHTIY8KybEwzQ02/SPH6lfNRJyntT1WhBwn2/uMDMmpRWpyqdTLmSh41iAVozLg5LKCl8hubjK3hBtNy81Wk3p4Mj3TAAUbDGO7TD7h1tr7ko/w7pKILxPkcibjFpwVDFvnx6790mEhWq/+AawERyZ3PzFJ3F2BaMHhfxuDRbxua6kAX2ruLn8AwA81jWGn/onrry3wGuB3HVLmviq62bRTqY0ktoAYNek/z5PGk15DkOK0Kk7BCRoZlkuHCDUfqMVofZ8IEU3hUa62pyoCRKpAEOelGbk2tlBTGhzdyGee+nV6ohwrpvxeytMwbCdwWzjLu/rv6gkzxkLjr7Asevj9hyq/vnw474yKJvUe21/cST6Sj9Hdm4Qe8rNLUImuGfl6VMGUwqs2KIn5H6VM1xgVPJCXDMMUpkZWTPQk1Jri0iE4mx82OdUqZ1mUv1LN77Dby5kHetQZtG82ljlv1TytK4/8+8cDUj55iC2zS+220l6ydYNpmw8jQ/BL8GT99eCZmOXLIrJzo64YAHwwYcHisTQvt5gj/wYvA2GYs9j7x+UOGo2rqSxsUMtcb5Z7c/uj5UmAmYrq380xwtNG6TqhLW1KDvXt0z+iB1ld5fs4bWBl7jf4F5J+60pbhn3A9Oig+vVzQ+ZFzCZ5PeFrx8kCOOmPaNvxJDWHm8vlm5vy0f5K2sPVyJ6X7XZhSerARi8N6lTl6+bVYx4s+4l7429Bsk3cD22mh0/8y2uI+hZT1S3MvBX1gA8A== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: I5YimbcIR7hzzno79S/mokoTxaoyBDWuBp054+ne4yoFvatEpzrlLZZ6ms+FJcQ91Vy9pfrynpt/tCL3xz/tEB2RpF0X47CmB4hTVaJozIZI2fNFMWud6TM7aF3MdnvkbNRZX3lDMFq4zUmrBTpJTmDcVdE7et2kkgg/cdsCC5Me3VyMz7ZlbBGFY6Rp7mqeVAfoMQgLJq1AP0QQyL/vS8Iq9x5M/nAk0x/isajvbNEzHf77ki7hkAMOHmm+/ljEYZw2nRwg6TYoWvzib21yw6kj84kR52PICjvBMi6NEYM7zc/JWUC7qCCJzoGOf65UP1lsSaEEJ+BHa8U+ppaDl9R5gyhjtzM6fsCCMUVPYuSjZYzwMGoRf8JRMKBVnKRF807egHqq+fRNrHOf8jorlUYhHhbqJ0bWxf66+OL2zswRU6K7UnOY5ylfhWp50FJn5JMhZwtP7gwmkcHbj8b/QklmbxPmqFVvcBhaGFLiXZ+znwOAeI/DsPCOlGCDTwHa1emAXGd5eR1uCUXB8Px1/SdlJkalLdFLLrFoOAoMGJcis8EcpL0X9IcbqN0bctW65F1+/o/YJFY00B4u63WkWiV3jTXrlufM6uw8SllDkc4= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: f93c710e-318d-4a3a-006e-08dd5a43cb59 X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4112.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Mar 2025 11:08:57.9019 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wwvxwxaJ4doyWXUQGQp1E+AAcjtnSdQ1Xhco5kXpDH9QAnzIx9aqOVxBXyyzBSdJBGUwmXHdCbFtiZi/3p1lHyC5C3YnsxjlGsjMw860s3Y= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR10MB7427 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-03-03_04,2025-03-03_01,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 suspectscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 malwarescore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2502100000 definitions=main-2503030086 X-Proofpoint-ORIG-GUID: iGtWUZAdCvS-U_l0Rv6oWk35Vgms-Xva X-Proofpoint-GUID: iGtWUZAdCvS-U_l0Rv6oWk35Vgms-Xva X-Rspam-User: X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 81204140007 X-Stat-Signature: 4wnf1j9fnk6t7ked1dtqi68gpgn61u5g X-HE-Tag: 1741004796-137860 X-HE-Meta: U2FsdGVkX1+OQvm5VXY8AnGAQEtehl1iu7RUQjnpXv1Zxma7Nlm6uXSMic/tiSnifOwjt0CbFD87Hfv/cC4qeBsEbNCyn2BAW+AtMYDZkqHJxdYJA9dsPqoCV3LIVNBi0ZFUb0ZvfmcReaebBbZjJBSovsmocoKJIJ8tV2Ibzgp/BcRR/ufoXQsh5jtX3jHB6Eco2mfdbT7VJdLRYPkAOvlJODaO3M+RX2GeXG4tB3kXYLY9xSGwnSjfo1BlzNekfT0QDNwJDyAt3yXg4JQZL+HF7PYDMjZd52s+2RkOGMtWD2qRJ332F+x2dijWHJTBy+BduhTQlJ2LAfMY/xmul8IIwbe1DCd6mJvKI7wTJ3eFeArzQtNdg/rSFi86nE8O43uEY8+UZYmIAujfsezVxzsvm4k7OLmbvhgZEzfa2dw/QCmFD3iy24i69N0nEbB2CxkDwPRBI/HkN1w4yUt2mtjImBwtNGibMZmvYrcthQ8l3OX1nL17pvZtpahyb+9EHIqqooqA3TSCp0i4pmzvIWtca3fKMdVGFQ9vkbaCIFVoCd4GZ/YA7z3oIB3DZfQse72G5FgRQvJWmhpeQB9NrRK8aa3NReqPzREBo2hWpZ3glELVhIktx58C7MYDTna2JZbtvVARn/jEm1OVUIvVoHyKzB+hm+OzNcQPf/1dYKVGOKOmQjB51JLiAiVIkrD4E2My7zJSAIJm/PGMYDrwnv9JT5YCJattbG37Hx0anGX0vDbbNlmHxIiknk/kLUSzdn+U6RJzOXcyi0shhF3Nm3GpPUSrlWowt6M1PH1UFIVwcuVhZu37jiJoJUTZesKn2WBHQbgGZC5bmcM/wr+mZ7TYdyD0iZRfOW7IQHR67ObO5EvImiB3mKzlvNUtSQOeYfjQmh+EM54HBCHeW4+8uny4LSkJmKYdBR8l03yDtM3xyinkdppEsonC/Ke9uoBAxAsWRc/XslDKF1Fo0RB g2T2stRc Usd0vWmt0i0kK+jsCiVuFU3ku1hWfZ5qCkzjVmF/HnqijWv28o5UQQ+/xU195mPjNcW1gSqusiIRMYKt/F7rR/a6FPgy6OZPGbTngj2XNnsN2h2fdMGWLctYRoAoSLu3Sw1+NXA7v3kUSC6jjyW+bY8r1HZPTrJyXI7stUHZ5gUxLPp5LMNgm71KL+Qcdq65sCYGnT0IBaOPBIa9lqazPRpHDHFXEDs3UQ+yrGIaoIOzwQhtaxolG4Fp0MODclzLg/zLs7TkknO7krl/DprMiaEBCeY4h4KcGhXLnHAfONGt4BOGMCwQtdV1EZoA4Fvt5CSzsswGmjK0IU2dgCkS4jN+l50Vv3PhVNT3HF15J1VkVOQyDERM+tHyqp9tQh5mb3zjRoZ7WnslS7F+v15+3duQFd95d7cXl+41BIDsp4NTqnhBd5rmG+RlpLlU7cq1tGcN4E4c3+RQMOz7MMsKHAPFakxIhrg6zRW5eSA4/Sj1zwAYy5LRVxMtneid34TgdLp1UD+bmTU+AkFiGvnGjhgC9w0mLY3wvYfK/tWQ1B77XAusd2FU73ZD1F8pjiR8DFS94leduBWMkGbxeKHca2+CqfbZiPAWVqBl3MDt3w5FG6eECEbcm5DMV6VXiKprOXlGirFAlO5Q1lT4JZFWxfXyn8g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000001, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We invoke ksm_madvise() with an intentionally dummy flags field, so no need to pass around. Additionally, the code tries to be 'clever' with account_start, account_end, using these to both check that vma->vm_start != 0 and that we ought to account the newly split portion of VMA post-move, either before or after it. We need to do this because we intentionally removed VM_ACCOUNT on the VMA prior to unmapping, so we don't erroneously unaccount memory (we have already calculated the correct amount to account and accounted it, any subsequent subtraction will be incorrect). This patch significantly expands the comment (from 2002!) about 'concealing' the flag to make it abundantly clear what's going on, as well as adding and expanding a number of other comments also. We can remove account_start, account_end by instead tracking when we account (i.e. vma->vm_flags has the VM_ACCOUNT flag set, and this is not an MREMAP_DONTUNMAP operation), and figuring out when to reinstate the VM_ACCOUNT flag on prior/subsequent VMAs separately. We additionally break the function into logical pieces and attack the very confusing error handling logic (where, for instance, new_addr is set to err). After this change the code is considerably more readable and easy to manipulate. Signed-off-by: Lorenzo Stoakes --- mm/mremap.c | 292 ++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 203 insertions(+), 89 deletions(-) diff --git a/mm/mremap.c b/mm/mremap.c index fdbf5515fc44..1ceabd0d9634 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -889,13 +889,13 @@ static void vrm_stat_account(struct vma_remap_struct *vrm, * Perform checks before attempting to write a VMA prior to it being * moved. */ -static unsigned long prep_move_vma(struct vma_remap_struct *vrm, - unsigned long *vm_flags_ptr) +static unsigned long prep_move_vma(struct vma_remap_struct *vrm) { unsigned long err; struct vm_area_struct *vma = vrm->vma; unsigned long old_addr = vrm->addr; unsigned long old_len = vrm->old_len; + unsigned long dummy = vma->vm_flags; /* * We'd prefer to avoid failure later on in do_munmap: @@ -921,56 +921,150 @@ static unsigned long prep_move_vma(struct vma_remap_struct *vrm, * so KSM can come around to merge on vma and new_vma afterwards. */ err = ksm_madvise(vma, old_addr, old_addr + old_len, - MADV_UNMERGEABLE, vm_flags_ptr); + MADV_UNMERGEABLE, &dummy); if (err) return err; return 0; } -static unsigned long move_vma(struct vma_remap_struct *vrm) +/* + * Unmap source VMA for VMA move, turning it from a copy to a move, being + * careful to ensure we do not underflow memory account while doing so if an + * accountable move. + * + * This is best effort, if we fail to unmap then we simply try + */ +static void unmap_source_vma(struct vma_remap_struct *vrm) { struct mm_struct *mm = current->mm; + unsigned long addr = vrm->addr; + unsigned long len = vrm->old_len; struct vm_area_struct *vma = vrm->vma; - struct vm_area_struct *new_vma; - unsigned long vm_flags = vma->vm_flags; - unsigned long old_addr = vrm->addr, new_addr = vrm->new_addr; - unsigned long old_len = vrm->old_len, new_len = vrm->new_len; - unsigned long new_pgoff; - unsigned long moved_len; - unsigned long account_start = false; - unsigned long account_end = false; - unsigned long hiwater_vm; + VMA_ITERATOR(vmi, mm, addr); int err; - bool need_rmap_locks; - struct vma_iterator vmi; + unsigned long vm_start; + unsigned long vm_end; + /* + * It might seem odd that we check for MREMAP_DONTUNMAP here, given this + * function implies that we unmap the original VMA, which seems + * contradictory. + * + * However, this occurs when this operation was attempted and an error + * arose, in which case we _do_ wish to unmap the _new_ VMA, which means + * we actually _do_ want it be unaccounted. + */ + bool accountable_move = (vma->vm_flags & VM_ACCOUNT) && + !(vrm->flags & MREMAP_DONTUNMAP); - err = prep_move_vma(vrm, &vm_flags); - if (err) - return err; + /* + * So we perform a trick here to prevent incorrect accounting. Any merge + * or new VMA allocation performed in copy_vma() does not adjust + * accounting, it is expected that callers handle this. + * + * And indeed we already have, accounting appropriately in the case of + * both in vrm_charge(). + * + * However, when we unmap the existing VMA (to effect the move), this + * code will, if the VMA has VM_ACCOUNT set, attempt to unaccount + * removed pages. + * + * To avoid this we temporarily clear this flag, reinstating on any + * portions of the original VMA that remain. + */ + if (accountable_move) { + vm_flags_clear(vma, VM_ACCOUNT); + /* We are about to split vma, so store the start/end. */ + vm_start = vma->vm_start; + vm_end = vma->vm_end; + } - /* If accounted, charge the number of bytes the operation will use. */ - if (!vrm_charge(vrm)) - return -ENOMEM; + err = do_vmi_munmap(&vmi, mm, addr, len, vrm->uf_unmap, /* unlock= */false); + vrm->vma = NULL; /* Invalidated. */ + if (err) { + /* OOM: unable to split vma, just get accounts right */ + vm_acct_memory(len >> PAGE_SHIFT); + return; + } - vma_start_write(vma); - new_pgoff = vma->vm_pgoff + ((old_addr - vma->vm_start) >> PAGE_SHIFT); - new_vma = copy_vma(&vrm->vma, new_addr, new_len, new_pgoff, + /* + * If we mremap() from a VMA like this: + * + * addr end + * | | + * v v + * |-------------| + * | | + * |-------------| + * + * Having cleared VM_ACCOUNT from the whole VMA, after we unmap above + * we'll end up with: + * + * addr end + * | | + * v v + * |---| |---| + * | A | | B | + * |---| |---| + * + * The VMI is still pointing at addr, so vma_prev() will give us A, and + * a subsequent or lone vma_next() will give as B. + * + * do_vmi_munmap() will have restored the VMI back to addr. + */ + if (accountable_move) { + unsigned long end = addr + len; + + if (vm_start < addr) { + struct vm_area_struct *prev = vma_prev(&vmi); + + vm_flags_set(prev, VM_ACCOUNT); /* Acquires VMA lock. */ + } + + if (vm_end > end) { + struct vm_area_struct *next = vma_next(&vmi); + + vm_flags_set(next, VM_ACCOUNT); /* Acquires VMA lock. */ + } + } +} + +/* + * Copy vrm->vma over to vrm->new_addr possibly adjusting size as part of the + * process. Additionally handle an error occurring on moving of page tables, + * where we reset vrm state to cause unmapping of the new VMA. + * + * Outputs the newly installed VMA to new_vma_ptr. Returns 0 on success or an + * error code. + */ +static int copy_vma_and_data(struct vma_remap_struct *vrm, + struct vm_area_struct **new_vma_ptr) +{ + unsigned long internal_offset = vrm->addr - vrm->vma->vm_start; + unsigned long internal_pgoff = internal_offset >> PAGE_SHIFT; + unsigned long new_pgoff = vrm->vma->vm_pgoff + internal_pgoff; + unsigned long moved_len; + bool need_rmap_locks; + struct vm_area_struct *vma; + struct vm_area_struct *new_vma; + int err = 0; + + new_vma = copy_vma(&vrm->vma, vrm->new_addr, vrm->new_len, new_pgoff, &need_rmap_locks); - /* This may have been updated. */ - vma = vrm->vma; if (!new_vma) { vrm_uncharge(vrm); + *new_vma_ptr = NULL; return -ENOMEM; } + vma = vrm->vma; - moved_len = move_page_tables(vma, old_addr, new_vma, new_addr, old_len, - need_rmap_locks, false); - if (moved_len < old_len) { + moved_len = move_page_tables(vma, vrm->addr, new_vma, + vrm->new_addr, vrm->old_len, + need_rmap_locks, /* for_stack= */false); + if (moved_len < vrm->old_len) err = -ENOMEM; - } else if (vma->vm_ops && vma->vm_ops->mremap) { + else if (vma->vm_ops && vma->vm_ops->mremap) err = vma->vm_ops->mremap(new_vma); - } if (unlikely(err)) { /* @@ -978,28 +1072,84 @@ static unsigned long move_vma(struct vma_remap_struct *vrm) * which will succeed since page tables still there, * and then proceed to unmap new area instead of old. */ - move_page_tables(new_vma, new_addr, vma, old_addr, moved_len, - true, false); - vma = new_vma; - old_len = new_len; - old_addr = new_addr; - new_addr = err; + move_page_tables(new_vma, vrm->new_addr, vma, vrm->addr, + moved_len, /* need_rmap_locks = */true, + /* for_stack= */false); + vrm->vma = new_vma; + vrm->old_len = vrm->new_len; + vrm->addr = vrm->new_addr; } else { mremap_userfaultfd_prep(new_vma, vrm->uf); } - if (is_vm_hugetlb_page(vma)) { + if (is_vm_hugetlb_page(vma)) clear_vma_resv_huge_pages(vma); - } - /* Conceal VM_ACCOUNT so old reservation is not undone */ - if (vm_flags & VM_ACCOUNT && !(vrm->flags & MREMAP_DONTUNMAP)) { - vm_flags_clear(vma, VM_ACCOUNT); - if (vma->vm_start < old_addr) - account_start = true; - if (vma->vm_end > old_addr + old_len) - account_end = true; - } + /* Tell pfnmap has moved from this vma */ + if (unlikely(vma->vm_flags & VM_PFNMAP)) + untrack_pfn_clear(vma); + + *new_vma_ptr = new_vma; + return err; +} + +/* + * Perform final tasks for MADV_DONTUNMAP operation, clearing mlock() and + * account flags on remaining VMA by convention (it cannot be mlock()'d any + * longer, as pages in range are no longer mapped), and removing anon_vma_chain + * links from it (if the entire VMA was copied over). + */ +static void dontunmap_complete(struct vma_remap_struct *vrm, + struct vm_area_struct *new_vma) +{ + unsigned long start = vrm->addr; + unsigned long end = vrm->addr + vrm->old_len; + unsigned long old_start = vrm->vma->vm_start; + unsigned long old_end = vrm->vma->vm_end; + + /* + * We always clear VM_LOCKED[ONFAULT] | VM_ACCOUNT on the old + * vma. + */ + vm_flags_clear(vrm->vma, VM_LOCKED_MASK | VM_ACCOUNT); + + /* + * anon_vma links of the old vma is no longer needed after its page + * table has been moved. + */ + if (new_vma != vrm->vma && start == old_start && end == old_end) + unlink_anon_vmas(vrm->vma); + + /* Because we won't unmap we don't need to touch locked_vm. */ +} + +static unsigned long move_vma(struct vma_remap_struct *vrm) +{ + struct mm_struct *mm = current->mm; + struct vm_area_struct *new_vma; + unsigned long hiwater_vm; + int err; + + err = prep_move_vma(vrm); + if (err) + return err; + + /* If accounted, charge the number of bytes the operation will use. */ + if (!vrm_charge(vrm)) + return -ENOMEM; + + /* We don't want racing faults. */ + vma_start_write(vrm->vma); + + /* Perform copy step. */ + err = copy_vma_and_data(vrm, &new_vma); + /* + * If we established the copied-to VMA, we attempt to recover from the + * error by setting the destination VMA to the source VMA and unmapping + * it below. + */ + if (err && !new_vma) + return err; /* * If we failed to move page tables we still do total_vm increment @@ -1012,51 +1162,15 @@ static unsigned long move_vma(struct vma_remap_struct *vrm) */ hiwater_vm = mm->hiwater_vm; - /* Tell pfnmap has moved from this vma */ - if (unlikely(vma->vm_flags & VM_PFNMAP)) - untrack_pfn_clear(vma); - - if (unlikely(!err && (vrm->flags & MREMAP_DONTUNMAP))) { - /* We always clear VM_LOCKED[ONFAULT] on the old vma */ - vm_flags_clear(vma, VM_LOCKED_MASK); - - /* - * anon_vma links of the old vma is no longer needed after its page - * table has been moved. - */ - if (new_vma != vma && vma->vm_start == old_addr && - vma->vm_end == (old_addr + old_len)) - unlink_anon_vmas(vma); - - /* Because we won't unmap we don't need to touch locked_vm */ - vrm_stat_account(vrm, new_len); - return new_addr; - } - - vrm_stat_account(vrm, new_len); - - vma_iter_init(&vmi, mm, old_addr); - if (do_vmi_munmap(&vmi, mm, old_addr, old_len, vrm->uf_unmap, false) < 0) { - /* OOM: unable to split vma, just get accounts right */ - if (vm_flags & VM_ACCOUNT && !(vrm->flags & MREMAP_DONTUNMAP)) - vm_acct_memory(old_len >> PAGE_SHIFT); - account_start = account_end = false; - } + vrm_stat_account(vrm, vrm->new_len); + if (unlikely(!err && (vrm->flags & MREMAP_DONTUNMAP))) + dontunmap_complete(vrm, new_vma); + else + unmap_source_vma(vrm); mm->hiwater_vm = hiwater_vm; - /* Restore VM_ACCOUNT if one or two pieces of vma left */ - if (account_start) { - vma = vma_prev(&vmi); - vm_flags_set(vma, VM_ACCOUNT); - } - - if (account_end) { - vma = vma_next(&vmi); - vm_flags_set(vma, VM_ACCOUNT); - } - - return new_addr; + return err ? (unsigned long)err : vrm->new_addr; } /*