From patchwork Mon Mar 10 20:50:38 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 14010665 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C0A0BC28B30 for ; Mon, 10 Mar 2025 20:51:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 76DE028000A; Mon, 10 Mar 2025 16:51:43 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6F4B6280004; Mon, 10 Mar 2025 16:51:43 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 45F4728000A; Mon, 10 Mar 2025 16:51:43 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 188D7280004 for ; Mon, 10 Mar 2025 16:51:43 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 721B9C1224 for ; Mon, 10 Mar 2025 20:51:44 +0000 (UTC) X-FDA: 83206837728.16.6458677 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf26.hostedemail.com (Postfix) with ESMTP id E13E714000B for ; Mon, 10 Mar 2025 20:51:40 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=JPG8LQpy; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=uh92e+t3; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf26.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1741639901; a=rsa-sha256; cv=pass; b=pGhmukkqLMd12uVA/cBlmLFFMm/jvXGH117Yef2M2o0N3viTFIkJDtkP9KPe9LlyVUWqDt P86263ZOl8aslvAVijV7m+9BXdMPM3YgTXuYxAzUjotS2NJWx2i9HJDb2ncrQ/qSOtpp8Q 0S5FCKpDVWlytapt51sYvNI6etruZFk= ARC-Authentication-Results: i=2; imf26.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=JPG8LQpy; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=uh92e+t3; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf26.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1741639901; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=kYvPH1xqsucyarxMcUO2+UA36cQxvVaeGgCKwzhQaf0=; b=3aPlZiz6bKIO3BnheCpkRF0k+wIitHiH8qTvvPlwb+o450kza7Xnsx+HPA9yrRWkFwI7J7 aFiHRyKCr/lK87ldA6msJxGsyB/TaSqHGXRpNPHdexGmPU9aldrP5FBClesmmi+qlNEf/O ZEs5wyDCYKMQhSQi9RN2wk7NdpSO9HM= Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 52AJtlPc023523; Mon, 10 Mar 2025 20:51:38 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=kYvPH1xqsucyarxMcUO2+UA36cQxvVaeGgCKwzhQaf0=; b= JPG8LQpyulAtBEODvzZ9K8fb4TMlpMOZE6jOV3A6h9YK1e5PYZd1eyUoyBWlxFWq grd6QqvPoj6ltdzKff+9uOP1miRKf0gJZG9T29EAFM017vsbQsLJYSOfBcLge7ZJ lSqc2zebe3XuHEgHLy0oSNQrYIjbTvRNDF5eJLlouucPaBUzMbQ4fiCI2KDevuhS LBdT3MqHWUeFOcfLm00r5pGjUaKCwn5nHnDr/8M+q+athXH69A8THyszVNnM5pYM zLTPsn6e9d7xsAeu27dCiN3bGWjv8U42eUk2sePWNVdtmczFX2CE6MwkfXCR/Fkn ExM2NCVeFi04Kf6BwmbHpQ== Received: from phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta01.appoci.oracle.com [138.1.114.2]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 458ctb3m7a-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Mar 2025 20:51:37 +0000 (GMT) Received: from pps.filterd (phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 52AJmvP4030644; Mon, 10 Mar 2025 20:51:36 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2169.outbound.protection.outlook.com [104.47.57.169]) by phxpaimrmta01.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 458gcmgcuc-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 10 Mar 2025 20:51:36 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=inID2LYUTqQwsaTgjdGveTDKVfpkCkH0kzkI28kD3Pzqu8Mvhm+sHJ0Hbao8NmmBLvXkVCSXRTh41Q1btOJTHPAfJdW9SNEfRAagKW7tEZglcFtu9SKNaWBBf3dlTJQ4lhw3xNNnidJqzbBa0xrltQbu06gGmiuhoFNwzYptzDzP3EaCBz8zqM2U/gIw3OmtAZCgslXgEDLVL7WOgQdq/34HwlYIXC1FsKHMp3t+vZsMNUvZpsRt4vaBTkO00wFQjw2jI7nhA+0BLFHrKFcxQwHnU+tnOgxWjgPbVkpwy1t/w51jVOHSep4v6r3vq3bW01EiJjsF1zbWYfSlJrWLYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=kYvPH1xqsucyarxMcUO2+UA36cQxvVaeGgCKwzhQaf0=; b=koM6tBriC3P/GwCGFvh8cV2b0uH4msrZa5OkWltwANNpzzYNsgGUFVwjxAwCspm/cDbZ6FJPGwtC7sTVzVu7R27g2s3HHBNT9WdM8nITuetg0dsyGVutehmAJluh4ZlcDgh2zrp6Yjjta2w5JQ9MVLdIOKiu/I9//JcTN0tYUw4etFM4dp1dBJTuDppOeNTot5euXOnhkmS1UK48Yjxt1umEryvLhTvPHcppAGWNlOfDFn6IediaLftnEalZslvVEUc3+Oj89x9+KUjvcHCyDspkCH3o2aT62id6aBabUHz7U/0voW6ICdJ2uycNn/RT+f69Mpa+nLiv/p+jysyWNg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=kYvPH1xqsucyarxMcUO2+UA36cQxvVaeGgCKwzhQaf0=; b=uh92e+t3g+mzJaQ64QRT5O5uNafRtNt7UK/sww6OepTWm7kwFmgoaTecURo3QMtydDXBRebqBiKYuSxT44Blt5QTCv0sBIxN6U8aU0fJGTxGas81fHr7fFw0uuE+tOMQJgwAOCDT5rR6jsEzYM+GLiWunHEpDv+ylMSzPT/BegM= Received: from MN2PR10MB4112.namprd10.prod.outlook.com (2603:10b6:208:11e::33) by DS7PR10MB7250.namprd10.prod.outlook.com (2603:10b6:8:e0::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8511.27; Mon, 10 Mar 2025 20:51:34 +0000 Received: from MN2PR10MB4112.namprd10.prod.outlook.com ([fe80::3256:3c8c:73a9:5b9c]) by MN2PR10MB4112.namprd10.prod.outlook.com ([fe80::3256:3c8c:73a9:5b9c%7]) with mapi id 15.20.8489.025; Mon, 10 Mar 2025 20:51:34 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Harry Yoo , Yosry Ahmed , Oliver Sang Subject: [PATCH v3 5/7] mm/mremap: complete refactor of move_vma() Date: Mon, 10 Mar 2025 20:50:38 +0000 Message-ID: X-Mailer: git-send-email 2.48.1 In-Reply-To: References: X-ClientProxiedBy: LO2P265CA0432.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a0::36) To MN2PR10MB4112.namprd10.prod.outlook.com (2603:10b6:208:11e::33) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN2PR10MB4112:EE_|DS7PR10MB7250:EE_ X-MS-Office365-Filtering-Correlation-Id: 0593843c-9e42-4989-cc7f-08dd6015579e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016; X-Microsoft-Antispam-Message-Info: Gj8tvzM+yfpZkKJ4opUv20PcfEmlYowfr03KhNk+Xu4LeAEfqwaouQNANQBcUM8ukSmjjw1IM4s6TYweOEnndmgL5DSVBlHzWm03OkkvY9UnE7nRdYGuHQ8eYuN/PlKgZMFulnhV9UlNHKLWzQr2ytzYOGENyyAQBplmiMG5jIHmgKQc3t5JOfFqrlfKhVMskRHVtlirP5mx9MUeJy/Wh9fl/zipZmRwtLWf56EhvgSChasAlXrn0pqiQoLiS8gezK/7N5CoTWr0r4bK+RFqDTmid+bMviq2XNr6OG4yrWubMFabVSMmc+/S1mcqIj23t27/qAj7L1GHBFZY5x9/H3R9usSRZ739ew1gfSGiNeFeJSQXAQVIqmGTCzYtKAx+0YtfcUHgd2+r065YYY+60qHQcPkJGBuF4YTCN/c3giwk7CxZ03u/SyRNPhd1csJhTSJCAxsYDSkcREnglY868dyg4pvXWD17c717UgjUQVWZnmkkRSOYqhkgW2JnSsaHUpRQT6WMBsbiBgc6LQnzfcbT+cplrGTuumGAhAWmLE6e7+TApdGGdM5vw9vsGAd0O/b0ks3jNR5pzgMKemSIins1W8dwzam3dxqhOn7IE52URUsknT0NX/EF5n6syYvGzVbWzC7idCHLbLmbLMl3Kso/HAj9H3pJtSPEM3xgFSuu9Dv7rO69WIWbcFKlC2qRATizIqbgkIsz9ITojt2Nq1gieCPiUYmpZC39mXU9cJvFOqluxcE8i6GahoirsenxUO81IEp9oFr3P6IsEkPYF78FBnydGmeTfxSwu2FCE4lCTzRrrdXKwR1xY0DwIR50Cx69qIehTVGpZ5gl6t+rSeVn51uEHjgxbY1++ACbWZhBJg8JPOSRnyFmGEqNVHgZVjQbxPTpykIdF7lmvSsr4+AZGM52W3sTB+F5hjO/p4kl40s0mh9/iSYrUX5Wet1UIgfgm41a1fLsLo9xhafIwlj1f5ByQG/5EW17xEKX8fHrnIjSLqbIBpjnmsF6tMvHvj5YB+ntASefh89VC/s0qoDclmYrfQbI4Xrji1ycQYkYwfPuL56nkLmGwZmv7qFgFto5iqVpfNh4rTpDZKL4NabF8ppXXLvs2gNDfxVaDR6kQs7hMaIrETbWV4neOZrz6gdmq2UIdbflT0P87ousYPKjpwOnTBam/xfeKDcKuEtr2M/onL6ervArFmx+OdPQYBYjIy44UGXCauzr0Mc/L8fJDpDm7kJo+ZcxAG+ksBkzG418QMIvDUrQ1nmp1W7Xsci/srFtMPK8cbkpikGtTcxSg/vksCxzd43WxdAZq7QamJnu5Gby7gMoYcssfaBmJ7u2+TTm4+B56yJMH9c0t+WFWqiftXnV10x1z+0gG+4fcQt4EPmhJN85k+6BEyGk X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:MN2PR10MB4112.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /E4BmOzNOxcT6/GokfDL13Stu2fuDI5bKqdH3/JRYNzrH2yTYB5hArNFxtzEXBANNHuBkpD9KhazkcIUKeg8WpLV0OVq+dwcP3LHuJhzwRUe0Zp9eo6sScxkos3xafbUvBTE6vO3QWuAv3IlmTuz8ipOf3TCfx2kQY3OktKVhsXkPTkAU8KRDFelKsgs40V6H9GsXkPk1V4RFfM9RsM8skll9tFaqXo1ClfqsXN5FA8wN95ktGJ2wH5/vcJ48iF5L6MLgQECU2AOoQRfe2umld5bJ9I5rnvyPIOf9TP4Ja/JCaiS9Dhyy63163JfXfI3Neo26xTE0CCve1YHwMnGYpQ+6ASgXfuvP7lRZesuqBOwWQow4rOxMT5qGmoBXRNomagZYcEKGRn0Tc2Hx51rzv9cwWpXSnfnyHlwlw86UdW2BBnwOdurFP4jL4T2/VXIzk5LTmzUxIueiY0Q07H6dwuUDV1UHwcR/40oVmV01gLmqC0iYWGv1cyHP3xDZ1ZYweummRmw5DsCCHH5y3cAX2NNNrLfqolD4xmrEQTijvEKjyH1suLWyKOXu64nOdnwDEWlIEqtOVl+hJMHHtv7Nh/zZ/abbCME4hqmG2N8QvC4Wl1jnp+CT4k00scH+qNG5d2xUQVA6Cq5o+dXew/KexA2G+62JGINjm/d5IkFtBreXFrifGTWkKY4L+sJl+jZEFJWSt4/GtbKMWd+skllAZZP+p3BPrzQoeUfycvhbcNfbR9R6JhlCrVEi35DcuBZ1ycBb+tADgoq/nZfLyYeyuRksJxJ9pS/xR5vXIF3aB9OP6vtHcH09cKBKclpl2GGeVn75dKQDPnmPL8n2mEWhAxt4HwejlFPHEYmmzuAkSdsKRmwjaMo92gxtjmX0Ign0eUmj6CTH4bohS4KdvwRm7D2KrOW5KOKgIL+tW+NeQshTYUSuUdcOHKH7ATnM6u8wngRZXwPGnQZz17Hzk9uraVBj7p21d93Boo6Y8hvLXeDLZQPPSM+9RajpD56nTcYdo+TKVMAle4tL0bMdzn1hl8bE/BZd8i62i6X/WozQaXbMVcbYZXf7ph5+BSsdXg2MntS7C61/oXKxpkjirjvQwEaQsSHtXgOcSYFuxlU7QYhE6GAIqaODXuk6gz3QtyEgWTJX1chQxrkAXz7BUd/XaY99rlcTUDZNOCqADg2XBpsTsdxiDWT3GkfTwCqMJ4H85gBloOx6LI1mRyCRui4diNIyljAe1/ch/9C6/ZwKHP7ZWoE78dbbTNGAFCV9uaucPCyP6pDW6Rzi1XWxRpPjrlWxvALZYRz5bX7RWiLeihV0Cr3i/+WfmJlIHKUYfU0hMvz18qUgLMeFiDqR/nSUS/CWMwDD3fOy+PiD+OdRAcgjEJACG1X8HE6q+WwO9w2aPKcXqswRYyDicAT2DNYK7p2YMonjwcoZAmsuY++gXTll8zMov96Pp7P8hwDtr+gfIeIlCzaxcjnbKcoHBUYqJL54U3EBD9xLG0LgUjydZdwLn1jK39KKyU0p2GcmUXwCdfM9IlvJEYY08EuSAp/JEgFsimgDjrL449PA1OpEJvCpetYG5HEL6qCxHjpy0GzZUfZq8BwyffUGi4C79sOxA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 63qCSfllHxN9mfT2rgnNqccA2QuCeqxw6I6OdZ4C0x15f0O+ab5ewsuEcx87g1OIP2DRxvNKLYTt8Qxx23zJ6KYr3VjbICr1VeoAfCTti6ommFQzdIgRWorqJX8H/++VdvI6Zrieb1lxeMpt1CQQFhvZYi5/cveZ0TMPI07g8x//ab2x1DqP3yvnofmihpPaBmAQR/WZ49SUs9FV5knZaY1jemPfNQBHbTxGkctpTRt2wMK47ds1DH+x+5AribllFn0NXbGedw7hCVzEEvueQiytrL6fSten/gzuQAAWSx+NQJ2+nwBmOfIAj2IUs55UWZqOr1Sf3jLRM/uxseUe38n96Rg1HVMjz47/xouK/ItjKDpAL5+bUH11XplkHT0T+2hSt6e4l74OCrB+SzOV4GV3AY4W9ySF7cwOyQXFmKZ0R3d4khlA31bj/SaJy7RdjJMJMWamyfXIrjX9833az8Lcq8qkYi+YJDjgnm8+707lym4+jsd78toOTiQx/5q6GUUHH7ABl/kGVc6j6XeQaJpYuqgKWkZEuwFno6et3+ajBvyox4uRrrItgL2wQ6D7Dat5Ad4irbDp8I52UNQ/Cs8R+Gc8U9hokSiGZ4EWgI8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0593843c-9e42-4989-cc7f-08dd6015579e X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4112.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Mar 2025 20:51:33.9355 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 60/YD7pzpUdPk8ECfdJ7pueLkojXU2uiFC64QVfPUOeWQGQbxB1s98o5v6bKADI+LoPm+BhUHaMK4KtThb5mGAI803ATRoEpoXyX1GULByI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB7250 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1093,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2025-03-10_08,2025-03-07_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 phishscore=0 adultscore=0 mlxscore=0 mlxlogscore=999 malwarescore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2502100000 definitions=main-2503100159 X-Proofpoint-GUID: hzCifiA9qfLwkauq-OUaxv_zvqcjJCAw X-Proofpoint-ORIG-GUID: hzCifiA9qfLwkauq-OUaxv_zvqcjJCAw X-Rspamd-Queue-Id: E13E714000B X-Rspamd-Server: rspam11 X-Stat-Signature: g5bnjqkbwywknsf5gz6hi4w8ad45ue9h X-Rspam-User: X-HE-Tag: 1741639900-372627 X-HE-Meta: U2FsdGVkX1+q8XzjX3BN4Qf3/Y0HLBws5QI7+VVzMXfdAaRGtxEgI16vmahIpsu4wHN4Lz96dFTb0Eqp6kHQ4RNwTdtB28tu0pvJK30agxwl5Hl0YqAaJSh0t+EULbSwj8jnvbfT4cHAQbsJyyWWo8q8PG+iYYVCTlHd+7rixrQECxbB2KBp0PzOPqwyP1NJjd2xf7Aupw5Naoz7ZGboNNm3uO3Ho2fvOrxJoTlbyytsaLUAj4ZufykM0IDlgx/HA9cXhZgfaVsyJBczRevrNm8Hn2VYz1HzHhgtMoHTFCcXBPFJ/4g8FpjrQ9IExxX9T7QJ8/6I+NklvEQ7sPlqHiqzK8yPOwS6g4l0l3a4BlmU6zqqrdd109L73V7aIFISefV55270Qw07WHXGmdG/Rvdp6kUnTI13BFPIWI7+I4anyDg603Xm68O2N4dbz4kwoq3HtWD5GhHnAIHv3+X9JaNIyQ9gJmS6jF1lm6IxnxB1AQT/6ot6sBTQ/POpRnUitomQdy40NOZtWRM0Ve7Q1nYtQS8wNOr7wTY5Np7+vmekJme4gaKdlKm4Rh4l1CAyC9P9R4PzM6HndM/MPNHFuYG0DKogcnTmAsVERyyQ9egIgpsAtYgIOmTOlBjMIjfId7+MshXJScnUlnYEjqJiNOWq/PHNmacBMAn7s5JF5rM2jsFXiefVgX0bm3DTaXZv0jw941Bgb1/ouRQQCgmGScJ+G7vCOUOpxOLnVM+XS0LNyVPpHUT70yfH8r2DXTM0mU2BMqaMtR4xpnyZcPCvnbJzHn16bWQrtVFnCHxUchRjO/9+QbvjW7pAvGx5PhEYT8ZDSG0bt1nmEtJmMTJP3x1JKDyrpg5MWBxbwmRx748ln5w/cmNHqZjXZSovizxncoGs4ASQpKtZp4tiAejdZ7NVB5StWV/wpLPXszzWjlvgdTKFEV4/hnCF76a/hIA7OfWu67qefuFPlxYIG/y DA3h4oGj HYcMO15oFUMRR7chIg2UB8s0vmJP5nX7GJvpmnnfhVwehmD+yCuN+FDApIn17OJL+AXylR1Xe2T5sGAHJswqFDacHzMFRct0r+1SKCy77Q5TmmwTo4C8vLAKyMXuZpZ4Yi1bQOeMbpwh6HbEp85PVjRLS1xMOLB7SFLS72OG7kDzJ0Jf/kxjagp16fqqmK+/x0uedAH8lZO+pUKipvOgOaDXOsAOKdrRYeOVtLJM4bqkINk0JOgj+JTwpRcbr+kNxOlJEvD/hHNjCB1D//roVDnh5227kEVIMy4TOYMGDxa+zLpJ9YtI2TXddsgaXw7sOHK5dbXn7L5yp3a406W/DdwykmY6hldZHJwjLBuenP8Wkj8U3JTBmQ1AtGF1GbpehH2PXFIms+nJ4WFJtu6vagf4AxpPGnWwoxgSgn9X3cmvYstJv2UxZMI7qq2q1SpoWY3fH2cqc56+Gta7nXl1QuxDAFp+H6vGdAMViZf1cUa+fHrHNDtS3b/62OuMzsd7L8EdqnFmt32v/EZ6THGaDk9MRfBRlyUuCNP0I92Winxlg3qWZvzJaE+/FrKf6u0VdTAhiHBHkwSgR8XCJ3VdhfnXnxZlC5ZxidBUEMS0xTqabuw29xbcPPXYR292szsEqiSIwmSWWYsBEtp7jXZAkRA+SNA== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We invoke ksm_madvise() with an intentionally dummy flags field, so no need to pass around. Additionally, the code tries to be 'clever' with account_start, account_end, using these to both check that vma->vm_start != 0 and that we ought to account the newly split portion of VMA post-move, either before or after it. We need to do this because we intentionally removed VM_ACCOUNT on the VMA prior to unmapping, so we don't erroneously unaccount memory (we have already calculated the correct amount to account and accounted it, any subsequent subtraction will be incorrect). This patch significantly expands the comment (from 2002!) about 'concealing' the flag to make it abundantly clear what's going on, as well as adding and expanding a number of other comments also. We can remove account_start, account_end by instead tracking when we account (i.e. vma->vm_flags has the VM_ACCOUNT flag set, and this is not an MREMAP_DONTUNMAP operation), and figuring out when to reinstate the VM_ACCOUNT flag on prior/subsequent VMAs separately. We additionally break the function into logical pieces and attack the very confusing error handling logic (where, for instance, new_addr is set to err). After this change the code is considerably more readable and easy to manipulate. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/mremap.c | 293 ++++++++++++++++++++++++++++++++++++---------------- 1 file changed, 204 insertions(+), 89 deletions(-) diff --git a/mm/mremap.c b/mm/mremap.c index 6305cb9a86f6..7dc058d5d5e2 100644 --- a/mm/mremap.c +++ b/mm/mremap.c @@ -884,13 +884,13 @@ static void vrm_stat_account(struct vma_remap_struct *vrm, * Perform checks before attempting to write a VMA prior to it being * moved. */ -static unsigned long prep_move_vma(struct vma_remap_struct *vrm, - unsigned long *vm_flags_ptr) +static unsigned long prep_move_vma(struct vma_remap_struct *vrm) { unsigned long err = 0; struct vm_area_struct *vma = vrm->vma; unsigned long old_addr = vrm->addr; unsigned long old_len = vrm->old_len; + unsigned long dummy = vma->vm_flags; /* * We'd prefer to avoid failure later on in do_munmap: @@ -916,56 +916,151 @@ static unsigned long prep_move_vma(struct vma_remap_struct *vrm, * so KSM can come around to merge on vma and new_vma afterwards. */ err = ksm_madvise(vma, old_addr, old_addr + old_len, - MADV_UNMERGEABLE, vm_flags_ptr); + MADV_UNMERGEABLE, &dummy); if (err) return err; return 0; } -static unsigned long move_vma(struct vma_remap_struct *vrm) +/* + * Unmap source VMA for VMA move, turning it from a copy to a move, being + * careful to ensure we do not underflow memory account while doing so if an + * accountable move. + * + * This is best effort, if we fail to unmap then we simply try to correct + * accounting and exit. + */ +static void unmap_source_vma(struct vma_remap_struct *vrm) { struct mm_struct *mm = current->mm; + unsigned long addr = vrm->addr; + unsigned long len = vrm->old_len; struct vm_area_struct *vma = vrm->vma; - struct vm_area_struct *new_vma; - unsigned long vm_flags = vma->vm_flags; - unsigned long old_addr = vrm->addr, new_addr = vrm->new_addr; - unsigned long old_len = vrm->old_len, new_len = vrm->new_len; - unsigned long new_pgoff; - unsigned long moved_len; - unsigned long account_start = false; - unsigned long account_end = false; - unsigned long hiwater_vm; + VMA_ITERATOR(vmi, mm, addr); int err; - bool need_rmap_locks; - struct vma_iterator vmi; + unsigned long vm_start; + unsigned long vm_end; + /* + * It might seem odd that we check for MREMAP_DONTUNMAP here, given this + * function implies that we unmap the original VMA, which seems + * contradictory. + * + * However, this occurs when this operation was attempted and an error + * arose, in which case we _do_ wish to unmap the _new_ VMA, which means + * we actually _do_ want it be unaccounted. + */ + bool accountable_move = (vma->vm_flags & VM_ACCOUNT) && + !(vrm->flags & MREMAP_DONTUNMAP); - err = prep_move_vma(vrm, &vm_flags); - if (err) - return err; + /* + * So we perform a trick here to prevent incorrect accounting. Any merge + * or new VMA allocation performed in copy_vma() does not adjust + * accounting, it is expected that callers handle this. + * + * And indeed we already have, accounting appropriately in the case of + * both in vrm_charge(). + * + * However, when we unmap the existing VMA (to effect the move), this + * code will, if the VMA has VM_ACCOUNT set, attempt to unaccount + * removed pages. + * + * To avoid this we temporarily clear this flag, reinstating on any + * portions of the original VMA that remain. + */ + if (accountable_move) { + vm_flags_clear(vma, VM_ACCOUNT); + /* We are about to split vma, so store the start/end. */ + vm_start = vma->vm_start; + vm_end = vma->vm_end; + } - /* If accounted, charge the number of bytes the operation will use. */ - if (!vrm_charge(vrm)) - return -ENOMEM; + err = do_vmi_munmap(&vmi, mm, addr, len, vrm->uf_unmap, /* unlock= */false); + vrm->vma = NULL; /* Invalidated. */ + if (err) { + /* OOM: unable to split vma, just get accounts right */ + vm_acct_memory(len >> PAGE_SHIFT); + return; + } - vma_start_write(vma); - new_pgoff = vma->vm_pgoff + ((old_addr - vma->vm_start) >> PAGE_SHIFT); - new_vma = copy_vma(&vrm->vma, new_addr, new_len, new_pgoff, + /* + * If we mremap() from a VMA like this: + * + * addr end + * | | + * v v + * |-------------| + * | | + * |-------------| + * + * Having cleared VM_ACCOUNT from the whole VMA, after we unmap above + * we'll end up with: + * + * addr end + * | | + * v v + * |---| |---| + * | A | | B | + * |---| |---| + * + * The VMI is still pointing at addr, so vma_prev() will give us A, and + * a subsequent or lone vma_next() will give as B. + * + * do_vmi_munmap() will have restored the VMI back to addr. + */ + if (accountable_move) { + unsigned long end = addr + len; + + if (vm_start < addr) { + struct vm_area_struct *prev = vma_prev(&vmi); + + vm_flags_set(prev, VM_ACCOUNT); /* Acquires VMA lock. */ + } + + if (vm_end > end) { + struct vm_area_struct *next = vma_next(&vmi); + + vm_flags_set(next, VM_ACCOUNT); /* Acquires VMA lock. */ + } + } +} + +/* + * Copy vrm->vma over to vrm->new_addr possibly adjusting size as part of the + * process. Additionally handle an error occurring on moving of page tables, + * where we reset vrm state to cause unmapping of the new VMA. + * + * Outputs the newly installed VMA to new_vma_ptr. Returns 0 on success or an + * error code. + */ +static int copy_vma_and_data(struct vma_remap_struct *vrm, + struct vm_area_struct **new_vma_ptr) +{ + unsigned long internal_offset = vrm->addr - vrm->vma->vm_start; + unsigned long internal_pgoff = internal_offset >> PAGE_SHIFT; + unsigned long new_pgoff = vrm->vma->vm_pgoff + internal_pgoff; + unsigned long moved_len; + bool need_rmap_locks; + struct vm_area_struct *vma; + struct vm_area_struct *new_vma; + int err = 0; + + new_vma = copy_vma(&vrm->vma, vrm->new_addr, vrm->new_len, new_pgoff, &need_rmap_locks); - /* This may have been updated. */ - vma = vrm->vma; if (!new_vma) { vrm_uncharge(vrm); + *new_vma_ptr = NULL; return -ENOMEM; } + vma = vrm->vma; - moved_len = move_page_tables(vma, old_addr, new_vma, new_addr, old_len, - need_rmap_locks, false); - if (moved_len < old_len) { + moved_len = move_page_tables(vma, vrm->addr, new_vma, + vrm->new_addr, vrm->old_len, + need_rmap_locks, /* for_stack= */false); + if (moved_len < vrm->old_len) err = -ENOMEM; - } else if (vma->vm_ops && vma->vm_ops->mremap) { + else if (vma->vm_ops && vma->vm_ops->mremap) err = vma->vm_ops->mremap(new_vma); - } if (unlikely(err)) { /* @@ -973,28 +1068,84 @@ static unsigned long move_vma(struct vma_remap_struct *vrm) * which will succeed since page tables still there, * and then proceed to unmap new area instead of old. */ - move_page_tables(new_vma, new_addr, vma, old_addr, moved_len, - true, false); - vma = new_vma; - old_len = new_len; - old_addr = new_addr; - new_addr = err; + move_page_tables(new_vma, vrm->new_addr, vma, vrm->addr, + moved_len, /* need_rmap_locks = */true, + /* for_stack= */false); + vrm->vma = new_vma; + vrm->old_len = vrm->new_len; + vrm->addr = vrm->new_addr; } else { mremap_userfaultfd_prep(new_vma, vrm->uf); } - if (is_vm_hugetlb_page(vma)) { + if (is_vm_hugetlb_page(vma)) clear_vma_resv_huge_pages(vma); - } - /* Conceal VM_ACCOUNT so old reservation is not undone */ - if (vm_flags & VM_ACCOUNT && !(vrm->flags & MREMAP_DONTUNMAP)) { - vm_flags_clear(vma, VM_ACCOUNT); - if (vma->vm_start < old_addr) - account_start = true; - if (vma->vm_end > old_addr + old_len) - account_end = true; - } + /* Tell pfnmap has moved from this vma */ + if (unlikely(vma->vm_flags & VM_PFNMAP)) + untrack_pfn_clear(vma); + + *new_vma_ptr = new_vma; + return err; +} + +/* + * Perform final tasks for MADV_DONTUNMAP operation, clearing mlock() and + * account flags on remaining VMA by convention (it cannot be mlock()'d any + * longer, as pages in range are no longer mapped), and removing anon_vma_chain + * links from it (if the entire VMA was copied over). + */ +static void dontunmap_complete(struct vma_remap_struct *vrm, + struct vm_area_struct *new_vma) +{ + unsigned long start = vrm->addr; + unsigned long end = vrm->addr + vrm->old_len; + unsigned long old_start = vrm->vma->vm_start; + unsigned long old_end = vrm->vma->vm_end; + + /* + * We always clear VM_LOCKED[ONFAULT] | VM_ACCOUNT on the old + * vma. + */ + vm_flags_clear(vrm->vma, VM_LOCKED_MASK | VM_ACCOUNT); + + /* + * anon_vma links of the old vma is no longer needed after its page + * table has been moved. + */ + if (new_vma != vrm->vma && start == old_start && end == old_end) + unlink_anon_vmas(vrm->vma); + + /* Because we won't unmap we don't need to touch locked_vm. */ +} + +static unsigned long move_vma(struct vma_remap_struct *vrm) +{ + struct mm_struct *mm = current->mm; + struct vm_area_struct *new_vma; + unsigned long hiwater_vm; + int err; + + err = prep_move_vma(vrm); + if (err) + return err; + + /* If accounted, charge the number of bytes the operation will use. */ + if (!vrm_charge(vrm)) + return -ENOMEM; + + /* We don't want racing faults. */ + vma_start_write(vrm->vma); + + /* Perform copy step. */ + err = copy_vma_and_data(vrm, &new_vma); + /* + * If we established the copied-to VMA, we attempt to recover from the + * error by setting the destination VMA to the source VMA and unmapping + * it below. + */ + if (err && !new_vma) + return err; /* * If we failed to move page tables we still do total_vm increment @@ -1007,51 +1158,15 @@ static unsigned long move_vma(struct vma_remap_struct *vrm) */ hiwater_vm = mm->hiwater_vm; - /* Tell pfnmap has moved from this vma */ - if (unlikely(vma->vm_flags & VM_PFNMAP)) - untrack_pfn_clear(vma); - - if (unlikely(!err && (vrm->flags & MREMAP_DONTUNMAP))) { - /* We always clear VM_LOCKED[ONFAULT] on the old vma */ - vm_flags_clear(vma, VM_LOCKED_MASK); - - /* - * anon_vma links of the old vma is no longer needed after its page - * table has been moved. - */ - if (new_vma != vma && vma->vm_start == old_addr && - vma->vm_end == (old_addr + old_len)) - unlink_anon_vmas(vma); - - /* Because we won't unmap we don't need to touch locked_vm */ - vrm_stat_account(vrm, new_len); - return new_addr; - } - - vrm_stat_account(vrm, new_len); - - vma_iter_init(&vmi, mm, old_addr); - if (do_vmi_munmap(&vmi, mm, old_addr, old_len, vrm->uf_unmap, false) < 0) { - /* OOM: unable to split vma, just get accounts right */ - if (vm_flags & VM_ACCOUNT && !(vrm->flags & MREMAP_DONTUNMAP)) - vm_acct_memory(old_len >> PAGE_SHIFT); - account_start = account_end = false; - } + vrm_stat_account(vrm, vrm->new_len); + if (unlikely(!err && (vrm->flags & MREMAP_DONTUNMAP))) + dontunmap_complete(vrm, new_vma); + else + unmap_source_vma(vrm); mm->hiwater_vm = hiwater_vm; - /* Restore VM_ACCOUNT if one or two pieces of vma left */ - if (account_start) { - vma = vma_prev(&vmi); - vm_flags_set(vma, VM_ACCOUNT); - } - - if (account_end) { - vma = vma_next(&vmi); - vm_flags_set(vma, VM_ACCOUNT); - } - - return new_addr; + return err ? (unsigned long)err : vrm->new_addr; } /*