From patchwork Fri Oct 25 12:26:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13850602 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 41AD8D0C603 for ; Fri, 25 Oct 2024 12:27:04 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id ADD186B008A; Fri, 25 Oct 2024 08:27:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id A8BB96B008C; Fri, 25 Oct 2024 08:27:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 890966B0092; Fri, 25 Oct 2024 08:27:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5CA7C6B008A for ; Fri, 25 Oct 2024 08:27:03 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6CF0DAB6C8 for ; Fri, 25 Oct 2024 12:26:24 +0000 (UTC) X-FDA: 82712048916.29.DA2E951 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf04.hostedemail.com (Postfix) with ESMTP id 9C77940013 for ; Fri, 25 Oct 2024 12:26:34 +0000 (UTC) Authentication-Results: imf04.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=jk2sLPvx; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=sge1IFQ6; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf04.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1729859094; a=rsa-sha256; cv=pass; b=LjC9ujfS4zDyi/5yEf+VbzOhtO8igzt1aDPSxeGHDboWZtf8jLs5G/lPyZ4pUDIGPsAULw fu7iN9/GdOLfe2mUEf4cwzbs3/jKKdljUF3DxCPRyQ8rMu/JGrqYFu1G0ZjwRMoS/WiuHb nb5Xt1zH20D8MCYXQeS+x9F/shNvsrY= ARC-Authentication-Results: i=2; imf04.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=jk2sLPvx; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=sge1IFQ6; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf04.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729859094; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=1RKAonmcILha8U/5gzL1MZCRAiyipTUTBI1zBY8/ua8=; b=gjUGkKjwfnaMUa/vd8Lk2QirDUtPYRbvFXN/1iSpyidT7vsKDWxehiYU5al08fnVSWfCDK oROnq43OGF9S7tzhYoTi9zvNcRrKSv8E2Ln2lqwfxY0KLbRo5eAbDfpRthy8ildnuHwJUA ppJRBMDjI0qRw5jAcTJOp+t3Hrm22Cw= Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49P8Babn014732; Fri, 25 Oct 2024 12:26:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=1RKAonmcILha8U/5gzL1MZCRAiyipTUTBI1zBY8/ua8=; b= jk2sLPvxcBTa+qktcW34pQG1spl3NzAuprtVz+URBnaOeklhuwG6UbYsyv62Z8dt TgbsaXJ0/BlQu6TdAzRf2uYoYG7hJ6ylOdjOoSgvFgvt7pwLJRHgEvj2czzK3EtK 1o6seh2pJjEPQ8VX8nhws1E9jezvOM176pgDUGm6kfxNe9w08mCfGpan4BsNuKk/ 3WbQdHH/tIIwhRRzkxlD+ionC7XG1rUSjKkrhu6HV576ZUFjmFcG7+EZ3N5cU9WW XIo4wPkVkJlrI7QZXs02ENgEdQMuloiaj79YsM3OrrQxRDQcFoJUjJJ32bOVKh0m Ep3QkJ7iULMbTADaAx8sqg== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42c5asmusx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:26:56 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49PBxpoZ016557; Fri, 25 Oct 2024 12:26:55 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2177.outbound.protection.outlook.com [104.47.57.177]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42emhdt4s4-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:26:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cDcSHylas1xZc8YOYyoNeOelaWe00gN6T4J9MPhjnGQuMXcP+YUSOhFQFm6/ALMtgNxeyprYq33za1YouSwLYfBb1K/BnCsBoW4K5x5a9CyPKelBISO0dS5HxAjOYqd0dr+BUXHO7Q8WMV8BJkwWsH+SFN4mLWpa+iIFQcdqpL21PbejvgB0fyvTDnJNkcuXiPTj7uor+m8owccVC0O35YSLZzl8RyLxsjbHGMf/bAmYxVzsANiKTGYC/mJsuDC1MQ/LwfBu2N9XgpnpmhcKfvI3HJCxTArDn2bwxp8ZfNBsbv0AvaJR2WV7f4PuSmqitqTHyuPLnpYKhWADdn8CUA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=1RKAonmcILha8U/5gzL1MZCRAiyipTUTBI1zBY8/ua8=; b=cvkKnk1y1ZOvX3pNWS4OKkA0zn91DqnvvrCaZ0aj/I8HbgjGzowiWLQIgLVD+g/7i19M6ArSbdH4wZ8MhtyVXyhzhUQQb3amNw+dRjBJLGH7dO2826R0wYnFyqaT8x+SwvkEi5XdDsA10zO9egyEuHMLEcslQ1bYzcLSS4t328XO8yIOBqIJSAPk5oSfqCLWwwJ0gjUzylbz6EiMRESBFn1UjdhTeiC/zne6K9P0oRPTqCd2ZATIGtTttdT+9GEU2FDte4grgWrCaYfnIMGRyYy6h04Q1DDORJ1G54TWtO3KeBgSjFqip7bd9/UackedXhvMgsyBisCgJQUOlBRPMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=1RKAonmcILha8U/5gzL1MZCRAiyipTUTBI1zBY8/ua8=; b=sge1IFQ6G1xYqSM5KxNLgBwuT0CQLmlVNVsmWWEAEecCU2fwei9zJaAdy7t4vsirwadq2QLGO1mKwNpALkL9+uZVjnHIdMMggsRs62pgtn8N/GkV0Eo0PgQrqPPH5zDczKNt8sex7kzYxqkuhzdhIAy5Cb7soW8nmFoKtAN4IH4= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by CO1PR10MB4484.namprd10.prod.outlook.com (2603:10b6:303:90::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.23; Fri, 25 Oct 2024 12:26:51 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8093.014; Fri, 25 Oct 2024 12:26:51 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH v3 1/5] tools: testing: add additional vma_internal.h stubs Date: Fri, 25 Oct 2024 13:26:23 +0100 Message-ID: <74b27e159e261d2ac1fe66a130edad1d61fdc176.1729858176.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-ClientProxiedBy: LO4P123CA0376.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18e::21) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|CO1PR10MB4484:EE_ X-MS-Office365-Filtering-Correlation-Id: 2febee1c-88b2-4f95-e90f-08dcf4f04d91 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|10070799003|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: V0orUsKOU5W0QrCZy3vutQ1kkVa/XsYApv1AyHvkFwYRK4tHUEYhrW+Oj29xSV364NKfKdCiCHN1r2oPJU4iQ5a0FNQ/BU1OzviyCEpAgH5mwWcVTepju8Oq3gUoaMNnO2BgqFkbnWodBvMyflBi9CdMAVtROO3k14s6B5vQUrzKM44n9Sv0kpsiWvzhtS/O+vtAV9yBR1SCW+yL7gOkuB0ebkIFdMv8U2wccFzPZCVvEkyw0hUqNAREq+5w4FK6gbYUo32vaskwG1iYSwWH+V7cV4n4IKwewzYXoKcNqtR1OZ/MrgKq68Ooxf5ht30cMMFSi3FhHPjUC23cTJGHMUuB44RmL6XaxLvUpjT2Zl8W8Tub6deHQiv3LeoI/ldR4vwG5FNeP47hIQbwNhyI6KO/XMi8/gnFV/vUFca1YclerZzKA2Gf0Cst908hvNsEqSTdegg8bNVoGvf0rLCaTJRIukDCRskMwC6uiDWmQUkuKcFnOM/w9HZH6ZC3ZQxoNjs+6EVXgdMd4TedZxrYFwE7ZAgmcOHGFVpt/TUgFOI+H/w03Sb3fJ26l4IdWOXUc9Q3xhgPROcJf6aNtmA96quRtU6D7mc9XdbUkLVl+b9ZSXY6C23TXrW7gOeDaGVEMpqPLlk5ZavSmm3CHNh4HPw9kVxKo0uGeNrhIFf0S4vUZRSQXqQpjfigbZ83Q/a5uxCjYdCgJwRR36GbS1VfE7zd76/v3LBgHabv0bCmTNm6qwAfdRhJkNuQT5VnCiML2G/cU0SZzPGBc5/x3N5NFwLV+t96AA88FWsM3OD5z9T0jAmhboOP4m5Yr5KOfO2taAHlkNvHQJNRI5ABkKRyZNMR56gFkkQCUdoWLwsjBAhIvXfp90zp/WPP5lIHZ7YJ2RbScV8ll5Q+XYhh25en9Rm2zfyirbtWi69VYofEzymO5HmrySS0lqiIbsXyE2Gcagw8r6ajpqkWQ8nD56836eaFoVpt6qY3Nfs9yq/cVEOqW4NftNUO+6MpQezNWbBwaT3YJq6ZJGZGNzvpxMhRO2WjbF4sl+YiBVagtPZtHmtQHsqfObjsi04VNHhbIaugqML6zSrDINJN7385G43Se5GjE0EHFVTWRKPoziQQhrbbedp1mRHCOZf8xT4I12ggH1SLkmTfPTuMjlhO2j/t+bIxm36qUosTf6OOEFopFlvV2TUQ4JEgb6u6sOhCT98jdgwwhbrrRPI41ecWTEojx09En8MphvE/iI+elaawid8LuIBFVB/dYyGquEmQv9bszKrW+O/su2pEOr2jAo8ss1oyfjrAf5vO5it+wcn7u3g= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(10070799003)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: x8p45yt9FKPcpbMPnRRprJrxXPIRdPsym8NQe00dySAY79JQgQ50XZRf3f5NIZ3NakYJDldIpcz35egynBNPsO8xEmgZ+0tsFN++lLrDiZ2Hc5qYPigCChSfAs231gnS64Zyusomt7DQH7SiMqfDgX8URuOpG0VVNlVN5TT+kG9m38okpiI/BKH5TUnuE076Te7z381TtLewUVVIk6jlQw3EAe0pnm/er7Ckb3PP178zJygf15LD1nwj5HrzkRvQ+SihDGvKoGd3VDWitgxFMasi7lbQV2iqDP25LZwMNwNUh9jlkYykiIeD/s3zokzLtrDkvACOD3BAkPqKO0CFXFJWSpZdro7ejce1z8OXMGHG5jENwNcFWJr2Heu3UWAeqPXs0PgHl0Y1d95E5qYuoDOpUwhKm3JPVPgmdKxMbt5YiMuiNAFBuJmCj+A0vxoIGztGrFIWNgqvlChpI0+z8GqDUMjL1dSWmIqYiLQuN6/f/qn75Td3d2T/VrrMFcH6VSpNJ5qJ0i1oqXfK9fuujLrfsm/CfQxymWjMoPm8AphRmJr9bOV3k/Ki0UF/pUbg1Hs4OYbY17THXqdE2xUwKin5fnXc6OTqF5mmaLsyCtihZsrtCcNs9KMmCBxFgDHRF3AHbMhuh0ZDkbxxTJT/b3oz0UPvsMeV+ToVZx9T5T0wXl521EScOD7kxt4j1yMy2ZEo1+68CIebDYK200fjNsaLNg+PQyKdb2OEC8SovjeKVEFWa0ZO4sHsuYckVUGjyqrNE2UjDaNmbBcAkkiCQJU6ZpnmqBy+dwe1dLV3CnSJTLgJ296SCwYrA2hmBIpv6Ayrep/pMCpoWEwIrxfXfzKzVYcH01wgkU6Ey9k8bA9fBwDDN5Sveuh9qu3O/1hdNgLDHKmRYBo7US0Vhdby9kUWp+NIwBTgjef4QFjMaJNN62aoUvYbm+jjq/2TLyhW3roephfGjiJis+XA4yd5DUfHlnYhZn7nEMzojupLks1vpdKOZ7uqia2f8Goz0dmHVsK4E/iONlcAta42LTC48hRAY+S132ufdhjy4j0f0Wx62kBQ+jwxdXIOg+uVuSZS7U+Oeirn9CO/+bUXWIaKXaxrwEUXeHbqxEDu/CxQkdXMOTJE9eNnGx3rXKEfK9K0749JHbtDJ0Ad85rGK1MraRqxINemu9TDbRE0wmA97zE9qihgQsDkxjw0jFzS7YzQSOf+dRYpJ7BjfUQFXT4SmFWwlbFg5xDLWyLpzBf0Dae1s6ulGYAMd4w4QV+TgaF3L8KHbyDgDnu4775eHzUg7INotBZ70TEV/3GALojbbL3Bw4B3IQoMpwU0enxMGN3h/IZCXodWo3iA5iLERLt79vjhKBlmUpiTKxgd2MzaBqBgQ3BKzHHJvvffA2u2C+4OxjEJOn6UZqi/aJM4pXCX1gqLY7eAQ1fDyaOzalr8Hgn/d3fmC82rpzK7qt8lFYTYhEIZ/wjHceiSJzXItxqBidouOERMmfHDmpaAxFEWK3vIja/pxv4LhdbAMoPscjzWkZ3UzpNmqBbiEBc2SthSsGeHFxCZBFG6OnbT6Cmo7rxHN/r+/kri2bSjRTZHGouiA3upRpKICIkNj4EQTD6XKFufmTlhZ4ZtSsgFK0BmKxwvLHI649S0TnU2ixq0wFcDOld8L/b0zIJh3O65ydZHwQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: Qineu5xaRWetUH6gU2Yn86RLod33WfYM17wl9VC5yOpaVxrVrT47vA7irwgQAYygVuia4XK+X6JvUfXaNEOjAhZxYPQkcUTeaxfwejgXfqFFMHaj4KSHPvGjRJDXsfXRNTPoDaVevbDjq1x9vkuX3EH3MV3aQq3BYuTSYq1AXPzvcrurRgLc4Kn+Tqs4IS4r9okiS3JE738LS/JHpMg9sYWNzosZGRbZRRXOkRhV931kCR/JAWPuHRVyWLura2Ikc2BXrBv90R3SNUguv3Fv4XuvD1T0JaM/2u1+uvMIYKsjF8ntpWTmlhLif7idWzr1N54nGsvtOVSo3Z8HdZVYlCzxP1CXGnkK4kMK6Iy8PikwUQp7vz7PRxebNFoymSdC9O7etQcrIiBbDkwmHZwGtpq/QFED3LtHiiCYI2whb6ur+oLuBtysnJSSXyVwtOnbKRcb+01fjBkL85WT6YuRVHw858EQSeQ4q+ZF2a6ufz7DBdRlr66jlP1iV8G3CHkSpzghPfClw44SyQqEZnsaRA/zQblccWPTLbSwmbSt2rYRekL5XfDnMSQk5hyuvy3N43IA/rhsA7CQjH3vRJOGNcsy7nqWLMBac21x11bqPRw= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2febee1c-88b2-4f95-e90f-08dcf4f04d91 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2024 12:26:51.2913 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: t1NX/qyhrqjVy5I/kPLm03VvmjUAE7xCgM6HIN9qnGwGLus1YZawizZH0yhZH2KpfUhBOG+yaGvtD3qQbfOM7R+Ft/5fE+jlhOSIfJhE+z0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4484 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-25_12,2024-10-25_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 phishscore=0 suspectscore=0 bulkscore=0 adultscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410250095 X-Proofpoint-GUID: M3vftKXBPRqZs4T8LzkxgK52g6k_AsMD X-Proofpoint-ORIG-GUID: M3vftKXBPRqZs4T8LzkxgK52g6k_AsMD X-Rspamd-Server: rspam12 X-Rspamd-Queue-Id: 9C77940013 X-Stat-Signature: be1crir1qeued8p4si7m1oy6xaej6f98 X-Rspam-User: X-HE-Tag: 1729859194-883885 X-HE-Meta: U2FsdGVkX1+Kb+udAwlmy+uAKsamfa4U5BgaWuhudENLF39NGafFZoGu+R+c4BIU/8V+hXcNbSccSIDTF6DVXZp9QqCtQVESCL7tDhPm7zoQBzAEtcqWmJnYWzUjFqI/AbkV7G3kNzOY7wfI7GVS9xECi+kFQxA8m/TgMHqYrCXA+yAWHBTWwz7zcEMju5GCbzT5+OzDySp1p8gCxt2SUYpCe9GFReJWozfxtoRG0NmLA2ze0ZiNc/l7SyykUCnQdO0H+2AbaauxOyAS9uNLrsm+lNsfrxaFCpAgDABnBwtHZeaL5LuOyTfwY8x7s8aUaePdAVPS0wNq4iL87Vb83z0ZJ29zHXV6vZNXpDl7P6DejUfS1l0dLGugRIhS+Hffzcix6LghEkceQyHEKXljAltTFJr7husVF4bJtzQ4LT5ZerjY+mH8K2E/koPKB+PyaIlAJiO/+UazS7FdA0EcknCIhbOGfzrG5l/Oh8rMd9APQYqpxpxEdg2YFTvkjkHkaYXcIC7ppaKxyrfDh+ghiHcoxzul5JQZ3BvpNZstlw58+0s/bxKO1FvdWUJ1Y90AlkLroP9Z/UvJgzWAjupSo2k4Cfw3DQ++VuBjq5+6kACbnnFldD3w8N3qzHBs8Myi6JgSm2cv8ffvEsXb6WJsL6MOe1MIBnHoW23EewaYBZGdbkQpCOB1TaWdiZzz4s488idQsIDA154JCo5b+XBjiYb/l5+ycAr7quY5KGUWNqD4CC0NhwoJNSB5bVqL1lBRCsUxN3kp/xcs5FPvT4H16Amx4M749IYeiYRh5mT8vCWCO5VLlhwee9GvFv0bKoiisgtPNX/CnbPLp7VcK+8+rVc7+LkuMZhGmma8LytuDjj6S19+rgxL3AfrOQ5slbC57GLjkIHLCUvseCct2ps+L9/2E22lOrmx/dd8MhBmXYnTe+YV/eolvcg64a16OHdoOYFsmZjge2CyIN5cWsc w76CxJAB NWN2XJG7TbHIwZ2e+Wv0gkg98mMi1xAL65GgimZsBZpRzFmVUabgLzl7HlvC0Oopxb5hBkMK5s3ES4R6TZkJ6PpUM47hEQAeIWfE4NraJZcP5Wh8Wb8NEkjsJyiFEsadZhG0XVu4j+RibXhOwSc1Ra/6Yqkph5fg5wxD2Ahv0bVANI9kChgx/8joPtVIl6lpl+0prlWKroY3E38U7knOVXNFMnbEQF/fzl147QCnb7brA0KyUocJa2f0jDdA0q8cIb9UiPaDtH8L1U+4yug2+r5QNsWcX0xw7ypIEXZXZNdLt9N3NXCfhbCfIPsuERDnPjK/i4NysVwHDXLi027ZRePuMRnuA4VVmux6I35u6glBoHkyGXlsXl9HO45M8jtmKfiGtbUq5D9GG6ZAgfiPbDShFunhWGld31GlxkkxoG8fFJJaL90+eETtnw3Z1xvtHBunmuzao55HXf2PJpI5BW18IqfhmIWWKQdYl5DwwO8joMMpSo8S+4iqvrxc/MBVY2Uif5Crf7uQlJu2A2NFxS8k7ykZ3H3GhBmR9Yra3hZKnPPc4wla8OEP1NY6qX+FoeyGo X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Add some additional vma_internal.h stubs in preparation for __mmap_region() being moved to mm/vma.c. Without these the move would result in the tests no longer compiling. Signed-off-by: Lorenzo Stoakes --- tools/testing/vma/vma_internal.h | 115 ++++++++++++++++++++++++++++++- 1 file changed, 114 insertions(+), 1 deletion(-) diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h index c5b9da034511..e76ff579e1fd 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -44,7 +44,9 @@ #define VM_LOCKED 0x00002000 #define VM_IO 0x00004000 #define VM_DONTEXPAND 0x00040000 +#define VM_LOCKONFAULT 0x00080000 #define VM_ACCOUNT 0x00100000 +#define VM_NORESERVE 0x00200000 #define VM_MIXEDMAP 0x10000000 #define VM_STACK VM_GROWSDOWN #define VM_SHADOW_STACK VM_NONE @@ -53,6 +55,14 @@ #define VM_ACCESS_FLAGS (VM_READ | VM_WRITE | VM_EXEC) #define VM_SPECIAL (VM_IO | VM_DONTEXPAND | VM_PFNMAP | VM_MIXEDMAP) +/* This mask represents all the VMA flag bits used by mlock */ +#define VM_LOCKED_MASK (VM_LOCKED | VM_LOCKONFAULT) + +#ifdef CONFIG_64BIT +/* VM is sealed, in vm_flags */ +#define VM_SEALED _BITUL(63) +#endif + #define FIRST_USER_ADDRESS 0UL #define USER_PGTABLES_CEILING 0UL @@ -698,8 +708,9 @@ static inline void tlb_finish_mmu(struct mmu_gather *) { } -static inline void get_file(struct file *) +static inline struct file *get_file(struct file *f) { + return f; } static inline int vma_dup_policy(struct vm_area_struct *, struct vm_area_struct *) @@ -920,4 +931,106 @@ static inline bool signal_pending(void *) return false; } +static inline bool is_file_hugepages(struct file *) +{ + return false; +} + +static inline int security_vm_enough_memory_mm(struct mm_struct *, long) +{ + return true; +} + +static inline bool may_expand_vm(struct mm_struct *, vm_flags_t, unsigned long) +{ + return true; +} + +static inline void vm_flags_init(struct vm_area_struct *vma, + vm_flags_t flags) +{ + vma->__vm_flags = flags; +} + +static inline void vm_flags_set(struct vm_area_struct *vma, + vm_flags_t flags) +{ + vma_start_write(vma); + vma->__vm_flags |= flags; +} + +static inline void vm_flags_clear(struct vm_area_struct *vma, + vm_flags_t flags) +{ + vma_start_write(vma); + vma->__vm_flags &= ~flags; +} + +static inline int call_mmap(struct file *, struct vm_area_struct *) +{ + return 0; +} + +static inline int shmem_zero_setup(struct vm_area_struct *) +{ + return 0; +} + +static inline void vma_set_anonymous(struct vm_area_struct *vma) +{ + vma->vm_ops = NULL; +} + +static inline void ksm_add_vma(struct vm_area_struct *) +{ +} + +static inline void perf_event_mmap(struct vm_area_struct *) +{ +} + +static inline bool vma_is_dax(struct vm_area_struct *) +{ + return false; +} + +static inline struct vm_area_struct *get_gate_vma(struct mm_struct *) +{ + return NULL; +} + +bool vma_wants_writenotify(struct vm_area_struct *vma, pgprot_t vm_page_prot); + +/* Update vma->vm_page_prot to reflect vma->vm_flags. */ +static inline void vma_set_page_prot(struct vm_area_struct *vma) +{ + unsigned long vm_flags = vma->vm_flags; + pgprot_t vm_page_prot; + + /* testing: we inline vm_pgprot_modify() to avoid clash with vma.h. */ + vm_page_prot = pgprot_modify(vma->vm_page_prot, vm_get_page_prot(vm_flags)); + + if (vma_wants_writenotify(vma, vm_page_prot)) { + vm_flags &= ~VM_SHARED; + /* testing: we inline vm_pgprot_modify() to avoid clash with vma.h. */ + vm_page_prot = pgprot_modify(vm_page_prot, vm_get_page_prot(vm_flags)); + } + /* remove_protection_ptes reads vma->vm_page_prot without mmap_lock */ + WRITE_ONCE(vma->vm_page_prot, vm_page_prot); +} + +static inline bool arch_validate_flags(unsigned long) +{ + return true; +} + +static inline void vma_close(struct vm_area_struct *) +{ +} + +static inline int mmap_file(struct file *, struct vm_area_struct *) +{ + return 0; +} + #endif /* __MM_VMA_INTERNAL_H */ From patchwork Fri Oct 25 12:26:24 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13850603 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 03289D0C601 for ; Fri, 25 Oct 2024 12:27:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 892206B0092; Fri, 25 Oct 2024 08:27:10 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 841A86B0093; Fri, 25 Oct 2024 08:27:10 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 66D7E6B0095; Fri, 25 Oct 2024 08:27:10 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 43D106B0092 for ; Fri, 25 Oct 2024 08:27:10 -0400 (EDT) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 7F2F5A098C for ; Fri, 25 Oct 2024 12:26:34 +0000 (UTC) X-FDA: 82712049210.29.0598252 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf16.hostedemail.com (Postfix) with ESMTP id 28A5B180010 for ; Fri, 25 Oct 2024 12:26:46 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=iDdFFshc; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=J15mkOls; spf=pass (imf16.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729859174; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=v2dv89kI7lw9WI2akap4rWRfGxfF/RsmR8cPxtvRT3Q=; b=CyjMM0401WCGc3kSjN+giiK8x/xWrpwd/Xm4BYTSVrGqgUgRMpUk/vhASYEVrl4YAPNSDf qS+zssKcZ34QGosCLaOeQkPaNGK+5tTJcoe/jbBIcp/3w++tnb8TYYnjNcFI5covIFmvbx iVe33bFRut/eoFik4izgRkOyHf8T3Gc= ARC-Authentication-Results: i=2; imf16.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=iDdFFshc; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=J15mkOls; spf=pass (imf16.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1729859174; a=rsa-sha256; cv=pass; b=yaJWBS5IS45LqVrlk61MyYaNPJZyA4gAnOnFqrsRGtt3LaUAcWL8/Prk2LIS5nxUsr5TVq +e4OnhvzxPl22FQWcQbSX1YgvdO7GiMfy+SoNfngZgA5QdhbNBTTjd8cJ0U8anqfTmXtYv QkzI6BlHmlACukuuRJiPtmQFqDlB6w0= Received: from pps.filterd (m0246617.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49P8BYFe014704; Fri, 25 Oct 2024 12:27:03 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=v2dv89kI7lw9WI2akap4rWRfGxfF/RsmR8cPxtvRT3Q=; b= iDdFFshcUMstesfit8GHcTErF7yUhtHFOmNpvYe6rjm2YOf8VSHRgDCCuBBTMwKI B3ZBDqiFsWu1C+M+oq3DBWg/dhjQpY0aQmNelmHtkebPACtVi9QYPzIh4dwjizQs 8bngK17ImM1STZBTZmiavgr9ZYPdNIRZ/yNrTDKWm0RK4YmT31c9DgerkFY3Mu+x yVpapRG5WC/o8ccFtdEaS79Il5TuksOS+q8KnkKfKrsymGN+YbtjkSwJIdVo+6rW AClm2vdyBudEoB/0vQJI80GdR7KNEyw55jUNOgGC8YDsg7zgoAnVHKn516/67993 tssUbJLCEdLxqIsfoXW03w== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42c5asmut7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:27:03 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49PBxpRi016475; Fri, 25 Oct 2024 12:27:02 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2171.outbound.protection.outlook.com [104.47.58.171]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42emhdt4th-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:27:01 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mNkyJstE0exslBTkAXPcTcZ6znOL2Q1GSq+NQNqXvnAMCEETSk2aCYMlGLeQOL0RYQJajdik7fyLVzZff4ufd/w97V+AIZp0K9dBu1uH3i7beRawz4C3T1i+md1lPH84d/ZrcQ3DtsHqKEp9YwQfTTau1LXdBgrXzlFGeGxTIrFb4EdteaKL50fiFa3rxJwzGXUupZN2fMPLX0krsGZi69Cny2vuJJKHtRmrucxCTt3nK8RlRi9V4LGa/1hIUiShXskyYVdAo4yPbUT89fCmLZMkCiBMdSec3m+Co/TJaW7zZyInHQ35gT89gVsYxi2xBjHmCbEyP72rbLT0vxePxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=v2dv89kI7lw9WI2akap4rWRfGxfF/RsmR8cPxtvRT3Q=; b=aal6jA9c8GqEaMfmpC0/vYyDclEsW/3p/LCwxs85/4LToaCq7Gdd7VR47PzIT9keZeEV7T+PfTYvKpZ7rjFpehAwhv8Ih2iooYX1ZEYrHL/Bxlqkf2M53lunr3Ff2/9t9h3GLv1WkSUgyVJmVDlXRiR8Isy1RaqOykzstSph5vyP5PGUH8xkejeLAb+DBQgaG3UTpIMXiAufiWZQm51BInSROokThF9811ypoPJTCz/zI5+gRVEosBF+LE0UavmF2GMByOSxwNXk4oGsJjJPJ9uqFyUJVxISgF0sETyf7BFTfrHDhP+0E34Gc7HOcJ0cS1x2lnvDpuzxd7KlDU+DVA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=v2dv89kI7lw9WI2akap4rWRfGxfF/RsmR8cPxtvRT3Q=; b=J15mkOlsjimKSakQyXM6u2Dsu+icU/hy2duH+ZbDzSGltN2L2LqoGz0yHKJJPv2NC1TnNm3nJDCE1yfVGca7rmPYqsOaCbLht0/h1BzqczhGVBYJffH2ZafCFjpXx6bE+8aeP9CtxhWftZMEMBzomGi+5ekthboUYWzxSEUXMMI= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by CO1PR10MB4484.namprd10.prod.outlook.com (2603:10b6:303:90::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.23; Fri, 25 Oct 2024 12:26:54 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8093.014; Fri, 25 Oct 2024 12:26:54 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH v3 2/5] mm: isolate mmap internal logic to mm/vma.c Date: Fri, 25 Oct 2024 13:26:24 +0100 Message-ID: <93fc2c3aa37dd30590b7e4ee067dfd832007bf7e.1729858176.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-ClientProxiedBy: LO4P123CA0448.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1a9::21) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|CO1PR10MB4484:EE_ X-MS-Office365-Filtering-Correlation-Id: ac415e02-f052-4556-7aed-08dcf4f04f69 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|10070799003|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: i3aMan+sFFNzvmXV/nHe8+sMiCCEk2WpRC5jI1Z463XLQZY90stVIHJuS64uOHKl+V1P4dB+VgY6vyyOm3aDwwxxBIMZ7AfM/ALX8EkPxmOMf5n5sZ7tUddUwPheFR0c9Jd0PQhW0I+sl1UmGAalAO64xVnFccOyH/CADnhJq8YVcZzhsyec9YHojPXaEb3RM6nzQCfvWFn1wYb9fRxaAoOr5VH66Gtbop5sJBycBeTsKQR7I113we2bdRJ9JYozlBg0h0GVicSY0zQmg8JFm5AiEMroghJ2nT7cbl7tS3vKHpDOTfoi8wNy5jRtZnYtOfZqp0Dc0mlSxiF+tRa+xaE/sjMxP8jOJmB5xORJzu1EeVu52cG/XzUhx8LaOT20sVHcO6uc2+PZbE5ebY0plrZ5SqtivI8d2g5M2mi7uMKsp1oOkpwtkc8mPeUXh05JjdKi7rpx1zRPjs5ZLUh339Ag9OERFJD0OZ4bOHqr4MMi+FD+JvBYQJBDyCj0Oet7IrkvD5qD9CKnNKzMqWXfqFrunPX1SlAyJOcIrLfxrSLYerUObQcZDMSFDuu9i/z+f3iaX1jpgV1/F1j4q8WhtrZi+1CPkeyZxGS6PgD/Mr1v+/BpB8Ki2L7I9Zpu8Re7qKKEJ19OEuaEfULIlJXbrXdy7UctEbgCCluPlJLPKiYhhD0VtgdG3gNw5/3GDVRSx94aRxWJgMa+udqXGRr/pp3u5ddSjP+fs8H3ZVngfGGXHGDyjoasb15J6KdLZjFocu17/PV4uS/hulFkY9hF2CBS5Rb8G4qwhUfakAZjsJnryycfsbMUPsSAUnvJe6OE4kW0U6/J6RCqY3KCXIYtlxuXln+TWcVHpIRpcpqfmUbQGhs31nkPZoSrStRm7YgHCP/kzrPviJj1QJcUPt5yXX5UUi2dqJg8PKYPR7XyxK1em/BR49EI2z9bbojLB3xtmRXzna1GLyYpWQJvHosVsGyMaWKXCnZ9zIorgjaBldBx8IoF9YU9g2r1T/GI4pLbUk9sugxRaTbc+RerIks0VDjjXYSQRN3dNY/UTFfNNc3a9LRhwfTJtcazotld/DiFRr1yH23jBCC4z4CFim8SZe/M6hLBB8DKxFc7/ORlt7Dz1UJ2vgSrEdw0KQuQ1bYLDUzzWMcw9gsZ+DOknhnYHAnBqPub2+qlL+RQTkWfncQAKcaigqo6rSI4B0d6JX1fZrEryYIYy75e6aMGdfKXi/RmVXoCBMH6eSlxNBHBOQ57Hj626YF1iAU+1B9N6zIxoPg3sKgG0QpPq3i0ZuYizw7Lsyx8lQsLnOmw3BrmvdZW5HYQzy21xV8Q9VXEUlv7 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(10070799003)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: RnLo10JZ/TyF63gvlKijympeTRx26Au35ui8JxqFWtKmDjru6/D9d4FHq2HOck66kcl6Qjoirg7nLosihc5t3l4eKkO1OriMjyrr8+f1jvGHdePPO3GFOM7K+L/X7ox17LodLAlC/XKksj2mSUZVN2q6a8KqFgzHpYLJ7E4HG/QruZo/370aq+PRJbzLwwI14e/UF5ju4cESkd3h0E+N+8OwQX3KoF64JdBq9bz71GjZw0LsJiWcG7Ae9pSEJnIjdDvuTWCA/0zhppU5lz5d+uL8fkzT2CpZWPVIijhlwVWv9ndIFnDIhVlFf8L0HYErZm7MLAqzk/9996noMp5h936oY+dRhqHqbW6/IGtfvo601ynxFyqqebPC/7coSaTd/NnG6MxBo/ywZqrbRdB0blUtpTOw8HClg1wf/THIoAG1ijA0te2HQRuCXP6KZaH5ku42zq0CMxjZGaVD2WUuLoYUj3lg1K7boAyMXErTQyHU+T+STyGE9S7yDZC8EfpSlUETUDY5XXtId9yRyr4SwCR9Zs2aLGWxR771YfYzbepKJxnNxp+1jDKWx2jPZlLaLFNrJcFaIkUjkgbu4G9lr1KLQK8QhCL/RCWmp7h2NJQRq29tYBLrSsc6jegoJXWgdP7xQskCUjOMVWa/Knd/RdpGtlJetSmmXAHv2MIy5MdVD4psDud5iFwdGsjAfl2Ztr6oQbob0oDGz7T3OvDMASKG+bK4x896QwYachKEx2BBVsLj6ZdMnvmW79VQvXjkHMByoqCe6w0Wyfbgl6Qr5TgQQIvMlBzjZPoWySj7RqaCnRQTdmyjbhp/VrXNVZpuzb5FJ+UBCdf60iyanieu8YMWRUjxvXUCGlLjjdjs1EZkmZHwy718ualEIS+7e0/mS9LBr4539E5nQ0UL4ADc/tbPhPGJyacnsBvEuSWY0O+YQueIXbZCynD980QSwBAW1fEdDNV4Lksm2r3Pxnfn0Tr7zBykNxIFONAAGrb4zn+VsFSYOoTwTSGF/kLnVWb5dzNJ6af54SesTSx17NfBqdi3oLzidBJAazyt9g5SJSqY+gOXMDH+xBDuY8uDknSlniQIJVEC/01DbkJlTU0hc8VkaZEguXvcyAXIeWmVPWJl7WDZsT22F2QIWqSlxnYWAwkSOqBg6ZLq8VT4PKvc98Pfbs/An/ap3nHqx3KL+uuWtaACBVtacMSbV18jhmQ5kvkwXcQmMyg92diQXxlqH0uumC4PWbG36Tje+KAGdNqpTXoeskckSgHa6eZ64cKro7MSOd8qxKPBiAYKirQE+LBbFGOrm9+pUv8Gv8kydJxq9BkBjXEppvE+gZfzIicrFEGiR26sLZs7BpN2xzz/3/H9QonMKzAzIa8Pd+bJ9aF1JXw52P1ZruizblYnH15kByO9JpRG978LozpbUoGhpnfTYaMwprXVNf/zhNk8UldHFPXbVErmzr/Y2gX5dQ15BT3piVVaMf5xBD1QdgkZzI+aEM9uxaWIZf/xqDBYdBZLcPgjXj3K/JCMmtR8dQ3m8vB79mNjVF46Adbmxqy+c6KXurCW08RYbbAMMO+Bc8ejpI6FuJpLhV8DaVoMeYQ7AXPlb5ExGZt0o8xDI5XINZPwzLE9AC01DmaawWcU2ExoaNMwoA1aV/0osMBOgDFpvP3MY6IrRP+QRd7P0NQkNA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: kwvxFeBzFuN+R8p/1W1GzTFEJojlxqOpkJYzzVVq3TvkfTHVWW37aXEDGAaZTICuG2y6geEUlFOZe3hXpr5C3cLKpbd396g4CzKD5Mknq4Vqc7zPKkhYYsnVdICeosZTbZuX7WrelYU8+xgnwff9C93zhQaiZO45H0/7yN1AhyBEviCmxQiutISE81kU+n/xFKZnqSNMNpBMyOsjY89JKvjOdlw5iLCidQN5/KFGkoOy4zOvFJ0Xpmv65trMKWtfFGnOuzXwbXCNm8Oj5QHpo2vkLivAvm0FWd1tEA6/pqjJw4KPJn6uIr/oAia+vJga06UINKz8GlQs6ZaW4S/ljaPoT7C2KWPIeA0YV9JNW5YTuhSW7rvityG0FXcOXWZOGgKMX7MDZ+HuZyY8CDAPhR8wXrtScRLyrg1v/YT7gfSiemMChERfsP6ig9kP5sWnSsB61bJLv5Ss5lKfporgPAftoHlTQIpHlOpl+wpIwdAnefrSRdGX8QmWgS2csZ25CfeLqqisjuvy3/sp1g/5RzASB9NuCHCHVCeaUZzir4XHIpIfdsZwrz+D5JiCobOiM9B6Zad444Sb15QCJdnc+Nl3gAmKJQUJIsROi1lrITo= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: ac415e02-f052-4556-7aed-08dcf4f04f69 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2024 12:26:54.4095 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: XZb6AR6beI8sEUi0jr6vDhWgWwQgFzMkemcYS1dft3EGtm5CydJFCW9Fl4CHugRyCVUla5oUL60cFgdfq7j1ChqDt6lm90C0Z+UkVu29xiI= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4484 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-25_12,2024-10-25_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 phishscore=0 suspectscore=0 bulkscore=0 adultscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410250095 X-Proofpoint-GUID: Rvno4F1x0mJy_mdNFTjy4FNT3LgaVXOE X-Proofpoint-ORIG-GUID: Rvno4F1x0mJy_mdNFTjy4FNT3LgaVXOE X-Rspam-User: X-Stat-Signature: wdf4mqkqgk6ya4ozmfd8hz1sffru7dto X-Rspamd-Queue-Id: 28A5B180010 X-Rspamd-Server: rspam11 X-HE-Tag: 1729859206-408284 X-HE-Meta: U2FsdGVkX19DI+UdT28TOMSdqEQSQV/S/eHc64QSgN6NGfd1sQWbxsnO3L9n0GfAjIxBWLqMby10Ozb5GppiyO0NPr+uW1F7+waILeKUlBSXbbNnR5iiLxeNShiA2hXrHuzGSxQ8W6Tq1X587YY9P0F/+faNAIgJlA1db8pOWYEvw3n0x6XB2AD/OC/dIW/Mkqvt5s/U6Y0VFyVZpmCdCw7L2dfXuJpbeWvjc1/biidXe/oWbXANzX1mSox03OzDjA2M98sq92gX7OA8AC5jrMs3sOb2+cWW9C9fPOvuN+WNcWOS09bf6xTGR57jYriBGXo/q/qZ4JPN9tDxcqAWdbcBMrnV0Jo0wa3A9eVS18Ddyjz8HH2lIBqgNK3wRKhu4e1uX++jowBzJHudDKBEFGEtJI0xsFTC31t1I6KXgetD+3soVFSV3l4Fc9bmQEdVg4rwdrUJAt2DuSA+tl4NZwdXfUsmqUrDvoLmhk8gGBHpowrylrWdf65IBYHd3Is9wnUDvmPWDxOMeZoE8P4NyIwAO41iqCYmm3Mn7Sb2YL8NTOVG/w7QcYKvzmYGBnNASNbxFa3qtisc4+Xw/AZyHuSsa8Xzh78A2NqFyBhstAWVtGtsbdYuMX6v3FHUewja/i79//27auBCa8W7f3b36d9BZXL5Q2KaiMOtcAaSRvG1c32cdw3/HmjOHSa6/YPWAWp+dtM1I8f7gA16fOUCYNIkFm7Q9+DPuaEZ3NLoSG6g4OOZUfYqJzVEqSPs4K/anWjGtjIrZ4vOOVN3tkIbF2jAiW2RFjmHgyrGQ7Mj8k0MbSnyvfP/6pzp6pG13BNvzjix9nBAFtZYg3BR7BlF/EhZdgoHYAvEPd1x8Gj+P63s+uDAuO01dhyWosmpzzLMwQDPpWrkupeANuwf7bq+lJ2zcTKOqOBDHHKrWXDTfWLvl4WO9Jx/Zfj9rqn57uiKUDV2Rj6PODD4OD1TN3l 6Wje2+p3 WFysqYf6Q4gS0QUQI6dPNnRdenXNfxC+slCH2x3a0rcrDLpx6CQ7vq5jNcLHiLak3sJw/y+dx6+gUaNj0LLHt+6RwIa2BTve/Mke3GPom9W2Y+lswMv8l7VZytj0PDPoy6kUcOAZ4pMgQs9YfwjWbU/HM7nMlVxcQLhNJryQDjyXeCyDJOkjUgwrZ9sD9ytkyRelX+oa/Cz52IcCZDHUqg5OtG1ppKQ/+sCb+8Z65mZAWQ936mlxBvO1xYQ+ImeYmHYscF8chQHaKaZiBKTaXbtCFu277T/6sYonAo/74E/4/PPzT1kA87nXSMt8Rvs/GX+nK6dFvuHFMAEOJLA3iv3ZafYk+gIWzV3RuGP35kDbUeS2jOZSy2y2J8I2jK2QhVvotIe+XRmyDhHPXFmtd6WxehQP8ooyxSOBHIMD7jDjlqfMn4syGb0XYODs/5BWTFtBbMRgw1fJaccJqq9T6J7N6Vi83JCBvScaT1VzmN9VG/u3IR/kPbe9hFsdTyBjLrzLygkdfFUETIqQ= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In previous commits we effected improvements to the mmap() logic in mmap_region() and its newly introduced internal implementation function __mmap_region(). However as these changes are intended to be backported, we kept the delta as small as is possible and made as few changes as possible to the newly introduced mm/vma.* files. Take the opportunity to move this logic to mm/vma.c which not only isolates it, but also makes it available for later userland testing which can help us catch such logic errors far earlier. Reviewed-by: Vlastimil Babka Signed-off-by: Lorenzo Stoakes --- mm/mmap.c | 234 --------------------------------- mm/vma.c | 323 +++++++++++++++++++++++++++++++++++++++++++++- mm/vma.h | 97 +------------- mm/vma_internal.h | 5 + 4 files changed, 329 insertions(+), 330 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index e686d57ed9f7..0affd1a0687f 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -577,22 +577,6 @@ SYSCALL_DEFINE1(old_mmap, struct mmap_arg_struct __user *, arg) } #endif /* __ARCH_WANT_SYS_OLD_MMAP */ -/* - * We account for memory if it's a private writeable mapping, - * not hugepages and VM_NORESERVE wasn't set. - */ -static inline bool accountable_mapping(struct file *file, vm_flags_t vm_flags) -{ - /* - * hugetlb has its own accounting separate from the core VM - * VM_HUGETLB may not be set yet so we cannot check for that flag. - */ - if (file && is_file_hugepages(file)) - return false; - - return (vm_flags & (VM_NORESERVE | VM_SHARED | VM_WRITE)) == VM_WRITE; -} - /** * unmapped_area() - Find an area between the low_limit and the high_limit with * the correct alignment and offset, all from @info. Note: current->mm is used @@ -1361,224 +1345,6 @@ int do_munmap(struct mm_struct *mm, unsigned long start, size_t len, return do_vmi_munmap(&vmi, mm, start, len, uf, false); } -static unsigned long __mmap_region(struct file *file, unsigned long addr, - unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, - struct list_head *uf) -{ - struct mm_struct *mm = current->mm; - struct vm_area_struct *vma = NULL; - pgoff_t pglen = PHYS_PFN(len); - unsigned long charged = 0; - struct vma_munmap_struct vms; - struct ma_state mas_detach; - struct maple_tree mt_detach; - unsigned long end = addr + len; - int error; - VMA_ITERATOR(vmi, mm, addr); - VMG_STATE(vmg, mm, &vmi, addr, end, vm_flags, pgoff); - - vmg.file = file; - /* Find the first overlapping VMA */ - vma = vma_find(&vmi, end); - init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock = */ false); - if (vma) { - mt_init_flags(&mt_detach, vmi.mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); - mt_on_stack(mt_detach); - mas_init(&mas_detach, &mt_detach, /* addr = */ 0); - /* Prepare to unmap any existing mapping in the area */ - error = vms_gather_munmap_vmas(&vms, &mas_detach); - if (error) - goto gather_failed; - - vmg.next = vms.next; - vmg.prev = vms.prev; - vma = NULL; - } else { - vmg.next = vma_iter_next_rewind(&vmi, &vmg.prev); - } - - /* Check against address space limit. */ - if (!may_expand_vm(mm, vm_flags, pglen - vms.nr_pages)) { - error = -ENOMEM; - goto abort_munmap; - } - - /* - * Private writable mapping: check memory availability - */ - if (accountable_mapping(file, vm_flags)) { - charged = pglen; - charged -= vms.nr_accounted; - if (charged) { - error = security_vm_enough_memory_mm(mm, charged); - if (error) - goto abort_munmap; - } - - vms.nr_accounted = 0; - vm_flags |= VM_ACCOUNT; - vmg.flags = vm_flags; - } - - /* - * clear PTEs while the vma is still in the tree so that rmap - * cannot race with the freeing later in the truncate scenario. - * This is also needed for mmap_file(), which is why vm_ops - * close function is called. - */ - vms_clean_up_area(&vms, &mas_detach); - vma = vma_merge_new_range(&vmg); - if (vma) - goto expanded; - /* - * Determine the object being mapped and call the appropriate - * specific mapper. the address has already been validated, but - * not unmapped, but the maps are removed from the list. - */ - vma = vm_area_alloc(mm); - if (!vma) { - error = -ENOMEM; - goto unacct_error; - } - - vma_iter_config(&vmi, addr, end); - vma_set_range(vma, addr, end, pgoff); - vm_flags_init(vma, vm_flags); - vma->vm_page_prot = vm_get_page_prot(vm_flags); - - if (vma_iter_prealloc(&vmi, vma)) { - error = -ENOMEM; - goto free_vma; - } - - if (file) { - vma->vm_file = get_file(file); - error = mmap_file(file, vma); - if (error) - goto unmap_and_free_file_vma; - - /* Drivers cannot alter the address of the VMA. */ - WARN_ON_ONCE(addr != vma->vm_start); - /* - * Drivers should not permit writability when previously it was - * disallowed. - */ - VM_WARN_ON_ONCE(vm_flags != vma->vm_flags && - !(vm_flags & VM_MAYWRITE) && - (vma->vm_flags & VM_MAYWRITE)); - - vma_iter_config(&vmi, addr, end); - /* - * If vm_flags changed after mmap_file(), we should try merge - * vma again as we may succeed this time. - */ - if (unlikely(vm_flags != vma->vm_flags && vmg.prev)) { - struct vm_area_struct *merge; - - vmg.flags = vma->vm_flags; - /* If this fails, state is reset ready for a reattempt. */ - merge = vma_merge_new_range(&vmg); - - if (merge) { - /* - * ->mmap() can change vma->vm_file and fput - * the original file. So fput the vma->vm_file - * here or we would add an extra fput for file - * and cause general protection fault - * ultimately. - */ - fput(vma->vm_file); - vm_area_free(vma); - vma = merge; - /* Update vm_flags to pick up the change. */ - vm_flags = vma->vm_flags; - goto file_expanded; - } - vma_iter_config(&vmi, addr, end); - } - - vm_flags = vma->vm_flags; - } else if (vm_flags & VM_SHARED) { - error = shmem_zero_setup(vma); - if (error) - goto free_iter_vma; - } else { - vma_set_anonymous(vma); - } - -#ifdef CONFIG_SPARC64 - /* TODO: Fix SPARC ADI! */ - WARN_ON_ONCE(!arch_validate_flags(vm_flags)); -#endif - - /* Lock the VMA since it is modified after insertion into VMA tree */ - vma_start_write(vma); - vma_iter_store(&vmi, vma); - mm->map_count++; - vma_link_file(vma); - - /* - * vma_merge_new_range() calls khugepaged_enter_vma() too, the below - * call covers the non-merge case. - */ - khugepaged_enter_vma(vma, vma->vm_flags); - -file_expanded: - file = vma->vm_file; - ksm_add_vma(vma); -expanded: - perf_event_mmap(vma); - - /* Unmap any existing mapping in the area */ - vms_complete_munmap_vmas(&vms, &mas_detach); - - vm_stat_account(mm, vm_flags, pglen); - if (vm_flags & VM_LOCKED) { - if ((vm_flags & VM_SPECIAL) || vma_is_dax(vma) || - is_vm_hugetlb_page(vma) || - vma == get_gate_vma(current->mm)) - vm_flags_clear(vma, VM_LOCKED_MASK); - else - mm->locked_vm += pglen; - } - - if (file) - uprobe_mmap(vma); - - /* - * New (or expanded) vma always get soft dirty status. - * Otherwise user-space soft-dirty page tracker won't - * be able to distinguish situation when vma area unmapped, - * then new mapped in-place (which must be aimed as - * a completely new data area). - */ - vm_flags_set(vma, VM_SOFTDIRTY); - - vma_set_page_prot(vma); - - return addr; - -unmap_and_free_file_vma: - fput(vma->vm_file); - vma->vm_file = NULL; - - vma_iter_set(&vmi, vma->vm_end); - /* Undo any partial mapping done by a device driver. */ - unmap_region(&vmi.mas, vma, vmg.prev, vmg.next); -free_iter_vma: - vma_iter_free(&vmi); -free_vma: - vm_area_free(vma); -unacct_error: - if (charged) - vm_unacct_memory(charged); - -abort_munmap: - vms_abort_munmap_vmas(&vms, &mas_detach); -gather_failed: - return error; -} - unsigned long mmap_region(struct file *file, unsigned long addr, unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, struct list_head *uf) diff --git a/mm/vma.c b/mm/vma.c index bb7cfa2dc282..0a2965be582d 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -1103,7 +1103,7 @@ static inline void vms_clear_ptes(struct vma_munmap_struct *vms, vms->clear_ptes = false; } -void vms_clean_up_area(struct vma_munmap_struct *vms, +static void vms_clean_up_area(struct vma_munmap_struct *vms, struct ma_state *mas_detach) { struct vm_area_struct *vma; @@ -1126,7 +1126,7 @@ void vms_clean_up_area(struct vma_munmap_struct *vms, * used for the munmap() and may downgrade the lock - if requested. Everything * needed to be done once the vma maple tree is updated. */ -void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, +static void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, struct ma_state *mas_detach) { struct vm_area_struct *vma; @@ -1167,6 +1167,23 @@ void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, __mt_destroy(mas_detach->tree); } +/* + * reattach_vmas() - Undo any munmap work and free resources + * @mas_detach: The maple state with the detached maple tree + * + * Reattach any detached vmas and free up the maple tree used to track the vmas. + */ +static void reattach_vmas(struct ma_state *mas_detach) +{ + struct vm_area_struct *vma; + + mas_set(mas_detach, 0); + mas_for_each(mas_detach, vma, ULONG_MAX) + vma_mark_detached(vma, false); + + __mt_destroy(mas_detach->tree); +} + /* * vms_gather_munmap_vmas() - Put all VMAs within a range into a maple tree * for removal at a later date. Handles splitting first and last if necessary @@ -1177,7 +1194,7 @@ void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, * * Return: 0 on success, error otherwise */ -int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, +static int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, struct ma_state *mas_detach) { struct vm_area_struct *next = NULL; @@ -1315,6 +1332,39 @@ int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, return error; } +/* + * init_vma_munmap() - Initializer wrapper for vma_munmap_struct + * @vms: The vma munmap struct + * @vmi: The vma iterator + * @vma: The first vm_area_struct to munmap + * @start: The aligned start address to munmap + * @end: The aligned end address to munmap + * @uf: The userfaultfd list_head + * @unlock: Unlock after the operation. Only unlocked on success + */ +static void init_vma_munmap(struct vma_munmap_struct *vms, + struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long start, unsigned long end, struct list_head *uf, + bool unlock) +{ + vms->vmi = vmi; + vms->vma = vma; + if (vma) { + vms->start = start; + vms->end = end; + } else { + vms->start = vms->end = 0; + } + vms->unlock = unlock; + vms->uf = uf; + vms->vma_count = 0; + vms->nr_pages = vms->locked_vm = vms->nr_accounted = 0; + vms->exec_vm = vms->stack_vm = vms->data_vm = 0; + vms->unmap_start = FIRST_USER_ADDRESS; + vms->unmap_end = USER_PGTABLES_CEILING; + vms->clear_ptes = false; +} + /* * do_vmi_align_munmap() - munmap the aligned region from @start to @end. * @vmi: The vma iterator @@ -2069,3 +2119,270 @@ void mm_drop_all_locks(struct mm_struct *mm) mutex_unlock(&mm_all_locks_mutex); } + +/* + * We account for memory if it's a private writeable mapping, + * not hugepages and VM_NORESERVE wasn't set. + */ +static bool accountable_mapping(struct file *file, vm_flags_t vm_flags) +{ + /* + * hugetlb has its own accounting separate from the core VM + * VM_HUGETLB may not be set yet so we cannot check for that flag. + */ + if (file && is_file_hugepages(file)) + return false; + + return (vm_flags & (VM_NORESERVE | VM_SHARED | VM_WRITE)) == VM_WRITE; +} + +/* + * vms_abort_munmap_vmas() - Undo as much as possible from an aborted munmap() + * operation. + * @vms: The vma unmap structure + * @mas_detach: The maple state with the detached maple tree + * + * Reattach any detached vmas, free up the maple tree used to track the vmas. + * If that's not possible because the ptes are cleared (and vm_ops->closed() may + * have been called), then a NULL is written over the vmas and the vmas are + * removed (munmap() completed). + */ +static void vms_abort_munmap_vmas(struct vma_munmap_struct *vms, + struct ma_state *mas_detach) +{ + struct ma_state *mas = &vms->vmi->mas; + + if (!vms->nr_pages) + return; + + if (vms->clear_ptes) + return reattach_vmas(mas_detach); + + /* + * Aborting cannot just call the vm_ops open() because they are often + * not symmetrical and state data has been lost. Resort to the old + * failure method of leaving a gap where the MAP_FIXED mapping failed. + */ + mas_set_range(mas, vms->start, vms->end - 1); + mas_store_gfp(mas, NULL, GFP_KERNEL|__GFP_NOFAIL); + /* Clean up the insertion of the unfortunate gap */ + vms_complete_munmap_vmas(vms, mas_detach); +} + +unsigned long __mmap_region(struct file *file, unsigned long addr, + unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, + struct list_head *uf) +{ + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma = NULL; + pgoff_t pglen = PHYS_PFN(len); + unsigned long charged = 0; + struct vma_munmap_struct vms; + struct ma_state mas_detach; + struct maple_tree mt_detach; + unsigned long end = addr + len; + int error; + VMA_ITERATOR(vmi, mm, addr); + VMG_STATE(vmg, mm, &vmi, addr, end, vm_flags, pgoff); + + vmg.file = file; + /* Find the first overlapping VMA */ + vma = vma_find(&vmi, end); + init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock = */ false); + if (vma) { + mt_init_flags(&mt_detach, vmi.mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); + mt_on_stack(mt_detach); + mas_init(&mas_detach, &mt_detach, /* addr = */ 0); + /* Prepare to unmap any existing mapping in the area */ + error = vms_gather_munmap_vmas(&vms, &mas_detach); + if (error) + goto gather_failed; + + vmg.next = vms.next; + vmg.prev = vms.prev; + vma = NULL; + } else { + vmg.next = vma_iter_next_rewind(&vmi, &vmg.prev); + } + + /* Check against address space limit. */ + if (!may_expand_vm(mm, vm_flags, pglen - vms.nr_pages)) { + error = -ENOMEM; + goto abort_munmap; + } + + /* + * Private writable mapping: check memory availability + */ + if (accountable_mapping(file, vm_flags)) { + charged = pglen; + charged -= vms.nr_accounted; + if (charged) { + error = security_vm_enough_memory_mm(mm, charged); + if (error) + goto abort_munmap; + } + + vms.nr_accounted = 0; + vm_flags |= VM_ACCOUNT; + vmg.flags = vm_flags; + } + + /* + * clear PTEs while the vma is still in the tree so that rmap + * cannot race with the freeing later in the truncate scenario. + * This is also needed for mmap_file(), which is why vm_ops + * close function is called. + */ + vms_clean_up_area(&vms, &mas_detach); + vma = vma_merge_new_range(&vmg); + if (vma) + goto expanded; + /* + * Determine the object being mapped and call the appropriate + * specific mapper. the address has already been validated, but + * not unmapped, but the maps are removed from the list. + */ + vma = vm_area_alloc(mm); + if (!vma) { + error = -ENOMEM; + goto unacct_error; + } + + vma_iter_config(&vmi, addr, end); + vma_set_range(vma, addr, end, pgoff); + vm_flags_init(vma, vm_flags); + vma->vm_page_prot = vm_get_page_prot(vm_flags); + + if (vma_iter_prealloc(&vmi, vma)) { + error = -ENOMEM; + goto free_vma; + } + + if (file) { + vma->vm_file = get_file(file); + error = mmap_file(file, vma); + if (error) + goto unmap_and_free_file_vma; + + /* Drivers cannot alter the address of the VMA. */ + WARN_ON_ONCE(addr != vma->vm_start); + /* + * Drivers should not permit writability when previously it was + * disallowed. + */ + VM_WARN_ON_ONCE(vm_flags != vma->vm_flags && + !(vm_flags & VM_MAYWRITE) && + (vma->vm_flags & VM_MAYWRITE)); + + vma_iter_config(&vmi, addr, end); + /* + * If vm_flags changed after mmap_file(), we should try merge + * vma again as we may succeed this time. + */ + if (unlikely(vm_flags != vma->vm_flags && vmg.prev)) { + struct vm_area_struct *merge; + + vmg.flags = vma->vm_flags; + /* If this fails, state is reset ready for a reattempt. */ + merge = vma_merge_new_range(&vmg); + + if (merge) { + /* + * ->mmap() can change vma->vm_file and fput + * the original file. So fput the vma->vm_file + * here or we would add an extra fput for file + * and cause general protection fault + * ultimately. + */ + fput(vma->vm_file); + vm_area_free(vma); + vma = merge; + /* Update vm_flags to pick up the change. */ + vm_flags = vma->vm_flags; + goto file_expanded; + } + vma_iter_config(&vmi, addr, end); + } + + vm_flags = vma->vm_flags; + } else if (vm_flags & VM_SHARED) { + error = shmem_zero_setup(vma); + if (error) + goto free_iter_vma; + } else { + vma_set_anonymous(vma); + } + +#ifdef CONFIG_SPARC64 + /* TODO: Fix SPARC ADI! */ + WARN_ON_ONCE(!arch_validate_flags(vm_flags)); +#endif + + /* Lock the VMA since it is modified after insertion into VMA tree */ + vma_start_write(vma); + vma_iter_store(&vmi, vma); + mm->map_count++; + vma_link_file(vma); + + /* + * vma_merge_new_range() calls khugepaged_enter_vma() too, the below + * call covers the non-merge case. + */ + khugepaged_enter_vma(vma, vma->vm_flags); + +file_expanded: + file = vma->vm_file; + ksm_add_vma(vma); +expanded: + perf_event_mmap(vma); + + /* Unmap any existing mapping in the area */ + vms_complete_munmap_vmas(&vms, &mas_detach); + + vm_stat_account(mm, vm_flags, pglen); + if (vm_flags & VM_LOCKED) { + if ((vm_flags & VM_SPECIAL) || vma_is_dax(vma) || + is_vm_hugetlb_page(vma) || + vma == get_gate_vma(current->mm)) + vm_flags_clear(vma, VM_LOCKED_MASK); + else + mm->locked_vm += pglen; + } + + if (file) + uprobe_mmap(vma); + + /* + * New (or expanded) vma always get soft dirty status. + * Otherwise user-space soft-dirty page tracker won't + * be able to distinguish situation when vma area unmapped, + * then new mapped in-place (which must be aimed as + * a completely new data area). + */ + vm_flags_set(vma, VM_SOFTDIRTY); + + vma_set_page_prot(vma); + + return addr; + +unmap_and_free_file_vma: + fput(vma->vm_file); + vma->vm_file = NULL; + + vma_iter_set(&vmi, vma->vm_end); + /* Undo any partial mapping done by a device driver. */ + unmap_region(&vmi.mas, vma, vmg.prev, vmg.next); +free_iter_vma: + vma_iter_free(&vmi); +free_vma: + vm_area_free(vma); +unacct_error: + if (charged) + vm_unacct_memory(charged); + +abort_munmap: + vms_abort_munmap_vmas(&vms, &mas_detach); +gather_failed: + return error; +} diff --git a/mm/vma.h b/mm/vma.h index d58068c0ff2e..388d34748674 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -165,99 +165,6 @@ static inline int vma_iter_store_gfp(struct vma_iterator *vmi, return 0; } -#ifdef CONFIG_MMU -/* - * init_vma_munmap() - Initializer wrapper for vma_munmap_struct - * @vms: The vma munmap struct - * @vmi: The vma iterator - * @vma: The first vm_area_struct to munmap - * @start: The aligned start address to munmap - * @end: The aligned end address to munmap - * @uf: The userfaultfd list_head - * @unlock: Unlock after the operation. Only unlocked on success - */ -static inline void init_vma_munmap(struct vma_munmap_struct *vms, - struct vma_iterator *vmi, struct vm_area_struct *vma, - unsigned long start, unsigned long end, struct list_head *uf, - bool unlock) -{ - vms->vmi = vmi; - vms->vma = vma; - if (vma) { - vms->start = start; - vms->end = end; - } else { - vms->start = vms->end = 0; - } - vms->unlock = unlock; - vms->uf = uf; - vms->vma_count = 0; - vms->nr_pages = vms->locked_vm = vms->nr_accounted = 0; - vms->exec_vm = vms->stack_vm = vms->data_vm = 0; - vms->unmap_start = FIRST_USER_ADDRESS; - vms->unmap_end = USER_PGTABLES_CEILING; - vms->clear_ptes = false; -} -#endif - -int vms_gather_munmap_vmas(struct vma_munmap_struct *vms, - struct ma_state *mas_detach); - -void vms_complete_munmap_vmas(struct vma_munmap_struct *vms, - struct ma_state *mas_detach); - -void vms_clean_up_area(struct vma_munmap_struct *vms, - struct ma_state *mas_detach); - -/* - * reattach_vmas() - Undo any munmap work and free resources - * @mas_detach: The maple state with the detached maple tree - * - * Reattach any detached vmas and free up the maple tree used to track the vmas. - */ -static inline void reattach_vmas(struct ma_state *mas_detach) -{ - struct vm_area_struct *vma; - - mas_set(mas_detach, 0); - mas_for_each(mas_detach, vma, ULONG_MAX) - vma_mark_detached(vma, false); - - __mt_destroy(mas_detach->tree); -} - -/* - * vms_abort_munmap_vmas() - Undo as much as possible from an aborted munmap() - * operation. - * @vms: The vma unmap structure - * @mas_detach: The maple state with the detached maple tree - * - * Reattach any detached vmas, free up the maple tree used to track the vmas. - * If that's not possible because the ptes are cleared (and vm_ops->closed() may - * have been called), then a NULL is written over the vmas and the vmas are - * removed (munmap() completed). - */ -static inline void vms_abort_munmap_vmas(struct vma_munmap_struct *vms, - struct ma_state *mas_detach) -{ - struct ma_state *mas = &vms->vmi->mas; - if (!vms->nr_pages) - return; - - if (vms->clear_ptes) - return reattach_vmas(mas_detach); - - /* - * Aborting cannot just call the vm_ops open() because they are often - * not symmetrical and state data has been lost. Resort to the old - * failure method of leaving a gap where the MAP_FIXED mapping failed. - */ - mas_set_range(mas, vms->start, vms->end - 1); - mas_store_gfp(mas, NULL, GFP_KERNEL|__GFP_NOFAIL); - /* Clean up the insertion of the unfortunate gap */ - vms_complete_munmap_vmas(vms, mas_detach); -} - int do_vmi_align_munmap(struct vma_iterator *vmi, struct vm_area_struct *vma, struct mm_struct *mm, unsigned long start, @@ -336,6 +243,10 @@ bool vma_wants_writenotify(struct vm_area_struct *vma, pgprot_t vm_page_prot); int mm_take_all_locks(struct mm_struct *mm); void mm_drop_all_locks(struct mm_struct *mm); +unsigned long __mmap_region(struct file *file, unsigned long addr, + unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, + struct list_head *uf); + static inline bool vma_wants_manual_pte_write_upgrade(struct vm_area_struct *vma) { /* diff --git a/mm/vma_internal.h b/mm/vma_internal.h index b930ab12a587..fc5f172a36bd 100644 --- a/mm/vma_internal.h +++ b/mm/vma_internal.h @@ -17,8 +17,10 @@ #include #include #include +#include #include #include +#include #include #include #include @@ -32,11 +34,14 @@ #include #include #include +#include #include #include #include #include #include +#include +#include #include #include #include From patchwork Fri Oct 25 12:26:25 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13850604 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5193FD0C603 for ; Fri, 25 Oct 2024 12:27:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6BFD06B0093; Fri, 25 Oct 2024 08:27:11 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 6489D6B0095; Fri, 25 Oct 2024 08:27:11 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 425F16B0096; Fri, 25 Oct 2024 08:27:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 1E79C6B0093 for ; Fri, 25 Oct 2024 08:27:11 -0400 (EDT) Received: from smtpin01.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id E4BA380880 for ; Fri, 25 Oct 2024 12:26:53 +0000 (UTC) X-FDA: 82712048496.01.F77463A Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf03.hostedemail.com (Postfix) with ESMTP id 21B0620017 for ; Fri, 25 Oct 2024 12:26:57 +0000 (UTC) Authentication-Results: imf03.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=m6k4klEh; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=ynLSN769; dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf03.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1729859175; a=rsa-sha256; cv=pass; b=vaalZGznwjGwSYzPbZuv3ZGjpNsjWQ7QSgFzm0CgpMR6Zd/y5dL2Tp0qyh09Wq9xZeu1Nm NxB56Lwv924z7weSL4+vhECbXhE7Y7HUPgyHJ7yVoNGsbjjChxhVTNonRN2maTKhK3ibAS TzzqdVdw8qQeUDAEKsNlzPTIxLCvgCQ= ARC-Authentication-Results: i=2; imf03.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=m6k4klEh; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=ynLSN769; dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf03.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729859175; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=tW/u9DgrsB/MKCUWnxSg/nTn3ydO0p05DZVgDkldlBM=; b=yLzpiuFK2XbNmT2cQMA9CHLZaXIprGmmVAAP9+f6/YD+vCEp+5x6tahqoDfb9mSq1YICru vEr0AwcJIV04ILnLZHaePm5aOaSB+6qOh767WSjk3KYKhVx04YxSxrK3D/vYAMAn9iNkZu YaYMbBaTCuLGqJpYVnQbo4JhdBJFzjE= Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49P8BZn4005842; Fri, 25 Oct 2024 12:27:04 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=tW/u9DgrsB/MKCUWnxSg/nTn3ydO0p05DZVgDkldlBM=; b= m6k4klEhZMTAqv1rfH8Kjx7SFwysJqy/4V3O1njAw/SiEUWNEV3GoMgw+Q2YG/sL qMptBZ2AVcsVCfA1pikazNQO33ZhvDy0thO0uT5RznlpIe4T9XIxuwPMlvR5zQy5 puCGTKolg6hx8gQLGb/lR0k8L3BGkyszCx1bxJKFKlcsBfbKGkVxJJDwp5mgMr7j ipPfMu6XQ8Rr7l5ik8ywzA8If34Y7hqXZ4ASfviTVBP2bBDr1kwHmdHzjMrV9SM3 SC38VGjrStWZBOjD2Q2psUGJE9+RRvu/7CShWxrcBXrSlNp0Fyq/QxeFD/gsa/08 aYucDczmkZ86uwiqEYDrrQ== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42c53uvttx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:27:04 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49PBxpRl016475; Fri, 25 Oct 2024 12:27:02 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2171.outbound.protection.outlook.com [104.47.58.171]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42emhdt4th-2 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:27:02 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=rsdrgkE9ZFZFPuiQsGU/nQ2iWNimlBe7hzpNeoEEswJARGk2dX8xLVXfC0EIIFrHW3tmG4jqF4+tLE3eExWOz05ZxhW2ccg5/YLVCR+W70Yr813loNtiv5X8yJGTugGgRjtQQDUVzAXqw5OV3jwpXeuXKTAcdRrTbRCjcmAzr5+oI0kNlaeUsowEm9rZf0+LO9neaKG32kISfs2NJH7yQCVu9E9PlKNlCQ8BPMHaAYoofbxTH9ilDJvh6L+CrUAjbe1ttdgTQuhO2/ca4fhGzn7vAwPZwVPbVwLZuZ81AWmwKX11kSKd0EoAZemphltv5HihdAiONy56UYItDjf9aw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=tW/u9DgrsB/MKCUWnxSg/nTn3ydO0p05DZVgDkldlBM=; b=RQGXV4fvH0pl/3KB4gdAlSSw31GJKnEDsjVIlZE2i98MXhNgrTfOyyiapnjF+uq9r81wdbDDKs0WSFu3gnpd56mWyToBcuPi0t7AogDuxXDVIS5DPGMuIhDTzlT66gvFrG/Z8Q+cD6JOtsp3Kp14Dbx2s1ksP/Dn0tAE+J+ynqvSkJlZoJU6vQ3sA14XLyq1oFZys7OJkFu2T6dsuljh8LbPjAEgl2N6H1vP6KbXBFaE04hXfHRcZkA0bwXItgVHp47Wnysw3N1E8hpx41ocoIVZGjfqgGaymeRAEnRqEb5/GovW00XZoQJKISRWVpy9C24gzsn5CUwWPGgEtewF7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=tW/u9DgrsB/MKCUWnxSg/nTn3ydO0p05DZVgDkldlBM=; b=ynLSN769BOlRDcAWIQlrfLrhY6gZ/el/zfM5rrw45bZEuPkWYJ1sfmbgXxFDMfgQQlhvh8AnT1lE/CdLuiUKV+kuuVKSyVhb4egKxkfonWdUmdTX81CkgEzrIAd5ozaNRPFWBsX5OPZpHEU/E5e+XRLTNpZaKmmD0NTyRo5z4vs= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by CO1PR10MB4484.namprd10.prod.outlook.com (2603:10b6:303:90::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.23; Fri, 25 Oct 2024 12:26:57 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8093.014; Fri, 25 Oct 2024 12:26:57 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH v3 3/5] mm: refactor __mmap_region() Date: Fri, 25 Oct 2024 13:26:25 +0100 Message-ID: <25bd2edc3275450f448cbfe0756ce2a7cd06810f.1729858176.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-ClientProxiedBy: LO4P265CA0164.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:312::13) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|CO1PR10MB4484:EE_ X-MS-Office365-Filtering-Correlation-Id: 914a8a93-d272-42ce-1a5a-08dcf4f0515b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|10070799003|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: osZhqhfKNS8pMS3EbYV+v+gfXfQocnPHpeAIxQyXyC8P9N14h74d8mUd90GUAq3EFnCSeVhZ9ZPAuehGo5oa7ww1BTQwuJb+shYxwCp4KDU1/RoEMB8EhWFSsaV0XS3QDjaFetrcT5bKc+cZ4jxUCJ/syYONDQ1NytNQDDJiZzKRTswR36aRVMnS5MA0/p9qhGDSlqP4fpxrjo4/HGLn9E5OIz2R5gSmKKrc9BdV4ada8pHfbJD/bS9y5TTqpwc8zDJLPS9TrxnXpM418zxkYGDEdnB3dfV2X9MpDrJC8o6R0FH3DpBvaj91gjU2eijRIwoHs2d0tj/30LgVdGuq1/zzGiaFUXuJF63KhqF7Qjsr5xMRMRFCXK8Ep94m2mnWUPQiCrWfH8zV1sp8BuDRkBcoUChX+CnpOpV1ttXneVi3jeVOre7muQjSyBMvdH3meu2qD+dqnyhOGe2HfOfxk+Z4LnxheyAiAJw12VHusArRXJ1FA+uVEondqITRk2JQPfS/sJcsIbkjJDly3HWej/iD9eT9LuXnI0XS4doaPQQaAtcwUXPFfs0P3maMEaZNiWG1haKHX8Hs4N+6qt6DNAu8FW9N29hEkdVZ9ixLH4XVY3y1Y5mCZipSXyC/r5IAmZr15ESoK0Z5TuMJjHEHKrxlnMmc9S3SA6DNw+goKh3JcIlB8q/0dMgUD51ywnXvbSj6Aew7dyEm81fwEsui7TUfXctg5rqTZs0gqdfFTIgXqBQOxP9Nr3Mt9h1xlgfCvGRtmvY76tAZv6ZAclFwtomZ5sQCnynH6GneTfrgAcUlNXytHL2ugj162mBcT0YOrzrc3Ke4ZfHMrLRBXVPx+rsIA+91qIhXSbWL5sjIQsuoz2hOmoaJsSCp08fP3uqDeI0Eq1qM8XNURICPxXimHPOAbsulphidJirLS9V3MXz2cu6EP04ATAJgOeSiONV3Y8UdJE6zO5fAeKDusWVYH8ZTeDBirupZA3PG0WAB1QqF+BHwTbD+Br4OHKx7HMW6PE1hJFu4Ogs4Bm9isX5jVyR5rEs2KHCogb1pNsF0FNffV5BryUrvXjXPcp7VNtgjPXVXHgsOdDb4kjZO/SD5KGQJoWXiLYtXsk7MWKYSeVgYMVORnBjl+vAGdgzGsrBTWtTml/7uDqbE5Q91DK/snvY24A0lpUqz8MRXMD4xZu1nyMPQo7Xg+DFbaT/94N1gAK/zD3p8flrSZCQmKfdpKkwxUSZ7OHYfT2V8TOEZaTaMcY5wjuQiSgqKhJmfsEJ6Fq0sKEH7YeBHOn2/KuW7O+IMts4R0gn17fmagqiX+8nmUJB7nyGF6hmxYBVIcVeP X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(10070799003)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: r6HNeyun50+FwmOiBnQmv6cpgDVyOHRsFTDfvwwXJPoJ28dGMfVKWolJRdA1qHhGRripqHSV9wEhsCTlBsLUEHOKEav59AWtm38BQJidvDyUNEWkhP7CeoQuHIyGRC7diVZV/dYWhzbKDHdvWsTzNhn81d44nvNZE4Odadb9lH5wa/g0H4KooKiWbSfz3Fwvtpxh0prEUKH+RwCNpnM5yplW1MUEZ77MYIa1rmabLXm5iuZMI+PRph1n0VXc03hl+PVVafWMsZi/VIeO4NpEriaEWhBghpyxS4upjh7+MF0uK4ENT/PLQxukvCDQNa4tQSmKlQALnS2Q5PCTN59XyXyKF4Mm1fC+t/SoobgzX5AwC+QP+kxwWpBu7/bp08lnx7Uy2kVndJH0kk6vjiVjes7lybQTjuZyBnxbO53RuFhsoASL4wlqJBarVQJIz5DDJEbC8tjAgR44Zvvi3lfZd9M9JiBK48+iBIDiEys+Mdsyimkz7t2PSJ4DjXsa4cNMpla20Lbe38C8+YH2ble7Qc1Y7OOiyjy+lfNAWDqiHD1evQXL7yZCCGX/pPWcRGI05XPL4SIk2Thqs83sdKRMSHCaRbUNwHORjbdjXEw6k1fF7o5rHDrWw/LlZBnjgS6BaP5C/iaz6CZNTLdii1lqi/N++6ePkd8sgkyscoO2kMse1ceGnXnVm9vsqGI5aDyTiHhR040yNSKhr5oCq/4jEafWQhgTt13mfWlRMVYPPyIBEZTOvMDJ0YZafM09HJ5CnO9uzinhucQ3VDf5tdeYacCB/8rgqdVS9z5/UFmAcHabRwyZpoKVb7CXqKt8oBo+IzC1hTelDq9Fm5xYm521PzBYSipgNE7gWr57VYmHakrbQl4eNBn6jekeMJmTANLW2SgF+p7RHoWhvsem/Uq4iey8fuH9d9bEO5lQwCAj+u4aenpdJpfyyCBIrBvmUtu2uIlacxWSwRZJOvDXXDyyvwKhDXCmIPgIH2ajIVfZmhrsEsW95hK510uUrmxk+5P0e1rpdDkpk8Z/jDGAVWT/ZY1f0ecuZP19JUwtx1l9mqjJMb49oBc+Bf3lIJk9Oh3frXYKQvP0OlM+fwDvaOF3Y0sS9Xm5fGoXCT9N0Nx9/t30KvXu1VUe1HO0uDVYsFmejLZa9JblBvUbz10Df66JFpfhZg5aUZuH7wHZyDy3bJ72jhcMMqEtZ4P0nwX3i2XaP27N/1rJUYq0WGaTr7u5wBo8iDur/2nD15byX0xKe8O8+bKtjqRMqnOD22K1GPvTrZn/2gRkBxjyRoqoaGbdovIIYwBcPOmUBY9TgIE0cnPA5wpBQlv732FDkJUHdkSBeRCC7gx49CFAJ6/nQoLdnwbA4KtA/UvVOQfBlqnFvPu61UwaPYu0IX6NnAkk7Ul5Tu4awfyaGnV11pODfN2mlPDheB6u5Dk+D3XeOASsUm9r3ZIXtwUsNo2UyYjYtcaTWerODfP8SIcW9TpHPQ7xCTqxX1RUs4/ftPtmM6OGuvwX5d75+rSYlsHvutGmXLSGoDNQeUCR6v+kHQRzeON7Q1x9XPXBaDba3Hg16HPy5yDFdJU23k+l5kiYwN/wAYKqq2bXGMPoND/XhIEfeIMmBl5NEr76y5uteH8pLtnv/sCfCqlb+lSHhRZvH29p7R3dvKdfXhRDCQqwi0vb4/g+eA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: etryJ5fLoaS1vvp81pFYFoLxRtvmdAc6uyK+iDPQyZUfRNH5xy16pTrqP+/aY5LMEtd8WSfpoejumL4fUbxPUbkOpw5hslWuBmjmAQo5UN/41H6lx+GmbRk4wyus4MUp/+K5aRuYa491bXADo5HPYSHUerOQnaO0qzi9vwDnNxfpuAVP4bHOEZYuYmmQxgJcCp26h/JutBjFC0nlS56UxNquTa4RlwZnz9KhNXImN9m3yEVFdusFVvB1amCj6JBg3lk5FBnP+El1AuIzJbHRR0EXruXs9D7GMlprcrBs6YlPWTQm1tXm0ZsX2vWZTZPtx3PnicVGb+FP+3aW/6PbSY7gbfquQFXPFpWJv3mdpSwGA6Pt816szI3vx0EuWNJ/6E5S87/glb6tLxaNwkPntaTtpSdIf5XgtdWdN5DWgH5VvxEoQd9RGfEeygEHYbVlVJCXKK/hJo2627HsvlYBH/FrITm/mWbdBg3Ctnvv6nTzD8XWMEl4//4gQtPRl3IqJZNBqoVq0VdG0fqvmMZpexI9icIFUvp90iuKxzkLWhrJ7k4be2Bj7SXtJRsT6pf5ivLIRVsFnLN/8E+US0J3Js2ESnjJmgB7G8w8UWJaD98= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 914a8a93-d272-42ce-1a5a-08dcf4f0515b X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2024 12:26:57.5594 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: UoAwdC64miYexY8atyLM0kplhvI55RkL8ISG5ZOSJt8ztVlB4DmA2q+41bZVZoSlbxbXFKLaiiRhXdvRJXR43xC311p1zExijZ/dwP/q89w= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4484 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-25_12,2024-10-25_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 phishscore=0 suspectscore=0 bulkscore=0 adultscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410250095 X-Proofpoint-GUID: 8HOIUugaD2zHS8gRfFuKMNjyXbOJt0Cz X-Proofpoint-ORIG-GUID: 8HOIUugaD2zHS8gRfFuKMNjyXbOJt0Cz X-Rspam-User: X-Rspamd-Queue-Id: 21B0620017 X-Rspamd-Server: rspam01 X-Stat-Signature: g6i1ydbgokiahnmwjssc1bahpx97qjsf X-HE-Tag: 1729859217-84177 X-HE-Meta: U2FsdGVkX1/ErIZwOT6lnB5VvKQ6m0xBpJa+se/JE+XOok13Bl1xP9WQxPah2mdq/Eh9sCHOQaUcItd6HS2wpPOdeLyCNAdPLN5G8tb5yUcRdsGKrSQq1lpS/KOI6OQxWGm8m5gO5DshHfxUOmwjwmFpNIet9neTcDdPk7cw/4s1RRAgbIGzXX393DSDWLKwD5/qO21+Rj+7jAddJoZ2Lk721S21jMYw4y7NhIckCo7HAYql+K2wvKSYJoDhpAQuJ5H5PpWX2djVmmNKDdcaSkGAIf4NuodzC4XpSs9/+Tv+46p1rxbsfypRgfqpTZhEgAHTR+i/8XuzCFe99/T+jyqaNjp2HdW69tu6Pgb7x6srfE91XXGTyH063qrpqdIfDU4IOSc8WSRe/jwGNfjaEPcd1BtAiHF02yLTC+pPWXps1uG8J8VBt0eoTF3vSkwfx52PDA5KjUTgSHhoIlZhQSOe9qXQ3bNWaAmQiuk0kz9CVwUFCze+lvWP7RXgMWAK9FaVbwqySuJHECtR/CLaBlWE4x/OLuDTZT7C2irAGxkW4XEMJ+THeaWcG1bQqkcKwRgjWPUCvSlEHY6dE7WyKsbEIiYp6b3g6oUirzoIzt4gYyk0ozui+Z0vDwsZaLIaEciN1qrlzyPkk0YmE1WV68W+QKJnP7rFj9RKXdEKSJUsAGGJP+4xyrpEAn/p67A6GoZ7b6QG2P554rbcVHSyHHz2uCkp+wrUEQgTKBmul/8Y9D7w8iGx0fVnnTGX3WaEjqtfskLjMJYUMNFoLEiqQ38D9VnAPLeRfqf9yioDAAqG7cpFfEy2OYCUkpMYFfssbcXootXI9sth5oZPi7c2D8/V2n6PExOUOCJ/ixdyrD4Jod68ceCl+Lkq1Dpn1jdGtXWXGKSpNSTr1YYdzHuyPJJ2TqMng2fTHa9rHk7xc//fajjUaqj414z6HdhmzCVQk0A3z+V0pj3PR179GkS gnW+8LkI wJE8S9/S72bebNnVOct1oFO0zZ53ZYgUrr5FdD7Xdx8tLzIYHxDHYqS4O2bBYLsTg1Dr+lFOQwWdvpn/p4DdJNUva1lXdeGvpZfdHE0BDt2HhjEk/i9In4mu4SrElvHHJt9jiyq03thbHXPHHP2suW9//qn/Hnb2j3vMavc7LCMWAfLR+aGznCAJL2mcM0jtjxUzzBE1OhN3Ui/BTY0uYFypW/Z8PNqN27rYMBc7+E8THSallel13WDP7MNuFqE0eXXjvw/Tmx4/zYnYyqm2HnQ16R+089dtkGyuKfL+o3nMsN56Hh2+h6vlraZXjVNxYHChTgONzdqDNYjzm42aFytft9AIp922OB6VkFcbRia+FG3rPk5Bb/28oIL3jy1CAPTREU75+i1P4LKHUeugCKMeVeWPFTdL48PFQxknL3TEMVo3MZmyswsFBArMbBsGeFsriJIg3CQqYG6oRkW3MK86Da3qzGqj+3BTLpS+ey00NeAsRonPuXI5luiaGM5BFJG+88lQ+syU5gY6LjJVkwCavD4+8/ucvIN60CxfVymZja4B1W31UGx3yCnnhoyjOMAry X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: We have seen bugs and resource leaks arise from the complexity of the __mmap_region() function. This, and the generally deeply fragile error handling logic and complexity which makes understanding the function difficult make it highly desirable to refactor it into something readable. Achieve this by separating the function into smaller logical parts which are easier to understand and follow, and which importantly very significantly simplify the error handling. Note that we now call vms_abort_munmap_vmas() in more error paths than we used to, however in cases where no abort need occur, vms->nr_pages will be equal to zero and we simply exit this function without doing more than we would have done previously. Importantly, the invocation of the driver mmap hook via mmap_file() now has very simple and obvious handling (this was previously the most problematic part of the mmap() operation). Use a generalised stack-based 'mmap state' to thread through values and also retrieve state as needed. Also avoid ever relying on vma merge (vmg) state after a merge is attempted, instead maintain meaningful state in the mmap state and establish vmg state as and when required. This avoids any subtle bugs arising from merge logic mutating this state and mmap_region() logic later relying upon it. Reviewed-by: Vlastimil Babka Signed-off-by: Lorenzo Stoakes --- mm/vma.c | 410 ++++++++++++++++++++++++++++++++++++------------------- 1 file changed, 270 insertions(+), 140 deletions(-) diff --git a/mm/vma.c b/mm/vma.c index 0a2965be582d..b91c947babd6 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -7,6 +7,56 @@ #include "vma_internal.h" #include "vma.h" +struct mmap_state { + struct mm_struct *mm; + struct vma_iterator *vmi; + + unsigned long addr; + unsigned long end; + pgoff_t pgoff; + unsigned long pglen; + unsigned long flags; + struct file *file; + + unsigned long charged; + + struct vm_area_struct *prev; + struct vm_area_struct *next; + + /* Unmapping state. */ + struct vma_munmap_struct vms; + struct ma_state mas_detach; + struct maple_tree mt_detach; +}; + +#define MMAP_STATE(name, mm_, vmi_, addr_, len_, pgoff_, flags_, file_) \ + struct mmap_state name = { \ + .mm = mm_, \ + .vmi = vmi_, \ + .addr = addr_, \ + .end = (addr_) + len, \ + .pgoff = pgoff_, \ + .pglen = PHYS_PFN(len_), \ + .flags = flags_, \ + .file = file_, \ + } + +#define VMG_MMAP_STATE(name, map_, vma_) \ + struct vma_merge_struct name = { \ + .mm = (map_)->mm, \ + .vmi = (map_)->vmi, \ + .start = (map_)->addr, \ + .end = (map_)->end, \ + .flags = (map_)->flags, \ + .pgoff = (map_)->pgoff, \ + .file = (map_)->file, \ + .prev = (map_)->prev, \ + .vma = vma_, \ + .next = (vma_) ? NULL : (map_)->next, \ + .state = VMA_MERGE_START, \ + .merge_flags = VMG_FLAG_DEFAULT, \ + } + static inline bool is_mergeable_vma(struct vma_merge_struct *vmg, bool merge_next) { struct vm_area_struct *vma = merge_next ? vmg->next : vmg->prev; @@ -2169,188 +2219,249 @@ static void vms_abort_munmap_vmas(struct vma_munmap_struct *vms, vms_complete_munmap_vmas(vms, mas_detach); } -unsigned long __mmap_region(struct file *file, unsigned long addr, - unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, - struct list_head *uf) +/* + * __mmap_prepare() - Prepare to gather any overlapping VMAs that need to be + * unmapped once the map operation is completed, check limits, account mapping + * and clean up any pre-existing VMAs. + * + * @map: Mapping state. + * @uf: Userfaultfd context list. + * + * Returns: 0 on success, error code otherwise. + */ +static int __mmap_prepare(struct mmap_state *map, struct list_head *uf) { - struct mm_struct *mm = current->mm; - struct vm_area_struct *vma = NULL; - pgoff_t pglen = PHYS_PFN(len); - unsigned long charged = 0; - struct vma_munmap_struct vms; - struct ma_state mas_detach; - struct maple_tree mt_detach; - unsigned long end = addr + len; int error; - VMA_ITERATOR(vmi, mm, addr); - VMG_STATE(vmg, mm, &vmi, addr, end, vm_flags, pgoff); - - vmg.file = file; - /* Find the first overlapping VMA */ - vma = vma_find(&vmi, end); - init_vma_munmap(&vms, &vmi, vma, addr, end, uf, /* unlock = */ false); - if (vma) { - mt_init_flags(&mt_detach, vmi.mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); - mt_on_stack(mt_detach); - mas_init(&mas_detach, &mt_detach, /* addr = */ 0); + struct vma_iterator *vmi = map->vmi; + struct vma_munmap_struct *vms = &map->vms; + + /* Find the first overlapping VMA and initialise unmap state. */ + vms->vma = vma_find(vmi, map->end); + init_vma_munmap(vms, vmi, vms->vma, map->addr, map->end, uf, + /* unlock = */ false); + + /* OK, we have overlapping VMAs - prepare to unmap them. */ + if (vms->vma) { + mt_init_flags(&map->mt_detach, + vmi->mas.tree->ma_flags & MT_FLAGS_LOCK_MASK); + mt_on_stack(map->mt_detach); + mas_init(&map->mas_detach, &map->mt_detach, /* addr = */ 0); /* Prepare to unmap any existing mapping in the area */ - error = vms_gather_munmap_vmas(&vms, &mas_detach); - if (error) - goto gather_failed; + error = vms_gather_munmap_vmas(vms, &map->mas_detach); + if (error) { + /* On error VMAs will already have been reattached. */ + vms->nr_pages = 0; + return error; + } - vmg.next = vms.next; - vmg.prev = vms.prev; - vma = NULL; + map->next = vms->next; + map->prev = vms->prev; } else { - vmg.next = vma_iter_next_rewind(&vmi, &vmg.prev); + map->next = vma_iter_next_rewind(vmi, &map->prev); } /* Check against address space limit. */ - if (!may_expand_vm(mm, vm_flags, pglen - vms.nr_pages)) { - error = -ENOMEM; - goto abort_munmap; - } + if (!may_expand_vm(map->mm, map->flags, map->pglen - vms->nr_pages)) + return -ENOMEM; - /* - * Private writable mapping: check memory availability - */ - if (accountable_mapping(file, vm_flags)) { - charged = pglen; - charged -= vms.nr_accounted; - if (charged) { - error = security_vm_enough_memory_mm(mm, charged); + /* Private writable mapping: check memory availability. */ + if (accountable_mapping(map->file, map->flags)) { + map->charged = map->pglen; + map->charged -= vms->nr_accounted; + if (map->charged) { + error = security_vm_enough_memory_mm(map->mm, map->charged); if (error) - goto abort_munmap; + return error; } - vms.nr_accounted = 0; - vm_flags |= VM_ACCOUNT; - vmg.flags = vm_flags; + vms->nr_accounted = 0; + map->flags |= VM_ACCOUNT; } /* - * clear PTEs while the vma is still in the tree so that rmap + * Clear PTEs while the vma is still in the tree so that rmap * cannot race with the freeing later in the truncate scenario. * This is also needed for mmap_file(), which is why vm_ops * close function is called. */ - vms_clean_up_area(&vms, &mas_detach); - vma = vma_merge_new_range(&vmg); - if (vma) - goto expanded; + vms_clean_up_area(vms, &map->mas_detach); + + return 0; +} + +static int __mmap_new_file_vma(struct mmap_state *map, + struct vm_area_struct **vmap, bool *mergedp) +{ + struct vma_iterator *vmi = map->vmi; + struct vm_area_struct *vma = *vmap; + int error; + + vma->vm_file = get_file(map->file); + error = mmap_file(vma->vm_file, vma); + if (error) { + fput(vma->vm_file); + vma->vm_file = NULL; + + vma_iter_set(vmi, vma->vm_end); + /* Undo any partial mapping done by a device driver. */ + unmap_region(&vmi->mas, vma, map->prev, map->next); + + return error; + } + + /* Drivers cannot alter the address of the VMA. */ + WARN_ON_ONCE(map->addr != vma->vm_start); + /* + * Drivers should not permit writability when previously it was + * disallowed. + */ + VM_WARN_ON_ONCE(map->flags != vma->vm_flags && + !(map->flags & VM_MAYWRITE) && + (vma->vm_flags & VM_MAYWRITE)); + + /* mmap_file() might have changed VMA flags. */ + map->flags = vma->vm_flags; + + vma_iter_config(vmi, map->addr, map->end); + /* + * If flags changed after mmap_file(), we should try merge + * vma again as we may succeed this time. + */ + if (unlikely(map->flags != vma->vm_flags && map->prev)) { + struct vm_area_struct *merge; + VMG_MMAP_STATE(vmg, map, /* vma = */ NULL); + + merge = vma_merge_new_range(&vmg); + if (merge) { + /* + * ->mmap() can change vma->vm_file and fput + * the original file. So fput the vma->vm_file + * here or we would add an extra fput for file + * and cause general protection fault + * ultimately. + */ + fput(vma->vm_file); + vm_area_free(vma); + vma = merge; + *mergedp = true; + } else { + vma_iter_config(vmi, map->addr, map->end); + } + } + + *vmap = vma; + return 0; +} + +/* + * __mmap_new_vma() - Allocate a new VMA for the region, as merging was not + * possible. + * + * An exception to this is if the mapping is file-backed, and the underlying + * driver changes the VMA flags, permitting a subsequent merge of the VMA, in + * which case the returned VMA is one that was merged on a second attempt. + * + * @map: Mapping state. + * @vmap: Output pointer for the new VMA. + * + * Returns: Zero on success, or an error. + */ +static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap) +{ + struct vma_iterator *vmi = map->vmi; + int error = 0; + bool merged = false; + struct vm_area_struct *vma; + /* * Determine the object being mapped and call the appropriate * specific mapper. the address has already been validated, but * not unmapped, but the maps are removed from the list. */ - vma = vm_area_alloc(mm); - if (!vma) { - error = -ENOMEM; - goto unacct_error; - } + vma = vm_area_alloc(map->mm); + if (!vma) + return -ENOMEM; - vma_iter_config(&vmi, addr, end); - vma_set_range(vma, addr, end, pgoff); - vm_flags_init(vma, vm_flags); - vma->vm_page_prot = vm_get_page_prot(vm_flags); + vma_iter_config(vmi, map->addr, map->end); + vma_set_range(vma, map->addr, map->end, map->pgoff); + vm_flags_init(vma, map->flags); + vma->vm_page_prot = vm_get_page_prot(map->flags); - if (vma_iter_prealloc(&vmi, vma)) { + if (vma_iter_prealloc(vmi, vma)) { error = -ENOMEM; goto free_vma; } - if (file) { - vma->vm_file = get_file(file); - error = mmap_file(file, vma); - if (error) - goto unmap_and_free_file_vma; - - /* Drivers cannot alter the address of the VMA. */ - WARN_ON_ONCE(addr != vma->vm_start); - /* - * Drivers should not permit writability when previously it was - * disallowed. - */ - VM_WARN_ON_ONCE(vm_flags != vma->vm_flags && - !(vm_flags & VM_MAYWRITE) && - (vma->vm_flags & VM_MAYWRITE)); - - vma_iter_config(&vmi, addr, end); - /* - * If vm_flags changed after mmap_file(), we should try merge - * vma again as we may succeed this time. - */ - if (unlikely(vm_flags != vma->vm_flags && vmg.prev)) { - struct vm_area_struct *merge; - - vmg.flags = vma->vm_flags; - /* If this fails, state is reset ready for a reattempt. */ - merge = vma_merge_new_range(&vmg); - - if (merge) { - /* - * ->mmap() can change vma->vm_file and fput - * the original file. So fput the vma->vm_file - * here or we would add an extra fput for file - * and cause general protection fault - * ultimately. - */ - fput(vma->vm_file); - vm_area_free(vma); - vma = merge; - /* Update vm_flags to pick up the change. */ - vm_flags = vma->vm_flags; - goto file_expanded; - } - vma_iter_config(&vmi, addr, end); - } - - vm_flags = vma->vm_flags; - } else if (vm_flags & VM_SHARED) { + if (map->file) + error = __mmap_new_file_vma(map, &vma, &merged); + else if (map->flags & VM_SHARED) error = shmem_zero_setup(vma); - if (error) - goto free_iter_vma; - } else { + else vma_set_anonymous(vma); - } + + if (error) + goto free_iter_vma; + + if (merged) + goto file_expanded; #ifdef CONFIG_SPARC64 /* TODO: Fix SPARC ADI! */ - WARN_ON_ONCE(!arch_validate_flags(vm_flags)); + WARN_ON_ONCE(!arch_validate_flags(map->flags)); #endif /* Lock the VMA since it is modified after insertion into VMA tree */ vma_start_write(vma); - vma_iter_store(&vmi, vma); - mm->map_count++; + vma_iter_store(vmi, vma); + map->mm->map_count++; vma_link_file(vma); /* * vma_merge_new_range() calls khugepaged_enter_vma() too, the below * call covers the non-merge case. */ - khugepaged_enter_vma(vma, vma->vm_flags); + khugepaged_enter_vma(vma, map->flags); file_expanded: - file = vma->vm_file; ksm_add_vma(vma); -expanded: + *vmap = vma; + return 0; + +free_iter_vma: + vma_iter_free(vmi); +free_vma: + vm_area_free(vma); + return error; +} + +/* + * __mmap_complete() - Unmap any VMAs we overlap, account memory mapping + * statistics, handle locking and finalise the VMA. + * + * @map: Mapping state. + * @vma: Merged or newly allocated VMA for the mmap()'d region. + */ +static void __mmap_complete(struct mmap_state *map, struct vm_area_struct *vma) +{ + struct mm_struct *mm = map->mm; + unsigned long vm_flags = vma->vm_flags; + perf_event_mmap(vma); - /* Unmap any existing mapping in the area */ - vms_complete_munmap_vmas(&vms, &mas_detach); + /* Unmap any existing mapping in the area. */ + vms_complete_munmap_vmas(&map->vms, &map->mas_detach); - vm_stat_account(mm, vm_flags, pglen); + vm_stat_account(mm, vma->vm_flags, map->pglen); if (vm_flags & VM_LOCKED) { if ((vm_flags & VM_SPECIAL) || vma_is_dax(vma) || is_vm_hugetlb_page(vma) || - vma == get_gate_vma(current->mm)) + vma == get_gate_vma(mm)) vm_flags_clear(vma, VM_LOCKED_MASK); else - mm->locked_vm += pglen; + mm->locked_vm += map->pglen; } - if (file) + if (vma->vm_file) uprobe_mmap(vma); /* @@ -2363,26 +2474,45 @@ unsigned long __mmap_region(struct file *file, unsigned long addr, vm_flags_set(vma, VM_SOFTDIRTY); vma_set_page_prot(vma); +} - return addr; +unsigned long __mmap_region(struct file *file, unsigned long addr, + unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, + struct list_head *uf) +{ + struct mm_struct *mm = current->mm; + struct vm_area_struct *vma = NULL; + int error; + VMA_ITERATOR(vmi, mm, addr); + MMAP_STATE(map, mm, &vmi, addr, len, pgoff, vm_flags, file); -unmap_and_free_file_vma: - fput(vma->vm_file); - vma->vm_file = NULL; + error = __mmap_prepare(&map, uf); + if (error) + goto abort_munmap; - vma_iter_set(&vmi, vma->vm_end); - /* Undo any partial mapping done by a device driver. */ - unmap_region(&vmi.mas, vma, vmg.prev, vmg.next); -free_iter_vma: - vma_iter_free(&vmi); -free_vma: - vm_area_free(vma); -unacct_error: - if (charged) - vm_unacct_memory(charged); + /* Attempt to merge with adjacent VMAs... */ + if (map.prev || map.next) { + VMG_MMAP_STATE(vmg, &map, /* vma = */ NULL); + + vma = vma_merge_new_range(&vmg); + } + /* ...but if we can't, allocate a new VMA. */ + if (!vma) { + error = __mmap_new_vma(&map, &vma); + if (error) + goto unacct_error; + } + + __mmap_complete(&map, vma); + + return addr; + + /* Accounting was done by __mmap_prepare(). */ +unacct_error: + if (map.charged) + vm_unacct_memory(map.charged); abort_munmap: - vms_abort_munmap_vmas(&vms, &mas_detach); -gather_failed: + vms_abort_munmap_vmas(&map.vms, &map.mas_detach); return error; } From patchwork Fri Oct 25 12:26:26 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13850605 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38C94D0C601 for ; Fri, 25 Oct 2024 12:27:15 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1DFCE6B0095; Fri, 25 Oct 2024 08:27:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 18C5A6B0096; Fri, 25 Oct 2024 08:27:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E364F6B0098; Fri, 25 Oct 2024 08:27:11 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id BB1406B0095 for ; Fri, 25 Oct 2024 08:27:11 -0400 (EDT) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 90E44AC2B9 for ; Fri, 25 Oct 2024 12:26:32 +0000 (UTC) X-FDA: 82712048832.21.915D073 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf18.hostedemail.com (Postfix) with ESMTP id 5185A1C0008 for ; Fri, 25 Oct 2024 12:26:59 +0000 (UTC) Authentication-Results: imf18.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=DbEvce0i; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=LjAvkv1k; dmarc=pass (policy=reject) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf18.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1729859188; a=rsa-sha256; cv=pass; b=WL/sn6AqQfKPGfSwS3PUO1XhYEcbPLb3FptyIUbnMQpI+cKaqo65Y8XC4ts04p9/HL7QRW TyVDD/j/aITj795z7JVwlgzh/QoWh2tyG1l6Cv3ehPL603RGYmBuEvIj/ZioCecSUXQpn2 oKr6BaGl4KcluKLXJaFB9qMSYHO3xRY= ARC-Authentication-Results: i=2; imf18.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b=DbEvce0i; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=LjAvkv1k; dmarc=pass (policy=reject) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf18.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729859188; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Fh+NuR7QCmcKWxRi86ErJ9aexreyrERuoTi9h/YGAIM=; b=6gvBGcvoTGSnBzxdDT5EkACeY0KMmRiI9aK2QfcSNJyBEAKW9pYXagD60v4gtPWgC3zxU9 vvbZK3Pul3k/ZapwGLt9olSJBYUvzDu/Fpv4IdF5iYnDNlx463ImWsGOuP5M/nyF3qumH0 E+ii1hoeROXWuNWRblxzHX8LBVd9VsE= Received: from pps.filterd (m0333521.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49P8BZ8O005839; Fri, 25 Oct 2024 12:27:05 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=Fh+NuR7QCmcKWxRi86ErJ9aexreyrERuoTi9h/YGAIM=; b= DbEvce0iBzEHNApq2ebk5pLRwE3WaPP2XCGHlX92+5pvdS+u/DPK86y3TqT5rImr RSgkJ1khh9JD+f4IKa+8wCI6SOf0dlE9v/Ntt5NISF6IJIf0qV0dKvv9giUJCuvw 5eOR0ynTm/1u5AAqNl9weR6yF33Mj0W7DhTZnCiB9x4TCarbuEz6D9sQMxQ5U0bo Vdtz7n1CtHby+ccE9JYD1m5CJPNpevdOH+4m8UI8jyVvHhygbVr67wZXQJsouvvy jL7fWyjOGWF/rmZ7UuSOIdzBEM0l3XgfPTDCJ5pSvv+GBPoHNDvI6hgh83F1JJ+R 6WWRJqeVxUs4mGwochTRXw== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42c53uvtty-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:27:04 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49PBxpRm016475; Fri, 25 Oct 2024 12:27:03 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2171.outbound.protection.outlook.com [104.47.58.171]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 42emhdt4th-3 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:27:03 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wJ8eefl3dxv+QsJWWMWUAyCX07U927502Jms4JRblLUdCpDt9HRjqFbKybnGIAKOnUjysaO1L+7ultaR+6kB528+ei+Lzl/bjMzvOxM+J6loxl5IdOQ5JCaAJXjYODwyEffGgJh/xCohd4oc5uSr3xPJqts5mr2hAC+oUupKP5tsN5jkR0D6Qi743qetASwQidJMubMlFIMdyqhzfIi7+8JMMFJ2dNr45OKcAHaWa3BG9ZKjJlfAbD2DA6A/mQzE0SHhNkyYFa4+7lv90k1kRw2OjnjzbTKEAGZNd0L716lVu9cT0hKeDAYy7aJTvaRDemlPb5KbBfZhaQyE6/1mXg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Fh+NuR7QCmcKWxRi86ErJ9aexreyrERuoTi9h/YGAIM=; b=POC/br9G1MSryKkGLnAzEPTQGkTi9UuA7EAYvoBED02LxXPXIZrY2JFRjbldSMJ6KG56myMTSbYp/WC9KvmXeWrFAjFduqPkradKPDTCsGo9GSanFcWigbPDNv/9kAKlFamFVLbvbSCPHAhn23KQA6bvqafRzBiEmXoYLMgw2dDt9tNJx8ENAw305NV/LHjgFCIWeyZmamIQkL7f+c0dK4DsAFBHFJMywSsxin2LSkEYQ8i+4x+V2fSFHqEqkCk0JFPsxBleoH+jjCv89cmQ3/g1z3rQ8CvX5qNR4g6izfnximLPcxTAcAP+tS+gacGz9G13aVf7d3iwlToAjn+/JA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Fh+NuR7QCmcKWxRi86ErJ9aexreyrERuoTi9h/YGAIM=; b=LjAvkv1kRsrkEn3lKQaukdhuy1M60SIV9Wk0FmjvsCKaI3Np0ZErYujTa6T+s94832OvnMCTod69xLW7hAeYud1z3Yt3vH1MqOz9dVZphbBXLgc+hr+S+1TGxKWZ0hyXPTbDd+vrz4liO6VvBoi0TdEIxCxXcfrmZK7LRkf2D98= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by CO1PR10MB4484.namprd10.prod.outlook.com (2603:10b6:303:90::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.23; Fri, 25 Oct 2024 12:27:01 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8093.014; Fri, 25 Oct 2024 12:27:01 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH v3 4/5] mm: remove unnecessary reset state logic on merge new VMA Date: Fri, 25 Oct 2024 13:26:26 +0100 Message-ID: <5d9a59eee6498ae017cc87d89aa723de7179f75d.1729858176.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-ClientProxiedBy: LO4P123CA0384.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:18f::11) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|CO1PR10MB4484:EE_ X-MS-Office365-Filtering-Correlation-Id: 6a062c4c-1fbc-4710-1f55-08dcf4f05360 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|10070799003|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: 2djdjVV85awVdBOnSsubmgLlKUOqMrCtcJyqsnThveLdzfMFQDqEIq+Y2JH+x7JgyzkuajQCnCW1nQv7OdTZI1hEaf5d3MCU51sTyKl5I1bIMwn8qQLBr3irKNwB12sLJr7zoxO3qTf9J2UP8JKHqQlji3znOiUV8eUmIdDGcCsSFhKEmcBZrAfPfkKzHaUWnVSYgsn4i21CRCSKgXliqsGEhHkBOvV6v5as3RXfR7L0KANB+I6OxHd/sEkWWIUohBeRPbCbXXzNdF01d+VVyvQqUQhwIFKGsoYsdqcYwnijmYOYNK9nlK0C5plVx+vzNjVXI+zCEjG0I0if2alxyp1+qUPjk4GyuzMlZpx98uLyugqtaeWyKKOF0hofUDqdVLvpD+k+lAZ5fcgyPYn2YR8bcBQxArn7Jfs2+JsYveEXC6JI0A6n4sbcBClEcES1ChaupcLl3Ppo/Dxeh49EQittgOs5eCo23NRmLvLwDPJuTBhL2OjLwwawXWqmk2NQFRE2dcNloEXOLeWRgRg0Ns2OxXxrO1R4tEPBcoqD837D0QXKIFjFNZzH2SO3wdAT5lPXOQUEytlHKjEzxaVRXrk26DsCnaE9WnqR/Ru5V33WzCTmTx4FBIxhXHICa1CAUMVXfQxrKof+eTspSr9vdh84KZ6zbk7xRtN2WitcLoLVRG3kB+h08skomHl9qhUwNBClL6WRzcFda+rUDcvYjEDGaQ0U8idJd4d3bbCGqMPXAjMqRfRB2Pqa7iJ+p74BArrmO4oms5obeRH3KkdoPAbgCVfdT+Q4GnoemLZUP9lcmH1evsesUh3seQOIuAJseiWnLHAXOeNLWeHfCo/p6S7OIWcm3vEIDHYDt0WDyHkG4S66sDeo7RLzwCejG7/lwQJHK/eKyULbMbBp6ehJ9gkiiu5q/cW6dnhzrpdflPuS+PqqhDiKrEmx6BhF8tpIta9/+H6rDuU4MtJVQNn/G4/DUxwZsl85Y8/q5Hf73SachqXe/b7SHyl+NPMQnKLZuQWwFCiiI/LS+MXZJN/7scpwGasPlef9SoJM1PbRvVgOcTJJ/MWRiJKLQzdHjiXHttK6TT5cz7U2kQQzeoY2xihJZcs8OIK1ZwCrNTKf1+hXdjgsIhcBWFCQsE1uNCupahuZZPR74J69SSdWeuc0s23Vsx3HotKnvyCK57U+8I/BkpEGDH3xUGApYbjz4JoTgWHREB4WPhC2YSJwaH9C0Wgb7rvbMfKlw8duX/vUa7yGNGy7x5yLckcPrwGbk/ND4NU275lggqy6fa0wotsMJfkot8Ufk3CR0T/TrMTjZm2cA+6aw5I9+alrU783XZR5 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(10070799003)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: SspEO2vJ+EnSxTZVbjQOSyNEjfU2Tl0PyFzepbsA2ljkk4vO3hE7uL3aMza8ZRFyUpE/q3aauM8OVBLo2Y8YfR/lUG0KJolWY7+R+039kWKuxeXib+fqJUY65h62hxuAlml+OsbpH9gvge5Y0jHByPwiTm+vry1dyRGkt7s2CdFP2UuWs1cU4y7KX+yPJQXefayZ7HLtI44erwFyzE51fjE6ADH9R4z5jLD1v4mB2fYczsJFYXhxdqWT0epmuaG90AVjKke13nQUNAiq35XYHO5hcmdV9rYBCHAv5obGUWTkY370yCsghBdRTV34p5ACbqL7dJZ4e75wqyQWbcmUapCJx3dmUVvH7QK8+zzm8BowgBYMPcdCt4fUWYTvOZJ5RP2Ebzt9JwzHZ9IcW93pfPJlJ2LSsikxju16SSaHDZN62gmcjk5Ycy2m5+TVK2Cq4Bqri0W4DNxYN0Dqqz94STvxRbCY8+xcYrrYxX5gZw7riNrNHQgeeQT+he01Ozl5ES9k1XFZ1dwzMmw/z1SFGBWN2VMw0UIW5ybxhEOpopDvtNKOp/Rq8xXPdESjX1KY/fOhmysDz8dSbJ0f9VznmfMFjARBtd3iduDU8n+MGAFB2EiqljNbMj4RIG7jhP2MPDpWS+RrOeGyk3rrfJjuQ3MuGV59TKoUkvgGoGcmOtO230SUKkfxuZB36ydGGpTnseB19MsAIatB9FsVEAimWh9lbTMeW+FBhxED5EeqlhW4Fq14g17oc8WsahlMq4aRMNOCHpuBXyWD8X8+UmnuJp6p4OJ2sC5CNAE1fx1+tqXoAx95FV7zWFgOwGc4R7vsef/gCzH682cmPAIHkHGPnzFnuFwPevs3YVqMHv0Nhml3af5uNC+uD4D5Gz00TgixppmieI6Y1jrxreky6LZfDSlMFI19wdjL7hV/BbW0GPbhwV+Xn121lRucqXljxJg3+Izv4ubaxLWhlsLC/+ghsBedGYtidwzJK2lqa527E9+ab1sJNgDtqFI7Si9tW9ldLypl6246Ji98qlae1fdCFEC/VyelIeyMzG+rt3vJIHU4+oV4FHMAOSgKLSKk9CC5jDMgTMaTRf9gSaL/uPtfmmJiwTtIrBWItz5kWbnsdzXH5nDJroW/PI+3t6bgjcG/lIhrMrtNDmMDp0LbdcMeGbd1ZEWONUWD1Bi0LCFUu88fFnHjgFOC2Mqs49pl3q1VcOOLMRD2AIKPoOmlq6YnB6yS8XK65atGjPkxrC/1nArOPZaPMTib2eLL6NEAxJ7u3eJEBbK6pTCIvNp7mSjCC1LFNUtpVa/dScsX9d3VEcbJOX84sF0STM+K8H9CWt5gHVVF/kBk9M21LOc6ens56TIrmZ8YxxQaF3Z3v3mwdh46kCbaw7+lYTXcbgllhS9q3Hx4esvDGa2oqG0yDJvDXwANEbEoRPtGNsgwx96RSI+jhrMiXo2V6KZFliyc+KY7wFOa7t6lbd/1TPB8vahM1dnSq59uKehvUIAJuFnURWaCpcQoEiC0xgD+EhKFLpoGVQZmi1hDdFDngDbuQzq695G0+tp622uldAotiAab7eZKu0KlMdG/4gmExoOMPdPCTyothkeaUylBUj9P0cvqZ3jDnwwMgmOSdC+ENYHJnQL6JK0m2pRfw2crFIv6w5sArEKuuf38YzELiHaBxQ1WbA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: E1zoyNcnWpKbpu1zUXuMhQEvW8QXMM+7QoLynOLya7ylMwWYGEX0M5dSv4R0Lird0Nd0wfjciyOU9M3cRnq5AGGWslJYM5/eJzQAA8RMjyKACV6hjk461ckcDZ8cXMGelhEbA21bXWWThLvWaNiEoSIJuX9EpBUpfI74XwkukCARArgzg9OVdNkB95xpnAgjRFz4G4Ox6G5MUDZxVkTs0gBd+o2YdA8YrXhVhkUgT0/hBS2tEFpuiC04715oHRKOmTJj1Fg4qyV6zdXa6WaYFlppbSrZBZA2kX0woozUwCwIy5eo3t4Hhy/X52CABf3nWigV05xOf3WbU3QMcJVwyxOP3up2yXc77FedbzmQQGn8ONVKGoiptgcwk+Qb0USfKao45gWCeSBrPRWUvuo74gqoisv0Y21UG8HTRumC2/98Uz3D64XUTc1Quyeh04FKL0C8yQNq3xJWgwf3FVM4PZ+HgY0rJ4nkKDJRVaqxn9uC6H9LYNI0wj+XrAhTYrSbQ0c34VykxOSymR7/d85r3Ma0SMmfLEvcYKmtNprfqBQmV0hpLN5JLo2Frqr8HeUPYjO8JD4KEeV9sCwO4oxjBjOb9+sQxc2oibYERH1cQXE= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6a062c4c-1fbc-4710-1f55-08dcf4f05360 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2024 12:27:01.0607 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rSzS7/VX+MtWwK3rb60IBMc/2AjT6d83SRH7K4vO3p2ithcjFDWXIsH+jsYl9k5azdm+01uxfWbqOLdK81bFvtA+nAwiF/EldDJBMq6VM40= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4484 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-25_12,2024-10-25_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 mlxscore=0 phishscore=0 suspectscore=0 bulkscore=0 adultscore=0 malwarescore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410250095 X-Proofpoint-GUID: fTZuV6PyLCISfuKZjl2HvDXEV7SoME3v X-Proofpoint-ORIG-GUID: fTZuV6PyLCISfuKZjl2HvDXEV7SoME3v X-Rspam-User: X-Stat-Signature: 6wh6crciuk7o1aw6stpmpp4rqf5eyw5n X-Rspamd-Queue-Id: 5185A1C0008 X-Rspamd-Server: rspam02 X-HE-Tag: 1729859219-231747 X-HE-Meta: U2FsdGVkX1/YHrEK00ocbSWQNz5mXE3XFjDJQE55ko5OB0qhBeD/4TMe0Rc39B/TaZm0XpmPYbiYkSu56t4b/NZDf4Oqdah8UG9v6j6XyAMEnWXtd7cYtWOxN2rxpXWXBbzvWE4BISSZyI+xDWpTd90DGn8eQM2r1GGL3IKhs0DZlhF2QLC+POMXl9Z8/SIZcAleGHEs97ibgQw9vhm/hgQcLrDPtxQEM1VmZz+iNCbgX7po5DqlhfFK75La2pyyMOa7cFgwovy8kXKCI8GDQcuD3GP3x6l8iErEwcCo+noCOkJzQIoWs4CaRRXwQqUK13EGo5el7weDCeSwZH3cYqihEijJO4H+vXOoI3URLytS4AkUc+UizkFFXuYTon4UafutAxrfq0DqgFAuiw4BZj85fmDrcQHIa3BO97os7BplzK5oR2pHnM28FG7FyuK2MbMzCKiv32IgH/039cQ1gU7MdI4a9DTiQsvHqNSakAQu7yedIoEOgUc/xLEpfK8iadNJCHdo/hJK+tTM5JU18PRBF9qeIBDWeibDmk4oc+NKfzRzNLi2rO9BX1g3qYhnyUIOk8rOxCq10yB7Nx/3LKUdIZlbPOnc2HqEz3TtTmW1k6dIrLQa27nxZCy7tRBY6xBO38qc5mQ3L5j+xKSaRlloDcmlFI2RKDR1Fvuu2kfv1qa9KKt4jPMUCvlhyE50ZcOnJ5+unPmCYfNveRvQqxKnXiHlw08Z/LQHxL5w2FSUIHTCy6BELEvQNcW7Ug4rgLxqHITmDBgvwygVWoNHo1CwoKfW9l5oe+lzujpEkHJ9qvBIxNGBxSimBROg4M37dnpAP1sHXqMOcI+XVBcfbwafqGtydmHtN3ed4j8hR/WrQu2m2GvnjP8s3sgA1OW/UfX0epWaBLE6+KA6+MlvbshxlKaEBGZ6HxMQD2e7O9UGmEGl+1mrGpv1SS6QU1zYUq+f82+X3ak2VWanBvE 53LLakvT D+osg2bhaDai6zJ6jvLOk39codEwWxNlJbgl+9X8XtLV+qjidS8obG9qTFktr3cz44adrZ6G1YLPGYTEOxNMJeNTja2rPFmoOSRck5m5tWGS8MtJOBC4FylpDxenc1xlO/j/nacvU5kMCNmn0jVGZWbdqK08U2hzCumQC1wOfY/wuY9R3Kt24C6lZzHJ/7sMBOCxWYQp3rmD9sr4CvIrMpsbFzTZSu+7VLelWSJsHlGWS++/ZH8oz4KPmzj+6UDntE9k8CKqsPat5aEr4gG3xoG+E4YOx/w4tbuqAmICuTBzhwzJ95YV+IbQKdC0YHflfqVPy64rktJUaPpDn0VuppsmLsUxvT50YU7O2YFW2wZgSJBZKT5TdWOQd1hdNF7dHuQI6Ta4SP166ik9z0gd/FilW/S+xbBqK6ig0nrTr/ybPxMRHLV8Jr2wKyCZcacyfU/c4X/P3c/AD73LrhJeBtQ2Sg24OcxPK6IcQI+P8xNk/V6Rijo4HfBCX7mugOjnJIKYIzQswrZ4STM4= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The only place where this was used was in mmap_region(), which we have now adjusted to not require this to be performed (we reset ourselves in effect). It also created a dangerous assumption that VMG state could be safely reused after a merge, at which point it may have been mutated in unexpected ways, leading to subtle bugs. Note that it was discovered by Wei Yang that there was also an error in this code - we are comparing vmg->vma with prev after setting it to NULL. This however had no impact, as we previously reset VMA iterator state before attempting merge again, but it was useless effort. In any case, this patch removes all of the logic so also eliminates this wasted effort. Signed-off-by: Lorenzo Stoakes Reviewed-by: Vlastimil Babka --- mm/vma.c | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/mm/vma.c b/mm/vma.c index b91c947babd6..7c690be67910 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -963,7 +963,6 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg) struct vm_area_struct *next = vmg->next; unsigned long start = vmg->start; unsigned long end = vmg->end; - pgoff_t pgoff = vmg->pgoff; pgoff_t pglen = PHYS_PFN(end - start); bool can_merge_left, can_merge_right; bool just_expand = vmg->merge_flags & VMG_FLAG_JUST_EXPAND; @@ -1020,16 +1019,6 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg) return vmg->vma; } - /* If expansion failed, reset state. Allows us to retry merge later. */ - if (!just_expand) { - vmg->vma = NULL; - vmg->start = start; - vmg->end = end; - vmg->pgoff = pgoff; - if (vmg->vma == prev) - vma_iter_set(vmg->vmi, start); - } - return NULL; } From patchwork Fri Oct 25 12:26:27 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13850606 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8EE5FD0C601 for ; Fri, 25 Oct 2024 12:27:18 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9F2D26B0096; Fri, 25 Oct 2024 08:27:14 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 9556B6B0098; Fri, 25 Oct 2024 08:27:14 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 782206B0099; Fri, 25 Oct 2024 08:27:14 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 54A966B0096 for ; Fri, 25 Oct 2024 08:27:14 -0400 (EDT) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id A4C0EA0981 for ; Fri, 25 Oct 2024 12:26:38 +0000 (UTC) X-FDA: 82712048286.20.E5DE854 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf20.hostedemail.com (Postfix) with ESMTP id 8C5021C001F for ; Fri, 25 Oct 2024 12:26:46 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b="NU6k//XM"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=d+nTQH8n; dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf20.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1729859178; a=rsa-sha256; cv=pass; b=6GVYsjXjkawf/NTDoeYWZgvCBfIerx3Fq4W0PpCbuCEbUUe3deuVWYKqsFdtcAmINM/DPz EYUEgzS8QiaF3SuAe9jd5d3gVqkz2TpvyKFNWYlNxP/wmeedNLMziyn/1BfC2undR5/boM hBgd0KW2oZ9Nu4oivHL4gD4Yp/K/d9w= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-11-20 header.b="NU6k//XM"; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=d+nTQH8n; dmarc=pass (policy=reject) header.from=oracle.com; spf=pass (imf20.hostedemail.com: domain of lorenzo.stoakes@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=lorenzo.stoakes@oracle.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1729859178; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=rzah8PtqmSFAEkrgx3+NN4GKeumy7rZse5pxRqGcMM4=; b=fgXlzSiN/CD6DrZX1vnkGxNLAwzC8JRQmns9b0J3IA96R2rG0aIN1g7ezUhmTg62/3lyrz veZXIipBxQLIahnwaS8ml6WR7andg/QNQrjpWM+X6tlB58Lshuwb8tVVQbMBG5pdmfv/v0 Q2pOifeIqmnD7/PJLW8eazZqrlvgUu0= Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 49P8BZgo026805; Fri, 25 Oct 2024 12:27:08 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=rzah8PtqmSFAEkrgx3+NN4GKeumy7rZse5pxRqGcMM4=; b= NU6k//XMb+TkwpLqE9l4ORvz7oc7dhH3bAjyRGPv3vkABROj6uuEc5FOm+bCVL+Q BjAL0ZtqqSmMAvC/RaX+jrbasy4W43/4saqpAInPOtDIg+5c3lM+S565yHkUG/Dq aNi0Iu6ahUQvYC8m9QuaUxFJzYXJ5Jflid494/zQIpfFuPAdnnmHS+wUsepmNHu4 jqEYRyy/S1R2vL4G3rs1wsAJcadseIDKYsWc5pr3gcB9k8kGvILPvKr7EnhydHKf SqMmAuviir44DaeAvqmjMEEj6x02mdfHQGM2oHdsrh6JmrtmJci/mOYQ2riAVPPm DiqeTNOeRJrX6+wrtpqCPA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 42c55v4u6e-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:27:07 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 49PC7U62015833; Fri, 25 Oct 2024 12:27:07 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2169.outbound.protection.outlook.com [104.47.58.169]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 42emhc7tsq-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 25 Oct 2024 12:27:06 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vuN4Kvecfo0MJjCubI5yCv+oQUhA0uNs8rSng/QWq2n9dxlvU9Y6vOhTlN9ysAUmh1a0ep35JIZU1PxlQeneb8wBLzsvk3TH0LrXdsmTXU6UsB/O9/eSXKJs9kpkxunP7ZOnYgUjiPGgLDrLiBK23QCN6GbwQ8K8ZW94xO9Zwqjmb3xMOpxAqRa6ILpS5fSL4GfSoJqPDURZWvLV7V81+J5dcabitzuoUe8SLPye0KzIb/fuBTHtFS4pxkRMfsQ4LmF+cFWv3XTMvZM78Q7uTw1apCC1rKd2M/gqGTqdMZXqTIYHQOvE+n4acEQ49cKTdsLMtTdTcJda5grbIk6uBg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rzah8PtqmSFAEkrgx3+NN4GKeumy7rZse5pxRqGcMM4=; b=D6oKmgNf/UXzpL/dXz+9f7KWHoxZYgb2JutVdRM6s/N20GNqwRZlNbSL9ns4TYsWRRxG+WIjXXhIOCsst4/dBl97adfdXL8eVidv+tV36kQPuC2Gh5nDdHA8JXFIikklgHDpYTu8K7BzmL2JOljvMbn0gLQTH0HQnbdk+gwNKdmydwjikV5uv+/flMDY0M7FfPIo4UTWljvAK+Y36tDexMzz2sg1oEi+T/jSAyZ/gMXUDTdN5nppmjLdwlqNF84mzEmWX2JZpEs+665TA2lKMRVBlz31aFpzPt0Z2/6f7LKE+VVfpK7MrDTIxpEITlMF/6FU9eULXUhsjaI96TCi4A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rzah8PtqmSFAEkrgx3+NN4GKeumy7rZse5pxRqGcMM4=; b=d+nTQH8n3jY/vlBTBqStNRSTTmeuMf1NZfjX/CTB+Jf/SJNr4ZxQW1d7xV+xNsabH0t7wGa1lA0o8J1NQ2j0EMAXv7b6w3YH1Yhjle96Xhx80Js4gDOLE1SXHDFBf+2V+fZZFlrMjdMgVnqyorh4iuQuag7N6fjOBS7A+moWOqg= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by CO1PR10MB4484.namprd10.prod.outlook.com (2603:10b6:303:90::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8093.23; Fri, 25 Oct 2024 12:27:04 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%6]) with mapi id 15.20.8093.014; Fri, 25 Oct 2024 12:27:04 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , linux-kernel@vger.kernel.org, linux-mm@kvack.org, Linus Torvalds , Peter Xu Subject: [PATCH v3 5/5] mm: defer second attempt at merge on mmap() Date: Fri, 25 Oct 2024 13:26:27 +0100 Message-ID: X-Mailer: git-send-email 2.47.0 In-Reply-To: References: X-ClientProxiedBy: LO4P265CA0175.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:312::14) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|CO1PR10MB4484:EE_ X-MS-Office365-Filtering-Correlation-Id: 8dfb44ee-6443-4906-9909-08dcf4f05532 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|10070799003|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: qYJuqreerQF9eUS63UnHKlkXtcJOCi7u7FFhi0Fretog8trb9ejn+9RZi3rcs3ebrojTEX698SSIuxzXJ4bMC7XzvyLr02xg77mEVYjmGohgyrNAijFovBExYdiXVTWg3cDi11OVVj96q40Ib/uagCgwaUjmL5V/ajFJDBC8oT26XL/rf/+KdLpA3ekKjyONPSk2IZsEw445LoS3iMyEdnm+Og/mX/h2p4E5DAnQpLLKMl+RGtFrPsug5nY5Tqr4tkKKFDKt7ifUtn/jkIx+Z4dFY7Ndn+KHfK1cE+wrakhzERIXAOfq1e70Kx/PIbbZFeFZoxJhNw9M2s3k7WAHY52bpsvI9rkoR7d3MXkKuFZoIGax+weg6c/eEDT+f7s/bxxI1Y1FArAmNXMw7RhC0d4zNr2tu/Ore7qZBB7Vw1kd6hUjbULMwqdil8POo+R6fgNITU+aOcNM35LutC6uk5gzkZ5T1Ne7SiZc2/oL9zVL0469Hj6LnhCBR3Zys2lVH3sj0ysdWFcGUgLQ3MW4Et9ZTkIS6v5Fa0prkwqTeq2CovyA7a7mDrY5IZpcgj3nWDzNroE67bPxm9d0Gk9OD+Jftwg5OwIV40OK4y26TgGTTJogo2UWwS6vwypwyiNomZOy6iAHKnTvKEnEiwK0tvMzei63HmwbtSg4w3TMgkBQRwU5GMa+rHOo1OqQogCPrx2TtScbWKsfczZRCl0+3ijgc98Hc6os0XKEAcLNG4phXB4MrrQJLhtXsFDKMH4vorFG1/+LCvZbxjGhGXucILYb3Y+DAbYXOzV2sYur69B6c4BqA5pcByyq8dpgDXHhhfkPWJSJ1I1YE4PM3Bg/GQQrgapfV6KK2DAI9FXftN3sbhWTpou1w7NVptndhwj1KotW83bblAMBFsohOY1Y5kpKHjtgb06MRBgPmLs2QQhUkjqnar2oJaXdgQscbHA5Ub7hLAMgeqT+kGQAluGT0eV+ckFKZJt0A3TDaKtV9JpXapspcXzKFcxTSLvKPEHnM5zFJkIw2/R9VpR6evCli/lhRrmOWPgkF8rt+Ltmd+IPRGD0ocE51UNfIeQ0hR1rUGg5+OFJj8uq92Wc6VxWD0A/QzwgxU0X61PO/c4pbGMqhA2TDjWMe2IUf+qZYAsdKKsDoBTR76air6Mt5MlbdYzpSS6j71tfGfit1JlrC6vvuE8RfVwgQJVg6fEcoMV89VvqLdjRxdQbV9lYZdt/O43uqq6HUf4I78W7XoDPAMyAtHmhnETFMwaOqhX03Ok5seOhQJL2QxwybAoE/JMH9lJ4Hv8iJjHSf6F//Sq2Je0= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(10070799003)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: +3uyN8lVUn8n+fz75PQhfHl7jSyB3C5kZoHB+3RdtHTZ/J7yLROjn8AJ9Qikf4J4KVVHeABFXE8bYEuPPC0XpI8jW9IrQXaeeYV2/I8MFRnzpYUCWa3n3et+C3Lzk6OIHzMhLjBwz/KS9ZhBqC/wqzUjQsKEQ4BlsATMz3aDE0FvGg4UQn2aZVLPoxKKI1PnWXGw66DGvYycwBcTIkzot76AvrHZ4cxq9AVZMVD82/tadtfDqFxcGH9VeSzOq8G3/GeUz7kyEAJkvNRICq00vjnG82XMpkM3ArGVvw8OgeD0d2RU9peIBNPA0RESnXoKWXkbVZhl5YBfYOGIP8OaBa7rX9u/5x0dwmL/C27CpW/6JGxe5P3PWP5oiNRmXUn0snNCArAc58PmTyK4IeKZQH7zyXKP2IpmgONZz8m+DRpVRB7zpw5PsVSRydR0O1pFTY2GOENHLdtV4SyGlOCmU7HI5ypgosswxAA+sjBE1RdN1/v2kUmGqV+Gci4X7GKRFoC5JHe/6FvKBr60aBOQ3Q4B79+8Sq7xIzoalKgUFrHFkR6qf1QmP/uLXRIty+oFN6eap1u70otqmV9dMxsolXCfjSgHaSluY73EFkxqs6jZ3QMKg7XPVuFl11zVtCqkRxrCKddLcsw9viQ3XqeMoiG6q7ukym8Kr3pXxP+pl3YP5+Wc/2nYSgW+IwuprsdU/E/SniQwneK/Rl6kxkGEvoSdBUYZbC4hm/KjaKUJ02dQgmM+pmKv3emjF2LPIIOwBuMPS3YZ9WyMt1NUVZiBDjgbRjAkhDrouMgFWsxKLTsdMp9PoUyAICuUp3E9Hzko4yTmRLzp6Z6U9yXixG5xs6ujP0KypKpbRiJc7yK1oDgQybj+2k/9pAY6bhYDHShTpIo0qD2Rq8tY9PKy7KWyN0D+tMHFnrNkRewEQRJO5DAOyrhlAEDOYBFuc1rRcSi7pTUxc639M8X1/tPS/d6YKOvQTNjSgHuBgRV/g8Tl30rzEHvhWVZn3kvITpA2cFH5XAwMBH3dZMaHaNbSwW+393qPNrz3KKRYtKFKN+7gUdI8prlM3n6tqpRA5GzVDYopR1fI4JtFeXynOj4Z5XTris0FkRfDpf4xX6hZkEdT6h+B5PU8cDgkxoTtZpeQmV5iwVB3oRq6LE9LiHGL+4ZCv7TYh+jyOCHHtzV6b5B/QI9ufn0WRb//TdhXtr3wdCmTBNI5WpQ9ylf58RNc4hXmNCT9PLfJm4b/OZSesi37vIu0v2QZpy2X0squutTqoW2du/odpLyloU33vo2SWMWioh3cXtdHjIv/zXInr5tapB8W7UCSaOAcVJD+WEkcyC0X8Ei8oLpEMHT7VCepXFDyvG+FwrOPiPpFJKzs7YWjLvTaFKprgWcngCJMqeCQE03h7o5EdSSyUXRjiMNiAF7WvuBFTA/fiS05Buo5T4PgSR3NJZqIpEs+JjmBZ6JPEzHVtsQHpAS/fZm7kd77hLp6GCjCZ3Q9pAPffGgTqpOLoudDFJ7W3DLmA9CZzOX/rCbPaFDnXqaEAN5p1h0q6MtxuyaRaaKQGy1Emepk6QIYLzPqVFtGW0Mf3/gOgDOpkuZ2VU77ahxEAfcQuL1JGT58Sz9EwkuRbNIQQWHCsa4yfB/t4WL1gPO1T89tf7aFFks7n5O9NqttqpRXwcDEZfQCfQ== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: lJiofi6nJtiajkXLDa0h3aGGlb+B5ShIJtcwI4UZes4KDXME9RLlwv5IXgVuXWH1FPFMx1KNs/bIO7E5zpsYajEbImvnJegmDGlImaIxvQArkv91EY6i3n5cTm1BSwOH3uFiewVMBT6XLm1YeU20cPnYFzkIUR1lPYp/Q4eNomAJMPQJQBvD10v3AuWsS1xGhkB8VIBZg2TDw7qnirOLqyC3qkiwfFh0r3rMAVTiJK2Hd2pE4/EbMFCCn6Cwou9KFlch5F7Dvq4cZb4hHzVO5Y0yB/XA7Q7bIbeg15riugzsCAskFB3muLYi/o7Y3jUi1eEb2PRD/7HBE4QjBE4WX4hGinuqnF1zzbwE74rBMRFIeFe5qhmJmWS34lh2gbbyE9jGXrzLFdp1ojo3QHQSm4rR31VmcItl9SGujJ975KiiY2HXj6h3Fa1CiievieAvyguKnojhNF5LPzZj4EG6MCc450Jk6LhAJ2WGQiLnE78vV4U7QSNqGkIarsPP91S7G3k5Eji9b4TsVWVCo420RwTIl9iLU+SEzcqUbS3vjJEIDAUtpaYD9T7KzMpttIytad7TIokqo11yqu2CygB6Z+vStonB2mgqgJQ3a+2DFoY= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8dfb44ee-6443-4906-9909-08dcf4f05532 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Oct 2024 12:27:04.0639 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Qnhg7qMjYjckbdbujNUcTYaZxDF9TdxeEieuYAb6NZNm74nj0sdSFbeUk0EUoSxrEu5+uEkGPUiCk0SkCoWUO8nMk+djPbYfJ3VDThCVrXo= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4484 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1051,Hydra:6.0.680,FMLib:17.12.62.30 definitions=2024-10-25_12,2024-10-25_02,2024-09-30_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 spamscore=0 adultscore=0 suspectscore=0 mlxscore=0 bulkscore=0 mlxlogscore=999 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2409260000 definitions=main-2410250095 X-Proofpoint-GUID: 4846HGWgGTtz7mQcL5vJE8pt_jl0mp-L X-Proofpoint-ORIG-GUID: 4846HGWgGTtz7mQcL5vJE8pt_jl0mp-L X-Rspam-User: X-Rspamd-Queue-Id: 8C5021C001F X-Rspamd-Server: rspam01 X-Stat-Signature: cz3uc1ik4335dj9wcx1wg5qhkfuakg5x X-HE-Tag: 1729859206-983771 X-HE-Meta: U2FsdGVkX197UMQsgTJ4yyRQGv3iRkabAIAHjkWKk//EyBX1zywIzBXFiOoS3yFjbeoJXOltVWjC8B/FMisxMAQhZ5+H9GMJq0hbZMrqN4igKzA4iT44cHNEvWOUflvFfU9EtywDLwujDzkE2INghRdVC8w0NTSDztkowyTm+AoBlYgpnQv0eCMHwXU37jjR5Hmub8iF70d2TNCclspOuqH6ZcuaJix3/I56qLqmgrcXLtvupNGOtjX6xKNeu8OzRj625o1Zsui4DEvIRpJ6RvciuWfNOouw85466XOj8ojuae4nn2coI91J0vnj6rsbOyAxAgM72QGYjdtLUMXdPs1vYzai0uWuVrd45TbCkKiMCMgVHRnH378Gd5TNuMiEeBa0kXBJrc/O49hzwkRfaaRa4DFOoO3LUTgbKTltXqCDvpR95F+grBcJqQjMxY0nAckmJgOmsPCAPzZyoaok1GB8nv7i/hO3HEe5zjKWq28ehZEXExLrr2YhXlinugCKxkAahRT2R5ZMiVKHCo4PNiD9v+rMU7mGkuym17v51BpY921M0Ffqdvg4Pe1MeoR7rS1byWVy3709gA0pBYUxrdDT4MrAi7gcjVrecFT4d3OvOjmKsSf791FYPoDYhdUSNIp3Ys3OcKNiKkWq93YY+SAaoe0XsCoseo52ePui8A987gk4ZmTczijMgUY5cn+sT6cE77/W3rPYhQRrYZcjzUpAgClCYup8QY3HKaa2PNNCMkUZ+wA+lIIVHHDUTj9HMJnLiL9KWAV178G/4Bh9HCBrMNDW7TucUToxG90vdFjUBtPl6q5WU+xvbNYPj5Kw0F/0gDNzAU1guIyZMD1+77OytAWKCMoAvqBKadj43e0xdkzGza/TnCNXh0uU3wJm14Nx48726ycNRUvWdmUfxH+YQMQsjtFUgDGcJGbTCBMgRuZo1Z23+EP7JAw7dHY2pN/8nM3oqi1PPuu4/Wh PzKROzVP /m/2E5aREPs029l02pj5i+9npnp8aW8Cy12fAHaaiXV1++YPKgKZkQYo0qfRBTyBFuCe7ExB8ZXjy7XeeSLeqscRicHMns1akrShcCTWI6NKQlwcDGbj5jir+DDib3B4DLwxQ1nDHSrzQIhUVdjECBKrcTfLIXYAu/M+fqN0ptu9uxchnv4rgMj+nDh6s1KP/rQFUcdbe2xiz4wWyu65O4Nz0m3R7PaKBTjyCpIMddJsO5kS0PL9UwLY/OmI46TYhcmdUEQu8Ve7koTv6XKSl/BvpauAWlm0QfZtydxZPSuHdgwiHergAl5jVJLw7AulGh3S3n6XIZ6PWNYtiAWu9OB3Tm9FJJWdiTVbUSoCchRdkaTI4WMJKaKSV6FuSxh57CWs3Z4uWKmfBud/h9SKuslkowQkwAgH3IgeuToDo/OO0OqgKi/j/aqpLgYh/2yr4S6i/EuAU6rR/CSWUjrTp8IQrXG48b+w8wcbTvgwe3f+MAjULCsXZ3EphOtgXz96eQk9iETiS95sbbPI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Rather than trying to merge again when ostensibly allocating a new VMA, instead defer until the VMA is added and attempt to merge the existing range. This way we have no complicated unwinding logic midway through the process of mapping the VMA. In addition this removes limitations on the VMA not being able to be the first in the virtual memory address space which was previously implicitly required. In theory, for this very same reason, we should unconditionally attempt merge here, however this is likely to have a performance impact so it is better to avoid this given the unlikely outcome of a merge. Reviewed-by: Vlastimil Babka Signed-off-by: Lorenzo Stoakes --- mm/vma.c | 55 ++++++++++++++----------------------------------------- 1 file changed, 14 insertions(+), 41 deletions(-) -- 2.47.0 diff --git a/mm/vma.c b/mm/vma.c index 7c690be67910..7194f9449c60 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -19,6 +19,7 @@ struct mmap_state { struct file *file; unsigned long charged; + bool retry_merge; struct vm_area_struct *prev; struct vm_area_struct *next; @@ -2278,8 +2279,9 @@ static int __mmap_prepare(struct mmap_state *map, struct list_head *uf) return 0; } + static int __mmap_new_file_vma(struct mmap_state *map, - struct vm_area_struct **vmap, bool *mergedp) + struct vm_area_struct **vmap) { struct vma_iterator *vmi = map->vmi; struct vm_area_struct *vma = *vmap; @@ -2308,37 +2310,10 @@ static int __mmap_new_file_vma(struct mmap_state *map, !(map->flags & VM_MAYWRITE) && (vma->vm_flags & VM_MAYWRITE)); - /* mmap_file() might have changed VMA flags. */ + /* If the flags change (and are mergeable), let's retry later. */ + map->retry_merge = vma->vm_flags != map->flags && !(vma->vm_flags & VM_SPECIAL); map->flags = vma->vm_flags; - vma_iter_config(vmi, map->addr, map->end); - /* - * If flags changed after mmap_file(), we should try merge - * vma again as we may succeed this time. - */ - if (unlikely(map->flags != vma->vm_flags && map->prev)) { - struct vm_area_struct *merge; - VMG_MMAP_STATE(vmg, map, /* vma = */ NULL); - - merge = vma_merge_new_range(&vmg); - if (merge) { - /* - * ->mmap() can change vma->vm_file and fput - * the original file. So fput the vma->vm_file - * here or we would add an extra fput for file - * and cause general protection fault - * ultimately. - */ - fput(vma->vm_file); - vm_area_free(vma); - vma = merge; - *mergedp = true; - } else { - vma_iter_config(vmi, map->addr, map->end); - } - } - - *vmap = vma; return 0; } @@ -2346,10 +2321,6 @@ static int __mmap_new_file_vma(struct mmap_state *map, * __mmap_new_vma() - Allocate a new VMA for the region, as merging was not * possible. * - * An exception to this is if the mapping is file-backed, and the underlying - * driver changes the VMA flags, permitting a subsequent merge of the VMA, in - * which case the returned VMA is one that was merged on a second attempt. - * * @map: Mapping state. * @vmap: Output pointer for the new VMA. * @@ -2359,7 +2330,6 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap) { struct vma_iterator *vmi = map->vmi; int error = 0; - bool merged = false; struct vm_area_struct *vma; /* @@ -2382,7 +2352,7 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap) } if (map->file) - error = __mmap_new_file_vma(map, &vma, &merged); + error = __mmap_new_file_vma(map, &vma); else if (map->flags & VM_SHARED) error = shmem_zero_setup(vma); else @@ -2391,9 +2361,6 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap) if (error) goto free_iter_vma; - if (merged) - goto file_expanded; - #ifdef CONFIG_SPARC64 /* TODO: Fix SPARC ADI! */ WARN_ON_ONCE(!arch_validate_flags(map->flags)); @@ -2410,8 +2377,6 @@ static int __mmap_new_vma(struct mmap_state *map, struct vm_area_struct **vmap) * call covers the non-merge case. */ khugepaged_enter_vma(vma, map->flags); - -file_expanded: ksm_add_vma(vma); *vmap = vma; return 0; @@ -2493,6 +2458,14 @@ unsigned long __mmap_region(struct file *file, unsigned long addr, goto unacct_error; } + /* If flags changed, we might be able to merge, so try again. */ + if (map.retry_merge) { + VMG_MMAP_STATE(vmg, &map, vma); + + vma_iter_config(map.vmi, map.addr, map.end); + vma_merge_existing_range(&vmg); + } + __mmap_complete(&map, vma); return addr;