From patchwork Tue Dec 3 18:05:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13892788 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 0B4AE1FC7E3; Tue, 3 Dec 2024 18:05:47 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.165.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249149; cv=fail; b=JNI1uUWtZb3JvACgrLRuW982Ymf2p1k1OZfaL+gHDcnOjSLScKe5KOYHTjk04ZdpuNgc4zWKIaCr9/HjcOYoiUclT9VCXZrXl3GwhxjHteuCmuLyT9KO1oy5PzLQ3U3lJjd3efV1Eh7yM9NmQH025IE4Hu10DpAO5mCbzTqOPd4= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249149; c=relaxed/simple; bh=cDNI6jzu43bDmxceP/M9BMewURl3fetf1kpz4IuYgno=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Z6stuVe6tfGJMZyeKmQfVQLY7URlldMs3uv0BaUWeyO7ybzsxLrH0doQmmVeUPuWiBJMoq/bzX8Lu1/azo6RAjcbYrAo6M4wmJWnyHeR1IFxx4hIViPZyiURs3621YmXkAfyQ3vGE+HRmEfCOUVVGVg177USqzs7u+lx4P6Kncg= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=KqjjE1I+; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=k0qHHnBd; arc=fail smtp.client-ip=205.220.165.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="KqjjE1I+"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="k0qHHnBd" Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3HtbW2015938; Tue, 3 Dec 2024 18:05:36 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=hltgmps9iDSMuG0QlMXpwxtDDGFu6Wuu6TFqLL2mhzc=; b= KqjjE1I+iBMm3MBAxDP9+95DGpR8w4fNjU4jGfYQBj1ZQJunmVHj96yHZDs5p9MW wKXOgbhP49rvIoLaLxv+UKZfSK0WyNRMFY/lnXUMudzvaGcpSJY7o/hoVpstoXYC AqHvuAfjQoMttBye15vT22epqlNX10oMolxWVMSSX0+2if2/GB1D+WGp1hF12ZD7 684j6t4scwTSY6YnoLdU2A+lC/BK/t2Qzd8pvGSAJA17bF5lMIjnjRTyri674v19 C9METtM/XZznMGPkvCAZAAh891YzqnJhiFE2mAB6Pw15F3WMqInIlPwoN9VPTFUH UDzbTk/bOKg+9qPzzMBNxQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437tk8xgr8-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:35 +0000 (GMT) Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3GpVdO038066; Tue, 3 Dec 2024 18:05:34 GMT Received: from nam11-co1-obe.outbound.protection.outlook.com (mail-co1nam11lp2168.outbound.protection.outlook.com [104.47.56.168]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 437s58r9fe-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:34 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Ge8K4GwnyIedgXbdNMLIdwZrSZJiq3b/xvQEkTZDHJV4OUET1O8X+KYr92HM+BvQQkw+PM2m0Oa8X4CcqloTtsiV0dg4ECXUoFKygIW836v+XJJp+EzM1K95dV7Wb/ZL/+4T/RhJhfgCCcWQ+1FZFyWyn0gIBY5W2NLG5mkqWtiywEhaSwdaEwCKJr/JUx5wQdtTdXXrHdF+CLvSVxMtMUb9DiyoHO7vWc+mMf4Aew8GD1338IGyFrik0mQGyR2ZCbWXZCM0UrTxYV+s6KJ4hURg8z869v/cA9WMp1NIjtvFnAU5DIM7+lVdLJte4DudcDKk9RLW/FLLLL6IxUDe6A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=hltgmps9iDSMuG0QlMXpwxtDDGFu6Wuu6TFqLL2mhzc=; b=suTlv0fKOaZ/MO4272OWvZZmO68krdTqyR64MZHmGqXuC1ZBKxxcEaQJXrlKyU4fhjYBamuRI80EQsZi/+MCPeMcZsBq7YhhnMiI6Gf7dhIl6t6bI7Grod+n82kD/V6JTh+Ozf3b4mvgVo39E8p+OkkjVzZsyd3ReTjaJ1gh5TgE7Js2zgHqEfYYVclc8f6JI0ObD01Wp3dkTk6Z41Vzb7EA3p+zJjyfHzbnVOteR86GUpUMcIWruQHJu+8laPmrFHtRdBKQ1CG931AUe/TQlGuRK0M5F0ss2HW8e4Pgw+v14Kfcq2zmcZQ3VqV6AMYaATgae5HDAJRBryy9w7XL0g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=hltgmps9iDSMuG0QlMXpwxtDDGFu6Wuu6TFqLL2mhzc=; b=k0qHHnBdNuA7ckmrnjJ1sSlZh+nZaSp16vO7kQwqSCX7UPOMp8G+ULvY9J77P/LHBabZ6HFm7BDOj30v/p/NnfULFZ9N0oBog+Lwx9RIBT1EXPtBXrmzYlSY6bft4YYgq4RdYssFaMiQIyl8EzWHvsG64z+ZAvYgcyCQomI8iwk= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by CH3PR10MB6762.namprd10.prod.outlook.com (2603:10b6:610:149::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.19; Tue, 3 Dec 2024 18:05:30 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%7]) with mapi id 15.20.8207.017; Tue, 3 Dec 2024 18:05:30 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , Eric Biederman , Kees Cook , Alexander Viro , Christian Brauner , Jan Kara , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 1/5] mm/vma: move brk() internals to mm/vma.c Date: Tue, 3 Dec 2024 18:05:08 +0000 Message-ID: <3d24b9e67bb0261539ca921d1188a10a1b4d4357.1733248985.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: X-ClientProxiedBy: LO4P265CA0159.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2c7::9) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|CH3PR10MB6762:EE_ X-MS-Office365-Filtering-Correlation-Id: b3867e69-d84c-4850-7013-08dd13c512a1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: gc1vywAMQIKtME34s60yjV5nDZdeCSaVbQ3mV3iQ1qoND6oanXTsFqk1pXwaIi/t4kROKHJvBVKRVUkCQRYKILJsw53+S9Sh5xKJjeH3Mj2t2TtOI3UDpI8ZoNiGPRoBtwiyr098yJfZf1wtgLjZ7NcJz7kPqgK28XWNZzKy/yvOdIVgfR/8to/3VuVsbezY+7v01I/FC/GFMKP7ryIJkHPhZKd5nwCVlfHHA6n+zUPtThXevIF1yz5YKSzuL3sSFLaPyfc/skGFV1IPDQf1bK1XV/gNU/YMvnqEjyG8ZR1Nyt9pRgJAX/Kfjw9gfk0j/1PDm2hVfY5L+F3lc9r4GEPz8ybPQfyU2tSjQ61mwqYPSmU321AY/zW/HFY9cqV7zbQMCLqCdxmwPl412SF3pH03UljzFf/sJ2AkaayGmZkIRWmcYUwNiTscDluecRyTfdIyK3v4Pyhf4Wyo/FP1MoP0lYZR5ZQduRItT36UwsIqrRBOQYU5D+wq4jDj6LwnExwAfIs8swySd3ct2B8/6Tv7mS296kjJ6jDb8gzcL9AsfbSEvs3j92tdxgZkkfB3onX3veIWXuP81nwO69/jjS3cAwCeg71Kqym0ZdEBCp3Q2jR4vPjFvs3zQn/P1OtNjh2y5lQa6ulrwzCECL6fEvBkp6mTZLI1hzz+1vDakxxOS914t1rfIxYj4lrPNfw8xxC8qtMK6p/Zf+osnc/3Rkxt7DX+SQFR7fCr/0CIM98h8bC+oP4IVYfVlFUYk8tnn4ckI4/uf6wLMlQ4x8rfFgtTD8SWwLwvXSnzMf1JuSRqj+mMqiruWQxLRXy1tX4ez5h54TUH6BPxN9Gtot9OWwlyOrvPcyiCbQfQo9W66LJVRl0YYw9PPHqfl/s16fF8HtU/dnPXpQfbr67praiI4vnGGCx2rx3ujnMdZNxHQppjFvbIKA+tp/mO80Hj+jjO6JB7JaD/0MXSmxsk171Ye8wd0DD2hnjzAQdKf3ImXMWbWtLRL2UXJtids21tOUvn4XwilAIaHf4mRw6qDQfZ2NOrMZG7w+6/tzhFwHeb8ea+4Uu3YjWklh5kkv/5xKq9fqPfJiErHFkOl9LcVSxm41pPO9IXaXdQfrOpgAm0ElehldfdxJNCfwZQ6YvrNNPbuBDLGlYR4dDafl+1NpMgxu9O8pYEgsGlwaXXmdzmpUt1LGMVPxfaP15kRKXqR9YNC/fxI7Lu6f7YPXG7tXbVPJml7wuhDR6ZZ+JRC9krLbnDTvl0a38cdEJZH2dta3cO+VlrNxLe500DslSFGtSnxrFTP3C702oMcq+3wUkmOaoP1cUX6yDlAKQhjn5NzBLRzMhSWlC5+BQxObqlTf1J7w== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 8oIn+51rhXvTkqTqqnCo+YnkF7Sw5ZOoah65KKLN8irZMqwAICHe76nQq1f+kEp/05MovFQcuPaJAimAh1TLkJz70V1/UL/l5kIfFScHN+TTvy/IYe1Uiuqv7w/lqpZ/Y7mDuFC6pNG06u+iB0Z9Uu8P84k3C895itlrsw9uwNAaG+HtM/QxNZouaXu+CD6QEJGlcx6zHC9F5cSxL11HnpL5WGSao+JOWLkZAFCe3HsBSq5XW2pTEsrEGLClUqyEyhfFCVaE9Yp/8Ab9B4bDjDLspK7fmkTN0V3HrboO6/B/qiEvKM+d/xuUSVK09X+2f7xcc49YjghjUZvKNcuNUr9VqH174PKfE+f8YSt2ifP217rtML2RvcXnCHcaCSmB/Hrl84EoSk4IxAx7OWwlFbthlAtqVwGOG/ic24A3FphAat4F1HI0TM24kqBnifZryDjkU4lL2qoG/Y/6FcfOUSEoxXpRXPv8twJaKVXDLhrarZrJrhD5dF1GiJKjTCeSMV8l/0s4pNgF+kaWBBPZfN/gpPh4X0+qUFEobr2vTbyoJKZwkOtdIFmir7X8t66rRQwRh+7o+c0KVruocGkPf8WtNLXLqlgK7JJCExKNzmEFUsDZi2HS/qE0kMXdU8ytV0kGogTLzmTzF7+Pp+wFo4xtFtBiEcWCHS3SQMD1SY6eHK6PxDwUoP3IB9GA9rAibZEjY5h81n7cCAP2pG097U8Z104a1itY3ku3uJ1B9WNgOoFvgw7WtU3jfz90QQQWg98/TAWvpmVzt6NvLIwn8/IKo0CmiU09gX8qEED0Rbt72I06uyloFaNzQlgbPxEGzMA76XjedSfLlBVvuEHmDGhC9DxveO3OCs1zcCD6jeC8zVW5i/DAEaHz6H8aDP9phOCKubQ443WaI+RUvZepnbULGL5VpJGeIONu9+QlQ89djrdsRlSpVrEK+hlAIdzXTaX2j6k63XAYOuPN6Ht1rcti/Ndcy2wHUA/dCqNYLs2cmPCEpHN/6Vynr9AaKfV3Mq2Nc/bMo0/TCFlGM2HAslcnKj7eBvERqA8HoTEwyZLs7Tedh27k7bcZACF9RGxhGDtWPwNgPTvtHE7JcErDfd5QTPGoviisS5h4wHQiAyBeh0SB4ICR1EITwGb51RRLMgDDN6ZdMBUE6/4MlqmHf1Z0tLYSco9sMnXTYW+xutz0H/5KXJr+eNa5c8sojx7wveaBLIpw00Ir/TlabbttTMfPaoQF8JaBrJRufZVFc2RYqJn2SwoS8Lxx8Z89dYhDabR1DpjSdlSwQcMguK52SV8JgqguIpbNsFM7QbbLz7s/t84pjXAjWaDD3pGx7i9iXUFgsGL3JV+u5e7lKprcv4dSGw/CatHNZHpONr2rCqlN2siqwWqnZMMzCLgnBBGOP7Gj/pEEFGQavY8hE4ezlKHj2lUKUbI4LVu0VIHit1NBwImI69SncDHcCASLZaAg8vIeVT5ZcrEfncg3tJShMsF2nt1QN67rpADuTE+dgY20/oZ6gzApS7O6Sn3PgpG0A0Vhg/aITseFayGlzL0wLBollaiOHjblO6hQd3XH1w8Yq42m3Zoi1Q+UMwW6awWPSeZFKjCNQgM4jqJF+zSm8A== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: JaybyQ4zxaxz5Kyv0+bqMWXDmEi0sdqQhPOmCdqG6h8GcEKbACtxBxgHhS8ya1NQhyJ1purkhu3LCOg0mNRjqk5NyAaXItQRF9v0+V7wwlJt31yjqt2x64kO+SAojvP6GHSs5qvFkwsF8JQP8PbR0LgkqfCN7iH3dABhPediBPE9RDNi0eMJc9wUsGfuHOgNSwE2rnJVCsI3r73wiO4HkyX2wMPrzRF8tzBOpsZ7799r+atD9QqfS2Nu2iLUP6/hMASSUy4lXXKk+YKN2Gvb/2L4YImN/ZQqA9C91n3RQ069HBYVtEaRvIUpQkA1gSX2IRaXYGNedq9xI/5KwZ/2ikcxo9uW7QB1CBu94r2wMca8YnUZ1uIUxnnOAVEKQI11sTw+SateItzOjFxTkOvtycodt1eUEK4qvcNdxYrZoRo1AsTxuSmpXl255HYG2RdXDhriFSWuMYNpQgt++WcW038fM3QuYI+DfThkLO1ZkVF7UNlR4PgPYTKkTDusRDkeBOhAMagNKGXhP9e2lO31am6W7XaonUYp7sSyIAEgXUUIRjmttuz3ju8KqM+pnSXOPKYcvVWhlzyjY7F4/iQ93pItWb55f6X4c6Q1jz3yuD8= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: b3867e69-d84c-4850-7013-08dd13c512a1 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Dec 2024 18:05:29.9971 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YBP9keJDsoUtlq4OtRAVSX2iRXyfnGDi7jcjRzYcTXW0EepnQPUYiAJuG6GJ045f/GW/0812p6qsNaGVqIRVy0td4MOAT9npbL+cKdDS5c8= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR10MB6762 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-03_06,2024-12-03_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 bulkscore=0 malwarescore=0 mlxlogscore=999 spamscore=0 adultscore=0 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412030151 X-Proofpoint-ORIG-GUID: c3FV1hkGjGUzLb5QGTpUzIy9MRnj5LKR X-Proofpoint-GUID: c3FV1hkGjGUzLb5QGTpUzIy9MRnj5LKR Now we have moved mmap_region() internals to mm/vma.c, making it available to userland testing, it makes sense to do the same with brk(). This continues the pattern of VMA heavy lifting being done in mm/vma.c in an environment where it can be subject to straightforward unit and regression testing, with other VMA-adjacent files becoming wrappers around this functionality. Signed-off-by: Lorenzo Stoakes Signed-off-by: Lorenzo Stoakes --- mm/mmap.c | 85 +------------------------------- mm/vma.c | 82 ++++++++++++++++++++++++++++++ mm/vma.h | 3 ++ tools/testing/vma/vma_internal.h | 22 +++++++++ 4 files changed, 108 insertions(+), 84 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 16f8e8be01f8..93188ef46dae 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -111,8 +111,7 @@ static int check_brk_limits(unsigned long addr, unsigned long len) return mlock_future_ok(current->mm, current->mm->def_flags, len) ? 0 : -EAGAIN; } -static int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *brkvma, - unsigned long addr, unsigned long request, unsigned long flags); + SYSCALL_DEFINE1(brk, unsigned long, brk) { unsigned long newbrk, oldbrk, origbrk; @@ -1512,88 +1511,6 @@ SYSCALL_DEFINE5(remap_file_pages, unsigned long, start, unsigned long, size, return ret; } -/* - * do_brk_flags() - Increase the brk vma if the flags match. - * @vmi: The vma iterator - * @addr: The start address - * @len: The length of the increase - * @vma: The vma, - * @flags: The VMA Flags - * - * Extend the brk VMA from addr to addr + len. If the VMA is NULL or the flags - * do not match then create a new anonymous VMA. Eventually we may be able to - * do some brk-specific accounting here. - */ -static int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma, - unsigned long addr, unsigned long len, unsigned long flags) -{ - struct mm_struct *mm = current->mm; - - /* - * Check against address space limits by the changed size - * Note: This happens *after* clearing old mappings in some code paths. - */ - flags |= VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm->def_flags; - if (!may_expand_vm(mm, flags, len >> PAGE_SHIFT)) - return -ENOMEM; - - if (mm->map_count > sysctl_max_map_count) - return -ENOMEM; - - if (security_vm_enough_memory_mm(mm, len >> PAGE_SHIFT)) - return -ENOMEM; - - /* - * Expand the existing vma if possible; Note that singular lists do not - * occur after forking, so the expand will only happen on new VMAs. - */ - if (vma && vma->vm_end == addr) { - VMG_STATE(vmg, mm, vmi, addr, addr + len, flags, PHYS_PFN(addr)); - - vmg.prev = vma; - /* vmi is positioned at prev, which this mode expects. */ - vmg.merge_flags = VMG_FLAG_JUST_EXPAND; - - if (vma_merge_new_range(&vmg)) - goto out; - else if (vmg_nomem(&vmg)) - goto unacct_fail; - } - - if (vma) - vma_iter_next_range(vmi); - /* create a vma struct for an anonymous mapping */ - vma = vm_area_alloc(mm); - if (!vma) - goto unacct_fail; - - vma_set_anonymous(vma); - vma_set_range(vma, addr, addr + len, addr >> PAGE_SHIFT); - vm_flags_init(vma, flags); - vma->vm_page_prot = vm_get_page_prot(flags); - vma_start_write(vma); - if (vma_iter_store_gfp(vmi, vma, GFP_KERNEL)) - goto mas_store_fail; - - mm->map_count++; - validate_mm(mm); - ksm_add_vma(vma); -out: - perf_event_mmap(vma); - mm->total_vm += len >> PAGE_SHIFT; - mm->data_vm += len >> PAGE_SHIFT; - if (flags & VM_LOCKED) - mm->locked_vm += (len >> PAGE_SHIFT); - vm_flags_set(vma, VM_SOFTDIRTY); - return 0; - -mas_store_fail: - vm_area_free(vma); -unacct_fail: - vm_unacct_memory(len >> PAGE_SHIFT); - return -ENOMEM; -} - int vm_brk_flags(unsigned long addr, unsigned long request, unsigned long flags) { struct mm_struct *mm = current->mm; diff --git a/mm/vma.c b/mm/vma.c index 8e31b7e25aeb..9955b5332ca2 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -2478,3 +2478,85 @@ unsigned long __mmap_region(struct file *file, unsigned long addr, vms_abort_munmap_vmas(&map.vms, &map.mas_detach); return error; } + +/* + * do_brk_flags() - Increase the brk vma if the flags match. + * @vmi: The vma iterator + * @addr: The start address + * @len: The length of the increase + * @vma: The vma, + * @flags: The VMA Flags + * + * Extend the brk VMA from addr to addr + len. If the VMA is NULL or the flags + * do not match then create a new anonymous VMA. Eventually we may be able to + * do some brk-specific accounting here. + */ +int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma, + unsigned long addr, unsigned long len, unsigned long flags) +{ + struct mm_struct *mm = current->mm; + + /* + * Check against address space limits by the changed size + * Note: This happens *after* clearing old mappings in some code paths. + */ + flags |= VM_DATA_DEFAULT_FLAGS | VM_ACCOUNT | mm->def_flags; + if (!may_expand_vm(mm, flags, len >> PAGE_SHIFT)) + return -ENOMEM; + + if (mm->map_count > sysctl_max_map_count) + return -ENOMEM; + + if (security_vm_enough_memory_mm(mm, len >> PAGE_SHIFT)) + return -ENOMEM; + + /* + * Expand the existing vma if possible; Note that singular lists do not + * occur after forking, so the expand will only happen on new VMAs. + */ + if (vma && vma->vm_end == addr) { + VMG_STATE(vmg, mm, vmi, addr, addr + len, flags, PHYS_PFN(addr)); + + vmg.prev = vma; + /* vmi is positioned at prev, which this mode expects. */ + vmg.merge_flags = VMG_FLAG_JUST_EXPAND; + + if (vma_merge_new_range(&vmg)) + goto out; + else if (vmg_nomem(&vmg)) + goto unacct_fail; + } + + if (vma) + vma_iter_next_range(vmi); + /* create a vma struct for an anonymous mapping */ + vma = vm_area_alloc(mm); + if (!vma) + goto unacct_fail; + + vma_set_anonymous(vma); + vma_set_range(vma, addr, addr + len, addr >> PAGE_SHIFT); + vm_flags_init(vma, flags); + vma->vm_page_prot = vm_get_page_prot(flags); + vma_start_write(vma); + if (vma_iter_store_gfp(vmi, vma, GFP_KERNEL)) + goto mas_store_fail; + + mm->map_count++; + validate_mm(mm); + ksm_add_vma(vma); +out: + perf_event_mmap(vma); + mm->total_vm += len >> PAGE_SHIFT; + mm->data_vm += len >> PAGE_SHIFT; + if (flags & VM_LOCKED) + mm->locked_vm += (len >> PAGE_SHIFT); + vm_flags_set(vma, VM_SOFTDIRTY); + return 0; + +mas_store_fail: + vm_area_free(vma); +unacct_fail: + vm_unacct_memory(len >> PAGE_SHIFT); + return -ENOMEM; +} diff --git a/mm/vma.h b/mm/vma.h index 388d34748674..83a15d3a8285 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -247,6 +247,9 @@ unsigned long __mmap_region(struct file *file, unsigned long addr, unsigned long len, vm_flags_t vm_flags, unsigned long pgoff, struct list_head *uf); +int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *brkvma, + unsigned long addr, unsigned long request, unsigned long flags); + static inline bool vma_wants_manual_pte_write_upgrade(struct vm_area_struct *vma) { /* diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h index e76ff579e1fd..7c3c15135c5b 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -39,6 +39,7 @@ #define VM_SHARED 0x00000008 #define VM_MAYREAD 0x00000010 #define VM_MAYWRITE 0x00000020 +#define VM_MAYEXEC 0x00000040 #define VM_GROWSDOWN 0x00000100 #define VM_PFNMAP 0x00000400 #define VM_LOCKED 0x00002000 @@ -58,6 +59,13 @@ /* This mask represents all the VMA flag bits used by mlock */ #define VM_LOCKED_MASK (VM_LOCKED | VM_LOCKONFAULT) +#define TASK_EXEC ((current->personality & READ_IMPLIES_EXEC) ? VM_EXEC : 0) + +#define VM_DATA_FLAGS_TSK_EXEC (VM_READ | VM_WRITE | TASK_EXEC | \ + VM_MAYREAD | VM_MAYWRITE | VM_MAYEXEC) + +#define VM_DATA_DEFAULT_FLAGS VM_DATA_FLAGS_TSK_EXEC + #ifdef CONFIG_64BIT /* VM is sealed, in vm_flags */ #define VM_SEALED _BITUL(63) @@ -122,10 +130,22 @@ enum { TASK_COMM_LEN = 16, }; +/* + * Flags for bug emulation. + * + * These occupy the top three bytes. + */ +enum { + READ_IMPLIES_EXEC = 0x0400000, +}; + struct task_struct { char comm[TASK_COMM_LEN]; pid_t pid; struct mm_struct *mm; + + /* Used for emulating ABI behavior of previous Linux versions: */ + unsigned int personality; }; struct task_struct *get_current(void); @@ -186,6 +206,8 @@ struct mm_struct { unsigned long data_vm; /* VM_WRITE & ~VM_SHARED & ~VM_STACK */ unsigned long exec_vm; /* VM_EXEC & ~VM_WRITE & ~VM_STACK */ unsigned long stack_vm; /* VM_STACK */ + + unsigned long def_flags; }; struct vma_lock { From patchwork Tue Dec 3 18:05:09 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13892790 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BE1AE1FCF72; Tue, 3 Dec 2024 18:06:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249162; cv=fail; b=oMJYSQRkPaRRDUaCWpPJmFO4Yq906agt8xAueA21BJR9TZRsoNXfMJox3wdmsgb41M1l11kU2nI3jH9mpQxSD57GDb41xiwn1Afuw/TEqPde7vTzi0ZO1vOX1F0bvP7vFOrRNl8eT6DXcEtYGXpGx8Io4dC5CpF7+ZVWOghtxSA= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249162; c=relaxed/simple; bh=S1B/Mml5yLlchv1TpVuq1OEvpxY88r+OaxSQMZEBufs=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=Aese9UZLC5YYgStpqJz+AQMGTa/hUrsAh/C/1SjQs5+eewMsqF8DMW4WLmZb2L6mCIDjRjYZ9XQMeBmRcH4TknOWBbQb1raHCaDD1LXHGYk9G9ZfGrQf+DTzMgfvPy9cUpVl69XqxNuN6rXHaJU88vxQDYtUb7TE1BR09A3ArmU= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=C0nKvadF; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=ENpe6prf; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="C0nKvadF"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="ENpe6prf" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3HtcAA030390; Tue, 3 Dec 2024 18:05:37 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=LQR0MlY3DN6I9QkGWPkfUs2y6O6Dm9f9JD8+KM54Z/Q=; b= C0nKvadFInBxmb6ZM3RJe9ZhpohZkepq5I2NnZ1zaa48qKslFcVRy6CX0qtpL5GN 1x6K1bIS59DKKAOTP/iil4fS3SGQQk6EU3KoWbPgLWZruxz2ImFPNIhL8xtJNaEg jBkQQ0vkoKM5vg5rSsEDY7Zp1rseITyNuiGX95J/P1mIKDr1a0Wc7vJQnZsKLCSV /UXe6CbvGnUU63TNPeQbqX8UsAJxzUpUB9YvefunsU7Y7JJZT982IjHFHqPqbiVV QE4M/plLugxZcFnkjoF4D2Vn/BS7A0plsAlCbD1qqhhJZdMUjBKbF0k6eUFf2/dh H+nbxO2zj+SOirCGlwW5QA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437tas6n4b-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:37 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3GT4YD031378; Tue, 3 Dec 2024 18:05:36 GMT Received: from nam11-dm6-obe.outbound.protection.outlook.com (mail-dm6nam11lp2171.outbound.protection.outlook.com [104.47.57.171]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 43836u8ge7-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:36 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=WtPp/KaZFwbhcjZeAKoPgv6nFIjTXyyrcpYQypcQcE0vHMTY0txNxo1Un0dhyDhe4tnhtMhwyI6w87/fu0A80rYMfWTGFCc101GYpvF8vgE1e7L4mqfQ/EpHtpODSYchjPq8Rf40KP2KK/7p7n8budjm9SPMCTe8rKLEZl3DjN/7qc89ut5klAaMRdJKfqLMGrCgAUZH2lIRfTk+UKbINHVercRHMDtOUwNQoktE9N9jKSuHuRn77F/GrIOPZmKyYrOjU3oLnqD6tiqVVM3x7rAvuUObMgJVoZNvy3lFUp5OeTspWPyvT/ScDsE7yWJHO2lWOlnqxURlQY9q2QLFYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=LQR0MlY3DN6I9QkGWPkfUs2y6O6Dm9f9JD8+KM54Z/Q=; b=WEiRIEfUEc7sO3hkscx1FWrGGQiFwSZr3Am4kRpLIF2GSiiTPkQV/JJlcXzXaWHsYdZqh0x6o8mJC9gnCyVEVVe7beWtbeS687qHxtL1ZcUOHROJ4nRQtBxH28DXEIFtf973e4h/Y2DXwiXODj0yZLbbVeDm41EGwaQgxr0BI0cW0kTRTNzUWh3hoQF41jPDL4HHDjUIpLGo1GwXdolkrMJRZ/3NtrR1YLbioeG6s7vtmwqrHJgULuuFOgVkOUiFRBav/ZQzOTINZd1+LKwpg3KtMk2h29xezS5xBinL1qNouEKH7TVN0VaQIV0UAWC+IrDptzwWlVWcg7moa9UyFQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LQR0MlY3DN6I9QkGWPkfUs2y6O6Dm9f9JD8+KM54Z/Q=; b=ENpe6prfQjcivjCc0eV4TaRgeootnb34VnJ353KWPhsrDW9lfNO4XcRaEWTTJQdacTSEtM/84tw1GSIKoUxxVTblUk7nv0DV5FrgKpeK+Mzoaeld9EQIVaxjngKv2zLZ1Myb6G+EC08wiZGs7pkJp8T0oIbVVYCOy/LaDiOG0yk= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by CH3PR10MB6762.namprd10.prod.outlook.com (2603:10b6:610:149::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.19; Tue, 3 Dec 2024 18:05:33 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%7]) with mapi id 15.20.8207.017; Tue, 3 Dec 2024 18:05:33 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , Eric Biederman , Kees Cook , Alexander Viro , Christian Brauner , Jan Kara , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 2/5] mm/vma: move unmapped_area() internals to mm/vma.c Date: Tue, 3 Dec 2024 18:05:09 +0000 Message-ID: <53a57a52a64ea54e9d129d2e2abca3a538022379.1733248985.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: X-ClientProxiedBy: LO4P265CA0156.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:2c7::18) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|CH3PR10MB6762:EE_ X-MS-Office365-Filtering-Correlation-Id: 71f13595-baee-447d-8a23-08dd13c5146f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: ICsVShJ5xoO5eb7w5vOEl2LgVGvh+fELOWx7haX0pVg7hcK/DgiUEYqH8PanWD8tDq1HObhdjQ3/n7equWnJJb0AuH1hw3E1TdYp68mdEzioUvsRJz5k2DMkvpbCjKzN3medIQ3nh9+SEDa9lhBXoEzg1JGAKxiJ/G6Z+T8nTkveCqG+fOt0TJQzWgOP8tVX8Ud15vQ1O5DJ52OeFv93iGtwy+ALGQ4rcHRHnQbzrcAGGlw2/mHsQWokCd7d5MfZjWMjCLq7DKamKk7Lfc1EI735Wt4g0vQkCgFiQR5ZxEcAIGArMaoKx0ZhAO+JTxeNZWi2PNmI1rdKusiYp9/kJYX13Q/x2tKpG6n5RcEK9mmFDUAluox+KDmWojZLYxojCgjaflZfULvIUnlRGcl8pcOPUspbawDtsqsxxKZ1v211JaGqcUZupTEGrAyKb1MZD10kIQ6nXP9m+8w57mfsIp9CVHpho7lTeL/yHAxVkJSJ7ZodAXemx8H7MboVcrb+oN9DonMxYuf35yRd4iBZhXGZWbASXqrtuwl1ktopUkeZCcg5UHWnQP4oFUK/GXSJot8Et+Plrh/xA1hxY9qyceTdq29QljcdqbDnZbGPDMSWY8dusRk/qOrEytEyrRnYQM8qwcKqCtzYdYMfbjK7jp7Ddo+9xMySuw1y4X95jPsx9mNjnSeeE2pL1Em4IkQUtD7TDgkQIaUsJJnUGRvPv9DupHqyXTI7wfxbaCI1Gk/uHOGf9uiAXNKnJNy3Fx0EdbklBUIOYVGgqaYcdz2i2kx+VuHs9PmaEstnXeZgAHdgJl9eZajkdWEoBvxB9BFt855nHRoBNkoY73lOq3h9dLFulXj+lkk+MTJ4Rbi6HLT6kq/ZSCkM7P2XFscb/J7+cZHEJdWWxomnTc5YPZw/W7dcfkxxFkLeOGlkj3Wrlb3A6SMzc4oEszl1tnxjGKawgyJ4xyipbcj/yaQ801Ku0Ib4+RYOY7KhgwcI6/y667U2kZn8LlXxA02pZBEcjel1hhsYmIaZgPXWRD1xgBxr/+PDsstSYrQg3qz6r4jr31NophK2Hf2rRQ0AEWvWYnjsBLvItAwxPIyuoBGr9B+pOUj6TcPDe5PpliOGeNzvXPnHk/hEOmfPuCq5FeqxyWG+VWOZeO6QspLkF5aETheETt1QUII3X1Ab4J8S8HlCgTDOgwQ814MDz5QDfIUU8WIXVgGfm+iHGZeSRoFDerUXKW7w48zaw4H2/FPUArdQLxP+0gYc3C0zvxdINHRNedIKgfX/ro/f1f1fPLK4WyN8zIHFAAqczeUsdC3NnotMxgsRDygcqKLuBuMXO5bT+AcnYQ+vQjFHhLjXknNKAPRZJg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HmhS/KZyXdT25548PPgG+650iJteWwkaISXzbcH0ARddxPdGop/IyH6E2A08fI47zM5kTTocfYKwyXcLkYIS+8DWGuIUID6jQUzIfjSNO1E0LdZ5bPlde66bNNinGTSlQR9Ebe8vC+B73HOBJ7vMpsYfG4paYdHosFqqibcbaQloWqMQTPZL/6+vhMcMnwuwfADT4hOYdoLpjGwi281mYujphAMUC+kmWdO/P0v3iRcVexira8TLSoPnXrPO/OhXtUDhV+YhCFePBS1uAQ5nTpaXRWKOHxQh9PyUqipr3r2JZ0dsOScma0ZE92GDh1MlfVufgUg6L2TFSEQ3vs8KztgUufi6MKLl28VQlKUddyGDizVQ60jj+4y1++gWncWgqxVMZ5OduqiD7eTtBr58Fv4omMjcyMnrkcLhVq12z+iDlLQRF8fsrl2h+8UX1OXCcUQcFCpateqYL5S9/h1BPR3R1uAnPH4IWEyvu12isI/6DE5zUqdXJArkJwb56w7HYconknCfCu4RcKqjkjWWwMrAshHn+TczWf7g9xvrRgeGifJmuI/YNoY/PWjmpcR2bEAGsJ+ezUqNIT7SCg+qqZyDJZxzR6UbDSGGtfWZjwzVTn72PDrnthmoZjP4NpvttINN5Mz8Jacd89VhN+EsZ6RtRZeXXr2GOKk595UsfUBgl7nGtbKprmx171cjs9nh2AscLax1vFA/NcPPf6/ZiKD9kOwwhzmz8l80iP5N7YDag4xPHMvLamyiVzDWZrvlt/i+ySm6S7O7OAELKurE3Sie0k5xjEXuqrPl/+DpOWhnw7XfQ1lQx8JmqGa378Vw+mvtSMKIHXWXimDITI81Gu6XOyUTpxjwiMa6f3d4WsnydVxGhPFM3pdRxT6jxshmOdJE1LzD2uXArUvVjawD/YBWhsJ8UFEA2Gb0rhsVpgPyMs7FJzklMKtYN9FR1AEPNhzO/7wXHoqk4zh921L4k/nJGJZF3dW/5rkRrbeBSmES9ATdTYJdVFsqbMQ7jiN7mAF5WT1w4OCNn/1ZL5KJ+Z6Ui3cOynhZTIEbF5FBWfPvM9mv+pctY345ZCfnF90QXIF96Xsmuy4XaCG8Pgh9430UndV/m9T6s3JEGg0IIibPQ7zV3b2GK/ke3tfhVct+VPNBPLkQJ+z836GmfCF+Ab8F/FE3ZitVjK0vqyjdtRLUWjcfQCaELmrdklNXkrCMX2pMtUrVrO71GwPA85YiVxmzuGgdVg3tjXBaQ1wI09ghV3+9R0c3QaBdfoXIXkt/hCorVogDkYQhlOK2XMs/kUJpM7szCgUPAyqzPtJGpE6g2wj3vNrYM1jyVV2zIZJSm0EQsDDeQqMWEmes7m4C7WnqI2IOLkPkAsqw9/B6RpBsSr0zhxL8zU9mVyKdgAxqHDftvXbVstSlCBeofUgkKUyPTeZaVIaSN09amQIZhMpvRs5uf4DvbHQkEXp/sw57bmbo5dIzzPSOu6o/UeUDTirHzNzz0FbR1zIAe9eb3OYvYR6r6k0yZ8jqlGRGEh75qIurn3axiClb1CXHfIihUlO1WsINIpDzWQ0dWT7QpLOBL+eZJIK1kG7f3Z6CJr6ZFyobw8JYo3Svrhn+DM79Bg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: IDa02bT/8L4pMa9RphvG3AcyV3WhdtfzZYoX6gFrPNbpdXt02tZPPGCAmPNVKBrCwbeBy4t8ucewK9IBgEswy50eJqGw5EhwSVO6nlQPMxygb+PYZIkIaxjCJFzfV5d9Xqbz0UH0TZr5CFENuxwCCLPxk2U7+jInMGT0lHn3ck1IXvqlk+9RgCn7DCEHL5r6c9Q8Xu25gksY4Y8KP7S3mwSaKWk/5zmMMw+eL5dHo1ZxYA6DmIexDB1jayAS2FLlF2mpGDkcE4ggYba/kdWwU/mxanU0EsOUvqXi860c42JOVYJUP3Xt68t8DvdhQlg9B8EqkGHZPL5YmFvOoxuQ1HwH02PB6vr4afETNWLVs+B8FaQuziQUB6n6Ot/SJ913bL3XB1VHtPS7xzjKgYLl6YLNCbk1XO5dkxAL/b50Poqvzsnx7dQEPl+ILJ9w5b5hgIiQIeS+hMjoRaAhdjhl8OfbExJ96CjGK0jqhxKn2l/McxVy1NwDQDU9qGLouIu02Sxx0Haqu2GIHfaMLFduiti60vz4O09fbUnTUMhmoFx0RcnY1L4Dp9J2TNaSEXwWo56EwwhyzNI2THjqnk5eHeFZ+rsvkFJ+3gmNKSBiVoQ= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 71f13595-baee-447d-8a23-08dd13c5146f X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Dec 2024 18:05:33.0420 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SvQIeyqxOPpalGPvGV43fntbeAvTG4oB8C8pvpBDQqb55Rn+BDFpKrf38GDECq5qc3tbIRXVuzEEEule+iWk+clfJQjwTYokTJjuVIw09yk= X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR10MB6762 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-03_06,2024-12-03_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412030151 X-Proofpoint-GUID: qndHoWZXSMVnuKX_pkRmQP1wbk7I9NP1 X-Proofpoint-ORIG-GUID: qndHoWZXSMVnuKX_pkRmQP1wbk7I9NP1 We want to be able to unit test the unmapped area logic, so move it to mm/vma.c. The wrappers which invoke this remain in place in mm/mmap.c. In addition, naturally, update the existing test code to enable this to be compiled in userland. Signed-off-by: Lorenzo Stoakes --- mm/mmap.c | 109 ------------------------------- mm/vma.c | 109 +++++++++++++++++++++++++++++++ mm/vma.h | 3 + tools/testing/vma/vma.c | 6 ++ tools/testing/vma/vma_internal.h | 59 +++++++++++++++++ 5 files changed, 177 insertions(+), 109 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 93188ef46dae..f053de1d6fae 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -580,115 +580,6 @@ SYSCALL_DEFINE1(old_mmap, struct mmap_arg_struct __user *, arg) } #endif /* __ARCH_WANT_SYS_OLD_MMAP */ -/** - * unmapped_area() - Find an area between the low_limit and the high_limit with - * the correct alignment and offset, all from @info. Note: current->mm is used - * for the search. - * - * @info: The unmapped area information including the range [low_limit - - * high_limit), the alignment offset and mask. - * - * Return: A memory address or -ENOMEM. - */ -static unsigned long unmapped_area(struct vm_unmapped_area_info *info) -{ - unsigned long length, gap; - unsigned long low_limit, high_limit; - struct vm_area_struct *tmp; - VMA_ITERATOR(vmi, current->mm, 0); - - /* Adjust search length to account for worst case alignment overhead */ - length = info->length + info->align_mask + info->start_gap; - if (length < info->length) - return -ENOMEM; - - low_limit = info->low_limit; - if (low_limit < mmap_min_addr) - low_limit = mmap_min_addr; - high_limit = info->high_limit; -retry: - if (vma_iter_area_lowest(&vmi, low_limit, high_limit, length)) - return -ENOMEM; - - /* - * Adjust for the gap first so it doesn't interfere with the - * later alignment. The first step is the minimum needed to - * fulill the start gap, the next steps is the minimum to align - * that. It is the minimum needed to fulill both. - */ - gap = vma_iter_addr(&vmi) + info->start_gap; - gap += (info->align_offset - gap) & info->align_mask; - tmp = vma_next(&vmi); - if (tmp && (tmp->vm_flags & VM_STARTGAP_FLAGS)) { /* Avoid prev check if possible */ - if (vm_start_gap(tmp) < gap + length - 1) { - low_limit = tmp->vm_end; - vma_iter_reset(&vmi); - goto retry; - } - } else { - tmp = vma_prev(&vmi); - if (tmp && vm_end_gap(tmp) > gap) { - low_limit = vm_end_gap(tmp); - vma_iter_reset(&vmi); - goto retry; - } - } - - return gap; -} - -/** - * unmapped_area_topdown() - Find an area between the low_limit and the - * high_limit with the correct alignment and offset at the highest available - * address, all from @info. Note: current->mm is used for the search. - * - * @info: The unmapped area information including the range [low_limit - - * high_limit), the alignment offset and mask. - * - * Return: A memory address or -ENOMEM. - */ -static unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info) -{ - unsigned long length, gap, gap_end; - unsigned long low_limit, high_limit; - struct vm_area_struct *tmp; - VMA_ITERATOR(vmi, current->mm, 0); - - /* Adjust search length to account for worst case alignment overhead */ - length = info->length + info->align_mask + info->start_gap; - if (length < info->length) - return -ENOMEM; - - low_limit = info->low_limit; - if (low_limit < mmap_min_addr) - low_limit = mmap_min_addr; - high_limit = info->high_limit; -retry: - if (vma_iter_area_highest(&vmi, low_limit, high_limit, length)) - return -ENOMEM; - - gap = vma_iter_end(&vmi) - info->length; - gap -= (gap - info->align_offset) & info->align_mask; - gap_end = vma_iter_end(&vmi); - tmp = vma_next(&vmi); - if (tmp && (tmp->vm_flags & VM_STARTGAP_FLAGS)) { /* Avoid prev check if possible */ - if (vm_start_gap(tmp) < gap_end) { - high_limit = vm_start_gap(tmp); - vma_iter_reset(&vmi); - goto retry; - } - } else { - tmp = vma_prev(&vmi); - if (tmp && vm_end_gap(tmp) > gap) { - high_limit = tmp->vm_start; - vma_iter_reset(&vmi); - goto retry; - } - } - - return gap; -} - /* * Determine if the allocation needs to ensure that there is no * existing mapping within it's guard gaps, for use as start_gap. diff --git a/mm/vma.c b/mm/vma.c index 9955b5332ca2..50c0c9c443d2 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -2560,3 +2560,112 @@ int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *vma, vm_unacct_memory(len >> PAGE_SHIFT); return -ENOMEM; } + +/** + * unmapped_area() - Find an area between the low_limit and the high_limit with + * the correct alignment and offset, all from @info. Note: current->mm is used + * for the search. + * + * @info: The unmapped area information including the range [low_limit - + * high_limit), the alignment offset and mask. + * + * Return: A memory address or -ENOMEM. + */ +unsigned long unmapped_area(struct vm_unmapped_area_info *info) +{ + unsigned long length, gap; + unsigned long low_limit, high_limit; + struct vm_area_struct *tmp; + VMA_ITERATOR(vmi, current->mm, 0); + + /* Adjust search length to account for worst case alignment overhead */ + length = info->length + info->align_mask + info->start_gap; + if (length < info->length) + return -ENOMEM; + + low_limit = info->low_limit; + if (low_limit < mmap_min_addr) + low_limit = mmap_min_addr; + high_limit = info->high_limit; +retry: + if (vma_iter_area_lowest(&vmi, low_limit, high_limit, length)) + return -ENOMEM; + + /* + * Adjust for the gap first so it doesn't interfere with the + * later alignment. The first step is the minimum needed to + * fulill the start gap, the next steps is the minimum to align + * that. It is the minimum needed to fulill both. + */ + gap = vma_iter_addr(&vmi) + info->start_gap; + gap += (info->align_offset - gap) & info->align_mask; + tmp = vma_next(&vmi); + if (tmp && (tmp->vm_flags & VM_STARTGAP_FLAGS)) { /* Avoid prev check if possible */ + if (vm_start_gap(tmp) < gap + length - 1) { + low_limit = tmp->vm_end; + vma_iter_reset(&vmi); + goto retry; + } + } else { + tmp = vma_prev(&vmi); + if (tmp && vm_end_gap(tmp) > gap) { + low_limit = vm_end_gap(tmp); + vma_iter_reset(&vmi); + goto retry; + } + } + + return gap; +} + +/** + * unmapped_area_topdown() - Find an area between the low_limit and the + * high_limit with the correct alignment and offset at the highest available + * address, all from @info. Note: current->mm is used for the search. + * + * @info: The unmapped area information including the range [low_limit - + * high_limit), the alignment offset and mask. + * + * Return: A memory address or -ENOMEM. + */ +unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info) +{ + unsigned long length, gap, gap_end; + unsigned long low_limit, high_limit; + struct vm_area_struct *tmp; + VMA_ITERATOR(vmi, current->mm, 0); + + /* Adjust search length to account for worst case alignment overhead */ + length = info->length + info->align_mask + info->start_gap; + if (length < info->length) + return -ENOMEM; + + low_limit = info->low_limit; + if (low_limit < mmap_min_addr) + low_limit = mmap_min_addr; + high_limit = info->high_limit; +retry: + if (vma_iter_area_highest(&vmi, low_limit, high_limit, length)) + return -ENOMEM; + + gap = vma_iter_end(&vmi) - info->length; + gap -= (gap - info->align_offset) & info->align_mask; + gap_end = vma_iter_end(&vmi); + tmp = vma_next(&vmi); + if (tmp && (tmp->vm_flags & VM_STARTGAP_FLAGS)) { /* Avoid prev check if possible */ + if (vm_start_gap(tmp) < gap_end) { + high_limit = vm_start_gap(tmp); + vma_iter_reset(&vmi); + goto retry; + } + } else { + tmp = vma_prev(&vmi); + if (tmp && vm_end_gap(tmp) > gap) { + high_limit = tmp->vm_start; + vma_iter_reset(&vmi); + goto retry; + } + } + + return gap; +} diff --git a/mm/vma.h b/mm/vma.h index 83a15d3a8285..c60f37d89eb1 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -250,6 +250,9 @@ unsigned long __mmap_region(struct file *file, unsigned long addr, int do_brk_flags(struct vma_iterator *vmi, struct vm_area_struct *brkvma, unsigned long addr, unsigned long request, unsigned long flags); +unsigned long unmapped_area(struct vm_unmapped_area_info *info); +unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info); + static inline bool vma_wants_manual_pte_write_upgrade(struct vm_area_struct *vma) { /* diff --git a/tools/testing/vma/vma.c b/tools/testing/vma/vma.c index 8fab5e13c7c3..39ee61e55634 100644 --- a/tools/testing/vma/vma.c +++ b/tools/testing/vma/vma.c @@ -18,6 +18,12 @@ static bool fail_prealloc; #define vma_iter_prealloc(vmi, vma) \ (fail_prealloc ? -ENOMEM : mas_preallocate(&(vmi)->mas, (vma), GFP_KERNEL)) +#define CONFIG_DEFAULT_MMAP_MIN_ADDR 65536 + +unsigned long mmap_min_addr = CONFIG_DEFAULT_MMAP_MIN_ADDR; +unsigned long dac_mmap_min_addr = CONFIG_DEFAULT_MMAP_MIN_ADDR; +unsigned long stack_guard_gap = 256UL< #include +extern unsigned long stack_guard_gap; +#ifdef CONFIG_MMU +extern unsigned long mmap_min_addr; +extern unsigned long dac_mmap_min_addr; +#else +#define mmap_min_addr 0UL +#define dac_mmap_min_addr 0UL +#endif + #define VM_WARN_ON(_expr) (WARN_ON(_expr)) #define VM_WARN_ON_ONCE(_expr) (WARN_ON_ONCE(_expr)) #define VM_BUG_ON(_expr) (BUG_ON(_expr)) @@ -52,6 +61,8 @@ #define VM_STACK VM_GROWSDOWN #define VM_SHADOW_STACK VM_NONE #define VM_SOFTDIRTY 0 +#define VM_ARCH_1 0x01000000 /* Architecture-specific flag */ +#define VM_GROWSUP VM_NONE #define VM_ACCESS_FLAGS (VM_READ | VM_WRITE | VM_EXEC) #define VM_SPECIAL (VM_IO | VM_DONTEXPAND | VM_PFNMAP | VM_MIXEDMAP) @@ -66,6 +77,8 @@ #define VM_DATA_DEFAULT_FLAGS VM_DATA_FLAGS_TSK_EXEC +#define VM_STARTGAP_FLAGS (VM_GROWSDOWN | VM_SHADOW_STACK) + #ifdef CONFIG_64BIT /* VM is sealed, in vm_flags */ #define VM_SEALED _BITUL(63) @@ -395,6 +408,17 @@ struct vm_operations_struct { unsigned long addr); }; +struct vm_unmapped_area_info { +#define VM_UNMAPPED_AREA_TOPDOWN 1 + unsigned long flags; + unsigned long length; + unsigned long low_limit; + unsigned long high_limit; + unsigned long align_mask; + unsigned long align_offset; + unsigned long start_gap; +}; + static inline void vma_iter_invalidate(struct vma_iterator *vmi) { mas_pause(&vmi->mas); @@ -1055,4 +1079,39 @@ static inline int mmap_file(struct file *, struct vm_area_struct *) return 0; } +static inline unsigned long stack_guard_start_gap(struct vm_area_struct *vma) +{ + if (vma->vm_flags & VM_GROWSDOWN) + return stack_guard_gap; + + /* See reasoning around the VM_SHADOW_STACK definition */ + if (vma->vm_flags & VM_SHADOW_STACK) + return PAGE_SIZE; + + return 0; +} + +static inline unsigned long vm_start_gap(struct vm_area_struct *vma) +{ + unsigned long gap = stack_guard_start_gap(vma); + unsigned long vm_start = vma->vm_start; + + vm_start -= gap; + if (vm_start > vma->vm_start) + vm_start = 0; + return vm_start; +} + +static inline unsigned long vm_end_gap(struct vm_area_struct *vma) +{ + unsigned long vm_end = vma->vm_end; + + if (vma->vm_flags & VM_GROWSUP) { + vm_end += stack_guard_gap; + if (vm_end < vma->vm_end) + vm_end = -PAGE_SIZE; + } + return vm_end; +} + #endif /* __MM_VMA_INTERNAL_H */ From patchwork Tue Dec 3 18:05:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13892792 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CB5FB1FECCF; Tue, 3 Dec 2024 18:06:01 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249163; cv=fail; b=Q3HRFFytavYpBPsvm87RWC3b+hWAQOZmP/TSM8eC0mkurL7s3v0UHaRvMXMN3wFBqSJC534lGwVp+JIYH0GKmc9Tav2irJUdz+pjo9YFTUO/2Cj3rcHSfP5XNcjf2KfbgbduJTPimQulP39ygaIv6cmzqpn65kztJ7JSjqb5QBE= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249163; c=relaxed/simple; bh=MFjaAUlp7Ligi12p8xZMO8mJ7xswaeuGlUCXDnAvqSc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=AcotIelr8Ad89lpRAjaA1kTtPQGaTRQC2H1vfJvLtytsWNYudUBE0pEq0ydCsxISI1W2xgZT/kyjC83CIv9LzGBMD6gPSuD6eMHpnB+Afk7IneMh287p/ZP4QtJDaFwPPGmnmbfOJCWJYReY3gpV8FjTonkiOoBNQe9Wo9GxI68= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=oXT2jfMn; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=b3SQ1GvG; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="oXT2jfMn"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="b3SQ1GvG" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3HtaHH012461; Tue, 3 Dec 2024 18:05:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=+zGfORM8vxcqIf9zAHTcbw5ZwMMMROGOU4OV8XXGU0A=; b= oXT2jfMn7oeEfjVs4KwvhPpwee/RtO2sjPAbwq1lLGu3tTTamI7z1nX0j57v5ir3 dxtKonzJW3HUdp+k0Pu2WwGk+V5NbnGCFPUm6KICvtajUlh/dk/OzCw8HyIAlWSJ CQNNjguucI3yiLIkdh4l7ieTa5zIt1vf3+imD6FyJY0iPat/k+dzgu77mxSfcMN2 k5eVcFcZ2qTDBWCljKi8L7COc9ZwGfTrYdvrCIbnzaLcfuYAvUPzMCQSUSmAv+zZ Mckp6lsch5pcgoHwX6I0lhTQqEhFKSgygn3HFkvfFVt1yK/8JwlVxpb9XjS3mNIV mY62BM2f7PYxXXZoqP/UBA== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c6nsg-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:39 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3HBlmm031373; Tue, 3 Dec 2024 18:05:38 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2048.outbound.protection.outlook.com [104.47.58.48]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 43836u8gfm-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:38 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=KZoNH+R/bUAJC56npUCWew4/jiQ1O1AUJ12PO1p0PQFVoSa0ZAbYl+mnhHWqUP9DBb29wlEsRoDPymSJD7LNuDHlxnSOu+ReOSsiT6nHHq2v1K+qgHlYEDIqfU5+UEzknr9EqrAnbpog44C3WQHK9WRuvLzXCcdranLlnhMAfIYL64jb1CIf2msMlLQOZFyJjKWy7Rc668Bu9smDeIoEt02mbRvNpyPipM7FmbNqDGSdfTsRLOSRgQrHvG0EJwdWoAdrmwDLTRliPYGwiTYCGRRH9MEzFtdPfQP+4EU3D+gxchDEpF1aiSLa68OkWu9a2LBBFMUyXB5rsEclfiSYVA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+zGfORM8vxcqIf9zAHTcbw5ZwMMMROGOU4OV8XXGU0A=; b=fHnHsZBnvy8GyHnIWaNOY/jv/u9S2WSIhkYr8fR7WoG0jT8+tAA7lEVQy9Q1GfsU1KDKekj75rGwoeHJqifmNnGl+5zDcQHGaqjbQ9aJaGYPaXYRnpBDtB3h2mmZr7jcVGA6tC4wuGmFN47UtMDbqj7eVBF301Bowcc173kGzNu7pOz7IZN0xDzXMqJyI28sy/+QI+PZoDFdLv3PeceDm7EyozLttd6j4r7bsJHZDoESmVJEuvFMcfCR0M1VdX00RP/VqdlVPYkrjKuPRtsZbPJfKNYwMb5zqSlVQNgYmPNaGmJjk7FMOW8rQIt45Fn9+hWR6Ar6d6kt/gTicPP7PQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+zGfORM8vxcqIf9zAHTcbw5ZwMMMROGOU4OV8XXGU0A=; b=b3SQ1GvGnI6puimNjnDbQaYaWSN398n2/CWwqfDLIvDpc+6EIA2U8Wwb1JGgymnys87Lsf1K7RbQSgpjafxU0v6752Tqsysuy+tb4IFk3qcR3jhm00yHR0i8NqLAgA7pGbtBqNVIUr2SQgp+wYZupJk3Qab0FSmOvy35LmFBVbY= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by LV3PR10MB8129.namprd10.prod.outlook.com (2603:10b6:408:285::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.19; Tue, 3 Dec 2024 18:05:36 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%7]) with mapi id 15.20.8207.017; Tue, 3 Dec 2024 18:05:36 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , Eric Biederman , Kees Cook , Alexander Viro , Christian Brauner , Jan Kara , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 3/5] mm: abstract get_arg_page() stack expansion and mmap read lock Date: Tue, 3 Dec 2024 18:05:10 +0000 Message-ID: <5295d1c70c58e6aa63d14be68d4e1de9fa1c8e6d.1733248985.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: X-ClientProxiedBy: LO2P123CA0054.GBRP123.PROD.OUTLOOK.COM (2603:10a6:600:1::18) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|LV3PR10MB8129:EE_ X-MS-Office365-Filtering-Correlation-Id: 3dacb44b-3662-46aa-fa41-08dd13c51629 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: o2rRmFT28JhEpUEfFSjsCCpDTA2FkoY94E8UTcm5ly+fMW30+gR22kOMNtTD2s94m19Z2YlgA4feH9l6lvwPoYIVU9xlqJw7C4Gvc5Gg5c52S4MSo77qEaFNk2CT6E+qUc70cjqF0ROai/cLL1ofj0o5SrWgFchOvInMSM6ZR3wRBVKCvE/wIkwjZpfi/VljnuvRZvzZGNRByCuBf2a81Rxk93xAQsY5OILsW89dzcF+HNnZETYpRxZPHlyLLAH736IZW6yLGsq/AfqKPOV1O1vLO6LBj7/rMaqeuly7TmnBJdK8aTq+ssTbolHAHSgN248cpBMZtGsvhYCk72HjHT78vtTFjsd5d1L1xvy7PthlvNvVONET5Jx3ALXcFEwGmezbKtrMWvSRrBU+cbh2k38Yl4d4GxbGcPeXzYebGRjQCGAog1Zzz2/YT0fFqOSYSOzAaEF8fgDP3rGXbXSdP+MHJJYi+0VdzJ09PW1M4nOCE0bjfcy9XiW6sBkVbkbH1eYlc7nfJi3LuwehGBJMRxera1KVchUKYt6ngJwiP1yMdYHIuGbOU2WrVKrJxPlw9mwaO5+o3FXerG81Lojvm1dgWXBPaaviC/sTPKLAL7oX9zHDcJ8IvkrSZn6O6hRuTBPVKe6gG979HzZeGElQD7f5uOnxYZoiIOouWZpVrWE0tkcBysb7bdPmPV09Gkqf3hhci7TSC73LG5XEuLFKkDTyNWPD7UjtNjhcrKr2ud7D3dhZeIxMUhi005U3Yat0ktrzbwLytWqp7AmLQz1zuZ7JIk3QE7n6BXh9JzQcH9+llspOY1y+R61Xu6u4wbpyMyi+j5yO91/ek6rynPjR6wrxksJvE602UQwru5Sn0A0sLx0NnEPZh9hE6A3kRDcLRgroT7Cd8fi/tS68c4SgOKRWA+T73XYJbcvFG3NLWWkDCPEC8NN5q3grxhQ6jcIHgeaOxmjJ/LzxYs+hzf9JQrUvPTBrl/d7aMObdy76W2I0dUQPgjLMyX3W4KF8Eyg1t4v0k4/m+fov4OCYtgutyTLrB2AMyIW0C0eUCX2Pk7JKFHDTO1RkaZR78NzJDPB1XIlpmMiIpWm3p9+zQD90nNnPG6crf1DsF8CnADVZtnQ1h76SlII5e7QcTh5Bp75n1q2BoB5VVd9GcKSo+0lHHbf7/sUZqiwT17pqfjZBIsyggXXMBKm3jCx3I2HQCjkUqexwOhZ372crufJrz2D4tzb1UTU1+3EBtA5mTsvT8LMN/+nx3RBpr02BOv9otoBxkquuZGG432jGCWfYQRvTl0ZsSg2gpaIaHJhOECcL8fLYcpE5EKeqJH6c27EIg1jJHaVgGnl8yoQ7YCZRn9TUQg== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(7416014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: occjZRIeZ6qr7jM3vm9ia/WQIwFbw8tHM7mgAkZweIogf1QBoeZD1xRqD6Aap17MjsCBdx8ipiySG7vlBRcZBJW0T7C0LfSiO0Zv2FiJoWGaN6eRrid/PTu2x+BrRD4NTBn0f2WguQ3KdACw2iycNHj5qISHknAsqxG4+Csu+BhWPj8oK2oQjewH5gEBiFnLx4SjU1QIwtnikTKvERHupvOX5mHwNWTVAnEv9jK9cevsifIr9GH74Jk791a3sfw52yJrgP7WxmUH/6JhfEBQKZ28dFtHIq1iFZO+R+heUEJTRDB5NuzoBqaWBg//nwXm9OUVGy6D5wsqpYus4c8/2dzVMtCwC3wudL9b6G5pTQfxrS0bTiSFRgG1pH8IsYGqqsyurmqXljrQBa4zz1S++9O0a2X8apwrhRc43MxE6qdoj0xUuAceboeO3enA94htMXH+4jcPObU5lhyzFJojjf6c00MqMxeji0EAYMDtjnUcWWgbjcUGR2fKxutOEloGYN08hqklW4QTDqKGgvHEuIv+1dRocZe+gWhe006HjxHTtDcH7wEvYQHTawBSowb5W3fSOv7bBHuxSwr9n+v9/kftLJ6fUw5WmhaH0tB4VAUl+A6J+MtaueEFaocsb7JWl+u/MR3DtTUpEUB+TOo1W6LxyG1vlx7/vtR933EfrxDbiwQRbRl0zUnutp9mCSvDC2PS7xdXtpOPvDfXA95P9heB+fMS4WMKGfKWrjjOwbD9TXQ20GFX54Vmwz99No59H4EYbGerx6u0LByONxyHOZduSOOHDKEKXJMb3yEzWnTwpygbQJeWqIdT4Xuykh16n2hgsSu9OOfbJqY0G+1VyNGJazjgATSJclmCSfcb/5tffVD6/v2YQOJmBq0l3jxfNUybQXlBLaWHRvRmV9EDeuFm17AsV9dx5JyC1p5hz2Ca1jpijoID8ZMwBKNsNM+zBj3gl+cMwlBw3PUm8wUuPCPFuIDVjw6o7/x6ro2vB4D2ko3A++l2XkaXvF6TLs06N4OBAV01P21k41xhCYzOLz1xDKOMP/UFQiTFaPn1TexWRiNS7koE1aTwD9enlfvdQ6y43EAe5CPTx26ljPli4fllqADfSzxKRAU8TkZbM4asjpQ3v4gNU+uX06GvUqDEVWpjlAu5EUuBhBy4dJUhNJQLSau1JMtSvsiJvnTYbYiQ6MVdmZHFl9qWZ9bJztcG3AEH92GMzSsoxTay7C29er+2zJAz/Ld5ZhoNZyHCkFRcy9soA8fQD4hWHia90MUPk2zCv76c/LtQwKpDwWlagCQImdEdAZjSQWEaydCBJCxN76VZHNTpfGhkU1wqPLO78vQrXpwPTv0/MfmMjUlAJ4s+KM1Boql24xXe3f0kgbgiHi0Kurbbmii9hk5YjvIsa/BmmbTNcoTfOe0EZArrRge5GzTrGb3Z6NF4QMEpU1Goas1tqJ3kWljYDgZTnnfXWChjYPu37XnphCMAxJIybFcutEXILGKwjpyyt8FWlDetEG3eQf1ZrfOMS+c77ejyop+YASXTQShO4bKx3P/+wKiC5v0K2yt5/tWCMquWK54Pw9DwuWGvyRbNWc50TU7zO6lv+BxvcIktQVVholfdVg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: u8gk3uHn3Bhow4/Qy8y86XL40uVM0p7Ghd7oJbzANO5rPVLQIbppjY30i46nyihlAy3rjl5bFXe/1G/Fq7DA7BVD3YGawRjp6boy4lHZtVElz98VQWjKtDhS2F9act4JUGRxOROYvicMp8k/bntCrya30c9DDmKPQA2PVc2TK3XqhxAP5mblSGCO3XnDiYXckleafm+lFH/sv9B509w7UC1STRPZZizwCMzp1gI36BMg2z33gP+IJyiNXlkbVtwAoxg+dmGLtB8xEJkfRc1cOPA9tJV2DVW1racW+vhK48l4X/SftjuhUFjx9fXfLITKsR2aabkPayZwHSk4pvtBKjYZitDniHzlEk+bG3QnD/85vKztixl4vO8NLOXWJIsIGDSvrKbHHIn4TO4rRqnNqRDeIKID/LDrVic2f4X8meIoPNKhb5tOP3lEjSAGtjQanL/6vqi11FBJ2HjRm1Iypo2VDJ6+CQQtoTqlDJicakpGVyCDxvu6CVt8+n8icXXA0sFnjG1tzHcDomy0NX9gDNJykgcchsRf2K9HX8J3ig5hd/urlkZU5AL9jNIvgFDBM9vp6kPZbY+uy0KHFOw+93TfxLxnBziFdTU83ptq2ro= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3dacb44b-3662-46aa-fa41-08dd13c51629 X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Dec 2024 18:05:36.0321 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WFzTcUNmv5TME4Sn9e/mw+gMzgI3LBIRdlqeVGn6h/Jfj1AqjWVD455zNayqPLQAjk8lj4EzTsYtkl6PaeyOuJUH7uyoq1bYBd0dvF5vOjA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR10MB8129 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-03_06,2024-12-03_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412030151 X-Proofpoint-ORIG-GUID: Lzh8xQqUisEYwiz-1sXK-ou1lh9FJ8o_ X-Proofpoint-GUID: Lzh8xQqUisEYwiz-1sXK-ou1lh9FJ8o_ Right now fs/exec.c invokes expand_downwards(), an otherwise internal implementation detail of the VMA logic in order to ensure that an arg page can be obtained by get_user_pages_remote(). In order to be able to move the stack expansion logic into mm/vma.c in order to make it available to userland testing we need to find an alternative approach here. We do so by providing the mmap_read_lock_maybe_expand() function which also helpfully documents what get_arg_page() is doing here and adds an additional check against VM_GROWSDOWN to make explicit that the stack expansion logic is only invoked when the VMA is indeed a downward-growing stack. This allows expand_downwards() to become a static function. Importantly, the VMA referenced by mmap_read_maybe_expand() must NOT be currently user-visible in any way, that is place within an rmap or VMA tree. It must be a newly allocated VMA. This is the case when exec invokes this function. Signed-off-by: Lorenzo Stoakes --- fs/exec.c | 14 +++--------- include/linux/mm.h | 5 ++--- mm/mmap.c | 54 +++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 58 insertions(+), 15 deletions(-) diff --git a/fs/exec.c b/fs/exec.c index 98cb7ba9983c..1e1f79c514de 100644 --- a/fs/exec.c +++ b/fs/exec.c @@ -205,18 +205,10 @@ static struct page *get_arg_page(struct linux_binprm *bprm, unsigned long pos, /* * Avoid relying on expanding the stack down in GUP (which * does not work for STACK_GROWSUP anyway), and just do it - * by hand ahead of time. + * ahead of time. */ - if (write && pos < vma->vm_start) { - mmap_write_lock(mm); - ret = expand_downwards(vma, pos); - if (unlikely(ret < 0)) { - mmap_write_unlock(mm); - return NULL; - } - mmap_write_downgrade(mm); - } else - mmap_read_lock(mm); + if (!mmap_read_lock_maybe_expand(mm, vma, pos, write)) + return NULL; /* * We are doing an exec(). 'current' is the process diff --git a/include/linux/mm.h b/include/linux/mm.h index 4eb8e62d5c67..48312a934454 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3313,6 +3313,8 @@ extern int __vm_enough_memory(struct mm_struct *mm, long pages, int cap_sys_admi extern int insert_vm_struct(struct mm_struct *, struct vm_area_struct *); extern void exit_mmap(struct mm_struct *); int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift); +bool mmap_read_lock_maybe_expand(struct mm_struct *mm, struct vm_area_struct *vma, + unsigned long addr, bool write); static inline int check_data_rlimit(unsigned long rlim, unsigned long new, @@ -3426,9 +3428,6 @@ extern unsigned long stack_guard_gap; int expand_stack_locked(struct vm_area_struct *vma, unsigned long address); struct vm_area_struct *expand_stack(struct mm_struct * mm, unsigned long addr); -/* CONFIG_STACK_GROWSUP still needs to grow downwards at some places */ -int expand_downwards(struct vm_area_struct *vma, unsigned long address); - /* Look up the first VMA which satisfies addr < vm_end, NULL if none. */ extern struct vm_area_struct * find_vma(struct mm_struct * mm, unsigned long addr); extern struct vm_area_struct * find_vma_prev(struct mm_struct * mm, unsigned long addr, diff --git a/mm/mmap.c b/mm/mmap.c index f053de1d6fae..4df38d3717ff 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1009,7 +1009,7 @@ static int expand_upwards(struct vm_area_struct *vma, unsigned long address) * vma is the first one with address < vma->vm_start. Have to extend vma. * mmap_lock held for writing. */ -int expand_downwards(struct vm_area_struct *vma, unsigned long address) +static int expand_downwards(struct vm_area_struct *vma, unsigned long address) { struct mm_struct *mm = vma->vm_mm; struct vm_area_struct *prev; @@ -1940,3 +1940,55 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift) /* Shrink the vma to just the new range */ return vma_shrink(&vmi, vma, new_start, new_end, vma->vm_pgoff); } + +#ifdef CONFIG_MMU +/* + * Obtain a read lock on mm->mmap_lock, if the specified address is below the + * start of the VMA, the intent is to perform a write, and it is a + * downward-growing stack, then attempt to expand the stack to contain it. + * + * This function is intended only for obtaining an argument page from an ELF + * image, and is almost certainly NOT what you want to use for any other + * purpose. + * + * IMPORTANT - VMA fields are accessed without an mmap lock being held, so the + * VMA referenced must not be linked in any user-visible tree, i.e. it must be a + * new VMA being mapped. + * + * The function assumes that addr is either contained within the VMA or below + * it, and makes no attempt to validate this value beyond that. + * + * Returns true if the read lock was obtained and a stack was perhaps expanded, + * false if the stack expansion failed. + * + * On stack expansion the function temporarily acquires an mmap write lock + * before downgrading it. + */ +bool mmap_read_lock_maybe_expand(struct mm_struct *mm, + struct vm_area_struct *new_vma, + unsigned long addr, bool write) +{ + if (!write || addr >= new_vma->vm_start) { + mmap_read_lock(mm); + return true; + } + + if (!(new_vma->vm_flags & VM_GROWSDOWN)) + return false; + + mmap_write_lock(mm); + if (expand_downwards(new_vma, addr)) { + mmap_write_unlock(mm); + return false; + } + + mmap_write_downgrade(mm); + return true; +} +#else +bool mmap_read_lock_maybe_expand(struct mm_struct *mm, struct vm_area_struct *vma, + unsigned long addr, bool write) +{ + return false; +} +#endif From patchwork Tue Dec 3 18:05:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13892791 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFE851FECC3; Tue, 3 Dec 2024 18:06:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249163; cv=fail; b=GZ+PKiGzmCzxwiEotXJBKGNGkQONlKmJFGcLEFsQZc5uqpUoVwMstD+4LGyOCH9zXJt0sj8laAj2DMtfIMyHnOW90CyvDNX9YdhH8H9cdJM27gex1HKkYomSne5K8K9GaAQ7KFslErQgBDAwFDu1Tu0QKJB14JiquN5kH+2PL4Q= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249163; c=relaxed/simple; bh=ijSqR7vQeesx9ToiOXXWqmTLUzHTbMcyZ/f8EIqPNrc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=DHZ7PJ7X7gbiMChpGT0vt+r+94NCAhVa0o4r5IuOHBsuuCDX4JAmZKd6BFGS/XOIJW2CoDbiWiue+p61sg6WBD6Je1rJ8xVK/9rONy1zvqNd/lRpkOYEQ1v0IAV6zGZmMgkNJ6DahHw5NyRo2zZxoI6eKoW8NrMc84MWLOs+w2U= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=RQH30V0e; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=eN85Fi2F; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="RQH30V0e"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="eN85Fi2F" Received: from pps.filterd (m0246632.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3HtY0v030152; Tue, 3 Dec 2024 18:05:44 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=f8U/gqFVS1QCBDY/iVkzXmoF1Dj4A8+V07OZpj6ShB8=; b= RQH30V0e2OB8NtN+txK9AtVkNo/pCz8GS9+IL50Ou1lnOLHm4TSK6ZSmDFhLSmiU dU8ufq5Rarg090USVxEzYXPYBhNNGY+FhUTJnXf11v8Q1QHtjOv2QHGeT6OsoL5q 7tVarRDwp5TU07jkiLyzF1dnJEDovWObGrQUf0pGosK2CZEuEyPpYzA8YqR2ETme do8n5BTELp+RpuyZepnFALVYjmLKE0RjHYohnJkJIINkAc8TxCRzMy0o+nKf4iyl fRUjBp2J/i6vIEtBp3Tqc8s4oUtOAbaDsA2Io/Tt/91Dh12HZ0HNqLpy7pPcCCuY PKfgHZS3xgIHgm/lSTowvQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437tas6n4p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:43 +0000 (GMT) Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3GT4YF031378; Tue, 3 Dec 2024 18:05:42 GMT Received: from nam10-dm6-obe.outbound.protection.outlook.com (mail-dm6nam10lp2047.outbound.protection.outlook.com [104.47.58.47]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 43836u8ghx-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:42 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=vg3VOgVAlcfndi7pvpntjJ1TWo62pOnBvX+iz/zqnmKrWnZyo1xFlYDxh4EjoF11Ccvj88xX2xReSGMiTShu01pTggChpFVAFjoFiIVAYzTLbI0CedQGQd6Q/wlxMv0KpoOI6t1l61HuMMrBYAvd9jHIqT6/acr10EdXJsI9eUP9t41PJThMYGL7EYveNDxBO6+K+93grGhwNXQw3FPdj0CkdUKR72IPpImhoXkU8lOQMokALaV+TheOjd94rXzl/hDNlzbAiKE5uOSpGTNYukP591QPQeOpxFz/o4Zwi19p4duR/GHlvPCzArMQ6Rg7koZ91f8elMSJqGbNsT11kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=f8U/gqFVS1QCBDY/iVkzXmoF1Dj4A8+V07OZpj6ShB8=; b=yAMHz9SPSnVQa1spYY/mq32JSXK1IitWWxjUo83g3NzSRV3ut1nhfojhkZ1R8BFhyQ3EVrtTsrkS7GF2x+IV9giorxrL0EDvH7UkVb3uGsovdDwdxTASuikX7jRtfcWqoGhcgUaBWY06ygDbZEALyc3Vl5f1Mgvdoii7KF3JxhR1xmFcmVhiJKmOI2iHgIqXgrFXAfRtDG4hZt0GZfQx9C9ORai0Xmfl0Ida10EHbthdAgOYox9hYkteoUEcKzt24xCyBe42c8ilH4LHM8B74T2L0TlXHo8bNY8AHSxty8mzoyEEzN4gJKlh9sWWmAhDZSq9bdkyTAJt44bdMsCMsA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=f8U/gqFVS1QCBDY/iVkzXmoF1Dj4A8+V07OZpj6ShB8=; b=eN85Fi2FmRso12FrFE6LoMNCl2tQZSp/4w8wS8q1YAKDepZU6Gvnf59Ttxbh/k5BmieUOkQE/2YQZS6v/0eROtXkXDZOGFBYCAimBSREVzTCqsOb+kPwCeYpI4i/78bbWENCQyPz6gr8xpkc9UAvj1nuLz4VFsP+03ozABYgkG8= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by LV3PR10MB8129.namprd10.prod.outlook.com (2603:10b6:408:285::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.19; Tue, 3 Dec 2024 18:05:39 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%7]) with mapi id 15.20.8207.017; Tue, 3 Dec 2024 18:05:39 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , Eric Biederman , Kees Cook , Alexander Viro , Christian Brauner , Jan Kara , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 4/5] mm/vma: move stack expansion logic to mm/vma.c Date: Tue, 3 Dec 2024 18:05:11 +0000 Message-ID: <0feb104eff85922019d4fb29280f3afb130c5204.1733248985.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: X-ClientProxiedBy: LO2P265CA0450.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:e::30) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|LV3PR10MB8129:EE_ X-MS-Office365-Filtering-Correlation-Id: e2221735-4f24-4201-2249-08dd13c5183b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: mO/7ZWbxQ7ryjf1GR7njAiVkUTPrG7h08jewPkfQfeZU37TsdHMNG5C9fm8fkJwEu8X/Ud/ZLn3f/3Xkd85lry0M//G7nswKH6J9cC7oTSWeBeVw6oq3mmQlM/JGR2mlhxiL3usjaujqUhhEoAjguATMpPlxoxm0MBF8EE++xDzdNshg/g7CBsq3SVSDx/wpj11FpbS+58ixARWWekAgdXLTz/HLjQKf1N3X04aN1/6Zkn5XeyXwNHENAKMk4ChqKSL5CMYk1+OTk4ape2elLiPp4+b/KrOgZLXUACKM+Fg+qsFwlyxxO6oAFrInZCE9CNP4cb7rMQ6Xm6fA7nKkTCncv7zKN0KntoyMm95EPskdfAb5P89DJZPbDr2VaRPfmIlIEWTUBdevqbLcm4VKxcIkbjjpJdNR5RkG4fV4olQAsZA3lR0Of/fT9+PCIHiX0C3iEf714Y4rhQjHEwQJt8/uINygt0vLsC5xoYxH3S43zlbawB7ttE5kUvcLZ6l1ehBk1qCrNEvyJYOGS1bEUa+28DT00FP9Be697q+mq5hJ595I3wzWIjPVXBUzerllE7JdnG1wpq8C30XcymcRlBws19goL+PGuOrt1Owdqe4XF4tZo4QnWNmqZZ4J3pxyh8vOUjAsGt6VeM6FEnkglzPlBdk7ShXKYmw0VTntmZ3bbuyUSGZsmYNgXo+ojabydQRWinXTI5auht63L3+pDWT0WY/z+ZMVjlLYxaC9k7UwKEyi5cFVDwsrZDsZra15iN7qoUErXbhZSwPLoGquB7+fqiyOEgBNrqwQJAkN5i6jtqgVstWjX8BlxWNjNUWEvxiCleFiOXNd7Lxn0xKkCj88/ltjqm7f9onU3owi0WRwOUqEKxTxISiqqcZW5FwzXfINosec4oLL8KU2XfKgJMR35QCvpREC3b/mqL05B49ktjxXy2/cZcGaiU78cLtdzBx70coMNMPyIBWwp/pzPdC/dCJNTRPO+yo5iXFUernsWgtyzf9urj+CdTyKxKW1EOa9hHf5AGz4OtOiCFj4LLAlUJuxXve5WETwob3eMAPRbKsskUbXP7VwvwAAKX+6K6mmweKt3zKpC7Pez+diBnXUWC1UFzjMrIcbilAT3thvzU7cvssT7/64MuvQUlpnW8C0nVpO84RlIBgKNIL+WwAJV3wwQ9m7UdpSDTXBidOjGzcOrJN+IgdeEHVPEMs6bUwJhl8L+WnG5BNAEadfaQlXtrrj+mlu2ik+6r4HPhRKl4zpXod5JSnkPymswtsCk1aU92HbkTNtWm8VyYz9etIlf1UgdNDaSKeMfF/xX+m+ljOSg710NxDn/B5kJlcwCddiASTRCAd9D6jVfQqq7g== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(7416014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: AmVzT7wNXyzQGS0RivGuYDYOofCRdfRagh2NLJU0OoWkq5vUeAwCSkuQSfNKgEDfsL4Do0Pk9DiWmtLbNFP/Be9ne4sH/fiCIOeaUDPGwJX/bzR8rh9JN6dUARbYNHjoWpMi6/oS+GRPie6x48L+xQ/yVr2PLMqdUBac4bZEgXw/mgs+6ysqxXWBe3q46eKW0Fa7BdqE0clXRFKhWMrYubBKoMp7y1m8F7mBx7SRcnzWsZtOBmon+0y6DP4uqQfBKqoDKjCQAom1aHgf0juIm80VpHRg48XM4EItVbv7LgwwaJ1Rq5JnEcrbwSRarx0kXhE3uNVBlMRN6Zh4asab4Ged45j/vsJqff5QppOvo6bhEC4C6gqS37qwxR17toguElzegWHgjzxh3F6M4VsRcGLGZyJKDUQwWMcPuWrNEd4MvXyiUeHRnBtUNS2B6lS8muBc6qFl7+jC+a5dCSGYqWZUEYWgiN5HJAcKftzhuq/uLpO75Es0+HwkMHsI3pz5Gi/xZf50vN/4uSiS5idOBzg73m8SUoLh8g267GImc/xe0C67utfU5ytSyGapa5N6f6qld7qHF4T8Ke/0WyNgvrljcg4cS0a0ZNKuOMtQWBPpUhaLEsdg4kRD8RQfvj+7JYrPiM2XA/MWCbiGC0T/9oAMPtbRfWCXX/dUUFoU+evs3UhlSy1ANVCamJYh5SI/bnVvIqi8mVs+INrezYpk0w7PxthUoo6nv0gbjCiH401QDIUwFlmNx0l4np9wg8HqPS2L72gKrc11UDNFvrP9vPu6KFV6giP4teOzsWIOB+oPoN88LCD4FKvlZ7YtWgTcd7ID+37dwpEknJER/y+sGlsrS66nkLaPcoZOJ6PHipbd3s8mYeIqN8mDGWz4uwRlD987YP628VSDdF/AJIwnM+6h+TdcFxoA5LXxACSNl42NAN6W8EafFfKWLVM9IwOqEN9wofKv9Lm7NRwEiCA3C3ZhaswzEQFgJUormNDFThSzana4+s8fLrnlBMYyVElsBcpIDYYLuFGkLqh1NEOlat2jbzAdk2RkH1BTjJBLHQoTPx+8UJl9r8cVDIKvDhVGzBSg/mBKkNsY3vP/Hk2WGaOtzO1TVUuniAfQBVWdUrXeGT1JgiLlfqA80ELl81B5B32CMPv4+Udo/x2N7BVagiS+ILGiHlj4x9N64xhq+O/adrjjY5HlTzTdXkzfRmZqtq5hIX59vd3QFJy/pxRS4xgyB7zwV2SdNxs/b6u54N+3LVRA+a7coB8rWhQyqDEJYt5s1lrd1P9QOB3lIvB7p2oMSk6r5YE7kgxaz3/Uedk+9VzuHQ/hr+onMkooS7Y+em2kph+C4HH7ldcEhZaYgY9oJ3hoOOYlK3iaDEkGakwRPanJmyusPsnzNBn+wcTOUcaE7+jXEWKtLJ0kQH8fZ6lQ/hwzSCq47PeaWqPsXcqcK54fFZXqwYJbBLUtMvOt8ZBFs4Z0OCOi9OJSoPZ/lJL4VHTKGadXsc5SOLVMmLXE1LVpzccPWHOPGSd0CwCWktgeI5C0aDPPiNJvENEK0CimJCE6Ij8k1UdjKQ5Qa5po4/WvSr4cs3FEL3lm4/XqOYkRWRF7oKc86S4HojARvg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: oSQLbxK4ZqvrEKo8LFQHwihuY/zkIJlrTfcy/n7DssK+FX9fCL/TIk9ybWTfc5//CQ/cV2wuDsPk3eBNdM6MyV30B+KOnHDcxl5BAtobn+zycoo7ERpkyTDX69QCDVtv5g495CfXqXp792PMYkhy8cVvDp9EzzR5r2G2n6FagUILagUpdyPepOuTrCb+H1WcJFsB/dp7pd9wVw/TfMgNdcY8474W8UiYFtyVl9Od2T68HJ5CPSMDwkdfsC2t8xSEtrXl7rsddARco5nEuEpviY5fuB+c4BpjDCpBXAYKihzzIWL+P2nfzrrc4Uzoy09vr5xHGiwoDGjGRjePwguzxiCStt5kIujPsEK+aBwo0teMXjvWF7W9/EF57GM+MMcsei2P0Obf4ltuwS1rnHEaz0YR4GgMpMgXJZEt3bYjUQ9KhzdqnTPhbsnsaM6Ab5MAoAPeopOAs+JpVJc106IzgO8Ugm3snkGRB/EZFlT1fp1VXOSWeNv1HKiDyFvPlqSl5lS+ml75i+nUvKQjnqmDnurJAzp88PmpRNj2M6njo1GnlLGwib9FgeV17gjV+aTBe4aRPBVl4KydgeSbRSQ7PH+8Kb6IjrAkpvpfc3NkY6Q= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e2221735-4f24-4201-2249-08dd13c5183b X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Dec 2024 18:05:39.4078 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tZScrNk+VWbsJY5Gs9JK8mI4Ma/+cdQL96X2g9j/OLPAzvk0qtdOdiW4SbF2j0s+B2Tc8J4si4xVAZQMjpgndO6+/Mx48Geh4wX2dRsx1OA= X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR10MB8129 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-03_06,2024-12-03_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 phishscore=0 suspectscore=0 malwarescore=0 bulkscore=0 spamscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412030151 X-Proofpoint-GUID: slcM9c0nE1THh8PRMfCu9KscHYkJdCbe X-Proofpoint-ORIG-GUID: slcM9c0nE1THh8PRMfCu9KscHYkJdCbe We build on previous work making expand_downwards() an entirely internal function. This logic is subtle and so it is highly useful to get it into vma.c so we can then userland unit test. We must additionally move acct_stack_growth() to vma.c as it is a helper function used by both expand_downwards() and expand_upwards(). We are also then able to mark anon_vma_interval_tree_pre_update_vma() and anon_vma_interval_tree_post_update_vma() static as these are no longer used by anything else. Signed-off-by: Lorenzo Stoakes --- mm/mmap.c | 205 ----------------------- mm/vma.c | 269 +++++++++++++++++++++++++++---- mm/vma.h | 12 +- tools/testing/vma/vma.c | 5 + tools/testing/vma/vma_internal.h | 62 +++++++ 5 files changed, 310 insertions(+), 243 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 4df38d3717ff..55a8f2332b7c 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -879,211 +879,6 @@ find_vma_prev(struct mm_struct *mm, unsigned long addr, return vma; } -/* - * Verify that the stack growth is acceptable and - * update accounting. This is shared with both the - * grow-up and grow-down cases. - */ -static int acct_stack_growth(struct vm_area_struct *vma, - unsigned long size, unsigned long grow) -{ - struct mm_struct *mm = vma->vm_mm; - unsigned long new_start; - - /* address space limit tests */ - if (!may_expand_vm(mm, vma->vm_flags, grow)) - return -ENOMEM; - - /* Stack limit test */ - if (size > rlimit(RLIMIT_STACK)) - return -ENOMEM; - - /* mlock limit tests */ - if (!mlock_future_ok(mm, vma->vm_flags, grow << PAGE_SHIFT)) - return -ENOMEM; - - /* Check to ensure the stack will not grow into a hugetlb-only region */ - new_start = (vma->vm_flags & VM_GROWSUP) ? vma->vm_start : - vma->vm_end - size; - if (is_hugepage_only_range(vma->vm_mm, new_start, size)) - return -EFAULT; - - /* - * Overcommit.. This must be the final test, as it will - * update security statistics. - */ - if (security_vm_enough_memory_mm(mm, grow)) - return -ENOMEM; - - return 0; -} - -#if defined(CONFIG_STACK_GROWSUP) -/* - * PA-RISC uses this for its stack. - * vma is the last one with address > vma->vm_end. Have to extend vma. - */ -static int expand_upwards(struct vm_area_struct *vma, unsigned long address) -{ - struct mm_struct *mm = vma->vm_mm; - struct vm_area_struct *next; - unsigned long gap_addr; - int error = 0; - VMA_ITERATOR(vmi, mm, vma->vm_start); - - if (!(vma->vm_flags & VM_GROWSUP)) - return -EFAULT; - - mmap_assert_write_locked(mm); - - /* Guard against exceeding limits of the address space. */ - address &= PAGE_MASK; - if (address >= (TASK_SIZE & PAGE_MASK)) - return -ENOMEM; - address += PAGE_SIZE; - - /* Enforce stack_guard_gap */ - gap_addr = address + stack_guard_gap; - - /* Guard against overflow */ - if (gap_addr < address || gap_addr > TASK_SIZE) - gap_addr = TASK_SIZE; - - next = find_vma_intersection(mm, vma->vm_end, gap_addr); - if (next && vma_is_accessible(next)) { - if (!(next->vm_flags & VM_GROWSUP)) - return -ENOMEM; - /* Check that both stack segments have the same anon_vma? */ - } - - if (next) - vma_iter_prev_range_limit(&vmi, address); - - vma_iter_config(&vmi, vma->vm_start, address); - if (vma_iter_prealloc(&vmi, vma)) - return -ENOMEM; - - /* We must make sure the anon_vma is allocated. */ - if (unlikely(anon_vma_prepare(vma))) { - vma_iter_free(&vmi); - return -ENOMEM; - } - - /* Lock the VMA before expanding to prevent concurrent page faults */ - vma_start_write(vma); - /* We update the anon VMA tree. */ - anon_vma_lock_write(vma->anon_vma); - - /* Somebody else might have raced and expanded it already */ - if (address > vma->vm_end) { - unsigned long size, grow; - - size = address - vma->vm_start; - grow = (address - vma->vm_end) >> PAGE_SHIFT; - - error = -ENOMEM; - if (vma->vm_pgoff + (size >> PAGE_SHIFT) >= vma->vm_pgoff) { - error = acct_stack_growth(vma, size, grow); - if (!error) { - if (vma->vm_flags & VM_LOCKED) - mm->locked_vm += grow; - vm_stat_account(mm, vma->vm_flags, grow); - anon_vma_interval_tree_pre_update_vma(vma); - vma->vm_end = address; - /* Overwrite old entry in mtree. */ - vma_iter_store(&vmi, vma); - anon_vma_interval_tree_post_update_vma(vma); - - perf_event_mmap(vma); - } - } - } - anon_vma_unlock_write(vma->anon_vma); - vma_iter_free(&vmi); - validate_mm(mm); - return error; -} -#endif /* CONFIG_STACK_GROWSUP */ - -/* - * vma is the first one with address < vma->vm_start. Have to extend vma. - * mmap_lock held for writing. - */ -static int expand_downwards(struct vm_area_struct *vma, unsigned long address) -{ - struct mm_struct *mm = vma->vm_mm; - struct vm_area_struct *prev; - int error = 0; - VMA_ITERATOR(vmi, mm, vma->vm_start); - - if (!(vma->vm_flags & VM_GROWSDOWN)) - return -EFAULT; - - mmap_assert_write_locked(mm); - - address &= PAGE_MASK; - if (address < mmap_min_addr || address < FIRST_USER_ADDRESS) - return -EPERM; - - /* Enforce stack_guard_gap */ - prev = vma_prev(&vmi); - /* Check that both stack segments have the same anon_vma? */ - if (prev) { - if (!(prev->vm_flags & VM_GROWSDOWN) && - vma_is_accessible(prev) && - (address - prev->vm_end < stack_guard_gap)) - return -ENOMEM; - } - - if (prev) - vma_iter_next_range_limit(&vmi, vma->vm_start); - - vma_iter_config(&vmi, address, vma->vm_end); - if (vma_iter_prealloc(&vmi, vma)) - return -ENOMEM; - - /* We must make sure the anon_vma is allocated. */ - if (unlikely(anon_vma_prepare(vma))) { - vma_iter_free(&vmi); - return -ENOMEM; - } - - /* Lock the VMA before expanding to prevent concurrent page faults */ - vma_start_write(vma); - /* We update the anon VMA tree. */ - anon_vma_lock_write(vma->anon_vma); - - /* Somebody else might have raced and expanded it already */ - if (address < vma->vm_start) { - unsigned long size, grow; - - size = vma->vm_end - address; - grow = (vma->vm_start - address) >> PAGE_SHIFT; - - error = -ENOMEM; - if (grow <= vma->vm_pgoff) { - error = acct_stack_growth(vma, size, grow); - if (!error) { - if (vma->vm_flags & VM_LOCKED) - mm->locked_vm += grow; - vm_stat_account(mm, vma->vm_flags, grow); - anon_vma_interval_tree_pre_update_vma(vma); - vma->vm_start = address; - vma->vm_pgoff -= grow; - /* Overwrite old entry in mtree. */ - vma_iter_store(&vmi, vma); - anon_vma_interval_tree_post_update_vma(vma); - - perf_event_mmap(vma); - } - } - } - anon_vma_unlock_write(vma->anon_vma); - vma_iter_free(&vmi); - validate_mm(mm); - return error; -} - /* enforced gap between the expanding stack and other mappings. */ unsigned long stack_guard_gap = 256UL<anon_vma_chain, same_vma) + anon_vma_interval_tree_remove(avc, &avc->anon_vma->rb_root); +} + +static void +anon_vma_interval_tree_post_update_vma(struct vm_area_struct *vma) +{ + struct anon_vma_chain *avc; + + list_for_each_entry(avc, &vma->anon_vma_chain, same_vma) + anon_vma_interval_tree_insert(avc, &avc->anon_vma->rb_root); +} + /* * vma_prepare() - Helper function for handling locking VMAs prior to altering * @vp: The initialized vma_prepare struct @@ -510,38 +542,6 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, return __split_vma(vmi, vma, addr, new_below); } -/* - * vma has some anon_vma assigned, and is already inserted on that - * anon_vma's interval trees. - * - * Before updating the vma's vm_start / vm_end / vm_pgoff fields, the - * vma must be removed from the anon_vma's interval trees using - * anon_vma_interval_tree_pre_update_vma(). - * - * After the update, the vma will be reinserted using - * anon_vma_interval_tree_post_update_vma(). - * - * The entire update must be protected by exclusive mmap_lock and by - * the root anon_vma's mutex. - */ -void -anon_vma_interval_tree_pre_update_vma(struct vm_area_struct *vma) -{ - struct anon_vma_chain *avc; - - list_for_each_entry(avc, &vma->anon_vma_chain, same_vma) - anon_vma_interval_tree_remove(avc, &avc->anon_vma->rb_root); -} - -void -anon_vma_interval_tree_post_update_vma(struct vm_area_struct *vma) -{ - struct anon_vma_chain *avc; - - list_for_each_entry(avc, &vma->anon_vma_chain, same_vma) - anon_vma_interval_tree_insert(avc, &avc->anon_vma->rb_root); -} - /* * dup_anon_vma() - Helper function to duplicate anon_vma * @dst: The destination VMA @@ -2669,3 +2669,208 @@ unsigned long unmapped_area_topdown(struct vm_unmapped_area_info *info) return gap; } + +/* + * Verify that the stack growth is acceptable and + * update accounting. This is shared with both the + * grow-up and grow-down cases. + */ +static int acct_stack_growth(struct vm_area_struct *vma, + unsigned long size, unsigned long grow) +{ + struct mm_struct *mm = vma->vm_mm; + unsigned long new_start; + + /* address space limit tests */ + if (!may_expand_vm(mm, vma->vm_flags, grow)) + return -ENOMEM; + + /* Stack limit test */ + if (size > rlimit(RLIMIT_STACK)) + return -ENOMEM; + + /* mlock limit tests */ + if (!mlock_future_ok(mm, vma->vm_flags, grow << PAGE_SHIFT)) + return -ENOMEM; + + /* Check to ensure the stack will not grow into a hugetlb-only region */ + new_start = (vma->vm_flags & VM_GROWSUP) ? vma->vm_start : + vma->vm_end - size; + if (is_hugepage_only_range(vma->vm_mm, new_start, size)) + return -EFAULT; + + /* + * Overcommit.. This must be the final test, as it will + * update security statistics. + */ + if (security_vm_enough_memory_mm(mm, grow)) + return -ENOMEM; + + return 0; +} + +#if defined(CONFIG_STACK_GROWSUP) +/* + * PA-RISC uses this for its stack. + * vma is the last one with address > vma->vm_end. Have to extend vma. + */ +int expand_upwards(struct vm_area_struct *vma, unsigned long address) +{ + struct mm_struct *mm = vma->vm_mm; + struct vm_area_struct *next; + unsigned long gap_addr; + int error = 0; + VMA_ITERATOR(vmi, mm, vma->vm_start); + + if (!(vma->vm_flags & VM_GROWSUP)) + return -EFAULT; + + mmap_assert_write_locked(mm); + + /* Guard against exceeding limits of the address space. */ + address &= PAGE_MASK; + if (address >= (TASK_SIZE & PAGE_MASK)) + return -ENOMEM; + address += PAGE_SIZE; + + /* Enforce stack_guard_gap */ + gap_addr = address + stack_guard_gap; + + /* Guard against overflow */ + if (gap_addr < address || gap_addr > TASK_SIZE) + gap_addr = TASK_SIZE; + + next = find_vma_intersection(mm, vma->vm_end, gap_addr); + if (next && vma_is_accessible(next)) { + if (!(next->vm_flags & VM_GROWSUP)) + return -ENOMEM; + /* Check that both stack segments have the same anon_vma? */ + } + + if (next) + vma_iter_prev_range_limit(&vmi, address); + + vma_iter_config(&vmi, vma->vm_start, address); + if (vma_iter_prealloc(&vmi, vma)) + return -ENOMEM; + + /* We must make sure the anon_vma is allocated. */ + if (unlikely(anon_vma_prepare(vma))) { + vma_iter_free(&vmi); + return -ENOMEM; + } + + /* Lock the VMA before expanding to prevent concurrent page faults */ + vma_start_write(vma); + /* We update the anon VMA tree. */ + anon_vma_lock_write(vma->anon_vma); + + /* Somebody else might have raced and expanded it already */ + if (address > vma->vm_end) { + unsigned long size, grow; + + size = address - vma->vm_start; + grow = (address - vma->vm_end) >> PAGE_SHIFT; + + error = -ENOMEM; + if (vma->vm_pgoff + (size >> PAGE_SHIFT) >= vma->vm_pgoff) { + error = acct_stack_growth(vma, size, grow); + if (!error) { + if (vma->vm_flags & VM_LOCKED) + mm->locked_vm += grow; + vm_stat_account(mm, vma->vm_flags, grow); + anon_vma_interval_tree_pre_update_vma(vma); + vma->vm_end = address; + /* Overwrite old entry in mtree. */ + vma_iter_store(&vmi, vma); + anon_vma_interval_tree_post_update_vma(vma); + + perf_event_mmap(vma); + } + } + } + anon_vma_unlock_write(vma->anon_vma); + vma_iter_free(&vmi); + validate_mm(mm); + return error; +} +#endif /* CONFIG_STACK_GROWSUP */ + +/* + * vma is the first one with address < vma->vm_start. Have to extend vma. + * mmap_lock held for writing. + */ +int expand_downwards(struct vm_area_struct *vma, unsigned long address) +{ + struct mm_struct *mm = vma->vm_mm; + struct vm_area_struct *prev; + int error = 0; + VMA_ITERATOR(vmi, mm, vma->vm_start); + + if (!(vma->vm_flags & VM_GROWSDOWN)) + return -EFAULT; + + mmap_assert_write_locked(mm); + + address &= PAGE_MASK; + if (address < mmap_min_addr || address < FIRST_USER_ADDRESS) + return -EPERM; + + /* Enforce stack_guard_gap */ + prev = vma_prev(&vmi); + /* Check that both stack segments have the same anon_vma? */ + if (prev) { + if (!(prev->vm_flags & VM_GROWSDOWN) && + vma_is_accessible(prev) && + (address - prev->vm_end < stack_guard_gap)) + return -ENOMEM; + } + + if (prev) + vma_iter_next_range_limit(&vmi, vma->vm_start); + + vma_iter_config(&vmi, address, vma->vm_end); + if (vma_iter_prealloc(&vmi, vma)) + return -ENOMEM; + + /* We must make sure the anon_vma is allocated. */ + if (unlikely(anon_vma_prepare(vma))) { + vma_iter_free(&vmi); + return -ENOMEM; + } + + /* Lock the VMA before expanding to prevent concurrent page faults */ + vma_start_write(vma); + /* We update the anon VMA tree. */ + anon_vma_lock_write(vma->anon_vma); + + /* Somebody else might have raced and expanded it already */ + if (address < vma->vm_start) { + unsigned long size, grow; + + size = vma->vm_end - address; + grow = (vma->vm_start - address) >> PAGE_SHIFT; + + error = -ENOMEM; + if (grow <= vma->vm_pgoff) { + error = acct_stack_growth(vma, size, grow); + if (!error) { + if (vma->vm_flags & VM_LOCKED) + mm->locked_vm += grow; + vm_stat_account(mm, vma->vm_flags, grow); + anon_vma_interval_tree_pre_update_vma(vma); + vma->vm_start = address; + vma->vm_pgoff -= grow; + /* Overwrite old entry in mtree. */ + vma_iter_store(&vmi, vma); + anon_vma_interval_tree_post_update_vma(vma); + + perf_event_mmap(vma); + } + } + } + anon_vma_unlock_write(vma->anon_vma); + vma_iter_free(&vmi); + validate_mm(mm); + return error; +} diff --git a/mm/vma.h b/mm/vma.h index c60f37d89eb1..6c460a120f82 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -139,12 +139,6 @@ void validate_mm(struct mm_struct *mm); #define validate_mm(mm) do { } while (0) #endif -/* Required for expand_downwards(). */ -void anon_vma_interval_tree_pre_update_vma(struct vm_area_struct *vma); - -/* Required for expand_downwards(). */ -void anon_vma_interval_tree_post_update_vma(struct vm_area_struct *vma); - int vma_expand(struct vma_merge_struct *vmg); int vma_shrink(struct vma_iterator *vmi, struct vm_area_struct *vma, unsigned long start, unsigned long end, pgoff_t pgoff); @@ -478,4 +472,10 @@ static inline bool can_modify_vma_madv(struct vm_area_struct *vma, int behavior) #endif +#if defined(CONFIG_STACK_GROWSUP) +int expand_upwards(struct vm_area_struct *vma, unsigned long address); +#endif + +int expand_downwards(struct vm_area_struct *vma, unsigned long address); + #endif /* __MM_VMA_H */ diff --git a/tools/testing/vma/vma.c b/tools/testing/vma/vma.c index 39ee61e55634..891d87a9ad6b 100644 --- a/tools/testing/vma/vma.c +++ b/tools/testing/vma/vma.c @@ -53,6 +53,11 @@ struct task_struct *get_current(void) return &__current; } +unsigned long rlimit(unsigned int limit) +{ + return (unsigned long)-1; +} + /* Helper function to simply allocate a VMA. */ static struct vm_area_struct *alloc_vma(struct mm_struct *mm, unsigned long start, diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h index 6ad8bd8edaad..fab3f3bdf2f0 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -79,6 +79,11 @@ extern unsigned long dac_mmap_min_addr; #define VM_STARTGAP_FLAGS (VM_GROWSDOWN | VM_SHADOW_STACK) +#define RLIMIT_STACK 3 /* max stack size */ +#define RLIMIT_MEMLOCK 8 /* max locked-in-memory address space */ + +#define CAP_IPC_LOCK 14 + #ifdef CONFIG_64BIT /* VM is sealed, in vm_flags */ #define VM_SEALED _BITUL(63) @@ -478,6 +483,8 @@ static inline void vma_mark_detached(struct vm_area_struct *vma, bool detached) extern const struct vm_operations_struct vma_dummy_vm_ops; +extern unsigned long rlimit(unsigned int limit); + static inline void vma_init(struct vm_area_struct *vma, struct mm_struct *mm) { memset(vma, 0, sizeof(*vma)); @@ -1114,4 +1121,59 @@ static inline unsigned long vm_end_gap(struct vm_area_struct *vma) return vm_end; } +static inline int is_hugepage_only_range(struct mm_struct *mm, + unsigned long addr, unsigned long len) +{ + return 0; +} + +static inline bool vma_is_accessible(struct vm_area_struct *vma) +{ + return vma->vm_flags & VM_ACCESS_FLAGS; +} + +static inline bool capable(int cap) +{ + return true; +} + +static inline bool mlock_future_ok(struct mm_struct *mm, unsigned long flags, + unsigned long bytes) +{ + unsigned long locked_pages, limit_pages; + + if (!(flags & VM_LOCKED) || capable(CAP_IPC_LOCK)) + return true; + + locked_pages = bytes >> PAGE_SHIFT; + locked_pages += mm->locked_vm; + + limit_pages = rlimit(RLIMIT_MEMLOCK); + limit_pages >>= PAGE_SHIFT; + + return locked_pages <= limit_pages; +} + +static inline int __anon_vma_prepare(struct vm_area_struct *vma) +{ + struct anon_vma *anon_vma = calloc(1, sizeof(struct anon_vma)); + + if (!anon_vma) + return -ENOMEM; + + anon_vma->root = anon_vma; + vma->anon_vma = anon_vma; + + return 0; +} + +static inline int anon_vma_prepare(struct vm_area_struct *vma) +{ + if (likely(vma->anon_vma)) + return 0; + + return __anon_vma_prepare(vma); +} + + #endif /* __MM_VMA_INTERNAL_H */ From patchwork Tue Dec 3 18:05:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Lorenzo Stoakes X-Patchwork-Id: 13892789 Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id E02D11FC7E3; Tue, 3 Dec 2024 18:05:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=205.220.177.32 ARC-Seal: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249159; cv=fail; b=nnpwT9E+8rZJujSHNn8FYqzbjU9ZZIKUGf5YWcjgDsO9XCFb2pbZ1ofoNozAvaEP+gs/G19USp+P047/FbDP7d6JR2UiOJHxc+RZ8a7u1srPJuXpXSf9tdmn4Qz4NuVy1HuhwiUVe2PC9ulUEgdyGIqJelD5dXA2Y1z3TeW1ETQ= ARC-Message-Signature: i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1733249159; c=relaxed/simple; bh=iaclXEsnfKTcPzoSuZrfKW1NEh8nEnC1PiF9+8LHBtU=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: Content-Type:MIME-Version; b=ZvqUChL3bO8CtloYaX+zLuGT+wkWoBNnooOiY1UfK7oCTa0/2FI8PQFKltyVrGWC7XtpLP4yyRN98/zm2sc8uhN+/2hCTaMxBurfmm4+aFSkfrfwwCNfHUckMoBJzF8WDpZW44UVUp5rOwqyfste0Lb0iAhV/ktJLoQpjws221w= ARC-Authentication-Results: i=2; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com; spf=pass smtp.mailfrom=oracle.com; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b=X3KZS1Yb; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b=PM8X86Qp; arc=fail smtp.client-ip=205.220.177.32 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=oracle.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=oracle.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=oracle.com header.i=@oracle.com header.b="X3KZS1Yb"; dkim=pass (1024-bit key) header.d=oracle.onmicrosoft.com header.i=@oracle.onmicrosoft.com header.b="PM8X86Qp" Received: from pps.filterd (m0246630.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3HtaQP012433; Tue, 3 Dec 2024 18:05:45 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= corp-2023-11-20; bh=h+C0HsktFzmPVDF+PnzGjib7ioZoAwoaSTtHI6BxIYI=; b= X3KZS1YbSPlN+2WPa1IQW4OGpkOMFmcNj/4gp0EHFLGVmTpCPtKezg3E9qFdKaNl dCB6A/SMefDnzytFF10iybdVgtB43UKY2XgVuRF+rQu1/6dU8wII1pL9U2OkpMHE mWUZHTxC12zucPDbyM5+cJPCW5edM7uKfTGk6VEYkELpedtwMz6ylHMyBlOiYSug j1lgnUScAE4KK/uXE8/D/BqIQgULqFnPVsEmzZOOjlgNboK4QAPY+Cin8lVVyOIB uYd5X35viTw7JCBqVQs3EDyWewZfTxA0OCmtRziWf1R1Wckw1IJTKRvlQwJMUl/7 Sgbw9Lzi3a5fdqPr8CyAhA== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 437s4c6nss-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:45 +0000 (GMT) Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.18.1.2/8.18.1.2) with ESMTP id 4B3HL0w1030979; Tue, 3 Dec 2024 18:05:44 GMT Received: from nam10-bn7-obe.outbound.protection.outlook.com (mail-bn7nam10lp2049.outbound.protection.outlook.com [104.47.70.49]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 437wjcuq4m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 03 Dec 2024 18:05:44 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Bn4EvTK9Y85bRMFRnQuqhHeoS8GaepPnz5wRlsPmTCRzTAeUXXwEqGFTXYG7nnbQBxqIv1q6GPOaFQsRJM8DTtjUFIxr0BKD5BuvBczfDnuVeoj1PZiFvwg1Iz5ZqKvnCu0eE5hOGUj33YrRZ96wTkygQyhLd5V0m+2EJsV2UFR9ex2Elt4MiNLDRKiC44Y4c7EZl6KyJP+gdBZt37AFDy4y2wWqbmIz2u5kvwqoTkSdB9R+dtxQVGbaRIfn74DiGJMxPfrk9I5IBKiCAGUhHQCGZRhwUYg7pA83aS1bnBOMjbmOTUH5KgmfPXRvXJUACgMIYKUj7VcbSSh7s6ShMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=h+C0HsktFzmPVDF+PnzGjib7ioZoAwoaSTtHI6BxIYI=; b=kPGoiR6MhqWostkoxqzBW0YyxeqD067PdRE85Rgeqqd6agrLR00eRdpIPCORaUUwM1JqZPIdSXdsZ4cSTfo1d6sMBqe9i2r80WanlCFL57pGbGvajgtlaNnt5ww3IOYXjGzrinEjrIls6ALFULZiv88ku+JrTDtuLaMWunZ6CQ7dNqAsoRHrkeHUXZJf/txh6G9MBEyC+5qVLXHEhYONdYEjGA0vT0PSPo58OV0X1fkQBiRyG+8OMmLRzkmrxS/2W9sw05Ky2XISQjHU2la+5psUaK/oKuvuWhQJVR+7VJVQhFg6H2ZS/EZUGbHt0AAv6D/MEQTf67hQHVfKVrWNAA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h+C0HsktFzmPVDF+PnzGjib7ioZoAwoaSTtHI6BxIYI=; b=PM8X86Qph/GITLYjFkuNQaWZCGHeRmvmIkcN7X9CXzYWgIeur/iAE7B1kS29PGb08+kdVDkKsTCyFSKfEm2QaovKaF79eYTbCudcusiPBSTX0z+Kxlqyrysq/akm6iiNqah4NHeZtPnexH/cmSsPmIEIJ0E4bdVLm74x/FaW2oo= Received: from BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) by LV3PR10MB8129.namprd10.prod.outlook.com (2603:10b6:408:285::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8207.19; Tue, 3 Dec 2024 18:05:42 +0000 Received: from BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9]) by BYAPR10MB3366.namprd10.prod.outlook.com ([fe80::baf2:dff1:d471:1c9%7]) with mapi id 15.20.8207.017; Tue, 3 Dec 2024 18:05:42 +0000 From: Lorenzo Stoakes To: Andrew Morton Cc: "Liam R . Howlett" , Vlastimil Babka , Jann Horn , Eric Biederman , Kees Cook , Alexander Viro , Christian Brauner , Jan Kara , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH 5/5] mm/vma: move __vm_munmap() to mm/vma.c Date: Tue, 3 Dec 2024 18:05:12 +0000 Message-ID: <5e5e81807c54dfbe363edb2d431eb3d7a37fcdba.1733248985.git.lorenzo.stoakes@oracle.com> X-Mailer: git-send-email 2.47.1 In-Reply-To: References: X-ClientProxiedBy: LO2P265CA0336.GBRP265.PROD.OUTLOOK.COM (2603:10a6:600:a4::36) To BYAPR10MB3366.namprd10.prod.outlook.com (2603:10b6:a03:14f::25) Precedence: bulk X-Mailing-List: linux-fsdevel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR10MB3366:EE_|LV3PR10MB8129:EE_ X-MS-Office365-Filtering-Correlation-Id: 3e767d7c-4787-4440-64a5-08dd13c51a0e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|376014|7416014; X-Microsoft-Antispam-Message-Info: 8YEm0AOYG1ZOkUJEdyMkJJFfA1h+pwppxEAMf+1NmbrpJbt3APGUYiV7PRZenc/c+1Sl0Qylu55Dz16Z2bAM+2yDaSpsBcHD4CNuDGp1ZATw4OKmUdz8FyM/1E1U//btC4iTGB6kKw3oT6ujMUDARORl70IX41AY+UOQtC0yuk5XpeOrFNTAw8iJPiF4jKCWqQF+V8vI8nQMmSe7uH/4xhmB6SLfUmPlYtDr2Bq9t19fnGPhCB4FBNvyHHWwI0n/Pb6VQu3zkSUu55K3iz8/o/oFYu0ymQIoxTknwQG5aY1wAzRm9JTMm8r3FzAogi13LnmeA9gKQ+pzpc58G+a69OIhH5hwy0rkt2zIp9PCcMXH35nylgiFxIjSruSQI/9wYgN3kfGF6s3by0Iuj6Xspg4k8L93B1PGdFFGU073x4M6U2QuA8o8kRZTFLlRqLTGzowGvPueNc6QN4l4cnt0/X0QoQb6l/qfyVE2d4b4pD7FaWr+V03vIEZ5qly+LRQyrk1L8vruXHXN69WN0Vt5kNc6y2haWkUfvnYV4/0BRBxOQql4x1nOIw2gfAchWhOg1zI8su9kbnFLbch6YHXU1QDK0Y2iQpI/JBJYrQvUMz+aoxgBbx3Mbc31ZJMxxnAPdHBlhD1udZtAncURgmrfhoqQiBmu2//gmGlZGQ4WAdoBkngDgM9hqQGvU8esw0Q7WanCFvD6pefr+BQEQ3MuPcvwn0eVFA9KYhkJTnawJ7qZMRm0lMyNiGDh0f1lJ4vE/hdTqLLt6WbYpu56uNXFKEcEzH/GENyTeeX1nlDA+DEgfHhlL9pbLhZBJqon7rlblFaSQjDREmCyAFmOsFKPRhToU276aYzSZGg6Ycxbca/V2dCbLOdnTcbqvXwfq9QIOH0gQ36JIde2jQG+7dv8zxZOUtGS/I0470hLPkbdpj41TwRGCSbhE0L9o1jOWEd/rr9AhSKiOBvGHH4e76g2qwdeqw303CRM9cfFZ+hN10Fqo+DDvbqy2A0orZiMJGlNODW6K3PqLWtHL/sKHt2M3VOMh1McndeA2Ol9+JfVmovnjicG8q1fL3jTHwm3mo1bJk8XOYNHnEX60r31KUkfJAr6ZbjfIHaRk7FFJ7TsBTCqyU0bXO4+VND8mbR9EtlBriBJbTqPdTG7nYpgOdldhqG0tfNkZLg+K8YuUnt+HpvNU9Kx++eW2SPSduoOhRKbsA8jT8FQxtRUopkAiEiYqh2loN/3Hhc9IWqzwMbCks9mgyAAIG0GVTseLXNquUw3xEPAEGn4j8y7zeSSw1nDegVamYBVEVdhRNsqwK6QRpvxv2AYmfhHObz/8RVhKsNs6RmbqDwINOdUqLmMTfvOvw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR10MB3366.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(376014)(7416014);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: aWCvxsMGckTL+VvGmXxRiI+uK8u3iAhG+GOsABdIZQhWj4sPlYcwEDqLpK1hhkx2btqMzj/x1l3dhiBZy/ENtZeogukvdNXbCf9+3OIkIHj6WxuIl+NPTPttDFnR00payfMUKok45bsOBdC1pcXvCwYMP5wwAFlPcgfMccuYhNPBJLV165MHBFrj3u9naNh9vAwsuhgE8Mm+JCPNTj2EK/hVIlY8EzqyO7d7WC02kYq4bSKWFCRWh8MKcypldRfp6Eu3FzjhwhIZNnTMIrQG0lI8PxSG08KIuPZFl+pKN+0nmEo3dpeajD5++6j9wiBC6Tfde32HXuPU+Y5mM0ljdMTCEYmra5QBVvJpTC5reiFBXymdbhzjdG9Rp/M2YJS9uzQglEW64kR5HrcS3tptE9Ua1HCf5dbJgwwjfM3qjvyyrHQByFUxxNz8pPNMA8hAU2t3+W75/KrLpUgLSN/k+FQX2iJLrtD74gT9IdSGqP9AhYyl9o+PoqzvB9QK8iA6Z3w/6/NAdbjpvZhSnFjzhUkX6Uq13RlcUllivuqV8xGX3BT2AgXTYN/fJHzNUmzwEtPqtaJLmswGjSotI1MDwsDGj1rmbtZEllVlH7dMopuvv5vUR0xy/rCAWyWeEE5YG5YNY7vQ2HTPAJXSFpSQROp4vAVEpM4ZcS+1YXhUmfV2720cYgrYDGdHRxPk2/wpP6flKMZ8b/db2AJeJPXTI6LWgR2JArUEC2SJ7gLF6ti734WeaoEJ1mN8tsJ4o16wUDuI1X/ytsGunAsr7w6DycEMb1eaG81+aXII4b5PXc9S0umIonNDlAfGaJsxGZwtGedDPzcmb0VwMI7kTL97oo3ofLwrmF9kCuvKr/62THBS/GB4rf6fnvErQVVTeZsL/aQ5VptgwhBlKxfrLUBM1hglbu5vd2nxMmrx7+dvY7cs2uUK5muYcrWHHnpKa1snPWqxyxpcu9Eu8Qr7vmyH6JeqejXEdoxJs3Be6otXcHrkqsX5BMF/OivQ+eGydtNYATndt7hbrRMACK1X5QkvY0iun4cnRhfN24lQ8c1ctBKLwNC9LTKIDgBS0i3qQx/UcdP2S3/KQ65203iFbSiWB3WXGEiXT8IG7VYCwkzS/n6YEpbqRyYQzmZb0Un5NjxB6EaRmzXZB9/24fqcuCOh3ujQ8N5uVc3snwOS9e+zMUwqmYlEVbvUbESuCqR7uTyAdRXxFZUMgufNgkyvcF3qkiz45TJ8vs2V+2p5gVi+gum657CS8Z6MteZH5evCnCOE7hRt+3GjwvhXNo7rWvkNphUoYHPc2/oWZTh4SG5BtrT7NoHS9tjdZEQws7uPKpI6ilgGwaCqwxxsDUbDHrcuKKvWjBdcPraHIw5HssI/bXmIIFcIWbibZCkyf1WvAVJg7fMTLgYG2wK27Z7RlHH7pyoBB+cDNzrFU7j2lKR78PbtXLyzEFcr9e7pseW5w1ij5MeGKjhL8+cdr1lsBqWwwYL7FyvKLrZV/vCqA77KeTPNg1ZFiS5HrR8I1UD3KKys4erCiEsJI6Lvso6zkeo9WfAGtONF6xALZ7oI5J/BZqofKHIo0F8CU1UUU9AnxbSr8Bh9ZVkRmlQrfy0IrZ6vlg== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 8q3FlgUZQmjsIS8i6MIg4XGFDqGvjiVGj7DHcD5dFlZPbZkXkszXldisogVb5vpEigBNooIwWpC2FaOcXb/C3n2I3pfOMqMlc/Vj6eeReZjWE69i2BNCM96OpgJP9iIw4rbkOlpmp/x71wHsmhfSWXMS1/XSjkSZfMUJy1tdmqQ3Ri8hw715xuHW3f/0jZEKzaZwD9fyCHW2siWSsmX8oFVwhLN88N/TFX+gczwXBxZV4IHq7RvlgyJDUhofMLPTG20z1Pttwajs/pfTtTP6neRAtiEUYY8adIcqdPLOslJP+q4GGnLWSwpq7SIoI9CnSk6v/lME9KsY75BohuCWWMofw910AmB+D7avw85H8cDiOrpPrLMc6zBBAQhJBPSZQLdEFE8SbknETn03XZlnXGDgMvkQtfWTaYPGj26p3sxgcXRjgJvkkQWw2Tv/cyS8oi9Mr8ZG9ZHijG9wpJYVsQSkXZ0z1v6I3UqFHy01X5pyLhLGWwPnHBPQpyR75e7hOQklkfUTUP9AZ7IxmTxkmpZOjSpRbPVvTREj63tFBYZCVucWT7vUUbUfIjQiPZtTfS6QHQy3aSCxHjFhbV42tS0qyQD/RbmPSwnZvM3j1Wc= X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3e767d7c-4787-4440-64a5-08dd13c51a0e X-MS-Exchange-CrossTenant-AuthSource: BYAPR10MB3366.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Dec 2024 18:05:42.5555 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TNPuZ7nJnXMrCVV8gaG/X1ZUwh37pLO1rU90z9OEx/Aw92sVxfxCvjiOyX3Ug3rZRQjA2GFX4mBMZCcQA/KVd/I4rJeF+zqDZn5Oxd19Du0= X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV3PR10MB8129 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1057,Hydra:6.0.680,FMLib:17.12.68.34 definitions=2024-12-03_06,2024-12-03_03,2024-11-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxscore=0 adultscore=0 malwarescore=0 bulkscore=0 suspectscore=0 spamscore=0 mlxlogscore=999 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2411120000 definitions=main-2412030151 X-Proofpoint-ORIG-GUID: x5YfrHLOjVs2IOJLTGqOk7u8lUkrr3Eh X-Proofpoint-GUID: x5YfrHLOjVs2IOJLTGqOk7u8lUkrr3Eh This was arbitrary left in mmap.c it makes no sense being there, move it to vma.c to render it testable. Signed-off-by: Lorenzo Stoakes --- mm/mmap.c | 18 ------------------ mm/vma.c | 18 ++++++++++++++++++ mm/vma.h | 2 ++ tools/testing/vma/vma_internal.h | 9 +++++++++ 4 files changed, 29 insertions(+), 18 deletions(-) diff --git a/mm/mmap.c b/mm/mmap.c index 55a8f2332b7c..1c6bdffa13dd 100644 --- a/mm/mmap.c +++ b/mm/mmap.c @@ -1044,24 +1044,6 @@ unsigned long mmap_region(struct file *file, unsigned long addr, return ret; } -static int __vm_munmap(unsigned long start, size_t len, bool unlock) -{ - int ret; - struct mm_struct *mm = current->mm; - LIST_HEAD(uf); - VMA_ITERATOR(vmi, mm, start); - - if (mmap_write_lock_killable(mm)) - return -EINTR; - - ret = do_vmi_munmap(&vmi, mm, start, len, &uf, unlock); - if (ret || !unlock) - mmap_write_unlock(mm); - - userfaultfd_unmap_complete(mm, &uf); - return ret; -} - int vm_munmap(unsigned long start, size_t len) { return __vm_munmap(start, len, false); diff --git a/mm/vma.c b/mm/vma.c index 83c79bb42675..a06747845cac 100644 --- a/mm/vma.c +++ b/mm/vma.c @@ -2874,3 +2874,21 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address) validate_mm(mm); return error; } + +int __vm_munmap(unsigned long start, size_t len, bool unlock) +{ + int ret; + struct mm_struct *mm = current->mm; + LIST_HEAD(uf); + VMA_ITERATOR(vmi, mm, start); + + if (mmap_write_lock_killable(mm)) + return -EINTR; + + ret = do_vmi_munmap(&vmi, mm, start, len, &uf, unlock); + if (ret || !unlock) + mmap_write_unlock(mm); + + userfaultfd_unmap_complete(mm, &uf); + return ret; +} diff --git a/mm/vma.h b/mm/vma.h index 6c460a120f82..295d44ea54db 100644 --- a/mm/vma.h +++ b/mm/vma.h @@ -478,4 +478,6 @@ int expand_upwards(struct vm_area_struct *vma, unsigned long address); int expand_downwards(struct vm_area_struct *vma, unsigned long address); +int __vm_munmap(unsigned long start, size_t len, bool unlock); + #endif /* __MM_VMA_H */ diff --git a/tools/testing/vma/vma_internal.h b/tools/testing/vma/vma_internal.h index fab3f3bdf2f0..a7de59a0d694 100644 --- a/tools/testing/vma/vma_internal.h +++ b/tools/testing/vma/vma_internal.h @@ -906,6 +906,11 @@ static inline void mmap_write_unlock(struct mm_struct *) { } +static inline int mmap_write_lock_killable(struct mm_struct *) +{ + return 0; +} + static inline bool can_modify_mm(struct mm_struct *mm, unsigned long start, unsigned long end) @@ -1175,5 +1180,9 @@ static inline int anon_vma_prepare(struct vm_area_struct *vma) return __anon_vma_prepare(vma); } +static inline void userfaultfd_unmap_complete(struct mm_struct *mm, + struct list_head *uf) +{ +} #endif /* __MM_VMA_INTERNAL_H */