From patchwork Mon Sep 18 23:01:57 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 13390559 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9A7FCD3421 for ; Mon, 18 Sep 2023 23:03:10 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B5D4D6B0463; Mon, 18 Sep 2023 19:03:03 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id B32CD6B0461; Mon, 18 Sep 2023 19:03:03 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 6563F6B0462; Mon, 18 Sep 2023 19:03:03 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 2FF536B045D for ; Mon, 18 Sep 2023 19:03:03 -0400 (EDT) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0160F120460 for ; Mon, 18 Sep 2023 23:03:02 +0000 (UTC) X-FDA: 81251245446.07.96ED98E Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf07.hostedemail.com (Postfix) with ESMTP id 8D17540007 for ; Mon, 18 Sep 2023 23:02:59 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=3x48z9Xg; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=wW+yoqML; spf=pass (imf07.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1695078179; a=rsa-sha256; cv=pass; b=wXCPLCadTP/cF++aBXyNP4qRlhg3mFmGeNw2CPLgwCZGfG/orIJPM+4JD7zEJvjkTXrlIZ gfZsJoXUd4P7qZgvvrO0d0wYlRMJePwyDCUQUrBZ78TUpG6TCT7SAeoFIn7l4CgvSdixPL UjE46fnlIfR5hTzS6C0Tvx4wDg9z84I= ARC-Authentication-Results: i=2; imf07.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=3x48z9Xg; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=wW+yoqML; spf=pass (imf07.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); dmarc=pass (policy=none) header.from=oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1695078179; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Y16y6xPVgvI7a8L2xgbgzpDrXOvAdzxTZsCIzk2fyVw=; b=uUy7e9kjfw9D7P9Z7V5ho6lxlHqUUbOzbO7IR2gkDHmmgbnFZcYt9T66xBi/ZwQf6eNDwu Hk3kEdw127zYxePKqiWgxlS8neCwELONnO4BE11tzO3chogWlcu+fPMMm9BmQrR3nAVmdu 2gbONWSBQUExw3lSlXGVpW1iisoy4Ek= Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 38IK54UD005659; Mon, 18 Sep 2023 23:02:26 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=Y16y6xPVgvI7a8L2xgbgzpDrXOvAdzxTZsCIzk2fyVw=; b=3x48z9XgPar73/CgIkpbWSMGfqKQUl3y1yn7I8wh01CQ5gF8vmLLlSxRfde4ANsKBGx8 gR31H+ADpLFtDhzQTmCm2qu2YlQecFpAAHFj8s8qTAxBeuDV4wkP8O1X73PD7CwcDqYP Wy4mdv4DQRH/Z8eHWiJ/dvOIpNCbhcZqml0+BweCt9zT+4EqjvwsG57Cne30TRbqXWHt PKap0+n43wb+3GG6ospP9Ozwx7XojBAbli2f1/s3xFYdzjSjYw36PdWjjAAhIegc58zg nQ2R8ZC0Xon0L2ctuGOYEqHK2hd6Uj3RgqZDfmI+GvJWpaUpSIiuOiRnWcdrKD1yvOSE LQ== Received: from iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta02.appoci.oracle.com [147.154.18.20]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3t52y1upr0-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 18 Sep 2023 23:02:26 +0000 Received: from pps.filterd (iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 38ILubHU016052; Mon, 18 Sep 2023 23:02:24 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2168.outbound.protection.outlook.com [104.47.58.168]) by iadpaimrmta02.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3t52t4nder-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 18 Sep 2023 23:02:24 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=gJ97kMFysiMp9zSD2wkpOXM6R/sZW9PQA2LnVE4dkn5bDE7JjLMY8MtanVM+1ZQLUFrTEOURLQKGZuUHQOGg2i2zQipV6aXPCrzyHOY6BvFATPzo94VzEJ3eAloNB12WZcbGmeHk/GSioTDFWd6iSi3uvyNxb4KsvXalGaLwXPm+fPKbcT8icpuOFu2p/CU2Rczi5qdAc8eKJnV3Mo4w+uacQy32SbuHHcMv1HSI0ehG1M5PjqT0WPgltMnJxuhxGIn5rpVvXVphTBDtExVcDBDjAltdntrRxEMFDvLCx/mp4QvC1BYWuXayg5VnHFnWbAKITf2juXCQWQVXO9tTbw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Y16y6xPVgvI7a8L2xgbgzpDrXOvAdzxTZsCIzk2fyVw=; b=fdyuzvF4X2Fk7NlCPWy4cQt2X6ZV5/CfqPd9uqbksqr93j7ZmV80HZjc0UQRF0zj5YMORfLBW3m/YrPn4X3nfuiMFIuYkBYI2edlr1sw6yQiCkCFqOC8JeGAtEiNpaaM4pyUZ5fuAzQtg/DNyaTrD6hG8u1ZZ/Mknoa2+lM0UFhcvBsvP4zDIBw3Ex3tPjzOpacv76Pn1GLn3MH2BgNlNjwrpsMlujrKNGsxpr2tbP5CTdirJcRl2m8Va1FoBObE0rmRw9cPut+MPVzSx+uiPlAmj7A5+eIvF+49VKey2VY1+dRNmkBrH3vwJxgoeYw236cWXeauA03w48gjG6u/Pg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Y16y6xPVgvI7a8L2xgbgzpDrXOvAdzxTZsCIzk2fyVw=; b=wW+yoqMLzOftf+0SXOS94q4bNo8xgEURYGhmGksBI5A92+L6vC/tM6D87ZAVJMrUVqfWz8LLgrr0T66jjNhPmgwXvW4wYn1+lo4W2ftu/g2BmtVYRfSue5jNsvYR5hjUup5HpsuiQfakHyivZi9zdA/lyf6RHIw9DUu6p5xOdKw= Received: from BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) by CH3PR10MB7494.namprd10.prod.outlook.com (2603:10b6:610:163::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6792.26; Mon, 18 Sep 2023 23:02:20 +0000 Received: from BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::c621:12ca:ba40:9054]) by BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::c621:12ca:ba40:9054%5]) with mapi id 15.20.6792.026; Mon, 18 Sep 2023 23:02:20 +0000 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Joao Martins , Oscar Salvador , David Hildenbrand , Miaohe Lin , David Rientjes , Anshuman Khandual , Naoya Horiguchi , Barry Song <21cnbao@gmail.com>, Michal Hocko , Matthew Wilcox , Xiongchun Duan , Andrew Morton , Mike Kravetz Subject: [PATCH v4 5/8] hugetlb: batch freeing of vmemmap pages Date: Mon, 18 Sep 2023 16:01:57 -0700 Message-ID: <20230918230202.254631-6-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230918230202.254631-1-mike.kravetz@oracle.com> References: <20230918230202.254631-1-mike.kravetz@oracle.com> X-ClientProxiedBy: MW2PR16CA0049.namprd16.prod.outlook.com (2603:10b6:907:1::26) To BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR10MB4196:EE_|CH3PR10MB7494:EE_ X-MS-Office365-Filtering-Correlation-Id: 1d3de23c-99c0-40e1-9a80-08dbb89b5025 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: h9qFBw9tL1ZNaMQ3gUxzkE6jX8P1w0+gx4cGr7Z1kUnG+2Us+0+qY8Nm8TsxyM71EHU1VEdfj3yOl7I/FHKgitdUDR2N6/q0bzuucK79pnAIwHe/vZOF17UUHmWb5aSdVgLv5KhRF7GyifwNqFwDhkDJ1HqMVmsjs5QWmM5UauyCRaw/k9zALcma9/2K9BQ/utN2iXQVwZ92hPF+ruAkclyqTVsvTERnhlx4QNSK5mOHIwGTsvs1SJo7u1uhzXyquGJPYtsGrlq2FnXNkgEaSWWzxI6mWRya8x3zU/e4+uzG/FUaxeCsyZjeXEKErliM+aQ1MpuY9oChyJwHGyuJDySFWNQ5pWP1Y6PxmY/aiojwnwDvzykCRX5c+edAL0KW+wCm8EOzYPQpn9AwuoTs3ZkEBEPsfHEkCbf0Uh2N3uyd/NkbuoNur8n7wyDYkJp7e8U9oAfsQ8tlPJ1yi221bnWN3mkLUgDHycskXL19C/mN1WyCj2R2hD9w8AMkr8KN0yNBW8kVZHNrCbf3n2Mjgq4kpSjVDovNwjZgpWi2ZTr8ex3btOaz+m4/d/Wsh1hE X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(396003)(346002)(366004)(376002)(39860400002)(136003)(451199024)(186009)(1800799009)(41300700001)(316002)(66476007)(66556008)(54906003)(66946007)(478600001)(6666004)(38100700002)(7416002)(2906002)(86362001)(36756003)(44832011)(5660300002)(8676002)(4326008)(8936002)(83380400001)(107886003)(2616005)(26005)(1076003)(6512007)(6506007)(6486002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: miGrKbcNZpW5j9hgE8fPdwljQ0DRmW1/XJOTaU2FLv5UirNhYK6M2lfsfQEQ32lZvv6kwg/YQkZ8+kYf3iCcMPm2MSvfVee6nXxTlE2ZTFJ65/5DepQ8aYRLwSAGIyWjLayrIDlUNgzLXvgqYhw/EWL0Xb5Rb50JeKaRNTvAhK3SdkawH/O/uywBIyA1SQiD240gYpabYI65KleibyG67y6qacDMYhurZXBoRhUOb/fmidhCuaKTvbO5WYqiqexTBAD+NdzJoa7PQ8MGHoX/+qtqXx9eu5Ome6y+CTxthlexSFPS018DIv20xHxyK40/8jxjlqm32Fhe0oofhTGMWjy9abkJcQ2MQtVOVds159ucDXjzk3KGIyi0f/z6K+H098ZYD1A0L3L5yOVTlN/oGrGUFApeyedAX9J1J1qUZcbaK6zG8jl+B0/0f6bez454XzSgyw6BIJXFV7bKZF4eWPyTgZFz1YGdinlb2jkMN0XHXAIUGhJOfbmHtw8CCbFJqRH6J6I/NN/SK7oPl7g6rWCLowx3b01HF5PAVUBUbEgRdYgBSs7EE3HNXLE7yS54QmW5QJf4xcHsotQ/H6CbYCecgYSLOcaJjz6BOQMBlyKUkxIHa/3InS/8c6q27JcgngyAKeMiKBEaCgknf+sz7ut4R51KU1ksftcHxoavZ0uDg3I6+q5SYj3KIF8cySJjJfk+VcMwGoMzV6WL2JhsN+yTFaILCLySX/fG6z0xXG2HuaP2hp2ZQQaSj7UJbX4czETSQZKH9d6AaFal1mNHoKplN8ki5m2ptOsWaRMGFHGmNrFVzIoyyGjrY5P5YE9l1/6xkoAPt/iGKjTpc+nQkjHxxNg01QOlmilYqvOSX997u/mqFQ3/H2a9OkNtMcZyYgzbpM9UvQXA3l9HFeijiGTzFOnbwqay+NeVUNOx/jKXUSRIiApv1sLfNJyKleBqjxA04ED/oQaMf+0HSC0d//BcPfWE/JJh+YbBnuBc8QlOAurmexM07p7CL8G3cm+na0me/libWP3r6LOG/RRY9BjJX7zzuuyU+ARrHVAKPAqWWO6vBOPd9RO/up/kji5yLOwc4XJTcYVN3Kf1HuqNd+SYmoyyIvD2PFz9jmqqgV8f3T36orsCOf9E/7M5LsCeR5XHq6slkgrTQ4NchTApHmxROkgZrls4k5x4rLfDL/ay8M19EE3pOwm9qP2kC/lrPBeivtMmvNEZHP1KS7Mim0kB+l5zW27hVvzlHxYCAfClWTZx+VfrNnmwRl2UiEyGO1D21HH5Z/rdTVVvTYAwKN2Q6Lt1qPwtr1lO+EJ8AzI0OIffNPeAH/qjSnbNKnH46M8ABM4gVkhb1PuxqTArLakOriui+eXSeh90mHoSb0/2cRjJv4LzQ5yio52KEBm33cidu/YYhcAQHFiWzGE5NgpkJZEX3CVYgg/8lTkPSmvalk8+gYfldu7DUh9boNPOsMEhkegA8XOBJqBCt3Dov2ftTkXM9LeJc/oK/VtzaQ0w4QM79AMGG45/twW03WOy6w4+weX5PGHKO+Wpl7r9aVBldPe04yHFEYgNkMhO/bWB10DNWmq8nVWkBjNWuKMe X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: L0Bwz1RY7TakwgtaALdP1mLCehayJnJ5os+FKGmg+wfUIDWxbRUv6il69v6z8+BBrbB8EUFEZ1krUUka3YAvpfv4RgEMbmNVpI/9hTh4EvjMflxRMgLApjak3dVuQbOHIHuUynlUlksPU8LD2TP+tp5fia8aEDlblbHZEddeNrAdLAUeI28XuoheLEAZSKiJx5+ZWSCNVpoJ7YODPCru//+0I2nxJk7kxo/7OwuG48HGf+P9R0kjn4jLLuC83dxjfvV6NQxv5D9KQMAPpjdB3s/8OXVnoYqI2wx4Od3eWl0p5HgLdRDvZI6HQBD4eiblQAvk4ph2FGPwPrj1Qysp4VBLIWIZmzNGS/B4ax02XMWYpGFFjlbnMuVueAMdAK0x1jIv4E2VI6i35bEpYwJYOfOGWUFzug7fK95v7WEtdtH2yPvb3dTFrLQ6T5CG0SUrLFz+M3QZihl9TLa6tmG9xPpBF4m0xynw6hoGSpWljZEeqVJShs6OhEs6wW8kgf6WPTgUZiXelOGCnz8y7IelpbfBTrvCQHARpkZwajyN7NrvA/VMXjnatzC9Le+nwEHZSc+ctZ6Kxu7ff1wq3nnpmq2H0z8KcbSN4QDA0E5S0OlGpNzw2V8WnodTewbt2/tKgH2LJsfe2cnYuXtIr1q74EBgS82m3kgq9IGOUD2XjL9E6ulyPuOQPAljCB2RQgu502JD+efJ4L/V8AeLavIA1gJEb17KmWaib+zaNMdbSourPXV9S87dUdD96UO5xFNDq0xmKR3NkEB4p8nByZzz2Qyms1YCg1jxfD2136v4xF0KZIwnL4V1xX9V4U5Uo36W4stf3aplxfRnea8rNy5phtwiKTQLjD6Tt3KpBemwdR2g8CjBSle1EYFPD6V2X574habkqzxeg31HVk/K/Q3YnCq5G79lgH07+Dteg4aUtK63Y+Q8sJuoL6Cqwt464aTkvCKp1GbYvlsiOdzrYsy1SHBx3+n8aScMQJbHrWW8/lyKwQZx4XK+f7PUstr4hHusGMHoy6/wD657NPjJvnhbNdb3h1l7dmmTXVOLg9MyE1knxxwsSdJRytxXdK1HjMmsu6QcRlSVnFYF08ShDZJVvA+GbvMciPypNK29LKy0qUNNp12gOpwDKCB5GzIqR9gl X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1d3de23c-99c0-40e1-9a80-08dbb89b5025 X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Sep 2023 23:02:20.8263 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: tlgvJR8qAn3balXj3gRSu3wCu7RmFuLS41Toijz6B121tSZ8Vv9A+lkIkmqqbD0PH8xASRbI8YVYtYX71Ss/Yg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR10MB7494 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-09-18_11,2023-09-18_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 adultscore=0 mlxscore=0 spamscore=0 malwarescore=0 suspectscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2309180199 X-Proofpoint-GUID: 5R7xdpdnCPEO44ZY8QGEPxpw2FGnSMps X-Proofpoint-ORIG-GUID: 5R7xdpdnCPEO44ZY8QGEPxpw2FGnSMps X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: 8D17540007 X-Stat-Signature: 7pi4p5oht5o44n9hkaw1commxd1zfrss X-Rspam-User: X-HE-Tag: 1695078179-833714 X-HE-Meta: U2FsdGVkX1+HF/3B2fiOIN/haYVxGvEB0WL+XAZbwHKRYUyTQoRbJhP7Mi3Ao3TJqsFOQYvfn6r9hSGzImXgYzuLvb+bjvpY/oATqRqx/Bg4pHiVdKTX36MnsF7R9SZ4weS/gz5KwSwch3QMXGzMIaGvBobN5oCaIRAAKxYmUAueiW8Ln74/qIKjwRPi9FXUhmqae1kFTOf2hyOW+d/Z9YNku7XLVbjF4PoqXf8qu+bnHECAypKbbIiDEaFqn54t9dlU52E2VkxbKJN2vnoQqMWi1eqhizx8w+9uKiSCijOd2qUiHSF1HW2JjejHBe/0TsqMlvlr1FBg65NOyA9k410HnCRJ5BIPwWOYyAvrtXhCUxjK/3pSWOX5h4Ct8/UykHdWYuKhyxVAWKS6vYGEWakQyxXM8H3oOGC/xAFQsNZI6zA5mRt+DKd/ahPg3RB8cdit0w0OhJDZzSgfy7xXh74PcvGcZsKOiPs4TbvUCTmXKzW7XhYG3CTLev5ox3Np/3d6SbNtrWDTwJEBf4tGr2ovIpTqA50VBHvs6FviPJGvI5Su7WvpzuS5Q0qlHlbH8H/Fpd9xeUNkkoMDdcyrMorpnO12lxk4NWJ9q/Cgtc+tVS2zWOggN4B5XdZJ29xP+9WaRxpsdJl5BsmxHvq5qWH5qA7Dku22UiXGbObsTgp2GQ59kvOOePNdM2WOqVV3/bT6xBjBgVZK52EA6wtxMUkTIJpfMC6Yt/j2SWBK+bd6YhDcxh7hObYGY19q5UjBG0u8yh2fJ8Ldtj8/Z6w7fvcCWTp5jBZc2/OXEWK4/Gp2LB/82zFd8i/dlBx8y+Ha9aTRkLm5l/BZoua+XUEKKdeDvb3NtA5hwaJTRVUMTe98QUiNm7fOp/udhD7dIk67siWK6FXfH2sf4f5lsGzr2vN7QxAoHadJ3ag7RFltQ/61L2Vp9hSPWvu1ZvRGXWMxndrRluBp+1+DQ3FJm1R m22Jct4+ ErfzlXKPwHolvEQKgwvTZS2RlXE4lHqX03SFBlqgjiseTPQVAF/PV7LWhcs/XvbRn7OZlr7Oxf2C22YCfKoSpt3y6OQib2U8huJq6tWrD5T4i/2pz+jx0fT20jgwwQ4aDU15KuAcW3RjKTPoeEyhVkjWXEf4+ssT89SU0SBBKYQUvNbT4tlkRB0EWJlSt2NYaJyOq7eulInCQJHmrlvyOGUulwLHq68+Feq97zwdleRmlNy3fw6QqhrAzUzYQPZITEYEYVG6oDzr6ZqFhsENzBANAGxYVwBpCRI8OpGgwG6brpj0IdFYHHGsjA5nh1U9Lr5qxxkD22toGE4mDHyRmoQA04xofu/FoTIDZ6cSvGXWDbPYQhgr1Tv4xk9/ebVDKyhO4pWTmOeWpu1remNdQzlF992GstBRBhqplUxZXhZmY9WbAAgnTP4cUuh8ZUcP7Sw3uE9UCGN9fQn8EbphUgEGL90UEL52L/Umx X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Now that batching of hugetlb vmemmap optimization processing is possible, batch the freeing of vmemmap pages. When freeing vmemmap pages for a hugetlb page, we add them to a list that is freed after the entire batch has been processed. This enhances the ability to return contiguous ranges of memory to the low level allocators. Signed-off-by: Mike Kravetz Reviewed-by: Muchun Song --- mm/hugetlb_vmemmap.c | 85 ++++++++++++++++++++++++++++++-------------- 1 file changed, 59 insertions(+), 26 deletions(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 463a4037ec6e..147ed15bcae4 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -222,6 +222,9 @@ static void free_vmemmap_page_list(struct list_head *list) { struct page *page, *next; + if (list_empty(list)) + return; + list_for_each_entry_safe(page, next, list, lru) free_vmemmap_page(page); } @@ -251,7 +254,7 @@ static void vmemmap_remap_pte(pte_t *pte, unsigned long addr, } entry = mk_pte(walk->reuse_page, pgprot); - list_add_tail(&page->lru, walk->vmemmap_pages); + list_add(&page->lru, walk->vmemmap_pages); set_pte_at(&init_mm, addr, pte, entry); } @@ -306,18 +309,20 @@ static void vmemmap_restore_pte(pte_t *pte, unsigned long addr, * @end: end address of the vmemmap virtual address range that we want to * remap. * @reuse: reuse address. + * @vmemmap_pages: list to deposit vmemmap pages to be freed. It is callers + * responsibility to free pages. * * Return: %0 on success, negative error code otherwise. */ static int vmemmap_remap_free(unsigned long start, unsigned long end, - unsigned long reuse) + unsigned long reuse, + struct list_head *vmemmap_pages) { int ret; - LIST_HEAD(vmemmap_pages); struct vmemmap_remap_walk walk = { .remap_pte = vmemmap_remap_pte, .reuse_addr = reuse, - .vmemmap_pages = &vmemmap_pages, + .vmemmap_pages = vmemmap_pages, }; int nid = page_to_nid((struct page *)reuse); gfp_t gfp_mask = GFP_KERNEL | __GFP_NORETRY | __GFP_NOWARN; @@ -334,7 +339,7 @@ static int vmemmap_remap_free(unsigned long start, unsigned long end, if (walk.reuse_page) { copy_page(page_to_virt(walk.reuse_page), (void *)walk.reuse_addr); - list_add(&walk.reuse_page->lru, &vmemmap_pages); + list_add(&walk.reuse_page->lru, vmemmap_pages); } /* @@ -365,15 +370,13 @@ static int vmemmap_remap_free(unsigned long start, unsigned long end, walk = (struct vmemmap_remap_walk) { .remap_pte = vmemmap_restore_pte, .reuse_addr = reuse, - .vmemmap_pages = &vmemmap_pages, + .vmemmap_pages = vmemmap_pages, }; vmemmap_remap_range(reuse, end, &walk); } mmap_read_unlock(&init_mm); - free_vmemmap_page_list(&vmemmap_pages); - return ret; } @@ -389,7 +392,7 @@ static int alloc_vmemmap_page_list(unsigned long start, unsigned long end, page = alloc_pages_node(nid, gfp_mask, 0); if (!page) goto out; - list_add_tail(&page->lru, list); + list_add(&page->lru, list); } return 0; @@ -576,24 +579,17 @@ static bool vmemmap_should_optimize(const struct hstate *h, const struct page *h return true; } -/** - * hugetlb_vmemmap_optimize - optimize @head page's vmemmap pages. - * @h: struct hstate. - * @head: the head page whose vmemmap pages will be optimized. - * - * This function only tries to optimize @head's vmemmap pages and does not - * guarantee that the optimization will succeed after it returns. The caller - * can use HPageVmemmapOptimized(@head) to detect if @head's vmemmap pages - * have been optimized. - */ -void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head) +static int __hugetlb_vmemmap_optimize(const struct hstate *h, + struct page *head, + struct list_head *vmemmap_pages) { + int ret = 0; unsigned long vmemmap_start = (unsigned long)head, vmemmap_end; unsigned long vmemmap_reuse; VM_WARN_ON_ONCE(!PageHuge(head)); if (!vmemmap_should_optimize(h, head)) - return; + return ret; static_branch_inc(&hugetlb_optimize_vmemmap_key); @@ -603,21 +599,58 @@ void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head) /* * Remap the vmemmap virtual address range [@vmemmap_start, @vmemmap_end) - * to the page which @vmemmap_reuse is mapped to, then free the pages - * which the range [@vmemmap_start, @vmemmap_end] is mapped to. + * to the page which @vmemmap_reuse is mapped to. Add pages previously + * mapping the range to vmemmap_pages list so that they can be freed by + * the caller. */ - if (vmemmap_remap_free(vmemmap_start, vmemmap_end, vmemmap_reuse)) + ret = vmemmap_remap_free(vmemmap_start, vmemmap_end, vmemmap_reuse, vmemmap_pages); + if (ret) static_branch_dec(&hugetlb_optimize_vmemmap_key); else SetHPageVmemmapOptimized(head); + + return ret; +} + +/** + * hugetlb_vmemmap_optimize - optimize @head page's vmemmap pages. + * @h: struct hstate. + * @head: the head page whose vmemmap pages will be optimized. + * + * This function only tries to optimize @head's vmemmap pages and does not + * guarantee that the optimization will succeed after it returns. The caller + * can use HPageVmemmapOptimized(@head) to detect if @head's vmemmap pages + * have been optimized. + */ +void hugetlb_vmemmap_optimize(const struct hstate *h, struct page *head) +{ + LIST_HEAD(vmemmap_pages); + + __hugetlb_vmemmap_optimize(h, head, &vmemmap_pages); + free_vmemmap_page_list(&vmemmap_pages); } void hugetlb_vmemmap_optimize_folios(struct hstate *h, struct list_head *folio_list) { struct folio *folio; + LIST_HEAD(vmemmap_pages); - list_for_each_entry(folio, folio_list, lru) - hugetlb_vmemmap_optimize(h, &folio->page); + list_for_each_entry(folio, folio_list, lru) { + int ret = __hugetlb_vmemmap_optimize(h, &folio->page, + &vmemmap_pages); + + /* + * Pages to be freed may have been accumulated. If we + * encounter an ENOMEM, free what we have and try again. + */ + if (ret == -ENOMEM && !list_empty(&vmemmap_pages)) { + free_vmemmap_page_list(&vmemmap_pages); + INIT_LIST_HEAD(&vmemmap_pages); + __hugetlb_vmemmap_optimize(h, &folio->page, &vmemmap_pages); + } + } + + free_vmemmap_page_list(&vmemmap_pages); } static struct ctl_table hugetlb_vmemmap_sysctls[] = {