From patchwork Tue Jul 11 22:09:42 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 13309430 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3CEF2C001B0 for ; Tue, 11 Jul 2023 22:10:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C6EC66B0075; Tue, 11 Jul 2023 18:10:40 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C1E426B0078; Tue, 11 Jul 2023 18:10:40 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A977E6B007B; Tue, 11 Jul 2023 18:10:40 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 9BB346B0075 for ; Tue, 11 Jul 2023 18:10:40 -0400 (EDT) Received: from smtpin18.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay10.hostedemail.com (Postfix) with ESMTP id 59CA5C03AF for ; Tue, 11 Jul 2023 22:10:40 +0000 (UTC) X-FDA: 81000726240.18.93160CC Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf27.hostedemail.com (Postfix) with ESMTP id C942540008 for ; Tue, 11 Jul 2023 22:10:36 +0000 (UTC) Authentication-Results: imf27.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=B730hXWs; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=yD29ExCr; spf=pass (imf27.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1689113437; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=sJdYG9TJ5ptIRFIrt8AEeUT1n1ZOgrXWD30BKh22RXw=; b=skg5rElpBB4y6EhlwoF80PmdW7GBI07llF4Y7cuUavPUxyR+aWZAgQICKpuK0IeXU6EPPe 9U8Lf2YBfomFXhL9X/yqzEiQafSbdJhkAqOrcaV+7psZ66DLSPRMbjdVzNoJUMVTjUfGep 29mVXeoBLnJLPf2+0M04JpBM0D9Ujoc= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1689113437; a=rsa-sha256; cv=pass; b=Is3IyWTzGjaiSrQ35JnlTUYzDZcvPqOK2voxNMX/fUTNyu7FuzvpXtXTC7xSAnHmy1KMGi GvBLJusRKKMaqBjWJgETCTauvFzp84sUfr/tCMcBH17TzFt/QlTdmoGd7IdKn0n9+Qw/wx xnfMSdKC1IDVc7N1UaH0BKXuprjC3ws= ARC-Authentication-Results: i=2; imf27.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=B730hXWs; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=yD29ExCr; spf=pass (imf27.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") Received: from pps.filterd (m0246627.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 36BIDDsJ011193; Tue, 11 Jul 2023 22:09:57 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=sJdYG9TJ5ptIRFIrt8AEeUT1n1ZOgrXWD30BKh22RXw=; b=B730hXWspKxIFzPsDkBCLW3TbN8j/P9oPAX/CXEKdtUj9uTDtSD+Z+n2b3tN8xkA2qbP MNUZxdahyX2QqOHwonC5cs6oK8FvhR4BjzMYtpW3Wra4ipQ7CCa9n7qJ5h/C2fj/5lDM v7LmRdMcnZAZ0Iu3tdThAQ5rNve/+8qqyMJ6xUhEqY2JXWDUOcZnVKp3Ox0JIBir2f6M tMxeAWIICtBUO89XSgYZva7XyObWmj16o2e8QpyA5/s0kO/eNlDODFT6Od+a4InLSOsa OGo7+Do6YaMckyfE4wlTlOzJ/eRshkiBfZBCkJOVvY0yzfEfZscZAB5pW0dyeLFZ7j5G CQ== Received: from iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta01.appoci.oracle.com [130.35.100.223]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3rrfj63xxp-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 11 Jul 2023 22:09:57 +0000 Received: from pps.filterd (iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 36BLnjKf008288; Tue, 11 Jul 2023 22:09:55 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2169.outbound.protection.outlook.com [104.47.58.169]) by iadpaimrmta01.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3rpx8btb1y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 11 Jul 2023 22:09:55 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=dUO+eDAsJ6kNquJMWHFVBD1kaNI3AIEJG3r3zwqCLTMPEtKoXQ/kOgBzNe4h+xX3RPSg08O2Njp3s1SZNULXpjtqtLLy2H7CYQhPbj/ekhzrTkoc1rxAIvNm1v+5kCC4qST3J186yHIm+5pUy3JGrisam/SyyVaiqo6tZsQc9p7vO3nsIadmsOlSR1R5T4+ampCwIbVylD2R2JmjnwGpAuqqmZ2d2h8IDNKfoT28HVtCC2sx1mSrQAIMmg30NiITkXNIywwmE223Cdpctji/CFqtTJXWghGxc6ioVyx0jrWK+8kNe35VgxLT5wFE+PdZUpBIoR8MsbRIXd4KqFp4Og== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=sJdYG9TJ5ptIRFIrt8AEeUT1n1ZOgrXWD30BKh22RXw=; b=ZUhvJjlnHv9cjrAdPJ+uOBRgrTOBp3EpzvjBiYTGwO9e3dnY9K4Is4dFQ+HMW1Fm4yACZdRVIkh7qCUkDRh/d+PCXDrPuK3GXWsJqAw6zJ40tbX5MO26bUsZ33FMaH5yHyN2OMD0ZASIBASjbvt2ZK07mEsiBQqmbNYR2KNd1FIDbw0vFCQmPo1XKnMqxonHq0TvvCY/v9Qb4EzemcHj8FE2dykFS+E4GyMoctYi2MuGMEr8IBE7SJ3GPC0VSEt8VHpL+8vJUYVaIiJKmgvO3Ob6+HDYa9DUHXjV+SjMzAuH5a16Ejek+ewOrbyvTvFHKB5Cbl4SBNlKVE+Z/dZqQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sJdYG9TJ5ptIRFIrt8AEeUT1n1ZOgrXWD30BKh22RXw=; b=yD29ExCrpmdnqrgTaYjIdWK3xGw0luEX/GhTKNu8rYd0sVZB4d7P0iGBmf8nSdAnFatOv5ZTGKwTYBvGCR1v8Hl8u5YZk2/G8VhXKUo8KGYgZniKCtGLEERaiY5fFGljeqBjPtDnfYWfX7CawKqk7tnbYl6DE8nn54sDKWwNqYQ= Received: from BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) by PH0PR10MB4775.namprd10.prod.outlook.com (2603:10b6:510:38::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6588.20; Tue, 11 Jul 2023 22:09:54 +0000 Received: from BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::4a17:13b0:2876:97f2]) by BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::4a17:13b0:2876:97f2%7]) with mapi id 15.20.6588.017; Tue, 11 Jul 2023 22:09:54 +0000 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Jiaqi Yan , Naoya Horiguchi , Muchun Song , Miaohe Lin , Axel Rasmussen , James Houghton , Michal Hocko , Andrew Morton , Mike Kravetz , stable@vger.kernel.org Subject: [PATCH 2/2] hugetlb: optimize update_and_free_pages_bulk to avoid lock cycles Date: Tue, 11 Jul 2023 15:09:42 -0700 Message-ID: <20230711220942.43706-3-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230711220942.43706-1-mike.kravetz@oracle.com> References: <20230711220942.43706-1-mike.kravetz@oracle.com> X-ClientProxiedBy: MW4PR03CA0054.namprd03.prod.outlook.com (2603:10b6:303:8e::29) To BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR10MB4196:EE_|PH0PR10MB4775:EE_ X-MS-Office365-Filtering-Correlation-Id: 30e7a8e0-80dc-41ef-8262-08db825b8df0 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dpmyNp5nzk/kh69KnARlekYnOH7h33pDQ85apHgfprMZx5yUThMCufUXEzwiWBZ2f0KqsIUKmYAuiXcvL1kdNDAoU/O0k5AN/yn//ZzIZ/wnG9zJQFVzgttRW/AiO3dWQWPXZco9rdwnk7EdrijQGg70V/wWKBFO0YeYMeQMvibx27JHV1R+eCBQiVD8kEgj0lW3qtgLsres/d6Bi+RmZzjX69ImaiKUYfWRqfWsYxM0de8TaJoxlg+0HiV0Akw6KYwyL1YNirm7kuRl0IrG/uDtkP+4pmDi7gtwYVLR8H7n2Pi5JLMta5Nq5GxtjNcKQ/SS/uGuUwvA1NWD8mZDoliPMiv6Asq8MVFWxUG8Hntc/aOglOqJM35WSVCYyICM9tAETwzjqUxmjtLaTMX4zlh+FdIkkAC69peUEhNA1mtyq6CyooT544vE0Q9Z1m10q44rrrDO1sqeD6lM/HBK7VaBWCDAQDWmlXX/cCTWPMgfTPipTA2JWzbcHJyTJBm8xefwIe9cgxMV1/XN0EHMEzjCKax2Q8TozW5u57uLcIGp55VsndLtrsSH/ndeF8QV X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230028)(376002)(396003)(39860400002)(366004)(346002)(136003)(451199021)(6486002)(186003)(54906003)(86362001)(6666004)(2616005)(478600001)(6506007)(1076003)(26005)(44832011)(7416002)(15650500001)(316002)(36756003)(8936002)(8676002)(41300700001)(5660300002)(83380400001)(6512007)(66476007)(66556008)(66946007)(2906002)(4326008)(38100700002);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JYI5W7j+tdTzC/YSvdnug1hzyZVqlK+mgfAxG7BAPGGzoJNbnbA7BpQ8GJdfhVJ+Tb+tZdsNbCWsbsgM7eppLyM0rxCIprnZJZy3nrBkiXcpvjUDTjN61Cb9hL2mq8sCAwpu3pTWmkWC7XcOOvYquzLeyHAQCK9qKKmn57/7FNIdvwuD2W53AZE8X4Ansi0Vnlpp0xt7/eFc+I1qsloVtBJpA2OuHhQgE8vveHLVfppsyeQnKz1TsgrE6VtQ8PHY/TrVi0Vym4HGEMWh0B/ZmvhbtlLxqDOpepInG0dbJ4BPwjnSH2X8k/h6uiXBRy73y9NZ3jt1VHN3/b0i+DL/FF3q/RaG9xL3xmP9wFSjicYSV68H7uxlCIoEh/nHx2Oa0+y0hAh2cZENOGzTUuF13Uoh+KZui5nGHJt2AFieKr6EnYBdmbEKkw3yGab4AXKdGJdOnetAKHVLd6U4elX0KaR9/xucbl2whlr96eNai8n4A82Idb0+HzQvEKZ0elDfUAS3JYdzbDc1Wo3a8wn7Zf8JGJbmbIqSsgc1DSXVkB1Yp2tpD+bI07zSCsZ/4SzcENXllPrm3A7GbPfCxHIxw7vk8xAa02hYoAhQkkebT5xa7b9rPme0woDYPLC5qlHsHZja0qsYWgDN/5TCDN8CCYxHhMclcIo3vpv2tnKcovrabF0AEsWv2oOKGBqhKJpd/PoaFUlW0aNZ8JQhXvO2OI/slVnwOyMdJSn/MgpwslY1OR0WlRBQsSksgSHYCeEKLStxe2p7Rw+FZUNJgr9BKf8zjUrL+JL5xNBUcbvBUPbMZXqy2Yel1Lv+R7y9N4WmKsGPGmQ07J/C9/gOjOFI3pkb6aq7fajbqkpMjJnX/ZM3+tM2YgA7eq/GUIUeT6fZMNFt3f3KgN1yujOioOaUnzkNriRU3OQ2Bf0Kv6j3f0TIHFQ+3Kkac8L1oGz2oMKkFhkRE1nvOaFgmlx/NKEeq70jw3KWW/e0XJyl9EJaQtjM79z8tzgHxRUx1DasZbfTONP74K85FEviZkJS1Dr6HEgMiKVIWMkIXJ1c8TZKBQDdhTgyK1tA1/g1tvqS6VbODwaeLYqnfReSEM/jepNsGXxPRdhXJBrhhCoHfY1Is8oUEzbh57ZEnCW0TKgTd5w2Sw6Ps1wnBICLfNsz1a3+nslL4TnKDduISgP0j98dEHVIlzbV0v1CCTLyOy3xkayw5YPGc5C9geu3sVSeZrLprjCEnxEkXKZRATjzj/l5p+/lseN6a0+snQk6dcmN0hg6k8ioX3uWinX8BUIbbUTz325ARoKta8++o3YhJbXuUBgyznfbMQCdHDoYnf0oN9DP8WAt+pBQ9TtRpiWdsmWWcmhG5pV1VHFTJVAK6xDch8uRlt/o+ehUypa9JXTd1a2Ijp6TAVxuJvIEL9ur4nx5bmKPXTYgOSg3HED+8RvweF1U2j8+DEnbAfRu5KKzAFCz7WD52W3CEqDuheZjB7fXuZCZALJ0ltq6veyWNqrcvbw5SW576HxvaJwPkjU1uU6H8xbMlRtbzh+dpHB6erwEt51Jhckim7bFGoMVUSFQYV36lHt8BKBaoMth8KxUXRK1QnCxo7aTk90+eIvVMuuaMA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: exV0cB5lgJw7yLa9LCOGPFF/Jj58qZ8WqSGJtDS2Y2bHc8zlyc8gSaVYfPrr4stdjZXchPi86vGJWCQGGcTjp1swDyy6SKKDsctfTHopXHrBypCJsEpwStMOJDmqw42yEKME18pcXpKkn09XRkrAkp/3MZLtGqsaD1kMlC+Aaaec6r2nN2SZwnRai2PfGJPY7vZIdI2lNHImSvzfMaErX+Q9o9XyOFQtY+9glCFO9YDBJvd3uLf1iaF31mbBZOVkMsB0IfL4y6VdRiNZKMA0OnZ/YJP0YSyowsAScPjAaTBF/FCojMBkRrg0Yebj36hKDfDKe5plDF9FkaJt2AwXOeZIRDUEcaJ53yj7yDs60hiaBKxl6sdH0wukay/jJqjYtMdXSRv5DDd8Y2T9FRsKxubeKSLeAs4FSsOE8lF+AqyYem2GxUkD8VYBzzdyqIweVm0CEaUy5pOfvC/Y32R9guT7cfvbY4Ftdfuka9RjeqAIx47njwToj0Hf1VFFmD1HlM6Raake8cMIK19ayWN16LKkvJokL31/ecnWUJSUTY0fnMvGnTl4qPeNz/Ie+1k8r1H1xGfAvwCKfjiq2Yax97ar7LTkqqlgGnKyypgj4z4TMAayzVafyytvROC05h/GoxtSi4xPHIy7wLT4Qv8c7AHMer5XNE10dbxzozghsZej3GZJXMlx3BdUzYwBPY0bLhLu0db9eyjdemeK6FdXNi3iorMSNe5Uqkk/IvM5xsC80h3kU7qlYxjeoeelAj8ZfhAVSkTi/aig1VRRSeEn8hG9YUfhw7jD9YqSXCn8ldITPGbmIJb23fwZKX5RinzfGKc/bvTbogO5m/wBHCGSJeJGHMVVZOGJz8/0gyMakJoSv7mzSaML8EWDfWPr0jVqpPi/bSugFSuQ9Ecik/p7WhP0X00eGEYRLUs7mSrOPOZLIPvkDalF/Lsm8vsiD7+Uwd53xTtz2YcAVaCYy2Fhg6mBoTBSZ7HTF0gm39hKPCk06Vg9TgMVn2HHvR7p0syr X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 30e7a8e0-80dc-41ef-8262-08db825b8df0 X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jul 2023 22:09:53.9310 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 2v/hRvAo7zkngH0dyxoMRbSbIqXTXZZ/Z+QdYaFpKrHw/gKDVbCrxPiwqpitoD/xQGtkUhqFTHN8eEOyXco8aQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB4775 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.254,Aquarius:18.0.957,Hydra:6.0.591,FMLib:17.11.176.26 definitions=2023-07-11_12,2023-07-11_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 spamscore=0 mlxlogscore=999 adultscore=0 suspectscore=0 mlxscore=0 malwarescore=0 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2305260000 definitions=main-2307110202 X-Proofpoint-GUID: sz9sOU-g3vTel-g6IUEDkDhHKVQ3_okA X-Proofpoint-ORIG-GUID: sz9sOU-g3vTel-g6IUEDkDhHKVQ3_okA X-Rspamd-Queue-Id: C942540008 X-Rspam-User: X-Rspamd-Server: rspam11 X-Stat-Signature: schz5yz1gu7iondy8d3fww73buddyx99 X-HE-Tag: 1689113436-130138 X-HE-Meta: U2FsdGVkX18V2Pf4tnwMI98zhk3deuoYN9qXp8ECuAFoTgdr6gekvDylzVAP1o3IDK9B6MATFP0wztQQZZ7q7op4cRZMu+xdi4xp9LBjb47Fmf5kawCRSGmWeG1sQQoAAdjeLwe0rRee62FdbvPw8ORYKKXlw3Aad//+AqIBirin+Gtx5HQHFzuvWd4g9OpzXkxeEUegkP538DrEmQgQI1QuUIKui55Dq3TO8VuQjKusvuupET4m3S8p26+pin7ducGyBOW0SPGP4l+ducPtFQMM3+1ESowJtstH96GM+OhvcnoMQVA7CFE7prdAOxZT45BIeTFJOvGybs8XRkSwBwZ++VhoPRWhqgJ3CweZZYo7TnzqvQW2rTJz1h/Ipgi2Dh3tYxSa0tM/WKGiWovgPwrZkJ33gZYnK3+fMdmTcgvKUnWQuYH9tj1Hi414vNp2JYum+J4a4nRVz0vYmvw8IHj44VSJgP1J1d1Qc859aqrSW8T2z4GRUxxC+2FDvK+5YwxiEhfZbaZYx8lSuAM7AiJRW2YU5N/ViAa87rHiMNya2Cpt0r/qUdtxE2iOryS3FqJZt5HlRhav9wHp09Dhj0ulHIHATb+l3DEbIql2jtZpFnXAOfFL9xxJd4jaImB75Yxh+qWJlSiRXoMuIpixmw+Kmn4TMkbmhd928jTS3uM6qfCJMxYybuk5doWu2JLfwDaQtvxVyZK07JoeiMSNzVaygwljwI+WXzsSrPaiKRP5kqDmj+gysbZ57jbfQ1YWboDboUjLLBScBiF5xoQ/QDnU7gCNJmDLdJOgsXCHStHl8h31LoQ7p4jcB2pPNNTQ0zipzRcJBosqTspIkj0wCjxHafgpqFGtKva2vzMcJuX1U32HNGwciqwUeYM4Fm62ZpCAwz2Ai1q3NQAYhfhQ6kYkjHqEBt/X+QvmGOPPjYue8bz+YgK+9YlQaCB+WGT5rMqgTJdxUjy1yHuEo6l k9z6po3b Pl7sR0pR7ETKXjM7vA1aYFr3VVIaQhDnjeuuo6MFO+xeQL7jUJEFJ8RPoGS3FTBiwv9yMSf9nS6Lslz413l5dvdf4JTST0+12pOcTzbYxicr4hw6cuGGjyALVYurwb+PWpqTWbObw6oTonDwcwL4h93YTObxsJzhjKzYf5yPBG5qA5y2bJiE+YZ8NvFyCXRX/33wx8BC1GZOyGZ5wUMTQF8dgLgAMLgVSQmGVDYwI+hbrvXci51GCuyWum8kU9ay7zH3bImU+ZOjxeG+d2o9IAV1hlFV76QHq9fstmBmiW2R7hNDg0i/oPyrB2XU7EYofZHG2R/w+rlZY8TT1FOy8jXXbGbSouziVEiMzQYuBnWVpCoUhlxdTqMf15IMpSzVDYFwaupVghrl8Qmbtbf8/O2Iqo59Xap2COeJeUSVKZRRkln70puNdXtuiv7tthwjRWy8f7/jd4rJMfm8YBzu8Fm4a8c3CDMAj8C8CyycGgK0bCF8= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: update_and_free_pages_bulk is designed to free a list of hugetlb pages back to their associated lower level allocators. This may require allocating vmemmmap pages associated with each hugetlb page. The hugetlb page destructor must be changed before pages are freed to lower level allocators. However, the destructor must be changed under the hugetlb lock. This means there is potentially one lock cycle per page. Minimize the number of lock cycles in update_and_free_pages_bulk by: 1) allocating necessary vmemmap for all hugetlb pages on the list 2) take hugetlb lock and clear destructor for all pages on the list 3) free all pages on list back to low level allocators Fixes: ad2fa3717b74 ("mm: hugetlb: alloc the vmemmap pages associated with each HugeTLB page") Cc: Signed-off-by: Mike Kravetz --- mm/hugetlb.c | 35 ++++++++++++++++++++++++++++++++++- 1 file changed, 34 insertions(+), 1 deletion(-) diff --git a/mm/hugetlb.c b/mm/hugetlb.c index 1b67bf341c32..e751fced870a 100644 --- a/mm/hugetlb.c +++ b/mm/hugetlb.c @@ -1856,11 +1856,44 @@ static void update_and_free_pages_bulk(struct hstate *h, struct list_head *list) { struct page *page, *t_page; struct folio *folio; + bool clear_dtor = false; + /* + * First allocate required vmemmmap for all pages on list. If vmemmap + * can not be allocated, we can not free page to lower level allocator, + * so add back as hugetlb surplus page. + */ + list_for_each_entry_safe(page, t_page, list, lru) { + if (HPageVmemmapOptimized(page)) { + clear_dtor = true; + if (hugetlb_vmemmap_restore(h, page)) { + spin_lock_irq(&hugetlb_lock); + add_hugetlb_folio(h, folio, true); + spin_unlock_irq(&hugetlb_lock); + } + cond_resched(); + } + } + + /* + * If vmemmmap allocation performed above, then take lock * to clear + * destructor of all pages on list. + */ + if (clear_dtor) { + spin_lock_irq(&hugetlb_lock); + list_for_each_entry(page, list, lru) + __clear_hugetlb_destructor(h, page_folio(page)); + spin_unlock_irq(&hugetlb_lock); + } + + /* + * Free pages back to low level allocators. vmemmap and destructors + * were taken care of above, so update_and_free_hugetlb_folio will + * not need to take hugetlb lock. + */ list_for_each_entry_safe(page, t_page, list, lru) { folio = page_folio(page); update_and_free_hugetlb_folio(h, folio, false); - cond_resched(); } }