From patchwork Fri Sep 15 22:15:45 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 13387791 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48E85CD37AF for ; Fri, 15 Sep 2023 22:16:59 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id B33026B039B; Fri, 15 Sep 2023 18:16:55 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id AE2696B039D; Fri, 15 Sep 2023 18:16:55 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 871C86B03A0; Fri, 15 Sep 2023 18:16:55 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 72D3C6B039B for ; Fri, 15 Sep 2023 18:16:55 -0400 (EDT) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 49D54160FF6 for ; Fri, 15 Sep 2023 22:16:55 +0000 (UTC) X-FDA: 81240242790.16.FAC6932 Received: from mx0a-00069f02.pphosted.com (mx0a-00069f02.pphosted.com [205.220.165.32]) by imf07.hostedemail.com (Postfix) with ESMTP id D186F40027 for ; Fri, 15 Sep 2023 22:16:51 +0000 (UTC) Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=cVlxG1zi; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=Rvq6XiU6; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf07.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1694816212; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=fR0foe/sV5Sumv5EvmNzyoFAm34AhjLE3kM7pxcILow=; b=rL2cHpTXt2p5vXr7ekgfm4vBBM5hMol6XQ0flhQG/6csge9lft8ytVcgPinZixah7CLQtX aH4muFiRA6R6OQLV12XBjWpSbUow6w3gU7X1TANAfzO6OdRux4sxBQ5qN7kILWC7ksMMJm YrneyXXVZY1GW2HgDFuJ88wJgfiYcEM= ARC-Authentication-Results: i=2; imf07.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2023-03-30 header.b=cVlxG1zi; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=Rvq6XiU6; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1"); spf=pass (imf07.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.165.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1694816212; a=rsa-sha256; cv=pass; b=j2EEKbaUEYvt801iF+dk70Jrl05sYdWfBB66fTvPTU6X04I6mC8YMEVoubN+rE6nx8PTr1 sXMlTKHk+v/bzJ9pVgqzmnJQIR1e9VCtPj9jZfa7/fpH8hZACvaTzmiOyY3Uet33r+Pke/ JYXiAUENpcyUj0dO+a+5oT0fsCUcBYA= Received: from pps.filterd (m0246629.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.19/8.17.1.19) with ESMTP id 38FLviaD025851; Fri, 15 Sep 2023 22:16:28 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2023-03-30; bh=fR0foe/sV5Sumv5EvmNzyoFAm34AhjLE3kM7pxcILow=; b=cVlxG1ziPRjon1lSiAP7Gx8+qeBaUba/zMsYSP7cnMvh3ZghrI0ThFgEDtn3LNoODhh1 3oBxSCiDq71i2Ufua6jP/c1IjrJWXgxFWLAV2LTXMPyIxXcdEvQCjtakluxEGugEmU32 rwQq7+Xpt951BEgucBtIeBL8UnPXGA2MUJdXN2JZSgjvAq3p8RiyqNXnHHADjSb/u0Xw +hzVLERmorw36n2OJfVhHUU0doEBBc6RawYrfU0TKfKwQixlGEegvx5XmTmVkBBJZyt9 PeWdySeJVEtEC3Wf4X2rVdS1pP60GtXjOchu7IMgzSbOpatEdRApvZ2mIz27FUg/CV1v jQ== Received: from phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (phxpaimrmta02.appoci.oracle.com [147.154.114.232]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3t2y7n0k4y-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Sep 2023 22:16:27 +0000 Received: from pps.filterd (phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com [127.0.0.1]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (8.17.1.19/8.17.1.19) with ESMTP id 38FLb0xI008807; Fri, 15 Sep 2023 22:16:27 GMT Received: from nam12-dm6-obe.outbound.protection.outlook.com (mail-dm6nam12lp2174.outbound.protection.outlook.com [104.47.59.174]) by phxpaimrmta02.imrmtpd1.prodappphxaev1.oraclevcn.com (PPS) with ESMTPS id 3t0f5awd55-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 15 Sep 2023 22:16:27 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=nMWIzJuAObHBG52dU6yhF0tD5WRm4WNISyH04Tt116RG0MWW52Futa/X2OSAJpjejjH23xZSManhbgFlm32yac/Mk+/wDu4rQlcbRJCwoVBmxbtDZUOO/aM91J0YvzcG8jm8ncIQPYoKbW35rixP6kvxxt3LCM5Fwq1TWC+PrWEDoZ61XVN5r+UzPE88OlJY2DMnAtg96cw19hyqZL4pEOsy1ZX/IUzs7OH4ME7j2NW9q2LPWT+Pk0NDDF7erT/VshsTbZAY+H/qggO2ntDrw7Ad6e3phwiWwOXsldiabCj+6wyWP5MXBfSM4bsNYVlWERPT2jZ9xvvowWdRRnBwaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=fR0foe/sV5Sumv5EvmNzyoFAm34AhjLE3kM7pxcILow=; b=npA5NsGIMmG1a+7X0arzQ70apfAQCOlVoYEh8KsI/B1/bdVY/2PvDT2Qf2KN4a12aw0XBZhvvgSdazS8I9IRJC7DvOS9zh2i4Q8/o0j0MP+Q3OgavWiXDBs247PrmcrA7FVWNE0RAhLXwcu+3/dxNUov5O8c3k/AYlGnoNEtu/r89dTRMM9aQhBIIYUx75FsZG8k0jQflJdpEuJaWlx3RGqQ2jovT8BhZTYNSVVTGgxVyLcQi4rBokHSLF0+losnr8fuIydvXgh1Dnbtkn0M8GuJlL1PLicVMOU3KdDXfHoNmQ/WKZRicAXrxZ+ExBE2ofz0E9EyoMZXizASW/muTg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=fR0foe/sV5Sumv5EvmNzyoFAm34AhjLE3kM7pxcILow=; b=Rvq6XiU6wjNWckoP9ZvRctqY9JXUAJ5ksgqfl3X/H0fQDGzGvQ5qtaZSx+vcTrBUTvaZVVk5l9G4oLaAP2XBUsb1enpp7oEXZhiBwbEjxptljtpVJiLw09MQdLsEX1BQjSpyW4SZAH6iEC1HGpXo5zaBSgy0EkzFtUXpPyfgH+U= Received: from BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) by MW4PR10MB5861.namprd10.prod.outlook.com (2603:10b6:303:180::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6768.34; Fri, 15 Sep 2023 22:16:25 +0000 Received: from BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::c621:12ca:ba40:9054]) by BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::c621:12ca:ba40:9054%5]) with mapi id 15.20.6792.021; Fri, 15 Sep 2023 22:16:24 +0000 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Joao Martins , Oscar Salvador , David Hildenbrand , Miaohe Lin , David Rientjes , Anshuman Khandual , Naoya Horiguchi , Barry Song , Michal Hocko , Matthew Wilcox , Xiongchun Duan , Andrew Morton , Mike Kravetz Subject: [PATCH v3 12/12] hugetlb: batch TLB flushes when restoring vmemmap Date: Fri, 15 Sep 2023 15:15:45 -0700 Message-ID: <20230915221548.552084-13-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20230915221548.552084-1-mike.kravetz@oracle.com> References: <20230915221548.552084-1-mike.kravetz@oracle.com> X-ClientProxiedBy: MW4PR04CA0201.namprd04.prod.outlook.com (2603:10b6:303:86::26) To BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR10MB4196:EE_|MW4PR10MB5861:EE_ X-MS-Office365-Filtering-Correlation-Id: e8ca82fd-ba7c-4bd1-62c8-08dbb6396638 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: FnYUU+jr7yYOLNKTYoVsW5nqTom7s3SrhSiJoExhqGgoC/rdPlqItVgBfBmM1U4gogovk7gQLLVVNHoy9OKdqH9RYtXWaJ9ncNVoc9EOyM6FpL+S1DYm9tDyAojHbi4TEbreEia7BDJd35fxKYiUDqt0HvsM7aVNAJPbCTiBHy0WgQo5i3DHt/5q7dwcnIqmiNP1EhS7Jilf5X7LbKE02/E4/EIcXwqVIQprvmI1tX96BXcx0mwWINxZS58inx5jjv/kNGCSB8G5RnTmdCs7BjZR06sU2OADkz0gMD/1nXnyaBkx+EC/XXII18K5FRI152fWm8rw1vidLgFPPiiiiFRCLTCHhIR0YaUmXHoFC6mMgCaDDLYhdJRoGQLku4vpfQ6bd24ey+6LIj4gkHyz8DYc4usiglDCgLppGLqgoTMox4sW1Tycl08X+/8uXSGbY7ZCRlfvyy+Z7vt3DF3Y5/LHhP2WWWBXQr+jHA2cx4kXNoOaad0rcY8uX/FXoBqTNN5BIFF3bzV6SGlUoBijC8FI93otxx/3WlvXAeYZqJr+ULe+EoNJYW7apT/rzxfn X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(396003)(346002)(39860400002)(366004)(376002)(136003)(451199024)(186009)(1800799009)(44832011)(86362001)(36756003)(83380400001)(2906002)(38100700002)(478600001)(1076003)(2616005)(26005)(107886003)(4326008)(6512007)(41300700001)(5660300002)(6506007)(6486002)(8676002)(6666004)(66556008)(316002)(66946007)(8936002)(7416002)(66476007)(54906003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 0uJwDlDZxbAHEdtFmkAyMsyQdOMk8QWEWHvgpWCHt0RBK59yIJW+FPIOrCOTGfbQMkmlAz6G8zX7thrqx4d9LrbqLZpYHaClHeGRmfkMR8HwkOkXkODrr+vBizD6H3Eew+4gf7AxoHo4gjkPD4Fi8wmUgTV0tUx/fQQOZmtT4RKZhlPkC4OZBV4zVekkFVvbRX+fi7T7MFf56lAfksRMmW1az5yPctW6PoTvu7pcr/e396IRbIsRn/mQRkSo/GOjuJ34VtqYhez3icf2PAnW/5IYWT8BN/ABNnqAahC+oNtSTauQmHSFAAC8ofImVhageKRVBwOGYaXFvAp+mczVA9My24lkxBuIWjOLKf85yLfqpaXMdpBnAoBFYwYq2QaIV173QqUDkJBS/xX48pWkDzw3277DkVj2pEbhhLUSfkfTYxk3M501iBLB0LZ431XsBbh3yYCFlsR3kNbrL7qQobii0WAHyDZBC1W6AnIzo4DAV3ypTd/w7IuqHKT7RXbR+xPmYSEAlzloSnIl20AdF4l+C2Y3qcJonaylm6HVZxhb+yi051pbvDYpmasLzil3GGXIQIUkmA73RkXvxjjgLgU3ZbyGqyyQDkDEs6lLKmlvmST+YSKZO4U3tjhP5pUeBMMGLZBpY9epVm8RdYFKEDPAfgQAJotd78CDVEwq9iEI7v48Tousi1jRgt4YdOS9I9XLv/TKbS7PhBl03hNv3dEFECtfeJanueRlilZzek7/HyHhqUStX28SSOYseeS9tKjQAhi36acSogCstzNezsWO4xFLHfUQj040Et0nZE3jYT0k8YX5TjrRmovitp1cfe8dj7zHVlKgECEQlUX2GDJK+IA6a4OmxqRzxB7PLkz3DV0zYyVKZVhkhv3/nJ/heEEfRXlhHTdMZPgXKHSwEhemKRsncnn7DaHFWnnGRl6xKAazHhffuQAG3mV95qOrytJEKY7cjCoqPc2cWShKHr05rbUQv7KAFXYYiBlGbVGFgsyqLwU8vE3wFGWY74Q5ubwenz4NZpLhCUL2FL+MLrK7XuYbPBiVnrnGiTqIJGCL4Sf7pTEkJeP864wkjW1WrBB+7ewbaufVJ9Kj0r9KP14yGnRLwrgrpuZ0pJlA4FJ1dMXvBkqvFDx6I3epWAer3ltYL6RxHfQtE3WA6vsR714y0McvLWd7g8OR7lZLZEheEG/ytOBpOSKo7oFfydyk+YXK8vVyYTP1Ki/43O8+wIo618pjjxtE+bGtShtDMUDqjPviGt3vhWZTsBUGNlTTanGfnrJX4JsyJWUKU+p6xtUVOQz8GvaPA8A2zOR1dMGlOLZ+U2uTlkqIGUKxDCYLf9nGMHn2X3uHoYn0TluVCGGEJvkN0dDuYR76xP/sbSp8acSwmwmL0bpGTmblk/wB+RrV2IGLuvf1SfVbPRX7TGuBmGjxyTnWWSc2Ff9FaleZwCyCZm7L2WJVb7subBsIkzMPZWnsYbJJKVWwUIrt3mHJ0xRph3vJO8h4/l/nusGXzmh1QG6D8LPJnbeaG+5JRZxMGJDGN2pHVOURkYS/H99/HGEiW6wL7OiD+qUiDeMYcWnntwo4JJ4ghA6cuVZ2kSiUQ51/VcSlEbMHeK8DtA== X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: noXzunz+VW1vfjvQamsOS6IEmED47Z7TSxbIp4gbo+QT6Nj7XcImYythUBKPj2t8KtZkjzvXhR5uq8tNWENn/VjkWPWhdLHSVcK8EQLj65Hun3q+ZOBkrb9pLFqzgw8SRZnei5ufFaZF9iPUjm7b3/+M+3qeFMiTejFIwxKQEDLLhWAUS+mwnjiIUx9HALwF/occtqh09A2b1SnWebAGNOfZ97F/oKNLBGAnTsPq3VstTEgRvR6OKC+owg7/yE1sWz19jQLguGFC9VbaRVnGQRW5zFVqD7/aWiw1Olr/fXKSLCm2tXhQE0vORYJ5RQilp4e8UqOdg6jR4/UEeFVmxeYusy22hP1lVl+Z8uAPSnqSC4+jHGZU2nkg8SnsRU93XsYFMwDO47TYyINskhHV5dx76YegUAaVzANp3RGMEIZkbZZKe3kArv8fQJOiVjKeLvhC+Prq9bAQafdvFjsgJVgrSl6DNsBfVRGZhNOTkSByCZzTQ0lV0f9k4bMboYwdunE4lBuNEfqeNVV4BA/aQWJcDyuqDJ08taSsC6ZChcTPASOa+/5uTsmy64ijCoovAmX4EYBgrEsKRbYu4fc+vmKlW61Xhotzv2owus0HqVUghUp5e+8lco8U+F/sA1LbvFTszLwkz2oD4fHBXS8eEGLN/gWafYDqQ0r1L/L4qYiJSD1FAlvHAwmosxNjI5n/E5Qf9cF/aYXPRYUB50GWLmbD+aKWF3DT/qds3DpPCIeL7NCQAyIumq6is0xss7p4LEPxFoq+awhXU4AXv3gpiE0/AWRSudkmCZInjKgMaccm6FW0tbiGmh24gbWsHzH9yIry3e7kjI97KpY1XhXein6Nu2cFC/YzTJTrGhwRYZpCni4cka1Wwti/jvnpUwTlM0p2scVkxY8oHo2AW6HCaksrbIj/mSz2ow5pXT63bmCMPf3mHXvO+dkcbt93pIq4kFHyDOVaBNK+b99iD9YKjJPmCsQp7MDuvmnq+yf7w909yhX9MUtna4aq/fuucHudebMM11HzSvJ2GNj+RqCxWd9lGbGE2Sdp0EChsJWKWtmP4IF3/G1AfL2pujxfSzFQZBRuTVqqYlu/7XkeC3Dk8YRrHIuM7myMATN3I1wr2T1RiR2ohsykU36cKEUc/Eh8 X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: e8ca82fd-ba7c-4bd1-62c8-08dbb6396638 X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Sep 2023 22:16:24.8774 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 4xh6rb46FVTquit+qOmfQh4Nx7cmfoIC8fMJuy28A8LajsLFEnXguRwk4dLstxJ1KqY1mgDNZl2HCGSCWSB5Mg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR10MB5861 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.267,Aquarius:18.0.980,Hydra:6.0.601,FMLib:17.11.176.26 definitions=2023-09-15_19,2023-09-15_01,2023-05-22_02 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 adultscore=0 mlxlogscore=999 phishscore=0 malwarescore=0 mlxscore=0 spamscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2308100000 definitions=main-2309150200 X-Proofpoint-ORIG-GUID: _7KwDLtVsLlD14n7b_LTNmnQAai-P1V_ X-Proofpoint-GUID: _7KwDLtVsLlD14n7b_LTNmnQAai-P1V_ X-Rspamd-Server: rspam09 X-Rspamd-Queue-Id: D186F40027 X-Stat-Signature: jd84oob648cjqqgfidwfdwbrs641nt1r X-Rspam-User: X-HE-Tag: 1694816211-168655 X-HE-Meta: U2FsdGVkX1+RWWO8C23Z8YR395jH21C6ABXpHhzFUZPrhrRVTos+8x7n5qUsiQLC5M0S740OtJcjg4fkaXKIlAcKFvVoly62g1BIibw0iAap6M6FzHWMfgeCWLDCfAKqvLc9NnZA2yGKfN7N6b/J7hoDRURRmJ8lss8uvljuDngpe80BIymfcbyZCkRubX+shMz1WsZC4ChH3/lfkZoHHZvFpwEZfJtwyqjxvqe/da3EiILlw5sxWuP9aQAowrjaugwFCuFlTWLhHcNGDIaNjC2kS0SuW/8yLxlncWOaUaJsTAveJ6PKaHZFdYSZ5EHcsutp/OBfPhp6uOc2jg+BoQ/a86mtW73m1gqY1hk7rY8k0Qc3IYLQ8IrQOuqlCLc6YjAYdSX5BHAF0P+ZDJ5QcYkhURw0YCO1G97zDM2j4AjXGOhHGHO+gvgbCuLMjPFXt+KscHPzk3Uhq78iEzYUDPklIQS5khW+FEPnltLFt1LirXYrX4muQujGjHcy8a2QYD9IRjse+9/LMh91PDVAy3O+Yw/mmUBBiPluS68CegP0O7JZqeZrdIycRJ5RXWekrglsnOPme/DRy+UVjf3MOGQgfQfS4N46m2LuiHSfoTtkh42OEp8AaFQIIZfMYfJgPLHqipxIraB3/N5QeeSO2ZOxTM4NpdAIWqgJ6pdbZLmmx5GxwjPk1zrorOY8jJ1RXZwac5QwQAY1JLREuEkpQ4RNJJIRy/Vc+DNi0LrJR9rIxCbS6x8oHLURm1nckRokmjkI2z6Z26zI+9Chi9ASJcwZtLlRyC6fsCRuINdWV4t5X2blsSGpjEdQXCeiZgA0RvDM8w83IPq1S7ikXqFHdSE4Lejou4s7m5d/aNFl7AtoSE0ZTy+t+QIK1KDeURty/2RHAtN0iFZc2MWmdsNqhbNA6CTRZMXb6WASiu7lG1px/Hy7zT7EmzEBICHg3WbgPhjb4bU0NzKDvxJTKlM 21V4wvoQ dDBwOAlbc8a/00doNAPmQkZrjWWqJzqMpTKfHxzCjzoVKAXSeiRIJ03Eq94MKk4GrZ/pPsBQGL5Q3Gwqe8y3dMPtsrSC5pJxGoFvt/7ujPzKyEOFv/493gwoD6UdWRqhE5R9G5coAQmYDrqVvu0kprszyJIug6Na1NQwhbL5GS+0a/HYx/1jpbI8fq9DbX6I6i8CBTPrJpfjxYFMcmL0zmpmUJy0Ow5c1YyrFUDKWw2xBYlufynxBB6cWCEb98s2sj2+8FQ/8nhAFMVRRICLzpB0pS9G77kUq4Hgxp82sgJ5iH9f5xfDG2qebyuEljRTO/IPymzP3+WjiKqHBYWzuuqrBiN4lXetMTU7k7Ncb9b77RVwAIFKe5Pa2gHDW9Jh7SfP/yZr3GDnBjMm+siaGoQE8z5l9kKIMA8fyYJfxaLnMh0EpARL9c88bnYeVFAxLli613fHGlcRjpWWLv7oIHc796a3XptntQ9gc4mb8K6vY2KJq7KQBPKrRry6nwYCvufqk X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Update the internal hugetlb restore vmemmap code path such that TLB flushing can be batched. Use the existing mechanism of passing the VMEMMAP_REMAP_NO_TLB_FLUSH flag to indicate flushing should not be performed for individual pages. The routine hugetlb_vmemmap_restore_folios is the only user of this new mechanism, and it will perform a global flush after all vmemmap is restored. Signed-off-by: Joao Martins Signed-off-by: Mike Kravetz --- mm/hugetlb_vmemmap.c | 39 ++++++++++++++++++++++++--------------- 1 file changed, 24 insertions(+), 15 deletions(-) diff --git a/mm/hugetlb_vmemmap.c b/mm/hugetlb_vmemmap.c index 921f2fa7cf1b..0e9074a09afd 100644 --- a/mm/hugetlb_vmemmap.c +++ b/mm/hugetlb_vmemmap.c @@ -460,18 +460,19 @@ static int alloc_vmemmap_page_list(unsigned long start, unsigned long end, * @end: end address of the vmemmap virtual address range that we want to * remap. * @reuse: reuse address. + * @flags: modify behavior for bulk operations * * Return: %0 on success, negative error code otherwise. */ static int vmemmap_remap_alloc(unsigned long start, unsigned long end, - unsigned long reuse) + unsigned long reuse, unsigned long flags) { LIST_HEAD(vmemmap_pages); struct vmemmap_remap_walk walk = { .remap_pte = vmemmap_restore_pte, .reuse_addr = reuse, .vmemmap_pages = &vmemmap_pages, - .flags = 0, + .flags = flags, }; /* See the comment in the vmemmap_remap_free(). */ @@ -493,17 +494,7 @@ EXPORT_SYMBOL(hugetlb_optimize_vmemmap_key); static bool vmemmap_optimize_enabled = IS_ENABLED(CONFIG_HUGETLB_PAGE_OPTIMIZE_VMEMMAP_DEFAULT_ON); core_param(hugetlb_free_vmemmap, vmemmap_optimize_enabled, bool, 0); -/** - * hugetlb_vmemmap_restore - restore previously optimized (by - * hugetlb_vmemmap_optimize()) vmemmap pages which - * will be reallocated and remapped. - * @h: struct hstate. - * @head: the head page whose vmemmap pages will be restored. - * - * Return: %0 if @head's vmemmap pages have been reallocated and remapped, - * negative error code otherwise. - */ -int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head) +static int __hugetlb_vmemmap_restore(const struct hstate *h, struct page *head, unsigned long flags) { int ret; unsigned long vmemmap_start = (unsigned long)head, vmemmap_end; @@ -524,7 +515,7 @@ int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head) * When a HugeTLB page is freed to the buddy allocator, previously * discarded vmemmap pages must be allocated and remapping. */ - ret = vmemmap_remap_alloc(vmemmap_start, vmemmap_end, vmemmap_reuse); + ret = vmemmap_remap_alloc(vmemmap_start, vmemmap_end, vmemmap_reuse, flags); if (!ret) { ClearHPageVmemmapOptimized(head); static_branch_dec(&hugetlb_optimize_vmemmap_key); @@ -533,6 +524,21 @@ int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head) return ret; } +/** + * hugetlb_vmemmap_restore - restore previously optimized (by + * hugetlb_vmemmap_optimize()) vmemmap pages which + * will be reallocated and remapped. + * @h: struct hstate. + * @head: the head page whose vmemmap pages will be restored. + * + * Return: %0 if @head's vmemmap pages have been reallocated and remapped, + * negative error code otherwise. + */ +int hugetlb_vmemmap_restore(const struct hstate *h, struct page *head) +{ + return __hugetlb_vmemmap_restore(h, head, 0); +} + /** * hugetlb_vmemmap_restore_folios - restore vmemmap for every folio on the list. * @h: struct hstate. @@ -557,7 +563,8 @@ int hugetlb_vmemmap_restore_folios(const struct hstate *h, num_restored = 0; list_for_each_entry(folio, folio_list, lru) { if (folio_test_hugetlb_vmemmap_optimized(folio)) { - t_ret = hugetlb_vmemmap_restore(h, &folio->page); + t_ret = __hugetlb_vmemmap_restore(h, &folio->page, + VMEMMAP_REMAP_NO_TLB_FLUSH); if (t_ret) ret = t_ret; else @@ -565,6 +572,8 @@ int hugetlb_vmemmap_restore_folios(const struct hstate *h, } } + flush_tlb_all(); + if (*restored) *restored = num_restored; return ret;