From patchwork Wed Sep 14 22:18:08 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Mike Kravetz X-Patchwork-Id: 12976592 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4235AC6FA82 for ; Wed, 14 Sep 2022 22:18:52 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC4AE8000A; Wed, 14 Sep 2022 18:18:51 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id C7388940007; Wed, 14 Sep 2022 18:18:51 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A29D18000A; Wed, 14 Sep 2022 18:18:51 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 8CB7C940007 for ; Wed, 14 Sep 2022 18:18:51 -0400 (EDT) Received: from smtpin13.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 69721AAB0E for ; Wed, 14 Sep 2022 22:18:51 +0000 (UTC) X-FDA: 79912106862.13.F05169C Received: from mx0b-00069f02.pphosted.com (mx0b-00069f02.pphosted.com [205.220.177.32]) by imf11.hostedemail.com (Postfix) with ESMTP id F2E6840099 for ; Wed, 14 Sep 2022 22:18:50 +0000 (UTC) Received: from pps.filterd (m0246631.ppops.net [127.0.0.1]) by mx0b-00069f02.pphosted.com (8.17.1.5/8.17.1.5) with ESMTP id 28EMIhCf027338; Wed, 14 Sep 2022 22:18:43 GMT DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.com; h=from : to : cc : subject : date : message-id : in-reply-to : references : content-transfer-encoding : content-type : mime-version; s=corp-2022-7-12; bh=YgYkt0QuP9ElyM3GYKSupA5PKhlljwrykt+N//yr6Sk=; b=O0aprwbDZJxPyxCjRnInoT10YiNSrEC8lpjxXpaFpDS8ijPvdi7R/Oe6vMXJ77SSWnjn 3KrYET85Gm6G1OPoaTzCflbChyL4fjhy7hyln/1xkxJiVhWJ6j2mTkh1qQ30zJqp73f5 ZHARUHHVGR5V1UWo+Y+PbzbXebvaTTy5WThG4hQJX7XxXBWYRFKuFgUL7OdcQza62urK k36wbLrIvJS5v2tK6+fqO8x24ZFfR0Jk8f0gOZPM+/9RoSbgYuJhBDYQhK+ri2ZCpDPq ciw00yFISGLRmSrmV5zpHthMzHX2kz4YD8QvTuGaoLJ0G80jxFOH/tNOBRMB+cacjnIX 1Q== Received: from iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (iadpaimrmta03.appoci.oracle.com [130.35.103.27]) by mx0b-00069f02.pphosted.com (PPS) with ESMTPS id 3jjxyduppy-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Sep 2022 22:18:37 +0000 Received: from pps.filterd (iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com [127.0.0.1]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (8.17.1.5/8.17.1.5) with ESMTP id 28ELuQLB009613; Wed, 14 Sep 2022 22:18:37 GMT Received: from nam11-bn8-obe.outbound.protection.outlook.com (mail-bn8nam11lp2169.outbound.protection.outlook.com [104.47.58.169]) by iadpaimrmta03.imrmtpd1.prodappiadaev1.oraclevcn.com (PPS) with ESMTPS id 3jjyedadwh-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 14 Sep 2022 22:18:37 +0000 ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YdHiT0oeWnUkBaGuBpns4akjaLB6DpfplDu7t0to26RGIdrnmx7gyswyzktCLiNq4fDdOlYT5dNU/ZSKrABW8EWFB8HFd7KOALNw9pR2B3hsJxtUPGGQYCERlsVWriY1CGgeQSQb2uH1ch9sTxrV5b2bO0UdEi6cqS9UK8uCNtkRoDPTjcVVLOic+saoZSC7Sqj3YNUFcsNlEBgjLdtQdV++L9AUdDmhMVI326x1ndgfM1TDvZR4+xFCsiwF7UbkqUyoYv28FxA9WnBEeVXlRj36d/ly2LJ/I8jDYTewOh/cbVRYSSdkZEetH/Ek79i6fenMcmQS6eskjK9PIwNr7Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=YgYkt0QuP9ElyM3GYKSupA5PKhlljwrykt+N//yr6Sk=; b=AaojvL71ykzR8wbHHp2vCWRLH4YT6Q8nV+fReKomNP0HzM6khh7EQjvAWoltQ2Z3gLg8W7o/OyiMH/++sos+xaAhLJAXgL4R1cEkXuegG2C9Gvyj0rbFUWK5MW6l4DhqK709AF8GY1hEnsiAmu9UBT0WWTQkh/GIhmSJE6Z8Z/p+BlfdX0yVnIo4ifozlNlONI7ct0m8IDbPDoMnqVJ2bKFCGeO6zjAs8UUXuOJJ83GxDnUPD35BQr6N9zTYU3pOK10LwIHskkysXNPj588AgHtnty9xry7h43tbS+cNS0NprIqc6B6CqOa/zW0800CMwsxDQ6R5b5vx2Q+9P9Ilfg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=oracle.com; dmarc=pass action=none header.from=oracle.com; dkim=pass header.d=oracle.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=oracle.onmicrosoft.com; s=selector2-oracle-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YgYkt0QuP9ElyM3GYKSupA5PKhlljwrykt+N//yr6Sk=; b=O93Zdu38aIhHKC96gJs4lDNvZhBeFmiU/8q3gdAoQDMhNvV2qTAQJZa91L7zLJpup+s5TPomsZbZVpNogauj7WQ0VRcT3K8+P29+LkfP8t4QK1rX3CCFnKL1lr6xxMgZv4t2vqFjgJp9nv/MhE/gxAkxkSk32N0jlR4sSc0P9yw= Received: from BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) by CH2PR10MB4390.namprd10.prod.outlook.com (2603:10b6:610:af::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5612.22; Wed, 14 Sep 2022 22:18:35 +0000 Received: from BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::e9d2:a804:e53a:779a]) by BY5PR10MB4196.namprd10.prod.outlook.com ([fe80::e9d2:a804:e53a:779a%6]) with mapi id 15.20.5612.022; Wed, 14 Sep 2022 22:18:34 +0000 From: Mike Kravetz To: linux-mm@kvack.org, linux-kernel@vger.kernel.org Cc: Muchun Song , Miaohe Lin , David Hildenbrand , Sven Schnelle , Michal Hocko , Peter Xu , Naoya Horiguchi , "Aneesh Kumar K . V" , Andrea Arcangeli , "Kirill A . Shutemov" , Davidlohr Bueso , Prakash Sangappa , James Houghton , Mina Almasry , Pasha Tatashin , Axel Rasmussen , Ray Fucillo , Andrew Morton , Mike Kravetz Subject: [PATCH v2 7/9] hugetlb: create hugetlb_unmap_file_folio to unmap single file folio Date: Wed, 14 Sep 2022 15:18:08 -0700 Message-Id: <20220914221810.95771-8-mike.kravetz@oracle.com> X-Mailer: git-send-email 2.37.2 In-Reply-To: <20220914221810.95771-1-mike.kravetz@oracle.com> References: <20220914221810.95771-1-mike.kravetz@oracle.com> X-ClientProxiedBy: MW2PR16CA0028.namprd16.prod.outlook.com (2603:10b6:907::41) To BY5PR10MB4196.namprd10.prod.outlook.com (2603:10b6:a03:20d::23) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BY5PR10MB4196:EE_|CH2PR10MB4390:EE_ X-MS-Office365-Filtering-Correlation-Id: 4d30b1de-46a5-4735-05b2-08da969f107b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Ek2fhnQWeKBdZE3NGSa3ZTktMmLHX2qwOSDFJB7YZzdSdVd4T/r6Y+hwmQk5IsSJsYIVevMdj205Jdh12XpCWQsYj7mdmsHBAMcj9feGRecaCrt8I3pbNrKm3kq8Jm8bb9B4dUTTq+mGs0L0A7RKLXf1oSp1yLtS580n9WsyPhEaJI1zlT3bkfzTGNRs4k1tjndzXgzjjNdNxVHLOMSvf5NImJkuDcxdUe0Dc+anrKQ1NL9o8YhS3f9K+NzZM3kbkz6opVLIlwDgoe4i+/pJkdnncHU3Fp+W7AUFe6ycPv7zWrSy7YeuuArnttll0A336eLv+b65I6+7ed4fvZnfgf2M6aMI92NbRxVmEjWs5GzhMikq3Zks+OlNdWk8WA3xwezynNuqcouVnMBHTBnf2Vs32V8jRUAR7o3diNQIHLmEFO6gYMjPTGrtSr6urq0fhnQ6oW4jlt9sQl9ivW0ypmU6UFUUDF3KNoBMc7mgipw6z+bVRWa4FIMK2ALmV26QPdlHWUwvXVvHSkciiHIx67H8jPiP/JIN1iqpALV35D0FuCystyzBiXzt/0/ru90Cvv9ggpowP/yhRn+UZGxNQ94u1Kw/07GWs+dn1MSW9qPSP+mhJ9y9c0gcDGYzQ2uer/+52ouT3wHhcHzMtYs8Cl1mY3oyiIsMk3YvksJFeDpDUrlRfWCiVfMI9i4ma+rzMO7CiqP780io4FJu7o4XVw== X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BY5PR10MB4196.namprd10.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(366004)(376002)(346002)(396003)(39860400002)(136003)(451199015)(6512007)(7416002)(6666004)(86362001)(66556008)(36756003)(8676002)(107886003)(83380400001)(6486002)(66946007)(54906003)(44832011)(66476007)(8936002)(5660300002)(316002)(41300700001)(6506007)(478600001)(186003)(2906002)(38100700002)(26005)(4326008)(1076003)(2616005);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: v93JHpVRFbb0PlIOy840rJlLYDtxWdSY/OBYkoI1HoN3Y+Hwo5pdBPyXKaE/5P+mT+CfeVnQuaL7Utv3n6rbnglMv8AxmtjGtdullvU9+8umxktSxu2wcLLIUuqcdAJ9pViZ8KNicnDOK5aViDzNrsluwN+pm0C/pFvaL1fGKlrswpCAN+weievXqYbO3pu+E0eg73caNgqczfaGlTikkvb6ZIz/Z+lUu4Q5Dya/rYBRxVfLe06RAc55UnfOBLFDKTHg/bf5+iDjJXJvA3vc0w6C+xPMj+k8XGC8SOpLjitmj1YozRbCFdAnReXcfNtBLADXxtz/zmNkJwJppvzwoDJSugTVXCzuuPLVqtbjr79MR3IJTmrYf9HY2PWFNREP4NnCJ8m5Xs9AnNo0GtuGiuAKb2NSYuiMXSWJ4yCYZT3k4nvpizf/WxMctHVrlr6+CfDLWg/UTk/jOmz7TXZJNfB4ZIuCPVGBiGecuSaZ4+ex+LF4TjLF4lft2EGIfnevbLUiIlr2eGDzTMFsKO4fQkNuAahMLu3At8BXRPkhRhRjWPh1TdpcuX20GtmNsw3DIMccabGJ6sO6mr3Ujb7gE5RxrQO2LtwEDJhPJKVMqpU4cs8hdYoH2XvP3Rf5jY2coHHNMeewL8W6osPQcZFzJ7oKP7rhL/vLGp5fFwkCNXjBqtayWAn5FEcfU8FrWO9dM36cTYkRoO361KgLtNPLgefT+OCXEO9OjusIkiPMkUXNPWRqSiSYtoucXRig9l9Yis23hxrjKExICMTNWvpl3iCglGDEMyKQzKV2LOm30UNkZqsLZHHtVFhK+oGlLy+L48vwmkTtHLRw0QdyAlvYeT366zbI9GcmmqoMV3Xqh9BSHCo/kigxFnSsMIWMtzz+j4+qllk8NbT4nJhbLk1i7LgqfiMruxIrMzC/kT4WPtj4yIF0mWVJqnCWoBOiQhnsQ6a+SrKXvkRz8Oj/WeZEsyqk6GV4auoET3sz0b+/wPXKRwiKlBXjhjC4IGZDL66OCqbeoRcQablSZcUs++loT7mynvXwLumhEMrfdvHa/FO+24g8yrOml2NesF+ZNj+A74siPUpoHBGVz16OSdzVD1QOz3t3oaJ9RaAEvyfhNRiraCvEQQKPnXq94JDvDRhMoHwtjXJKJtl8KD+vxr1k6npqFm5raXerOzh9s+alwfTn8/+G3nr8rd03AlPomRc2SqiDziO1h+Ht+37PVh9izm2qyn5sgk6HQP8ucEG7iXUASFhq/QvmAL0TA7SePShctwbkvb+UlLNj/Ib5FmnweA9o88EgHiWGEU/C4DDqu2V85Z2x7ZmFF4QLxJJ15nCgofnyeVZ57xwftcfMSwIw51MsCUHzQjngi7raZPV1uiFPz8Pp2yf5/bTQYpuALEb8x3cGHGFSxtFNiTNJjlxDAs8fg/6HCnyXUvFSZcjeuKIhkZCpL82D8df8CkaYrYetsTs6lCQLF+elWX09FtFnxsEfteeP94rVhgp68XS9YhSBRQ/11/vwpAaV1rTDWd6xb7L3cEKLIkRl90UTaqcCSEDkLrOgvS+UN17iEWkEY7+LocPZBGeXZyIGt7Nui/eHhoJxlMFpg93wWYgUQvazJA== X-OriginatorOrg: oracle.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4d30b1de-46a5-4735-05b2-08da969f107b X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4196.namprd10.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Sep 2022 22:18:34.7591 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: t5brwB27MUbXHWRq+WQd1JHiQ56b8HFyk8REbfJuQMmEH3OW0DVfoLI8hsBVjczE3AgUlwldaEVTRn3uMvUIug== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR10MB4390 X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1 definitions=2022-09-14_09,2022-09-14_04,2022-06-22_01 X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 suspectscore=0 adultscore=0 phishscore=0 malwarescore=0 spamscore=0 mlxlogscore=999 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2208220000 definitions=main-2209140108 X-Proofpoint-ORIG-GUID: CCej2BkDRArpAEAYlJMrnFMEMtH-umz8 X-Proofpoint-GUID: CCej2BkDRArpAEAYlJMrnFMEMtH-umz8 ARC-Authentication-Results: i=2; imf11.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=O0aprwbD; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=O93Zdu38; spf=pass (imf11.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1663193931; a=rsa-sha256; cv=pass; b=OU+sEz1KocfrXp34x2RsnE0+wejbjM5Kv+fKQt+nKNmnIRjqgTAAIxX1AfylFsVH6hurTg 9B58EmwLsSaJ4GjISfP3w/fnupf4scrPD7QHP2aQWvjjC2Cgdv1xqVsQn1zHUymZkuPQey capt47NZtVOZs9XqnsCARcMsPaUgx8o= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1663193931; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=YgYkt0QuP9ElyM3GYKSupA5PKhlljwrykt+N//yr6Sk=; b=qx1yTO+mFwomsTgBLygeBIAtwZsuC8dZOxD8xGVPAsG1UOHr+hSmORs72xejzxSAaPQYGr xY0/T2uPZR7q9nuNOK2DBQ0uX8/AFL7efCXNd0bf3fvJxSRVVV77sODy46gi/G+snUVFJx pH0oyzPhNdni34+pRE/6DUFxBEUNQ4k= X-Rspamd-Server: rspam10 X-Rspamd-Queue-Id: F2E6840099 X-Rspam-User: Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=oracle.com header.s=corp-2022-7-12 header.b=O0aprwbD; dkim=pass header.d=oracle.onmicrosoft.com header.s=selector2-oracle-onmicrosoft-com header.b=O93Zdu38; spf=pass (imf11.hostedemail.com: domain of mike.kravetz@oracle.com designates 205.220.177.32 as permitted sender) smtp.mailfrom=mike.kravetz@oracle.com; dmarc=pass (policy=none) header.from=oracle.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") X-Stat-Signature: 8d4xq43t3iubci1snqfuefttxe69ouxo X-HE-Tag: 1663193930-416644 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Create the new routine hugetlb_unmap_file_folio that will unmap a single file folio. This is refactored code from hugetlb_vmdelete_list. It is modified to do locking within the routine itself and check whether the page is mapped within a specific vma before unmapping. This refactoring will be put to use and expanded upon in a subsequent patch adding vma specific locking. Signed-off-by: Mike Kravetz Reviewed-by: Miaohe Lin --- fs/hugetlbfs/inode.c | 123 +++++++++++++++++++++++++++++++++---------- 1 file changed, 94 insertions(+), 29 deletions(-) diff --git a/fs/hugetlbfs/inode.c b/fs/hugetlbfs/inode.c index 7112a9a9f54d..3bb1772fce2f 100644 --- a/fs/hugetlbfs/inode.c +++ b/fs/hugetlbfs/inode.c @@ -371,6 +371,94 @@ static void hugetlb_delete_from_page_cache(struct page *page) delete_from_page_cache(page); } +/* + * Called with i_mmap_rwsem held for inode based vma maps. This makes + * sure vma (and vm_mm) will not go away. We also hold the hugetlb fault + * mutex for the page in the mapping. So, we can not race with page being + * faulted into the vma. + */ +static bool hugetlb_vma_maps_page(struct vm_area_struct *vma, + unsigned long addr, struct page *page) +{ + pte_t *ptep, pte; + + ptep = huge_pte_offset(vma->vm_mm, addr, + huge_page_size(hstate_vma(vma))); + + if (!ptep) + return false; + + pte = huge_ptep_get(ptep); + if (huge_pte_none(pte) || !pte_present(pte)) + return false; + + if (pte_page(pte) == page) + return true; + + return false; +} + +/* + * Can vma_offset_start/vma_offset_end overflow on 32-bit arches? + * No, because the interval tree returns us only those vmas + * which overlap the truncated area starting at pgoff, + * and no vma on a 32-bit arch can span beyond the 4GB. + */ +static unsigned long vma_offset_start(struct vm_area_struct *vma, pgoff_t start) +{ + if (vma->vm_pgoff < start) + return (start - vma->vm_pgoff) << PAGE_SHIFT; + else + return 0; +} + +static unsigned long vma_offset_end(struct vm_area_struct *vma, pgoff_t end) +{ + unsigned long t_end; + + if (!end) + return vma->vm_end; + + t_end = ((end - vma->vm_pgoff) << PAGE_SHIFT) + vma->vm_start; + if (t_end > vma->vm_end) + t_end = vma->vm_end; + return t_end; +} + +/* + * Called with hugetlb fault mutex held. Therefore, no more mappings to + * this folio can be created while executing the routine. + */ +static void hugetlb_unmap_file_folio(struct hstate *h, + struct address_space *mapping, + struct folio *folio, pgoff_t index) +{ + struct rb_root_cached *root = &mapping->i_mmap; + struct page *page = &folio->page; + struct vm_area_struct *vma; + unsigned long v_start; + unsigned long v_end; + pgoff_t start, end; + + start = index * pages_per_huge_page(h); + end = (index + 1) * pages_per_huge_page(h); + + i_mmap_lock_write(mapping); + + vma_interval_tree_foreach(vma, root, start, end - 1) { + v_start = vma_offset_start(vma, start); + v_end = vma_offset_end(vma, end); + + if (!hugetlb_vma_maps_page(vma, vma->vm_start + v_start, page)) + continue; + + unmap_hugepage_range(vma, vma->vm_start + v_start, v_end, + NULL, ZAP_FLAG_DROP_MARKER); + } + + i_mmap_unlock_write(mapping); +} + static void hugetlb_vmdelete_list(struct rb_root_cached *root, pgoff_t start, pgoff_t end, zap_flags_t zap_flags) @@ -383,30 +471,13 @@ hugetlb_vmdelete_list(struct rb_root_cached *root, pgoff_t start, pgoff_t end, * an inclusive "last". */ vma_interval_tree_foreach(vma, root, start, end ? end - 1 : ULONG_MAX) { - unsigned long v_offset; + unsigned long v_start; unsigned long v_end; - /* - * Can the expression below overflow on 32-bit arches? - * No, because the interval tree returns us only those vmas - * which overlap the truncated area starting at pgoff, - * and no vma on a 32-bit arch can span beyond the 4GB. - */ - if (vma->vm_pgoff < start) - v_offset = (start - vma->vm_pgoff) << PAGE_SHIFT; - else - v_offset = 0; - - if (!end) - v_end = vma->vm_end; - else { - v_end = ((end - vma->vm_pgoff) << PAGE_SHIFT) - + vma->vm_start; - if (v_end > vma->vm_end) - v_end = vma->vm_end; - } + v_start = vma_offset_start(vma, start); + v_end = vma_offset_end(vma, end); - unmap_hugepage_range(vma, vma->vm_start + v_offset, v_end, + unmap_hugepage_range(vma, vma->vm_start + v_start, v_end, NULL, zap_flags); } } @@ -428,14 +499,8 @@ static bool remove_inode_single_folio(struct hstate *h, struct inode *inode, * the fault mutex. The mutex will prevent faults * until we finish removing the folio. */ - if (unlikely(folio_mapped(folio))) { - i_mmap_lock_write(mapping); - hugetlb_vmdelete_list(&mapping->i_mmap, - index * pages_per_huge_page(h), - (index + 1) * pages_per_huge_page(h), - ZAP_FLAG_DROP_MARKER); - i_mmap_unlock_write(mapping); - } + if (unlikely(folio_mapped(folio))) + hugetlb_unmap_file_folio(h, mapping, folio, index); folio_lock(folio); /*