From patchwork Tue Feb 18 23:54:43 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13981103 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DB120C021AA for ; Tue, 18 Feb 2025 23:55:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 771552801BC; Tue, 18 Feb 2025 18:55:17 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 721412801BB; Tue, 18 Feb 2025 18:55:17 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 5EB1F2801BC; Tue, 18 Feb 2025 18:55:17 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 435872801BB for ; Tue, 18 Feb 2025 18:55:17 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0E0AB120769 for ; Tue, 18 Feb 2025 23:55:17 +0000 (UTC) X-FDA: 83134724274.20.A116B74 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2070.outbound.protection.outlook.com [40.107.237.70]) by imf23.hostedemail.com (Postfix) with ESMTP id 46E4D140004 for ; Tue, 18 Feb 2025 23:55:14 +0000 (UTC) Authentication-Results: imf23.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=CELEFCGS; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf23.hostedemail.com: domain of ziy@nvidia.com designates 40.107.237.70 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1739922914; a=rsa-sha256; cv=pass; b=JXeTgfIlJ/HkTRraRed0CHJ0CHhyiWk00jgIz9yw3WMh810P1C4NQ0kNbU30fi/ts8YxH0 wtF4+yT4YCxIPCqFb/12TSwnGrGLtyX6SEcCbxSLdpaQPmY+ZB60/lTX94F/jd9X99b278 xjHJbVsvjtb+Zov7uSIWLxi7jcWS4ZU= ARC-Authentication-Results: i=2; imf23.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=CELEFCGS; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf23.hostedemail.com: domain of ziy@nvidia.com designates 40.107.237.70 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1739922914; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6QuCbMHib6zbVMGyf2zEu2fYzLX/UadW3QhlKETvqro=; b=Hr96FnIUZ3XQHm/arM7id26+F3Z+aQtYDEi98dRmk+zmri6synvPMnNWJgCnX1kbylJgrp GsNcCk1yBz5cYOROKVS+hi+0TVhZTL+H2P1oqO0vf3EirEVsEGf0fvv6ZclcreVnrVd58g 2Rp+4uW6NurlY/2f0ymwXNdTyBaipFM= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=HSxK2K3HAA2cXtJ8wcGRoyHu3k45C6cJa7DZ5Kt70P8RdPDKWIagydtjKFUn4hqAxqkcrA1XYuMk0wFOrkCShIyW4NkiCrGsdhWu7b5pyo5yHwUAgJPwLRy3rGk/nV9v+/lL2PCfA8IanVjoMLE+bT7+kcPdzNBP5NAisNA4djF4JTLquxRAPHUNaXSlXw8kGCmoePR4gQ9yfox/u/V4inTXBwnvjzV/kEdpVD1l0B5nTOLW9Mck+Yo5724KIsYQOP+T4JI7sLzwGFTvWFIw4Qj1rTJT3jkE2jGkeM0ApU4o/snl6yULK0zsWesOQRs38RzMrIg5qCoA4IrvxSY4Zg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6QuCbMHib6zbVMGyf2zEu2fYzLX/UadW3QhlKETvqro=; b=FlYzRR4iAFvrA4LOsic7VAddFnqug11Cw98z6HFjaNEPhAXrFnSGEIUSczfR+zmTS451EiCuQvYjXN1aTZMvBqPATaSd+Bi46TV3bL0HPyNewrJXZGigCI1feAV2GNv+Q3KIQzKeOjht0dokeEMelUbTD3wu/i6+y+C2lUWEDgR2TWYhPVeoiAhNNucOIkF1Krp6gHD3PtrJ3utD9SNMxwgPL1wX9CdpODQn7+r7i9cI5oR6soOPOCQ+o5//YvKAsFCbPbUYOeo8Z9MQUlF67NTfdtPwg2PYZyn6tAbA/PqdTCz4L2JQCzwIC1KenJ/ixkErPz7SrHyTLzd6zQQ4OA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6QuCbMHib6zbVMGyf2zEu2fYzLX/UadW3QhlKETvqro=; b=CELEFCGSaH42frk3O1qLdSmpR46U42A3BR1mWs6Bev5Mje8Xepz0FkwpUNNWoVLdb/xJhj5/NN7MSyEcDPiMpl21xYKqC9IkRJVtGoiM7IObHGK8Nrl9t1FaYGy5572ZMS2/PVdnA0Q3+hvBD4Owhzm0iaDJXIRmQNHUIUNWyE/mESZE2ED1QOAatPlsVhu7ttP6uneOBO8fALabz//dZUIUuGRY03yXvbEy4QG+WizMQPRvL7tJsFnlrXTeewSeay4+pAbAIcxiWyR0lXKgF6ProoHhBjSbHRzEOX+/4PspVHlBeFAwXKRXlhAP59wyejfYHHH6biGDDGbLYRkZ7Q== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by CH2PR12MB4326.namprd12.prod.outlook.com (2603:10b6:610:af::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.14; Tue, 18 Feb 2025 23:54:50 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8445.017; Tue, 18 Feb 2025 23:54:50 +0000 From: Zi Yan To: Matthew Wilcox , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: Andrew Morton , Hugh Dickins , Baolin Wang , Kairui Song , Miaohe Lin , linux-kernel@vger.kernel.org, Zi Yan Subject: [PATCH v2 1/2] mm/filemap: use xas_try_split() in __filemap_add_folio() Date: Tue, 18 Feb 2025 18:54:43 -0500 Message-ID: <20250218235444.1543173-2-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250218235444.1543173-1-ziy@nvidia.com> References: <20250218235444.1543173-1-ziy@nvidia.com> X-ClientProxiedBy: MN2PR12CA0004.namprd12.prod.outlook.com (2603:10b6:208:a8::17) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|CH2PR12MB4326:EE_ X-MS-Office365-Filtering-Correlation-Id: 3afa03cb-0c92-493a-9f75-08dd5077a1f5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|366016|7053199007; X-Microsoft-Antispam-Message-Info: Vlx7hFIuVZHAmaqDDHO0s3WxK7QBCF3DMmN+ZsxvSb4ZwBEVVUVJJ8h31weRWvADxvPTwl/54Exk+jVQW1whevX1ebaYJoaF3Df+vi9YCNRX3SJIaSYUitSOeDnkkioWjUKYZrgpbcKlDdLX4HllBdiB+3OvfQFz+TPJm4pyPDXLfvlH/ZBbZfdeoFS3dYhjaNzwL0d8XgDRfuUk1SIj/M/NdYA2/FkjXzEL6jCP5qdGzMCp1vVu4ELLYeuGTjXl913CSpy2B9dc0NW6cvdVW3p+35mcBEjpY/Ra4AFghM9fqhEMeikIcm4GXFZSgNCUR27Lsw+kEo4D/hdnfvoO6ERVZ6ATuw92S/H9qB7MibKEhGbm+iFZQN0miKW3oyCHk+LQ28FYZsBsk1EAm2LvKvAyBh6lkRz9/O2MCyfQzap5qM37EIdUWQgytyHAvQ1vI5XOCwq4ulCVQ12GS4uGhS4FpHVSE/ta1MDdCHIVBKleTnxpMb8zV3eoaySwEEnlJANillIbpszxm6drwall9L7g+DUOHcNbvmYsvKdbsUkE3NmS4VFwtr3MECpXVCzxkO9b5H1es8LNCoQWrKdM5ZfB3rxbQGE+3CS5+PeadYyxiuYyOua8K8OO8XlEEKh1SmWesMnLiJPYPFKuPTjxtRYduVqe7kT3srUDsNLtgex4ZgphGVbpuAea+AZDElFeGKqw12it1xFbBqg5+j9d1rqf16OxKsjkvt9qifrjroY+jlgnRb3SgIf9Pf8LLlrg0+UGl8U8RmGts55mkCR1m6wwLmy0sV+swqbr5chqnbSExSsV+ugeWih50wgT2ZxSSObi9ltp58o8AO0uDbHGXulCVCn+kkZpl52W4KZu81sOs69Ya1lT4ljpYroDwmq27+mKX0giwj1oRk1m2hx1Qd/EoepefKK5WKRP7FQXg62/Zd3SysL+jpUYEKPSnHFK2yTlCxP/Mnq/zpKBXHkqGTyeCJ6EAeJuZVtotj6i12RQSUs8mnRCMAmx97+8eEnlNk2dSGefrUKFOWe1f18sqBFSozG4sgZZrF0dYSiub5WL98W9wVdYT+cZ0zMp7PFV1O5LDNwy8pq6mOy7ZWe7DssW+EkRH5OCKVdI4IsiD10KPbprAOGO92vlK8PUs8FJXQFFqNF0bX32uslhhIGKEj6SvnUXTlPJiGo3l7swx+f36OenL+B07k6yNIchV8jVR4IaTedCTkiXrYqzWOg8UFF+0EZLy26zBUwJJzEgokdKn+VXKI9mWPR/EBcj/xRhdoxEeIXGoeo3b9EaAG79Y3wXLQAUC8Ldg4FI2G/ldcuzrvU1Do2MIATXZ+H9f48yiYkLYlaEs0mEjJRK6EwwFPGh3IJkB2dFFrcnkiYMU8F8TPaCzSkDDDKa6yTUXYhe X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(366016)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: lfn8KFSqV11jr+GcImt3il76RzJRSXwerw4dbEDbOZ117DbKwv3yGUrXoYGRR5/WlMnU48S5xDuLnnSUKMGXSoweNGnX7xeqAr118EbOXErORmJmHA94XMhDGT2ZJjTPM9VRaYWq+PGQoFsO4CK5Fnl2Fy4gO/54P00Qa0lW+axO7iWa/hL9HGa7sVFCUwSm/yDHmsim+og0hOyXHo0PXmowJdkWOAJ9TxLX+VXl9GFYBZVWsUgrFTg2b0zlB0n8oRgyjl1mA9HTS9F90iCByafoPLm7OWZBOCPufGJsp/2GpeIv9tRrYyKG/THnUsmgQnwot9KI43ox3UTqIHM8F3z9iQC5htma6vGWnNznaToRZOPw8d9CxQT2TPEGnTUMnsGNJTtRKZ9KcZ1e6oISfKamoTZSnv/F781mhoPoe9v57S9VLvQjut9hd09WCtPnSw2eKIAYney1F6ty7+4uamEYlSILH69RSSw20UwTs0TN320XNqsuvdam7omKCMqpS28qmtaxr5XF+tVPH0hBJ+6Wdifa3dPc8c0ow7TNqlDxiaiGcSFaTeGPYDVPsUMi7Z/F6pz4hWeuUAEUfxQ4azvH+R/3UO+22pBKytN6PAcLKlihJxc8rVYv07/snzXCEY8H7o6els1DZ1wH9tvBLIcxF0NrfLD+ArM0msicLfSYNV8aiNTxnVzZekyeHQWAxJ8LNgt1ZGUV5grjOICevZQK8o7f8QOJ1Z4s2uuxPxudhKMkRgRS7KxB2uEdyAWo1DPWOqXp+/MPxzoP4KsF2YnjmkLLw3efRVReBKLXemSljrGnsRHNj/wd7j+kjHsJr/iwlsIonu2EW33bdOUyYsdQ8KDNbUazgT5w8+v3WeRkDS3B3OgSk0jYz8WMQRd2R6XVIKSbP6fZl/vbY/M1tdXYOU8KcKrUuEsLCrYEaqXIyt5nCio+AlhbcgU0tPdEqmDDceOeNxHjJHXrGuRgCjtIxlUwEMJS31Tf84wohHkPNGwu+aVzhyUeILapzZG3BePFCOxsGbZTHVS3D8rrlRHA7H25dos0F3u3gtbQubHQRWr+Wi9wnpa9+xSryNqQn31i2lsDXyK6HCUyg26LOLzEOtZJGvL1QQLmPe2pdCVCXIQr+8b1X6+Xgh12Sz3Izu3vuNSIoq2CaQP2e+kmhllw6Nh+ZiRXRjRi1qmJM7zlRiG0C0yaX0BJW53+cbj5wSZQ19m5jhZxLDUkl5MlrCtheIINW5r6JKfjl+JZ6EVyDMs2zY1YSQG2B5WR8n5VqlV+af3GrkuAt0mR6L39qgvMpvgW3hceTvWPPkFGX5QlYRY8pZILD7d6NLYhHmp0k9SSS/wAsi7atlAhLBouW0x4Ucw7Pb9CSBEO25FGPmwTB0j+1Uo61uU3gTACiRWnquW7YLvA+XSgo1Ag8vjcmoeClK6uMk668tjrlofa5R2kO1cmRE7wf8FYQjQrYbwZnw5y1wWxxCym9rj8d7AUJt4SDqMV5as8mmJXWq8hRhntSG4Y6cT95BmTfqohxPFdVh21J2Rg8AHbPQd9unePlXjJ8m5KSFiQnkurGKOPkhJj2HySAPhSBBPRnI258/lX X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3afa03cb-0c92-493a-9f75-08dd5077a1f5 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Feb 2025 23:54:50.6132 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: G3TY6gyEslgopXXqtiaz5V/b3pnb3GIN2DU0SijRwcIXCOXy1ftCZ75ghZvo7Pc7 X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4326 X-Rspam-User: X-Rspamd-Queue-Id: 46E4D140004 X-Rspamd-Server: rspam12 X-Stat-Signature: hh9hb99m4exo8i9hs7tq79i7pibpraj3 X-HE-Tag: 1739922914-141960 X-HE-Meta: U2FsdGVkX18tTR1BQeqH5TNMX12mrGgRRO0mcSlDvE0GnKxBDhx1RzVzqt6L8F8zWCMJnU5d8GGPbgTgl0AWVZIU/Z0vuUvyitP8bh4iTItCtrQsHFEBhg7rB3xUTTKlfCS28LLnJBXf30R0gB0O7kHUzvaUWhHNN+SBuGCNzeKJspjJrdWCGrxoRgJ8Jnfy4LQDhaPQ/d8bfwLTGM/sRb0Ou/Mh7xmSWDFVSL9UPn0UIpQRB50TLtp4Y9jTabGlS7juPKGevmtPgGOo8NCh8jUf0sSJRf4SlkQXtpeRPkg5rxWKE6QeOAFx4hGKlk1F1A9jm5j6xYbtWnD0k4Nw3NlTf2z+0MgV+jlsFptbD6ML+n41i3Qs1RLaIYVc/3gmBQZ8XJfmnK64lmZZ7bTCBZKUauWYBB+5muy+wV7LvUXqPdp2O0uoI0Ccwj9MNKRwW/32nmEVBxHBHlOoQuTD9lZab9CRVgkKXafC29814h518hfr2Xumgf7vZ8YHbwrO6iHU3K24VgrFV8g9lgi1++rIljq2Rr9UcR6m1UvGm6220YzYjtgc0VhvwMAdEwV++2WNvmdM33SJAdevtBGyj1XHXYAZI8B4Ec5Xv/lu8YsxA84D8Rs9+0mxn6Qd9BvG+ApbgfbNB1ZDzJv6ibbvq9gNjcaQ5OaByXUdg3ZELp6TgJ/h4WgVJRRlZVm4Ci6LJFwuFqajGyFwty1S8OUEi+2FhlbKJ0+XjB+KcABhPlTakmHtlRtICRnD78+Dl10GQaec7WEztfkduHAb5Yppbl4LXW/SbGYR/Kk91gYnCDj1IejNcZ4Yzp1m2nNlJJu/fMFwcWStTTYR5QH1/zQCD+iUmM8w9DLBA1UrEfmNuYXJguGTOOFXpuzSRn21quPSmgo7PKsUljB9JUVC/shbHzED1Dga0PCfrYzZNKATOClTWt5IO+InarDQRcYlVNSBUI7oNOWiaUE+Rlxaz9p ZJ2U1b15 z4CpxodZHETk249EbyVevtrbNWHnMs86mqo25jm0o7mrderyJXHv2rD4YJW6iZPCVu1lpTlxC/gJtM0/qn83al1IMwgLocPrwYBvgq6Fi94MK07mshqCTKCTK0kE/bCayVNnzohqZ/FyriBmv3d1AjFanj50CwPQAehnaCtRUYULAFMRDrjYZkCNDN77g5PV9EzJ6OBHPojByE6sXU1WkcmzDM6XXrsRMSyn9SEQOvtbCAKuInjWh9iIrSvu1Ie0ajPL3JutauoG7x9QSJzTGUpV5jfbIg1hU83V9KYV5k/YSuQ3PG8u95ct95PWXu1f0vy5KnUQHBHor5p6jYDNXwK16UxXA8JCDVpyz2abl31Q62K5H7t2T2rJSaGgWBRQkOYOuER6C/5afY10Cn6o2r+IfHAsgup8xRLtNeWuclyRnekkOmtmFiic30FYnFfy1YgKvaxHntyL1K2TzqXTl5Yio022VquKw6yqdb17SefBVdoUgKuc2TDWmXzIqVQjVWrrJBHAU00Hu9j1S2dhIBISfv/XpoDfK93YL X-Bogosity: Ham, tests=bogofilter, spamicity=0.000002, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: During __filemap_add_folio(), a shadow entry is covering n slots and a folio covers m slots with m < n is to be added. Instead of splitting all n slots, only the m slots covered by the folio need to be split and the remaining n-m shadow entries can be retained with orders ranging from m to n-1. This method only requires (n/XA_CHUNK_SHIFT) - (m/XA_CHUNK_SHIFT) new xa_nodes instead of (n % XA_CHUNK_SHIFT) * ((n/XA_CHUNK_SHIFT) - (m/XA_CHUNK_SHIFT)) new xa_nodes, compared to the original xas_split_alloc() + xas_split() one. For example, to insert an order-0 folio when an order-9 shadow entry is present (assuming XA_CHUNK_SHIFT is 6), 1 xa_node is needed instead of 8. xas_try_split_min_order() is introduced to reduce the number of calls to xas_try_split() during split. Signed-off-by: Zi Yan Cc: Baolin Wang Cc: Hugh Dickens Cc: Kairui Song Cc: Miaohe Lin Cc: Mattew Wilcox --- include/linux/xarray.h | 7 +++++++ lib/xarray.c | 25 +++++++++++++++++++++++ mm/filemap.c | 46 +++++++++++++++++------------------------- 3 files changed, 51 insertions(+), 27 deletions(-) diff --git a/include/linux/xarray.h b/include/linux/xarray.h index 9eb8c7425090..6ef3d682b189 100644 --- a/include/linux/xarray.h +++ b/include/linux/xarray.h @@ -1557,6 +1557,7 @@ void xas_split(struct xa_state *, void *entry, unsigned int order); void xas_split_alloc(struct xa_state *, void *entry, unsigned int order, gfp_t); void xas_try_split(struct xa_state *xas, void *entry, unsigned int order, gfp_t gfp); +unsigned int xas_try_split_min_order(unsigned int order); #else static inline int xa_get_order(struct xarray *xa, unsigned long index) { @@ -1583,6 +1584,12 @@ static inline void xas_try_split(struct xa_state *xas, void *entry, unsigned int order, gfp_t gfp) { } + +static inline unsigned int xas_try_split_min_order(unsigned int order) +{ + return 0; +} + #endif /** diff --git a/lib/xarray.c b/lib/xarray.c index b9a63d7fbd58..e8dd80aa15db 100644 --- a/lib/xarray.c +++ b/lib/xarray.c @@ -1133,6 +1133,28 @@ void xas_split(struct xa_state *xas, void *entry, unsigned int order) } EXPORT_SYMBOL_GPL(xas_split); +/** + * xas_try_split_min_order() - Minimal split order xas_try_split() can accept + * @order: Current entry order. + * + * xas_try_split() can split a multi-index entry to smaller than @order - 1 if + * no new xa_node is needed. This function provides the minimal order + * xas_try_split() supports. + * + * Return: the minimal order xas_try_split() supports + * + * Context: Any context. + * + */ +unsigned int xas_try_split_min_order(unsigned int order) +{ + if (order % XA_CHUNK_SHIFT == 0) + return order == 0 ? 0 : order - 1; + + return order - (order % XA_CHUNK_SHIFT); +} +EXPORT_SYMBOL_GPL(xas_try_split_min_order); + /** * xas_try_split() - Try to split a multi-index entry. * @xas: XArray operation state. @@ -1145,6 +1167,9 @@ EXPORT_SYMBOL_GPL(xas_split); * be allocated, the function will use @gfp to get one. If more xa_node are * needed, the function gives EINVAL error. * + * NOTE: use xas_try_split_min_order() to get next split order instead of + * @order - 1 if you want to minmize xas_try_split() calls. + * * Context: Any context. The caller should hold the xa_lock. */ void xas_try_split(struct xa_state *xas, void *entry, unsigned int order, diff --git a/mm/filemap.c b/mm/filemap.c index 2b860b59a521..c6650de837d0 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -857,11 +857,10 @@ EXPORT_SYMBOL_GPL(replace_page_cache_folio); noinline int __filemap_add_folio(struct address_space *mapping, struct folio *folio, pgoff_t index, gfp_t gfp, void **shadowp) { - XA_STATE(xas, &mapping->i_pages, index); - void *alloced_shadow = NULL; - int alloced_order = 0; + XA_STATE_ORDER(xas, &mapping->i_pages, index, folio_order(folio)); bool huge; long nr; + unsigned int forder = folio_order(folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(folio_test_swapbacked(folio), folio); @@ -870,7 +869,6 @@ noinline int __filemap_add_folio(struct address_space *mapping, mapping_set_update(&xas, mapping); VM_BUG_ON_FOLIO(index & (folio_nr_pages(folio) - 1), folio); - xas_set_order(&xas, index, folio_order(folio)); huge = folio_test_hugetlb(folio); nr = folio_nr_pages(folio); @@ -880,7 +878,7 @@ noinline int __filemap_add_folio(struct address_space *mapping, folio->index = xas.xa_index; for (;;) { - int order = -1, split_order = 0; + int order = -1; void *entry, *old = NULL; xas_lock_irq(&xas); @@ -898,21 +896,26 @@ noinline int __filemap_add_folio(struct address_space *mapping, order = xas_get_order(&xas); } - /* entry may have changed before we re-acquire the lock */ - if (alloced_order && (old != alloced_shadow || order != alloced_order)) { - xas_destroy(&xas); - alloced_order = 0; - } - if (old) { - if (order > 0 && order > folio_order(folio)) { + if (order > 0 && order > forder) { + unsigned int split_order = max(forder, + xas_try_split_min_order(order)); + /* How to handle large swap entries? */ BUG_ON(shmem_mapping(mapping)); - if (!alloced_order) { - split_order = order; - goto unlock; + + while (order > forder) { + xas_set_order(&xas, index, split_order); + xas_try_split(&xas, old, order, + GFP_NOWAIT); + if (xas_error(&xas)) + goto unlock; + order = split_order; + split_order = + max(xas_try_split_min_order( + split_order), + forder); } - xas_split(&xas, old, order); xas_reset(&xas); } if (shadowp) @@ -936,17 +939,6 @@ noinline int __filemap_add_folio(struct address_space *mapping, unlock: xas_unlock_irq(&xas); - /* split needed, alloc here and retry. */ - if (split_order) { - xas_split_alloc(&xas, old, split_order, gfp); - if (xas_error(&xas)) - goto error; - alloced_shadow = old; - alloced_order = split_order; - xas_reset(&xas); - continue; - } - if (!xas_nomem(&xas, gfp)) break; }