From patchwork Wed Feb 26 21:08:53 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13993197 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 91186C021B8 for ; Wed, 26 Feb 2025 21:10:33 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 0B930280009; Wed, 26 Feb 2025 16:10:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 04252280001; Wed, 26 Feb 2025 16:10:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DB02D280009; Wed, 26 Feb 2025 16:10:32 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id B3272280001 for ; Wed, 26 Feb 2025 16:10:32 -0500 (EST) Received: from smtpin09.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 5585B160BAC for ; Wed, 26 Feb 2025 21:10:32 +0000 (UTC) X-FDA: 83163339504.09.FB036E6 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2058.outbound.protection.outlook.com [40.107.92.58]) by imf25.hostedemail.com (Postfix) with ESMTP id 76649A000C for ; Wed, 26 Feb 2025 21:10:29 +0000 (UTC) Authentication-Results: imf25.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=iSamyKXN; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf25.hostedemail.com: domain of ziy@nvidia.com designates 40.107.92.58 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740604229; a=rsa-sha256; cv=pass; b=vWcDZv9/+u9S9JSp6YD0EPMDA03EFcp9CTcrgSfee4la9myB7q974EvQi69xS57lxPJtlk DDxNV0aEJweBlutTLGGB5JUjUaDdzZdpAM9RVlw0evIrEIdLkF1zttYYo9416LqWMw/hS7 41jVBFby1vo7/3fDufLUqtWRs7wd6C8= ARC-Authentication-Results: i=2; imf25.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=iSamyKXN; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf25.hostedemail.com: domain of ziy@nvidia.com designates 40.107.92.58 as permitted sender) smtp.mailfrom=ziy@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740604229; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=2QqGevh41/tC3V+T2ChKF52ncLDVut8qgSqk7CzqYxg=; b=UwNwRgPwzXkN1bmG9Ccd4WspsR3wlGFF8woMtGeNUBlwwA6nOuRrytzQaLUfQTBrDIcjTt YbJ6wUDPv6XpbFSX3Y710qMTELhjFQQzsAevLlSfgSenUaSLGK6UXp/AZEtqD1lbkc51aG gS7C8Q9toVTcjLb/mQJxku6QVIpJKt0= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=RSs8FkFzh9wqoJ2bz6eT9sd37vEFPP8JEASxTvdVW7Iu/ydpd5OP0j9jW4sTRPiCbdnHn6NT9uShHcx/ebXX0JWXBkTtpiu63Hy3IRVu65QJHis+ETf3I+gbJJLX7WlADkcAwkbYSS60OYXjWKrRz/6fSaMWGha/IyWQVpLu3L/QAg5LOubHE4db+IwAip2g2G7AJ/ziRgC7wAvINkn7ZJV6Vm9ILwvad61y1wPc7ELhDVtShX66nzMjufKpNETqmu4cg5nVcmJ49l6jCryA5byIZ9ZAKHKl3Y3S27NESJyM4wJRkWXecJBzuyeQuZ0UfM1ELuISEziXzF31jK1GMw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2QqGevh41/tC3V+T2ChKF52ncLDVut8qgSqk7CzqYxg=; b=G6LlY+za9Cr8Mc999SHTbddlJGnNVQWOA36CDxmjSXBgcDk3ooBGQzMCD7dm+4NIkJlLhmwbar8pvy9BNnNw0U5PpflYouXI4zmuhtFQdZ1D61R0tQuKHRpuTKeofOVB73C2ejHvCcob0noM1IJRSKSY8IVs+fLpMlIUoIY9N0OjT2NeluiqLIC1UL/0wrQ5x7N40FvVCGm0nAT+WczXFEEaDOkllw3bSTIJUhmlXEmzzG7LkGx4Tx+ivYe4RyNzA59jrQ4dhi5mTc+fAhMURDgBKy5L5mG43XuOA0q6b+9iual8xAg3RS1ZnzSojx19HcRdryivwyYAvYp02n/Fxg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2QqGevh41/tC3V+T2ChKF52ncLDVut8qgSqk7CzqYxg=; b=iSamyKXNsbx10zig6Z1AYEJuqsls/aCp9b1riCo5TXqTrPp13rulSdNb3DeIKEwRQ1MJzq3fDps9zLIkHIxLR65Eg2/BDld6/qjEwTkCq1WPmNmo+Nw6QakrYiFvaOho9yxNx4vgZWq4JUbHmedmCuXqiugbA8oHo3nPv6pd9w85DLPIIvzmk+JNoPd9InP0zL6SJwtN/LcMPNn2eLRrqdAYFrbgigX+IqJwx8KorxseyyJ8wTtmorRnZ5oglSnwaDHPv0PFoah4y0C6bjjdgFvYenvOFiC8L9+sBUnU7rVA73Pr2MuSQ/EVwduMHrRNXCKNnqS9w8K7Eq9Sg10FHw== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by SA1PR12MB7221.namprd12.prod.outlook.com (2603:10b6:806:2bd::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.20; Wed, 26 Feb 2025 21:08:59 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8466.016; Wed, 26 Feb 2025 21:08:59 +0000 From: Zi Yan To: Baolin Wang , Matthew Wilcox , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: Andrew Morton , Hugh Dickins , Kairui Song , Miaohe Lin , linux-kernel@vger.kernel.org, Zi Yan , David Hildenbrand , John Hubbard , Kefeng Wang , "Kirill A. Shuemov" , Ryan Roberts , Yang Shi , Yu Zhao Subject: [PATCH v3 1/2] mm/filemap: use xas_try_split() in __filemap_add_folio() Date: Wed, 26 Feb 2025 16:08:53 -0500 Message-ID: <20250226210854.2045816-2-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250226210854.2045816-1-ziy@nvidia.com> References: <20250226210854.2045816-1-ziy@nvidia.com> X-ClientProxiedBy: MN2PR14CA0007.namprd14.prod.outlook.com (2603:10b6:208:23e::12) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|SA1PR12MB7221:EE_ X-MS-Office365-Filtering-Correlation-Id: 97cebf45-10c6-44ee-1896-08dd56a9c9b8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024|7053199007; X-Microsoft-Antispam-Message-Info: 2Xp1VGrxgI/mBcNCel6SzLdW/odQZvizBGyEr0O7+j/B2oefxtEYhwbm7ToQO2DdEPw0IGwedcvxzxU1HlRiyCsHgEWTWcd8l4UddvF5K9r7c8ZPb9AJh0sywssaKCm/FXdLLfQ5oC0OFxwFY0pq6NPgKGbPAk4Zq0vZ/ocxWPc7GJ929bFPFK+eG2Wlr5X1v2doxSwCj+zkHtxcptBqCFSTre/T7S2V/DTQitPBPILW3InNNbZjHJAdeTlY2O3doIPm0IqVSK69ppFuVjHu93Bus4/npA9i57oLUKW5j2O73xTvMc4mM4u1RNKj4WdFuYOX7m2Qt4AjzBZ9T3FgajofNyCZCoN8FIGa2WmLfY67zLNWknyMLv8YHNf8CBrYbdo9KTraO7mYujoKdlghMRFXBEQs3A//HE1Y1/Vtdbekg1SSOfsDpmAc93l9f5URFnUJm2BSMGzjWwTP6SCVumWlv59tlHaVmW7h+AtFSRK2xmG3uGRe/McZnPFzyUAvRZS4qBsgIfl1fVLZLyIQwkMRWTd9wxyH+FhwzxfWc7YiGG02mWnIItlMHAvqYeFURP0kPXSIrD00Q2Ub7KJ5cKgamu+X4W0pngrViVPXVe+dAKmFQPbfBYasR+SAEyf+ow0FEvg1FRw5SR7j0M2eCamLmhFbP57ZpIc3J59zbrecK8OVAauq9CWOFvVb7wAB7QeW9G949j/i20LwEovjXp22DBaLHkVD5ZunWWZi6CgKl3aQ9GkCESDC6FWNJddy/VlAc5EojIhEAKMXavrEG2INWq1ueBYYG1K/K1cZca2I/brp5DfIrEwRkHaPUmsYNnIbxlzyO9QTJUe9INwEfOlhWWxf8TAOGN5r5mb0CYnL4qI0jP8KwUh0k1n5dSJp5dvhx+G+mIjHVNXV5Q2mmSn0pyI2CKYI21qoxh+eR6n/r2f2W83ejdt21zQK7RBBh14jq1fDWxBC6QIUllGv74KBIUA51kycrdOtMrvcn5pVkr68HH+yqdbw6bwe32akzG3Pe7VfrikVgGj57E6/wJ1iqsCPavLE9o9LQTCu16GzVRzM6ilW6dBXuGEZAXdNQsUukvH9l74OC1uZuHy6/ZukxOOQ98C4BLRvhT7qOUkWzjolUrNKSAdsfUUZpZfPvepQADfeHAYQO5P5FzZkMgA0zltkCHwlRfzSKLRGMo5okMcjXIa3YUkmPJcdeBNBBSI5/2hBh9PcBiEQE2o0zwoovaUtLIQqiEIcT+H0BNrv4RkbNm2osy7luaXul39ij+2f/AWblbY/mSBoSaKO++CIEb2YbC+xEvwg5x58f4siByhR1mdZ9J2IduURjtNCwpRB8SE6Qk3TYzccSpwauy76v14IVkAhyaHKwKw3twICwjnPwzc/4lK2HkusU3Dq X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QIC+fHEY65P34XFGRPq+XpFgKmkoUOusxc4Ir879CjMH0s48lB6Sg67aVx2o1rKnExKIDC9OaSF+jnCMrscZuMin1k/pU1oa0VQeNK18vR5vfOFuUllY7jLoLBJudUJvJ4U1aF4PmIdxmLB8Z9fYnIk1hdYGUyETo2niVXsvWo4IWRFa6qObTsxQMbahewIvqi7UNE8G0KSEYb7dUGgUwzp305eUkCKc6hGwMTv3zVekT2roGGcv/rCsFr6LLVSfhH72Ig4i7sKzUwCdCqJ2a8qGVw2DDWrUxKSw5/mk/riD7nS16tB/iC+cwr35IiWdUjhQue9N0dkXANWVR7k6248A2uxbcMkh0nCpqterr+SD8JSxAzYQHEBLy/txbp+Q4ljTEjCZgif0iJvjMwFMKMRIYUVgR46IPs0mmDnl+8BtdCbBhBGNjUQSR6MXnHC0TPAtI7zF751pSVbddJLtFM9OALqL+jNsbEbE1iOJWETi4JmS6J+CfZgtHmqiswsx7x26aaEO13xLtknpBkT4voMo+Qb8m2cpZi9zYmwia1EytWu0+MuwOOWDHQfZR5msk6ogryjgw3QWn43/zIrtHb0H0AIhJFKuzK7ekwTtMWiWezmMtV5qNvKhb9TTf2x9/j6ehbUPITHIe+MxEoy1nueQ4kS887P1FB32P0BtdEBzS4O9ZWOpEwkyKgOw5LaHjkFwN7VCqhdq65GtbOXR9raP7OrLl1vYJWpNyuR8T8brHfcvFjR3FGVw9HdP7+nJAythgzxxe/yvM/XhXp642w0KKiqJF9h9WF3hxUhROK2ZnkXyZ1xHi7+wmePqi8LffAh9wANEfTiT4d4TvTJbbUjoxYlvvGAkVL7zgKiQeU97fqxuAVoMk5kjdGLPEBt14p0LuoMzYH9YqZpfC3Ef+BtHlvDm/LXGJUNwX/VZ02YlLkNyVN8mBAmogricTd3kdt+6glmuR8roTFjKaxN4lUi9L4rnbIrf0Hvw3v+kxcO+RpBYZazn0s1c9O4ZrALqeWHW195AOUo7zsyQpN5rLTJDSf44+7JZMu2lysOWZl2KtPObrgNQ0ZPHnmVjsH6x6Y+BTjkYZRw7r8ysy6E7C/uf8xAKN+ClStdhtLpCBxDO391KqOwvW/Znd5GDbSLTYVBKTAB5+LjFzlyEdMBhDvPxRG6R0S6hM/PMzToBS1hqURE+5ltrWYa1s4tUnzHr7EnFxdJd95e0K6sZ8OTfZ7dMu7O4DfIBshMGWL0RXjxdf0ngXg4HOknkPEXcv6tESpon4PI63SuxmXHAjRuT3PHL2UhMQivnn3OCxaBB7Dqkbgi84PRUQnLoX86G1TYRF+4ybOTz0+BHJ0JvwI7mF4ne1XT8uvZrefqtP07tBWuoA3wHve7usHRf11FgB5CuIiXUypSR3HA9LiebRAXZGZG0HoSMdeie8YSX88e3EZnWCco3hbIY+2JE5dk3eXUdjTxSRFEc7yQIXW3cgyJ5uuOvQRO60qpMq3xgW6NGgFWGW+jFMeMjdSn0Rcpc2iq80OBtCqMGM3JMAPuuMAQ6ns349JXlAe2P6hBHmyXIOj7YIi294FIGZsnnWnoSBOfn X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 97cebf45-10c6-44ee-1896-08dd56a9c9b8 X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2025 21:08:59.1894 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RSB3yRZN0E8MJZjC/wqlnI7b+V4Yjdys4o1QK7H87J4Vx/1xGw7wMzkg/RSM6NfW X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7221 X-Rspamd-Server: rspam02 X-Rspamd-Queue-Id: 76649A000C X-Stat-Signature: cfrkuissnb5jcpdwwj19u6umhmgowg33 X-Rspam-User: X-HE-Tag: 1740604229-667869 X-HE-Meta: U2FsdGVkX1/+/dzmnyp4cRzVe/8zFHYVD6JFiOj26L7oM/fBBpkglt68ORLESA10ImnaqhzydDI6t0KESUq/DMZcpH6vA/A6P0Ywa7upsn3j/Anu65we+460mAOhtvIFgT9/tGeGJkoq0LAPAsGqXZAR9fIyAnk8qXWSJiWCzGIHZX5M9MjEIwy3zlZ6ypd5fq4xYfHwu/Cs4ekejhQlPXzp1/Kk2oit4RRgXxaDitRzPVOhOtZ6v82GoiBF5sTGIxo6pYbaxLyomUetcOFm7nS+u8jKj7U0sUavsYTGzACVRc2cdT/5KdedePMgbJ2j3AyXs4lDJdU/tJIahqyYcYgaUXm3ghS2ij9ZizDvwfYNwd/pGD4YNp/ELKJkHwM3V+uPy7Fm/oU3QovCEyI5wQaLcGuJQ4DRZ96DcW93ng9HtbggrenpyS8l+3xN8g/w7wLjyBq9hA0TxVCXbOs/AMwp7Dp86b+8nKi/oRJKEhHbochq/FEmDg2XmibQxMrPUAPphoEPmLi4DRZmDxEaoK5cf6Eu0niMxb4z7NwKOJ5ZTCivVzG3ltAXMFXAWUlgORLblN+lGlSobyXzsyY8PQP+QMLyLIE3RZvV9k7cmwPTqNLI29vk0ae4V+TEWs2X1fiq1pFKhLZGDup97Mg1R8E1ODKNwFSb8WE9hZ5ig3Uf3tg+N6jxvf5UMMLWbinyimFuTPVvncP6I9Z5unCRIRWluvLVDG3IdD+14gmmyr2aW3GjngqakpZ6BhWwYzXM1Kda8l026Rvbnu32nqCOXo9OI20QltSUlrGSwnSaWwnV85a2DX84Qiq3uZwjFlCFlkfh+YbmnU6iVKr5pheccR8F4CkotrYRltow/Tg9JpyzdD71Xa1RrBj+b2isrzzTdRsM5PZaW0JoXXI32nMWdXRAXScnQ/7Fwix34TdCxYtCjIP9HHE/6f7ls7K0uWQSiOIz+WVYrX1k8863fBh X3Yly9Dv wXmUxQXdUTwKY4t1w//k+topybA17poX5V79fvZd37RAHR5Cxycw6T1KINvr5nVS2eSxC60Co5b0bUa6ThZFrR2l5OB9jtjFehbG7ZY8odpBAKooXmh/eN9GL99OgvgdWn6zKJlSSqjFkj47YUWgPhxB8DYLIO5VCrT6Q4qCro7Ju6BoTVaKmScTI/rs2HIPx6JALTOHB194thPCrN6F/DsR3lTe4EC3KGjnpaSIEQ41xE/rnyzxoqk4vsHKV5/YGM6sVX1le7/Mhuip/g40uYQRmTb2jKxqZKijCAVuJQX/Y1tMIOhOJrYlgG0EmeRgL+ZS75ok5773HDMr+LuIe89v2pNeSwWmW8Ez/dN9oRzJKzbv/nSbxHZjK/BejoX3OM7RSIZnIYoLAquYctx/P98ogLEgTkLdQCx9OsBKcy4maM7bYSL98Gltj4eTjyaID7snD2RW6SGV5u86gE4Cuuve6MQzJFbytCfu7ogIZMnorPCLV5u7Qyy3x4fxQdn/DDaytxnDwxEPN08N9Aap0i5vUWrJDssM+81QJqJDvIY3mkm89l/S81xX2n1Lm0JuQ6xaeO/YHJFyG/oZwkLRYmcp6g9lM2H3Gk2h244kRUuC7sGMJQCF99/MrRquoYAYyBx3z5ZpUDR8fqWpHiiXbg2rM6ide7aNjgf8MIPTq2GdKgr+pMkPJjqhVXH1ZI4zCoTTa4+KgkVk8scE= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: During __filemap_add_folio(), a shadow entry is covering n slots and a folio covers m slots with m < n is to be added. Instead of splitting all n slots, only the m slots covered by the folio need to be split and the remaining n-m shadow entries can be retained with orders ranging from m to n-1. This method only requires (n/XA_CHUNK_SHIFT) - (m/XA_CHUNK_SHIFT) new xa_nodes instead of (n % XA_CHUNK_SHIFT) * ((n/XA_CHUNK_SHIFT) - (m/XA_CHUNK_SHIFT)) new xa_nodes, compared to the original xas_split_alloc() + xas_split() one. For example, to insert an order-0 folio when an order-9 shadow entry is present (assuming XA_CHUNK_SHIFT is 6), 1 xa_node is needed instead of 8. xas_try_split_min_order() is introduced to reduce the number of calls to xas_try_split() during split. Signed-off-by: Zi Yan Cc: Baolin Wang Cc: Hugh Dickins Cc: Kairui Song Cc: Miaohe Lin Cc: Mattew Wilcox Cc: David Hildenbrand Cc: John Hubbard Cc: Kefeng Wang Cc: Kirill A. Shuemov Cc: Ryan Roberts Cc: Yang Shi Cc: Yu Zhao --- include/linux/xarray.h | 7 +++++++ lib/xarray.c | 25 +++++++++++++++++++++++ mm/filemap.c | 45 +++++++++++++++++------------------------- 3 files changed, 50 insertions(+), 27 deletions(-) diff --git a/include/linux/xarray.h b/include/linux/xarray.h index 4010195201c9..78eede109b1a 100644 --- a/include/linux/xarray.h +++ b/include/linux/xarray.h @@ -1556,6 +1556,7 @@ int xas_get_order(struct xa_state *xas); void xas_split(struct xa_state *, void *entry, unsigned int order); void xas_split_alloc(struct xa_state *, void *entry, unsigned int order, gfp_t); void xas_try_split(struct xa_state *xas, void *entry, unsigned int order); +unsigned int xas_try_split_min_order(unsigned int order); #else static inline int xa_get_order(struct xarray *xa, unsigned long index) { @@ -1582,6 +1583,12 @@ static inline void xas_try_split(struct xa_state *xas, void *entry, unsigned int order) { } + +static inline unsigned int xas_try_split_min_order(unsigned int order) +{ + return 0; +} + #endif /** diff --git a/lib/xarray.c b/lib/xarray.c index bc197c96d171..8067182d3e43 100644 --- a/lib/xarray.c +++ b/lib/xarray.c @@ -1133,6 +1133,28 @@ void xas_split(struct xa_state *xas, void *entry, unsigned int order) } EXPORT_SYMBOL_GPL(xas_split); +/** + * xas_try_split_min_order() - Minimal split order xas_try_split() can accept + * @order: Current entry order. + * + * xas_try_split() can split a multi-index entry to smaller than @order - 1 if + * no new xa_node is needed. This function provides the minimal order + * xas_try_split() supports. + * + * Return: the minimal order xas_try_split() supports + * + * Context: Any context. + * + */ +unsigned int xas_try_split_min_order(unsigned int order) +{ + if (order % XA_CHUNK_SHIFT == 0) + return order == 0 ? 0 : order - 1; + + return order - (order % XA_CHUNK_SHIFT); +} +EXPORT_SYMBOL_GPL(xas_try_split_min_order); + /** * xas_try_split() - Try to split a multi-index entry. * @xas: XArray operation state. @@ -1144,6 +1166,9 @@ EXPORT_SYMBOL_GPL(xas_split); * needed, the function will use GFP_NOWAIT to get one if xas->xa_alloc is * NULL. If more new xa_node are needed, the function gives EINVAL error. * + * NOTE: use xas_try_split_min_order() to get next split order instead of + * @order - 1 if you want to minmize xas_try_split() calls. + * * Context: Any context. The caller should hold the xa_lock. */ void xas_try_split(struct xa_state *xas, void *entry, unsigned int order) diff --git a/mm/filemap.c b/mm/filemap.c index 2b860b59a521..cfb49ed659a1 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -857,11 +857,10 @@ EXPORT_SYMBOL_GPL(replace_page_cache_folio); noinline int __filemap_add_folio(struct address_space *mapping, struct folio *folio, pgoff_t index, gfp_t gfp, void **shadowp) { - XA_STATE(xas, &mapping->i_pages, index); - void *alloced_shadow = NULL; - int alloced_order = 0; + XA_STATE_ORDER(xas, &mapping->i_pages, index, folio_order(folio)); bool huge; long nr; + unsigned int forder = folio_order(folio); VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio); VM_BUG_ON_FOLIO(folio_test_swapbacked(folio), folio); @@ -870,7 +869,6 @@ noinline int __filemap_add_folio(struct address_space *mapping, mapping_set_update(&xas, mapping); VM_BUG_ON_FOLIO(index & (folio_nr_pages(folio) - 1), folio); - xas_set_order(&xas, index, folio_order(folio)); huge = folio_test_hugetlb(folio); nr = folio_nr_pages(folio); @@ -880,7 +878,7 @@ noinline int __filemap_add_folio(struct address_space *mapping, folio->index = xas.xa_index; for (;;) { - int order = -1, split_order = 0; + int order = -1; void *entry, *old = NULL; xas_lock_irq(&xas); @@ -898,21 +896,25 @@ noinline int __filemap_add_folio(struct address_space *mapping, order = xas_get_order(&xas); } - /* entry may have changed before we re-acquire the lock */ - if (alloced_order && (old != alloced_shadow || order != alloced_order)) { - xas_destroy(&xas); - alloced_order = 0; - } - if (old) { - if (order > 0 && order > folio_order(folio)) { + if (order > 0 && order > forder) { + unsigned int split_order = max(forder, + xas_try_split_min_order(order)); + /* How to handle large swap entries? */ BUG_ON(shmem_mapping(mapping)); - if (!alloced_order) { - split_order = order; - goto unlock; + + while (order > forder) { + xas_set_order(&xas, index, split_order); + xas_try_split(&xas, old, order); + if (xas_error(&xas)) + goto unlock; + order = split_order; + split_order = + max(xas_try_split_min_order( + split_order), + forder); } - xas_split(&xas, old, order); xas_reset(&xas); } if (shadowp) @@ -936,17 +938,6 @@ noinline int __filemap_add_folio(struct address_space *mapping, unlock: xas_unlock_irq(&xas); - /* split needed, alloc here and retry. */ - if (split_order) { - xas_split_alloc(&xas, old, split_order, gfp); - if (xas_error(&xas)) - goto error; - alloced_shadow = old; - alloced_order = split_order; - xas_reset(&xas); - continue; - } - if (!xas_nomem(&xas, gfp)) break; } From patchwork Wed Feb 26 21:08:54 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Zi Yan X-Patchwork-Id: 13993196 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id D0B00C021B8 for ; Wed, 26 Feb 2025 21:09:19 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 747CA280006; Wed, 26 Feb 2025 16:09:19 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 6D0F1280001; Wed, 26 Feb 2025 16:09:19 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 4FCC3280006; Wed, 26 Feb 2025 16:09:19 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 2EDFB280001 for ; Wed, 26 Feb 2025 16:09:19 -0500 (EST) Received: from smtpin05.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id E54964FE48 for ; Wed, 26 Feb 2025 21:09:18 +0000 (UTC) X-FDA: 83163336396.05.3E302A1 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2065.outbound.protection.outlook.com [40.107.94.65]) by imf17.hostedemail.com (Postfix) with ESMTP id 11C9440013 for ; Wed, 26 Feb 2025 21:09:15 +0000 (UTC) Authentication-Results: imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=L5yaJhbc; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf17.hostedemail.com: domain of ziy@nvidia.com designates 40.107.94.65 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740604156; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=vZTe+vj4+GDLRrlRyis+VS8JV73i92eh02I9nqzdSr4=; b=Pb5OJIrdKPzVRmLSOiv6EVrolvsZ09Ze2I6m7a+zehUunYeiXlSoD439YwJCpNDH/XJD0a t6RNzfWlIeMeKN8yiX/+OCwczuLZQK/CaSLobj2hIfgqiNYd6gv2lBMYAZQUPuxQOhGnxK UJBpJLqx7rmkN1aaNKySiC2Epkk/by0= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740604156; a=rsa-sha256; cv=pass; b=IxzJTKlUF5/4SPZXDU+OiUBC1OWEDQFi6Gb2xZYwUb0PsmCbsxPWwaIpLLuWtQX6Q/q8Y1 z5CEeGZXkGBq92I47siwS8BmnPNdGChEZqDtgK15/dba7nus2sniVaL3l4+bC84fDM1WsS Wl4bl1YGDdM+A3zql7LS03bJUjGn69c= ARC-Authentication-Results: i=2; imf17.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=L5yaJhbc; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf17.hostedemail.com: domain of ziy@nvidia.com designates 40.107.94.65 as permitted sender) smtp.mailfrom=ziy@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=cxAwyj1RBWWzcPOl8CnvEg1eD7iiDiUY7Vx91dv82VwtQxVn9tsKKD3i2bs4gB+7fx2RuCKm2xCiHSL/8J7wrbEVvpA4dvQ1KHPwy7pxGoxZXAaz3iBHJTsl9Xs6Ufnt9XuB9yt861hxwx65uM6g3vkf2QYfWh19JjUu0vhBRropQiJo4WElT9AUYYbJqt9QEV2xYJxQJIX6ql104s8vG4laUSXLHqXWa3S3k6HjiLZf7RBaypw0Nh+fKz5tNoPIcY/3Zai6HmRkq/xWUpgApbdbzqA5xQjW57Xnr/PatxBg+oDyHBOTEhF5rd2bS4t/3dcHdo4w3FRZDAbrmaTOlw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=vZTe+vj4+GDLRrlRyis+VS8JV73i92eh02I9nqzdSr4=; b=Pga9t47UW85pw0BMlHTD82aZG2ZrByohxV5in4YEwCmjXz+N73w8RDj/1uY7WdVsa6qX3chqQq4j1adzHS5FYKamtQtY0rrkYfhnr/2xXXaNmp4o0ICXY6rbd8yLTWSwI72FLz4zsVecKYbJ5qKBEESHqEFXPAA1egnaG/FiEUHcbViB1BuCxsri+FUSPOEnjywlEWoIZq6cajBFaVsepQ2EknO4lguT8u/hnxKpKodK5oI6PvJnX2YWzV7y+P3oRkfjY0ZDGZH2mFGRZMKLjebVmE84qZiAH8R3k/u4KTxj8lXlIDBgfOfz+uMcOs7KEyz51Vd7xmz7PBAK3zIN/A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=vZTe+vj4+GDLRrlRyis+VS8JV73i92eh02I9nqzdSr4=; b=L5yaJhbcCk8MA1/Ak1saN1QAhD/+cagKIJNMoXP+tYHIdhweda2CsX/q4BSfwmwsHczgUsR5/MKP1yIBQs7PV+95xNvD3+sM7UfQBmOaDH9o3RgeJCocBTqsUPqt+YV8BMwGpdKCF/jzwn947CjBQsDV0Wf2Mcrw15wRbllDF0a91p3+TKiMTNQh5wOmTe0NTUwkq8+5neHI8eT91d6EvYjosetzXt0pIYXakdfCOuqMUo8IZmPBoo5so5EY56dRcz50VgvS8zammVaL9SHvTIqkjlnk1zFXq6GvPakQGuSWo6ofpGZ5HCH6Ai/ma8YVr+qQHn/PP3bDLdu8gxStQg== Received: from DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) by SA1PR12MB7221.namprd12.prod.outlook.com (2603:10b6:806:2bd::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8466.20; Wed, 26 Feb 2025 21:09:01 +0000 Received: from DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a]) by DS7PR12MB9473.namprd12.prod.outlook.com ([fe80::5189:ecec:d84a:133a%5]) with mapi id 15.20.8466.016; Wed, 26 Feb 2025 21:09:01 +0000 From: Zi Yan To: Baolin Wang , Matthew Wilcox , linux-mm@kvack.org, linux-fsdevel@vger.kernel.org Cc: Andrew Morton , Hugh Dickins , Kairui Song , Miaohe Lin , linux-kernel@vger.kernel.org, Zi Yan , David Hildenbrand , John Hubbard , Kefeng Wang , "Kirill A. Shuemov" , Ryan Roberts , Yang Shi , Yu Zhao Subject: [PATCH v3 2/2] mm/shmem: use xas_try_split() in shmem_split_large_entry() Date: Wed, 26 Feb 2025 16:08:54 -0500 Message-ID: <20250226210854.2045816-3-ziy@nvidia.com> X-Mailer: git-send-email 2.47.2 In-Reply-To: <20250226210854.2045816-1-ziy@nvidia.com> References: <20250226210854.2045816-1-ziy@nvidia.com> X-ClientProxiedBy: MN2PR14CA0016.namprd14.prod.outlook.com (2603:10b6:208:23e::21) To DS7PR12MB9473.namprd12.prod.outlook.com (2603:10b6:8:252::5) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS7PR12MB9473:EE_|SA1PR12MB7221:EE_ X-MS-Office365-Filtering-Correlation-Id: c34acc3d-dcae-4bb5-619a-08dd56a9cb0c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|366016|376014|7416014|1800799024|7053199007; X-Microsoft-Antispam-Message-Info: ydfgA96WeLZaFN3GXE3XwY/KRs79pQw8m/GG9aSIZ80Liudgqq/tWVi0wUGWKyroYu4OLMDAQUayW+FO9Y6QNi7CV0XxSxrveUa2+4L9zZXgiF/VyJLVGnU143aS8Aq0FRq1ObCKxhU4Z+wgiBderJ2XUcDz6OdXPAamsJS/UA6CC6474crxLA9151ZVonaDWk5vMBBiRhyz4FLvAJzmwJj3LQZOiaCJG/qkQL7U+L8eUN14rP2wpuXZuVwm1zoZDbcHp6UOpLkqBn9bIHDNUQ5pIB0ND0USbMNTMHXtBDfxDzxW8FBFdrqIlVfweZ8d0qlARL4BxpqrVAcu5PUTt+G5xV3PYjH4Ha4lIbSG8FNylLWke35/GPUWsEq5hlMThPZtkSAT6ty5QsD8DmNeWa//QjKL/sx5hIENCxEuJEx8Z+FZJCHUkSlq1q2ZncP3h7/ruDcSs488r3ldxfENzIK/o9mkYK2EIvv2ExPfkfAphoZKI+Md6qhbxpNB5Wg3Unb7b1x3PAdaHCnyX/dOR+dM+wOUZVIxvY7fY5a84tsWTw5nf+9a6EBME84nJyZl8FZ5xeYzZNqqKWoIYytjDD95GzabuZu0uMs9QKyc5W2XOVciz785/kr7cV3ZEdmQBz2FsfcHIEiI0pRWR0oahQfEjqbZx0FM6NL0TXZqZU/31jW//6i7FytBXyRK1KMz4L2yN/tuvAUPqczct1Zi254wMc/MJ+yolfw6Sj9DRip42QcnXB0sAXeJ/tbSuylCeHBmy2ItRfRhD6egWymEESLu+9Y1+ms+K5wMspHTcbQGvzI6tADWUcOiplcj/aWgnwC1m0FcuUeYz32BNqcHJgRLVetBHmxBz68dvIEgOoN/53A0fkRT/9Lhy39rAFPoO69tUTJW4NPB/isr6EchKjh17/kzEPnwzyQDaiijagYtaK5oV5gQ7yGjWVBcj8kJxltGUuNIevoXeurgvtrhlk1/OD1gRWpGGk4iHh/pWoASWIPg4kqmam+USneNMEYbse1W9tskRBqLzpPQtyFkFEmEgyXO1A+ev7OBchUfuxlmgt0BDfIMWdj8LafQ8zSxYDlvD6mwS3B7nn5lmtDNhJLmNw+WHzh0zJVmii4XD2tHAVYhZPjClJJpyO10fM+59dMn8bU4/RIs6LGbu+VSgrDC8o6feCeuOhwVYr/MePVWD31dvUcQMWWpKwC5wjWTRQnpLCYGrLqGGB4UyWsz8BPE/GO4zGdeHg27gmyR1ulkwCdUQ/Z8b0OAe9JAOZfmPBOgbeoYkeVfCTpYk5SYuJCZ8FgvhCuq72bKuRSZsnpatljAxlEWPkj03NkAL4VSEjIrygBrhWRn4N3HylvF7yE95HmyEGG6qPYnxHQp7MDDCIrtSYSFqMZs3tHO+UEk X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS7PR12MB9473.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(366016)(376014)(7416014)(1800799024)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: QrALfEJD+w+RrMLepKf7+A+HZ3lGLpX9h28eHr5QMr8X8TMbqUFgL/Vhnis5wLlQATmI5TdkbLSTRgSsi9r8hlrf3yDQVVVcYoQ9AxLs1dXuToYpd331P7b6TUqW8bW41KUEsrpQ51imiu2HC1NINbvZ34OatqlgdzGzuSUNbdMH4BRC9jPqSwOxnGlyadzFbUVnBim3Aw34VcTBhb0+h+FjK5kJSYPmvReJQ2sT0DJCi6HhjhpmMd/sq2TgtrYx3J3IFDqKgoVbw5IXZGkdhRnqt2ZmIzckc26+shizz0vf9/r2at62JqkaH92RLY35N1Cz+t34DfcK72HjSFFolebTrVT4SYwloOiogEJIVS1zQWWHRqYrYDLuxKhpGK3hFFwih1nXT2C13a3Ah9KNKivXno4z0XtNBk9DGSPx+J0e3xxWlBn1/KkOoLxkDMOtkw78F4y6JxZAUbqJqa5IETiD2FuVhFrdpU6O12JA4QSSyjp5gA8XeJDxuGnVlqFm8MKM+dJrgv02Awz5v5+MdrLMn3DhseFgHLPQy+qjMneXNEejA/yDbMdZwlE4BLRIAi7pbrWOEbM8ejY4+hx6G1oeOWQ4qlJSuGyazqpiG3Fb2yICgl540DElbHUmS1354zPbniwSqXlNCKXVwYVEcHwBhm1YK7oQPH8/KQOMVOIk4XW2j7t+SL73lYciOWYj6IWH+nGwfv3bW7Hk6dc53wwKiT4GVF+VndnG0ykdwbmXbaKFdYLAKuu1WWiViSRNVZjWdGaTmxdtGGvG0awZDUyiZS33+Ol2ikpWgQfNhnL5uopgKvhcprAW2+QimYTaQokeT2jzXdIqOvu6C/iD/JVqBxdLDcaPnT5y3uZP/5/n/31SCJAryszxskJbDFKsrsosxgjqPK2Sv3UwDzE5ityHHLEaXa09jWE+sY/zxpcX++VxLaayFHqDS3SsLe6gh0+Pe53xk3LJmsnS4bwmiqyrmTv1fHHgwMnsYZN1oLufzPzE8eKtZ1HEZ44bHlmB94v+QKsnNAIKHCZR83p/iMxucaFV6MN+Qdfv/1RiRd+NL4TVXF/ckRqKklNexuP3ahMhg7k99fsR/NhTCDRxTBoTK1LAI5gWk8FoZdN+wY1MEp6smR9RO4hYiD3gngCnCuCq7WBNMpLd34g5ue3lUVcgvGUN1q7Bg7OdU2mulV9nL2ZQpROnG8BJjCujtYwr9eRhlm7G1UciCVCKkfetRqeDGQ6mkMcXwrbxJvnEaNWB9QKFtfXj+7MR3lYZKPuvbdg441VQNOyJHA7a33ObRrusfsx+7jNMfZshO88P+dR8N4pK1It7OJr9gINa3wBrmQq6uaBaGVZMdOEbXjwX/mqTpP3jQPDuHp05i3P/PXjHHSYuwTGQd+kf6jl1pv3/paucKr2mHb5T06buTHOIvdybPao6MXxztLDMYXEWXlb4c1Gly5sckAhDS+/65nosJ/hKXFUXFb2c9zxnh2eYcwveqPfSMB4B1VHu3NJcZDHlJKrjn5Dhqj5NmoHN+teYW1TjhpMDn+yVu4kv/66eJkz2kDE/oslMxy3omQlMxatdedDantR5IWUyfnDTc4Rv X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: c34acc3d-dcae-4bb5-619a-08dd56a9cb0c X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB9473.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Feb 2025 21:09:01.3537 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: GiR6kv/7HCehPedxbSev1XIxvEyBSz+iLqM3+EzU/OUV0/yaLArNE0TuncZB6ZqN X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7221 X-Rspam-User: X-Rspamd-Queue-Id: 11C9440013 X-Rspamd-Server: rspam07 X-Stat-Signature: wa1n9op7y6d9an5wg5dpap4intkkds3a X-HE-Tag: 1740604155-636683 X-HE-Meta: U2FsdGVkX1/QH8ZKoDIH+STZlThdqEnMWjeXsuRj02XbNVkY9k3g+wzcqToEEoN80NQiKV8HQHLaZBffgAIePMx453DbE5ZGctPpGzlZSw+rKSqWMCDxa6A3Mxb3BrfvThJ0Am4jC3HcLHI0oV/yGtG6BsHNZhki1PHxFiKE0UlRGGVOdxpQ2vVkXhvkNZm66DFKgOFO38H1Pifqa7h7PGZ6Gva49wWjEhZHgs+lJCjtoMFBKS+REXfxlXq5R1IRX60/aGeC7wuSwJGmuwwnutZc2STjTDz3mhOWMJuG1ti5kWw4w/vS2e7zQMkYrgz0yZ+25Zk30pb3Zub6UFCODdseQz/nrmpLGSsMulbAAhmT0ui94LSguJU5RvH5yNkHPXPulipp6vNCOLxo0gSblvvJzigAm2MpyY4Iq/6AmVwiN2hELTvhagJdq090FgzA/ED7/XpZ853VuSIYjKCPENYaho+r3F22nMQlzLRqQ7Kx/u7PsM9TDUsFYPIMkJLpHLvo7d446Rrswe5M/vyk6QldxUpcDGAZf6K/RP7sOyWVt/uR1PRV9drBx2Lm2H4osRXcdTwzduuIudrVDvqmkqazu/rTDHF0AgbFcX+XFh8DltKOqwBoNTcUJkYh9AeYRMHNStBwKVtP3/qh4IlvkzJSzUMxZfoyBqzW7/AnE79lN4vSD7GY9/BKRRgYr8XbVdBFWHjj4FBAQHBGCeSZd7PCoWnUASmM3ThwglYZVEmdHJ3WFotrIOyx0zmINy1eKPVkKrcFGsjuoJEGmBcirV7Mcu5iutU6ZZzBl2Xs6fEM5iZ2kskC/jdKz1CjSMLcRoKJ98opM8khKqqXeQu4deiPfIrUWZcs8ECuOlXGaTacL13R/V66n7bQ4c/yAdi4ayS+iGzrGWTZmO6YNKfimkzNxCs/y5rfLtB6GbJkAzUnkpR/6aahRIGNbA4jqshX77rzvjhOkh4dv4r3LZX xjmh/WL4 sYPL7zraMPNMKiBjuTAe7ZN6LlfUScqQPfjNgvWWlPxpb0oDCLSQ2BmcWW3dE5VzF1pSYSflEFSAa2I/n1IZommp7Q/EPM4xwwkqEVu4EPYqDUBLBq+TGkzInDjLYzRmTr3YKbfBBDn8a3HfmwQB8IGwo4eRFJ+AR8L+5mSHYgMPlbw0lXglZ9rob25Rw0vD3Dx9dAM51dNfn8ljJScGsGmO7nnaP3RY+5Dw5ULbUk49yf7Lx2q//sx0GlFvZpnsT6Bx1hq/T9k4k+bQRWEWpTWqys1EOULx006080MKA8b02tJX9ufJv7uTuBYGegZzbEKUpDu6WTSGhGK6pIPvbSUuwPxNuc6ijGpQPZJOf2jZuV4fIWoHbofOAQK2946IWb76qO/EJkrvjhlZSvK/VE+jqX3KffWfb6XWy6AaRugxCl+6RplANbWzMri/7aKH90aLAT1iS0Db1g6rlYSScICwS5qJ/zQsfJg+US/hDNnKnDWa53P8kWqA2u9as5a2xRjZ9Y+6qxtcgBN8gd+/mQ/AyoNIPQSf9ctZozz80haXMg0mZtrCSHvUis3gCUMNZZT/uWCIbLgd+7iDl69jBUQRstlknmyYyQNFwayXgZV68WVP3X1cev07ttQoylUTL6npdU9QxTOMFFqzEeQkVGmvy8xCMbXpG5Zb7YejzLycm9yesHgAG1X0IUh0AhIbF9VHaedLoDwf40Ww= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: During shmem_split_large_entry(), large swap entries are covering n slots and an order-0 folio needs to be inserted. Instead of splitting all n slots, only the 1 slot covered by the folio need to be split and the remaining n-1 shadow entries can be retained with orders ranging from 0 to n-1. This method only requires (n/XA_CHUNK_SHIFT) new xa_nodes instead of (n % XA_CHUNK_SHIFT) * (n/XA_CHUNK_SHIFT) new xa_nodes, compared to the original xas_split_alloc() + xas_split() one. For example, to split an order-9 large swap entry (assuming XA_CHUNK_SHIFT is 6), 1 xa_node is needed instead of 8. xas_try_split_min_order() is used to reduce the number of calls to xas_try_split() during split. Signed-off-by: Zi Yan Cc: Baolin Wang Cc: Hugh Dickins Cc: Kairui Song Cc: Mattew Wilcox Cc: Miaohe Lin Cc: David Hildenbrand Cc: John Hubbard Cc: Kefeng Wang Cc: Kirill A. Shuemov Cc: Ryan Roberts Cc: Yang Shi Cc: Yu Zhao Reviewed-by: Baolin Wang Tested-by: Baolin Wang --- mm/shmem.c | 59 ++++++++++++++++++++++++++---------------------------- 1 file changed, 28 insertions(+), 31 deletions(-) diff --git a/mm/shmem.c b/mm/shmem.c index b959edf15073..f08ba9718013 100644 --- a/mm/shmem.c +++ b/mm/shmem.c @@ -2153,15 +2153,16 @@ static int shmem_split_large_entry(struct inode *inode, pgoff_t index, { struct address_space *mapping = inode->i_mapping; XA_STATE_ORDER(xas, &mapping->i_pages, index, 0); - void *alloced_shadow = NULL; - int alloced_order = 0, i; + int split_order = 0, entry_order; + int i; /* Convert user data gfp flags to xarray node gfp flags */ gfp &= GFP_RECLAIM_MASK; for (;;) { - int order = -1, split_order = 0; void *old = NULL; + int cur_order; + pgoff_t swap_index; xas_lock_irq(&xas); old = xas_load(&xas); @@ -2170,60 +2171,56 @@ static int shmem_split_large_entry(struct inode *inode, pgoff_t index, goto unlock; } - order = xas_get_order(&xas); + entry_order = xas_get_order(&xas); - /* Swap entry may have changed before we re-acquire the lock */ - if (alloced_order && - (old != alloced_shadow || order != alloced_order)) { - xas_destroy(&xas); - alloced_order = 0; - } + if (!entry_order) + goto unlock; /* Try to split large swap entry in pagecache */ - if (order > 0) { - if (!alloced_order) { - split_order = order; + cur_order = entry_order; + swap_index = round_down(index, 1 << entry_order); + + split_order = xas_try_split_min_order(cur_order); + + while (cur_order > 0) { + pgoff_t aligned_index = + round_down(index, 1 << cur_order); + pgoff_t swap_offset = aligned_index - swap_index; + + xas_set_order(&xas, index, split_order); + xas_try_split(&xas, old, cur_order); + if (xas_error(&xas)) goto unlock; - } - xas_split(&xas, old, order); /* * Re-set the swap entry after splitting, and the swap * offset of the original large entry must be continuous. */ - for (i = 0; i < 1 << order; i++) { - pgoff_t aligned_index = round_down(index, 1 << order); + for (i = 0; i < 1 << cur_order; + i += (1 << split_order)) { swp_entry_t tmp; - tmp = swp_entry(swp_type(swap), swp_offset(swap) + i); + tmp = swp_entry(swp_type(swap), + swp_offset(swap) + swap_offset + + i); __xa_store(&mapping->i_pages, aligned_index + i, swp_to_radix_entry(tmp), 0); } + cur_order = split_order; + split_order = xas_try_split_min_order(split_order); } unlock: xas_unlock_irq(&xas); - /* split needed, alloc here and retry. */ - if (split_order) { - xas_split_alloc(&xas, old, split_order, gfp); - if (xas_error(&xas)) - goto error; - alloced_shadow = old; - alloced_order = split_order; - xas_reset(&xas); - continue; - } - if (!xas_nomem(&xas, gfp)) break; } -error: if (xas_error(&xas)) return xas_error(&xas); - return alloced_order; + return entry_order; } /*