From patchwork Fri Feb 28 03:30:56 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995529 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 59BE6C19F32 for ; Fri, 28 Feb 2025 03:31:34 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C48686B008C; Thu, 27 Feb 2025 22:31:33 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BAAB06B0092; Thu, 27 Feb 2025 22:31:33 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9FC116B0093; Thu, 27 Feb 2025 22:31:33 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 7A4FE6B008C for ; Thu, 27 Feb 2025 22:31:33 -0500 (EST) Received: from smtpin07.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 0A343B3FE8 for ; Fri, 28 Feb 2025 03:31:33 +0000 (UTC) X-FDA: 83167928466.07.590B84C Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2062.outbound.protection.outlook.com [40.107.92.62]) by imf08.hostedemail.com (Postfix) with ESMTP id 3EC1C160006 for ; Fri, 28 Feb 2025 03:31:30 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=EfZ0OHr2; spf=pass (imf08.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.62 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713490; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=7fpOKX5mt97+LmuzK0U3LyecTQPT8dlAshklE5pRhO4=; b=8bw+KJkxT8/8+uO7cgEar8WhccDeBkf7EnDWq4WB5zRO/gtoXQdn34ZfzUN97gOl3S6Jv6 Z5QD1PSJjkft/rPgkB/AAe0S+yQwC3C/L15qMMngPF/r7/N9wyefG1vI977CIMw93yzOUP uOsXBUaPCEoAbTljjgu/eGEjS7DSoMo= ARC-Authentication-Results: i=2; imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=EfZ0OHr2; spf=pass (imf08.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.62 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713490; a=rsa-sha256; cv=pass; b=clbsOQICOL01okoyEb2G5M8fXc/cVTO3ElyQNKtS40fb7kp+kZlxhwcf1Q5HBAvwCoridL DPh2U9fHiLsg4E7XJLRqa1qeIONMJ8DTRDAweb4f0w/3fppx71PT9hPL9YPKXooTXwwgqB 5Wgw1JbYDZWMmCVWpSnMrVLZC9WLHRs= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=axZRmNhuh0GMFUL2TdBtHhgGOXIc3fy0GjhtemjNp76fi43CvqjRrkWUnK4ih+GTrcYFdqP2xyEi8/5N1njyhJdT3v0oMwjKKfzoOJpuGAclCVTvN4gjaDqpGvU3KkfaJeTFtYPHJc3F1AiMGW7C+YNU2NYD29LJibFbAdbSOyDo0/wx4c6bDlLBiJcCyxtf4BCz/Y+15yPXVg0LflLERVhM9Xdjof7VKI3dTCN+efRsBGClXOGq8IswLe+L6nXTy7DpFf4FYt+hbnAoyGgACxY/+GOGweRzDNMQpegqSkpB9xTf4Kl1MWgLA88ffI9a+RRcf1hJ3sqnTK41uZaiuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7fpOKX5mt97+LmuzK0U3LyecTQPT8dlAshklE5pRhO4=; b=A0Mo+qyFZR5m04ZRiPvVy1C0ASZReL2BsWTkW1/sWn0zyFAIvfjPaaDwMqaZSJjTffsnE5woj6JdpNld4I5nw4ObW+Rjgdqs3dYP5wUrETVWnEN0Cj2KoTIs+O1Uw6gzHx6s7lLRJtns6adTEXFXrAQcxwkWVlGAky7k95eqHYBAaMs4+SQLrFFPADiC8B6H8rMlVOt9pnE8k+qpbVx6I/Mxrc2+Fuxpz2k65aXtqCMGghR0OP8/XSp1NrH6Q2EdKhwk33NW7w8lscHzSlY8a1vjgI7+esQqzBBoyIFywL823xqw781106decJzI4oeTVRSBOgkiuau7rmj281aULQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7fpOKX5mt97+LmuzK0U3LyecTQPT8dlAshklE5pRhO4=; b=EfZ0OHr2XvQ3rSDEGxlkwv93I/lEFrqYSSGLCkEBYi9U8UUvhdnXYNphUqsxSofh0mM1dQSkrn69WpLnT2WZje+7xH6Je2g6k7+1RE8WfEivZF7VJY0X8AQ9utlWZmdHYkZZAe2AZ/z9O25h8IeaLallqgm/UpT6HEcocfXfmmM9xyTqIkpeVLQNIqCEYnzs8DjsNLZkT2GONSS2wXIYiwzmBJ0fTn7VPbfqP2tE/oD4kTdPzaD87f7HzkmyYpByFs+Lj24fJWclQNy0d0RoR+gNaMBLzRZvxauLnw2pHEVI/QsPsT+cS2tl9SRh7ESqX0dkxS2Eu6O+0ToOrZrZeQ== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:31:27 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:31:27 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev, Balbir Singh , Vivek Goyal Subject: [PATCH v9 01/20] fuse: Fix dax truncate/punch_hole fault path Date: Fri, 28 Feb 2025 14:30:56 +1100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SYBPR01CA0050.ausprd01.prod.outlook.com (2603:10c6:10:2::14) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 462f3f6b-e815-423d-893c-08dd57a86241 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: uaxkybciKBx9pL+Z7k2S3EyQ+dnds/q1XVVjKieTZfXEG1w9p5XjxDCSbunZZx8q+MdHfVeNzMQoPafrTtf3wm/31FLEAkz1oNhfcR84BUF79nSNOXQ+CE41dhl2RKiGSF58YdKdwUj9vOH9FUUXkCjPOjh/98uW8RMY/j3MH/GxFGnizl+e82+q3vAxVuGNf8u/ItsS7cjwxQudPXR36ATHf58z+ym0tQ2ZgWtbXKgYuil7oV2svjStx4RVa58+jb2Zinl62nKpjI75tb76afgUQ7AFGcTB0EpIvJW7DO59NW5gVkaVQHM2YKHKFCLelOcNfiZq8bSoGIRHW4rKyI54fLqLSrD5kWyjfWGESI+Zd5ODINQ4Z5OG2/UuF3tci6jxLrJo9rj6ALftU5JZ6HeGFmtI6/8/TX28VMyBQs9Bzh7coh0aVBKptye1fhng33Wvs58mj92omFdxMCiqlX3GMof4hIyKZ9ze4ZL0vvcnQrWsUUemHQ9e7zI1qQGoG89fpEmbHAoRBVcnQCZcprKvDHtIES3HN9/lhgmRZbpAxViHA5jTjbA/8vKneukJG5pHlnWI2YdaAZpnmaxEs7sl43cqUREHgdEYHFq7BcFh2YS6Uu7OBhM3+kxD2oqFwCFTVxgf2x7cPKPzsiYZofOgjFyRpWIV0CKQwzhxhfrmiDLMq0goj/5Z79HZetujtA+LV539JNJLGnOHcyPLldilXsA7T6lGod1MWyD70HOr86at1PihiV+e/GDkKyMgEJY34mnfud8J3GIKfHAvLMm792N9IP81a8AelPrn31/AlPpjqFPXfiD6aF5Nb5uALL71ncwOJtXmkTx/k19XC14VEORqfFo2rVGuLRvmLPw+imYfyBV0EL31l9AOlkW632HQ8boXWXGLSL7REdxmQShY48EgwRMXC3AE9VaqoBQTo6gFIQiIQBkpbPHo/b+GBIJV64p8cW/RtMRJBSNxdR4JLPALnXwPXnW1Glp8yLHqtzAovHVJguk9pC8D+/jLMgLRRvsYdl8OTpGg+4Qbf1vsIcctERmOjUzeryWOVuL9uSj+dqqzWomd8kOrZoEUUZ0Q90JbhUUoF9Yy5W1ps2/5SB2BOpm8Wa03VZY3aZBBroQaBavz4ognVfJRMBrdkm59TrxlZwFjgEtvjfK+tgtwqYTCgcT6y4I9CLxLlcTHB0SIEi8wYt9z39DgHL1ru4sLqxk4+d67zDh/AtIryAXkTBJ2sIX+c9hQmf3UXK+PrEPXPwklE3FnbCt6jKxgvgUYwOLcrOv35WsxGEPCO/zja18hLHmCiWpHUafUNgjKzhqDhCGWaCJDnZ9tDH4kLKXFUjPi+L9ulbRMWcmkoZ1IVSS1j+qNkOumXa9DOq3QpXfjH4UrFr9r2kbJaStR X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: sHwKyQwvSsAQpRA1U/ghpgjinyvWXEEeWl1pbHIFWK+iqkapS84O8qIxAp1AzWZpYaos4JSulhbipKBYPtySoyagsj+hF+5w5/sHzwbKGBbvs9WrZNsSjiE8DPmw1TLQk6YAR5DAWfhsUtlkDQqjFdmZ8CJbs+CglCEEqgKPjc3Wr4J4ub2rztpbMgZNUggu+Wv+o0emgBnJwhowFleSGHW3LblNYft1dohlS7lnLDB6EzVWYXPJzxYyYrgZbtfvEg8hSTRQq7iwovWW4ragb4S17jPA64krokpteQQedoaONxId7IYzfS95dizoUVskfpH/ea+FF1wya4Ky2lYD0FpP8imvizAfXreKV5OTKhplSx9qbVILiNd2Q6ytb1YVcmRyW0OXedNGVSDGlgEqmcOMKSSQHIOHyseQS0UyIPjad41sNKrr4eJZiYx3c9sVVwO0GhorfItY/kXgYjDB2QJm7csPK56P+8CaFgDIAj/uXcUmTLVw8v0OIp6Ew1Ok+KeiPrQzaPDAxQQOUQGQBBl3ngEDh50y1a0gdPx5YkIO548UGep3lzDlj4rrPOXwgRw70DNpknHS11mXQxpD6hRgaaFmCjyZPyk+i5Nmqq7AaRGSfsmhXXiR2CqFVBOLz4bVOgeDpCyCAJ2NQRhdwGu43l+rMCXisSTM1KEnJfgW+q4R1iRAnkMH53UinNUFpuUvZuWIUluArY8FFr7g45+XPp2P1G98TimSPuUcJyfFrMB9dpn1aZSTDH6+I4BhZUKPx1PztP1qgqWxVzAqoWujXgotMtkJifyTA1UUAZ6HWIOVD+Z2RwsC6JeoMp/FObqeF26x8hrx7yGw9k1baGW7kXuJE30my49AuEeHDQfw5FJWrDIybdXcXi1KjumB5hXBGRa1tCdwA84titQ3y1Im3sQn0CQvmFmZ+XkuqVH8J01dF3m/8jm98n7ghXpCQ9AoEfIJ2OznGeJq/Ett9fvQd/kJkcyUZgXtzfhKn4ZHOW0+rg1YlchYDDhaDxhmTVXiTz+x412HXJX0Vhn+KXZhgVhPiiYKKuEHZFVcyryKa9UifD/4ngWCjq0razkSz0MYHT7YXa5FrSVEWdEObTxI9M1lX8HafiyCQTPLj4QvM0K91kIjpQabmddp3qAiDorlQYj2lc1uZrGrnmlgV9cXN3nTXumfykv75ICG1XvYeshR5BcPNB95ec9P4yW1EPXCLIjKBexOAoMIpEwhHXqIrvRUy4yEZoi/d5xg1o9jTQ+7ZrxlFUEhNQ4ppbg3XUslAaGvaYzgFhYvcIzTE9mIhTjZLE9vt5jncONXa/N2kOJ+QsiQH2YFsqPH2Lk0ebmotu9sRQ3A6hqxjtjC7kiZfcgq4HIqhIYWkk7B4gjU5N3S4BfuILFtxZMMkqiPzZHAPGkbBazMHkKcVgL4hQAqF9V2rIjyeBL5jplh+lccM5QfCBqeEur24dvXg3YJ8IyRoOBN65u+PwtnauA+nbXeL/PmauTskri4yN0/NMQ4PEoZ8siKKAQrPGI8tiyxupIhbsaBVbhOy4O5h9nDlROcJ0rI2r8IzyKdX7eFEeqStA5BOV5otZg7G90z5V45 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 462f3f6b-e815-423d-893c-08dd57a86241 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:31:27.4940 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +C+gNYtBEVdIk6n1GGM15g9rWRqwqEeQQuuXT6mE8gWGUIyluFJnY1nbdM84n6pTfhs3M7ToGHjIJhhZsAaj0Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Stat-Signature: nqiouxox57tiepzpp1ggom7a3qzhd89c X-Rspamd-Queue-Id: 3EC1C160006 X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1740713490-151978 X-HE-Meta: U2FsdGVkX19Tlt4lJpY7IkDJ2n5bE1v5M1OuYfwjxZgUp3uRVTV9bO0TitbMGrAvaky3LofIs0OAQWKp+A0aWl1s1BjvaewEzL8cp3lj8NYMUzekZmdKSQa8ePtVxGtvbED+fokvhTxMDjyeDX3S4QHv7bvsh2Bk+rlsfGaq8SM6BOeT14QyFawOYWri81b9LOxtUr4cIbAFpQICCLiFQSG34FcV/4k4fD29Lj/6siNjnIf268Ml7bSCu+FqpI92ylAcaU3yiSsqiLNjZNeEJ/d8YJTsK95Ydr+eaAqOLDZJxDkINcVmQ7V+UsIHVYMHGPrFmThxEfdhkbAR/u5WovRE6DSKL0h4pwk0RgLU/e5ejV8iGOoHXpj0dJTSSUJhmLQuyGYRlcBJKBkpZc31k+PSJbVQLyXzTxGmMbtuIHHqS/7bz/K1MvGOOjKtFggNaHuwEJo/43dJUAwDE82dmckNR36AOOrrEqpXB3pwoluix56b++eT48V/cy79RnbGfRekBACXMWAiWaJAE77d0z6zx/EGlKAm+x4C1rqr8X58ANho5T8K7ZZLoS+47vqJHBk07QuoVk7HhNAcK9QQplRYu1WkkVo8OCg3sdHKBOye0Lv7jr1g9HITs+A33yTnSJBe9Zeo8o8XLWe8D5Idi1ulcVU8UWeAb3h6qUCf1fB96bybQsh6fsxyoLSyy/85MEQv9soqST03s7cqJE2qjpukZafNI17vEGoS2ISntygHa0oJu1lsh3bXmkc//sPotT3VppeJh8aic+pQBs6rN0t07GHQPlLzxhrkFcvGfG7FnKExcySisMDa8ac85F84C+U4PzhGZlz8B13AlaHip4MXHUskNOh/jtYHblcvN2BGCuI6oLLavri8AKTQ6y9HVzjdksmJoweOkjuvCmcIT8GnizbcKhgVvl6KE2zQWfeLX8G6KXno+XFIG+mJfdhxokxv4CzzE1ROvrwpNNQ +VFaaNox NHOaZOXgA5dLM0XsvD/bn9pla+lFgsIoOp5Y7KFyVvb8VCCSG8G22bLcpfKc2ydv1IzsqWRO0344tjuq/znS5ctYTkl/+df+BAFp+qvQVPhxu+QPJnLt8zogJrPLJ4178Dq6Ge3O85hvGzBz4N9iz7VUhRbqChc2DmSXpHbzowS/Zt26ZDyo3lZEt9CF1b5RmJGcPuldapLzLv906HQBF0crTYg9RqaSBfaeVWZViswkUhWbzBM5eUVZfv2ZBiNQ3ozHajqM59mLGahhD+Jlm/pngsPWcIh1iUGlOQlCPjmRNnSOLT1kOzSRiCDPNIBw9j9nXetbzE7fkMKzBESuDEtLgbca0gLbtTjtGWRrHitMDmou9/vupzb3oZCgcoc2wR102Ciyd5jWWD8tHJBRkF+IDSHy6ie8r22yNul2rU6yuPjk0qFh5ixjrhE9zpYeHppDlK8mHEdfBO/dFLZyJFEErnZOPM6VRwq1k8B++34J/1yvVVJixafa9wM27lK0hvIvS X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: FS DAX requires file systems to call into the DAX layout prior to unlinking inodes to ensure there is no ongoing DMA or other remote access to the direct mapped page. The fuse file system implements fuse_dax_break_layouts() to do this which includes a comment indicating that passing dmap_end == 0 leads to unmapping of the whole file. However this is not true - passing dmap_end == 0 will not unmap anything before dmap_start, and further more dax_layout_busy_page_range() will not scan any of the range to see if there maybe ongoing DMA access to the range. Fix this by passing -1 for dmap_end to fuse_dax_break_layouts() which will invalidate the entire file range to dax_layout_busy_page_range(). Signed-off-by: Alistair Popple Co-developed-by: Dan Williams Signed-off-by: Dan Williams Reviewed-by: Balbir Singh Fixes: 6ae330cad6ef ("virtiofs: serialize truncate/punch_hole and dax fault path") Cc: Vivek Goyal --- Changes for v6: - Original patch had a misplaced hunk due to a bad rebase. - Reworked fix based on Dan's comments. --- fs/fuse/dax.c | 1 - fs/fuse/dir.c | 2 +- fs/fuse/file.c | 4 ++-- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c index 0b6ee6d..b7f805d 100644 --- a/fs/fuse/dax.c +++ b/fs/fuse/dax.c @@ -682,7 +682,6 @@ static int __fuse_dax_break_layouts(struct inode *inode, bool *retry, 0, 0, fuse_wait_dax_page(inode)); } -/* dmap_end == 0 leads to unmapping of whole file */ int fuse_dax_break_layouts(struct inode *inode, u64 dmap_start, u64 dmap_end) { diff --git a/fs/fuse/dir.c b/fs/fuse/dir.c index 198862b..6c5d441 100644 --- a/fs/fuse/dir.c +++ b/fs/fuse/dir.c @@ -1940,7 +1940,7 @@ int fuse_do_setattr(struct mnt_idmap *idmap, struct dentry *dentry, if (FUSE_IS_DAX(inode) && is_truncate) { filemap_invalidate_lock(mapping); fault_blocked = true; - err = fuse_dax_break_layouts(inode, 0, 0); + err = fuse_dax_break_layouts(inode, 0, -1); if (err) { filemap_invalidate_unlock(mapping); return err; diff --git a/fs/fuse/file.c b/fs/fuse/file.c index 7d92a54..dc90613 100644 --- a/fs/fuse/file.c +++ b/fs/fuse/file.c @@ -253,7 +253,7 @@ static int fuse_open(struct inode *inode, struct file *file) if (dax_truncate) { filemap_invalidate_lock(inode->i_mapping); - err = fuse_dax_break_layouts(inode, 0, 0); + err = fuse_dax_break_layouts(inode, 0, -1); if (err) goto out_inode_unlock; } @@ -3196,7 +3196,7 @@ static long fuse_file_fallocate(struct file *file, int mode, loff_t offset, inode_lock(inode); if (block_faults) { filemap_invalidate_lock(inode->i_mapping); - err = fuse_dax_break_layouts(inode, 0, 0); + err = fuse_dax_break_layouts(inode, 0, -1); if (err) goto out; } From patchwork Fri Feb 28 03:30:57 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995530 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id BEA5BC19F32 for ; Fri, 28 Feb 2025 03:31:38 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E4526B0093; Thu, 27 Feb 2025 22:31:38 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 31F7E6B0095; Thu, 27 Feb 2025 22:31:38 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0AE866B0096; Thu, 27 Feb 2025 22:31:38 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id D805C6B0093 for ; Thu, 27 Feb 2025 22:31:37 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8C7DA1A19AB for ; Fri, 28 Feb 2025 03:31:37 +0000 (UTC) X-FDA: 83167928634.02.C0EE78F Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2058.outbound.protection.outlook.com [40.107.93.58]) by imf10.hostedemail.com (Postfix) with ESMTP id C2167C0005 for ; Fri, 28 Feb 2025 03:31:34 +0000 (UTC) Authentication-Results: imf10.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=cdZCsdT0; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf10.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.58 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713494; a=rsa-sha256; cv=pass; b=oXRO+MW7FkuD4cKigFV/7HyMJu9gcSiqMyzC3sIbgPcueATQN0N72LE75UKu5Ok+8Fsli7 +rhgBBe51SuUdFnpjRnwsY7u7ZSyewodeKd/FiI4l2KDoTL78pvZ80wYaw5W617K2/9cuD lNHDDzYGJ/uothUmC3C3g2DVa8Nlo+M= ARC-Authentication-Results: i=2; imf10.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=cdZCsdT0; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf10.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.58 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713494; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=W+P+HGRiqDCmj10CrZO8lY35WcAxWTj70gicDvDjINU=; b=wd2ouvepEoB1TKOMw13QVrpLdwPjroEfNq03fqg/EefpCgsk8OdGHoFuMW7zc0vengVgwW BgG7hbpdRBe1/fCIcUJzP1fURMEUNP2+nmINIqOD600GCtFNKXR9Yc2yLA9D2DAxvWwU8c vW2SYat4UAJGFEWah/6nRbcQrPwUhXE= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=gG4s7VmANFJLBlSqoRgu9ZsawpIGtnbt0AY4IfujbmcoB27kwPiVB7Lidm05m0+2bfhRejizIelIg6sLg77rAAZcSsAYMUI6wP7OQSGG+VXCmXDMAbarR7dorB0p4Gv6F0LpguNrx8WmmR006GxOon5uTCT/xx14MlhhfrQTuW96rHuBe0wPY5grx4umlbL8XLaTv0NGH5sGx7lrbNZ6E/ejZICq9JWN1o+LyVcDh82iCemmF4YIk1AiG0zhR9SIBoXmHB0X3CRE4Z2C8ueAKKoXV+rg+n70MqIcZOGwp1KWErTGbvMklK0pTGYOL0XHMmRgbo/pPcRLAEtg+dxf3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=W+P+HGRiqDCmj10CrZO8lY35WcAxWTj70gicDvDjINU=; b=AxixWnNcPZj28LPFljghMk+iFNinEBFTobh+VCBWtroti6sLzSM3wgAXG2+dgW/yTj5EfXboK9arLnEqaehQinWCVvbR6BxIn/gpP+NICFoxJu4W1MLDRWKJWICQP6IYgCB3x561lOwQloreyMMz+FKV+6RuV4qd49undYvg69TqtEaTstFDxNzOAfpMkAYfNEQuYOiJvMeVxq0ifcAH9xo8J0bPhR0zOXJAuqkgnl8qikpW7obpfOjd0JKez1q+uq7ld5iNAKyH0/BQ/fEGOaCso++KReATTVsnzeimn0uYXUfoc3MsQplWFMv8pwUaH+p9lzHRDGMEQifzH9/bEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W+P+HGRiqDCmj10CrZO8lY35WcAxWTj70gicDvDjINU=; b=cdZCsdT0P2PlCAjPedZsDgXhgTlE7nz4PrJw2QpYY2zoXqNJq89E3X3arDp41JvxHQw8+nYaQuVP5sZtb1DuKNrWYxAmSG/mVotZqIv1Y6HQrxWQ95Y56+yuea5FS76YAD8mhM6W22KNL1suONUB5cO95xkEghPlozh1sgS1mE4xP7WbJ8yhw0T/J8+8TLL3pNiDADhaupNFd5dTxeit7MMw0spwVXEg4sjL/ltUcZ75tnB0EHTR2Pyy/7zKincJG5Ua8QI4XgSg+Uwh888QN00Hlf+xcWQsEs1/WlFO7yZx+ppfZIV62tzCUh92ou1+ftsNuFs9sDAwcCvTS1SOag== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:31:32 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:31:32 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev, Balbir Singh Subject: [PATCH v9 02/20] fs/dax: Return unmapped busy pages from dax_layout_busy_page_range() Date: Fri, 28 Feb 2025 14:30:57 +1100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY8P300CA0020.AUSP300.PROD.OUTLOOK.COM (2603:10c6:10:29d::23) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 8801b662-b1f5-4b0c-987b-08dd57a86518 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: OnvgJFBRUSNrk+CBJHuQoT8NVAHkoVi9OS/BjpsH1JzkSouymwduqXKnzMnVdUj7BgBHV0+GEe/mYKRmzUk5djSLFetqJI0/lnxkm3pbnDLw2l186rYqU7EV9PodPkGQqScytyNopv+My4hSSt9/e/KQDX3sQvUirPTPr/+Lkml8PVXQ7CuarhkmWm+xlWqflccIasdGl9uqoqS6VWc9Co6Wnl0ni+KIalfs1L29T7z0LlXfcNsfc+KS/9k80F17883otU6hNOI4kmHa8LddpZnDi6fQzOaJeLLP+3cqdK5IVzN9ZJLXz44bOZvP9cSxWfaJyHlCq5JLK8Z1idEGbhCeMZVqEfoH0BFQckiNiQMRLHkYQiA6Mi31xVjj9JYhdyaNlCNIe/KPapP/ylGt37PAaVIu72ghqrnwSK49aOt2VgUjkYe7Q5DzLdI/SqMO2PyB3uBpBM5seZODHGg6NZY1mGp6uZa3AWaIzg8K2u79dQ927SEqwSoJHQpSMZuRHLxX39qnel06YwLfYfdXAwIS7S03STgOK9bE3+B5WjPf183H9BBsxVGk/rbxgszUZLc7OxHHbsaMj7GzGSK/m0hc2iee00fQ2HGfIdSMF2DDzLLsGJbsQ1nlsjL5d3FCYTDcXOjCzullNx0DtwThiNfb0NGcpJotbS5o7KjulnxGn+JABl6F9fwaIHXLXdwmJ5XAOw44qkQgKIeqR1bfmnGU4+rWX2uwFaPdnUaxzQZrcMhLjVhSAZ1RnyIyK5iFV+/zbTo/44Xb9uF4m+DV9uAaiVGPE+Tb/0Y2YyfTKGzlSkvDiYvNCIGGPmEJToyO0axQgZZFQwF1BsCq0bF26FQenWxnbe+oUww/Oa7/xblgOwrtwrWfmPUbs07dpWIR5XV+E52QjUmfuwJLdjM9FMJXlGnuQjlkD5YrogZs/FBIGhYMR7xeYYBv+J4/aGtEHglveoGUl259iQxaL2s6p2P/0jbaCO5ejVLj0HBPvKJCMQXMgkJ9TmXd9sWjK/lIspJxYzv1gCDmRx6raeV++wreEehgbkQ6kKeks2YZvMt/0dS/GFNPlsYU54fJ2MIfbtrGMdxMZmfsZ9cMj82B6KszhIxqRYD8Up+Fo3+NaHC0HJFyyFZKn5nxz+zitehx5ahm1PYlUdAILVHtqY2edHGInUGwJMcWO8DLgi3f42KRoInoiNl/EgyC3uDxwwUORLgGB9hnfN/UOI37ZW7ymaWXJYiEd/wfO7JB2IAC8/9pBWzn8KRYb4LLBY0DuXYcxfAtl7ALpVEedy+H2FdZxGbu2gwLei8q7cebiwMRCwsXd4wEE5JBMVJNbGBhG/1854GQGm3Ub8bt21g1iAzlF88MuWHXmwNjOnBE+4DCts4MscLlvPzOR6z3qOtjUHv8 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: /BzhKb1yh3yhGY3YTI0N0ICOqBFKM2XTWPWXSfYrwxu53r8ZauhQRgB06mMnOL754x+pIY68/nv+6QeyBrzja3SCd+ZmNjVtA1Gnk2hw5rKra1ZqcnhWHGaN5I7Qk+xQjfA2xA63AwM5q3rR/KaRdBAuoROYhCHQxanT7Z55WAbQkV6X6v1ZCZkkA6F+7QG1lrkqrwy/wDQeVpr+BQczdXq91+LXrCE6XFcpaJIf9S5b6hcr/1+dhaHmZAxfQGGrZ2oHk8DQfOWYLNtquKEkHlUaaM1CaW4bIQGzpqGYSVTYHw4dEDUklA+pf94KUm5NINbMBpqzNl0eJRUgCtYgrfyV/C9tI6NqepzSUGXo+9rAUxVCbJ7OxLOiXaKtmBu+A+7OFtPnkskoma7SOC4egUQAqCGeuWHCeo4Z0LjLyluVM0Iw971ov7K3bYEoDo4/M1Y53QZjG0lDj1TEUIGdA96NbKybw/s5/M2yssjU5tg5OWQ6MmLkUTiE5irkN7BCy99ph6OAv1TBubnwXEPV74vu71RLhDv2YtnM/RuBWOuNABLTUdYXngEshcwvcEXwPmK7F9wDisFNZ48SqjtJfCszwoN+X4yO1rGbtZTtClabJSaXfyw0ENGxkPeDpREQov6yV50IbkJqZPN5oXFTie0OYz5wGwSmu2rHsq0bvLnYyVHfeoavcnLbDIii4Uh49PzBtyDC14nzHg6PuBgf+NjECtFdQrTCBcUCQPAi4Aowpz+RfcnLm4PCT5SMgT2dSWdoG5WKr3/U+JvB5F/ZOdtwRJ3yzO36L2t8rJo3AJPVd1D9u+iueNWkb2h218WfY+scnZ+Ij7c1CXtgIXeRUJ7rIqDLXb61YSp5ZpQbSMBLhwEUUZX87I6a4D/dfYvbQ47UdOOuytvG5X3Kzbn1q8sZHTcLAsHt+pxCGHQm6TDHUw4ZOs7x9RMKTmHrAvTyM4nrS7VcJN4c1OVaNQE8ycveU/tw3ovjGKSYeJN3XNUl2H/NDw8lOCDE6jyon19taNZ4CCp6vfO86rHo0UB0Zr/FdwbZQ+M+N/69p/lYxZ2ATpldNhT+IBwFvRTFv7zpxWQSYD0pMrf4nW7lD9ep1jVLij4dfSAmFtfFSkJZF56SGcOUaLgazEOgDxLdFGaWA5PHMNTjOnRTjExfucNhoxBTP+v76A8g8rx9viDJl64XDYTk1+VLCNFDshCIJRR9d99MWtEkwYcW9utZJJgkFDxQKxyXq4QZZ7vRS9qTEba253hCTbgs58oeE1gTt8Y1pNuEF6qgGmC8N+PFsnqtE/qWkzeVXmsM01Rl26FuUrtpQhArjUjO51nnQtwCJHCyTNyJz9Ovek/fDdxt1BWBujBLeHyVZmdGSEgyjQu2PlhXKwq2gyFo45Xtq1AmuPf8BND3sOb4pYDcDZwmmb1HrptQRaIUR9SKZHMJTLFCKA1ciVs99T8BrajSalBRSytGqZkBCMeugBEF81D7MJVPMGVBtxG2EOHOVPcl6m64/JSXLa0WXD54d9f73YDaK0flVCGhQ6Y7yY4YJE0/FqNwMKBbH+a1u353czIrpGa8K/OyDV081lXV9r9Cp/IyGZ0K X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8801b662-b1f5-4b0c-987b-08dd57a86518 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:31:32.0364 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QJ29k96Fo8a1F26QUQZrW6KIS9VqCNXptbJPD7KmfKxJZGg2sYX97UIevZCLhLU/9m5w4/EjVgcEwCgQ+XFHXQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Queue-Id: C2167C0005 X-Stat-Signature: jtcqyhcgpzbuegixco76g5ew4tw69p34 X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1740713494-915080 X-HE-Meta: U2FsdGVkX1/XrMacpNM7yH0BDH5rEAU8mv1VUz6KUgmYQnPJBkmegqv6aZHYJRHHMieDknKpYv0dcTp8w28ylgkpdVnVd8cIJO+jyuvEgYxcVlNR0RN75SjVX+7gHRXcSykAJ1EQlBzVGPrtV6UgwY9W86/D4C6Ap2GEm/1fnKjQTUdS10J7jQulkTJjYljy+pS9CziBA6pwKRPGrXaVze7Q3o53MH8Or6P2g7G61SIfvOrbh14QGbj/BUVkVhkipP8wIi8rEGxiPkV1I0pc2VXDxgO30xnYYvkYzV8iL37xSkSU9Mt8MquOBZhs+gKSJsqDzuiYXcRvjrlpPDoBJutCj+nKxvwCv/BnvKqmQbzrFx9I3oVsEnSyOE8kS3p+TOhsRJI285gioVYOqa1LIcVhK2VYJ7oeaiWi/QxlmcB9BWK7EaRzAl4EnrnYhi188tT8XXKDjpXEfosNOOO+X5PmjgpWr/F4w1D1O8edUWOJR0gNdit4WSxKhI8X/VJVDqnUzng44hk50rvFIq/WPgSUptPM3FhrJsfiPkYpLeDeNbPaAD04a7jbz6FaZJ4QJZAt31H0R+VIkk/Btw6TucSJeVowymZGffnkcpkdXUNZ4JnMsCMtM1bYJ+txdsdrTpnlW9A0lN2EyC8Tz1t/50o6ySoWz400602io9YFS/zcYPLEVL4GHEe0sbp9ADP8Zc70iz6vxJdY3xYIf5lbEWjgBvqhXtqF9lZwUQkYru3yYmIQ+ml6wpf+DzPFTCeDLJAWzQIsXRtg4RU4ng39Qr0t3L/tBG5Fd/mAPPaxqDAFonYjtQR3vJ/OO76rRnOOtQgfLJEMcs1gxA8WKIMX2obDiffJQQ+bxVnClviiquB+L8wUVQ+wsSUT+berahecI2pVbS0S+XYjmgWMFlWg7lmUb8UPSQHVUGuqj9xRo5RIEeAEHm/w52VMR8S+knL45YrXpQqROcE9tPmIs4L /Z12BIem F3EeEN4JDWjBZtjmmTHFQue1vHNFbM9Qr+lCBIGBcbccLaqgNeWCnZIEhRxVfYJAmTsXf1/DACueJW1ZW7I1Hw4+TzdP9WyFN/NmIU+jw4Sj2NpaVbePZZAkym3RS/ixAPonzNGBkoBdRd7heMvA6WbAA9fOgnRQDpOaMZSfzMUBgXLg6aXQdrZXy4mnsmD76iVe7YKM/FdLU05c83Z53+HAUZ4u8VNaV5DeOU8p6MYUw7zpsligtcfmMRUbg9xugcDcxUzKdrEbHhyIN/4Ffw+p5/XotU0vmZfFe1mUuv5uLoUhu6Dj75RkiBtAgRvStgBQnuwVuzH5pEOL3ASBnPWj+QORkFJMFM0pvIPgKgwHm1FdqmnUIxD4ratxuRdFvpsK89Bz+XCUm3+HBFJi2u7Q5Z4zGfeJG9qjmMzX5ot4NGw9JFLfR9W/IER4Od5PHtwfSC76GtCrjsQR2oVstSI55CY8vkL8RXUSim4NQqkTNuPc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: dax_layout_busy_page_range() is used by file systems to scan the DAX page-cache to unmap mapping pages from user-space and to determine if any pages in the given range are busy, either due to ongoing DMA or other get_user_pages() usage. Currently it checks to see the file mapping is mapped into user-space with mapping_mapped() and returns early if not, skipping the check for DMA busy pages. This is wrong as pages may still be undergoing DMA access even if they have subsequently been unmapped from user-space. Fix this by dropping the check for mapping_mapped(). Signed-off-by: Alistair Popple Suggested-by: Dan Williams Reviewed-by: Dan Williams Reviewed-by: Balbir Singh --- fs/dax.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fs/dax.c b/fs/dax.c index 972febc..b35f538 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -691,7 +691,7 @@ struct page *dax_layout_busy_page_range(struct address_space *mapping, if (IS_ENABLED(CONFIG_FS_DAX_LIMITED)) return NULL; - if (!dax_mapping(mapping) || !mapping_mapped(mapping)) + if (!dax_mapping(mapping)) return NULL; /* If end == LLONG_MAX, all pages from start to till end of file */ From patchwork Fri Feb 28 03:30:58 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995531 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id C5600C19F32 for ; Fri, 28 Feb 2025 03:31:43 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 4D05C6B0096; Thu, 27 Feb 2025 22:31:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 458BD6B0098; Thu, 27 Feb 2025 22:31:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 25CFE6B0099; Thu, 27 Feb 2025 22:31:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 018E36B0096 for ; Thu, 27 Feb 2025 22:31:42 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id 8B8E91A2236 for ; Fri, 28 Feb 2025 03:31:42 +0000 (UTC) X-FDA: 83167928844.20.87206BE Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2062.outbound.protection.outlook.com [40.107.92.62]) by imf29.hostedemail.com (Postfix) with ESMTP id C8DB5120002 for ; Fri, 28 Feb 2025 03:31:39 +0000 (UTC) Authentication-Results: imf29.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=S1GjkDV8; spf=pass (imf29.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.62 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713499; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=P0+55lF1ewPGjHL4a+DdtnQpGk+9xCAh01YxiyDIjWQ=; b=LZv6Bg2Q3Uj5vbBLNk7UnSb48T/oi3PHU/f6pCgfkB1UxqIMI/Vq8/vFDLYUUD0xtOhzD1 ALeUkAgU6ErXTViWtiJeNsDrErs5bjpGGkjX7gGJfh6QFuY8s5ZfCFFEjvPDICoFotVzEA rQQ23009fCsmtiWXV6lW5ClaxauCra0= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713499; a=rsa-sha256; cv=pass; b=7x9ZjZaVr2qUMVFcmSzBqUsX9S6Da7qXnWLCPPynlUtZFcnHpZnsF7k14JKH+nhJ4g95lQ CqDUEr/JhGmXcM/cDR9Gp07CRSg4F7DPcnoNrRgyaMP9mdBT6ta9mg8q1P3Qim4eLpBP5X HP+nD0xgDtlsuSdQf6yXaMG0Il9OwgA= ARC-Authentication-Results: i=2; imf29.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=S1GjkDV8; spf=pass (imf29.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.62 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZcYMKwqh1xo5lpt0K5diHXjFAF1XuVhu9fAT6AzwGbIs5II/xxUTNZSgGHYOxU8OmsgWb86gaj8nZKUwpXdvOaZLUaOchRHyXWYQEf65aCrYHTx3UPenyt70lxv4mQ6dycCcUTkQ9dFdsUOGfheHnLM46yL49hevEbljm0CXfpgnqFU4t1l/1Hp64D6GjrT4b0psNGFwnNPmUZj8/nCyP5ktOjA/G5uI0YUKldV4DRTdFrAYYyPznjMLXBvxuMOYqHx0BGnafX7ZI32wWuLluG795glgzWpbhfluOcinxE0l5uLrbrwg+FZ0VFczNQ8npO2BvAs7KGV7qAHrJsE4DA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=P0+55lF1ewPGjHL4a+DdtnQpGk+9xCAh01YxiyDIjWQ=; b=JmzIyl7S7Gxm1Q/jp1G0GZZNh6wWQ9oFZupLitY5xZGvwfVWz1pwURN0+4nLXKHV7H3k1rnLKn83v8q7HKHyfacfjWyVf2IfpE7U3KfWWIFRJbgiiSCd0C3G5EBSXMjk/823z5oai14Y5TCS9obee2rmpifTzNxklNaYymtqNQaVmBtV0kchELXbiBADaJSdNqob1ApJ5d7iDXiJEk4to8XPiRAW6CNl2MFAhkJwdKLO15qsFsIg+U4T/hTmpfVoZDc+2fZYJXAPlqoPXSJe8GSXz0ATVIZ2psx6k4DIVUHWnUIRQnb/YU3+A9kkrNATpAOuWQJRhtGs4xfeKZCWtw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=P0+55lF1ewPGjHL4a+DdtnQpGk+9xCAh01YxiyDIjWQ=; b=S1GjkDV8uWDQ1V3dpJoeGYfPQ7aNFOwvw9pziTYrcD+6WY8c4KUVDPz4GtGqSbILPbgmXaUtSiigKq9er4ilhpkLcSr6YjdN+0H9DZulcDSrkcMrCF8cwUYwhJ4nuDC4i/Diwo+S0Tq54OBeWAxwMAeTb7HV/1HPEjshS50xfAIl0GC7WKhuxIEecQZuFUZBK8pFjmpdT6IrvjU/xOhFtRwXLoKN+6kUQg3mtdrt+MxDqq8GrhQa4XkcuGvBRhNZAo8+xnE0WADbnSgnUo3xsDB+HpzjvXA14lscMqOKVgN8ALu9LcFHV374DvpJ5EREZ0kRtWZrkCd7CGCwnVWnWA== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:31:37 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:31:37 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 03/20] fs/dax: Don't skip locked entries when scanning entries Date: Fri, 28 Feb 2025 14:30:58 +1100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5P300CA0023.AUSP300.PROD.OUTLOOK.COM (2603:10c6:10:1ff::13) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: b426778c-13fe-4dfc-a2b0-08dd57a8680d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: r449XtztPjVa25o6+uk9d1fQBvhYfXxP28WJhLdA+ZeQqzdtjzTgAPMIAlLnGtPZAuzodMt9fsrUacOPSLeEuBFxkq2LqlN3FU5AAq98BjagmQzdiGuC7SmExslTN4fGrK6J5WdQmiQ6mB4T5v7XVbBu4fsCbKWsNnFDXfbQ9kje0GFCWzRRnt8tPmDNuXuMOPcXqqtPZvlxqelWkdOSUtvvnryRI3uwCInb1BySnbqcibL6VFVaAKoi+sOUjvW4RjUacZeZMXNxVbwaPjXt7p47pgbOxVHbloe07qN/iefxINAF2svKi3UKz8lk7OVR/sHpUOUiRgu97W8m70CzOM8jIO7kUdZ1DKkplz+jCL38eutbjJlvSu9iU7fUMNtSzxxD2HBzGDAbyviS80oAHUUEIPjgd2Dl4n0BAu26OWDNXMoSLJeUHs1KoQHzESdU5U5vpaGaa5ZCXDkpf7dZ3kYUvFe9WIPxSAsWVhgw/J96KEHcJPv6H9SRL/XvDgkGWemWvOdoIFESc8KnQbU14vX3xunCpm25kadF+AtOz0PYVA6IzwooHm4dAMw/Ixx0sKZXWrKyRJRCWcUVrAZhN7Yp1+aeJ5kNTCmAb59cbAF1ujxEROSHv0+rxodXG/CWPT2NXyEpmW35ClPCQjiS5oj0BT5SOIE2DMMXBS3sDYRFhyu12v6yVZP0+yMtWMXVoRhB0vRK/HDM1IFTtfjsCLI39JTSVkq/AlBYp6xOGQeQLupwhG0hHnDOiRPzkN962399jAJ6xrpdboYUd537YWPuo3/lG6V9GPGZNdloGZKs3N4iVjZsJhnLZyE0pkSYHBsxn6ZtriHB+oFfY1Vpjy8sMl2g1wTDh6DAIE46Se/TtDSb1gG+kz3K8UqilhUxeWq116CiCAToFJX+UOHcFcqiFfigX70jrSNtguQxSXbFtnz+qU25FRBnwzRaWVdzGZX75pGGmi1vUqJyRSYA3VC5plBUP72Ocgd+hMRA/0iRNwUklXkiftCCgHvu96POmO5SsVfBNPj1Wq+A7tbN5pMSrqSmYC6izrvcbXMLSFfP593KgdsWqclExUv4og+773WWlGfDrcN/QrDjQo2OSHpN16OifYYKH7TrZbCMozTRii0RF5scZVA+wZ75eg0MdFK1hViqnmAzNF5RLIssBgQIQaaM1ekgZayW8C2Q79XmNcAr/BQ3My4XQO6rYAlKF+1wpeYR50pkz1aL9uIidoBKK3VDBWdpsAnB43iapC2VIxUSywA/j84r+eNpF/Qu1KCuavAteS3Dkng71Aa7+fpeGDNqhIEdNYDHC3Btbe0moslh8O5YYW4nUehgHY0RjqGoGBg34vYsnuT5JnElIB1f78P0AyG+6AgUuNnt+05PF+4TYAFZgwVDsS8x3fN/ X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: JPhIN83ED/8Msq9MjD8XlUqX16BeY3DuS8KhghG215L/5hXb0QbYmwp5qRmYZQjX/+WK9ETnd1ynOo5lgH3PX/Qw85/MommUeJoObmozYiP2c+qzaBF/NXn2oK+hC4utCtOf4Vg7QmVL4ObX0E7rtbiZBKDDz0SmEjTc1f9KbKD8jUI6rUrtqcfPfVyMcCr3GZ+v+fhmX6b4KlCRnFuvjuz4//pi17QBYSZ4nNOueJKBMuFCfYfeyp8kAgruguhq3iIYVGcXONWkMhDVC3GNXh0ErJktA7oLz2Ax3/j1GBEhDaXChYDvvONlZu0gziLn7Od/JSzG34x+rbzEqr4Fsl8cllg52PZhi9rYVybY9IZ1UofwokeWvWR00ylISZ129W4HRSWpIw9gvOr5hc6yMRH7AhrCSCbzdEdXdRG3jVfrMWbr2YMwoKBC2vN/ALF86hxjO8bpT312gGOwCCbW/DRWPnz0dRw+EoprlbPtEsGPjK8CisXD+L4RkQ25qbJaWKSdJgl5QfhbrZYkJ823N4AlOhhI2b5PIgkyZ3NXpQZSqb+ATD4wY2UkWb4ZGY87Gf9iytCZPvAGD9Va9bVIEufuNanF07hRBvR6s4vWfflZuzqhA0b4YfzOfnUFL6me38elXvmRAsa40CVhrOJEpRigZU9EsNoJUvDheiHawg/TCOLxnV+M5A6dzCUgUd14Ix42wHJUa/HJhIn2F+WchJYAfOW8ttGLDAT25r6flGcpgbjzT9ENVFKZjmjSrrZ8K/vKzuBio4rb3qibSmt+aJNNihRTdfewz9ximXplRkNtbidKqO2Wc2hrWmd1wyRcAOT7BFmUIiip0w1gQPhD0UYr/CAiMLNUdQ7aon3+N3gaVwCUB32TiYq7dqmc/dyYnnyH9fZk+U33xvchDfRL3OaLFYKmb6o+PqgxKG/sRY+QhK6x3OLfoFs6iD4mpgvQ/yDwKZue+TVz34eklU+9x8jTZHuNgy8VnvXh+HQaYxhXJkjH1uFmdNc1nDeOXOZuCsLy+heSKEoIweG5Q6XBAfsFD7TTVz1RiwDy+GjnRypKZKdGYLh1W0swZtT0740sn/48vP/B1/+c7s41tjM9Zp8LR8kiuq5zKTI7o97jzg7lVS9VuN6GPTxHkhTxVeF9EwDta/lh3COx+rAtQolsr4V+TXotf12OSEwjoJZw/GFPnfWUccdvi7iUq9WGZLrp5yIswZxv2KtvgrwCbPzt3vNewAxJY5v2GInD74tv7Z/xJk2ZskVz18qpxmUVyoGrWHnc3GAbxLBCV7EDPcgk7DOogOPoLCvaVwu3q/rQMuHGjbFavtCZE2ZI8l1AgX7sT4QIoLU6nUoiPdXB3KUyU26BgjYyvYUCmGvwfeFJJEHhaVTozxJUjz66dppb/MHDA2FV3qFeohI1smkLZOAvaUQnTGhzdiANdMcELK79SpTZE1fdwC4j8dh7ChrwRo9rdGV9d9mW1H0yfgspHQvHK/TOotRSthcWqB5rLuaw0OXeeYrl97GXhUTEdYYRmmTsyuHPYnunHNOSW352Hs01EHLWqJQGDWBn/BRfbvd3dnRTiNBujExvDKzwsZ3YEUvq X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: b426778c-13fe-4dfc-a2b0-08dd57a8680d X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:31:37.2419 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 93/N9nrCmRiQL6lqndLCzp9ttLYRt7VQLCVcxEu2OL1art0SyjdLzWdo+4jL0mUL8H5K77F8QwQUD7H56HbeVw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Server: rspam02 X-Stat-Signature: p95habczykkwbtiswn3o4dfk881y7eg7 X-Rspamd-Queue-Id: C8DB5120002 X-Rspam-User: X-HE-Tag: 1740713499-399459 X-HE-Meta: U2FsdGVkX189Us+x1KvpNzaMh8m+kyEtaaB5/CSutZ9PXLDdapadnRphKb504/vXkrZAjpmF1blcNHoTcl4EoXwhkBvBl8fLN8ZaB9JeFWQS7cLeCKHTcJJYgwpFiFZeonKVofJPZ4odrSU24MONO2j7A6DSekMv5iIon2Xtvnb6iaHQoOoyNntuw8TgHxCHmMnllfTRk/G0sj2iWaj1QtGJ03OcYnSo0GkbY+mYJnkAH3XAe49VI9MZl+bbRrxrp1S1FJGBkqw9l9MCekx1NV7JqK33mNw/jSQ2U9/OoLj4VETu0QbjAHuhe7C9OvkuxfubIUsE7yRmRzRrx/XYxbsbZESJx9vrS/U78FMt2smQlHPQd4guyBTm+dpERYCZXGr4QyFzsb13jzTRnDOLu+r3LRx50BP5AhIDixl4v6IcgG+ke+8IWiXKs9l4g+KcoNlSkH6Xo2ko6jCq1RNkFaOXlEzs+h+wsopwbCdk8Ywm+CjuFL1fE57vE3dVhTFbrTy2XLCeaDrXo9xI2yUbYKg5c2Yw78yF2RDhICNwY7BAIgjQjh11X+4Gsoozdt0W+XopaTkLDVpcyrAYGb4VIMLxyM9gErXpJ4iTMN8TUPPw/9QDmWJREqBmCsvniWgAdI3bXPV/8ETXHdsi6UDMga+5a78IlZkrzSu+PO1OOEGYSl2hwm+C5MM7nGdWYvnqhaALRemlUSfNKLfIvCJdW4LdR7jjaVLNpiKfoGWnQCIoqMhwSAt5XD33oWYYMtOasvBXPyzL86PpDZ3l7YtY5KUKaIaFQKzBxbxYSyryjs8dAJ5vHs/S9SdctAAxpm9G2B7iXR6xbqRkQ+7U8x3SP1JDrD/S4LuSHqCcuf656p07Rhka6ITikNCIbRGkvyiwdL8wO1vZWwn0bpPi2az5aBhVBUT6eXAZ4CqSh1kPJhYCKwaBvey8F2Un8hVHZ6R9/WWlczJ14oTtTHsbHo3 DZ2d1CrG Nlec2Zs6Tk8srb+JGXHPDWEb/vQ/DfztRjeFcTCeF0KoBGAM2PUyK291b5nrfOUw75YjaaG8baRQKa21p3rUZaR0YnUDAvzNs9wfeJIygUXS2G5iLzIvOcJ282CHc+SKvvBPwF9gazQm1lk7Yje3lU8lxIaHTc/VQUggjI8xKb4x2li3d6NXY/+85X+bNIRwwJpQZscM4UCJOwzHLS/ccXANqehpFQ+WdXC6X+jqUPI3r0yn+TqsSWfPkr8QpiX7wrqqauyDNPUhuMg1ODkXDREueIfbEmhKmW/vBp2lKO/cLHPj1Czcmv4Ac4q0n4WpcuGxjHvrXamIcC6rE/tjiUbBJRrZAb0AjfnDgDL6N1Ncc1JmXe8EPE1iyaV/dPSkc35zK4ptx+ADzaiwP31jJ9JXWFwBzABuVdhk+rIT5YEAq5Aavrjef9SQb4g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Several functions internal to FS DAX use the following pattern when trying to obtain an unlocked entry: xas_for_each(&xas, entry, end_idx) { if (dax_is_locked(entry)) entry = get_unlocked_entry(&xas, 0); This is problematic because get_unlocked_entry() will get the next present entry in the range, and the next entry may not be locked. Therefore any processing of the original locked entry will be skipped. This can cause dax_layout_busy_page_range() to miss DMA-busy pages in the range, leading file systems to free blocks whilst DMA operations are ongoing which can lead to file system corruption. Instead callers from within a xas_for_each() loop should be waiting for the current entry to be unlocked without advancing the XArray state so a new function is introduced to wait. Also while we are here rename get_unlocked_entry() to get_next_unlocked_entry() to make it clear that it may advance the iterator state. Signed-off-by: Alistair Popple Reviewed-by: Dan Williams --- fs/dax.c | 50 +++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 9 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index b35f538..f5fdb43 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -206,7 +206,7 @@ static void dax_wake_entry(struct xa_state *xas, void *entry, * * Must be called with the i_pages lock held. */ -static void *get_unlocked_entry(struct xa_state *xas, unsigned int order) +static void *get_next_unlocked_entry(struct xa_state *xas, unsigned int order) { void *entry; struct wait_exceptional_entry_queue ewait; @@ -236,6 +236,37 @@ static void *get_unlocked_entry(struct xa_state *xas, unsigned int order) } /* + * Wait for the given entry to become unlocked. Caller must hold the i_pages + * lock and call either put_unlocked_entry() if it did not lock the entry or + * dax_unlock_entry() if it did. Returns an unlocked entry if still present. + */ +static void *wait_entry_unlocked_exclusive(struct xa_state *xas, void *entry) +{ + struct wait_exceptional_entry_queue ewait; + wait_queue_head_t *wq; + + init_wait(&ewait.wait); + ewait.wait.func = wake_exceptional_entry_func; + + while (unlikely(dax_is_locked(entry))) { + wq = dax_entry_waitqueue(xas, entry, &ewait.key); + prepare_to_wait_exclusive(wq, &ewait.wait, + TASK_UNINTERRUPTIBLE); + xas_pause(xas); + xas_unlock_irq(xas); + schedule(); + finish_wait(wq, &ewait.wait); + xas_lock_irq(xas); + entry = xas_load(xas); + } + + if (xa_is_internal(entry)) + return NULL; + + return entry; +} + +/* * The only thing keeping the address space around is the i_pages lock * (it's cycled in clear_inode() after removing the entries from i_pages) * After we call xas_unlock_irq(), we cannot touch xas->xa. @@ -250,7 +281,7 @@ static void wait_entry_unlocked(struct xa_state *xas, void *entry) wq = dax_entry_waitqueue(xas, entry, &ewait.key); /* - * Unlike get_unlocked_entry() there is no guarantee that this + * Unlike get_next_unlocked_entry() there is no guarantee that this * path ever successfully retrieves an unlocked entry before an * inode dies. Perform a non-exclusive wait in case this path * never successfully performs its own wake up. @@ -581,7 +612,7 @@ static void *grab_mapping_entry(struct xa_state *xas, retry: pmd_downgrade = false; xas_lock_irq(xas); - entry = get_unlocked_entry(xas, order); + entry = get_next_unlocked_entry(xas, order); if (entry) { if (dax_is_conflict(entry)) @@ -717,8 +748,7 @@ struct page *dax_layout_busy_page_range(struct address_space *mapping, xas_for_each(&xas, entry, end_idx) { if (WARN_ON_ONCE(!xa_is_value(entry))) continue; - if (unlikely(dax_is_locked(entry))) - entry = get_unlocked_entry(&xas, 0); + entry = wait_entry_unlocked_exclusive(&xas, entry); if (entry) page = dax_busy_page(entry); put_unlocked_entry(&xas, entry, WAKE_NEXT); @@ -751,7 +781,7 @@ static int __dax_invalidate_entry(struct address_space *mapping, void *entry; xas_lock_irq(&xas); - entry = get_unlocked_entry(&xas, 0); + entry = get_next_unlocked_entry(&xas, 0); if (!entry || WARN_ON_ONCE(!xa_is_value(entry))) goto out; if (!trunc && @@ -777,7 +807,9 @@ static int __dax_clear_dirty_range(struct address_space *mapping, xas_lock_irq(&xas); xas_for_each(&xas, entry, end) { - entry = get_unlocked_entry(&xas, 0); + entry = wait_entry_unlocked_exclusive(&xas, entry); + if (!entry) + continue; xas_clear_mark(&xas, PAGECACHE_TAG_DIRTY); xas_clear_mark(&xas, PAGECACHE_TAG_TOWRITE); put_unlocked_entry(&xas, entry, WAKE_NEXT); @@ -941,7 +973,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev, if (unlikely(dax_is_locked(entry))) { void *old_entry = entry; - entry = get_unlocked_entry(xas, 0); + entry = get_next_unlocked_entry(xas, 0); /* Entry got punched out / reallocated? */ if (!entry || WARN_ON_ONCE(!xa_is_value(entry))) @@ -1950,7 +1982,7 @@ dax_insert_pfn_mkwrite(struct vm_fault *vmf, pfn_t pfn, unsigned int order) vm_fault_t ret; xas_lock_irq(&xas); - entry = get_unlocked_entry(&xas, order); + entry = get_next_unlocked_entry(&xas, order); /* Did we race with someone splitting entry or so? */ if (!entry || dax_is_conflict(entry) || (order == 0 && !dax_is_pte_entry(entry))) { From patchwork Fri Feb 28 03:30:59 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995532 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 19595C282C1 for ; Fri, 28 Feb 2025 03:31:49 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9B14B6B0099; Thu, 27 Feb 2025 22:31:48 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9123D6B009A; Thu, 27 Feb 2025 22:31:48 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 71733280004; Thu, 27 Feb 2025 22:31:48 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 4DCB16B0099 for ; Thu, 27 Feb 2025 22:31:48 -0500 (EST) Received: from smtpin20.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 0EBF3120618 for ; Fri, 28 Feb 2025 03:31:48 +0000 (UTC) X-FDA: 83167929096.20.BFE6A68 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2068.outbound.protection.outlook.com [40.107.94.68]) by imf16.hostedemail.com (Postfix) with ESMTP id 4BFB218000D for ; Fri, 28 Feb 2025 03:31:45 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Yk+duUal; spf=pass (imf16.hostedemail.com: domain of apopple@nvidia.com designates 40.107.94.68 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713505; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8uq3O8kXTMfzu33bTQkJE6cujPJmeDvSmwlQjmGtbl0=; b=IXvGdklF1h68bZ3IllpPnSdmTi+merozaw24rYYcuyaW+st9HfAotRwq9JDoDI/Q0hSxjx 1JAFB8naokXWZcrG63DnStkv83ShQJHJzdxnxbYdFBIuoFGTZKnmQt/s9WgyEWBft1jBej PhX6GHUk6N0f9ttpd44s0ZMiBm/Uyj4= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713505; a=rsa-sha256; cv=pass; b=OZm1X7Uf9TAripNsa9GberkgPSKdzeBpfuamwRaYlYTjCO5AInphyPTNXpDNDGTcJD/h3w 2Ey/B+96sDTBncbsVX3UoLbPPXWj6V320U7T3DvFWFs+tR65eU/wdfQ7MA6EV0BZC9I8rV PtOAW1EhR0aJmaytxqeRJvVzX7Q9HW4= ARC-Authentication-Results: i=2; imf16.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Yk+duUal; spf=pass (imf16.hostedemail.com: domain of apopple@nvidia.com designates 40.107.94.68 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ri4RFeS6YGrkOEqZ175F/ZnHVeoSgpOVyJf60Vs1eQCM5vIUWbt9w4GC25GvWdUD0OGLRyWeWlm8/CQ7wQzTdfN0aPpNuCweuvRaeO8aOFTORMuanVozQInoN6edpn7GH9aI+LQ/xWwo/Z3g9wdDittd/jXlGLulyq17aI6eHTd9qJiEDO3TOk3O5Bgnls/Nz/+iaEGZschP8CyIVnxhWxggt6KbbTCd38VcHkHLE2AO5UrpNxQL7iqiM79igjejeO1Vc7XaRly+xTrQJBT9K6gSLvGgVJyArJU2sdqbUPyg7fQyj8d8CO3wCTNhhVaY29ZF+zNTlluHFxb1X0CFwg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8uq3O8kXTMfzu33bTQkJE6cujPJmeDvSmwlQjmGtbl0=; b=wV51G1VCZN6gmj9do3+M0WQokj1i0a/QOx+WKZ2lrdg+bMPkuME/pB7uJVtudMzlYGhhoMrUWMruBAhKD9Q/FB4GGoD3Uos6YVkYu5SUIxzC6kRDfAJwftqYQPPLWiduHY9qUnn4RXv+mm4Fqx+HxacPiTucISRWPtrSqxlQ+zR8/cyMcLsSDPB+e9n4wBmOXG9rciN7JO3W/fY2ot4Vl5/1grgmr2tg9pWMYm0OAAEi2ZBZnESu4eqUWaUQAKdEeecfkpWTsS6yDB8U+vWfcNWKP3K+X0k82761uXNzyzrztqD5+UkZqFxzaA4B9vcxAlXd+qMvrQzFuvCPHuWJlw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8uq3O8kXTMfzu33bTQkJE6cujPJmeDvSmwlQjmGtbl0=; b=Yk+duUalhvc//f2HV7QpbUR0hPa2RbU0oNOGKVD+OEuywv1esNpK1dglkGxU4dHkVAkSettk/tXl3SU2r/KMHhXnqHQnsls4ghSykwsBeg/1kA2eXIwv+Kvge5hh0T6het5809/NTo/V5FwSXUwnNkW+UzfHUyxjyhknoLDX8EYIb79bKfr7DIVkMBX1yJnZNibz2bixqBNWQbVNjst3r+qGds/HCGEyoYy/ICCTiQMU/+anWAW6+6mC7olPXx6Fsim+I11AemTHCZekwDMMphnB3ZaPkdW4px7bN5A3gPjRaUtkRZpdExHKxN8F1FvRNL83mYuix1fCZwMragN9BQ== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:31:42 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:31:42 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 04/20] fs/dax: Refactor wait for dax idle page Date: Fri, 28 Feb 2025 14:30:59 +1100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5P300CA0026.AUSP300.PROD.OUTLOOK.COM (2603:10c6:10:1ff::15) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 97a860d7-7e48-42f8-4808-08dd57a86aec X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: vLorP4ItNgA3291zW5oI5epydjJIy+IaHaPo42jTQNaCZ+1iKRBN5m47KKnt+yXUBGOy3awTkV4CU+h7GXLZKiCIK1PJDbkdZAkJMqg8hPhsS3eG7cGqEMmfJCtPk9wUMU755ND4lcl9siG2qbcuK93cNGr8r/Adj8sCr1cVNV8c4Gfkx1ujjIK4F3QmqO5y7k3MMeBZT30Jo2WjDXPEt/Tw8/2Gurw38zXqXOqEnGSk2l5EFVKKke641cp10BMi26Ayt/cHR1xD12Jwue496kzT2u9vDTvPZHu8NixeHCngtqsOupw7dov5eD/dJ6VCBTY0fTSfpywmmas3E2ea2NOzyHvz12JwMKN+xYrTOQOVn6zmZ8wRKRKwDsW11Y1YgGO27UnuCdB1vYqk9JSAZzFZc5l/6yESWPZ8KdmvAojI6g49A6N568kSpXsOGN0KVHHCXsNDNLK1fbMPpNkBWXGBux5CupV7jgrxDZh4kQ2RszWUYvIpPkFfuXS/z4YFHGDRfBCbVlrZgGTYiT5H/3a2NPAn2ZJ6hKvy3zIociHxKAvYjBOzwkoeOsbL/rDahdwHWsDDJVXOWDkrY/w/EBgmsRC/7JCjqoeFWbHKs+iC5J6TBbGZmoV6Z3AV9u+kLSEqfH3lPNROGAxDvIAjDROgOVXe+2/g2ctRe73XZuYoCws74XXR64DxYTKiCLt84+olB/+uP7NzLH/CBYJqNHvXfrj0rHX+7HjcaMkfOFmduGoqYBupB7UJJ+8I3jji3yZncv5HIu2UAfzHwGUrStoQYmuYuaEQU1rT1OUqNi7gkV4K1ztrg9eAmk/x08eXj7GIW9Y49DaW99SsmQIFVzMGy1F5ZkpCGJ9hH6Mnax+2+WbOUUyUetTn1yR2og64jJCOyNLi2Y9vVKCLcaKBzBPSWwweY0qnpPH4QFlMPCECoGtsEtbMxVvwQIA3UhXyn5sZWLnb+/fKLHE2P0Bi7t6AXu+Yi4X9iORFiGFEMxYdIVjWrmOqxRl7/bHMP3ISMSdd68LDug3BQ6sqYvxkwYa1Y0IKdu0Tu+g+I4u/9oayjj3ZtjuqPq5u/5HKBgrxqm4Lv5kUs4RtCJvBvICbSiFcr604otxJmUkVIlY5TM1E+k49v85gMSALGnTAKIBcNMEl9seYZfeiL+Pdy8XsCkPxHJA6JiWfFmk2zOAt7m7ZLEjR+oLy8RCQkS9ZzY8OQ32WANSW3ZzRpqnWzlVjpVM0QQf1KY4/2CZzA9+4cPgecQI6eU34N0UrRr1K1SAPPWTKz7UkM3fzxfKlG2ga0+RPSqP8v1KBrCYolo4mIsEEt2TSNdOp/6l8RgCsnZeHrFadkekelFkx8YY7WAEgJxjLuvcg6D3CByVbcjujvdPhNeRElSS8aULkiNKkPqQL X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 2vY8fFn24+bffGt6hCujBVLwAjdwboesZAKmydlk7AQ7xa6T8leOXdu2xBMPtpGJOzk/lbFohNNhtlvk8qeedBkxokkHrNw5jdX0QbOMjpOb78IIt7RjRl6NvBcJDMorS4JpoO/79ktV9uXblVBV3QJ9vhogSS9lJIETuIhOREDq3ysMIsfcdy2DHYgbEFrZkBpUVs7G1eqcblgcriGexB4hnGSUqgc/TDXxbdLaMuEbPnrSQez4AjOLMe6ML6L95mAXFNHS/8KEj06Ru4WJ2urGtq34sGJCdtWyFKMFx9Dz+FEY6T6/ju8i4Xj8lVg6RyVJg6v2aBhG/nfZXidtmySN/l1n48m4iddH1W5/9djr8DZyo5GS1qz94WtnzXAq1YU9cXcFBYWPKQOOFa4gpTxKXo5YQP4EYabPZDs4IsSKYIGQT7gG4N54emt9+4bdNgszJqZDK1M+z1VcRyYNn+Fa+TvFPIM7ExbV5o3xFH08QpEt3R3RnsJcKheIB3sOnZLQvcjSxXNywBeTE4cQCq+EEZk3f6f0KjJvC8rLsm44r/CLayl1kti7PhUOYCpk8rScTz4505ZAXT3Z/eGnPvm3SkxDYadtr4/kIleQmc8CiUeeHIe3W5bDOuDWIgbZcdr1RggidES334HC6aYrTThbN/DUZnHZJtVcZOC5yoAFBm+ris4uOJ/tfR43eCKDAHBvhSaay/KKiR2tPWl9oYR6Pvffp36ltSQWUs7Xi0SykuFL1V4DvyagK9VDrTqD3+5uQVcoSzz6HL7IAR19Eqss/SRpzffU0XhdbhvtwWJwEHb4v91e9KHQmQsrrYaHBzQ1/hmrbbVMsGqQdNUcZ+8Wx3DE0RetagW1DgS6qXa5Hm0NSn0QrZ0we9rmdX68YdSw0NlUBM2lhTmNoJe+Ie4l/XExiBDYwMA3o6XAypKdEZAEj7bNk+S7WLp2XftoolOFHJIMEF3PrqS3nK0FYlKDWaPwu6uknCaw+wogKzZkmo84xy8LhriQhcaOhAqNKS9XDDxJWyju8Hy7Erpph609r3mfKl1agCDGUJvPClRgDRARhytObaWWZaXQMuC2ZzAcOFu/66zJrCBo03F4eA0xBE0n2T0x9rZnlY2QL2632KOexqRoaCAOzRigiqNQv24P4WyPKxLse76HB4x+ZY0OaCoWrW+wzgUO/1v4J+hBd4dlSPgN55FM9RN5poUrrXeNpGXAhOx8HCcb0RzIulsFkcXClEz/KrAjFYXDCNBYlsZT+F08X6qmV+HLHzjn2ggEgYeu8rGQKuWy8ePZjf88nWbayz1ZDhb2iVz7DWuciDn/b9SJ0CApJ4cdkSBBww7DR8S67rnIVLjse2YhLhxwqFtnQqxtv1cacPOAETLJD1sMGpWuTnswRaINL/5ka5UM1og5dnn86MveSnye7y1Bg6yNhRpQ6a7OoJjK+CUbbFiViQJSteJT/OJBIAnXFNq4mFFIxLi8hq2GzH05eWNGFhzxlTof8ZkLb9/EHMnrQ75lw0yW0cm7AVfi9So3DdRiIRuYBpz396gmNsgqxt70Av+7JWCKCZEdPOJ6v6FZjGbXvuiMi2URdSaWGVxN X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 97a860d7-7e48-42f8-4808-08dd57a86aec X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:31:42.0361 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pM+G6CnOGDCEo9dW+4G+4/iYi59urSlaE+FY9Zg5KN2kNvBPR6aFKggjarxrH2T15x+vczGE5PzypnxGwMbkew== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Stat-Signature: xfcfnmqyt3ma4y7kj5qbriztgc34cqrw X-Rspamd-Queue-Id: 4BFB218000D X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740713505-488309 X-HE-Meta: U2FsdGVkX18AxyeMJZnKC8bCO0PmH7S7UBNU1EK2mhkqHa3wZVw60KwfAJKFQpMBwIIBar1G/MO4xkLxq1Nd2BxMgSjKLeb37+x76j8I/XmxIqwizyXmLy2IPXVTVSFCz8vCvVTB+xEnGHO32/JKWmOpQJX3zWt/qDC3SRhOS78stGWJ+OGkUCXS88Dtotim4MK22PfLBmdvFTLQLWspSo5WItLx+NrcECgmm9/GWdmKc/nkdFraelX6S49VuNCjpE22jFzviP3XvqXsJeIo8ksYA73wtX2RLjy0sGjwVlrkpB6lqaE3smhzL29vumlsZdk9Qi9QuuuJ22NzH0sG/UhvzclY5XjbvlK4AIxAfhcTBqt2TagNieQDMiE077FLKy0XX4y2A62H23OJrjzi3TF8EC5MB3d0hoXy8U4i8BLUzEaqsFRo3VKGwzWw/FMeN195XQpJ8s1Nk5ezmqbjWxBZqwvRXZngZYQdsE0GxO8OJqypXGYxKEF83KKLmd5HMro+pJRVDUNP6QUmOkDuvwtkFWvue9wX2nYzUjqD7bLbQxl9C/ivmwGznAbodh3DFbHV3ZMoO0kuNWPT11lflQ+aGE6CG8+Q5JEIChTD7N15wjbFtGQwi2u3OnW78TppuQaLfPFtxxLsQEzphG9FTpAEbA699BUBic2uwAxyylxf3Ev80hJvQBArMlkfeIWhg1cuCQdhmlHW6IW324VssZgjhuxYKUNNCvhJid2CLuefTwDjQcSoZErOAu2IE1juauDxcyY3c7OFYGtGKwESw3VzJhU2VCwP5cwy7cdVAcnTDk2FPST/UtMBx1777EZiiznDgn1VS7q12tf664cMUifiEY8IEFX5li+R12Y18l8yXKbtpeYgMOdhpiTDBk6Ncedi0cUvNLW2x2oAMV13n9UlBkJefHw4XzywOdbjKywGZLCEL9cTNjLb+uL1HnoecZwGMvqwhZV8noF0qXi mtt4ss6t TI2wtigC1XkOPFBr2eum+KvL7ZQ7tYnPNLkpEM93mAT7SxLiHb4ZZ8GjpwDcB1voDWts83Qi1r9/L7UxfKD/TO3vr4HBs7vdKrxasJ39cZDhksYU3hGh/DvTabwmIOZVGwf7uT8zO1nuBwD2jRUkrBOnhCYUSmFPZanmGpNmE+MBYKgStWPHtnvXtOsh4LLw9LjqnlKaQq4yWd2GoBLlpANElwdbOfy4Bz8e3KWWTQmMG04v5E9YBOYMPKXWEbuEz6D8CVqkzV3eSCBgkkGjXLfV3UE1ppvCm9qbTe9HLwqIee270Qno0ZaO9BZZlIdJ5A0pT5wOAzteaWtH4YlDE3eszqYcJhZbs101LNQwh+HfT141u8zaB/OXR/ieSscb/wTTihQ7ZgNRTXCT2SPuHIpVH3BQjyC9IhNNTTW3vbXTOIDjbpF+O8WPd/g== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: A FS DAX page is considered idle when its refcount drops to one. This is currently open-coded in all file systems supporting FS DAX. Move the idle detection to a common function to make future changes easier. Signed-off-by: Alistair Popple Reviewed-by: Jan Kara Reviewed-by: Christoph Hellwig Reviewed-by: Dan Williams Acked-by: Theodore Ts'o --- fs/ext4/inode.c | 5 +---- fs/fuse/dax.c | 4 +--- fs/xfs/xfs_inode.c | 4 +--- include/linux/dax.h | 8 ++++++++ 4 files changed, 11 insertions(+), 10 deletions(-) diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 7c54ae5..cc1acb1 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -3922,10 +3922,7 @@ int ext4_break_layouts(struct inode *inode) if (!page) return 0; - error = ___wait_var_event(&page->_refcount, - atomic_read(&page->_refcount) == 1, - TASK_INTERRUPTIBLE, 0, 0, - ext4_wait_dax_page(inode)); + error = dax_wait_page_idle(page, ext4_wait_dax_page, inode); } while (error == 0); return error; diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c index b7f805d..bf6faa3 100644 --- a/fs/fuse/dax.c +++ b/fs/fuse/dax.c @@ -677,9 +677,7 @@ static int __fuse_dax_break_layouts(struct inode *inode, bool *retry, return 0; *retry = true; - return ___wait_var_event(&page->_refcount, - atomic_read(&page->_refcount) == 1, TASK_INTERRUPTIBLE, - 0, 0, fuse_wait_dax_page(inode)); + return dax_wait_page_idle(page, fuse_wait_dax_page, inode); } int fuse_dax_break_layouts(struct inode *inode, u64 dmap_start, diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index b1f9f15..1b5613d 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -3020,9 +3020,7 @@ xfs_break_dax_layouts( return 0; *retry = true; - return ___wait_var_event(&page->_refcount, - atomic_read(&page->_refcount) == 1, TASK_INTERRUPTIBLE, - 0, 0, xfs_wait_dax_page(inode)); + return dax_wait_page_idle(page, xfs_wait_dax_page, inode); } int diff --git a/include/linux/dax.h b/include/linux/dax.h index df41a00..9b1ce98 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -207,6 +207,14 @@ int dax_zero_range(struct inode *inode, loff_t pos, loff_t len, bool *did_zero, int dax_truncate_page(struct inode *inode, loff_t pos, bool *did_zero, const struct iomap_ops *ops); +static inline int dax_wait_page_idle(struct page *page, + void (cb)(struct inode *), + struct inode *inode) +{ + return ___wait_var_event(page, page_ref_count(page) == 1, + TASK_INTERRUPTIBLE, 0, 0, cb(inode)); +} + #if IS_ENABLED(CONFIG_DAX) int dax_read_lock(void); void dax_read_unlock(int id); From patchwork Fri Feb 28 03:31:00 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995533 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 8E256C282C1 for ; Fri, 28 Feb 2025 03:31:53 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 174A16B009B; Thu, 27 Feb 2025 22:31:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 0FF5C280004; Thu, 27 Feb 2025 22:31:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id E1C9A6B009D; Thu, 27 Feb 2025 22:31:52 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id B7DBA6B009B for ; Thu, 27 Feb 2025 22:31:52 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 6AADFB3FE8 for ; Fri, 28 Feb 2025 03:31:52 +0000 (UTC) X-FDA: 83167929264.15.B6FF632 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2042.outbound.protection.outlook.com [40.107.94.42]) by imf15.hostedemail.com (Postfix) with ESMTP id 9300FA0007 for ; Fri, 28 Feb 2025 03:31:49 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=SiWqxJcF; spf=pass (imf15.hostedemail.com: domain of apopple@nvidia.com designates 40.107.94.42 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713509; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AZjp39mH9a7emt54vkN8geCoCp5VBgqHEdOopdg44hE=; b=Ds1uJoLKsB3EgY7IIgNzKuz7yCa0rnGsRkJdDyx7nAmU/ZSJv8gVdGtx7ec7fpGD3Vza+0 Sxyeiytbvu2C3q7ukMHvXNH8DaTE5PZiorHjpm5H11zdQXQ/wMmm/+nnuKxLqjizvLvnK+ xWKvdIDOphLHozTxOIhXTGe4lz71VuA= ARC-Authentication-Results: i=2; imf15.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=SiWqxJcF; spf=pass (imf15.hostedemail.com: domain of apopple@nvidia.com designates 40.107.94.42 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713509; a=rsa-sha256; cv=pass; b=5DA1kTzhwC5Wnx45cvQU4L2uWbjOoo9sm21P6fo6GiUpfZl0H4PAJekgwekBh96iqbFBXa cC5eM9xtNVw+WcOGreszoLujxm7m08zcMxYONUcXLWJ3a5pCKmbWYzVLpf4Lgjdgge7k3P VY0V1m374r7MfJpqg1F9tKu0lMn9ipM= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=EmQMfIjh+N55liiz82JaGx86lsdd8zIZQcsD1fJA9lys5tGE+6wcRag2lV+eQWbg95zpKNELfsDF54L1pgMTbu16hVgo2bDYlEnKU/n5QGaMvNLuAzqmpzO7OPDWn6yvNwJ47bBuXur8h2E3sjJ31DkZfmHB293gc2kCKjiPbserX9uYXmiRmDv5BUZYBY7hJMMF3eATxMQH5eJY2mZ1pBuSBhswAd7cEkCJNM5hIQ1tM+J/lw3P/vGlH+PvYJtBkuN+vqP4toAgTTkSRlzHiESMn8ZXgWjibBSNiWxilQ4j5UuhbbOCxcxjKjDG7g/PyZI7tngNA1dMiAXCbQ3J5w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=AZjp39mH9a7emt54vkN8geCoCp5VBgqHEdOopdg44hE=; b=CNKwazJBYtgGJLxeoArSp/RjJfJtizd8/VBJShLj5NIQ0kCRi9fJRZjC/nveF/SgBCa1wjKrKWrtMaEBdt1WO9AplNNTq2E179djDFISnGKx1vnErXY6QVrfiHmWOlp3P7tts30ETiBwMsFdvYOYnXLIHPtv2n3Lu3Vrl4E0womL6RHrfMlqa5WJA/x49D7F8JwU0wPpeSpTatEUU4JUxlyteVXKBQxg3kuIKGJCXtmBAdb3ABrAlTKQV70Y+FpfQ7ar1p0ZVl+TmECMNIKx2Y8/Qi3w088/Ln4Lq0lFgXcta4dIP+ZKEgQdbkF8oPGXSca2FI4f1vXBjvG34gSsJA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=AZjp39mH9a7emt54vkN8geCoCp5VBgqHEdOopdg44hE=; b=SiWqxJcFGzeRo/kDL+TR1QSIDjroMF7qkVQZDOhBJgwNiCJXmIW1iZ5/X9mpHcnYpeOSuuce/wU7fYQHO+QOxtu8t30G0okHiXiX6gNwJBm6siTqBJDNurx7LQOs6S/IBJUE6ueSoTAkjFd15bmRZU0E4zceCgvLa2BU6KD7wVxne5aIIFRipwQ/jqhRQi9qLqFBC74/IJ+Ra1PWgWtRAWE/7rMivUPmBuirOqslN56XnXowyPm7ttesXKKz7OlYPrnZTTkLZkKJBJQSM//XJnMTatPbV3CBgyrkpbKWJ5c0DUSFB6L9EWrW0ryNUdy2Sturdvp//6fI7gK54shW0Q== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:31:46 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:31:46 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 05/20] fs/dax: Create a common implementation to break DAX layouts Date: Fri, 28 Feb 2025 14:31:00 +1100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5P282CA0059.AUSP282.PROD.OUTLOOK.COM (2603:10c6:10:20a::19) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 6fd4a13c-2b2a-4015-8a32-08dd57a86d7c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: WAs061k8MhsthjQfMI+nKOKBVyVZACYxHLXDvCqhblEewSyIEzwSIVVx0BhCpprGnt9QqQiTRe37cO9Y7fQFOIZlvXcXUGB0FinxrxokGiHdRe+nkplLgA5p8ZGOMEQRyyk49uzC9ul5Nfckb6O2sm/QZNSSLO6plFIiXDKmvLIczpWHSOnPcyeSvRrq0xo55KsID/AaskWfJMmA5lUg+QFEc4ir0WScd15hoVBTE5UIW/ZjXn95YjLH9t4SOrNhflgg4thRw411eiUShQHgLh2Irop0BJuNm/SuoOihyClpQmyvzBfBFgBtcC8S84gz/3BWXoOUqdmnab0yucJOcUPh50drtYuFkzBNyDtufsR4QvXHaCmmCTm1PlhtpWhQ55yVE1ryRnPnwukJd6F4xsYZPAkfJSWQI0IDN1Qt2vxnNsQ0HlOuCD0azDiLGRIkMf9NaBaBFp4FKvH7gm15jJXhdKDE8bc7EVEa0P9G/M0/4cna75HHCL/M5r2V3DQ5TxrqZPSwkTu/QeafQHUEJmrmVHhadw4KjOZvRdV6oeMYAKKGzzFfq7hTCqKC4PDcAqSbUuxCwOM4tKNk5APWiks76mdiMauJPKqiHM12tlb4ExSf60UQoiszYn7OF+1ys1FyFARKM1aGqfK4jv5FbyqlY2V5soAgwyO4oKwz9fRf6b9PBkyXV/Qr6boRVSPZRP/v850yPhOPs5uHg7oMoXrzryGLgQfpwAoVh20lrdBtSmbH4jEt0dwekSmKe/WZqDSKw8uUmA0EhrsO/laJuQ0VBeYhKJ9JOopS0DQu9z+6ovDVe99DsC/I4uDbix/Zunxp1Sfkt+YOe1cDgh6c3a37TfZxM8ig2n92qTxhvECKW4rFacbG9rTei8wMk1e/gvEcLMtjxkuwKW+WnayGWCCWrK6xfMEGjo/h2Gw216dB+jSjZGPCvIhUjUWPwezjWgciPa23PjE2yz6LPDkLaqvF5aX2YpzbpBWTM+SQCwV68amcrQJGkWA8lSPsS4tRYH/U/GKBF3Ox0pTT6l/NN5Aj+VKJ9veL0NDB9DteF55mfK/e44UranswM7s1yx+HAi3tykeJVKdc8kBk6/hsFZO/TFdxKD/aP9SFDmZHJUM630es3K1cRFxcN/TseQHjtWmJjWIQioj0LjKfYvHFUycjjtA817WgroIP+eHm9XQoNviuYBGWaLsHY7YY7Ke8P7MVKzFC8LbEQxvqX6d/1CbSkrVERUMUVC/4IMwJp8XOA5KZ3e4WukrUxI//j5PpBj+fLP3tOgS92nQ4iI1bkNCLuon8r3RPqf2TZLTrgIsX3WrZ6XGn82lObzlmEehbFZ4kAP54IcjzKWdgqMzbs/KEpdfeJyeSNjd9ZeOMOAaryPFQQA62M6kHbRErjVuA X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 4kLcM0zHZSj1N2SP5r1cFCyqRl2FBjF4vMmRhbXYYr359HyCTqdY5u9px1OyKk2HLvhwCtC2ds1dLJKKap8u7LtRc2cbzHpTUyG7051pN/d8a9PexCU+lgrfBiWf2Is/z7oLItLLcGzCRaadXXDpQq5TQ37TndttkgqARqxYiZ/uLc3o702KykIJ3eV/TxQ0TEOBLtLy3U4g26KWeHrIN3NgVirJvAHarhAzBcWd5vQNKltf6Rm+WrJWUGyKr5GxkMemqJtrgaREZLYGkQ0aN/8DN1D4ycDXxxhSLdgRiC8MA0a71R5Xnk8dBLUtOhgJSnxoWm+xqPZql+1YXBvi8LYU4eT+lrcCOrEYbul0uNlsLxcadZOSlptrUpxU4TY+bGf1mOtKgHunmhH9xjDim6+ZTVRDFvKaLxBZBYEgZ3fIZ609DJ74b28CdpD04OALcbSzAcjIxrTluxjYNifJMPh6gbHdfixtGafNNYQtkoS19KkWAoZ/UZS9HceljnJZi4s3IqjzwwCQW/jDKeLyoadbqYlbv3a3osxyBStNvilvGe9lzWKbdlTQCkodAgWeDpZj1l/U99Gq6a/OmJ3Ca+SjqbLXvKoM/nrN6cJs5fk7vUKc2M0XdGQ5waVRBMImVSIWcutAkU2rpWswBww99U+wecMeBI36fhEbjJ+VtttB01m0SuYexTfKxyXpXU6Jhg8kchacgtouUoiiKsBVyqLPpT5KKwLiL0fgNXJEPOsk/bUt5Q31/7xsnhn8hOvONAD1esR1M+7j50UdS3YwE2hD8oZ0ESEBVqH4VGmo2VafYTDb7uZVFmpVqJQUDKgC+kKQ23YzLe+jjjdT1YIT8NXbphSbtczKbxQnl3YCbWymfH+gqkzKHHCX+DCbCGajLmHqSVaLYpnSodDpkZCDfmZWza4ImV1POVG2a6PKWpc9dXbQ5iarX3Q44M0QAtwyXukIhms6siZLaemNVYM5o3WYFKsuk62hO/YxBaByqVzigDm4mml8RnbHtyqyhl5o+KM/NA7mKHoUh9hbhX5tN6VmYdSeHj93V4WZU7W6oL1Dc5xoHW4b1XlS42K3NNlCSPFQgUVPEn/XGU+fE+Z2AfDa0NmF/AoS/n6kN5xjuyi/gRljvcqqDTFCkAk6Ew54rkEtuYQGcP1lcvQm7nzw0PT/v5ZPoW+0I4RzyhZkcWIAukmqrW/6OIvjZ7uCuTSsO6cBwo7vxsWEZVkLhN/8FfgD4+pXfc0e6uCmJOy0F4uEmdewCDvPyN9AbeG8DOl7uMNZ0BNIP3lO/jnCvLQyxVrjvdQ8FScBza8eklBJubFhpxVeOW4B5vDeGxBWzFlA+nh4P9kJC/Y28d3xG/Eo7pV205pJABshgzuRpK7dTPXkKf44+x3De/IBcI8DojlcgFgOyme48w9lMLDQcT6HDyXN5EXbc2iUQGg2ziJQn7Rdsfx8jntF0SbcBpPVrPRmtW+o9W3MoDf7QYkdoX6GpMGsB499RIl9FeWl2H20ShzO5mRdT+KCXksb5Qq1pNJ36WUyeGD+zy5abr6gXn6s1vE/Uwx/9MsJPt61s+59B94kqbXHN7M/cfLMqh29hUTX X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6fd4a13c-2b2a-4015-8a32-08dd57a86d7c X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:31:46.2462 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: d4vmBq/SgMpwoa00+mL7irn/zE/7UAXagxbart1ho+HHmktpZgIkCV6X5IKSNOvZ/nuly9/wXvE77LXZRE2kgA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Stat-Signature: j33bju7oc9d95af5xt3zhhuua6pn91pg X-Rspamd-Queue-Id: 9300FA0007 X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1740713509-595249 X-HE-Meta: U2FsdGVkX1/9v4WaSAqYiROtg4KAquryL9RC08WhOS7jcxLLuAS2yL2qVG5BWtpNAwkcsbwQhV/DpbL+T0sw4uwO3uChZPXXXknstShJc2G/jL9Zejis3GxvqBPjsqms159BbnaTk5yjUxpoHr2pZRkcydLVWiyJR8t6xmkhpcrYm/bclZFTK53zpSP6Jg6g9nSD325TDrfjM4OCqpUIsMJmesPb1mDHfHcK91GRqBhQ1Oux1LjsajKuag6Pr1y6mIEli1LznNFq4T7aEmvqQBmx4PisfZ8gsOu/fq4CpVOmu9hOFl8Dz9Ci1zxbWA7VKy6A37F3FxnRukz4WNUIIsw33CI09li0GjVlNKkmDCYcFGPZDJWw69O3oIU0Ul1vEe5xLwECzcWxMrbVjz434GjP/F593QjDDz6ezcdylMGZ7moWSm9D4X21kE/sccVsSlMYTlSvomoF3Is5N/ljbx9WMypLeRBqn+zZhrY9JplHJHpJojIk4MMfzxuSSB5uC2JLV4SryM/onP2WWZCSWUCebHmZgCwCbE7aaDMdt5vKs4pQCzVdfhPLzWSbu3Ck/eliP3ml9ATLGmDzfg7e5X0k1QdnB8kF+lrbD6QxqlZBTJAnzAZDlid7/s2UNHk8z+ybnGF6XrDLs7jRiSnDnXA/T0N8jSf8ZLCu8JgBSA5EbrvAtYqR6WbZ6bajUFtVbu/KWV8uLH6Lig/fcAuu4ZKNOusmBcBvnVQIsFOX09PvylV5K9cFnQcG1yWSL1HW00AkP9R+wK0SfVeQdYM410kExAmcbHMmLbRqc6A9v7l8QsuGSLCavhtd5mwIQwQUs6MtlORpBUGgHJwsyZ3VozaED3EbWrL09HdbhiGcDAHwZbUudgYZ4GHUOkQAmzL44sFC1Z8f/XyiUopOvTfsjtCrtz9Qfw3JHCvqtUDuhe8zlPOf9LxI1Ehg94r6S9HM6EV1GtS8QIvfOuvQ5QB 0kVbqqWh I9kzjGN9XhDhW4JIAh94mYPrSKxFtTC86zH/Q68NpResT29jzQ7U67vAPa2r0Fw3UlD6G6E1NVl2vix+zny3DBce4G9E/E2JL7mhz/YH2MHuPU+bzk1SX63sWe/2zoIZeKwv8FQRBiNS+6aWgJIVm0KwTWPPq4ny15YO/p2ed2C7o9VAO8WNUbeKhG25mwxPPg02Oe+9oLyxJZ4eEpNc+PmaD9LncAKhPyD+d+5osqbS9N4tumbPcoAfr3cKg52Mo38YdtKp09NNTdUzoQC7AS/Fc5AK8DlNeyt7kipRyMXniJEvcaUe3HdY/kYTUii4GkCFnTkK7AVVOCXOJ9GisDbydA5SLh69AHp75gEMnPj5dueRRByoWLcA7IU+g/EgJm2whjdfFE7vdwy0qR7ddYqf/OFdTkAEXrhPzJbNCZ8MHiT3WCB94jJ2wJg== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Prior to freeing a block file systems supporting FS DAX must check that the associated pages are both unmapped from user-space and not undergoing DMA or other access from eg. get_user_pages(). This is achieved by unmapping the file range and scanning the FS DAX page-cache to see if any pages within the mapping have an elevated refcount. This is done using two functions - dax_layout_busy_page_range() which returns a page to wait for the refcount to become idle on. Rather than open-code this introduce a common implementation to both unmap and wait for the page to become idle. Signed-off-by: Alistair Popple Reviewed-by: Dan Williams --- Changes for v7: - Fix smatch warning, also reported by Dan and Darrick - Make sure xfs_break_layouts() can return -ERESTARTSYS, reported by Darrick - Use common definition of dax_page_is_idle() - Removed misplaced hunk changing madvise - Renamed dax_break_mapping() to dax_break_layout() suggested by Dan - Fix now unused variables in ext4 Changes for v5: - Don't wait for idle pages on non-DAX mappings Changes for v4: - Fixed some build breakage due to missing symbol exports reported by John Hubbard (thanks!). --- fs/dax.c | 33 +++++++++++++++++++++++++++++++++ fs/ext4/inode.c | 13 +------------ fs/fuse/dax.c | 27 +++------------------------ fs/xfs/xfs_inode.c | 26 +++++++------------------- fs/xfs/xfs_inode.h | 2 +- include/linux/dax.h | 23 ++++++++++++++++++----- 6 files changed, 63 insertions(+), 61 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index f5fdb43..f1945aa 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -846,6 +846,39 @@ int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index) return ret; } +static int wait_page_idle(struct page *page, + void (cb)(struct inode *), + struct inode *inode) +{ + return ___wait_var_event(page, dax_page_is_idle(page), + TASK_INTERRUPTIBLE, 0, 0, cb(inode)); +} + +/* + * Unmaps the inode and waits for any DMA to complete prior to deleting the + * DAX mapping entries for the range. + */ +int dax_break_layout(struct inode *inode, loff_t start, loff_t end, + void (cb)(struct inode *)) +{ + struct page *page; + int error = 0; + + if (!dax_mapping(inode->i_mapping)) + return 0; + + do { + page = dax_layout_busy_page_range(inode->i_mapping, start, end); + if (!page) + break; + + error = wait_page_idle(page, cb, inode); + } while (error == 0); + + return error; +} +EXPORT_SYMBOL_GPL(dax_break_layout); + /* * Invalidate DAX entry if it is clean. */ diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index cc1acb1..2342bac 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -3911,21 +3911,10 @@ static void ext4_wait_dax_page(struct inode *inode) int ext4_break_layouts(struct inode *inode) { - struct page *page; - int error; - if (WARN_ON_ONCE(!rwsem_is_locked(&inode->i_mapping->invalidate_lock))) return -EINVAL; - do { - page = dax_layout_busy_page(inode->i_mapping); - if (!page) - return 0; - - error = dax_wait_page_idle(page, ext4_wait_dax_page, inode); - } while (error == 0); - - return error; + return dax_break_layout_inode(inode, ext4_wait_dax_page); } /* diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c index bf6faa3..0502bf3 100644 --- a/fs/fuse/dax.c +++ b/fs/fuse/dax.c @@ -666,33 +666,12 @@ static void fuse_wait_dax_page(struct inode *inode) filemap_invalidate_lock(inode->i_mapping); } -/* Should be called with mapping->invalidate_lock held exclusively */ -static int __fuse_dax_break_layouts(struct inode *inode, bool *retry, - loff_t start, loff_t end) -{ - struct page *page; - - page = dax_layout_busy_page_range(inode->i_mapping, start, end); - if (!page) - return 0; - - *retry = true; - return dax_wait_page_idle(page, fuse_wait_dax_page, inode); -} - +/* Should be called with mapping->invalidate_lock held exclusively. */ int fuse_dax_break_layouts(struct inode *inode, u64 dmap_start, u64 dmap_end) { - bool retry; - int ret; - - do { - retry = false; - ret = __fuse_dax_break_layouts(inode, &retry, dmap_start, - dmap_end); - } while (ret == 0 && retry); - - return ret; + return dax_break_layout(inode, dmap_start, dmap_end, + fuse_wait_dax_page); } ssize_t fuse_dax_read_iter(struct kiocb *iocb, struct iov_iter *to) diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 1b5613d..d4f07e0 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -2735,21 +2735,17 @@ xfs_mmaplock_two_inodes_and_break_dax_layout( struct xfs_inode *ip2) { int error; - bool retry; struct page *page; if (ip1->i_ino > ip2->i_ino) swap(ip1, ip2); again: - retry = false; /* Lock the first inode */ xfs_ilock(ip1, XFS_MMAPLOCK_EXCL); - error = xfs_break_dax_layouts(VFS_I(ip1), &retry); - if (error || retry) { + error = xfs_break_dax_layouts(VFS_I(ip1)); + if (error) { xfs_iunlock(ip1, XFS_MMAPLOCK_EXCL); - if (error == 0 && retry) - goto again; return error; } @@ -2764,7 +2760,7 @@ xfs_mmaplock_two_inodes_and_break_dax_layout( * for this nested lock case. */ page = dax_layout_busy_page(VFS_I(ip2)->i_mapping); - if (page && page_ref_count(page) != 1) { + if (!dax_page_is_idle(page)) { xfs_iunlock(ip2, XFS_MMAPLOCK_EXCL); xfs_iunlock(ip1, XFS_MMAPLOCK_EXCL); goto again; @@ -3008,19 +3004,11 @@ xfs_wait_dax_page( int xfs_break_dax_layouts( - struct inode *inode, - bool *retry) + struct inode *inode) { - struct page *page; - xfs_assert_ilocked(XFS_I(inode), XFS_MMAPLOCK_EXCL); - page = dax_layout_busy_page(inode->i_mapping); - if (!page) - return 0; - - *retry = true; - return dax_wait_page_idle(page, xfs_wait_dax_page, inode); + return dax_break_layout_inode(inode, xfs_wait_dax_page); } int @@ -3038,8 +3026,8 @@ xfs_break_layouts( retry = false; switch (reason) { case BREAK_UNMAP: - error = xfs_break_dax_layouts(inode, &retry); - if (error || retry) + error = xfs_break_dax_layouts(inode); + if (error) break; fallthrough; case BREAK_WRITE: diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index c08093a..123dfa9 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -603,7 +603,7 @@ xfs_itruncate_extents( return xfs_itruncate_extents_flags(tpp, ip, whichfork, new_size, 0); } -int xfs_break_dax_layouts(struct inode *inode, bool *retry); +int xfs_break_dax_layouts(struct inode *inode); int xfs_break_layouts(struct inode *inode, uint *iolock, enum layout_break_reason reason); diff --git a/include/linux/dax.h b/include/linux/dax.h index 9b1ce98..a6b277f 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -207,12 +207,9 @@ int dax_zero_range(struct inode *inode, loff_t pos, loff_t len, bool *did_zero, int dax_truncate_page(struct inode *inode, loff_t pos, bool *did_zero, const struct iomap_ops *ops); -static inline int dax_wait_page_idle(struct page *page, - void (cb)(struct inode *), - struct inode *inode) +static inline bool dax_page_is_idle(struct page *page) { - return ___wait_var_event(page, page_ref_count(page) == 1, - TASK_INTERRUPTIBLE, 0, 0, cb(inode)); + return page && page_ref_count(page) == 1; } #if IS_ENABLED(CONFIG_DAX) @@ -228,6 +225,15 @@ static inline void dax_read_unlock(int id) { } #endif /* CONFIG_DAX */ + +#if !IS_ENABLED(CONFIG_FS_DAX) +static inline int __must_check dax_break_layout(struct inode *inode, + loff_t start, loff_t end, void (cb)(struct inode *)) +{ + return 0; +} +#endif + bool dax_alive(struct dax_device *dax_dev); void *dax_get_private(struct dax_device *dax_dev); long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, @@ -251,6 +257,13 @@ vm_fault_t dax_finish_sync_fault(struct vm_fault *vmf, int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index); int dax_invalidate_mapping_entry_sync(struct address_space *mapping, pgoff_t index); +int __must_check dax_break_layout(struct inode *inode, loff_t start, + loff_t end, void (cb)(struct inode *)); +static inline int __must_check dax_break_layout_inode(struct inode *inode, + void (cb)(struct inode *)) +{ + return dax_break_layout(inode, 0, LLONG_MAX, cb); +} int dax_dedupe_file_range_compare(struct inode *src, loff_t srcoff, struct inode *dest, loff_t destoff, loff_t len, bool *is_same, From patchwork Fri Feb 28 03:31:01 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995534 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 63982C19F32 for ; Fri, 28 Feb 2025 03:32:00 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id EB8E16B009D; Thu, 27 Feb 2025 22:31:59 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id E427A6B009E; Thu, 27 Feb 2025 22:31:59 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C21546B009F; Thu, 27 Feb 2025 22:31:59 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id A1BE86B009D for ; Thu, 27 Feb 2025 22:31:59 -0500 (EST) Received: from smtpin19.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 169121CC1F9 for ; Fri, 28 Feb 2025 03:31:59 +0000 (UTC) X-FDA: 83167929558.19.5B9C501 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2042.outbound.protection.outlook.com [40.107.94.42]) by imf16.hostedemail.com (Postfix) with ESMTP id 4FD80180006 for ; Fri, 28 Feb 2025 03:31:56 +0000 (UTC) Authentication-Results: imf16.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=lOFlUXkx; spf=pass (imf16.hostedemail.com: domain of apopple@nvidia.com designates 40.107.94.42 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713516; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=w7VcP7ZEvUthj5PzZYbtWmdjUZ5HeQZyLdoZRveFFxw=; b=fpuS/h78OyQbNfMtvyHSIQagbAvJtgXOFA0+/bh6xDjQvaBEPkf/gdwyw29x+F1g74uei7 Sy4tWrlHlMQ3Y0XHdCpL8ElRa0cYVMnbDTZZQp9UDGqeD8QCpjztaKHQIKijBUht3sLnRB elMdhgrCydxfQr2fQ19uRVI+9SMKK5o= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713516; a=rsa-sha256; cv=pass; b=PunCU5uNsSF/Iucj6NJRYNi5Uir0oujfDqXFhXWAgSJ35OXYJPnGfy//1/Mv2IGeH+5ZbQ drPFGih03j9PhpzqmImcrfQEQUSVQDFPQCtG8hlRKY10cXv5nXaB9FjCLwJQPBpdY3dhtc 1yoR7D6i4mGNnOdwb+2oFY2R+AplvWI= ARC-Authentication-Results: i=2; imf16.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=lOFlUXkx; spf=pass (imf16.hostedemail.com: domain of apopple@nvidia.com designates 40.107.94.42 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=CUJ7haef2wyK+CmE2PUbv5z+FHQJv5ZHtBddMq9KNRE952cBGLP9EtggixEdSRwMJlMbnFgzlkuVymvISgfE7wWP0Tdr5RpEhbHQx0pe9bRDAV+YySyBF6lWi5jnpiCcGBEpXsT1gjNFOALBjG3C9+xhRKSlRy26fPewqmU3Bh0DWjnbvd4Q6RSW0iY3DHWR2S0v87h0NkNJM7+tLAennNT9VcFmmytG4Q6NkmAnAu1ETkZDr8hCRk/qebbe3GQ1uH6dp2TDck0I/mfCHagPrntjFPlAX5GKDNMWbUTGwRAUx7AVNY9L095YEQjCVriKbB1vcw9wVFRDxshRzqWb0A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=w7VcP7ZEvUthj5PzZYbtWmdjUZ5HeQZyLdoZRveFFxw=; b=g9JtfrLkJE+SUqP3gBB+AYwCC1lwxs+vdLXutvrbHWJWWskHX8wSO6jm40t+HIqlyLfc0ulmGK77I9Eje/ouqHz45d/qMJH1kY+6EnsdgcQxnhWLpjZdvBwJVEYV3UTYenFE5cOqsSy0/r1ckMDHz56DfXDCN2wRJEO/kYXH+pAbZL6khvexqi61AB71QkzgYQC8lxGqnvWm6ZW9103ao644xBC15XTCnuQzXUr7+pUcvxoO3PUEDfeOBOTbm9wXkbl+d0a3TnPayiYJtbCpTmf5Nk2qDa5oaKZ5QgeOArngZg4qPdKPxKHXvUMyuqaasra1RUy0uzl5AXPt2+LCeA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=w7VcP7ZEvUthj5PzZYbtWmdjUZ5HeQZyLdoZRveFFxw=; b=lOFlUXkxx47GXPFIzLFIn/TJnK7aZCiEbhxaeNgkxAnRrk28tiqXDNj6Z2VPAFo3P2CAJwVXEBE1GElwNG3fvQdn0n52cqZ/o4WFLHGX2XNqqh85/QchUZUClXRPrJAgdmFUluOTyZPNtvM8x796goxlehbQB/suCU0Bdh3f9HnOf7gymS29Ry0Ns7RGkLoPUetFduOW88KfOwOukEgtrr+EUKUwpC0qAwLv9broBQbYK1TgrfwM+G7CRIlV+LjXq7Po3ksX6w6Tz4+6vJZxN52EB7PQ9rdxrO+S+o0NL14Rgvnh0bijAL+N6dRu4dov7Ko0w+QGwxueOEoRexj0qw== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:31:51 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:31:51 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 06/20] fs/dax: Always remove DAX page-cache entries when breaking layouts Date: Fri, 28 Feb 2025 14:31:01 +1100 Message-ID: <3be6115eaaa8d28fee37fcba3287be4f226a7d24.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5P300CA0054.AUSP300.PROD.OUTLOOK.COM (2603:10c6:10:1fe::16) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: be7cbd1f-7a62-40a8-6ab4-08dd57a87039 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: RTKB4wjE0Fhw3O89jzimQv7WwhLw6HsEo9ahZlXaimWTDmT5PSQx9Lgceu3jRnTV7zAObEMaR9C22wvJx59J6WDYO0n8ihDDlUY938u8QeOceqy30rbgjPcl2ScJwdj1JLX/7PtK6Ml7iuB/jFpO+qBp5JUfwFw75ij0SC1so8EkPntJynbMRPjCgXJMz/nnCXehUMYwoZD+OBZve5oITfM8GpDGgk13zzfNDdFfnJLxp/Y/HCjYwh8/TEyBbMHDRCBI2JbHJsaE6CFxZJOuU1BXlMrGljOZQMjOezyS6ytYCARlGMen+K+7GABCJeviZKzYuxYGd6kveW+fSKZyXuXg1Sv5/Au41BlzyX9s/b5o3f0Tac4x+d8/AUDgLIk6UVfxNLg65K5WTQsozHHnqwRE5xGqUinXjLSITp9QpJSPriSBNwAMJRdSbsh1ubv5rr8oOb71rU38UxDAzoWSz8Ry3CVIsBP5dKVxwHENxIpa4dtM1yUqU+xUDcqVQzC+chzhZiCvCocdOHQxyk+bILbGH2RTNr2Ogvp8pTw9s49KGz9tFU+dykva9fb1Z8zypPTt+0DHCnzZ+PAcTlZ8/8xWQE0xzE7CsFrVN13ld76rwPxs+0OPqcdL/LhV3Yng0YRlT55ZWgq0j700SSMh+V62ftYjQLx3Tgbgmjzc37SvqpMWWLuA/HrlweU/ZJjR5jWJOFUXg+/ebH0lQW0JafKE0fBeqblFUczhj5lH6IwPtTImJgvMIy1t6PsuSRKqDuedQQHWFqr9UdizumY4oLyAH36lOx4d1IfaOyjc6vz9UnYu+m3z5Rduy58y9aFD5pxKyJ0u5Jh6YRAgQiIIHsL++r5q8ckjN0mvvPTdSBELdYOn8KMiOpzkh94wJy1Gykv9P1GOmAUxTnh5EApzLuPKSOu7YslanvYCYF6t6YBw1ardeOXgtg3aPVPWHx4sCJ3ZjkTGJooPSwd8TLLRHf14u4oiRrPIWe6k3BHCtk/oGagF44a1/ocMUbz94qEdpA0IH4VRkPpVg74r9kKGd5QpxbUp5zW5mYNGVmzZaMuHtsS6l2hsKTjlYQY6MzeN+8DMQbaNYVD553L4wvvQskPxXzAoJgIF1K86EzOSP7LPx+Wdw5iDmOgx+vsYZrsXsJjpDhiW+Zc6oFQtm0iAwD8Rybdr7yGR55RNdYEIMrL4iXa7gibJgdRIcwrXc9rjOYmfDpGPo14ZQkPOjJEvIw8FaKxSu4DFt78h5uT4Iu0aLXQAb7sLu7n+UyPnFyGF5JRNgyQRnco5+SVw4lkOl7epS5+wjlWDLgk/ZOBGdWiMKn10EnRy6q4PaHyTXMX69sHX+zVAps15lBrQltx9sURW4L3sEIzilbLGjM8IG4XYD0BfFaGlHl4yr14vBmSC X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: UxyMp3sRNR7f+rL6JCiP9HtnSo8DA8ZLbQYDFlVGlmq8E0cc7xSGqq2PnvRsSc9Fgr7hPNNIJKjTuLBruxNGuVuam4lAwfyOM+fGjLk1EQUfKAZJ0Eu8/3f2Sux4pieI7FGJswPwb9Ow4OMBQhuDg0OSwPwiZuHQ/ba6Ujoe+n58CxToYsWYn9QMxO0WUeUQiWdynCQSN02FBTySx04Y5okMu9eTmjmuDIzB2agDTqGpc6pnHylM6iqTZRnZloggcRm4PNtEO1eQefK6TXjsbKteSee2oqjKOArYIA5pi8jc/DmQnCcvTDwvKhHL/NDa6HPsaCFeBrcLdq1wezKbwtBfEjkYy3KrqZdVLD9d14pxam3TP7JF3nk/7byMVQLU8tPhwk0aoC9IPlVqwWBSM/7o6EcdShUJnnzXHOyz4kssHx9Zp0TdgFE3ndD9gMZ+9/NFJ7vWneqWiG+cWVr1WwBhPOhinD3QT0Nwilqn5be1LKUsX1bbme512YEX2wteM9FuEJvxoDT5oYaUHBUTOXLm+OMLXkmjDUFxeIe2m0fSRpn1/632VNrfLXEN9C4fs3AA/k08iNgR7HjmbW8zZSOQHFYh2SEi05x5G+kwI+g5XHeC3FKa6exuA2IoGofqwxxeYvuwdLayKbd+djlIxNWmU93vgX9srNJQt9Me/3wIFKkkjfvqVqwIcLIX+LeHlu1sFkwqmGfFb42s2r1BzB6D6gXxwCC5L0rIro1z328xMTvOwrV9iX6tB+f2FRN9Zg5aAbUl6lkhNkE5AquDAku/+6qrm1y0QELeVZctyMlKaxsUsBjstwl+s/lfAMD4PtVTqrLMSo/dePeyW/PCpQXhoqHtxsNP7r9byZeDlg8zMcY6sQMgDjxhpag9rsF41Ct7VhUTm6CzU0QqfdHE27l3R2s1HnnOmr1HKhb6r/G7zl7lffZGDb8nuoyKLBHjYcKGRtYYAvVkyJ0cG+wOT9DHV/RaUOsjHM9tkkdCRbtAurAg3rWadfNbTtkiTp4yajhmIL8bibUImN7dSw/zCXFyOJGcX4d4MF+L4Nk3NoOhIJDUXcsxH2nrhvWkMMJAaJsHUUN6AO9wQLVdf+5Q3WALSxl0ypOE1Rmjd9XiNWM8ohmTz/W6UzFooUJumzx1BbQU1qjNamzgcgXQ/5uHMd+erC7zveW8g5iUA0dvJ1qO4RDDVJl9Z53b9U2JN9uLoZ/EGGB5SFd34iZezWLJJLwSQ9AWny1AfWH+EaSQmjYKZN9CKtx8OCWT6PGPsHQXN2rqVDFmK9XKUVRi4CadIhPaiXUTzXRwNdL8JAPCRzVgOCTeqgTM3FPAE87mhY5CWpM5CYSYyZLZlHOkqIFnYhSrO2V9vAhCVkgqyZe1jE50XSNk/8mQ8ku/ALyTStgByREB2gtCm/92A8JyWlspTyUGG/0bt5w7RUvGeIit4isvPYLGw8ExeU6HGIVp83rTnq8m4h5D9iznUqR+lmnXeOWr/SC2GU8dsqKD6vVLhWALi9+XHa8lvA18GrUfHQrfXz6S69C5sbNgXWP6lFI/oYOz4Zc53FM0OXejWfQwYHW/meHOGUZAGWc+045YEGTb X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: be7cbd1f-7a62-40a8-6ab4-08dd57a87039 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:31:50.9806 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: xBULIaOTPNu7KKZRyiSSIA5wzP5shjVoqfT/Zd1BFijlOfYxaSTKUXK2Kc+r54YP+EdJNxCAmSJgluCRSbnfXQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Server: rspam02 X-Stat-Signature: c4b94z3p6sigqdz6amjkksymmb1roofy X-Rspamd-Queue-Id: 4FD80180006 X-Rspam-User: X-HE-Tag: 1740713516-767076 X-HE-Meta: U2FsdGVkX1/08/w6h4heo57E7WftWVMv/WKqYSlCr4H2P9SEZDH5iNzgu+k4uFysELzdDn9HG003PuRxnPFn+7SMYjR1E49ejJxNRAZhgTAENAU6Fb3qyO1qKqFULr7rfuSlXOufXIhB6cgIbhAYJwUkOc+1F3Gt8Ryx467JHONvVtAXdFQjh/f7YejUh7iBR41JZw4LFv5gUl5RiGJlW/z1gss4C+IcNwjzvSPKc77a6fmHNan/OR5n9PhHzm1veRKCJTmJypUGqlSru5RIrPUqI9yiiBdMgePK8vEfx0DLPVHwUp/gDy7UVSJ04KhzX4n2mkyce6j1OmpTtLtRemorYK4NDH+vxyC1UoRM/H9YhRbvooNlw48B+wR3NkxObKY6YjA6otzqXKUPJ12xVHFmxxTzfYR5y5ScgUwNKD+ygvgzXIRu9CVplEbmnUtlykWbu9nUHd8lSywwynGHdVdconrkq5pfeZBEmgII5rxLoBfCHZcs/wsN6dawH40l0vfmYotR75Q291nClyh9JrAToApYCjIVo+1ew52zkOMve0EHcOq+fY0o5V0l8hiJbOGqXOE8EB0DaIOppzipmUv52C6QPqs+nKEK7WQU67SpxEoNXiEdtV5l6HkoBmjEW+msDYO5bI5cNSAEMTjoobEo/G+dvLbU0XHUElMbXbNWzmvCqmnMlmz1doLCjhfoVdvBiweiaYp4I3wbcGAtuPdmoILAadmg/ck8pA8flrKVeHoTi2D6NoGOccC8Pl1rkWFGn6iE82sQiZwxnbSpBkkmpJjLjLVLaV2TmBrgyGXQxZcc9j8Eopql2/qyWqp5lgGJPE7Hal7xuk4xR6PmS8FfZL3wdEKGJQUpPNXVUYQw3nCPcv07MAkrXXautX1X6pAa7BzDlpc+qBqaC/ntZJyH4bKGBY97MzpptjBu6U01AIhq0AVj4LAer1MBx+/uWvBsoeLGeiBKIEowcJp pCnaGMrF L00Z1h77mUrCDDEhb+4vWldnl8S3IvTLuuy74KVe/ky1bZ3aoLNcZEPoMZckAc5GHTHrsvuYdTJudJdHAwFdj2Fxy10Jb/G5X/gItF6njXq6AdjiHxytL/5CgVsbymjuPY42PASQ8jSgM0Z/6KJWAcH1h29CKdR9hGjkku7de/BBcxvAZiS4+nTRECvcdNhzl66SJ7chzJROImJqpSuQ9w0y502Q1E0ElIba/UddVIrTzX93ihJmVQ+ZnfNYfQBgXfcWs6g2YY/d2YsxmccqTKu3oiJg5ilU5uHyfuIAplhu0PQocelfDPaOgqipldmC/oV2yq/m1+egrN5gc7RI2gtHGgHziJrN+U+MzZcoNYbitW5ad+mxKneVEDBRIpG31rmPBliW6ghmJO3aDcssGKb7HQlUe2KUQ3QJYXIwhkt6AsySNuXhW2rGOOw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Prior to any truncation operations file systems call dax_break_mapping() to ensure pages in the range are not under going DMA. Later DAX page-cache entries will be removed by truncate_folio_batch_exceptionals() in the generic page-cache code. However this makes it possible for folios to be removed from the page-cache even though they are still DMA busy if the file-system hasn't called dax_break_mapping(). It also means they can never be waited on in future because FS DAX will lose track of them once the page-cache entry has been deleted. Instead it is better to delete the FS DAX entry when the file-system calls dax_break_mapping() as part of it's truncate operation. This ensures only idle pages can be removed from the FS DAX page-cache and makes it easy to detect if a file-system hasn't called dax_break_mapping() prior to a truncate operation. Signed-off-by: Alistair Popple Reviewed-by: Dan Williams --- Changes for v7: - s/dax_break_mapping/dax_break_layout/ suggested by Dan. - Rework dax_break_mapping() to take a NULL callback for NOWAIT behaviour as suggested by Dan. --- fs/dax.c | 40 ++++++++++++++++++++++++++++++++++++++++ fs/xfs/xfs_inode.c | 5 ++--- include/linux/dax.h | 2 ++ mm/truncate.c | 16 +++++++++++++++- 4 files changed, 59 insertions(+), 4 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index f1945aa..14fbe51 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -846,6 +846,36 @@ int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index) return ret; } +void dax_delete_mapping_range(struct address_space *mapping, + loff_t start, loff_t end) +{ + void *entry; + pgoff_t start_idx = start >> PAGE_SHIFT; + pgoff_t end_idx; + XA_STATE(xas, &mapping->i_pages, start_idx); + + /* If end == LLONG_MAX, all pages from start to till end of file */ + if (end == LLONG_MAX) + end_idx = ULONG_MAX; + else + end_idx = end >> PAGE_SHIFT; + + xas_lock_irq(&xas); + xas_for_each(&xas, entry, end_idx) { + if (!xa_is_value(entry)) + continue; + entry = wait_entry_unlocked_exclusive(&xas, entry); + if (!entry) + continue; + dax_disassociate_entry(entry, mapping, true); + xas_store(&xas, NULL); + mapping->nrpages -= 1UL << dax_entry_order(entry); + put_unlocked_entry(&xas, entry, WAKE_ALL); + } + xas_unlock_irq(&xas); +} +EXPORT_SYMBOL_GPL(dax_delete_mapping_range); + static int wait_page_idle(struct page *page, void (cb)(struct inode *), struct inode *inode) @@ -857,6 +887,9 @@ static int wait_page_idle(struct page *page, /* * Unmaps the inode and waits for any DMA to complete prior to deleting the * DAX mapping entries for the range. + * + * For NOWAIT behavior, pass @cb as NULL to early-exit on first found + * busy page */ int dax_break_layout(struct inode *inode, loff_t start, loff_t end, void (cb)(struct inode *)) @@ -871,10 +904,17 @@ int dax_break_layout(struct inode *inode, loff_t start, loff_t end, page = dax_layout_busy_page_range(inode->i_mapping, start, end); if (!page) break; + if (!cb) { + error = -ERESTARTSYS; + break; + } error = wait_page_idle(page, cb, inode); } while (error == 0); + if (!page) + dax_delete_mapping_range(inode->i_mapping, start, end); + return error; } EXPORT_SYMBOL_GPL(dax_break_layout); diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index d4f07e0..8008337 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -2735,7 +2735,6 @@ xfs_mmaplock_two_inodes_and_break_dax_layout( struct xfs_inode *ip2) { int error; - struct page *page; if (ip1->i_ino > ip2->i_ino) swap(ip1, ip2); @@ -2759,8 +2758,8 @@ xfs_mmaplock_two_inodes_and_break_dax_layout( * need to unlock & lock the XFS_MMAPLOCK_EXCL which is not suitable * for this nested lock case. */ - page = dax_layout_busy_page(VFS_I(ip2)->i_mapping); - if (!dax_page_is_idle(page)) { + error = dax_break_layout(VFS_I(ip2), 0, -1, NULL); + if (error) { xfs_iunlock(ip2, XFS_MMAPLOCK_EXCL); xfs_iunlock(ip1, XFS_MMAPLOCK_EXCL); goto again; diff --git a/include/linux/dax.h b/include/linux/dax.h index a6b277f..2fbb262 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -255,6 +255,8 @@ vm_fault_t dax_iomap_fault(struct vm_fault *vmf, unsigned int order, vm_fault_t dax_finish_sync_fault(struct vm_fault *vmf, unsigned int order, pfn_t pfn); int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index); +void dax_delete_mapping_range(struct address_space *mapping, + loff_t start, loff_t end); int dax_invalidate_mapping_entry_sync(struct address_space *mapping, pgoff_t index); int __must_check dax_break_layout(struct inode *inode, loff_t start, diff --git a/mm/truncate.c b/mm/truncate.c index 0994817..031d0be 100644 --- a/mm/truncate.c +++ b/mm/truncate.c @@ -78,8 +78,22 @@ static void truncate_folio_batch_exceptionals(struct address_space *mapping, if (dax_mapping(mapping)) { for (i = j; i < nr; i++) { - if (xa_is_value(fbatch->folios[i])) + if (xa_is_value(fbatch->folios[i])) { + /* + * File systems should already have called + * dax_break_layout_entry() to remove all DAX + * entries while holding a lock to prevent + * establishing new entries. Therefore we + * shouldn't find any here. + */ + WARN_ON_ONCE(1); + + /* + * Delete the mapping so truncate_pagecache() + * doesn't loop forever. + */ dax_delete_mapping_entry(mapping, indices[i]); + } } goto out; } From patchwork Fri Feb 28 03:31:02 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995542 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 5B237C19F32 for ; Fri, 28 Feb 2025 03:34:11 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E6E3928000C; Thu, 27 Feb 2025 22:34:10 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DF674280004; Thu, 27 Feb 2025 22:34:10 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id BD3DE28000C; Thu, 27 Feb 2025 22:34:10 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0014.hostedemail.com [216.40.44.14]) by kanga.kvack.org (Postfix) with ESMTP id 9D2AC280004 for ; Thu, 27 Feb 2025 22:34:10 -0500 (EST) Received: from smtpin16.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay06.hostedemail.com (Postfix) with ESMTP id 496BFB1542 for ; Fri, 28 Feb 2025 03:34:10 +0000 (UTC) X-FDA: 83167935060.16.06CD138 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2055.outbound.protection.outlook.com [40.107.237.55]) by imf06.hostedemail.com (Postfix) with ESMTP id 666B618000B for ; Fri, 28 Feb 2025 03:34:07 +0000 (UTC) Authentication-Results: imf06.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=TsNOptE3; spf=pass (imf06.hostedemail.com: domain of apopple@nvidia.com designates 40.107.237.55 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713647; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=9tvVmUd8MTa61zBY6TluXmKwkBsI5wuLIycCgCcsc2Q=; b=p90Frf/Kwrga4STW8YKLSInqOzj8a8MgvRqvhWXzTEJO5urYafurVhFobgV6ryFbSnDawx Mmol+l4j9V6BkhIwB+jLgve78crdXyDQGQ6SWNtSoOfpgqQzlH32h+BIoUUS6YqitI/ssZ 2bwHDDn39OgsCghnWuplT4JQX358ito= ARC-Authentication-Results: i=2; imf06.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=TsNOptE3; spf=pass (imf06.hostedemail.com: domain of apopple@nvidia.com designates 40.107.237.55 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713647; a=rsa-sha256; cv=pass; b=7i/+swJcPYve8ZaJyeIKL5znoLVs42hkDS1t7QV2gYvEJy4jEhsGAmUomoam5H06F+V5r1 pFiiuPEhUGQ1a0uegZC8xsKZC+nDOye8NJwsKI3YWF58rNsaYSGhfbluwJjmhagquzHcLv LBjbJhbyux12LLoGDyhcdR6NC2lQfaY= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=wMZrHcF4OKXpm/vE939fXQ0MOakSji0qF8a7N7NG6Mkef8D65VlNRTt1KUeDA8iv0ckdA7kv2A0xDRGCip1U++ae1G+2JrYO7qCeAFryZWMZWvZ6uRXjIAoQXWNkIGKjLUdGiDHyP8TW9KiOLfuPODdiQRwwTOBC6waRJ110eL97YlglCXCVouUBN0rzCArBzIPOahhxw6ZVmeFupYTfcWqscdBfe1kO21jyCws93oHdew1aQqaQ9KuylRv41zVBKrlX0IdxYSfu0nZqPoTSlmu6exsmU5sDprburzyyR+WNTbHb8v+Dr14Ia3xP57qLegkxYuU9FY1uK+znIqcE3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=9tvVmUd8MTa61zBY6TluXmKwkBsI5wuLIycCgCcsc2Q=; b=sRUqUkwmqAXOU+DgwpkhnqeEJyo66la4UkcWj+c1RNT9F+JQD0eW0XMPJHzeyDt+cbzo7me0tE99IhDl3pxi0yZofaKHABEtFNHz7YOfBSM/6qGOs0EKaVIJ5krkww39GcQmWmB2hEPN+j/aZefM3unZHYRTKLexGhp1x8XJ4bmucGG6RP70myETXSkaWbNUziXbLJwWUuVwBdtI3H/R9BeHFDHhuxL1/2ZK6EQIO+iWoP1q1V6wzQYpdWeReG7Q3iJ9bx5BubBF4mGl7ICb7kEipuBrPU2vrbidafRkHOyUHBhGORAHJSdPCdtd/JgfHEvIvYGHRLPpOsWrd9eGWw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=9tvVmUd8MTa61zBY6TluXmKwkBsI5wuLIycCgCcsc2Q=; b=TsNOptE3eO1VQJam6ltyTEpFD59TN8Yck4G4nF5zDfrRcVabqx9lRzQZ4uKLwpCGJHYj2dFGwPdK4EBPaSt+BY1zIWM4KJ39gH/2Yr06meUEr1hdtQKxi/u72hh/bij4nO0QAA+Gx2l+d5YxbyClA+wNadzI2CZEPxDd4YR4ksxG8lwKr1MeFSxWxZPVDOyURXEnCLtxtEbp87+B/gXQZZeQPg9GIuUSJqU9vw15odDfEFBHkm5u6Bs+VUYzwA80EyVBFK/EwchedeUiqyYCr+jZyPCC/BVEVHTv540vVLd+iHEQFx0xq1LRNIL4gFKDBmNEHZ6CxtKFKWdXS+2IkQ== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:31:55 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:31:55 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 07/20] fs/dax: Ensure all pages are idle prior to filesystem unmount Date: Fri, 28 Feb 2025 14:31:02 +1100 Message-ID: <2d3cf575bbd095084993154be2f0aa7442e5cd28.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5P282CA0086.AUSP282.PROD.OUTLOOK.COM (2603:10c6:10:201::7) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 2c36f48f-c702-4d74-0d01-08dd57a8730a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: cyix02rE7G8bJNw6rd1yKc1SBIqX8b46Xco1Lc1/fUtQyxd4OFwtbwUALnr17gfkr0qAg/uCe92qgT8443ksSv3WGWEQFNVPNoxvQrr0YFNiTLDCAZPIFYjHI42I7Zgnpsyw/0yOfAADTYJ1Bvga3A3Dy9nh7g8DW8n9D4R12FPNQvMQVAUTWyu1Y/ds6A4kKPZ7+MbthUDXMbAZ2drJY5ZcNHWgKz6cVhD9rjh3XWX5DRzECR2YV9ulsMyop13X2nnT5vwFxsaiqr2K1rifryYu1ijujS5vK0IxmF+B7yClR4MdRQUrlcnKHbWY1jxOgv+dFRWumRPx2AoJgSHJKijsvKbhHTI3BDJ+aXSwJU2nd5RauH+SCLjJSSI87GjyAXniR1cQD5XIcC979jDE/W/akmTL9extFSJxWBiLws4Lk9auTDLfqLsHPCdhwZXIffdzNnXMLDLyVQKf8ZcT7TqYV6rsZpNhkkxYaQHsbWt88Or28nXLDc/pWhrmp71iIAMJYyR/kpXw8UMCEMWyA7C5S1PQFnMuQRfn2Ru2VyJYDLktaZytuNTDoGmAtKsz/tf4H8zVrIGVvh6h9mNfXtETTk1pvbziqWfNb2POQ2Oup/qxpOwokD05OzxyLo+ETecEWh8fOU0+6Lm90DkVv5hDZx9fCsY/eEmQuRvGnFdVR/jnIXDGvV2pYAddlB9pwm9ZBe3YDHCxkEXEPZqdUWBVcYmZhJ4Gn6lYOZ+PO8wKgsVcVXRrr4xX5/Pv2+O3wmFxNPd0ViF1UQKFj0ly9moQcoHFkU/HZVsMnt2MAjUnbiTLqAxN3U6rF76FiIKKw6XY4KbEw5fbmUGUSutf60BrGkEiB87cDXZm81or5PsIU/LJz+0tPYJNliYmYtdt2pERBiLuovXy+Sym5m6SO0VHSyhWgRu5oU/LydB0rqwa9marjfXlvT4wmwwrXwctnkWt4UzjaCxRntpUcX3nj9VGZMh+dg+n8UQyuUjryqui4YqnjD2f2cWZlNcDwhniTawsykhwuYvNzCWj7mdy014ONBHY5nW74cmWWf8vhtvqCd+JEinhShjPcUrWkshvos7FHgHTKdmxhsyWUEkRHuIE3VSO+R+b9w423z4GdUbMsFP4eSout7zvLZURzuQmFPyRT/XVCFBz1yvv80vY9rST9rJ3p0FJegRP74DztMLRkfBa3ttxJi17fMzGEXfjgtweR/fjoivIZmVKXJCSTb8JS7Kuc2I8Puj2k2Ycsi/WjuYxaEpDtJKHvFYDXCBDtpDnvK3qiIWUztDnCLf7huXqLC3FRpIQvTaNdthRnABDt8LSIXndtaw0nGgMY3+2PBBVpn9e+g0ofdJDzVPpJdNpak0mu88NbQtFsV/eLXO6+5mofva8oBljkqj8X3Rq X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: RV5kI1JWw2fVPL5HgAmkOArKonRODPpMxS7Ou4sKyQm/mQfReLfyr1dQ5XQa409zZGGdSsThmmCCSaYYSG4YRTAoYyt4u1njW79AG7NMdzpRI2b+7VIr2gBhgyyY7mam+JWMnz6WwC6uPXd+/5D/a4jiNBV3g9Qpz/WYKEhSKsv7XOuPXQt8FWwlzx0KeBdxlC505eWKXq8o5N1EP/H79/JJMsisP03zsvZ/ABqEhszOLbHDnPWOT+uNZ5JO9QXP5VBRCDQlYDhKc1ibPk2qHcSjAPe52XF3iMpjfuTuWksZMveRWWMFnMXoGwapgWDuJbnIC32cABJWQswVceitCHvH7sRo1EMjT1eB5jjagvd9nkJ74J48iR6ek5hafnqxhDW/AwzjFAYOqDp5VGqVyJQmqJcpZW5iDaBdmvzLJ1PqW1jNZV6CjEJHTbQRkYLKr5aLmNW4lTLa9vDNgnWpxvP3XvzbAFRyfDbqtcrutmP5Mif3VE2oH6JnLAuehT1wnELaf9NzNwmytX4sAtgBDlHt2VKw+L2uKqluCZPIKHThf8J24hG/mTYjsGI206WT5W7t6LNic2hIJHrvLgKpcTTPQiNGtrAvSVuW84JdxFr4paA7v4AMq4WVjdDCrYzyeBm7LdKABHyFXB5cb/6FV04wmUTuTduLN1a54c6nDJmdMq9OIwdnyivwB6QHgY6NeUv5nAExKEzc7cD5G3DRNBiBIJWGdomH9pQelAQcRaz6MWRuVJbPgYR2jNTeIAEfnzx495DiFPxYf2VMHAV+1YqaNC6FeZndAQE4FFMLAzF63DX1jKJNMPCgsWaw+SbBvcaswmp5LpYosVjOoUZJV7KG0erU3SBzObJ1K9lDmwRpOe5dKe8XmDOVNRfQl0fGTxpeX3zqUCwAkZcD8cwDjRU0pLo/blxW9G9KJrh8P2UOaVqHNFj56RVLnplnbUgHaASATu7XMWP5OP+0mnNNsB5YoQU9dRWq16SPgUf60EFygiOmLQc4Ign88lj/BxLu7alBcQ4LUANH/dYTLxGnognaBj1BQeULyPSplXDWCP2VNIKs/gh/YHNHJonqNHGy6vgBCOB6oIiE6A7uyEayuXzTLZGmVUX5qXtrTJNpaic6alKeSQC9afUaZIbB7blnJrx80i3nQjXPh0fFFNNe7MH5PCZmApJCNrRb1oWy1wbbnioUkgfaj4fTYQxjZRqukQGrXupXkzQuDXZOgHQW21buGkPIEAVg6mhrHzKfuGHNnA22v53gwRErDTLy+Ddln/pEZrhuCQ9FaZIFm0lPsK73wFv1ztOV7EG3RWDjuPDqibJVv2kjymEPc6Y2VwtPuCpHRIIRWaVGEhO4IUC1XyDiHyub6hy494s+kHT/L/js7Qmf5i/4E7mFZO0DTiYzYcimrRqtrzqa3BAYBlOBD2e0Kkk1miESB57nnJ5lZ871/2fJkDcmTlOkfUmOXQoE6q8K1ueLQk67he4txypmF2dS3mIti+L24LUqzR7W8Raj18zASWk21gay+UPUwz8JP4bOn+wysb3pfEi2h9Rm6yMSAzV+FOI2OY6RhAdgqXV1C4ZxNrfts7nSCNUspu2L X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 2c36f48f-c702-4d74-0d01-08dd57a8730a X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:31:55.6644 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: oz9QN5zQjgQJNIeDOnAv9YmTdqb9ZsuP9CzG0uI9z6pzaVrZeKRMIQqX9/6sXLRy0xWYxedXmP3FRLqttzij4w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspam-User: X-Stat-Signature: 9w7j74cbmq74oryjf1qgx4grfq5f6xwd X-Rspamd-Queue-Id: 666B618000B X-Rspamd-Server: rspam07 X-HE-Tag: 1740713647-714575 X-HE-Meta: U2FsdGVkX1+ilHf98M6NMHubdkFMYdWGaqLfM8DuFOjcsZAv8uhoN0/2Rx4h9B5GgN5VYIvMAnLNrLUI33btpK2f2d9+TTKx3iuN2mI2YkiGXR23UT8XBFz42BHcnSPainznAkH03ziWJru1Tl4mcKh2Lqz/wfkRyFWTOGIWdX71C1SUPFOTp4/WjsKyJa1DlNYx57G9IrjeZcYD8mWKcUNz0CxwLtwPy1BnSb1a6u9vEQv4+VUvkU2cdOvjgp3vBxJdzppCfFVTbsJx9ErDASufl4wDjZRgpoXZ097Uwag1VR2YyyiJ6/FzkVUpc2x1XcYalaK61nJFIzbxMc1ZA6cZRb2Ew20zVBDpIH4UvrwnYclD3Ju0doIr2na5SK9IZq/BYzbCkyknEna6/jPmxcVJczB1G5UCh9K7vmQgjMHpBBnBDIMODvYcdw8YzxIf1OHvvjNC9NLyfO4oKMcCuMSLqcYXSN2vMYO1/UHvjrUANmsfts75/gSiWZ1PR6d76gBIAhLCa6zOfe3MZRgYa00HJsTZzkdJUKyPpo0ttCkcggxSoHZZ/bU7pTFVJ3kP8JKPN/ycHmOokzjTnMSmISDxPMT3jrX7wkIbnheUznb/NXo7W9loOThvnJUcdUFaWHux2w2cJfY3i0bFSn2RHH3q4gj+wmIf6NCECa9x3+UMeLT2xGKP2PM4pd5UpEGHY0PUZhXUy7ebKdigNQqBchuvQh1A/XwTxGY2b4jwPntenK1Nt0G11SoGXaHIhqngbdjwAERheFBmNsI9XqOwqSP95uTPW4fFNl2nyFxLIzKnihGRhVmaD4XTX7cF/F79MV3xUiWoJ3eEL9bH3DbFQhHUzAT2TTGz7sQUciRD7keZGWUAsyAhSxoFHhxZOE/hangrTIaB6yQWrZI2DVcWpAwi4betlBtdJB06DauAHqfsxYPYvUtHEHLvZ3K6Xa/glqtTRfcPF7Kx+AVTYGC 4yYlrmKC +nrkJiUEDHJ2iB3gxGRwk1L6q/qY45j5YQD9rm30sNaZSDZulsTuwITqJwlKPY6iFS8pF9dgE7loU7daPC3Q4PQrkgsbGUMcXfRCwctf65YmzGqDRxVU6lGsSeG37SfsVEwsuSZ+5Qte69+jzUcsslxePBMbKUhUUkYbabfSpEb8A7RBhRsd24ZH2AlmeMCtJHV5pCv1NcUzFW8GAD+f5dyQrnmu8rbRVmx1CLu/5t7xwWIjUbqNdz6gEtQSJ+MvEqZsLn5A/d0EfPY2tScNyRMdHQFc/7oatBYOS2gyRCdoDFVcWwce3gHCXJKTf9mPhrM6W/7vgIrlWFO33CgPdVVsCi2hwHDQldVI3oj4Gb6dIDpM4busU+BKPmyaw4/r7nyoLBzramu9SOw3WJmKtsDXwTkCySx24NO4J X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: File systems call dax_break_mapping() prior to reallocating file system blocks to ensure the page is not undergoing any DMA or other accesses. Generally this is needed when a file is truncated to ensure that if a block is reallocated nothing is writing to it. However filesystems currently don't call this when an FS DAX inode is evicted. This can cause problems when the file system is unmounted as a page can continue to be under going DMA or other remote access after unmount. This means if the file system is remounted any truncate or other operation which requires the underlying file system block to be freed will not wait for the remote access to complete. Therefore a busy block may be reallocated to a new file leading to corruption. Signed-off-by: Alistair Popple --- Changes for v7: - Don't take locks during inode eviction as suggested by Darrick and therefore remove the callback for dax_break_mapping_uninterruptible(). - Use common definition of dax_page_is_idle(). - Fixed smatch suggestion in dax_break_mapping_uninterruptible(). - Rename dax_break_mapping_uninterruptible() to dax_break_layout_final() as suggested by Dan. Changes for v5: - Don't wait for pages to be idle in non-DAX mappings --- fs/dax.c | 27 +++++++++++++++++++++++++++ fs/ext4/inode.c | 2 ++ fs/xfs/xfs_super.c | 12 ++++++++++++ include/linux/dax.h | 5 +++++ 4 files changed, 46 insertions(+) diff --git a/fs/dax.c b/fs/dax.c index 14fbe51..bc538ba 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -884,6 +884,13 @@ static int wait_page_idle(struct page *page, TASK_INTERRUPTIBLE, 0, 0, cb(inode)); } +static void wait_page_idle_uninterruptible(struct page *page, + struct inode *inode) +{ + ___wait_var_event(page, dax_page_is_idle(page), + TASK_UNINTERRUPTIBLE, 0, 0, schedule()); +} + /* * Unmaps the inode and waits for any DMA to complete prior to deleting the * DAX mapping entries for the range. @@ -919,6 +926,26 @@ int dax_break_layout(struct inode *inode, loff_t start, loff_t end, } EXPORT_SYMBOL_GPL(dax_break_layout); +void dax_break_layout_final(struct inode *inode) +{ + struct page *page; + + if (!dax_mapping(inode->i_mapping)) + return; + + do { + page = dax_layout_busy_page_range(inode->i_mapping, 0, + LLONG_MAX); + if (!page) + break; + + wait_page_idle_uninterruptible(page, inode); + } while (true); + + dax_delete_mapping_range(inode->i_mapping, 0, LLONG_MAX); +} +EXPORT_SYMBOL_GPL(dax_break_layout_final); + /* * Invalidate DAX entry if it is clean. */ diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index 2342bac..3cc8da6 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -181,6 +181,8 @@ void ext4_evict_inode(struct inode *inode) trace_ext4_evict_inode(inode); + dax_break_layout_final(inode); + if (EXT4_I(inode)->i_flags & EXT4_EA_INODE_FL) ext4_evict_ea_inode(inode); if (inode->i_nlink) { diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index d92d7a0..22abe0e 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -751,6 +751,17 @@ xfs_fs_drop_inode( return generic_drop_inode(inode); } +STATIC void +xfs_fs_evict_inode( + struct inode *inode) +{ + if (IS_DAX(inode)) + dax_break_layout_final(inode); + + truncate_inode_pages_final(&inode->i_data); + clear_inode(inode); +} + static void xfs_mount_free( struct xfs_mount *mp) @@ -1215,6 +1226,7 @@ static const struct super_operations xfs_super_operations = { .destroy_inode = xfs_fs_destroy_inode, .dirty_inode = xfs_fs_dirty_inode, .drop_inode = xfs_fs_drop_inode, + .evict_inode = xfs_fs_evict_inode, .put_super = xfs_fs_put_super, .sync_fs = xfs_fs_sync_fs, .freeze_fs = xfs_fs_freeze, diff --git a/include/linux/dax.h b/include/linux/dax.h index 2fbb262..2333c30 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -232,6 +232,10 @@ static inline int __must_check dax_break_layout(struct inode *inode, { return 0; } + +static inline void dax_break_layout_final(struct inode *inode) +{ +} #endif bool dax_alive(struct dax_device *dax_dev); @@ -266,6 +270,7 @@ static inline int __must_check dax_break_layout_inode(struct inode *inode, { return dax_break_layout(inode, 0, LLONG_MAX, cb); } +void dax_break_layout_final(struct inode *inode); int dax_dedupe_file_range_compare(struct inode *src, loff_t srcoff, struct inode *dest, loff_t destoff, loff_t len, bool *is_same, From patchwork Fri Feb 28 03:31:03 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995535 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6239C282C5 for ; Fri, 28 Feb 2025 03:32:13 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 3E44A280005; Thu, 27 Feb 2025 22:32:13 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 36D52280004; Thu, 27 Feb 2025 22:32:13 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 14A9C280005; Thu, 27 Feb 2025 22:32:13 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id E6FF2280004 for ; Thu, 27 Feb 2025 22:32:12 -0500 (EST) Received: from smtpin15.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id A1AD5141BB9 for ; Fri, 28 Feb 2025 03:32:12 +0000 (UTC) X-FDA: 83167930104.15.697C677 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2046.outbound.protection.outlook.com [40.107.93.46]) by imf30.hostedemail.com (Postfix) with ESMTP id CF93780005 for ; Fri, 28 Feb 2025 03:32:09 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=YsXq1U5i; spf=pass (imf30.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.46 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713529; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=6feSz4PxPdf/fn8934Sb6xeZDwz13w4MaVyMgVEG/Hg=; b=qYISw5AKfgWAl/NJ5WTSQmLiyk8+x+KWqMuUVamYSWzu5nPzNyxqEPPqrnIKPkM8+nRKDX kD0Xve3JLms1K5Jg6MTKRMzOmINxGFDx+4/yRDqBW48skeCRNMlRw1uAQrf8Dp8D9VDBW0 aiXzODSK13np3MPxiua2guIYPh21fu0= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=YsXq1U5i; spf=pass (imf30.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.46 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713530; a=rsa-sha256; cv=pass; b=oHfcfaG0gMxedpAlnctdlu63y1DeM49lbFHht6y9KBTAwA/u7OcNFRetjuFl4Soc892x+f 9dkkKE4ZzoZ2mfdjsnjAq+PAQmiulpNyyDXw5fhG0ezlhfRR03uXuHgwwm5s0vGNo9MShI kr9bbDY8r6Ha8j1nciJk+qldvMA7nnU= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=tL78RxBMSB9HVR7UokHRj6ygos22S56LHj9bBxc7390JN9NVuUhYBE6d2vl7OVWIxjF+x36Kp/E3gCBuKlqXFQ2eKXRpPgbGpUjcRTuSVHbZeum6NcXf1tp5B1aecRRWI2PaQfVjx5uE2x0SkV5TlQwRkRyfBPrnkxmdtM+5k7mXAZuy4uAhn4R0hB9vr6v+2S2aEdbX96wwLceUQt08le+O2y5D/DH7jq8Jmxy9wYIs7qEE8ZX2lfhU8u13sGY/IkcnWY+hryMRqwn3kphDPgksmBoanx4G6zvI1TCz/sFj+ttEldmrJl/01u9Q9K9ZNXOAqVFh9xXd+sW5FmgXGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=6feSz4PxPdf/fn8934Sb6xeZDwz13w4MaVyMgVEG/Hg=; b=HHS2BVAn1p9I5N6UHuDpgHJYeFF5vquaCWQzYfxh2ZQeO0lt7IB5Xy9h1mIekCL1VYAwGsXa4eo8RCGWqWrcKkvc/norhc2tfU77QSpQ4TBMU8YLHhY8EwXZr77Dbl3raznz+8jfmQAsgDvBgBU5yOBtp4NCMspAAvWgLYjK322K73sXFsCAkdw7cWzm9wroQQD3BPo39tWBBeAEsMU0wBm/7a49FvtvNDRKNPD6Kv/5reSb+cCcmpvaEKSKi+Y/+7l+BGwIwHaPmdhkfbtJN4f58K/l5GKqyiE2F+6RFDHvR8dDJNOh4rcnuJ6t7Oq9coRx7hTPSK5Hpj92P1Wsdg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=6feSz4PxPdf/fn8934Sb6xeZDwz13w4MaVyMgVEG/Hg=; b=YsXq1U5iSHuvAS89o7WFNsQTJPOrclXo10GtDQ//B8kvePEqxGKCowyYIVQ1lm1+S0NS/c94bCN34RUaL2hWHnsb86HiPpMU1sG080pqSTF0pcQz7bFgMg9XRJ5PdX8zC35PiXQnUHFCdxo7CdOrt012DrAp7ni6StfE0o+14D5Zvc0YuyEkbIplBqwAsWKnzkUPCeR4s/SMg4s318EnqPI6jOuaAWATPJEx+xO1tbVPjilv5IF80oX7Mgx0UNn+b/vxZcYBgDGwYljjKFiOBMbB+qgxr5PLVLkXKTW2GNhYQDZiut1PsifZannNdSe9mjXIz76VHXSYuM3IUn4mvA== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:00 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:00 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 08/20] fs/dax: Remove PAGE_MAPPING_DAX_SHARED mapping flag Date: Fri, 28 Feb 2025 14:31:03 +1100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5P282CA0173.AUSP282.PROD.OUTLOOK.COM (2603:10c6:10:24a::27) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 8715a1c5-45c1-4fd4-5a72-08dd57a875f5 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: bzCXVS0hsyngQj0PnrM1m/gHFgrj4wNzuWfYN0Gkuf8SuBHFXPIxr7P8W5oIdk+fLUI1U2419z0JYrRmQZ5qcJLJrMEQOey6HJeZZfaIxmzzyusww7BObDL7cZGr422WpD/sHhzGhjYRv431PfkOkqWlD59c6l0PJ6aV/na7kdbpE75feRBWjtWEOqpBRMNM0CKQXnrkdAGbyLLz6ZLjo01jKUV6GNtGmVjss8oOTM+jAbhsSjJNqSfeTKwMTevlpS3a33N8zDhnpxtVNXXo04CVXwUtJ/kfyyQEFsIo/aEuSDeHYWABfD7k/8VF3oQeKI2lI7rndS56dTx/lWZfO0dYxP7CizlraBzjgG8BtI00OsR+gPG45PeDOxTLPbzt4RBSOpY4VVuTBOBL7G5ANyQX/Vx+aTf4Qs4NHs7y0Wxu9Jn0I0L44RsTnpJWLZ4UOnDHLePKw7xtBmZjqobD2yHYWKLcpAtKPq6jK0dhZXRxgor0WjEYfa4mTvoMtgtRNSfr6nOLqIjAb7S1HU6cc5uppLjrU9zeXj4Tw/m9t8k+yZryUVtgKHKApu+/U9p5DAnjtk2tKehA/QNsquvU0t3o00R2tlg9V/sl2/Vitq9IG+MHG/DzUavKt+i9rEyP5HwroUjyhWxNLzlm2vkXaMpDzJ5RynoaUQhCNNJgmvRdI34hcnJXY49snmNf6wKIL4QzeLEQLrpBmV5G8nOv0xEiXs8tXKYFAUOQyo+ya/UtCxWObfUP1hs0gEFHEXNj0crMRO1XQwOXjdAiHdAQmI9bK9GGFr0d4HTbw+VpsCVTgQxAtltFbKiBxYs3/YMbzxKmMqAa7OG6qKm0P9nyMZeSsgEomChnJ0A3pqFoIA+Tm1RlzF+VIhftqYegqfHEo4rlLNGAghfT+xYJuzh9yAhR+tL6KzcOAzyv1TkRVF5fL34q9MbyIWRYU0+2Bwk1PbaM/ti7sd3uyUVBlE2fVgbmKaAxNQATVHLJ+3iKPXqxvgb205RUQLnn4VaweyvFGNm6EW3EqTO/j0xgd6gbAEzbZTPVjQTRUt0kv8gDelKEwqSdk81KD2TWz4mhK5e6nsrCQKwEZ1xy8F4iQ7uaCnqsgyk5fDO83COTFNBjAee4X7DtQ8LCJObSuymVfsafgPKH25Ohh4fI2JTiUNswrXkqX+dtYzKl/M6gE5blp+6oCZK1g+/kZ7fL0YOwB19YCHP2tFZ1Yz+4GPzIidIl75QVNBjy3Kx6D4YjDPgxaRbzQ/k/8Q14aQyEifM6fJfKGfT9uddJ6c/YF2Df+xaJsN5fgmVaKWkuAevWN7VpTgbSN6ywyu6cZpnKaVWYzSWc9pLScW+lwRAKpXpisr21clrTFI85jXJGbCZHM4W6DwuabY2HQ8t32u0uyQpc2uIi X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: IQ4aOFEzExeaPODi0/u5BPH1wRMeV1e1R4OSXYqc5r0+RNNDAUiEVQ9X/4xdZV/6gzIqOymfhQ8P0dnKMJjOR8yQJGJzu//Za4gr/c5/sMzHdp2bSXG/JjeeDQ8ROUXm59W7DaGN38U86yaAIAbOLvkPJeKv41bAoyvbA1SB4+6iIXVTZcLYOCAIBGHV2xKMGOweI9XIg603GpYOxG5C7OBBzxK6hkaFMa2/wWdRezlVO93nFB3aUSCrlI7GLoNP2STJ7EufijAA2CSxjku85+isJMH7v9c0Ivo7kA6Yo1f7U8J9rIMxewDgie4Vm/e75/8vWuJHj3/S4P14B47R7cxnhrpcviENktrW1eP3yXESk1dDXd4dn5eLLtqNzF0iyQlPfd4jqyH38upSSxJVX4hbVmOwflwW/a80ir3zDLAmKATD9fSFwZxMHGtIZGg5uVmIRUtkefLvl6tMyG8OroixSMfwBoso+S8xL+b/cTOyn0F+LbHU4BFN2Xg/8j2bBpqM9EGiDDGeWN27rlAte98rnEBy8i2PznoXTq/JPYvc8XU4hDq3wmPsQYFHD0qh38GHJ3sX4+hKJg2vyw6MbhbJit7NId/KB8qyiSWCucLyfyr7Md0a/rGP9e2j6S+lupJjR3Q32xrhjCaY0I5rAnRXKh0Imt3eLTgPEtZAt9FgQUoGTf0Jk2CkYte1i9s1YxuN/DjCgVT86tEmZrj361mAbW6qktQJLX4eYZEqw2bTYowRt4ubjYMNAjNc6W3gM9NkttMYQw0+zgr46ZwTlqrdpPQA7CSJwWpIllKucwjkWdcNS/+NwG01pFJC8ZDv5UnjgfhYyf7noANzQA0T4UHQuhXbL1hbQhbGEUrvpRYAr5b6+1O4xKXYvY+z0RECTYAFyy4fsJURFJIA+Z21b0Gsefi6ozvpgSVXUVM+0MuMwqsIteqjRstg166kXAOcTAWnCTclKjr/AqPEZy8qAg0KbGIlVQJpJ0DFnmiSUGZf/Lp6VeRfrFBpyN3pOqIc+45+LXR6QYgnW5cCkKeJXTqO+kNzvP5UPLSBl6azu1+UyxQJD8gdhQD5Y1uxVbIQzMyqI+UqfbqImDSk4X02MNofvs1Gp4i0zO0UUdr8VhKF9MCZ5vh17lSKDZF5jpU0QB40te+aBxj2qCxzf5vyWjQpYKu6axuQHGVvkKnlkT1TkdqIdYopZyV4XmfliIeS96NlEWKt3jCa6bOnVeeEJ9Z/a1qE8ybhxShRFS/MsCzjX1ykIIgdY1P/ATuFWhPVPCGFj5t5Ek1eWGg8i8h36XHhzkLgOT8yTPWv/8R9fGPbatb0ghbjyHH1uSftPf4DxR3qMYGbXnVmIM4DjoFVtR6WDjZdCpiw9dnxsvijZcbJZ9uLVQqqgaImgR4E5jiVRooS1TjSpAkeE6EC+tKiJtyJOTAMJIlNyyeZctShpdW+jDqaqtk6s0UDVbAr6dFiOhNGyjQ3MXQSuIiaVg29HIe0ra08Nne9C02D8E6Me/Kmu2CBHKcAcvdaC/yLwqH5MqOLuBLzWcFV2M461sx13DEJ7pCRlpr9vIeubtvWeo8jF0Z/422L1W0eA0fIQ8tx X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 8715a1c5-45c1-4fd4-5a72-08dd57a875f5 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:00.5346 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: k6aRYN0ndN8eprhNjHtU61ubW/m6kMQ5hinO6nlYCm8dbU04jsTQRd3XcNlQGqhgsrp5+XUtsRQn3a0uLFIzeg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Stat-Signature: iwagwhj843k7eiehcouyfsfukafzm5wf X-Rspamd-Queue-Id: CF93780005 X-Rspamd-Server: rspam06 X-Rspam-User: X-HE-Tag: 1740713529-852027 X-HE-Meta: U2FsdGVkX1+7xf2p+LzoffAfjd0faRJRKyNuiC75ophcqbxeOnShMlxk1VkV2yxZ+QP4VPhEvK4OxiJZlYPHCmRyJ50YXwX+IIebphselM08LdGi5LN11M6rb+oa0zQDXzDpZjGCsIOe8OQWllyCUgYUC98yvxdxBJKr/xZ06X+U/m75omeq6ZNdcs6Z7y43X78JZHG8zPWldu9EeiTKSpgTCngCOM2ZQHO1yZIbf8gsqAtH3cidQfJ6gf4qVfcdGcYCrMosWWQ/ao0xBFxAYKRbVO4PabWeYNW0PM3/Ov7+hbuvLmcLKWF1HX1YI+hpHUqQgMs4pslO37zzowpl1Bxu/GQIp/vHIqLwmuRggA60TGakXoXptwSlRuG0Y+JaaJ/bYhPw2LvsioxlD+35jQ46t60vNJsPzvMyp4eL9wUbht6VelhVJ7o6DLXfcSfNwDJRJq6qsytVkHoRTO4wVQgJVWN3RpMVkLwDVjxoxxOqQeTK94e8/Jpxjk1+gDcFT+/pD5poylG8m/MmoHsojTgF2xZaoo7G2VPqv4N019opo6DpOuqzMO541nQFIL4uhYBS2cePgoiL4enM3Cbjejkmq4CgeYS9cmHCXGHW2j6EtRWZQcHNw83HkSSGcU0Mo9/CiK3j3g6YHs9VtWCrxYigKq78hV8No06EPwowtc4gFDNt4ctDySVDWGHU1I47pCjYQPJ0HDQOGbRdWdK8QfIrN3qnkzt6UR4MAj0pAPWe3P++xJnbCea00GyjFeC8NX1IJEnXWfoCYsdEO84+q6zVGiUUVqctrinW1lCy53gENy5hdXYYnrbJ+UYkSnlHnjn1OK7L+2yGjOXCJwPYbDQbPmUD2aa8pus+gAD4+hzqai/F32Fw4c9+nBYoP10djy84Ucw/BVztk2vXG6F0f2PXWcLUmHjEJm0TZThjbNcoyu1QL34M0eC3ituUugXHPvdxOp8gz3Jr9s5wgTO eRPE+zlL tTbJ4JhTgbDtzIWt/qRuIcPdLFcRrUKW4gtPVlIZSrLiaxeUp30jKt8dpW4GhexUZturS1ebgnEOHNc6j/D2tspCRTywccWIGoc1Jz5dp62NCzIAxg1kdlRIIUAlEvUArncT/YFkUTak4/HqwkXZMAmjnozIPZgs7issz73wYenUkL9Y5o8amN7FCt/4xxVTEImgNUWbfwQGhvnjSU6W2ktygephYu1ijeNQ8kdLkIeo+vwEC6YCaIAYoWPThRgFsJ30UefCsS5uTiCRknlE4b86HQcnFAVuU2pT86okPy0vG7UoOjEd3hV8QrsdUFafq2lDfqWuwfflLwmioTuQoYAga2un8CJEFDdvvAQw1DPBrcJg8vSzsfPQWe8BFVzKW9MkvZxR5BPVx2Ozy5GRj0agGRZkWf4ANVBRij/15H9huMl9JHHey4Flr7i/sYXVdpqFX9XIy2H3Wwqo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The page ->mapping pointer can have magic values like PAGE_MAPPING_DAX_SHARED and PAGE_MAPPING_ANON for page owner specific usage. Currently PAGE_MAPPING_DAX_SHARED and PAGE_MAPPING_ANON alias to the same value. This isn't a problem because FS DAX pages are never seen by the anonymous mapping code and vice versa. However a future change will make FS DAX pages more like normal pages, so folio_test_anon() must not return true for a FS DAX page. We could explicitly test for a FS DAX page in folio_test_anon(), etc. however the PAGE_MAPPING_DAX_SHARED flag isn't actually needed. Instead we can use the page->mapping field to implicitly track the first mapping of a page. If page->mapping is non-NULL it implies the page is associated with a single mapping at page->index. If the page is associated with a second mapping clear page->mapping and set page->share to 1. This is possible because a shared mapping implies the file-system implements dax_holder_operations which makes the ->mapping and ->index, which is a union with ->share, unused. The page is considered shared when page->mapping == NULL and page->share > 0 or page->mapping != NULL, implying it is present in at least one address space. This also makes it easier for a future change to detect when a page is first mapped into an address space which requires special handling. Signed-off-by: Alistair Popple --- Changes for v8: - Rebased on mm-unstable which includes Matthew Wilcox's "dax: use folios more widely within DAX" Changes for v7: - Fix for checking when creating a shared mapping in dax_associate_entry. - Remove dax_page_share_get(). - Add dax_page_make_shared(). --- fs/dax.c | 55 +++++++++++++++++++++++---------------- include/linux/page-flags.h | 6 +---- 2 files changed, 33 insertions(+), 28 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index bc538ba..6674540 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -351,27 +351,40 @@ static unsigned long dax_end_pfn(void *entry) for (pfn = dax_to_pfn(entry); \ pfn < dax_end_pfn(entry); pfn++) +/* + * A DAX folio is considered shared if it has no mapping set and ->share (which + * shares the ->index field) is non-zero. Note this may return false even if the + * page is shared between multiple files but has not yet actually been mapped + * into multiple address spaces. + */ static inline bool dax_folio_is_shared(struct folio *folio) { - return folio->mapping == PAGE_MAPPING_DAX_SHARED; + return !folio->mapping && folio->page.share; } /* - * Set the folio->mapping with PAGE_MAPPING_DAX_SHARED flag, increase the - * refcount. + * When it is called by dax_insert_entry(), the shared flag will indicate + * whether this entry is shared by multiple files. If the page has not + * previously been associated with any mappings the ->mapping and ->index + * fields will be set. If it has already been associated with a mapping + * the mapping will be cleared and the share count set. It's then up to + * reverse map users like memory_failure() to call back into the filesystem to + * recover ->mapping and ->index information. For example by implementing + * dax_holder_operations. */ -static inline void dax_folio_share_get(struct folio *folio) +static void dax_folio_make_shared(struct folio *folio) { - if (folio->mapping != PAGE_MAPPING_DAX_SHARED) { - /* - * Reset the index if the page was already mapped - * regularly before. - */ - if (folio->mapping) - folio->page.share = 1; - folio->mapping = PAGE_MAPPING_DAX_SHARED; - } - folio->page.share++; + /* + * folio is not currently shared so mark it as shared by clearing + * folio->mapping. + */ + folio->mapping = NULL; + + /* + * folio has previously been mapped into one address space so set the + * share count. + */ + folio->page.share = 1; } static inline unsigned long dax_folio_share_put(struct folio *folio) @@ -379,12 +392,6 @@ static inline unsigned long dax_folio_share_put(struct folio *folio) return --folio->page.share; } -/* - * When it is called in dax_insert_entry(), the shared flag will indicate - * that whether this entry is shared by multiple files. If so, set - * the folio->mapping PAGE_MAPPING_DAX_SHARED, and use page->share - * as refcount. - */ static void dax_associate_entry(void *entry, struct address_space *mapping, struct vm_area_struct *vma, unsigned long address, bool shared) { @@ -398,8 +405,12 @@ static void dax_associate_entry(void *entry, struct address_space *mapping, for_each_mapped_pfn(entry, pfn) { struct folio *folio = pfn_folio(pfn); - if (shared) { - dax_folio_share_get(folio); + if (shared && (folio->mapping || folio->page.share)) { + if (folio->mapping) + dax_folio_make_shared(folio); + + WARN_ON_ONCE(!folio->page.share); + folio->page.share++; } else { WARN_ON_ONCE(folio->mapping); folio->mapping = mapping; diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h index 3f6a64f..30fe3eb 100644 --- a/include/linux/page-flags.h +++ b/include/linux/page-flags.h @@ -710,12 +710,6 @@ PAGEFLAG_FALSE(VmemmapSelfHosted, vmemmap_self_hosted) #define PAGE_MAPPING_KSM (PAGE_MAPPING_ANON | PAGE_MAPPING_MOVABLE) #define PAGE_MAPPING_FLAGS (PAGE_MAPPING_ANON | PAGE_MAPPING_MOVABLE) -/* - * Different with flags above, this flag is used only for fsdax mode. It - * indicates that this page->mapping is now under reflink case. - */ -#define PAGE_MAPPING_DAX_SHARED ((void *)0x1) - static __always_inline bool folio_mapping_flags(const struct folio *folio) { return ((unsigned long)folio->mapping & PAGE_MAPPING_FLAGS) != 0; From patchwork Fri Feb 28 03:31:04 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995537 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B3681C19F32 for ; Fri, 28 Feb 2025 03:33:32 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 2865A280007; Thu, 27 Feb 2025 22:33:32 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 1E7D2280004; Thu, 27 Feb 2025 22:33:32 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 01389280007; Thu, 27 Feb 2025 22:33:31 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id D2804280004 for ; Thu, 27 Feb 2025 22:33:31 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 97887821CC for ; Fri, 28 Feb 2025 03:33:31 +0000 (UTC) X-FDA: 83167933422.26.09D9B9A Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2071.outbound.protection.outlook.com [40.107.244.71]) by imf09.hostedemail.com (Postfix) with ESMTP id D590C140002 for ; Fri, 28 Feb 2025 03:33:28 +0000 (UTC) Authentication-Results: imf09.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=In5t2oNe; spf=pass (imf09.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.71 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713608; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=92BKGlNCPFH50JMw+ilesSOP0I/ZNI3TrI6Rb+xr1ag=; b=ZAKBnXTthJfblS+6XW04NcnSWiqtUapHSXnLy6M3iHOfC4KIiAXTox/ztcX6uVwraV0nsj DdGLz4L33/w6wty58KXei3valOLDpeDiRd52Y6c5UafhBhqeDsnD7DzueS7zhjfEbyLr6H ViDxK9H13lDmOozGcu7nJeOfjuDZCy0= ARC-Authentication-Results: i=2; imf09.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=In5t2oNe; spf=pass (imf09.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.71 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713608; a=rsa-sha256; cv=pass; b=BK7KfcZ+Ia2xglvJLsGkF8OmxNJWaOfioITdjXCSIyahP8p/jfx8KBnMiRyAfCl559zXmM pCouCcX6uH7HmyAzpCmuYUgqtZzvbfmSZXZ01oLMvJSPe9hkz12OH7DkkXT4Cm28jpx0uQ 0S+J2ObfLkMVg45GPMTU1aC5CrHCtUQ= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=PPutJRstmMMc6MHC/zyKbMkMxA30mKD5jh2WkVMpRiZNekUaJB/q7fYpLfYe+sgRvJV2xNgyUtpEwPZHbiRMIeFjNusrkx7n4KyN0D8vRSjufEMI2MkVGSL7b102Nqd0i6nZyv/ZRt2mY2qFBoNPgaqMnOUOTkUKccg7awirfAypLnstdQwv0CsIPHmNfMuN/H3ZKd2dqSD0vtynXHXXHEMopykHy4osprNU+MyXnnKYuEhkn5TPhmGkNhxqKtxoim8PYrG28T+g6/kL5CYjR6uyDusXAoWKsI59agGHWdDy/NyFK62aETztNIDxUUQ1zVYnHDZS9Ob6UhvrJRFvXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=92BKGlNCPFH50JMw+ilesSOP0I/ZNI3TrI6Rb+xr1ag=; b=ngFrwpPYWm1/XFZqX1l0SaU0tEspMVUDmJshEm+I4wR5ufF9BGzpaf2J0ER0zDqhs8Naawr6i2/1xj/xieUEcHYXEbnhxxTs1Ijf+i6nIpdo9Zs4Hfz+f0mdtugdQi5rrWOWBTgWPsc9du1u1iSaXWJxVKoVHgi7/AjNoSuf3f2HCten7T7q7b6TU95HSmBG4uIbXdRWDgoYFFfn1BJsvsZpEcxfLfpBq3s5tDBHaap7CW5vPXHy9yYxVmH8Lde7MvoZ969+5JpuzFunTv8+dQG19Pn0HhdiWQ92dkmnFcQp8b80gRxtaNqDP+aqgwNbQ0cgCTcSf/rm8UWmFHMmCQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=92BKGlNCPFH50JMw+ilesSOP0I/ZNI3TrI6Rb+xr1ag=; b=In5t2oNeMARo4jtdGrHkvFeVbkuVu/0f8b74Gy+xyxu4hlqoJInKkJ/cZZXlxbmVhGToJ4+n39tUQWPi+1YEwiNOITIPNlv02/vCZBvBJEdlKQPgk58gRxCe8EoQEHCRBwDz22438zTyvwWkAiLkrRWUyf/GBB6unh4TL74InnAh6wyAlvZpH/sGNlVWY8PPA2sUQ5JaE2jL9fdOsqUOpKQp6ookxNFAMN9RYj6MdyQbb+XuWbd9muymOGHSb05LP1H507zlZ2IdwGWAnFhtLJ0Obq4mj11Of5iwPC6pGKcYH9sb1wfN8PSUgnImcS0wRLhIKS3a3zQ8zQHt1uqVIA== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:06 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:05 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev, Jason Gunthorpe Subject: [PATCH v9 09/20] mm/gup: Remove redundant check for PCI P2PDMA page Date: Fri, 28 Feb 2025 14:31:04 +1100 Message-ID: <260e3dcfaf05ff1c734a49698ed4332b5dae04c2.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY8PR01CA0006.ausprd01.prod.outlook.com (2603:10c6:10:29c::31) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: fcd65de4-2769-4efc-53c3-08dd57a8790c X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: xcAI7Ka5WZsjZFFokCtuwJGj6QifVFtsT5uQnCRu3kpK6f2s1j0PIvVZu7z+gQa3l6TV6FxZ26AuJFdn9zsCgRUh+3d7Q1kfNInFSlLte/ROTkb7Z2weIuYJ/qEZtE9RTWcwdH6lu8wGzcm7yhP7gERNHkD3KnlXqJJEk5JykL0SXtcKMSBbo8MHCPonbP6b4E4Nb2vHeLzbC0v/xVipn3k/tZTMqhdzQytp7JwfwkxbeaokdV27iemSIUwgYccn6uJrlVo7CExJqdQoMv34g11Yujjnx8sp2LD0sdefhq1tA4LNqPyIK/RgDnIDieds/YYdgDEUUFtgIO1vGJC4HiqiXkskp7s1C1SMqrR/3RwmjQhw3et8GJNoKKHZ8gayoMrFELMp9MBViw6IfeiYQoB5uPDcxsMm52AnJF0lAjZLpVvv8pLExgObQGlHKpQdAMzGVDaHS89hf1MGnMZVuCm4mjcZRKkx4jUwI9NnWSTLd3Zt+CFHi2rfOnBfwUlrEPpmfpUk0ca3arOMydaIqheDcHNpJLE9djTo7YZ4Bk6RqNlVjc+eozf/+1RkVh65JBzinqTx+1rSVzvNqFmw2+4oyqVcP0F6o9nLyI0zEh7FkKzydQB/To1gYEiYzWo172hisucly8oG8f1MTFqV3Z1iAEun09CwPZ4EL5KMUX5TRhn5Kb09kvfigT9PANqrvWAg7/y5/3PFYwpu8pg+PhdMM/0h4Zy+qY6ZVOJi5tOJrcJY7r4plfbpWJC2Ruo7N01MTe1131hU9tWo1jqwJWnV768XRVH4/dudtsHbL9w7+e5qGWGss1Cz9gzL3EmNK4+SzuUXjUj/flrw+kmAaVY2cs4j0wKm54U7CxhZybTEvdAHC16aCLNNCDHmBmj4GfZk92I8gEP8lmdN8CozcAs96oatlksoATxRr0oWdpQ0k0BL8COQVldExS7sItN1eiPzG4LNnWEbjpzrzDqwgH9iK4ed35YjQP9ZAFVuAtXAN3Za2Vp8bxj3vOy8r6mG608zmSt8XEQ296mGcs/V5zttb303psuR8LcWFQ1F6s41j6TRbbSKIkg1NmA4PlLidgrac2QerUXeMgIA7nyggvzueJn0B3iZOwRseZUZhBsB4EEIFsnKwNC/rgbjZ0G+rfLvGW+flMthT9tzjFCV4FzzQTT+M7dNltC1NS41T5jjsXc+FzNmrabNRZ8W3z36bVx04uIjUxRN+iDB3ydPP+pOuw//aD9TfcZbYWe68z3Jl1lRXBE2h+g/NzcHD2BD807jB5sNySwHQPkl6vqQ4S0LL5Kk3ThofxYtSE+8doXBZzOcslwbpuRpsV+stRXSlsvTJeeSno121sKB9NVIcXBGURfVpXipIiyHP/CRwyXXqy5zyckTfVkz9jX/Ynx1 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ZNSYYM0VgFoTINIbMPdQK3yFiXkRQZ6DjgYShPyr846+Dz5pzCyYWJISak9MvAAKSvdL9mLk2Xp8VryLNZoo+ReIlcExATNn25ak9Cv318EJ87LOAJ7Mb6UJT27B+g1m9sjW3vHfMWzr68Inu6sOEeaNbY3Iu0D2yLwjmS49fmtgYkLx0q3y7jfF7s0BlNrNRR0b4J7NyUHv9sLmiegP+VF2juUwIdXbZL8+4+SxNfelYkzwgnnPyrq041O460bJgsGyRFqdXLpyLxhb4f4+DroVmaAE+jb8qFB6kGrHYIfDjzz2efmYSO385wwiBGxgSda6cxiQ0q2Q/i41hXG1gKS5EAzdwSnv8wsAHggaD5Xrm32A4sJkucLk0tjB9bdUYi9sGWxjsi8ZnXKtDEE10jZV04r3BEVch8IlJRf7tT8MSnD670y3oJCkkx8bHOVQPunscbdVqPPrRsuo6dxnspgazc11BHyXxypvkfhhjKDwgNLQMgdgRcY2w2kJv8KyYgBmGJGZSYgqKxb4Znhv74XC8vQps+lJweu57IHrbT2FlxqPwp4HE4DO+MikkD3FFXM5JS9R1/nXgcclkEj6cxZHlhWZVNPDR0oh/3P6MQdRDSsVL2N9kolctfgSinU8Ox9JXfqWl0znE2XaRZfyKytz7A78EHbPbc4l+LCJhXRV0GD94DPwpXKCgDXyIko+DTFmzvNly2rlLBzywrh7MyW+YQ1WRAXKW1yB1YnXKo6rtlsgFj1SU4/d58KV4dPoLszdhKzQix7CCjGX+/DT7xDcOT7F3RTpTpimlBGGaDEsFn896sb/pxYskK8Wvy7Hl+GxrKh/paFkqBoIwYF/nXd0385tM227RypSmReiIfAAu/xyYSON2ZGfVIO4a6AVEUCz8HQJdruO7FUT8LM2Jmg/GcvdfcM76qyNIH1l0pefh0YZ4AamhnVNxxijKw98FBWYINQob7NuTRQ2ffAMCL2yr/ksJmiF891Dm8DZrnVCjKCFN2VfxHbY/IONSoVENybr4YKqMOLcUqCF6tcvthLgibc81drWrEUWGasU92HLHw+LF4kHUWGaPVsXz9tOzXsICZ2gFDbLHWCSRPC76itmLtR3DjaZu5uCZ/PsAD1aO3SbjlxERq11BMKX8gDx54jMj7NJEeraybfadXYf5Ugmf8YsSXcrsLp42/smDbYN0Y14uyyp6Fyoif6+77Y8iiwlrIvwa4auFcVLn+LxBB9TmZ8fTlK5fL4yC54XMz6yn9BNdxciJsntgOBmVEbqk9kCCoxvO4YVT4p41tx+NTFmgPQOQiJAp/5InQ22vg05nRuh1sQ9HU22yGpGJahEFhvYjhuP4myzJglef4wlyDNa46XlSmB/uDu/mcTfqSsc0ao9qQRdj0nHxulXnw1hITP0GaLJ/sF6jAN1AvLaf9pcWKHRFZQnA2OCDOfFTatS+RPpuPxdBzURiFfL7WKzn6AR8fwXR4omNbUG3QRKqBahXmjuVoxxTWRYbiY4gBLB1+AXqfG93/xdS89K8vqSgjOfKZQVSc0FMcByaOm8H2K45PJ+ApbjpZM95Tbuc7B9mUPxHUl3XaG3rRLwozPa X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fcd65de4-2769-4efc-53c3-08dd57a8790c X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:05.5876 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: r9BB0W9PBNUCspUjlmeVV5947m1zx+SORTEwDRvF8qKXsvSuoZCF6zIC24gVidxIuzKvJ1l7SbULmV/v4fIKkg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspam-User: X-Rspamd-Queue-Id: D590C140002 X-Rspamd-Server: rspam09 X-Stat-Signature: nazw698p4zr6e3dj4iidnjq5kowft1zs X-HE-Tag: 1740713608-987626 X-HE-Meta: U2FsdGVkX1+YVYKOJwSvh1R3QnPVzv2pnCVfHcQXWRWpbb4MKGOlUXfuBpz9QTu3nTdwWPZ8YEJjFcOgG6dK3ZeW7/SCpI7tQBkR+3ROLXUhI2rzBoTOWA5Hc4MXoBczBGjAWxfMi2Hqfy03IQb66W9QxNh2FpaYCYTUt5zGDzY38Qkp5UiMM1cWWNTxk+bMLjkO/gQNnRJ+pvMkEvzjJanIPWnbbWDmXUPJmlFYjKmbCyg5eQ8BpAf5LdXHQMMaLeMRJ5uRh9WRo2oNvtK13THJ0CFOjoCCL3A6EM4Y66jiQTK8aKlsKmQIP5/cnmv25AZc/+UaW7Rt1ZioJnPFepH/mq+BvPKk42I8j5CRvcq1weWzUADUV0jx2ewkL9gOt9WjuswZybo8TkKyzgPjFdEpH38XqXgPVbPDJyJnJQcm+G1eqcB0d818J+8rUAUPiPVsy57COYEBfomMijCZQdAWaTDASEIaR7nmfDZu5p7qeTpmwwaFSLJQUSCsJ8cJdjzYLFgpZh1bRoOMTTW2ko98ZuCkhSzhF6kFOru9QZ9s9Yx/CCI5uGLuHlvsGv+2fS1HJr4TNpnJ/8mWJzk4yFi6dvsLk27uI1P/W3eOFdmoEh8e/knI5vdvamqg3FgsFC2IKL6TVmECpkyPJD1tkfpZyDqz2tl63nL/Tp6V7414p3+x2foK6VJ46Xm/AArfCAM/X1ovwSNgv9i7ID3lm+zgIOPielaJFetnyP1APQIBxfOTQaCm18xgGrZQaOTCYE+gBuhpI6kZhLP9UaMVROmdEf4idd6+SbXMSUkhAMNoyyTdT37wZ/+al0ceoRo+NV/AlqoaSaa8nWATmAlEaYuKMoSQ1gDxwYOGGPIg6/yJV19kMTW9VMIqxZ63Pz8EFzF2Hc1KTekH/5pKTB//PX4WVQ4GEQEHP4NC+S1aC1r9ml6K2ZY3iBSA+Mj0X6P+5vQO8NRRhSs6lVKLgHX VGsBYwAS XTKAZFFm8eZxdG26crZYqoytUqZe5jOxrkFZpAbGHyrAZY6cZahXmICtj1jzMh+O724SN01b3nlUgxTtkNWTBTmN0USEnC5n9l1q5nBlAfmhdAVObinmIe2RWEyoONivf911PdgVDexz1tWpoBk/m3FGg7gopbA1KF26QSlosGHHZLGvK3B5DiHv5dYTlj/MHxzWruCKVge4bLPa0LPB0PTrrxxFsCgk5Ft0ESd4PHzCqXkELTv5n25sziGAdqPfc3s+MMUp1GNH3y2PdFLC5QF0eLQAAR1KSXnE8E6flDhAoX8jBUOxRCXg2r0uEVCcderCimKic6PQNOFRRAkPbafzNGRweFEM2I4yEmcbMVWCzQjDKRF/QC5/0Tzpt53YNHVw4iBJUm/ZR43zbGKamh8SBRGw6hdM/K3TF+sUcC7g3c65yBAfQfWhX75awr11wHdAMDXJNL4oaNr0= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: PCI P2PDMA pages are not mapped with pXX_devmap PTEs therefore the check in __gup_device_huge() is redundant. Remove it Signed-off-by: Alistair Popple Reviewed-by: Jason Gunthorpe Reviewed-by: Dan Wiliams Acked-by: David Hildenbrand --- mm/gup.c | 5 ----- 1 file changed, 5 deletions(-) diff --git a/mm/gup.c b/mm/gup.c index e42e4fd..e5d6454 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -3013,11 +3013,6 @@ static int gup_fast_devmap_leaf(unsigned long pfn, unsigned long addr, break; } - if (!(flags & FOLL_PCI_P2PDMA) && is_pci_p2pdma_page(page)) { - gup_fast_undo_dev_pagemap(nr, nr_start, flags, pages); - break; - } - folio = try_grab_folio_fast(page, 1, flags); if (!folio) { gup_fast_undo_dev_pagemap(nr, nr_start, flags, pages); From patchwork Fri Feb 28 03:31:05 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995536 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3A326C282C5 for ; Fri, 28 Feb 2025 03:33:27 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C246F280006; Thu, 27 Feb 2025 22:33:26 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BAB94280004; Thu, 27 Feb 2025 22:33:26 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 9B023280006; Thu, 27 Feb 2025 22:33:26 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 79A69280004 for ; Thu, 27 Feb 2025 22:33:26 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay08.hostedemail.com (Postfix) with ESMTP id 3F359142179 for ; Fri, 28 Feb 2025 03:33:26 +0000 (UTC) X-FDA: 83167933212.21.4C0B217 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2072.outbound.protection.outlook.com [40.107.93.72]) by imf15.hostedemail.com (Postfix) with ESMTP id 78DFAA0006 for ; Fri, 28 Feb 2025 03:33:23 +0000 (UTC) Authentication-Results: imf15.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=fedegiGZ; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf15.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.72 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713603; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=Vz0xPUgxEv2VmYCP8Dm5xz29OCp4yeM8XyN0+NmpTis=; b=EusBX9RlE5JSVl4FT6mU+1x1OEio0vGSafPf5IkINg3Fz+3I7fhhzOn25/FOBLI2RuB6oB yxqAhM8KNouuKBAu8PVC+ui/sk5+YPV6ee4t+NhBxoFQOm9aKUYWWnZDUAiHsnyT5KdIgn 7zKnnZ7u7UKN2kzr9nkBMj29V2M5IPo= ARC-Authentication-Results: i=2; imf15.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=fedegiGZ; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf15.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.72 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713603; a=rsa-sha256; cv=pass; b=PzaGZEm9t5jawsFf6KCL5iYxRXqeroLCNH0t0eekgdsMwK84HO+iiFKXqElDQY/WVjFMEz Qwx1Kvs7pF4fQ3C042h5DQP3aRlAAdpigoGjrjtETQ6RwgomOlOoJZ+34+erOR1CNi3Qqg LDoAcYnsKLyML+jQXfyKS01vayfLx7E= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Nqn48H1nwZbYSj71N+VJ32l2Mx6/s10QC7xP4aCvNTs1O5DeYLUKFRYNxBVyH2HxpHvIm96ohpxFBjD3xT4DGnRuMDkakDsh8Q3IEfrAdT8mwrl3Wg9aVimFvdVBc9oIv+zmfEaHL/lUTv+gsyLBS+gpLuzzP64xzxnaTGEMDsjgxM6TaWtZF3WuUq31dGj+ImfU/WOJ4euLLOeDB9rBxZZgqI6rgj74PXMnrFQ4bw+kG6RXbtQuGAdt38rVIllQbffCDUL9R53AfhRAbF4qm/zbdE5Iu/x0HAUA6D6SR36eWnlKn6bHlpb1yWjxDT4QvZcm+HRV0yChjqQtD6LsvQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Vz0xPUgxEv2VmYCP8Dm5xz29OCp4yeM8XyN0+NmpTis=; b=MoN2ygR74C7rZqggF8BNZIxfRyQs9wsW6dSsvX1ggDSD9i3+CiH2fJ9CljFEyS+/zZR8DkQd6/zbzzNvi9taiun1TSY1KC4QPZGFtRqxNwSSWypYLm0gDk7lxHhmLnEqXVfEcUoe3t3u3KJjzD913y4wzr6X1BkmiO+CbSLiPZBeOhLx6leRnrqAkYqlSGRc8/FlK7daETVUEfxSA+QPH9bF+D+mB/ySlsuIScNHXqf9krigFt+MTTckihzscj8CtXIAt3f+Wczf88a90O1McWYF4ISZGxhHtY0Ki4tYXR+Zn2aaw2+sGmvjZGqzakBxjPj8M8YK6u8LPKDxCaRejQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Vz0xPUgxEv2VmYCP8Dm5xz29OCp4yeM8XyN0+NmpTis=; b=fedegiGZOad+UK6hWLjarIM3lk4RZ+HkEqTu73JZrdHDtKdNEvablalJGVHdJLxShz33QJXrCKLhgCJq91k5RshHXRY8iw+j6eUF9KfsMNDTTGqqPb+csIu9302/huQnpBaI8l20VB9Le5rAalPD1EsMLNbv27FPq01NrGi+o6gOd7EC1V/LHij8LXW0xzi0LQlou06XCjJQaBiM3FQC/e2sscAF43DWbHRd0fWIamJSlGMm3SvoKoCZT9l3wJPF0jUVd/CSVzvQKQg6pYB98IwTTSEEbLvj8SetX5mXD22lYTdniqsDYT4pQ1+MZhkaCe10CLMue14HJsTgpdF6yQ== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:11 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:11 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 10/20] mm/mm_init: Move p2pdma page refcount initialisation to p2pdma Date: Fri, 28 Feb 2025 14:31:05 +1100 Message-ID: <6aedb0ac2886dcc4503cb705273db5b3863a0b66.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5PR01CA0020.ausprd01.prod.outlook.com (2603:10c6:10:1f9::18) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: aa179763-44e5-48df-9aa4-08dd57a87c22 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: 2PaHrVU7ZGUgVnpFO0LXflvelPa931IPI0PzT44sNY9abZy5WyOQxHld9KzORoAQJIb2LB6HV1Yh2tpG3drqBwu+2s9El9qvqUP0pXi0ayiG+FixMXWz5Rx6V5EodJsT0F9gpMSgVC8Y8hZVGbeUDdCqzw6HUBqWv/cd/842mEMgyOacZAjBY0ItlT+RXB8x+SVJ+LDBBcbv/nU/4eVMfLfRpkLpBrVdwn6SPLNLskgG9XVOx0N+RYBx1raY00xTemKJv6iEMtIFEunjEbGc7z0xnm8Gon1oeJqsVQ7hUTbwSy7y5emoaXttEpVyarJTxPc6PekjXJNxxptiIgAWRSzq8xgqUXV2RhNx+QkZZ/iT30pLKJsJRLP7LAl4ebvv7UqQkyboCQPW8LM6xZapZ5n7TFy3TJglCgjoCEV1kWXlUlSckDVQfg8WYpI2nuA10iLF6kxuntA3YSk8EFbMXI+TBDwU4hzewoIN0QzP8TW/7ObopuG2qNXV9zdgSwGR70Ns8wU2blxJdgWo+djxH/LVZXaiG1mDqBwdHJ65tAbAqXeFF2YHnS6e8oSWf0zxEASzpvFIMm2TRjbpkspQarYXUt0Dg3DemlDCXgiBAm+suIzRpdy2MEj8a4T32XOzVJVzDE+4VY2M4WHyw+4gUFFjyc7d/BaX4Rg1RLhSwvQ54gOQKmPBNKmsEXo5w38Iw7soulVf+j8go1OshOhxc1Qsr/tehfrHi7PTuuPFQ3vDWI4WIecCG/NbVre3wvbMOAB1oZxhoie9C9iLhyBbHImWabo9lpN87F1qMAl/A3XyxQYsmY+hx5a0oK8JuY/NDTwkEIH82rS59qrhT7zl149ntJ2EWAokBHP0DEK/rIy6rLoBSFX8Cnm1u66jB6K3G2JdbkNRJdj3qKjb5ksmG4AVDPOa22psi42O/9LVsCzc9jVV3P/btZVOgZZftmC6TO2P1Om0TDmo4sYA5GZwtJJCgBUPc0Yi8USapYzTzOjKaSXg46/Ld2DjOaS02z9D4pgnPeDRCNsBwR4o8LeBGFEASQ0SYxx7DpXl9Or9XfjBeesz0oVNsluD8kBzFIBwwIEpJfVt5cxfkULE3RRSDmbei5GDpjuQKpbWhUMtLZK16VKhU6PQ2STMB7dEZ8bX7Bx1dKd8u+2kpgP5oc5NqBAQp4o77EQngYcMso/LNyqkEqjuwBbBYmmw+88J1ooAHmNhoAR69EW2noTF6x0DKavwn1+f9IRKNbnwAaZTO9hgEIKI+WwdMK7/fpWkWO0RGOK35SrS8SKLxIz3LKdrx/xqs9y7hMD4DdDN0z5oQl8GFXPqUdoxa9fQQ6nzvargfmzCg5S8QgPsE+bs8UE7EZFuhIUghZnZsXYtPfeq2LioF+KQGbUZHxApHWQtbqzz X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 3P2qb5aEmhhe1WiXfXHFIx6LN+vFHZElkP2MWZefb9STEtABcxNKeeTf0RUckW+E6xIsGOyjtjzKES5a48HdeROFlrWrqoCxpwEPPUIc5DjV+7NJ9VKFDl+zj4tXPEx/UetW4UXfi3ggVTeZdmp2eIWJ2+tpRFzTJzTh2OEs6aaAuSi1gLS4uMsTr+SZ2GWizQML66wdqeIxSIUDiaObicOj2pcqWBrQottIUmqWJ2Ds7EdlsO+ctHpYxFoH36LTqrY3VJ3IQj/ag+ef2Yi/QqMEkHSWoIn6IhCAa8VZ40V516NbgX3PVe9yKg7L7NKeQDCAR0Wqj+gxoPrKi5ypjMsT/TfTvCMm8eUUrdyTH0GFPvPMB2clr+ucAXBBsJ7oZuTvILzMp29TQnCGkNRXixEt/yhIxkVyEpE8sEOUWdL8efiSmC/LmZogSiQIn1MAKDEEs9SCtKo7pzOo6TnHWBTtVmIWo7D5q7pxt0DPAoEb0S6nE4r9ogerhiLJR9GNzr3cC7tjRJglCRPPIQ6bsGuc4nEx0xz6gBWHhCvtM1CeCbdc/neTHZoij/ZXut1FuUZ0F8rZJd8aV4uhL7ZEqzLBzE0hFWE3vHeWERUHWG4YQ+21JzGcLjql7S6P87xalmYlf6Ci15pMO/p45F5NPNMBCb4MGtCaTbQVXG1TSGLAggyi65woYEZMysuSCsC28SsXLZmwf9f50UJ+hDLQ8K0gSoEstwqsCyqFPyuyc+i7tVcPRz9jobROKhyEegHKfuGGTsQp1wT1yx/ueegz8YfocdY6eMux9Izbk+PDwqdNPqkNU0fI6pblFnC9zN1JG+CF3u/PdOPburEGzSmt3JpqDnvVjZQjggOlrfUOdd0XIZAxHQ0kXmmgnfkrxnwcRHFeI0OxvoNuBN3OCJ6t2HAIgPM7c1197Yj/S6ne9VFsZL7k9yQBFiJLA/m5CFOp6a5ED0ZoVizfYXfLPMQDMUdPjdCg/0ignGg2GG7AjFTDQKuzkzYsY0zr77T/6gUS6nakVKCKOQ22TZGxDryQCkkxmfKhcAK8loroDDNpWmhqokHgG9Z39IPrIKAdQihzfKVAMGcTryhO+JReDLsO5QT6+Ect3mKEOJ+BgQtg7pOXCkN8Cw+sA1cfbEqZmFRONZ4H/J+AezjSbT5pSXw62HsBZrIp/2YnQsn1FCk9pibRfl6q4FgMRZAIrZvZKh5ZGCmC2zpHyy97QC2qFiOPE7L7HFLFFrZp+3rZRRD4TkQiyS4HU7AA4zIu2pdC5qBpL7qS+TZSd6V3VHzxwUZIuHLTiMY/OrG9ioMUeM8O/WTGpu/A1jsply8fAr9gcfIwZ8HxeqoGJLl6av6qiKjWPcQphUzNYymJsgjP2kf3TLLfLe7qyNnRHU2BWdoYToqmM4/+ae9R8jGIgBOxN/Q/a+i/aSDRgDPl93EhMvUeiC8BXyUaSNDTW9Nh40CDPQUivPd3S+kZZ+p5jRw53OVx4SEI3NA4xaDK8iJ0r3Nchxk/XaDHeQ/h7v9Kp6tAap6VNeJf9Zdtsucvqd2OG8gdX8QqwVINvirD9A/bxm2dhn4sGA1voEOi0sKE0+6bpujM X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa179763-44e5-48df-9aa4-08dd57a87c22 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:11.1195 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +rBgEosXjI87aHKmpk9XebnF/BLjdYPf3y3kC1v7vQJY228rwd2lo6fY0sc9/e/Vnfw6+bJbUbYTp2JspuV/tQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 78DFAA0006 X-Rspam-User: X-Stat-Signature: jwj8dtjyg7kb6t8q5dtdwuxu1mpajg4o X-HE-Tag: 1740713603-289477 X-HE-Meta: U2FsdGVkX19roxFhEHYjCMycaaS1SIHxSSVlG2VFIYjVCaDF1qBieHXZJsXeJei6ZHIQdTJrix1/md5ehVGhHg7lN3lN1XtjSR5KDzmEDJYPrJwNNAHDjQ9W2640ZDrdwQfnvtboREq+znA32mJ90K+1/wrgJrVIG/9e7hYVTUAiusnpkPav+AaKjScLP+FlFz2sPwfM1V4qurw5araPeGmcNRix3Riati9VNuWs6hkcfOsoeSYIkmzaDB8QpReN3uygpdV0ZTZCf45pvL8qkkskrwwAT23Ic2uhNao7wO+47a99Zd2Tu9ghRxkYlHjfsPcJsALM+RDAUOF+3mUKdOnWGtVSiLA3ZaPi8BrwdO+X6WiSZ2kxMqoyzHUjqodPfwI3Dggcjbs3sF8E79UFc5Afeu7hZ3atUlq7Jh/WenqrTsGI3YVYSid4M1WsSdfSX1VB7yZwISJpvOOzdph+1mFTryGBVbjwci+f5rT7UbeZ42BpNjP776S91yOzW/OWlBhdUOTsc4Nke2JnMztxCSLdC1B4WBE0AM7OvXmRKV9IL3FHuJSu2kkqFcldebRd3UjwdM9/idLybgJXqS0Q7GCI2F0w7vBgMisk3tMVWiKKcH3LAQQObENM6hijp+nO/BEp8dWycZWYCw5nrLm+xx6COFUeW9DJFJAgU+UpfoNjw2kvyRN3fcnzQlkLT+VKr7sLhBabomWDyywqCmknsVjOt0YpqqesWDLZt0AXO/ozS01XXGaISlbrq/zQqplnXqlNxwfAnlMVSf6SP95iyZeRjQp+hsO2tze6RE6Yr3O54zFib9Xy79qQLDeDQlM2EAGp6FnLrBUXSTntuOhwlEnm7ame//6FC+Ky27qj5BIZqYbMXQ0mqumx5Fn6BzkQty9PFisiX/uDg5OpLPQ682jre0xhJXYTHrpjOBnpDQxIrz6fVXMmghAxI5rsIdHb8BviVcjLK7BeHQqNhX8 We6dIjvE uJvlHY4uLaNOaysmfwBsQT8NaofVgKz+F7IUHvJv4oia0Q8xFPHCLFzk7NAm1JKxO/7ek5mXGxlWLEAV541mMthmIoOPphH31nwHK5m+ck1H9Pa3v7q5OHkCxO/qmww9M4ooXJFchqt16h2IlJFr2nhsmjgJVVm9vi0UoKlvhv75BDymM1lzyQnA8FmnSKMR5DZMufTKOLI3d7/XoCuEFZp5caQiBbSMapC/oe0sFAGR9YnFxoQgQrVKp1vjFiB7AUU7B7cTXxritiwnobtsk+APlRhhlOAt8SVBpwYHjr6hS6XpFJB8BCt3Jqh5UgwxOrG1FwN1vqvIs6GzHKyhKm/wmcebxKKiCTHQnIwkRYeuvNKGBZDd4lfjpo7pKJ5+qnQS/TPkMA9g8IYLUXN9iao5PQZeXxMo773xHCOW33wHMExwZYiD3/8PRKK7RkdNN+wyQebikdiSGuAfVcmL8FEdwsCWpDssNUEQexU/yNYfRP64gdR30MOCR5PE7Gem0Uii1 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently ZONE_DEVICE page reference counts are initialised by core memory management code in __init_zone_device_page() as part of the memremap() call which driver modules make to obtain ZONE_DEVICE pages. This initialises page refcounts to 1 before returning them to the driver. This was presumably done because it drivers had a reference of sorts on the page. It also ensured the page could always be mapped with vm_insert_page() for example and would never get freed (ie. have a zero refcount), freeing drivers of manipulating page reference counts. However it complicates figuring out whether or not a page is free from the mm perspective because it is no longer possible to just look at the refcount. Instead the page type must be known and if GUP is used a secondary pgmap reference is also sometimes needed. To simplify this it is desirable to remove the page reference count for the driver, so core mm can just use the refcount without having to account for page type or do other types of tracking. This is possible because drivers can always assume the page is valid as core kernel will never offline or remove the struct page. This means it is now up to drivers to initialise the page refcount as required. P2PDMA uses vm_insert_page() to map the page, and that requires a non-zero reference count when initialising the page so set that when the page is first mapped. Signed-off-by: Alistair Popple Reviewed-by: Dan Williams Acked-by: David Hildenbrand --- Changes since v2: - Initialise the page refcount for all pages covered by the kaddr --- drivers/pci/p2pdma.c | 13 +++++++++++-- mm/memremap.c | 17 +++++++++++++---- mm/mm_init.c | 22 ++++++++++++++++++---- 3 files changed, 42 insertions(+), 10 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 0cb7e0a..04773a8 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -140,13 +140,22 @@ static int p2pmem_alloc_mmap(struct file *filp, struct kobject *kobj, rcu_read_unlock(); for (vaddr = vma->vm_start; vaddr < vma->vm_end; vaddr += PAGE_SIZE) { - ret = vm_insert_page(vma, vaddr, virt_to_page(kaddr)); + struct page *page = virt_to_page(kaddr); + + /* + * Initialise the refcount for the freshly allocated page. As + * we have just allocated the page no one else should be + * using it. + */ + VM_WARN_ON_ONCE_PAGE(!page_ref_count(page), page); + set_page_count(page, 1); + ret = vm_insert_page(vma, vaddr, page); if (ret) { gen_pool_free(p2pdma->pool, (uintptr_t)kaddr, len); return ret; } percpu_ref_get(ref); - put_page(virt_to_page(kaddr)); + put_page(page); kaddr += PAGE_SIZE; len -= PAGE_SIZE; } diff --git a/mm/memremap.c b/mm/memremap.c index 40d4547..07bbe0e 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -488,15 +488,24 @@ void free_zone_device_folio(struct folio *folio) folio->mapping = NULL; folio->page.pgmap->ops->page_free(folio_page(folio, 0)); - if (folio->page.pgmap->type != MEMORY_DEVICE_PRIVATE && - folio->page.pgmap->type != MEMORY_DEVICE_COHERENT) + switch (folio->page.pgmap->type) { + case MEMORY_DEVICE_PRIVATE: + case MEMORY_DEVICE_COHERENT: + put_dev_pagemap(folio->page.pgmap); + break; + + case MEMORY_DEVICE_FS_DAX: + case MEMORY_DEVICE_GENERIC: /* * Reset the refcount to 1 to prepare for handing out the page * again. */ folio_set_count(folio, 1); - else - put_dev_pagemap(folio->page.pgmap); + break; + + case MEMORY_DEVICE_PCI_P2PDMA: + break; + } } void zone_device_page_init(struct page *page) diff --git a/mm/mm_init.c b/mm/mm_init.c index c767946..6be9796 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1017,12 +1017,26 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, } /* - * ZONE_DEVICE pages are released directly to the driver page allocator - * which will set the page count to 1 when allocating the page. + * ZONE_DEVICE pages other than MEMORY_TYPE_GENERIC and + * MEMORY_TYPE_FS_DAX pages are released directly to the driver page + * allocator which will set the page count to 1 when allocating the + * page. + * + * MEMORY_TYPE_GENERIC and MEMORY_TYPE_FS_DAX pages automatically have + * their refcount reset to one whenever they are freed (ie. after + * their refcount drops to 0). */ - if (pgmap->type == MEMORY_DEVICE_PRIVATE || - pgmap->type == MEMORY_DEVICE_COHERENT) + switch (pgmap->type) { + case MEMORY_DEVICE_PRIVATE: + case MEMORY_DEVICE_COHERENT: + case MEMORY_DEVICE_PCI_P2PDMA: set_page_count(page, 0); + break; + + case MEMORY_DEVICE_FS_DAX: + case MEMORY_DEVICE_GENERIC: + break; + } } /* From patchwork Fri Feb 28 03:31:06 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995544 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id E69F3C19F32 for ; Fri, 28 Feb 2025 03:35:30 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 7DFED28000E; Thu, 27 Feb 2025 22:35:30 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 76801280004; Thu, 27 Feb 2025 22:35:30 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 56D0F28000E; Thu, 27 Feb 2025 22:35:30 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 34704280004 for ; Thu, 27 Feb 2025 22:35:30 -0500 (EST) Received: from smtpin28.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id B2AB11CC603 for ; Fri, 28 Feb 2025 03:35:29 +0000 (UTC) X-FDA: 83167938378.28.1A09C77 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2053.outbound.protection.outlook.com [40.107.237.53]) by imf14.hostedemail.com (Postfix) with ESMTP id D0433100007 for ; Fri, 28 Feb 2025 03:35:26 +0000 (UTC) Authentication-Results: imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=H4yB8pmI; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf14.hostedemail.com: domain of apopple@nvidia.com designates 40.107.237.53 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713727; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=srcf6zqUgBIxokzx1McHz9sw+qkphK+rkZW51E1Zzvc=; b=vUk2Nt0YkHcJdFxHt48iAi2gGUBOz9+dzzwLTn4SDVSJerMYUtMCBfAr0VlBULo4Y0X3pR CR2UeinfmRiVll3CgzGuv7NJMGQeAAb77gmsQ6XyrfVgLUJqHq+LHgo2y0BPkhQ+sMtsmI mT7JK8adl10BHZYYJZzuMtyIAO20SAk= ARC-Authentication-Results: i=2; imf14.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=H4yB8pmI; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com; spf=pass (imf14.hostedemail.com: domain of apopple@nvidia.com designates 40.107.237.53 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713727; a=rsa-sha256; cv=pass; b=3DAntQENjFMrYiN0L8JlGfDAuO9a4Dz4T8BPIKT7dSz+sLRq5p1NxLuLBBhDFJEwmjH4hB CXm4KI+ydz5k5LEm+hjh/v7CjHiJlVzobgKZRyWWvsI3prufg0g9XPePFlwfX8Ib7f87fn 4PMBEtvRlGYWJwAAy0w50V9hN1/elIM= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Ro9m7QpfBUirFtQjOD4KkVA/NQsexHw+Lvld5ziEh+HqicNP1FkIEnqluc7YMmbB1M7Es51MLFaD+7y/wXNI568VGX6+ecfOFpYVmCguur3UIdM58x6iGxYVFGD+XVfpxPhXXGmyMCsHgiVx2sc3V6AK0F209KKov8q1zy9LxSMfvUT4OQys8F+r01Xp9IKzDd9RhVkVn9/+ORa0p8EzKsBYkN+TJJ2DjRC7DuGMfNIM4uriG7m2sfQ1Q5wV5gx9fRD8xNTJsZTU5XODXBkYfXzpfGoC6+lcejqJ6FyeQXsZEQPQNaeryzKDc72l8vknwqGkn1OMKWM6jbdtVbJKuw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=srcf6zqUgBIxokzx1McHz9sw+qkphK+rkZW51E1Zzvc=; b=Ob2VsRABf05ySI+oQwGLanXKUxT908tDW8q7OrrS2X8ra+cjqQVuVG49xLI7uDd+yFhEJ/IKpx7DQxF9+MBL/N+4fDPSuC2sUDocs9Feaz38tzzNvy0Yb3GV0RPEOpZLj6dSmJ8P6L7gd8zkFrBYy3ZXuG3ZMNSkwTFK48CNqkK9/IhJw2hg3oYMLqpXAffzmpmCv/dc6KFPtu6hd7SdCvhFQRz8CQn/hAeZcGGV7UtNUpmlRDRQmx86KstJiYqrrZ9CFm+AWhyloao+r/TmxzvOe/hui3so0mKX8LJazer9nXoEv0xoYzc/qEUGSbIjVYrvPQaSjmc/JtvVOi8T9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=srcf6zqUgBIxokzx1McHz9sw+qkphK+rkZW51E1Zzvc=; b=H4yB8pmIMsasBWtvDEso2Fwkvuq4lhh0jD5Pp9sxLL8MmAy6Hi+O8T7n8F/bU89IBo1DEnIMonqkm63Hn7z5bfuG/+pt1t6aF9LYIDP7XWcKgW4Aavru67xe9HWNW3jO/it2Fnb+nwZPd1jkbKvGSv63w4o9PD0Q42jR1RGDtoKbBjx/KzbFsGDsDs+Lss6Dok5HCoU1gUYUOu6zNBTd/i9nHzENxwijv4BydCAJBWppG9NyyF33nA4tEGuJpkYvkps1FD35JE4IMU2l0R60+79X4kXZgsGU1tv9JhSyBLnqOaTNA4HywYyqYzLydv4JjM7oDGNPlI/K2krwUHTYMw== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:16 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:16 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev, Balbir Singh , Jason Gunthorpe Subject: [PATCH v9 11/20] mm: Allow compound zone device pages Date: Fri, 28 Feb 2025 14:31:06 +1100 Message-ID: <67055d772e6102accf85161d0b57b0b3944292bf.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY6PR01CA0113.ausprd01.prod.outlook.com (2603:10c6:10:1b8::16) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: fe6f5feb-6072-4cc3-b92d-08dd57a87f73 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: X2V4PFIPUuKKc23dkRIOEPAO+q3b/wUKemcU4sMH+QDvQyugPb5nryNzpcAmrOZLAKNMmon0WCSCr2n6L+UPMexvIBS8YaHKV1Ojo+D+cgZXp+9MVuMXuA2/FmuAXfEsfDM0Wh1vkGg/KEenVLBiey9b1Xq/c4zr/AaTnu5QGXmYcmiFa2xoxGxGZmMy2p5S6PyYAJb0g8Su6NNUm6PbY8LGV3zYMPQp88yOPubNHQGcwODB17alNfkFhbCpaisk2+SanJqVh7LCHu1cSPpMmlRTc5NajD9U/mB/lbKtICv0utOW/WVVv4kYjEqbVR3TIx8KRE6fn5LYcbW44VPQpK261tmM+paD1Xojc6Z8zMQDaS04rp/OyjcUxZXPA6kkGCBm61rm+nVu/BochkJHnpVY5ZnODxGKjmg88mRlRjVWklFmhBr/LGaMT1N9ZllniQNQ06VzhGCoqc6erYqeNWsSaV1CmncXfTLssHHTR+Qzuk8wIGWJnyvR3GjDIrhZpPDXDzvjuGzPCNBSXY3J6/0CjkoOScZVTgCsxebRW6jjmTvWTqHIvUXnSp2KkWpnN86bON38PYkjHzm3ergjz9HfF7u156yh7LiKRRHp52WIV7s5tXo3zHvrmsMVMj+F8R967F8fzuRqLKxF8ROhaTpO8WN4XMZqt8u0maxmGpVXKDPdCwcyalpDxmA8sO5GtyLIUxMU8WK4gXLZ+YJUG4z0JYqiDP06IHSo95vIEgCk0cjwagumBhadt5rZkF34Gky0NAW+Ybfy5d6/BEoJXlcotF5SmaFaEY1W5sYZFcLOy2vc1KzACazBKnlRHXnbsHkiD30i3qdmUu+CqNJuz+bFmQN+QN6C/itz9JVMfSiOARJcTcs4xr9PK1W3OxV6hygdYmxxF4vf4GPNxDbTjqOgu+27YxmD/DDvhNPuJTbqM9Qr4Saju4NF9OoIRR5UF7iUQx1fmKcECWaeHOfbMrbemqV/UCklcXO237Vp62sRfdaJI9hrPbmcaLoLkvrG+T+ZiJqFnNN5PsB80NRclCADLLTYrCMcB9LVxiyT3X1vdloy0inXshFQ5MOIK5/dL22wcvoX0INJqsgLXgsUyoSB65Ftn912E9NaSnx7bm70cpHWtgP2RLmGHQz5dsRq4Hl4aD+F33Hm2VDDaBrDypo/ogD2yXX+IVc5+E4ZsVMhr83dfNJ+ljTZfDMdbHm0cgeyzSyNt2UbxwQPGG478W+1UcpUoJ+2lPTfiBZQgZn94u/10NmpxG0au1tjN4DDPchgCAkG+2xxkf2ENovbjzX+H6QyZiUt8pc2XeZWx/AgZGFQ9XBHKFyvAA/TKf3wWWI2EzfIq3YM/CJKQxRj9ChhFFMNuflpvkDWyE95fx2gtR8EHHqQOSeKZnVos6Zf X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: tM9tWhUK8swCkfP6lHXkc2EgkLvhhi1dF0M+fClT16W9BwXPCHqlhaXvUO4J+e568GbDEgBlNkoNSZNiwutsEVy0+PoGT2UC3aP5j2DWmGjqpqnfRbHrLO3riw44uu6E5BK4BuAL5joY4cPddQ6Cmh1IZwPZPUZrZFOuVE5IreOAOfyPJKyVKQeBIs9GWtOV+Uv+LWVrtuh1FD79pnFZRtcAO9FInAPoqLWZ3G0CbEofN2D0+My8cMCWDSbyaqvpSInnCOpekqt67iSqjdExX3QPAeu+H3KrUNN/vFLKMF3MFntEuZXuKjJd5ncvjWImbZ4B091eLMIt/eP6Vo/YozdtElqairFTAkckrPqoiiqfxiohvgDJB1xFI1s8N8BBZUc+u/XIcrCjhVO8dMzhhBCOKJKKvPO8c6EMG6VhHwyXRiCFVOtxoXyvQibbnkHXCleUMJosqw8NAEus8VNlv6AzFZTLwprxnCjslzJr9QUKJlPyWsvj5N5o8Yrs9g49ABr8LpgiXXgGMF2jPAhRqePzERb1NkMeJ/MwYgTZhRVmQobywZmQDDXORUCuSvX24EVYIhjGbss1UAeeMRz89G62E+KvH9EfkOogZuAl6rwVDuJQFbd7kIFAF75KHKdyX7rFJ+CUzv9tEOMb4se3PW4ZvBo8TTMTpBmpJWEbmOLp59OMvtGa4HsAxlIOeprKWjjIUXtC6olDRvY2jr0SxtezntB8ueEHOqBsEAkOuBIpGrZQYJHCwd+8mRmfRoHFuPwmIPXKw8NsISe2nb79kg+lXkfPSkFHM8O4J8hG67dT3RrsTfMJlvFPRiTKkh7GGEowhBhGJnINDjzp+t/kN96sdxPIoYdZrEbl/eSm0tnPBlrCutYTCY+dy44aty0AZT2/6NQ4Cc/Dxoz7MAv0QqLB3v290S/j1TYPqEH670xO060EET3X+tVvfuXILZGix6zkyGREA6zUfL/0+1/nCYt0UIE7ERPWO6yBZfNbKV/LhO7KchNPUvTtxm2j+R/+paJ4/1CRAnAnh/oKgrnJ8P9t9aRSRxUTkzSocyX/fQWqtC7lyldDGBJuvLmmfQZxMcoFu8Dhmq0kZrYK4mdJSgDDzJqDYFTUdC8Omba/3aOXMmTpMZTnnV4JpGf0JNUyNesRLwWMQGj5ew8qvj9GZd2DLQUde4TxOEv9vZaHimmUw2Qp8sZIaouW1CLQ3PHo8+BuogiPMDsjSz2VqiNxBjgpYGaHjsV7JA9X9cqiBQG+aHC/0OkoHF7T4ynVJ50NU9eVfoK9ENlfzIYNVLZpSkVIucM0VgDmTgA0rAalizcquWbvNcGt4CwgGz9mGzMVppqOcX8VCMNN3/P9U8OASDHHlN+b65lZSoCjMQYS3dtbDDqocToUs4NUg+hMa+I+9lmMT95/9t+RTwnnkKm7ES3RKiA5g7BeCwLHO7Ad68AEPO+AYOTQn/xowKW+EJJfF2oay5hMfVVZmuePTIDLw4vT4ap8uWGO+SBBvNZkfDX6s/OWAzxh3LnclHzQU2kJSTA4T08OCSpUqP2NQcPOJN5WJKUzVHtRGol7GREyyNd97vUcmRAptOkgxZLkrOrL X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: fe6f5feb-6072-4cc3-b92d-08dd57a87f73 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:16.4529 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mKnieurFBl4CpkY2f1l+AxX3Gaeftkb8i4nBevmquQxk296cqNIWaRMpkM7AACzFXgck2EukhMucEtpB6sQpXg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: D0433100007 X-Rspam-User: X-Stat-Signature: 1wt3aob87ykxdm8bf3mur7e8yeuejazq X-HE-Tag: 1740713726-214013 X-HE-Meta: U2FsdGVkX1/S0Rzkzk9jR5STbwFbhB7/n6sG402y28lgSAL8CmqR3BoCQD/d7ZLBdUfPAQC/H0+9i0bIk5QbIsRNsXIsHU8bXhelUPEo3gYEIeyM2I7C/N5Tg9yrwikCR07dlfPcZvy0kLm8Pfs3XsfviKWI4/NakHmWeesKFjEcSWpr1YnYdlqIkk6vPRhYM9lCRXK+ow90r4hJ8DnNOQLPlRLRaD4FtAdCFEslENOEzsRyYpA1l4Ft1YM6fWqeb7ppqF/v47LdMVZkttIXEnxmfOV04Q4UbfUsLoAnpBd4Wi8SMDnka0i3MbbgHqT5APeNSffvGh+DWUOcYWXmxOQ6mGnYCIptgudLNpfQ0Q7RRYoHbCZtecGbEKvN86dkp3NO48cYutCbALuAskKTbdvSZ0Ihs5EdyOWeKKcjmmwPw43aI3TNAMY5al6WVS3bGiERCmQllbdulTR718xZpveeYUGZeFrKhgEKUS/ItYwzGbo1l9RmWdbbtb0ZN011U93gDHnDL30YcfGwoFBed9u064s/QvysAQBXm6KngA60HQx4/eKSWxmRQsTxHBPkPIPAsrNzINITHBuRUAaLaXA1GHbHYrlR9Ugs4/GdkFtH8VihIOhT8YYtFbFcoD+OWuqKVEe8mh95iOa3vsjcyILgKXLgl1IKorQw2CmHR65dfMTdoDYiAeyDbIx9wAWpNXcn/7yduehBp2cUikhOhxFoyfCPJ7RQgRss6ufQ8jYNegvpa80u+o8rbHeWfBVRDg8pggjc0GlgSCbmt0TobePJ6TEiIKXItJMrfdeNGkgGF67fNQGCCtBEe5r452gR0kKkz6dMgv0//YAt8sAuJ9l+l99JZuIXvuoPfmdixs94zjAYnatFMlnoO3/xZ+U5jWfJ+OjPK7uQm+ZGErrpJITkr7RE7wvvaYpZRPzabL/wHf61LMekLlOUPsb2rJEkd2kdIkrEWPwJVBf0sT0 Sail++Q/ fFvb60QS8pQCe+jub+ZTXmTxiceqOsnFSNtCrhmZssc8dHkfjIIsnEknM8dyuNNU7dS03iRdgC8uk0xZwQQlt8fjLe8Qfv9B4A8j3TDon90TpUm5B9UlChjcUmxBNRiEg1T1Tky5AQ4wqq38dEBPgnVXheZB3T2XHHV62nH2foi0b9A7F2wnuPKvWoLlmr4Veo3E9NZ7+8FAoIj8Nzmf0guAnWSb5pMNLYvPgBPXMlk64nagVQoK4+0jP9MuJ4IR2whouaBxSjrfgewkti9IjjQ6YULIYY0Hcy/z3YkPHbQm0xtZhqFrp6ZBng76DKNteCDEOOyIqIbn+m48gvgbH67CQ/6LhXrO+97aHPAOzVame62idX9e1VBytpHR4RFhnvtIZOlMrbsk6/Fc2GRfteif3wWIhwZUUb3PL0iPHBgUiaPil+IrcnxO7UzucNeZvqWGiH8Rupo6gbYI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Zone device pages are used to represent various type of device memory managed by device drivers. Currently compound zone device pages are not supported. This is because MEMORY_DEVICE_FS_DAX pages are the only user of higher order zone device pages and have their own page reference counting. A future change will unify FS DAX reference counting with normal page reference counting rules and remove the special FS DAX reference counting. Supporting that requires compound zone device pages. Supporting compound zone device pages requires compound_head() to distinguish between head and tail pages whilst still preserving the special struct page fields that are specific to zone device pages. A tail page is distinguished by having bit zero being set in page->compound_head, with the remaining bits pointing to the head page. For zone device pages page->compound_head is shared with page->pgmap. The page->pgmap field must be common to all pages within a folio, even if the folio spans memory sections. Therefore pgmap is the same for both head and tail pages and can be moved into the folio and we can use the standard scheme to find compound_head from a tail page. Signed-off-by: Alistair Popple Signed-off-by: Balbir Singh Reviewed-by: Jason Gunthorpe Reviewed-by: Dan Williams Acked-by: David Hildenbrand --- Changes for v9: - Fixes from Balbir Changes for v7: - Skip ZONE_DEVICE PMDs during mlock which was previously a separate patch. Changes for v4: - Fix build breakages reported by kernel test robot Changes since v2: - Indentation fix - Rename page_dev_pagemap() to page_pgmap() - Rename folio _unused field to _unused_pgmap_compound_head - s/WARN_ON/VM_WARN_ON_ONCE_PAGE/ Changes since v1: - Move pgmap to the folio as suggested by Matthew Wilcox --- drivers/gpu/drm/nouveau/nouveau_dmem.c | 3 ++- drivers/pci/p2pdma.c | 6 +++--- include/linux/memremap.h | 6 +++--- include/linux/migrate.h | 4 ++-- include/linux/mm_types.h | 9 +++++++-- include/linux/mmzone.h | 12 +++++++++++- lib/test_hmm.c | 3 ++- mm/hmm.c | 2 +- mm/memory.c | 4 +++- mm/memremap.c | 14 +++++++------- mm/migrate_device.c | 18 ++++++++++++------ mm/mlock.c | 2 ++ mm/mm_init.c | 2 +- 13 files changed, 56 insertions(+), 29 deletions(-) diff --git a/drivers/gpu/drm/nouveau/nouveau_dmem.c b/drivers/gpu/drm/nouveau/nouveau_dmem.c index 1a07256..61d0f41 100644 --- a/drivers/gpu/drm/nouveau/nouveau_dmem.c +++ b/drivers/gpu/drm/nouveau/nouveau_dmem.c @@ -88,7 +88,8 @@ struct nouveau_dmem { static struct nouveau_dmem_chunk *nouveau_page_to_chunk(struct page *page) { - return container_of(page->pgmap, struct nouveau_dmem_chunk, pagemap); + return container_of(page_pgmap(page), struct nouveau_dmem_chunk, + pagemap); } static struct nouveau_drm *page_to_drm(struct page *page) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 04773a8..19214ec 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -202,7 +202,7 @@ static const struct attribute_group p2pmem_group = { static void p2pdma_page_free(struct page *page) { - struct pci_p2pdma_pagemap *pgmap = to_p2p_pgmap(page->pgmap); + struct pci_p2pdma_pagemap *pgmap = to_p2p_pgmap(page_pgmap(page)); /* safe to dereference while a reference is held to the percpu ref */ struct pci_p2pdma *p2pdma = rcu_dereference_protected(pgmap->provider->p2pdma, 1); @@ -1025,8 +1025,8 @@ enum pci_p2pdma_map_type pci_p2pdma_map_segment(struct pci_p2pdma_map_state *state, struct device *dev, struct scatterlist *sg) { - if (state->pgmap != sg_page(sg)->pgmap) { - state->pgmap = sg_page(sg)->pgmap; + if (state->pgmap != page_pgmap(sg_page(sg))) { + state->pgmap = page_pgmap(sg_page(sg)); state->map = pci_p2pdma_map_type(state->pgmap, dev); state->bus_off = to_p2p_pgmap(state->pgmap)->bus_offset; } diff --git a/include/linux/memremap.h b/include/linux/memremap.h index 3f7143a..0256a42 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -161,7 +161,7 @@ static inline bool is_device_private_page(const struct page *page) { return IS_ENABLED(CONFIG_DEVICE_PRIVATE) && is_zone_device_page(page) && - page->pgmap->type == MEMORY_DEVICE_PRIVATE; + page_pgmap(page)->type == MEMORY_DEVICE_PRIVATE; } static inline bool folio_is_device_private(const struct folio *folio) @@ -173,13 +173,13 @@ static inline bool is_pci_p2pdma_page(const struct page *page) { return IS_ENABLED(CONFIG_PCI_P2PDMA) && is_zone_device_page(page) && - page->pgmap->type == MEMORY_DEVICE_PCI_P2PDMA; + page_pgmap(page)->type == MEMORY_DEVICE_PCI_P2PDMA; } static inline bool is_device_coherent_page(const struct page *page) { return is_zone_device_page(page) && - page->pgmap->type == MEMORY_DEVICE_COHERENT; + page_pgmap(page)->type == MEMORY_DEVICE_COHERENT; } static inline bool folio_is_device_coherent(const struct folio *folio) diff --git a/include/linux/migrate.h b/include/linux/migrate.h index 29919fa..61899ec 100644 --- a/include/linux/migrate.h +++ b/include/linux/migrate.h @@ -205,8 +205,8 @@ struct migrate_vma { unsigned long end; /* - * Set to the owner value also stored in page->pgmap->owner for - * migrating out of device private memory. The flags also need to + * Set to the owner value also stored in page_pgmap(page)->owner + * for migrating out of device private memory. The flags also need to * be set to MIGRATE_VMA_SELECT_DEVICE_PRIVATE. * The caller should always set this field when using mmu notifier * callbacks to avoid device MMU invalidations for device private diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 369f76a..6f2d6bb 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -130,8 +130,11 @@ struct page { unsigned long compound_head; /* Bit zero is set */ }; struct { /* ZONE_DEVICE pages */ - /** @pgmap: Points to the hosting device page map. */ - struct dev_pagemap *pgmap; + /* + * The first word is used for compound_head or folio + * pgmap + */ + void *_unused_pgmap_compound_head; void *zone_device_data; /* * ZONE_DEVICE private pages are counted as being @@ -300,6 +303,7 @@ typedef struct { * @_refcount: Do not access this member directly. Use folio_ref_count() * to find how many references there are to this folio. * @memcg_data: Memory Control Group data. + * @pgmap: Metadata for ZONE_DEVICE mappings * @virtual: Virtual address in the kernel direct map. * @_last_cpupid: IDs of last CPU and last process that accessed the folio. * @_entire_mapcount: Do not use directly, call folio_entire_mapcount(). @@ -338,6 +342,7 @@ struct folio { /* private: */ }; /* public: */ + struct dev_pagemap *pgmap; }; struct address_space *mapping; pgoff_t index; diff --git a/include/linux/mmzone.h b/include/linux/mmzone.h index 9540b41..8aecbbb 100644 --- a/include/linux/mmzone.h +++ b/include/linux/mmzone.h @@ -1158,6 +1158,12 @@ static inline bool is_zone_device_page(const struct page *page) return page_zonenum(page) == ZONE_DEVICE; } +static inline struct dev_pagemap *page_pgmap(const struct page *page) +{ + VM_WARN_ON_ONCE_PAGE(!is_zone_device_page(page), page); + return page_folio(page)->pgmap; +} + /* * Consecutive zone device pages should not be merged into the same sgl * or bvec segment with other types of pages or if they belong to different @@ -1173,7 +1179,7 @@ static inline bool zone_device_pages_have_same_pgmap(const struct page *a, return false; if (!is_zone_device_page(a)) return true; - return a->pgmap == b->pgmap; + return page_pgmap(a) == page_pgmap(b); } extern void memmap_init_zone_device(struct zone *, unsigned long, @@ -1188,6 +1194,10 @@ static inline bool zone_device_pages_have_same_pgmap(const struct page *a, { return true; } +static inline struct dev_pagemap *page_pgmap(const struct page *page) +{ + return NULL; +} #endif static inline bool folio_is_zone_device(const struct folio *folio) diff --git a/lib/test_hmm.c b/lib/test_hmm.c index e4afca8..155b18c 100644 --- a/lib/test_hmm.c +++ b/lib/test_hmm.c @@ -195,7 +195,8 @@ static int dmirror_fops_release(struct inode *inode, struct file *filp) static struct dmirror_chunk *dmirror_page_to_chunk(struct page *page) { - return container_of(page->pgmap, struct dmirror_chunk, pagemap); + return container_of(page_pgmap(page), struct dmirror_chunk, + pagemap); } static struct dmirror_device *dmirror_page_to_device(struct page *page) diff --git a/mm/hmm.c b/mm/hmm.c index 7e0229a..082f7b7 100644 --- a/mm/hmm.c +++ b/mm/hmm.c @@ -248,7 +248,7 @@ static int hmm_vma_handle_pte(struct mm_walk *walk, unsigned long addr, * just report the PFN. */ if (is_device_private_entry(entry) && - pfn_swap_entry_to_page(entry)->pgmap->owner == + page_pgmap(pfn_swap_entry_to_page(entry))->owner == range->dev_private_owner) { cpu_flags = HMM_PFN_VALID; if (is_writable_device_private_entry(entry)) diff --git a/mm/memory.c b/mm/memory.c index d337eab..905ed2f 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -4316,6 +4316,7 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) vmf->page = pfn_swap_entry_to_page(entry); ret = remove_device_exclusive_entry(vmf); } else if (is_device_private_entry(entry)) { + struct dev_pagemap *pgmap; if (vmf->flags & FAULT_FLAG_VMA_LOCK) { /* * migrate_to_ram is not yet ready to operate @@ -4340,7 +4341,8 @@ vm_fault_t do_swap_page(struct vm_fault *vmf) */ get_page(vmf->page); pte_unmap_unlock(vmf->pte, vmf->ptl); - ret = vmf->page->pgmap->ops->migrate_to_ram(vmf); + pgmap = page_pgmap(vmf->page); + ret = pgmap->ops->migrate_to_ram(vmf); put_page(vmf->page); } else if (is_hwpoison_entry(entry)) { ret = VM_FAULT_HWPOISON; diff --git a/mm/memremap.c b/mm/memremap.c index 07bbe0e..68099af 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -458,8 +458,8 @@ EXPORT_SYMBOL_GPL(get_dev_pagemap); void free_zone_device_folio(struct folio *folio) { - if (WARN_ON_ONCE(!folio->page.pgmap->ops || - !folio->page.pgmap->ops->page_free)) + if (WARN_ON_ONCE(!folio->pgmap->ops || + !folio->pgmap->ops->page_free)) return; mem_cgroup_uncharge(folio); @@ -486,12 +486,12 @@ void free_zone_device_folio(struct folio *folio) * to clear folio->mapping. */ folio->mapping = NULL; - folio->page.pgmap->ops->page_free(folio_page(folio, 0)); + folio->pgmap->ops->page_free(folio_page(folio, 0)); - switch (folio->page.pgmap->type) { + switch (folio->pgmap->type) { case MEMORY_DEVICE_PRIVATE: case MEMORY_DEVICE_COHERENT: - put_dev_pagemap(folio->page.pgmap); + put_dev_pagemap(folio->pgmap); break; case MEMORY_DEVICE_FS_DAX: @@ -514,7 +514,7 @@ void zone_device_page_init(struct page *page) * Drivers shouldn't be allocating pages after calling * memunmap_pages(). */ - WARN_ON_ONCE(!percpu_ref_tryget_live(&page->pgmap->ref)); + WARN_ON_ONCE(!percpu_ref_tryget_live(&page_pgmap(page)->ref)); set_page_count(page, 1); lock_page(page); } @@ -523,7 +523,7 @@ EXPORT_SYMBOL_GPL(zone_device_page_init); #ifdef CONFIG_FS_DAX bool __put_devmap_managed_folio_refs(struct folio *folio, int refs) { - if (folio->page.pgmap->type != MEMORY_DEVICE_FS_DAX) + if (folio->pgmap->type != MEMORY_DEVICE_FS_DAX) return false; /* diff --git a/mm/migrate_device.c b/mm/migrate_device.c index 5bd8882..7d0d64f 100644 --- a/mm/migrate_device.c +++ b/mm/migrate_device.c @@ -106,6 +106,7 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, arch_enter_lazy_mmu_mode(); for (; addr < end; addr += PAGE_SIZE, ptep++) { + struct dev_pagemap *pgmap; unsigned long mpfn = 0, pfn; struct folio *folio; struct page *page; @@ -133,9 +134,10 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, goto next; page = pfn_swap_entry_to_page(entry); + pgmap = page_pgmap(page); if (!(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_PRIVATE) || - page->pgmap->owner != migrate->pgmap_owner) + pgmap->owner != migrate->pgmap_owner) goto next; mpfn = migrate_pfn(page_to_pfn(page)) | @@ -152,12 +154,16 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp, } page = vm_normal_page(migrate->vma, addr, pte); if (page && !is_zone_device_page(page) && - !(migrate->flags & MIGRATE_VMA_SELECT_SYSTEM)) - goto next; - else if (page && is_device_coherent_page(page) && - (!(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_COHERENT) || - page->pgmap->owner != migrate->pgmap_owner)) + !(migrate->flags & MIGRATE_VMA_SELECT_SYSTEM)) { goto next; + } else if (page && is_device_coherent_page(page)) { + pgmap = page_pgmap(page); + + if (!(migrate->flags & + MIGRATE_VMA_SELECT_DEVICE_COHERENT) || + pgmap->owner != migrate->pgmap_owner) + goto next; + } mpfn = migrate_pfn(pfn) | MIGRATE_PFN_MIGRATE; mpfn |= pte_write(pte) ? MIGRATE_PFN_WRITE : 0; } diff --git a/mm/mlock.c b/mm/mlock.c index cde076f..3cb72b5 100644 --- a/mm/mlock.c +++ b/mm/mlock.c @@ -368,6 +368,8 @@ static int mlock_pte_range(pmd_t *pmd, unsigned long addr, if (is_huge_zero_pmd(*pmd)) goto out; folio = pmd_folio(*pmd); + if (folio_is_zone_device(folio)) + goto out; if (vma->vm_flags & VM_LOCKED) mlock_folio(folio); else diff --git a/mm/mm_init.c b/mm/mm_init.c index 6be9796..d0b5bef 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -998,7 +998,7 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, * and zone_device_data. It is a bug if a ZONE_DEVICE page is * ever freed or placed on a driver-private list. */ - page->pgmap = pgmap; + page_folio(page)->pgmap = pgmap; page->zone_device_data = NULL; /* From patchwork Fri Feb 28 03:31:07 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995538 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 40945C282C5 for ; Fri, 28 Feb 2025 03:33:41 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id AC42F280008; Thu, 27 Feb 2025 22:33:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id A2527280004; Thu, 27 Feb 2025 22:33:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 829B0280008; Thu, 27 Feb 2025 22:33:40 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 5E487280004 for ; Thu, 27 Feb 2025 22:33:40 -0500 (EST) Received: from smtpin17.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 087E3A1891 for ; Fri, 28 Feb 2025 03:33:39 +0000 (UTC) X-FDA: 83167933800.17.9AB328D Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2070.outbound.protection.outlook.com [40.107.243.70]) by imf24.hostedemail.com (Postfix) with ESMTP id 36F56180007 for ; Fri, 28 Feb 2025 03:33:36 +0000 (UTC) Authentication-Results: imf24.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="L9u/dcB6"; spf=pass (imf24.hostedemail.com: domain of apopple@nvidia.com designates 40.107.243.70 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713617; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=yNPEvAsPe8G2yTsS4zF2Ou6m25axDDKUOssAOYWV7/4=; b=8QMV1nvt2wq85k2AY9CHwqrUntrDqtIAIYZMKqUu0wruJQ9gfJJLDhEzh7FLHKaaHFvyde bnNhYQCMzOLe2pNqrTMp/rx6cajWt1LIN7N78EtIChUy4gHnCBXL5s6N88UY8zwKMmr3nY iRWGiC6Qg0efFEMjjM0gl8GS3C5Pn+I= ARC-Authentication-Results: i=2; imf24.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="L9u/dcB6"; spf=pass (imf24.hostedemail.com: domain of apopple@nvidia.com designates 40.107.243.70 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713617; a=rsa-sha256; cv=pass; b=nHZx5Dxoq7Zc+srMglkD5eHwb9jm1eLxOGntwe4pldzmvDwW4mHv8Bb6iuwfPAIBkiaez7 hSXy/2kdI+6z/9Dd/o2FXLTCxjl/tHOjpOBTOqDvP4S4spyGQ8/VmcWeXDHZLawyjX9w/E +Ac4OFN1PMYxBzZh6jjHvgPjUhGckVc= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mBB+97phwKzRvIZZyzhTDI8NhAo/aQhokC9bsOgShYQe/vSqiPrx6hOJ9W/SzrBu8ZkfHZyUoI3ivj6WUEus6j9UvY9K1tDaZDB8vNWmHgBSJg+/K+JrAaefkidv9APUQnyMSUbbtwjIq5rayje+uE5pRHisAnpfeRFn8KVL7EEiT5/RqtIInSh1HorKzmev8ykLtZXotGn5T0TGYjTx6rN40MHx2VzVqpZTgUOHKTRFm27HGPDb5OB5EBmrrIuQFsAhUJvvqGmSMv/dUe7xlMrVQVpXLQBY0/I40Y0jLk4vfRcPIhzuW9pBzfRgpU8Yo+LuNud8Jr1Th8ZIIVuEBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=yNPEvAsPe8G2yTsS4zF2Ou6m25axDDKUOssAOYWV7/4=; b=hoD309mzoTJNaNwIVVjus9IQez8Mx7T3cy/otkBlaqOSgPsKRiQB+4yyT3Ia8ch/oeGvpsgVwqeHMFu5EtqFLr++8XwdGuZNXkyHjL02LOwLFHR9iMaLG/i1Cu5VPkzsZ89Y+u+nZiXG7B92I9ueYzhw3krlOcZ67kvtJ3xLvPA+rmluNOLopiHhuiIroXbZN8tOB36hjnq9PSzJLeXVqqzXycRlfgTU9+IWSWN57BqtSx2JaIoPMiJJqwpyY664YI0/TFq41HoWtlDFNZTHRXCmJmXmsvhuGYeaLFIQbEwBbjXxhugrJGZ6loPrl4lCnSsWiiBnRjF0efJckNTi7g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yNPEvAsPe8G2yTsS4zF2Ou6m25axDDKUOssAOYWV7/4=; b=L9u/dcB6w8plRd9kbTR8spcNO5FMuRTFP9+YahZeMdtlXwmQF8IEGbHlqKb9LTLU2pc5L/Djd7CeHmT/MQIAhm6T9+NG4V3+o1rPfPqzkvxCLPcwwFExEItNDlVCVRLuKigVCDWX5E8F1PT/hvoRJ7E5GDnlXQagZALXAVgDDbiaqEjnFDaJH9kvGNlXExaWlB1ofGNc1GoabrJBq5ec9fuWvc+gpNOAj8X8j9VfpYQPwOBOTSJP485F9Xg2bAhsRa53bz13+fQ50P7XNFApLdHd1bq2WXiiTIxZ/IOOiV5a7L8ZeMNXgvnepXlyMDYAq44oHdek5uulBv9eie/FaA== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:21 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:21 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 12/20] mm/memory: Enhance insert_page_into_pte_locked() to create writable mappings Date: Fri, 28 Feb 2025 14:31:07 +1100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SYBPR01CA0146.ausprd01.prod.outlook.com (2603:10c6:10:d::14) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 9bd29e86-fef4-46ca-0908-08dd57a8822d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: CxtFFC9aCUAnMKS9Hz4xjpv8X/FLsEMac8vKJL8HQTEJAXBWlp3FyW5HWjSHgk0r6NOlho3C3nCLWFQSYIKfmbc4ByMzKlh7dkIu/1N9Zi7OPxmQGZ8I5sRAenX4+ZgzZo+soIPPp4Li+8L/JrgfIoN8yM1KYQocnKFm4gMb72XWizxUoyqRSiueyA0F5zVvA5UW/czYVJc+rO0W8zbUPRla9iv8f21D8hiB8Ygi9uXiq7Emcs0nAF0SEaJrK8agBvxGzz+7m9dWKfNj9homcF/b50hdD/FUkbpfBQYp4KhJuNMRaQV54XE86cGVufJ3VjxIhja7Buxj+wymPbOCDwxR2yoQ+skUvB6jdf42JQH3i1EoafI7zenJ0R037gKGjq2CrqFYNR6wgCKdODyO63Gfap+wG5x7puU/FpeIgx23KZ1qJQE5xdR0luJvVEo8IM/WD0tGbp7r+yUndg4QrNBLTOXwkbh9FBeAZNAZP0tVbVLMU6bmU89q6qqiCEGjL9YWSp7jol9VyFZLHBtHiL3boqRXgeekouRnqFCukdFTQmsU/kVOgOwN2P1pTriuhlSMJbF0veIXFYG/ngO93HzPf0cRA6gi9X84w0NrZpx9UN1MopxeO/UTcGTzBn4/XWHWV2BywLFJ+HaIyVcuqt0Q8g9LOdoPKkIp9zEwtzJRW5QjynG5GwVDMtgLk98OskcMSfLr/wBSkIiLljlYE3pTIC7VxpiY3v164PiWOdRg/XJVwM2gGLGXtyEXL9kKZGRX6gfII9hi9RBXXQe6Ag8F43lVbbSk/3uKJLR93hp/HyW0X/NK0N6J+BZHI0c7IMiRTc2w5FQkgVDiByG1RsvsjlfjVD/rvnXl5KgTt33M7R1zRkfhP+BIaqduCkuduFraxHlsRrdtQADSOMChARQuqE7iX27l/VDzTYP5P8cgJg0/Dozz07tMPPWmBDluPXZ1eEkier/KXX+OsZvd+zgvHXHQxsN5/Z4NyaRzuSfXIO4DnBBXvm+Yz/NEhEFTSzH6SazrJm9HWYSKOOTjVMtbVFSt2fusL2EcqvFZRXT1lyL3LVQw6y+ZRJAbbdo1nyJjcQJLal2HEWpkiELCPAYBR8VatqPEmavWCdVzi5lh8tcIlhTJb+f3eMg3BIlJ4lSxuxulsdJeeNzHiuMYW9/Cz7dxaWwz2S4yXjApYLn5TwCTDf//NTTVqZPimZxR+mzu3HBkEPm08i2FBm8pJZxRyF/s1IsRaiyFhUeJmzE194DGulnvx/51fCK5fApHw1+Uhrl2IohKB8yfgZCDk1oVb2AjLVJN48OpPzXhhZbtD9rJFMJD4sTOyuIMygIFzIWX1g1aN4ax8R9hoazd/iy775ktEmqqHa6vWjNPLvOU5gedgU0WaPuKDNImcbdE X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: xLFgM3yWNTSzfS1255VgCfHHTHATH8X1NRvueVWfhfmvc0Fs/xlWOFUYIiqXMeJOO87zXX54eUaqpJHiJCxVNnas2NyZxfYOa//B+T/+xxggngecONO9OiFvxQ5rN6TQG7aZ2R8HdSdeQRpb7m81Ja3+hLGfBXD/59oxiBFekfkEl+Nk4+gnl+PMx4VvHdoeUjyHLrK25l1iM1cppoVm1aLBXw+eM8+WrTCa4AvxRIDghRBHZ8xDaQ+DD83ZcHJ6WdGbQXsCQp47lz4F8trfhWTxACQ4OY2ohjUclGU4rKSXJx3Ny0eGMfggN4pelDYGW9JOka5fH4FL/C3in4ysRgjmkbeiqlypuzRv81ZY2iHVL/hB+TF2B78NlAq0mqNAHoSJ5E853rEcRup0cPDuql2Gb7w5zwWb8UFgVIKetWXLOo1ZL3jdrmoXOY19yugiw1wGKl/zPlrP/UJJoYB/WqQDJ2zgOC/MRXbgpvK7whBScvlywfHllOpEmA0Mu+OBpcK42WpBFoT1e7pebQWBZ0xpi9GuV6irO7g+ggo9ffQt7+tOanc+CCPhycQ+erPRcbu/YWFUdiKyqp5HChL6SqDAZ5U3hqOJsjjRmnAG+4dGc9O9anmT58yxay1AL/RGaSW6HciNSfa+umVeG4V7e/NWpszwqV07uvponzPl2iDD7vXBNMpKe3bDY4fE4KbFPZZ9tYHz9a2dx0oGWtilJWU09XKVT/Q4ALUO1ku5A2hpDquoZz7XayxBIcusDk8vJExhfxN2VQG7bmCoULIF35A76VCd1/Sn+aat7gDn23bk6tKGYGnzrrYv8lw31nwIUwD4jcxNTy33O2HM3/0HeLEBadcXX4TCYVHJTYtNO/rMlthHhYXcXz/xtjgm/5wmWWE0l7V+qTr6CeIgT3TmmidUeyBwK8LDxFaO3a9RLwzDOxD3DeV29S0QIq5JakiAOu3Ny/a30y94RH/hHsU0DylVhEm2MSSEGFqJXT9SBfQOW2y37BJIc9n5ZIOeyk2HuiIh/3tsp/anuFS1x+YV++lXegGSvkEewjd7r7KXicpwmz/IfNrj1c3T9Alo7PK49xSJSV8fnmoM6ecGQ5sLYXwl7TScDI0VhbYcnnEeCb2v12c2B2OpvdZeDH3IoHbCElTn9Nrqzwc0WheY95MHCXEne43E2i7VVLEPQKhlKKhXI/C1GN9ITlk4Sa8oIcuBLC5bzz7diOqYcdpViRJT5IoZMBw0R8jUwaFKgiKx2qVlX7Zv7A2esDRsUiAq/TKcvczMlRB1iYfItdCx6KnZu1vfbYOLM6UMtkjYdVFht+I16MVrDCp+59ar0rXy7cK1PMP7unVWIAKQrPbhahUIGvedIdY/nw9Vjo78ge4rfDgmsRA7UHXDowg+u8/agWV3nyF3e1P0yBI0CqpnJcHgwWd3TCtFi1/AEih0EOQ3kGnCOYveTm1WJcU+g5II7Phbzxli9Z1SVIbKnt3jkzEcm6F7TsBVILm4IpR9vniWak6k1dnX8lS235aeATOWcH04ExCGwj4g2azJ+v8DqWSbwpxnKLI0aF90uluuNCawA0/g01aHDhq63QbTm5rVxxtZ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9bd29e86-fef4-46ca-0908-08dd57a8822d X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:20.8961 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: lr6H7zMdW/ylEQS4aKCpEL6HkQxzqxLuP8a4ebLDsSxQ086EphR/kCyg4jvAE1icg4I79nrnJzWodrHhdiM0LA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspam-User: X-Rspamd-Queue-Id: 36F56180007 X-Rspamd-Server: rspam09 X-Stat-Signature: ytb3khpbk49wszy4efsf9dbifjew4au9 X-HE-Tag: 1740713616-123685 X-HE-Meta: U2FsdGVkX1+6GLYXjF9v14VUhkhSJV7v09D2L28vKLi+L6bhioww5ncCz/p8D63cp29gJRgir4eMbiVaAefuA1Mur0PJevE2pt16XVfGTye9gJ8F2I5mo4iTGCpnQQE7XHyLAfgYv0dhNyrfSMjPayUSaCWNF4Pa8/TVKa6Ll3hlItNo35E1Kd2kFckUIxENTjsJviFwFcg0CUzt0W0AtAG5GeEb0Mz/hq7dbMGqNdrsDE3tjUCvN7yDF+aSEiuXwGwBCBy0dzQSyU3MBNeHeXCKjbYxhuOP/vLszfMptIV3q2/fqz3LbJ4/NksnImzP0qMbT9CiqOYaWgp99iBeEFRbkeAj+TlRyj10GbDmFOwV1YPiOfe8Llc8XQEEH1bYgFzLVSlgujj81t5++uehTNxX00EGxjjVCUVOwdmiDH+RTR3ezzNRQHACSplZzBOukoJ10djBU9J8gaAIK2pa5YgbL4YsU0Lvr2OoiIXZbjXH1mbrSutdufaBP3yhBxdOKSeVvgO9bfU+oX3eV+mBnHFMH85JuDWyLkm6R1a+SyifON7k75hF8vroHOSLDahPyx0c8WwsfRQwb5Eep3+qx+JTRs9gXhMmqPMPln7aU9wh0N1Dka0sj8KSWn/umbS1eGXm+v/ejoCMdKMXl2peYuHMOUWHVqqNxoGgiVMmpXLcakSk0zT7Q9XOhyKqRQ1hyU4HICPgZ1s05B64D1f/y7Cg1TOfaNJw0do/6zaeXR2xIFl1gJ6jJxRgqey3TsojBhLoirG8pLYkxvD/+pPHgRAJw4dHeg2lWz+K0oydZzS1hCxvzfpUrCfsIEMsbAfQDE2U7U9s6H29vq54AZo0woI2UGuqKPz9CHvBzLWkz4XuPF9HIHPCzVASoqXn3vh4dML0GetowhJYViA5OhyB3DwHD78ooldv0wCZjA8fCC3E6j2MJx8+NYkm0ZbUUNbXGqQu2H0I5E9/B6JlqON jbovYXQB MEJHh4aNg/i4uNH08N18oHc3FUyNhO1HVZj9xQqAwVd+uztOFG805uVJau6Z9iKuS6j0sjytsTjMDURLM10VpSuK5aA+7dEMQ8SZmTqJeRVmpddi557N2cdXAVg0inBPrS+gMvbblsYDSnpCbhKZM7Pxn1IHO36gZpDzInRMqC0p+Vz1ucoe3Ql1Yu1kx4BpDkx0XPnc5vMCejzMAp8hhgvDg8XT0Aisj+AdRVyd3Vw1m0X5gEYv5Kc+F3GW3hBXkyU+egwtfAxm+6diBu9sDsAyI55DV8/QQ5p7EsxVNvJbA5/9eCvnbsMjE/GLNmjewifdPvHkcIJaKC1aWDbZasl8DSmiPSE4tovFfI6+yFlk1DSo6a4LUMEubZFLMJsKIH0PtJEUlOX0983IzZKTAXNYTFOqZ7FX0RoDAn751Gbc3D6ydxVFsTZtA6JfOgvIwvhEg4qnRGqyQa19k1lQiWcrin3zrNiuEbRAkLJmYwwJUTbnrQArlpvWbR6Gme2JwLF9b X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: In preparation for using insert_page() for DAX, enhance insert_page_into_pte_locked() to handle establishing writable mappings. Recall that DAX returns VM_FAULT_NOPAGE after installing a PTE which bypasses the typical set_pte_range() in finish_fault. Signed-off-by: Alistair Popple Suggested-by: Dan Williams Reviewed-by: Dan Williams Acked-by: David Hildenbrand --- Changes for v7: - Drop entry and reuse pteval as suggested by David. Changes for v5: - Minor comment/formatting fixes suggested by David Hildenbrand Changes since v2: - New patch split out from "mm/memory: Add dax_insert_pfn" --- mm/memory.c | 39 ++++++++++++++++++++++++++++++--------- 1 file changed, 30 insertions(+), 9 deletions(-) diff --git a/mm/memory.c b/mm/memory.c index 905ed2f..becfaf4 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2126,19 +2126,39 @@ static int validate_page_before_insert(struct vm_area_struct *vma, } static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte, - unsigned long addr, struct page *page, pgprot_t prot) + unsigned long addr, struct page *page, + pgprot_t prot, bool mkwrite) { struct folio *folio = page_folio(page); - pte_t pteval; + pte_t pteval = ptep_get(pte); + + if (!pte_none(pteval)) { + if (!mkwrite) + return -EBUSY; + + /* see insert_pfn(). */ + if (pte_pfn(pteval) != page_to_pfn(page)) { + WARN_ON_ONCE(!is_zero_pfn(pte_pfn(pteval))); + return -EFAULT; + } + pteval = maybe_mkwrite(pteval, vma); + pteval = pte_mkyoung(pteval); + if (ptep_set_access_flags(vma, addr, pte, pteval, 1)) + update_mmu_cache(vma, addr, pte); + return 0; + } - if (!pte_none(ptep_get(pte))) - return -EBUSY; /* Ok, finally just insert the thing.. */ pteval = mk_pte(page, prot); if (unlikely(is_zero_folio(folio))) { pteval = pte_mkspecial(pteval); } else { folio_get(folio); + pteval = mk_pte(page, prot); + if (mkwrite) { + pteval = pte_mkyoung(pteval); + pteval = maybe_mkwrite(pte_mkdirty(pteval), vma); + } inc_mm_counter(vma->vm_mm, mm_counter_file(folio)); folio_add_file_rmap_pte(folio, page, vma); } @@ -2147,7 +2167,7 @@ static int insert_page_into_pte_locked(struct vm_area_struct *vma, pte_t *pte, } static int insert_page(struct vm_area_struct *vma, unsigned long addr, - struct page *page, pgprot_t prot) + struct page *page, pgprot_t prot, bool mkwrite) { int retval; pte_t *pte; @@ -2160,7 +2180,8 @@ static int insert_page(struct vm_area_struct *vma, unsigned long addr, pte = get_locked_pte(vma->vm_mm, addr, &ptl); if (!pte) goto out; - retval = insert_page_into_pte_locked(vma, pte, addr, page, prot); + retval = insert_page_into_pte_locked(vma, pte, addr, page, prot, + mkwrite); pte_unmap_unlock(pte, ptl); out: return retval; @@ -2174,7 +2195,7 @@ static int insert_page_in_batch_locked(struct vm_area_struct *vma, pte_t *pte, err = validate_page_before_insert(vma, page); if (err) return err; - return insert_page_into_pte_locked(vma, pte, addr, page, prot); + return insert_page_into_pte_locked(vma, pte, addr, page, prot, false); } /* insert_pages() amortizes the cost of spinlock operations @@ -2310,7 +2331,7 @@ int vm_insert_page(struct vm_area_struct *vma, unsigned long addr, BUG_ON(vma->vm_flags & VM_PFNMAP); vm_flags_set(vma, VM_MIXEDMAP); } - return insert_page(vma, addr, page, vma->vm_page_prot); + return insert_page(vma, addr, page, vma->vm_page_prot, false); } EXPORT_SYMBOL(vm_insert_page); @@ -2590,7 +2611,7 @@ static vm_fault_t __vm_insert_mixed(struct vm_area_struct *vma, * result in pfn_t_has_page() == false. */ page = pfn_to_page(pfn_t_to_pfn(pfn)); - err = insert_page(vma, addr, page, pgprot); + err = insert_page(vma, addr, page, pgprot, mkwrite); } else { return insert_pfn(vma, addr, pfn, pgprot, mkwrite); } From patchwork Fri Feb 28 03:31:08 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995612 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2E92BC19F32 for ; Fri, 28 Feb 2025 03:36:58 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id C25DF280010; Thu, 27 Feb 2025 22:36:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id BB022280004; Thu, 27 Feb 2025 22:36:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A0309280010; Thu, 27 Feb 2025 22:36:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0012.hostedemail.com [216.40.44.12]) by kanga.kvack.org (Postfix) with ESMTP id 830CC280004 for ; Thu, 27 Feb 2025 22:36:57 -0500 (EST) Received: from smtpin29.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay02.hostedemail.com (Postfix) with ESMTP id 3BDEC122132 for ; Fri, 28 Feb 2025 03:36:57 +0000 (UTC) X-FDA: 83167942074.29.62AF318 Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2072.outbound.protection.outlook.com [40.107.244.72]) by imf11.hostedemail.com (Postfix) with ESMTP id 78BA94000B for ; Fri, 28 Feb 2025 03:36:54 +0000 (UTC) Authentication-Results: imf11.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=mu+doOP7; spf=pass (imf11.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.72 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713814; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=o9MotQOSY0LVsCQPw+Q39CqMR5NnyIi9vRj90M2RfZw=; b=D46DtxubrPsQYxb99iXh+66iBcOqKPHqGY1xUkf45yze7ZMMtuWnu6qbWm/CiQ5q8fA1N3 rDyql66O80ZdGFJyGAF+B1X4mdkPZyVHnFwz6F61wRwkjDD1N24jmq5iJCZew6AZ3/tg3/ ZimM9zhZbRT7ozicdFPSQoV6v6+6OVU= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713814; a=rsa-sha256; cv=pass; b=7dpYi+I9EjSuja63XVew4rwO7PPPyvct58QY8qu35WZ8R2d71UlgHs+W6UQMm7SnQICBJk CMzVUU2D7VxZ2y/+zWZHd5NknuYLlhQyPFYMdeEmrlR4cl7wKkyPMbEr/u9TW7kUqy5TwS IY3sGwZutiYRiwDB97V/tI777mDBbQI= ARC-Authentication-Results: i=2; imf11.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=mu+doOP7; spf=pass (imf11.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.72 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=mTciidwqJU0jHr52dekYLPeelCAkzIDd8ceMh7RFDgsT2oYQCp1hH40IQ2z2lnVYJx71XtekseL+Z1FR+/9bZxhgtG1W5B6DJpbYbAMmThNwSJDelSWaf0KSkVRlamPVt08mRxn1vkFzXfGkl+JoXw2nCV9bVxWxZ8eBdBprKoWX01/043CtG+9slLW+SVKaclX6m32FcTMqC9TvN8/WRw+dl8aEaaOFQ3ne/LdftR2EolL87LffevwcVOFl6SdntQXHkntx4v3zP7TnoYKYdm5txd0i4oJVmU5SejLmoXbkaFv3AHkA3p8cGh0HuWIQK1shBhY+JfmeQBfcubjhRg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=o9MotQOSY0LVsCQPw+Q39CqMR5NnyIi9vRj90M2RfZw=; b=fwny091JVd0pdEyknQF0uIzlE58GJ61SLNlEa6wQcugzPXYFxoxmNCbrbbSD/QS75Xt0q56OEHUEliz31UGKEFoGgXtnAZzKUGv7LNoEzvg1MifC/zJAa3nCIFAQygrqy/TWEymMNAPex9HcgfPP5q+gfIzPiJCNOEx+MKqn05whll5wjzDv+bXArFWEvavuvhEkwj57pbQ0heBt0qVMyRsB7Abs9Di7VBUt8af5OihkMc+7GSSxN2mTdwlyR7UReFNPxgHj6mH3LV+Ivi0RGtCLf0P+DFlLzp11txF9qFYMCTwr3VzjZYfwC7S0or2OKcHTQd9Llgmhh3//uv+r0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=o9MotQOSY0LVsCQPw+Q39CqMR5NnyIi9vRj90M2RfZw=; b=mu+doOP7uN02urvArLCJ75oOSy+cI/AqZjlh7PvXqjwuxfCYQDkO3bj6oLQ49kF8Lt39U5m2D9GdrQm38/+DpbJpmVWCSiRKmukGF6mtL8UO4NVRK2Ek1fAvr03qAkdNaAA/7/LojPfl4HYYhQDCBOOjcMY8eK53WUo9WfQOzfipCx5f78yb/WiUtVsV/k3G1ZK6+UwnoN+CVMyRTKuJ0brYSoZuBtZqqm7skvMg3CIAHm7W6zHvyMkq4YGlgY9r4LZZbQAboUaWL8rdaPImuvfgAS0hzMmDHWwnY8Q5TuQVef29CUwjd5LCPe4Ti6O78T2FbByvdSTO47bTRTpqJg== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:25 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:25 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 13/20] mm/memory: Add vmf_insert_page_mkwrite() Date: Fri, 28 Feb 2025 14:31:08 +1100 Message-ID: <4ce3aa984c060f370105e0bfef1035869578be47.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5PR01CA0051.ausprd01.prod.outlook.com (2603:10c6:10:1fc::11) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 883ee873-7778-4a3f-3954-08dd57a884ca X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: /DsuOmG4D2FZhQtBclOH7l9I3CTZYOVupg8Hha01e7UGqkltXtsJslrmsR3E/m0K26/NjGAcw6mALzRXmzIrT8/NHlBAXbKI+e25tn/gPyJI//qPqYtHKOOVxqFdU63eRq98b4QHFOIWa/6mRDIa2xblN9w/aAMazNs2OCXjSp4XV7lVMscz4oGDhHAQDSmxeoU6hfpcLT4lvdEYH8/fxXH6hEa3lKZDFjnEq3si7VcC+c7BGSmZmPJXtIb1EfLKe+B3IsaM5kAxOPM52ZP8HqRVrBi7pdoCt9HIUy/JuA4HlXSp3keKwj4ugKBNkAava3vgZjaxJeh41IFLWlh15cuZmaXQkOFKAM9bQTfSx2LF7NXQt3StBBK4/xTn1O7XMgW9SdHuWSmj/UQelEALPS5Orbe07OVJ+MMrdP+UWINQxqxfMSAraEc6X3tlQuvGZD+oThBsW2Qg6K4HCifHq7SAWvi61s9uXa4JKhG/3n417qs7eTxi6Jpsqsm0eBNnO/fUSDmE61sQ1/4YN8h3BylBFJwdi40mXQ1K1+G2sBiGEDJUSE6xMmIYGIOAkzXjqOxmxGo/XIGlJ3eHjJZ+wOzLWXQCwlDUgcrAqR+QuUg78R7mdBWW3wJ4YtQILmoep1CBYKncWeUrWucivayV3iFmBjyfrhSVZYcN80+2WytU7RWrzFY8xz9Is4g+QBPsCd0zQxC9bPcZ2VBTbBGfzj+oXte3UiHZR1KqMHbz6rzHEjNEvoO61DWUTwuLBzMCWxCr1jEx3stU6JcJwOr07et1JVTxN40nxR+YXWovMVIA+91Ihs1XRy1tfh9B5ZylHb9KlqnIcFFJTBO+Zg5VB5omgRriPWxeI/n+GswHWDXFAb7wEUJCVIU4rfcpDBvp+nNOrX9XqGW905trUUhbIlTshsVObje2y38Mj+acSuUpeFUPtkwK4tKnRM+LdMeVxm0FOeDiEVSzbtVb4jBeh+NVa1GZUH5/CFVVJkm1llEl8theEI3stsy8CH+V+IHorsK1y6oPl7grHBTdM1HX8vScxbDd4o6gL774dBXiFoU0oqayq4Edj27bsF4ShcfJqo2fDrQlJ0T0+e7AT4Ods75TGRg2HA+nnjRt/wBvl2k53UUkOR7keTXv5/RqlkV4x7P86cw1UuQnT/lW0lAR/AVrtEBACd4m1Il4D18S5aqb8tTbGrC+YR2yTryHh6NK49+w/X7nckNkGtmMpOiT5KjP/0ys7akijIhVjLiM5feww6v9lDNuEbMseObGA+Y92GAy5bDXHWvklJKY83s3PeEYxcCCNznbqC9u+sa91Q+gquDgNbfXu8vN/QsLm2Jd9MbYHMOZaMqh41ihLXpl1Reqbx+QCmwlPS7lyHaYCNzoASPSKXxDguO748j7c0RA X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HI9RyjoQRtff868b7y8ASyId4HgMAMe3BGnDouAnWzu/uYKM7wQ0EZZSNd/DZOShzghQqSOY2y9JIgBBlCmZ7X+Scn3IkYtuswlDhP+wplkaK6DpiSr3RohplJszNdCVbEOz+g9Z82vry6PShFt6FW6692cEBhbTbHTINwOGJYWryKk5gR97O9+tJKpSZxjGsZQiYaz13DyXTDv54uE7RlfI+IN7CnBmDMe/LVr/wDn16rt0P/2rlBL0ig8TwNOW8OmtcpwazHtQA6Kawn7mlzV5zc5s1fOQQG/9wWOfxZTSPL9Oz/UrpJJPFlQSX4FOjP5CTyoYH1A/9jNO7jMAu7V7b5fFjMdF5WNO0y8l0Ua8xrYmjMnarU4EoJ5Alwgh3YS0dUj/V8kWHm2Wqtpc38SYgKqQVruvnpZWkbpcO3TKXZPIIFVvBucjYj9/EgHJNW7dAqucF6vu1xEW5ZWksrlMaw7tv3izkJC8AJhYTqRXLLuDvtXTvOOmOaNCvJRscn3DnoMMWc1NGa8BS5GHbyQHl1rKEpcVThmKN3zIMW2sLu3AuT8EjgszykpuS3jU0NkAGTH2yM3czyY5+z6Yji2ZkZtTVRX+ZTCxB+o0hZNcozsF9JD+ELSMbbXV8K7UmG10n+eFTtFPts/xOuehEgjR/sdSn/0jXUoA3VmjpDmfakJD+e2JNS3erci8ojYnPk7fTUs+FxO4UUJHyqBw5piaimfvP4/RHDV8eqZZqDssRT2Od+QAc0JHVeh+Uhze/t8iwC5Lv75jO/WQxVJ6xFs+flNcm8PaYLxih7+9+yhg3h31/K4E746dn+0nK81njrWbnL9+YFY7t2+03h6NM+d/b8uK2piworovpnrAf4p8CBxN1ycgHKSUHA5Klx61HkWxKoCcluYnqy3b/1OI3srr/2d2rfhX6v5fIGNekb1oZYio+9ng57bL0g5ACXr3y0X02chzIBDJzpkkqA0b1LVm5MeKNen5ptKEOhqkVk2E/SfqGGzEaJzse+ZmrCQ5P1WvdXlvz3Y88s1VQEkWyRUzG87Ghm2dppPqaKKjQZup4mtX4sxC6RrCLZ7MJpSOlPDg1wLmrWRFVEhfr4IGervmwD0CKgkJEarTL3O3Oyh/EET+EXvi47kD16Q29N3WSQ81u3cRSfMcgLIHuNYL4uyIBlkMc3pLgBOGlldVJkANAcmeB+tzSW1oub3P9Nid/d2XsT7xe7wJUpTWa2xRjiA4qWrY1clZr/m98l+n9uLM+x+PHiFLSVnFFaIPYddUJb9+/o4Nvj8oMS32cgNqb8kwk0RrpJwjJc5aamZvfNMzY5/0M6MWZkKAr94ZIq8GOImoDIxBhmOcc9Pg3Xa31CvyEd06KYRHFwQidDxLcog1064r4EdzmmTm6d4xeBEVd2Jqt42KM5lNCTpoGqQ6E83YDaIM9S282CwMEe6+MmKDHnwtpklqP+UhIgkI+Rwxyn6CA1u5gTnX+91Ho/tlLPcXptU/iDLFKYXuM1NyRfwsWCj4rrnP2qeGsQH3ZsL9U5ub7+NVLi+nOZfASIsbFmv14pqOLGnKIaT+FiH0k96wXgfBVWAhCffZiF/gE7yi X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 883ee873-7778-4a3f-3954-08dd57a884ca X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:25.4061 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 7jBjNko09tisvOWAzm3MedK7t9041xmxk6n9sPT4ks60612mt1ur4HnCCMpJQaRcBYcVejPDtiZGydRyrH3yWw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Stat-Signature: cd3ocrhxyippwtirmff5j71jgihsheh1 X-Rspamd-Queue-Id: 78BA94000B X-Rspam-User: X-Rspamd-Server: rspam01 X-HE-Tag: 1740713814-669875 X-HE-Meta: U2FsdGVkX1/AC2nDvXzKjVYtUfhERYv6w6DW5AhZ3WKKNgyNCeq69tNGc86CUK9Wqze8ZofcFZ4mIueTgXxJf4TnU+H48OYAwqIw/UvVST6J3Bpi9aaJRguYGMaqd3Ecsfl/Y+OoF/VI3vZ/6BPJUt4RWd695ODEVoLFR4YI9csyBOsQzfY+aw1qezyNObch3QipNdwFZ+djkpM8tmFwEDeCFCUKBhG1vJokLMzqGK9dNNabmbmPqISqUWft+a/brgGeI5CIvFJldmtVo+Z8Rm18TwFgVyT+/9J3kQu/EKq/zQLWnq/BLDY/B0UaoZt0YhJN+comOG6THrqauD9UMHt6v5LqqaMvDxM28kuy1K6y48sz6Pjx+m+8mw7t6cmystYrrsyVq9v5+sm5f3tPcVvgLvpDKuyCgA67mb2aNPfiOFG+EFV4H3sDhBNneB/KuxuWCVvm2JzNICPClgR7wm+UibUipT8nvCxVkIjhvwbDfLzQoVWwpradnOvwz69LenL9TDKgRvcrOIBU1N0K/0fRAMUHKbz6bkgn6H0PlGv8hoqyP2AwzVSBjiFFAmluWxtCh78VsptqjQrPcKBMmMCsmgdNwkx/Lf4IFKheRHxC9w/WIbSF9GFOzmHup1strqyKccyqaz6P6ppS6WYCRLqcu9fKdhJHzxHdWwaGQ8Rv58fTmuxEmKCwJR865nMzdsFhBo0rseciWjkQVNvJJDRlxGmjecrIUf0aq9HNmgYvtJ8NW7WqnUJZ/KDwqgpwG/eEeUYJ9OiFo1KLA5j2N744Qo1JfpljkL9rYkVCRg++gy5RQ6UYkt7pSza7cBqCST+EfhkaM+1Rpf4x3ybnW/Y31brTEppsLbSRP4cV5Y5kQuAI00d53yc+qtgfWiCzvnxN+QCT1/lUkxOUa2Se8qwTGA5KnAwa10jilJ85pGj0oNZIsP/joC66lDPWm0FTtRkjnYelgAianJlsazj jsffBbhj YT2yxPUKtud8XUjQnEwEiFAC1Bowo8wckdMloOMcPa1nSL7ygrI3Fde8XjOUf1wX7ApO/NpnjoPVwFP9A3e7uUGmlJN2E+QeZ1b8ksq0wdt0wnJ6GuZBQFyiZs+zWgYKL1Bw7zOY/+yMbL+/1pbbmFEtBUw5bGCl/jo5vHe9xh21cztL9ksUD361xBlyFmp5JX0VGta9u7kRNQrPLhai9n0CXCyj66Xsx6AnwKfLJGIknazp2y1ODrY9OYblg7tWyCXgZSs1PRTnlf7PXAAfq/GbFwboupgtcVmfkan8bLn4IQIjAeM63BOBmnx3twFK7A/agSTSOvIMjA+Hy3iBKkySsQC1MyZPYI867v+S2D47PiMqQzK2diGOBLZjXywNxMFFoZg98XCo5Th6vQ5C0vr2LYna3iid4i9t+J7anTLC9HXbhIVqXISwc8ssB3h6+eDZhvAD7h4Sy0SOn1H6o3p2owPvGr+aHGcXflapc/6JJ+Ic= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently to map a DAX page the DAX driver calls vmf_insert_pfn. This creates a special devmap PTE entry for the pfn but does not take a reference on the underlying struct page for the mapping. This is because DAX page refcounts are treated specially, as indicated by the presence of a devmap entry. To allow DAX page refcounts to be managed the same as normal page refcounts introduce vmf_insert_page_mkwrite(). This will take a reference on the underlying page much the same as vmf_insert_page, except it also permits upgrading an existing mapping to be writable if requested/possible. Signed-off-by: Alistair Popple Acked-by: David Hildenbrand --- Changes for v8: - Remove temp suggested by David. Changes for v7: - Fix vmf_insert_page_mkwrite by removing pfn gunk as suggested by David. Updates from v2: - Rename function to make not DAX specific - Split the insert_page_into_pte_locked() change into a separate patch. Updates from v1: - Re-arrange code in insert_page_into_pte_locked() based on comments from Jan Kara. - Call mkdrity/mkyoung for the mkwrite case, also suggested by Jan. --- include/linux/mm.h | 2 ++ mm/memory.c | 20 ++++++++++++++++++++ 2 files changed, 22 insertions(+) diff --git a/include/linux/mm.h b/include/linux/mm.h index fabd537..d1f260d 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -3638,6 +3638,8 @@ int vm_map_pages(struct vm_area_struct *vma, struct page **pages, unsigned long num); int vm_map_pages_zero(struct vm_area_struct *vma, struct page **pages, unsigned long num); +vm_fault_t vmf_insert_page_mkwrite(struct vm_fault *vmf, struct page *page, + bool write); vm_fault_t vmf_insert_pfn(struct vm_area_struct *vma, unsigned long addr, unsigned long pfn); vm_fault_t vmf_insert_pfn_prot(struct vm_area_struct *vma, unsigned long addr, diff --git a/mm/memory.c b/mm/memory.c index becfaf4..a978b77 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -2624,6 +2624,26 @@ static vm_fault_t __vm_insert_mixed(struct vm_area_struct *vma, return VM_FAULT_NOPAGE; } +vm_fault_t vmf_insert_page_mkwrite(struct vm_fault *vmf, struct page *page, + bool write) +{ + pgprot_t pgprot = vmf->vma->vm_page_prot; + unsigned long addr = vmf->address; + int err; + + if (addr < vmf->vma->vm_start || addr >= vmf->vma->vm_end) + return VM_FAULT_SIGBUS; + + err = insert_page(vmf->vma, addr, page, pgprot, write); + if (err == -ENOMEM) + return VM_FAULT_OOM; + if (err < 0 && err != -EBUSY) + return VM_FAULT_SIGBUS; + + return VM_FAULT_NOPAGE; +} +EXPORT_SYMBOL_GPL(vmf_insert_page_mkwrite); + vm_fault_t vmf_insert_mixed(struct vm_area_struct *vma, unsigned long addr, pfn_t pfn) { From patchwork Fri Feb 28 03:31:09 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995539 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 597E2C19F32 for ; Fri, 28 Feb 2025 03:33:46 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D83B8280009; Thu, 27 Feb 2025 22:33:45 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CE403280004; Thu, 27 Feb 2025 22:33:45 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id B101B280009; Thu, 27 Feb 2025 22:33:45 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0010.hostedemail.com [216.40.44.10]) by kanga.kvack.org (Postfix) with ESMTP id 8B866280004 for ; Thu, 27 Feb 2025 22:33:45 -0500 (EST) Received: from smtpin22.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 4AB38522DE for ; Fri, 28 Feb 2025 03:33:45 +0000 (UTC) X-FDA: 83167934010.22.AE9A4F5 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2052.outbound.protection.outlook.com [40.107.93.52]) by imf13.hostedemail.com (Postfix) with ESMTP id 6F6C820004 for ; Fri, 28 Feb 2025 03:33:42 +0000 (UTC) Authentication-Results: imf13.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="f72b0zK/"; spf=pass (imf13.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.52 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713622; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=NqUAyvSreWQSx6y7bcA3/W+0/nsi680Xnn1rDdTp16A=; b=cPHxHMu2XxIHwYemwWchwj/F7tkUWi9VEAGBNOaWdC5NkNGRWv684jEbOTmnjfSxxGdlvL 8sHoLnNSFMrbI0OhNP0cHJOndSkAx1VQ/NmYzoE0zifZHhf/grhHtmLTAIqAShFK3hF6+s +BJbafZjTxJ+3ZQayZ5I3BeEiLSLabk= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713622; a=rsa-sha256; cv=pass; b=ze963TnHSBzVHQbg5K0e+FPcMBAwdz6RDvQI+YycG8FE6SRunmKdujeCsyMOI/jGUAbzdn Ur129qqAi1ZpXxzBwh1tREwxWin7JlGhM3xu7fW1DAb8EX5AZdRZSicPL7ffYGYFVrUCzO tqQUpnHR3QXnb2QRxuad8PlxmWsA378= ARC-Authentication-Results: i=2; imf13.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="f72b0zK/"; spf=pass (imf13.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.52 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=ZLftpql5nBKOKOc1fmqUUu8apTQRadIkKCguU7WIov2hcuZd0oJiOE/qmKJrIFBk2ygcRnYQnO+MUhVdu+YkwEEsNArcOc0ZVygnz+OqHPY04sq/tZ9fKwkgz/N7hycgeTIG18G3qK0fuwdhRjX7ML41VuBvc03SEc4amLJ7SqZaBwKDmzuMtocPhRrYgCT6ob7udQpijzB0GYZ7uzOM45NpwUl7gz8goUyhNqQh6jPP3E43ETI7+qUF7p2qDVZ41pDw/StNcG/DOJgq+2jGbXqLjFqZ+JOjhiKBzDarDKV4hstnm1IMXLp3/66btljpV8PInFX/FJzccz7QLG5P2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=NqUAyvSreWQSx6y7bcA3/W+0/nsi680Xnn1rDdTp16A=; b=gEZAQkCcxEflP5xxFKmFv0kbPt17rN7nER+3auAC4yyQ2FKVtOOeCB0b21QMqxQLCZp3o2TN7xp/WMGdryb87a1yHoxpc3BE1YZKLhAOWgjr8mBi69BvcDLKgRSR+ceNgGokG3Jb+RkLfmm9ppGfus1v1sytsQlBJZOd0r6FL2RpCl5BH0fwJD1WH+OSnX5v+HH51oDcuyQJj9s+zS9r3dqw9BXrDr0n718a8VpY9toCQVvGq3Ms2MWLh9Gz+blQiVn8L7Md0Aq69oMeVsAQ8vIrXWS4ZxjT+i+wLsA4yjnhiZZdKbeT3s5ffkpWUXsGbHfCY85877FbkFzRrjxdCw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=NqUAyvSreWQSx6y7bcA3/W+0/nsi680Xnn1rDdTp16A=; b=f72b0zK/zHjJTHIIqXmy2M3qWN7fZH9Ln29YKAC9fizbC0k5bDO6vajj06FODGr/DfvQHB0iFICJLVY22vNM0jwp/WgFG/Oy/TeZHGYZKF6tD0/gLPy56IJYqpMdKaj+UcV4WHZ6+MZV8iLBjQEJHUcAowRxgXuFSwAoKSwav3rYlVFtXMXBrkIaGnKU28cnTQ6PPs2SkAX2z/2zr40lkxCwMHUtLv9euMpL4Q1PJlHqaaHeXdJUI6gEhqy3v57G2gqhukNa1EJpXGLZjJU+UunzkicE3Hs4FPDrXKRNfgCfi6HtoI1swS2/2npVjFoicakjsoZXEa2k58Qkbatksg== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:30 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:30 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 14/20] mm/rmap: Add support for PUD sized mappings to rmap Date: Fri, 28 Feb 2025 14:31:09 +1100 Message-ID: <248582c07896e30627d1aeaeebc6949cfd91b851.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5P282CA0128.AUSP282.PROD.OUTLOOK.COM (2603:10c6:10:209::8) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: ceeab409-4438-4988-4925-08dd57a8879d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: oeeCy5iUmCo3ajOydYaIEMK5H8pLtFYl2kfoMHPAC7ZACFgtpvSoSA489Xmzp0ds9d4LTDCLjErxfA2rLg0vUvBg+OPr7Si5vmJFg+mHzIH1fqhOv7E5Y8YEWMjC21OG93oBnKHl6+vTyAwfh8MGS9YaYsONuz8J2fRfASVj8PNbNyJW4S1ghVBxk/FpsASI0XFD/ZUqgteNumtDPmNkEnNMwTO9h4NErN0zZTaAd/SCvtlybCl2CUoPDcUlStKKuM3/dM4twhIxmh8FUdfPVnqY7q5LtyH7BTZ0Z4HyXhJb9HFXUo81BHHoM8Fw2b8o2KQkqLo6Crmh6He4G3GpTmKrEZKHDQ5Vb0UDF/01HxA9dZO6tEw3Z+1eiNAFIlk3lzvLzYVq4K/f29erw7lw02i4ZDvh+rWXlfvlpBFvwqhoH0XeZW1fXe4bGYU+GPz4RrB67vWQHhDmHjjTHm/YGDI69madSTQ8x/gLVG3ckF+PAxxq+9wq/Pty1WUyREvDBrzHZ9QbyhxbNj9XWuAioCglpB30efyoje8MGypMyieqgVj0iRecoWEEzlMTEUmW2SRBo1ES/yRT4k1coeLf735EIMIrNN0LUV+ak1yS27h7UByJzgoeqs7LpfRJGSn1y7ez5X3neNrR4xX92y+yBHYEfOXOnlYkxZ4lyiEE3teF+h0U37SaTXmf1woxhyHPzdc8/VqZdZZMnBvLnG/SYuj380nmxcZ3+ZHSuOyqModxGQstffrLZ9M97XyW54iP+obvUqirViR5HZkG609t0b3pvOiYmGBYiU8TQ51dIHX1SIErf8fDb3O0EM2a7zT3xYC3lhcqF7KgGHWEWEcacGveUsOX4PtaRGzLrCkN4iK9fQC7TuHKfb8gsWPDf5hVTofelFg1dGih5vjT1RvRiYor8Eb/7P2rfCVFm85qqGDjFEw5k7CpVlMXoGQSFAeZeTRq7CyYheU7SEsD9AE/nCW0tSISKpMFKUQrL3FW2ep7lmBQwKSj3RWKMScmeDYBluMSWH+f1mT0chsxHWeDsFycZ9LehQyfyPbNjdWZkS8JX4YnwZmc+LPmM4EJqarXzA5cjdIFAyaxW7drnUXIMIptXq81aMttVCglq43732y7q7ux2/ji1mWVmRNcyR5yebX2xdE5dCPGAD+ORFNfydiMjBCbLzipiZyBzUk69imztUVzEQ2FCASt9BAExog/1qw2D70Fw4XKy8wn+UWJA25tUeZxI+US/0XnWVaM5W/i6ksXEP0jRYwMtiQYqrzfiNVX82/8Z+eu/ZyedIuNIj7GLo4Yyv7FKiToVLnumgH2kBxuG6jz7z51kOjAvWx7lnMQ2ggVmjzEVquHnNhlgB1bnk52t413vEEb1djFsYaggRuu2w8EhTtV+7YbkscH X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: D4evzlrrf8XbL4GuD4fm/XG/91tcO7oOZnC0VKIMo4sUtLx5Ggxzn8LwGZ2I1jIFd8VFGmtoDwsc1i+Kt0/NaTL5Jt7UwK2E2amkr55IBUfPO/wYjXdnhVrvoYDddKKEqhKc+QOzsRdp5abrFkdFX5A9y5tLAOD2KEKb/IMDVsssgdtOwSKySJvSpMi9bjMJ70jsg2o/6shvrRgub1H2aAlVTIovEN0vneyRG5lRJHlg0GtOSo52OQb4FpisqwmVajw1Y7lsgbX6RyknAY5vEzsjQuWUtC2dijTG5H9u7+ZSzB0c9jrqqf6jqcNNxZmYt/sltajdFJ5qc/E+2abNDMKW2uEkWxDWnIf73a8A8MEqQUHj0fcnvvnzR9BrW8OuZHs5gEL2HOqbQz8R0pcJxh3IFERzWXqhFiKvpDEKyzAmmX2xVuqFugowrAZoNKL0lf57uDdaCMyhpGVAcO6mkyAZIIPzWuxubtGRh9NaMTttYo0NovHvx/HsQOM+jmYHuhbKhYvlxLyNdVGXL77CmYqo4YSczTDKpjQfaXPos4teJjxIXbVEc5XHqHVxGYWXldLjbXcuZNqSAqAwtg14p6jw6cXaqhoA53hlxH7lbrbDbDpMBfwpRTeHPCSlX95tOzk/UniQAFqbo41DqcBqwnqblbkuUPm9vTR4iHtLSYpIPtT399asVbfFpCfs/oYTn3FV5+5Io+VrxoBFxoEz3+8mbI5Cj3F9If98f1szhzMnGoxUtCk9cik8TtDhVXe6VhV11Kp4VeIyNFtfji/ma3RyXPnOP15oHekhGpLrBVTbCR2hyC2aMTFwpA2KIqnZWQ57f51NQC7Y4HA6pkeohL7UJLxrNVAe9bNG5pligvRqQXKZiYEEqk7TdG0pTKbXXlGX5OUYUheYOmHyalGWg+vXl4CkKQfBr+MDegKnzyNeitOq+sCz/O1mOZgNN0mN7x7BMAfK1t780aJqVIXIBUuCjoY+GYBja+wRthRHnM02CMjoX6TtX3pAAfQaDjqnBujbcwurSqSJZdF2mDhbgkN3ffJdUfry7H9yUgTyNxObbtbsh2oY3+rffK88TCSVd9OwA4WwujtnJt9rIgAJD7+9KSA3DiWvwloVfZf6ud5skvpjSBFGxHww0jdoI8OEK3zDPaPNtgJJsV81lg8GKtNBtKpjeer/0UQyhVjKJ1BpGrVoY2K3oxwPkdeMzPxMgAbQdKd3D1uZ/Yc+EB4Kg1sUzJx7YB/3taB4ZdvNEVISCs9/54ikDoUHXToqBnwEv6jlW60xZs5Wy3fYTnnq0y7fIcZjnNW1NofNLmPgS0ZlyF7u+IbeuzAujCnKitLv71AF9QxYdK6v/3J+saGTmhJiYj1pv9g776LZSolYmKxg4elRjd9JZvrvuNcNC39QADBZXELJ9qMDfbIBtxFhu7q6CNobH72XOiAiMR1JzDYKrZvR0Tv+HHsfP8gfLQ2QSOagj0LSfAKCXX61kAPaXTBQP87kR9ttOk9L3SlrXeKSD1HwQ1dXTQyrEJ79VgKp7HNSIDdHM/sfXVK6sb5SvdNnqQ3V1YQESB9CuZ7HOjc4jytmtWG2DH9QnguBjdKK X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: ceeab409-4438-4988-4925-08dd57a8879d X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:30.0081 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TRBxsA8DH3JtyeAS++Tt5cMwRnqEyiujCMYaWWyLHDSzoSGuLqBqyjnYyw/ubjbBsRAy7pkrDFGaJZYArtxYTQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Server: rspam02 X-Stat-Signature: x5knisr3u7if85wu49ufd4gmsthkcu73 X-Rspamd-Queue-Id: 6F6C820004 X-Rspam-User: X-HE-Tag: 1740713622-152953 X-HE-Meta: U2FsdGVkX1/FMprDnRh5ArVoHxCkG3vRH/e4DVu62gSvkyPIADUCMk8UeAMoAlMtnrHwGNPomNKwVzp3Od9f1/elAswB22dCsSK89P8RqeuaCchZbeRjgo+cJeZZhra9eCQBwhhWrzQ2agrEerIlZvY7GoD9K6spxR6Pdp2ZbdfioUmXI3pbrCSFoj0ztPmEJxSEcoXlQFLDUk+ady6i8KHLQEb4Cu8P43QftVSIod4/fH5pf23AsQCb0KeeuiwrQNInobOGsyRwRR2i9uQTpwB/yGvtqxe/MizkzrlkeEp9Z+EsRJ+fwJ8KTArzRV7knjzUAIHsMvBmb5223gLvCRMoPyxynlkiBQ0v3UOxJRwPOCcKPV0eeVyT/E7MulgPjxA6z9axbGKRrz71caEDknjtJ1ldVqR8r5Rp37X8xA6MBK3tOan/RBV4bANpFr/Do5xjBxfCbeR7Tkm3FgeH3oQ8Xj53Oryhyhd6pKMRqwilNR+6t/mUhINdYT4X7RxVa2197cLuiMJtToKLqHjZvGu3TiVIewS2MGUBqidDLr9ka16GcjbcK4f/Grs+WOYacLTPnOfyrAjoLnswEmOrTN/T4ERzRYkY15yhbK6FGu2Q8p+AThxbWRVm755FwIwpgekJNKE+eNRbrJoKLd/JeW0ZwvnfvyP9jV+oIXgrNlqg2K6THySy5A8JzngLZW3MYWdEF8E4n+WrQPwUO31DzkhllS5h2YR4lfoTmb2yQIW0192ebBFuTSZGlFCVbaGtkzP+n2YbZe/N5UuDODFrD1GGjEfVXQ8r7AMiYTGRP6i9m3QdBtcH0N3RwB4vtwLTwjTAKy/uWK+SdEBsDozuuum8gGxvWNSsy3pGxqr1W8/a7jcyikE0ROOwuoz9sR5g0ZfLNmsJM1gSbPFY94kT5mgDVmGGCcW3KfiqPHD9D65aMyzv26GWHxyLphjlv3Q353HXFGE/uqX71LTosF5 gnDV9uGg 4x6dy0mfDgwFbLkHetiVGcw6F2uoUB/WHP7HBUTWdrVf9oSwPrkLjWCkX1ZSqO3D/GWqw2vtUeCYCf14Kz4iljJ3onmt6CxVdTM5y1/eS5xfrtcS+lzEpHw4Ie0EjuPuW6Zxu/rqkZ32z/tmFgVYibPy7a+IOltBOuOK6c2wwTnYskq9gy0dbIBaJboZTyFDV39aSlHsDnCBa6ET0OC2Ef+yhVazGckJmFkmKBMBTYyog2hINOFicJWtaSaKNm6mxL3oVDCgXLTIMteeeRVWQixvjWR33zQ21c+L9AX2rYcUdpxtvW8X9SQ6Hpx2lZYY3ZM4xB3sYhgug4AfsJxd3OOEzu/xWkLZQ6T5fvTFgwI6TelPt6aMRSkh4834gVbTfX+oIfJ/e/jH2ycUu09SD2soz26fbUKYY/8EKoJBymQd1vNLFZX007x05XtjQ5TNhZ1QtNDCV1mkWKQc= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The rmap doesn't currently support adding a PUD mapping of a folio. This patch adds support for entire PUD mappings of folios, primarily to allow for more standard refcounting of device DAX folios. Currently DAX is the only user of this and it doesn't require support for partially mapped PUD-sized folios so we don't support for that for now. Signed-off-by: Alistair Popple Acked-by: David Hildenbrand Reviewed-by: Dan Williams --- Changes for v8: - Rebase on mm-unstable, only a minor conflict due to code addition at the same place. Changes for v6: - Minor comment formatting fix - Add an additional check for CONFIG_TRANSPARENT_HUGEPAGE to fix a build breakage when CONFIG_PGTABLE_HAS_HUGE_LEAVES is not defined. Changes for v5: - Fixed accounting as suggested by David. Changes for v4: - New for v4, split out rmap changes as suggested by David. --- include/linux/rmap.h | 15 ++++++++++- mm/rmap.c | 67 ++++++++++++++++++++++++++++++++++++++++++--- 2 files changed, 78 insertions(+), 4 deletions(-) diff --git a/include/linux/rmap.h b/include/linux/rmap.h index 69e9a43..6abf796 100644 --- a/include/linux/rmap.h +++ b/include/linux/rmap.h @@ -192,6 +192,7 @@ typedef int __bitwise rmap_t; enum rmap_level { RMAP_LEVEL_PTE = 0, RMAP_LEVEL_PMD, + RMAP_LEVEL_PUD, }; static inline void __folio_rmap_sanity_checks(const struct folio *folio, @@ -228,6 +229,14 @@ static inline void __folio_rmap_sanity_checks(const struct folio *folio, VM_WARN_ON_FOLIO(folio_nr_pages(folio) != HPAGE_PMD_NR, folio); VM_WARN_ON_FOLIO(nr_pages != HPAGE_PMD_NR, folio); break; + case RMAP_LEVEL_PUD: + /* + * Assume that we are creating a single "entire" mapping of the + * folio. + */ + VM_WARN_ON_FOLIO(folio_nr_pages(folio) != HPAGE_PUD_NR, folio); + VM_WARN_ON_FOLIO(nr_pages != HPAGE_PUD_NR, folio); + break; default: VM_WARN_ON_ONCE(true); } @@ -251,12 +260,16 @@ void folio_add_file_rmap_ptes(struct folio *, struct page *, int nr_pages, folio_add_file_rmap_ptes(folio, page, 1, vma) void folio_add_file_rmap_pmd(struct folio *, struct page *, struct vm_area_struct *); +void folio_add_file_rmap_pud(struct folio *, struct page *, + struct vm_area_struct *); void folio_remove_rmap_ptes(struct folio *, struct page *, int nr_pages, struct vm_area_struct *); #define folio_remove_rmap_pte(folio, page, vma) \ folio_remove_rmap_ptes(folio, page, 1, vma) void folio_remove_rmap_pmd(struct folio *, struct page *, struct vm_area_struct *); +void folio_remove_rmap_pud(struct folio *, struct page *, + struct vm_area_struct *); void hugetlb_add_anon_rmap(struct folio *, struct vm_area_struct *, unsigned long address, rmap_t flags); @@ -341,6 +354,7 @@ static __always_inline void __folio_dup_file_rmap(struct folio *folio, atomic_add(orig_nr_pages, &folio->_large_mapcount); break; case RMAP_LEVEL_PMD: + case RMAP_LEVEL_PUD: atomic_inc(&folio->_entire_mapcount); atomic_inc(&folio->_large_mapcount); break; @@ -437,6 +451,7 @@ static __always_inline int __folio_try_dup_anon_rmap(struct folio *folio, atomic_add(orig_nr_pages, &folio->_large_mapcount); break; case RMAP_LEVEL_PMD: + case RMAP_LEVEL_PUD: if (PageAnonExclusive(page)) { if (unlikely(maybe_pinned)) return -EBUSY; diff --git a/mm/rmap.c b/mm/rmap.c index 333ecac..bcec867 100644 --- a/mm/rmap.c +++ b/mm/rmap.c @@ -1269,12 +1269,19 @@ static __always_inline unsigned int __folio_add_rmap(struct folio *folio, atomic_add(orig_nr_pages, &folio->_large_mapcount); break; case RMAP_LEVEL_PMD: + case RMAP_LEVEL_PUD: first = atomic_inc_and_test(&folio->_entire_mapcount); if (first) { nr = atomic_add_return_relaxed(ENTIRELY_MAPPED, mapped); if (likely(nr < ENTIRELY_MAPPED + ENTIRELY_MAPPED)) { - *nr_pmdmapped = folio_nr_pages(folio); - nr = *nr_pmdmapped - (nr & FOLIO_PAGES_MAPPED); + nr_pages = folio_nr_pages(folio); + /* + * We only track PMD mappings of PMD-sized + * folios separately. + */ + if (level == RMAP_LEVEL_PMD) + *nr_pmdmapped = nr_pages; + nr = nr_pages - (nr & FOLIO_PAGES_MAPPED); /* Raced ahead of a remove and another add? */ if (unlikely(nr < 0)) nr = 0; @@ -1420,6 +1427,13 @@ static __always_inline void __folio_add_anon_rmap(struct folio *folio, case RMAP_LEVEL_PMD: SetPageAnonExclusive(page); break; + case RMAP_LEVEL_PUD: + /* + * Keep the compiler happy, we don't support anonymous + * PUD mappings. + */ + WARN_ON_ONCE(1); + break; } } for (i = 0; i < nr_pages; i++) { @@ -1613,6 +1627,27 @@ void folio_add_file_rmap_pmd(struct folio *folio, struct page *page, #endif } +/** + * folio_add_file_rmap_pud - add a PUD mapping to a page range of a folio + * @folio: The folio to add the mapping to + * @page: The first page to add + * @vma: The vm area in which the mapping is added + * + * The page range of the folio is defined by [page, page + HPAGE_PUD_NR) + * + * The caller needs to hold the page table lock. + */ +void folio_add_file_rmap_pud(struct folio *folio, struct page *page, + struct vm_area_struct *vma) +{ +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \ + defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) + __folio_add_file_rmap(folio, page, HPAGE_PUD_NR, vma, RMAP_LEVEL_PUD); +#else + WARN_ON_ONCE(true); +#endif +} + static __always_inline void __folio_remove_rmap(struct folio *folio, struct page *page, int nr_pages, struct vm_area_struct *vma, enum rmap_level level) @@ -1642,13 +1677,16 @@ static __always_inline void __folio_remove_rmap(struct folio *folio, partially_mapped = nr && atomic_read(mapped); break; case RMAP_LEVEL_PMD: + case RMAP_LEVEL_PUD: atomic_dec(&folio->_large_mapcount); last = atomic_add_negative(-1, &folio->_entire_mapcount); if (last) { nr = atomic_sub_return_relaxed(ENTIRELY_MAPPED, mapped); if (likely(nr < ENTIRELY_MAPPED)) { - nr_pmdmapped = folio_nr_pages(folio); - nr = nr_pmdmapped - (nr & FOLIO_PAGES_MAPPED); + nr_pages = folio_nr_pages(folio); + if (level == RMAP_LEVEL_PMD) + nr_pmdmapped = nr_pages; + nr = nr_pages - (nr & FOLIO_PAGES_MAPPED); /* Raced ahead of another remove and an add? */ if (unlikely(nr < 0)) nr = 0; @@ -1722,6 +1760,27 @@ void folio_remove_rmap_pmd(struct folio *folio, struct page *page, #endif } +/** + * folio_remove_rmap_pud - remove a PUD mapping from a page range of a folio + * @folio: The folio to remove the mapping from + * @page: The first page to remove + * @vma: The vm area from which the mapping is removed + * + * The page range of the folio is defined by [page, page + HPAGE_PUD_NR) + * + * The caller needs to hold the page table lock. + */ +void folio_remove_rmap_pud(struct folio *folio, struct page *page, + struct vm_area_struct *vma) +{ +#if defined(CONFIG_TRANSPARENT_HUGEPAGE) && \ + defined(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD) + __folio_remove_rmap(folio, page, HPAGE_PUD_NR, vma, RMAP_LEVEL_PUD); +#else + WARN_ON_ONCE(true); +#endif +} + /* We support batch unmapping of PTEs for lazyfree large folios */ static inline bool can_batch_unmap_folio_ptes(unsigned long addr, struct folio *folio, pte_t *ptep) From patchwork Fri Feb 28 03:31:10 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995614 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 6E5CAC19F32 for ; Fri, 28 Feb 2025 03:38:26 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id E23A7280012; Thu, 27 Feb 2025 22:38:25 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id DD364280004; Thu, 27 Feb 2025 22:38:25 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id C25D6280012; Thu, 27 Feb 2025 22:38:25 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id A01E5280004 for ; Thu, 27 Feb 2025 22:38:25 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 62E8C16197B for ; Fri, 28 Feb 2025 03:38:25 +0000 (UTC) X-FDA: 83167945770.02.081542A Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2047.outbound.protection.outlook.com [40.107.244.47]) by imf30.hostedemail.com (Postfix) with ESMTP id 7F01180010 for ; Fri, 28 Feb 2025 03:38:22 +0000 (UTC) Authentication-Results: imf30.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=YC9TMgfX; spf=pass (imf30.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.47 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713902; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=HqzD7sEFSC4LaRKIRiHCWZ4+XsjlEZSCqu5EnQeyOr0=; b=4FhSZZKDXpHMVOTtMdzF7hZWMEUAvAl0u8TZfmitbd/A2SBtLtIBKVkPe/DVDFUWnZRqOS QavEnAMuRE22sqXunczQN2tTSyhRVqhE1HbxO4HpgjJjXEXVEjnUgcO1hLSOSW8Bs2BUjj VV1Ok77dXYpsDyFqfpSzxUSbAfYY4+c= ARC-Authentication-Results: i=2; imf30.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=YC9TMgfX; spf=pass (imf30.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.47 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713902; a=rsa-sha256; cv=pass; b=IFrGYUlOe7GWopm6PFRCUqQf7s7neUwYz5WC3yV6Y1mokWworv8gucZro/VgvaV/g59fNy /qTZpmbgkNrP/iKqBNwR4isVvHagpS7fqYRvCPro/goSqgeklIu0Ur2t6V6dKOqynZrVH1 RrVCoWPlzBW0Hs8u4BgpVHxqmwUfenk= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=h/RbGRmYP2oUJycSSYAnUixa8yrDBUACn8DvSSZ5NYe61igx7QqN0cReObOu2vj5nha41pckFBFWFm/kpZi9CQAzLSg7Kc8f5FTRLQUAUglOSJuACXkzjWajMzGIi6leY9qgOk39Jf/B/fpSMqPvsxy9VDwFsjD/ojlLz0Joyy9SgpHyrt7Z66X0LRNsoFU+G93HiYqIrJv5K10zonIBQwn7esNu+YDmhWW7WTbinH5uHbjD0rFsQPwq4LLFgh9CyltJR5Wx+2BQqm6Rl+YtcSLaWKnqGsKuMStRemgih4+Z0/w5KyIVeyUw7zHIxlqIXiEln8j9F5pn+cK744r5QQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=HqzD7sEFSC4LaRKIRiHCWZ4+XsjlEZSCqu5EnQeyOr0=; b=StjCyGL7pXBfG/QJbtyKTm/ao8wcP7vZg9eAmEkv5K9EqmpxMUDK1V6991hwQeRN0yemtSu8I/Pus8mnWQrpky9sarHONuIraVhTKw6oPsGpBjcdp7hJPj8bdQDJUishEzw9rKOqeJSmydhtLIq/qzrq6BmZRdQmYQOA+BzU7DGFI0o76/TNKxUq3q/Nr5aLqshiBVjcIlT652XUwAKOi5/R3zfGfhpRoh6Qh+coknudQd/+Kq8sRncJ8YMObuqcmbzK0O2R+FQs+fnDf8So3MVzY+N2vPDcQwgtJPhH+YmraiIhXHFALXncoQmHv6/glqewZbJF3BEfy4+kWUylzg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=HqzD7sEFSC4LaRKIRiHCWZ4+XsjlEZSCqu5EnQeyOr0=; b=YC9TMgfXwvvEYHYORK36SdOdqBnkMSUB+S9w/my+0XSTPcaYn0pDomYWEbxKfS4DO6xhMqhyfjZ7IA3wEHye1HEsmrOjHcVIqePQUCcndkGKvO/Y+Usz06ei46WeT8BN9MmkvX4zTcmjORXK0BJxaQBtrTu9JETv/C/TATj5+Ok97pTaVqy3rhp0EQrk/uf46Aa9+fXVQL025S8zvZFrrc9hPuz3hP5QFxShY8Q+0qNupsqoGVaUA0Ukmzy6YkuEwOODDNXSuVEPviZq/w2oFKSreLUS1DziuaHwGq0+aLtj1Y6J97siVXTZydqYuvYc5oLzSA6J9QNqoflC3I7ZbQ== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:34 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:34 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 15/20] mm/huge_memory: Add vmf_insert_folio_pud() Date: Fri, 28 Feb 2025 14:31:10 +1100 Message-ID: <649a1ef91d556593948351e94f51ef73a14f6794.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY6PR01CA0090.ausprd01.prod.outlook.com (2603:10c6:10:110::23) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: cbcbde85-6452-49a8-2193-08dd57a88a20 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: aw57hOUoB3XnUGz7yMAGWb1+0qd1OLt17xv48LogYELvGT4LE5BR8IgO5XKyedgXG8YZzOY2RsaL/Oltu8SfY4hcxQktlhpHNfgz4Qu/QM/TdSk3MW037fZELoHZpFrh/KdMqiTPIkljGvc3RFuvrV8J+1fAdH110fWmS2k9V8KgC/H7WIe5H3xGwaJzLJiS8QyFM4Leyfq+aeQkSFw5FC3ObERNg5BcPH+KMNxKmRvkoHf7efZhhLJb9XOrE3YNUKMXt4dx/V5EDiGRoNkyu4Y42YojT1mQCrFvLtrh5VDFPi7il1bHGjS4CkepwYslUbsdK/txtcIXTPNl0+WLbyte6bx4hLpAsSauoeTdnWlty4kFKw6efMTt2Uh0c4b+CfX7cenwkuVk7+9VhGYp9rJeTYzckyJ/ihRP+u2V2c4OaBWdJRxMu6/YQPIS5H6anUUoGhl4z1TKW/weWveUw8O/KNf912tvssv4mb277gW2kLJKppOOKp3c5kAznd8Uue0YmDEjVdLqTWIf3mUsQVPAeIttp36JFxKHs/Q7myn5RmtGN5kdF1a9yAcuw8Idri88JH1cQpVW4qSkOCsDazAE+knJ2Vm/Q2r50JtbmOlgfqETvQQWxe2sdgPVlZGarGP/kU+tyYSgPRFT92fixQ4vdGB/KIj5hM+XrBdWtkUancMl+JTFewiyH9lByDoHmgM1ZhfxgcLp8YiOMkPTtsjxDRwaqZvtHZfsJvtPN8nTGzRy9ioHDF10qNG8RomDkhCsGtjYr6Ml8P40VHEbjFBZW7hIEXUTRYK975Ex+wmnhj9DEGyRh10qgKC1pByCQuSyQFCWsN/lTWxjqYom1b1mKuXOVa/LfPqoJEsDK/a6T9lEJ8F5ten/O8VXOdTm4lmA8+06/MHsXs1tlKUIZ0kmHARUsltvNFR6dthhWI11w9lhjnKi2iTQZ3AcJF1Q1s9rnweq8Ut5K4tZhiqxdbk6acE9IymdotDktRcY1FgLA+GG7VBewaL0j7mg1s+dAKaPb3HUtjzg2eMzwXGSeLLMz8lEjZPaY39bEHiaJrh60Xh3HsFtS2unrOC8IrstmgpzeVUmum7UVy22thKjN/Luo+r/bClxk44zH1RehTxv36H4iphdekfLPftq1C2XHnegiBdvWHdoCac6fN4sEbW5lEnqTqH36uljh3pG7+EvR47e6ElMDpSPvFX+B/xmq6MPYQ2XCqbu+XoLBHNJ9uzduFRPxL2mBP7NZFu9ZBS+PYr6WEDkFHAbQeeIYflM5tnio4FzvjjFSBUXqpI/C9GXu7dd1tuepv02Uk55n6aomEqGnp5EJQkK45qoqH9hZjQA1demWnZsaZWlI+sjGD2IjmclP/0mnZ/Zs9r+PFsk7lDEEAFLjjLHKcNKKnNv X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: mt9saix1+Z4vsLXCHe62ZNmYqVUyawRke7YY/Z6vGDz3VeY5W7p68FVOmXzXfNZZBYF2UTw4I17BlAt3Ketdi4Svt4hxXBil1XwmpWsrBQJ2SzgUhUsu8eEMH9EsIa/nFisCeevA5UZgHF7c/OOHkdQTnVng78iUFxOhqGhpmH0z9fivWgaPRL2s83pVScaWWdk9SJle+1ILyshPPqtLFYbCqP0RshrCOzBHVmQbuB3U7/U+jGgnlgaQmPpJlf6qaOxG8R/rUxza46l6z3W1Fl7M8rcv6FZJni6rWDLH7gcFnIe3/zdopmyX/CngWOC63pCZ8lmkPZlMSb5xsgPs5tpdpIpk9wqtzSDnBJynB/MfZwbYvVID3DhYjAPT+SSzdX+Px6DQzHvYqV1TAZbGntOnfY9OFoC4epkFvlgv2eLkLpFulc47U42+TalctBeoXq+aEtDODAPtUlU1erRqQ5nHnZAmgrUDs17A2V3A1dH8bPpMBFpiInHik0/B+tMWbe4OAXUnM6kWC6zEl3oGC1Q5IMyyyWvq8LjfPIGuorcykQOldQJRiv1yp+0/onkNu1mJZK/Z+9sIBjOotPH0fsJvSVJIBQ/BZJkV9BDIMq3TR8yiDXcp6CpvfwK+XObS6yu4cl5feR3mhiuz/xD5DQ25kzz85CsHkgUGSee5V1hyyJUUKfThCamMh/QMdKebjO7/TXOxHikypLab87Sn7ynq1XCE+RDArlhuVgaiQ6fF6Q2DbHojyPvHHNfDRmezcrYMqID8PlNXQZvcocycNf2vpXSGkwboTDG26DEyoeGu6aylBqCFIITYkAaWTDaKPlInmfWvrpex2D+xy/PLlrrYGjo9WMUHzxTNTetCiFEMGFj+Sr4xdKzkUA7StpZ11yBCQD5ufDNTXvL/Lhu2CJ+2qoBXTvoAGU35IxNvZm/QNrmPBaJL5rlo5gK/iNTjjWNbI+9Y9JtC4kIXvd7O/jOIrdho5laOgvPDIL0QnRpax9pNeuJdQxgyjKlk/MWedlCAkk3eZyY2YmlrPdpUIybcfBz4Ol+Wnkzvjb/RaOuwlrCo7jlYQqronEUZAybGRWb67UP+dtzov9CfevlY0swA6993YJnJoWfZTbh0amz4XdzinpkzV5XYS/2JuJoK1u1q3LuXkyVoO4mMUpT5BnaEmrN5lYAdtdh7kF+dmpwMkaouMsMMvVc2+XIMCL2kdGh+AgFcjJnd6R5ndIS0jZK86nisKvtzLJJey9GmqZpn3XJnat//zH+ZiUhLYhIAiPZ0DN7rmRFX+05ZYj/KoanNwNJr2GeErlUOsFNjkpm+ikI5SXvrpwIOZkUDl8mI2rR+gr25czJoxEMoMlQsFLo/yYQVuc3QVF6hkdlFCs+2uv/S0fS+WKPmS23eTdm3YhZrCbKjzIlZEPcLIG2P881xzTQJBGqsxm6q/FxMDZ16QGrOl/ne657Blmv3o/TMUIAQZ2jMB+YoHxDH8DFil/6gKbIwRFUUw6GTH3dMnZQTWj7Qu4uRs2S1OgOFvuRBBj5OYuLgADQHhXSzzM8QXBRXIyKIGhsFL6ektMkZx193G6gaiok+lPuk/vcHB8ax X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: cbcbde85-6452-49a8-2193-08dd57a88a20 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:34.3010 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: esEgplQ1/Wgu+5NrSWpyLtAOpKalIwvL5aLhxdXFBdlrdoVMYb/MKs6zaNI1hsrxZ7zd915Ds+NIfpuOYt7Zgg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspam-User: X-Rspamd-Queue-Id: 7F01180010 X-Rspamd-Server: rspam09 X-Stat-Signature: iwz8m7grrpob5g8u5qgggzp1wsirk4z9 X-HE-Tag: 1740713902-857191 X-HE-Meta: U2FsdGVkX1+mGObVd0LcxUwbqToLYMqvp+/1bJ8cpc/B10/Gkdm8GJqHmvPffMWYvJaWW3DAbKCmU6R7OCZScdTYej5XeNZ8sBOJ55FuMhwI9Whv+ZlzVjWVXv/20sOdJYDdsbqeOX9aIPoBaEOdKKZVzKQvly47X95aBcjpRE+MQ6L2LnVf8ovYouVMeK0qSnDxafJYGZBK1W6UNFaOOUUp+Av6upe1C6s8gXhySNQiYpHoxqAGC6U8ESBokXNMPinpRhb5IZtNvQuFaEkiss8fdAmlGTL7gHXN5pPgSJ1e+MvFt3BLCGIYO+rGIYdmNAnuPxZwO5T8C1ukjfJuxriSMYpPUKcusiPIKPzjYvJYW/XTCqTiCH03EjhKaPLw/fQUd4bs3d7iWa7Td2G7fMExWiOFg6srTsNgIY78z/3uibmunqTV460oLFPwUQ2MaIay220xl+oliB4xsr5IssvRZ2DYmE332xUh0SlNunoOoEWf3ON3zi5g5ny6kCRnAQ+bIBUXCbt7HoAJRH+CL/W1mj28u+jQ5Y4sHx+pr7fwDlUwlM4Vx22GBiczJ1IvBig9WB+FArpPfy5EkrJEMgrOWtFLWdTP+3JukhLAtQPFEQPwhvNmJe2TXr2r+cFj8kibF+76YE4W0W3R+SDwwSWDxjQu4Zdkx/zDQd8Goi7Tv9mSzjahVTJdginG9NyD3jQXLpZvbYoCNKFYdCynA8R9nuaIqduyUVakmYqa4Vwyc6sT9Jc7+2Bb8+Zf/eCAeSqErdJcXE3z8A3aXWrJ66ImBruNtCqk9NxUbQ5YGyWMaRSdErOn2zxJ6xuKY4DmuVgJ68olygfWx29arQerfnVGn9KL8TAxdOMNeQcaeDRDKSoVHAImLV3hmkVHHqj++5ezpFMdn+LhkD93qaWtIlomlNqviGlLk1jCOZNmsmpIrbiy+wMNrj6VHJzSa7c4mEJNwE0PUS9veDXRk8n BJzhA9rG MP8yMeyns92O59eDkf++rMvOxj/juFMBsfSfe1vK4kcVQLZZCB8EGwqYJbuXtbVABX8ScEE1UIlYuKniE7Z5dwFzS1j0fhN/DFRgmmN4GzyanOBU7UgkecSmcZGmn8xUyeEyMD/VPR3/qmfVw1Bvg3eA+OXjGocAfJTs/O55XMi8C28lubCkGLV5Z1VgigPRAMGoNhxMmENcD30u3hUB7jFwE3i6+RYvQLrkvhZgiIUZTPut788DvzlWEOYKTDtPPkSQ+XNFdmsegs+ix7/8FfxfWFdV+CvOeNi80Ga1asJDRUhzFCsTV1KS6gvgTzMCs4NhqvbpwvGbYK8i3XWy4Oj6KVTSIufcgTFDQU1tDLf0/uE/VUBIo93ZM1CiCEj1gmNc2i3VD5F9fj1ivQOKZIRU8w/DEOfCoKo3vQDrP//DpJcfEXGPDfEolgXxVuHynPnrZ5VKMJ65xv5EKAhFWtBE/V+gDndoXLELgaqjAjk7XWJHd0y0EovOpyCl11Ku+amYG X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently DAX folio/page reference counts are managed differently to normal pages. To allow these to be managed the same as normal pages introduce vmf_insert_folio_pud. This will map the entire PUD-sized folio and take references as it would for a normally mapped page. This is distinct from the current mechanism, vmf_insert_pfn_pud, which simply inserts a special devmap PUD entry into the page table without holding a reference to the page for the mapping. Signed-off-by: Alistair Popple Reviewed-by: Dan Williams Acked-by: David Hildenbrand --- Changes for v7: - Added a comment clarifying why we can insert without a reference. Changes for v5: - Removed is_huge_zero_pud() as it's unlikely to ever be implemented. - Minor code clean-up suggested by David. --- include/linux/huge_mm.h | 2 +- mm/huge_memory.c | 99 ++++++++++++++++++++++++++++++++++++----- 2 files changed, 89 insertions(+), 12 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index 2bd1811..b60e2d4 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -39,6 +39,8 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, pfn_t pfn, bool write); vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, pfn_t pfn, bool write); +vm_fault_t vmf_insert_folio_pud(struct vm_fault *vmf, struct folio *folio, + bool write); enum transparent_hugepage_flag { TRANSPARENT_HUGEPAGE_UNSUPPORTED, diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 3159ae0..1da6047 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1482,19 +1482,17 @@ static void insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr, struct mm_struct *mm = vma->vm_mm; pgprot_t prot = vma->vm_page_prot; pud_t entry; - spinlock_t *ptl; - ptl = pud_lock(mm, pud); if (!pud_none(*pud)) { if (write) { if (WARN_ON_ONCE(pud_pfn(*pud) != pfn_t_to_pfn(pfn))) - goto out_unlock; + return; entry = pud_mkyoung(*pud); entry = maybe_pud_mkwrite(pud_mkdirty(entry), vma); if (pudp_set_access_flags(vma, addr, pud, entry, 1)) update_mmu_cache_pud(vma, addr, pud); } - goto out_unlock; + return; } entry = pud_mkhuge(pfn_t_pud(pfn, prot)); @@ -1508,9 +1506,6 @@ static void insert_pfn_pud(struct vm_area_struct *vma, unsigned long addr, } set_pud_at(mm, addr, pud, entry); update_mmu_cache_pud(vma, addr, pud); - -out_unlock: - spin_unlock(ptl); } /** @@ -1528,6 +1523,7 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, pfn_t pfn, bool write) unsigned long addr = vmf->address & PUD_MASK; struct vm_area_struct *vma = vmf->vma; pgprot_t pgprot = vma->vm_page_prot; + spinlock_t *ptl; /* * If we had pud_special, we could avoid all these restrictions, @@ -1545,10 +1541,57 @@ vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, pfn_t pfn, bool write) track_pfn_insert(vma, &pgprot, pfn); + ptl = pud_lock(vma->vm_mm, vmf->pud); insert_pfn_pud(vma, addr, vmf->pud, pfn, write); + spin_unlock(ptl); + return VM_FAULT_NOPAGE; } EXPORT_SYMBOL_GPL(vmf_insert_pfn_pud); + +/** + * vmf_insert_folio_pud - insert a pud size folio mapped by a pud entry + * @vmf: Structure describing the fault + * @folio: folio to insert + * @write: whether it's a write fault + * + * Return: vm_fault_t value. + */ +vm_fault_t vmf_insert_folio_pud(struct vm_fault *vmf, struct folio *folio, + bool write) +{ + struct vm_area_struct *vma = vmf->vma; + unsigned long addr = vmf->address & PUD_MASK; + pud_t *pud = vmf->pud; + struct mm_struct *mm = vma->vm_mm; + spinlock_t *ptl; + + if (addr < vma->vm_start || addr >= vma->vm_end) + return VM_FAULT_SIGBUS; + + if (WARN_ON_ONCE(folio_order(folio) != PUD_ORDER)) + return VM_FAULT_SIGBUS; + + ptl = pud_lock(mm, pud); + + /* + * If there is already an entry present we assume the folio is + * already mapped, hence no need to take another reference. We + * still call insert_pfn_pud() though in case the mapping needs + * upgrading to writeable. + */ + if (pud_none(*vmf->pud)) { + folio_get(folio); + folio_add_file_rmap_pud(folio, &folio->page, vma); + add_mm_counter(mm, mm_counter_file(folio), HPAGE_PUD_NR); + } + insert_pfn_pud(vma, addr, vmf->pud, pfn_to_pfn_t(folio_pfn(folio)), + write); + spin_unlock(ptl); + + return VM_FAULT_NOPAGE; +} +EXPORT_SYMBOL_GPL(vmf_insert_folio_pud); #endif /* CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD */ void touch_pmd(struct vm_area_struct *vma, unsigned long addr, @@ -2146,7 +2189,8 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, zap_deposited_table(tlb->mm, pmd); spin_unlock(ptl); } else if (is_huge_zero_pmd(orig_pmd)) { - zap_deposited_table(tlb->mm, pmd); + if (!vma_is_dax(vma) || arch_needs_pgtable_deposit()) + zap_deposited_table(tlb->mm, pmd); spin_unlock(ptl); } else { struct folio *folio = NULL; @@ -2646,12 +2690,24 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, orig_pud = pudp_huge_get_and_clear_full(vma, addr, pud, tlb->fullmm); arch_check_zapped_pud(vma, orig_pud); tlb_remove_pud_tlb_entry(tlb, pud, addr); - if (vma_is_special_huge(vma)) { + if (!vma_is_dax(vma) && vma_is_special_huge(vma)) { spin_unlock(ptl); /* No zero page support yet */ } else { - /* No support for anonymous PUD pages yet */ - BUG(); + struct page *page = NULL; + struct folio *folio; + + /* No support for anonymous PUD pages or migration yet */ + VM_WARN_ON_ONCE(vma_is_anonymous(vma) || + !pud_present(orig_pud)); + + page = pud_page(orig_pud); + folio = page_folio(page); + folio_remove_rmap_pud(folio, page, vma); + add_mm_counter(tlb->mm, mm_counter_file(folio), -HPAGE_PUD_NR); + + spin_unlock(ptl); + tlb_remove_page_size(tlb, page, HPAGE_PUD_SIZE); } return 1; } @@ -2659,6 +2715,10 @@ int zap_huge_pud(struct mmu_gather *tlb, struct vm_area_struct *vma, static void __split_huge_pud_locked(struct vm_area_struct *vma, pud_t *pud, unsigned long haddr) { + struct folio *folio; + struct page *page; + pud_t old_pud; + VM_BUG_ON(haddr & ~HPAGE_PUD_MASK); VM_BUG_ON_VMA(vma->vm_start > haddr, vma); VM_BUG_ON_VMA(vma->vm_end < haddr + HPAGE_PUD_SIZE, vma); @@ -2666,7 +2726,22 @@ static void __split_huge_pud_locked(struct vm_area_struct *vma, pud_t *pud, count_vm_event(THP_SPLIT_PUD); - pudp_huge_clear_flush(vma, haddr, pud); + old_pud = pudp_huge_clear_flush(vma, haddr, pud); + + if (!vma_is_dax(vma)) + return; + + page = pud_page(old_pud); + folio = page_folio(page); + + if (!folio_test_dirty(folio) && pud_dirty(old_pud)) + folio_mark_dirty(folio); + if (!folio_test_referenced(folio) && pud_young(old_pud)) + folio_set_referenced(folio); + folio_remove_rmap_pud(folio, page, vma); + folio_put(folio); + add_mm_counter(vma->vm_mm, mm_counter_file(folio), + -HPAGE_PUD_NR); } void __split_huge_pud(struct vm_area_struct *vma, pud_t *pud, From patchwork Fri Feb 28 03:31:11 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995543 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1A5D9C19F32 for ; Fri, 28 Feb 2025 03:34:17 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 9DA8528000D; Thu, 27 Feb 2025 22:34:16 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 9622D280004; Thu, 27 Feb 2025 22:34:16 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 7401528000D; Thu, 27 Feb 2025 22:34:16 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id 51A4E280004 for ; Thu, 27 Feb 2025 22:34:16 -0500 (EST) Received: from smtpin26.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay05.hostedemail.com (Postfix) with ESMTP id 1263152560 for ; Fri, 28 Feb 2025 03:34:16 +0000 (UTC) X-FDA: 83167935312.26.0FCF9CF Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2089.outbound.protection.outlook.com [40.107.244.89]) by imf20.hostedemail.com (Postfix) with ESMTP id 1CF6E1C000F for ; Fri, 28 Feb 2025 03:34:12 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=bMoMcWzz; spf=pass (imf20.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.89 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713653; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=BPkfSG+iXeKAyAjVqa1PJ1E1IonWgQrZ7MBYoYvRJTg=; b=iVA9aDhYTN6eTeXUaz8hqvhPKaJ9dncu/Twh9TOJLbCki3JZ2HSKjBwvdKPzTiyBhFUUJ6 aydWJ0433cKCd/KeU1qZcHCJZtu1iMeqBqZm3v8DFdtGuHr5HPqE0uB18jdIUf6+E4yVsc oBVxyL5iGtN6C1Uz4cEF6XaXg4Mpzps= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=bMoMcWzz; spf=pass (imf20.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.89 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713653; a=rsa-sha256; cv=pass; b=cYXTrc3KANwX0Vxhakw0BI9eqXniraVTi0FZQxmtm79R3s5pLJoM7dQZ91L7xsB9IV85sq 8mhWz5umdzpse8hDgjpE6Vn1S/5l+lxYy29cPWUsk6D54hrgKCTRw+8BYzoaNZpNgfUPjS IajKg07XTbk703fEyMHc+NVENbfPWhQ= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=srC6fd1+4kPTJxcPm2tIMmc2P8e0/3Z44XVNc6q7T0EM5oFHllUdX+9WVlLTRprCAPRRQOdSfZZEMFY1kc3eyPElAZHSN53QnS7Apmm5F/KXbdB2aJdqHvRjucwO+G/gaDs9hh9YSGeIkLmTOpxnLJG54Vc5igoHCTr84YX3GPapn5efeLUCqqIYz8GexAxzzO7/HB9kxdQS8kr8KyaEm9Q4FcEe0FtBQbhG1O6jp7YlzTaJar2JH8QWjucxcDqFEp8D8npL1GtwFClxeIBtlAKaGwkcjNzsxrbDTFO+ripvCTwk+lwD6zgGLrYz3ptA3EOjb6TOezva14k3p5sIOg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=BPkfSG+iXeKAyAjVqa1PJ1E1IonWgQrZ7MBYoYvRJTg=; b=k+Li6wv4p0BCgq16DSeYWzxX26i+/2+IDsOC+WhuIikiGQK27YXFD9PkM+joHvNRMJS9JsCo65EsRfeA5lxO82i5sX+YI90jU4tpmrcax1nJJ2QonMpQG+GcHsCcrYSXTU7b+aJoN6cvC1wDbvD+9wzsx1DTbKkyS7EQ5dEATyBFfcYIBtiZbI+jojb6HIELe2BIhSjRFTSSH/y0eSN7zqsD+iP+2pJVUxm7JC7hiHyOvihR5PIbmoZXADT+DWFg6HEQqn27o4lI6qeFyY9aB2r06nBX/Wdf97zxFWs6MbXQUDe78X0HPgjM0rsePm9PziqcJ8X+KYiQg0Zj19ajiQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BPkfSG+iXeKAyAjVqa1PJ1E1IonWgQrZ7MBYoYvRJTg=; b=bMoMcWzzngxN1jSMkj2ASvWN7Wq0JNbJRq22ehOn+BuBMD9SBPyM05dJ5o2m+Gux1+e9exVBW+1xkrkZu0Zpam9BzSr2bxTvMu7c+e1qBxdtORBbvgFEA+beNtFJIs9MNivEpt4GZjT8FvwwDLScQZ5WcJ8l6L8SD9rA5x5KRq/3n/7VST+hfa8JHBxF7DwrcPkz37h0qUpXHJeMEP0WoPrfq/e5/1nLGlcMluzg5CDNwu7gHtI0wlKGcQhR09HJczYOudj8jzKGmDaICdhcRcbpemzzBsZ2LhPl6B08xfPoZOR+z+Rc8wzHkjjG0Ur47rF3dJwA0AdUZTxcxc5fkQ== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:39 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:39 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 16/20] mm/huge_memory: Add vmf_insert_folio_pmd() Date: Fri, 28 Feb 2025 14:31:11 +1100 Message-ID: <7bf92a2e68225d13ea368d53bbfee327314d1c40.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5PR01CA0023.ausprd01.prod.outlook.com (2603:10c6:10:1f9::7) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 5941775a-8968-47d8-81b0-08dd57a88d04 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: MvC7HML4Aah2rfTieShn566whk6Rlg5KX+keQR1V8QmVm+neQdtMU/C6TPYXo/m0QWWlNT4l/CZeNUjr0DdQmVkEseDtT7x/nOrHbXSr13S71OnDEZE1b5rOKFywpHcoChfNStOxe9P/veUtzI9gR2RKNsVW2a3ta+oth3zsgcUv/zbqjG0IPoUtaOImCW0S5PVLvpDWiI82Eif+q0FCaItHs6P2+JRYEfJTkicfIYVrwH6y/ELCFtR9iuEU8JSFoiawjPQQU0xN8U+38y3ZYHWtcBQ7uhx+OfIZDFWo2JliOwT7lH0TKrHiQEFXYhemlmEMSBNHrZPG0Sfd8VagXX3J7TcRDr70pD8m0MmR/oNHOa9hFS5Z970YFFEkAaeBAesCR/XWsScMGIrUtNezbc2WqNYn6dXx7TcKGQ72fU2HXI4wr2kWTWTFtPgu86IR7ZR/9hjMCnisOPdtjh9IgYwI0+BYGN0twqoKxtYad4Ht3YWQgCpuaEcyFK9fU90oMJmyXskIrKPIqO6oiKkSxcEb0cWgrvtfIK9ryNSKEiKckWhtDC/brSvgSeMhlMXQN12478KrLFxLfrw5MIYiBegjTIeofdyzeWHrwd0DUPBPUvJQrifuS1tCLo98CI5mq6PJpgC/yqFiaHJXd3WUsI+gDc3pnCHLfUDUDLhh2T599Z4NAULaQr/1ouEP9BJGBYR7DItc0i0jwo4VfwKmOSTruRKg9kjyqTi/2RaRe90cDZ7zm1P0w0rk3I8tOV4vcUnikrrYLWjWF1kudMxcVaDR3KyNbIde4N0YaMLsHlxzLATcEKNVQ1KKxWd5hFXXet6Fc0qofZ8ruebWZ+7KKGk9ikkdxteMNvW5pjVA3XJ+z8/7Wt1iwrRVATvDjsE+q6c2D6a9mhepAFIi/nQAXAsMX96ZPrCDGyaDZXnJVpiYDcaNHQKLoec1vZrS7CX9DIwFW1yC0XiAIoJqdT01XMqBeziYYEZ/SNk2iiMhx5yOqDyi6UAEPlJtj/2Fl4jhXOque8qg4GTActqmQU/bpg4s7W3ySBKvsaxAtWgy5c9BP2G4jN92oVaW/FQWi6vJFNcH0ldX9riwdtqcSqnb5DB2Vy+dk98lnD9wHc3egKrTXd2W6Ljoix/cDL7OWqDNjljBOl3HdUL2VJbvOvpW4iN43+v2DURr9Q38yK2RjGKvYnzDbmKiXkW2jZEsh9f4RsnLIlRBTqekgzFi/AmWWiE9wkQXdSGLWYFeo04ObehPK6jFHVQKhC5o1BN+mJUtaXUAl9+DND5EMPMhWCzAv4IYso51td3bDqubYMuS1x1CyN6oOUISLAK6xyNmpSQtZiCrS8OFMFAMCPQ9KqjwCaSlMfKKAPJdmmgfqNOHFbcxGzb4PZUDJnOIr+LCpBJv X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: HFBFmEU43RPxA6pvEJ+EhXNguuPyKNkV+aUbO1NIkLk5dgoCEPBhsCBsAUqyNs5wD1AHDjExb8ifz/5zxlM2+mXr7ilx0F+rvYycxkWbP2oOmjiylxcKlxkvixnsoG+kbkt7YFaN+vQTRVnPyXL9Dop/1wlPz2H3Q+9ocQ60QGzn0Y7+lrIG7ltyk7rY2UoZrb+9yHEBjxfak+/bE8ODUWPfK7/8Ue1OHWkGKDmRsFe5e2GqkT2zKiTWYujbKpEGEajMXaMV445YgDqE/nNbKUkkAeHziJ3TQX1Jhr2Og8CqC4HdtC8I3uDnJ6O3MXPAsrp69dYX7sdOSR+ExOei6jhP/HXs1Prq6Fjo7Alva3+rq5I9OayRsjFr8iWVLa5k3TsoGnHU0I8KS1ZaRupBnsnqCqf7dinEuX4vLrkacHMf7s12GGgLaIxQ/fWV4FqdBf1Fj+LaZwOWWUuQ2lxVq9okPzROLYcfRXKg1/aSxHZ42n+a+0Cf6H8T4ak+O6frwBTJ4y8DMAVRKzCSYJTGJ6AO8IFBv0XQk3/ivRZEvx1I5AY7jSvvFASNrJtfHoA0FR5blsHKeVnBsJhUexLGqsREnPty4JLv5lqCXtZSXiCSYf1r6avU6lZHEtSlEDFjHypPtxLmcsN4btPn0Z7x3cnCfnMkZYzQ+gdjzYOZK9kMFdwyqG7aMto020sqAi7OndqQNNTEcAGTRSZJlhyeIoM5vFOtnf/PXQjiShcJ8fj7xGYT6kIMgCz0HpfcQN6r9Vgd1/rtC/jIOglcnmv7CJt2ls8+CgCFSxAJ6Qq8BVzwK8SJAHWa6IM+MQZhoszG0oh/GU+QkT6kc4DJ3hSsZk0e5eUrYDsNg/ZjUY14hO9ynv7HUU7k1VpgV1WUVKFHsZPCtTY+AwWeLENsScNxB+SyvDD2MRVEune5WWQJsqQUVaYhGrpVedpoS1TZ5RrJsy6AoQZM15ZoKuIpQZ4Wx4thYmzPqT2KYm9BdLYTsISl8CUbBhvX65OFOYec1DLEh+8fltcCd5I4I0tKOmh12vAv6FtumDdHsltPZsfikPYUtqcu7m7856ZtOJFkSeAAqP8rIzaJ/pJxaGLjjLVP1PN9iM2q7s//As6Ro5VzLJ0Jt9tG0W6YCmzp5eIN8XZ+nSIgtCMf5rmYKQCmAAl4S9/LrOGODSzQunJCdpGCFWVhFhY1ESw2iTB2SQ/k8bhxb058PK4CdQq4BaMSRJXvurN4D5fSU8hotNkhUdJAy/zEow05giDqFOJg+Whb5u4RXMYfqQQ/3ad07SzR4snAiVg2KzBBEPcxd0dUsbQj5bbcYlANISDQJ8uXrEEP+VAO0jBCehmFz8nQbXYiyJ7YfNDJIRKbppWz5JChLlI9v/swLyOiNVx6+TGhRKdQWVrCk71mAYjPTAQUZH5nqR4auHgldZJIfcgB5QEBhiSXCMuAN++dxN+mtc2VZ/3jZRGH61LhRg6ghTEzxma4T7oK5E3SqZZq9sADlwMCWw3OJ5NC7+8OtyOfX/ZJcS8KDKTfKUnd9FHJmQz1E/kF+wAu6+Er3qcnYE18dJm8Azn1JFju3a+tOy1QWS8/fHz1iNu7 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5941775a-8968-47d8-81b0-08dd57a88d04 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:39.1551 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: OA9hEPlJMjbw0LCs277PihIa2LjXfBarf5XLG0yyLM3HKIWC0qMUMvHVFd8srdB19hx46hQAvKkBeg6IoeQ5fA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspam-User: X-Rspamd-Queue-Id: 1CF6E1C000F X-Rspamd-Server: rspam09 X-Stat-Signature: xwxmjddt3me6copnu19mterpt7qnhmnd X-HE-Tag: 1740713652-832693 X-HE-Meta: U2FsdGVkX18Z7xGlc8vHTHTV0KIGn33FL4tNl2rO4pp2/00FFksltrpjXNWYppF4SxyX7g/+qM/AGsOm7JXEHwvUsyA0TEscgtTT2aA8KhrEp6b5EVvp16i6lU8dSVVX+6mcpYTdik/YqvS4pb6FwW4cKdc0QV/FUBLyqOxA/p+PCXmU8NYjSL7VP0YL70rvuSyKOl1tjS0KmXUGkJhjaKMkUzkjWqkBj911n3M70vzy7GQq+yPHBj73Kh3BNfvegMYhF4L5OZHvWuWQX/68X1WEa4oUefkBDF5NHXESjBKeJH0nNg9qMbAQbhPxnSFFf4Uu8kh2HIe3GEATKsogdxd+kLxuFEozX2lK17x9yekjEqXyeGqmPkbqQtNk4EDrndMv9kCM7ya2QwxtVshRBzh0VWic2yK065mlTNFgZK810Yj2/U6Fx0V17ft2FlG4bzoyrGKllg8SpAxwgsJIvENCanbRaaQhcWeD+CyxA+7mozF7pwGipKiuaJTE/BUEPoQkXRT0TUqT0k5BN1OSIQsE4Uj1sPfUdm7zE5fXLB3/kPyDGPSgnchahje76GRyhw6HQnCx7b574NL5mYe8bsEzpL7gGTOXG6pJYuLbBnVOfnSZjPmfoVn9YJ+fvshTlZyBUuPueNNqHDkFumLoRGc5EKED+ktc3wrXdfFsx4X0YyEutfWWQ0019+L18vYLFLsSYpxFwOIbecgk6loDzNejJ33LgEQo/W1auqJEGQVMYo07WJGQ7URy6z9CCHvLAFct8MWDp728Ts0dchmqsdPGz0bsc+trrkPVCL6DVJ8ss+EoJDWceKe12t8hrUSMcTFraPpwm5NhnoOzux9PE69UrC0l6sAfMVEDc72UaTIjGKQVmAxO8GydSwr9zCzZwmtZU7F+OA4uAVr53m+8YqOiMnde5exfCNBW/Tji+9CWcuYtHh6kDAB3KsyMe0DnU1EYWz99K/SHqFE552B T4kRiBuK JS020nMZQ4JcBu4+yCLGyRuhmeaqvUj2VhPNMfMguihK2D4DSWCqkfcDdwWM2QOqWGjHotyODZjYt+Xtwlec8137iRIJOXYKifibWZDtvb9hspksbWDbh8/5QSwZqF3fzj8TXBPAL1D1gXZLhcmc6LZZEaz8u1t1Wgla/YggMK3VSWMcW2zn3DKiga7VAlRbghIfIC4+bYbraU45qBXJGiKmaPG9iems6Ty3HFbN/huNVZo0H1nlcu+HEw2OiZjPp39YfLIYtnUGLZZNmmTJMJT/jpsmbbHb4QS9aI/81WmG1/KOfK6DfznphhtiW+am2s57n0h/RH3rCKrsgjc3Vo6LkEhXsqoaDn73Sx4gdu75NjPNczr19Kec2suvPv6WkUZZxy4+MgKmeoP37qy32gvAjGwttdakdHVgcD0sjbGZDpIVfNFy5zSTm6wfBPgBAPzVMA23n21IOPh9ujpMPMMZusffvqI9Unu7j1DUIUKH7Cmo= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently DAX folio/page reference counts are managed differently to normal pages. To allow these to be managed the same as normal pages introduce vmf_insert_folio_pmd. This will map the entire PMD-sized folio and take references as it would for a normally mapped page. This is distinct from the current mechanism, vmf_insert_pfn_pmd, which simply inserts a special devmap PMD entry into the page table without holding a reference to the page for the mapping. It is not currently useful to implement a more generic vmf_insert_folio() which selects the correct behaviour based on folio_order(). This is because PTE faults require only a subpage of the folio to be PTE mapped rather than the entire folio. It would be possible to add this context somewhere but callers already need to handle PTE faults and PMD faults separately so a more generic function is not useful. Signed-off-by: Alistair Popple Acked-by: David Hildenbrand --- Changes for v8: - Cleanup useless and confusing pgtable assignment. - Fix line lengths Changes for v7: - Fix bad pgtable handling for PPC64 (Thanks Dan and Dave) - Add lockdep_assert() to document locking requirements for insert_pfn_pmd() Changes for v5: - Minor code cleanup suggested by David --- include/linux/huge_mm.h | 2 +- mm/huge_memory.c | 65 ++++++++++++++++++++++++++++++++++-------- 2 files changed, 55 insertions(+), 12 deletions(-) diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h index b60e2d4..e893d54 100644 --- a/include/linux/huge_mm.h +++ b/include/linux/huge_mm.h @@ -39,6 +39,8 @@ int change_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, pfn_t pfn, bool write); vm_fault_t vmf_insert_pfn_pud(struct vm_fault *vmf, pfn_t pfn, bool write); +vm_fault_t vmf_insert_folio_pmd(struct vm_fault *vmf, struct folio *folio, + bool write); vm_fault_t vmf_insert_folio_pud(struct vm_fault *vmf, struct folio *folio, bool write); diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 1da6047..d189826 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -1375,20 +1375,20 @@ vm_fault_t do_huge_pmd_anonymous_page(struct vm_fault *vmf) return __do_huge_pmd_anonymous_page(vmf); } -static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, +static int insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, pmd_t *pmd, pfn_t pfn, pgprot_t prot, bool write, pgtable_t pgtable) { struct mm_struct *mm = vma->vm_mm; pmd_t entry; - spinlock_t *ptl; - ptl = pmd_lock(mm, pmd); + lockdep_assert_held(pmd_lockptr(mm, pmd)); + if (!pmd_none(*pmd)) { if (write) { if (pmd_pfn(*pmd) != pfn_t_to_pfn(pfn)) { WARN_ON_ONCE(!is_huge_zero_pmd(*pmd)); - goto out_unlock; + return -EEXIST; } entry = pmd_mkyoung(*pmd); entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma); @@ -1396,7 +1396,7 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, update_mmu_cache_pmd(vma, addr, pmd); } - goto out_unlock; + return -EEXIST; } entry = pmd_mkhuge(pfn_t_pmd(pfn, prot)); @@ -1412,16 +1412,11 @@ static void insert_pfn_pmd(struct vm_area_struct *vma, unsigned long addr, if (pgtable) { pgtable_trans_huge_deposit(mm, pmd, pgtable); mm_inc_nr_ptes(mm); - pgtable = NULL; } set_pmd_at(mm, addr, pmd, entry); update_mmu_cache_pmd(vma, addr, pmd); - -out_unlock: - spin_unlock(ptl); - if (pgtable) - pte_free(mm, pgtable); + return 0; } /** @@ -1440,6 +1435,8 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, pfn_t pfn, bool write) struct vm_area_struct *vma = vmf->vma; pgprot_t pgprot = vma->vm_page_prot; pgtable_t pgtable = NULL; + spinlock_t *ptl; + int error; /* * If we had pmd_special, we could avoid all these restrictions, @@ -1462,12 +1459,56 @@ vm_fault_t vmf_insert_pfn_pmd(struct vm_fault *vmf, pfn_t pfn, bool write) } track_pfn_insert(vma, &pgprot, pfn); + ptl = pmd_lock(vma->vm_mm, vmf->pmd); + error = insert_pfn_pmd(vma, addr, vmf->pmd, pfn, pgprot, write, + pgtable); + spin_unlock(ptl); + if (error && pgtable) + pte_free(vma->vm_mm, pgtable); - insert_pfn_pmd(vma, addr, vmf->pmd, pfn, pgprot, write, pgtable); return VM_FAULT_NOPAGE; } EXPORT_SYMBOL_GPL(vmf_insert_pfn_pmd); +vm_fault_t vmf_insert_folio_pmd(struct vm_fault *vmf, struct folio *folio, + bool write) +{ + struct vm_area_struct *vma = vmf->vma; + unsigned long addr = vmf->address & PMD_MASK; + struct mm_struct *mm = vma->vm_mm; + spinlock_t *ptl; + pgtable_t pgtable = NULL; + int error; + + if (addr < vma->vm_start || addr >= vma->vm_end) + return VM_FAULT_SIGBUS; + + if (WARN_ON_ONCE(folio_order(folio) != PMD_ORDER)) + return VM_FAULT_SIGBUS; + + if (arch_needs_pgtable_deposit()) { + pgtable = pte_alloc_one(vma->vm_mm); + if (!pgtable) + return VM_FAULT_OOM; + } + + ptl = pmd_lock(mm, vmf->pmd); + if (pmd_none(*vmf->pmd)) { + folio_get(folio); + folio_add_file_rmap_pmd(folio, &folio->page, vma); + add_mm_counter(mm, mm_counter_file(folio), HPAGE_PMD_NR); + } + error = insert_pfn_pmd(vma, addr, vmf->pmd, + pfn_to_pfn_t(folio_pfn(folio)), vma->vm_page_prot, + write, pgtable); + spin_unlock(ptl); + if (error && pgtable) + pte_free(mm, pgtable); + + return VM_FAULT_NOPAGE; +} +EXPORT_SYMBOL_GPL(vmf_insert_folio_pmd); + #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD static pud_t maybe_pud_mkwrite(pud_t pud, struct vm_area_struct *vma) { From patchwork Fri Feb 28 03:31:12 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995540 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9FC22C282C1 for ; Fri, 28 Feb 2025 03:33:57 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 34A7A28000A; Thu, 27 Feb 2025 22:33:57 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 2D22A280004; Thu, 27 Feb 2025 22:33:57 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 0D6E628000A; Thu, 27 Feb 2025 22:33:57 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0013.hostedemail.com [216.40.44.13]) by kanga.kvack.org (Postfix) with ESMTP id E0D2C280004 for ; Thu, 27 Feb 2025 22:33:56 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id A0B9A821CF for ; Fri, 28 Feb 2025 03:33:56 +0000 (UTC) X-FDA: 83167934472.02.F3DF8F1 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2054.outbound.protection.outlook.com [40.107.93.54]) by imf20.hostedemail.com (Postfix) with ESMTP id D9F4D1C0013 for ; Fri, 28 Feb 2025 03:33:53 +0000 (UTC) Authentication-Results: imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=d5kkdWEL; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf20.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.54 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713634; a=rsa-sha256; cv=pass; b=q2aVvpDgpcoLtTdGCynGN2+JBOdDTMdFcWfqES8S0TujEfeASAQBjwvUsvwl05WH5EVfgk FhGOATac9vMfPDlDqfO4yHPRpOrBdtAq412WzFlbZc/qQkZxyJIDA3LQUoKLsH6DcEmID9 ChQwEZmJlXppLGPFJmkt3k8aJ1HWcsY= ARC-Authentication-Results: i=2; imf20.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=d5kkdWEL; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf20.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.54 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713634; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=VENjcw892hS6S6Ehnh8sjwSOcKFsy/z5WoqOWPzB/Kc=; b=zQfcz5Pmj+sNNdTi2d0wlQ/SedQc6Cfi8mNj8gQUDV+b+NaniXxvlKk7gsLYvUu+kkVSHa 0vWt/+TNaBqm1Wz9F5HxC288R+rBkiis4MX8bJK9S2499+4MgMaQADqgudvkrmLQ01cpQU LrRz0jix1W2rUyj0DHfEVVuF3FdTjw8= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dKPYxj/LhcO75zkrJSvXheE7nsnAPfofAe4bNu4GpAaEL73Emyb6SHZvKnWWbpV9AgC01perdAb8BdGeHOKmbg1pY/GFitjpcv50vcZppzoockRA4seahU9CzimM/eFtd6ZT5YCy6e1PmfI2WfOiUboWZISFc15p9XCclh8b00d8WTryiRgxkwYJz8Mq49Dbu8RbN0vSI20U4zX4ttLNxaAqBwJdtPP01KyeW17gylapjO+5iGvwGYwsD9y+mBibw6qNrbGKWxsZlCpPUCfe5NXxvLaqA53Dr2HJymU8ePvVwB/JkVcOWm91HsBjZ6yce7L2Evv16yBOlZAGLepXEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VENjcw892hS6S6Ehnh8sjwSOcKFsy/z5WoqOWPzB/Kc=; b=Qmd5QPfRQ1mBq9XoYSyJonGdyy2k0Nc37Bmk+NVIJqsE+pDmuzb5lXr08U1g8adc5kzd/kti3otuM64r+apwK6Am1CXA8/0QoTZPjd0iX2iByhEsj1psFUzgAzmR+Yxlaw1eC2WJne0YPIf3W5+XOklwWTm/KKN2sYzHeErNJTf/w8qHrEQT096WRTXhZmOB0+AosoVeJnT643Mz8tK+FtPa4aPiV6kdvYAbXSYGCHiVVgm80//EMLGrQMCEvXeu0ER+Ja5dHdgX/YXqxdUU5N5/SveZpKvto0AFZwtzW51Q5wrZqBctv0tgR14/0YCZuI+1ziJQ+JYFRePFUdUoMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VENjcw892hS6S6Ehnh8sjwSOcKFsy/z5WoqOWPzB/Kc=; b=d5kkdWELQG6vgoc5cQu+L1d3pKPR+ymTcSNkDQiWnqCYyrGjrcJz5rEP3Ycu1IpP9J6tLIRSJuu0PB7TsUZeyEdFRW8SADH9IJJlIh903/t4I1Xx98uW4ComGzva231dWQiCi935f+mER6NESpdqpoZlIcFd2FS6aaw27hhQspln07fq5W5YQKJrQzLCLBEvrcBCgmyvkxYbdrbb7U+J04oSp0Yfy/R6SW+g0mK+/if13bC9oXnwjN/MX0hKl6T0jp/qJEgapOvl5oYUdT+LonqT9g3Z4HYsyR3vDeSkRIgtdajY6vCYfK5jZnv3NJ3PLTeFvbCdFv+5qrWxUmmaUQ== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:44 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:44 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 17/20] mm/gup: Don't allow FOLL_LONGTERM pinning of FS DAX pages Date: Fri, 28 Feb 2025 14:31:12 +1100 Message-ID: <250a31876704b79f7c65b159f3c835e547f052df.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5PR01CA0016.ausprd01.prod.outlook.com (2603:10c6:10:1f9::17) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 3f2c25fb-8a8f-4ba5-572a-08dd57a88ffc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: PaVWVr1K7H5NLii5nNPL9JPr/H9C6/2c9vY9pve8oNUDmqVpGI1t4/qnrznByHvW30NoC4tytu7sn5lCRWCyIzDJHO7XpUw4p3p890iiw1llmnCWVUxRbsRy4DUaoeeoiRPOus+/A9DnMn3hcU4IzDPk1ZxOshZNHL1zet1nNZwMoTyDPlhRFgniyB7FMQhWP8QcS0lTaPBOtNbnJStX9EsKo/gCWAUHhPzWsJw+Y6/EAPjUbh0VE1LrE25qxLPoDECitMSzLlSyll/XXKwq2Ae7tMKVtdmYxcBPqDvJt1IX0Ktdhb0xuxPvTIcAox8kJ/0kWOemjEiOybh9u8AwMAdZiHpzVeQXepedFMZ0yOvsVtP7xMCeyq0H5eaNzmKmFQUeYcHkkL9aE+q7wIRythXtgUDMFN+3FXNFdxZlHBNOFYM8N0vtkzKNtNgaphMuA6zqcGeyL9AX1L1Mdt4PJ7q/Mj+lYCzFk4AzY87Sgc72uJHPoxbzzItEqQVkoXd7x59oVAEJygT24EY0Od3xwsGkZLMJjgT6zvGdD/VlcpKT77SnS7M4RjtNed+QJRL/datdiEE792cVEaF5jXKZW05zWuUsZPkGwxKzBD4ORWMlvFW6yst6GYXgO/uhXQUOLDZzuCAEzxgsbqWOltSERNdBF67d33UNSxuQvtuTDh7ccQ7zpDzWq1oeJ7Sc7z76qM98V5Z7cQcuIYHioUc2g5zGB0pNocXpJUyQeGO+k1p1d5BYld8eUT5tO6mSX2KoKbI0+oLXWBX3ovCub28NTcdWgyTrrfaFLas3NV0rtJW8gIxzjczzzf4kXgEG4eKKib7rlXE/j63NTVl2dvzDbviq9xmOrKaluWmBwHu79UNd2/FtN33CRtZVhLFTkmratSQX81WGtc6/yiwK+pMHwVrG3fxWp6PP3+D5yqG0RvjEr28nNFmbSQG7SgzP4UC+VAExCKUsCnqtNn8io03Tzr+Xm3qw4Z6ZLtMUUtbxgz08BRK0H8zHGHZuEjMAPOWiiZurY/3lzqDRmP4ege377mbIxJL4obYPplPXKrj9Mkks6t67shSjsVpHuV6upG32sRzUzA5+ZyiRsKtlIa1/g3DNkfVZaVThR15896DUrtSut01lH3IzrFZeuRHgOvXJ7xpLUhZ+jDCCtSRgAmqVnGT5/qkOPWmg+BN1EDTGpIzBipZwBfBg4rdvKKnihUiBCuOMpbk2QxcEz8fgPXWgqODnubFutMREs0Po8YOGFdxpyGkOILXqw7NFzPemwuaCBjsPrF6MB+CSuRSYn2GKVJqFGGKxjxTLJ4j3ALFoIVgpNA6+3Xhyf8EFDNccG2gudGwFcV6xl+bOaQoMwTV6B/IK6Qp+iPR3mpfqo7LKyA1Lyef7A3H/XD5PW+ccn+L9 X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: 17+2Yc0d+py+vZTjUdDAr7fRzfwqbkVIKmDpHPYCYweOcGCJjsXMlCEB1qsojdrjkevm1Qg76gRVyCoc7qUOurDu21Rh9iDmjRmpJgZfnNLdltGj/Kw0dcDXk8RtKGcu7NVuWoiTDKYLxM4l8NY0LQp6ctdQW6APRkvWG5UzTssEiG+XqC9zOMecC0Nob5ylQSpkSP05EvmGMSY2lAHA3RPQX1wBb9PfDxNJc3CveWtoEHpyTnllV/x/ob3Y5s8tCU9L3vZETxfu9pV2YTENG+Qt48O01t+pdh9rCelNrqB7yXeJ3V0bNr4001YDqniHQ3csXORHABVm9guEgXxKBnGx2MDQbLeRkp85bBNrE3XUVa1pJoRJZ4pXfCtGR1DFst6KxE6RzllBaWyy+IgR4kjIeG4vI1nW2oTa80gNioVpsupvKPzbYS9fbdOEL1wStJTnY1HwpqZJPd4xBWBZLXy164/gjM+1NOMOiLDdEVhZ0CS+YEZsHywd3VvIbh/lPBGC6X9Z73e0Ih8Ln661n1phz8t7QnyTH3+QSG6NVEWQSkZnNhGGHtY92Dxf4mrCuIqn+kadLWJt8pOPWtBelFLKxUQ5LgwQygWaTM6mVBSemksTZGfkwsPNKNd0fQ5/4r3x4KnbDdEH8PQ1HIc/9PmiHv2WkXT89K38btOfRb9UnqxRkGC9+8YFTgZ6hmWFto7P+1fdg4i5/5YAe9cp/fM4dHk3OdR6G8GVFYEChGS9afKwugEAOxiQ/7I2UPKYxKzcduQCohpZMbQsXepJ2EvokPPAvG6WDpY53rT0icw1t3OqD9IVcltnFt56o8GKMMTtAYa3CoRX+50pY6JPCz3O26FKZtfGGol4kUKycB0rBM5bPRi/qGXOgt07ehlj7bOx5StCd3BD+yHMWYsxDkxxK/i2zfe+h2fo6W+uF0ApM/vTaSyw7LqQiQpgIncQxhTQ19BLXk1oouLOnVTuxTAMOYWn2u8JMyctXTNv9d7DAFlzd0Eiwn+uLyxGqGYKUIREbDNW0GAksZOTP93UJ6HCB12NZsPCY794C8BgiJ8WYaLbPIlvYGDs74OVNVpZBl7Sfkty72e5h777P8HHr1TcWk+WPj/xYpwBFmGZv+I23T4aQT5Q53Gz4JSQzw6GuyO2qde6EX16VnrX/inSCyZSEA1YHZo5Ofm60hrg7NKw/SZbsZLKGn+1maNVQ5S4ZGHRL7DWBtrwLyuPNluYRCk3K53aB1p+kN2q7Khluv9WwzILPm+AanJk8GQ9MAGCBbkL6AB3TLlIoOAhjYMYD71/3SRTRuKhLVnfo3dEdhRqN4DmIywDPHYGtQDiod8kOfNq5x0WIMDMvyqJWDSZeihZTVcGdnFdYqWGqlUry3JZb50gZd0+sCODuO50ZX3QWvVu3hVTPEEPF6LKy9sm8h5HQYfbtxlw5YGTJKckj/6h/MfHcRTV2vAN6yNs3AzTTaZMZhFY+TF5kxdGyQM0k+The6BF3YBv6/1dIFElyw1BZKGrHKY2mcegIF18YH0Tq57Z51LXu93qCdCqaqiDd6V0G0E/9RTIhel/ahxVNKUMgy6ijri/F1YaFP1urGn4 X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3f2c25fb-8a8f-4ba5-572a-08dd57a88ffc X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:44.1210 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: kDAa6kFMKEQ5qBVpvdjk1gPaNXdKvTW8sYUxiL0TtTRzQgLoT5tqtuP4Elj+bcYucUbL6Ot3axIuRPZby+DkDA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Server: rspam08 X-Rspamd-Queue-Id: D9F4D1C0013 X-Stat-Signature: 17bbe99gi99e3eafjuqgyj7zdzfej19e X-Rspam-User: X-HE-Tag: 1740713633-110514 X-HE-Meta: U2FsdGVkX19fBQhoYYPKjYxYZ7mPocozYDxpESmxpcHFLKTn47NdGqZEWRMYhMQVFfu31b0DuyFB8VKnbrtBQAzFlSC47hTuqZszU3qILRJEVa0r5LpHLGpe+lcrzFiFlROtCDbBpMQmGRGC/Cs5e/6f9xZ6vRmcQ2kyMduT20Gf1vkBEMxqK/Mxbr7olpIZnWKdiQif3hvn3uYJWhUFVahlbIwxHyOKXGryUWqZ6Lmrx+5M+wre2u3Xh/Jw9OV+29/O6ozOo9f1aF1kfXjyQs8wCoJmA7/+7AK6aoB+140/BJ0xSxN2y8MJVSiM/BNKtgUlOcMArqR7prsUiU3Uw0GFUqTvwETqVljcfLlfw/mJbu8tJqirIeJsMDTm5ickHz60eZelcvAmHCiBqGSOnFB0pOdAQjYjVGEylQo4VuhTpUqPNmGjhF3q1nNhO4uhP+40jclRqxWMMexjHYPfOjlVXsdbKXE69BPV0otVxamAm1+AvT6C/+2HoyoJg+vfI3tGX4zhLUZjIg95cgG95RIGYyME/r35blBWNkuGb6EvTlE2DSMNEuoSGhlTNZNOGYXedych2+SyTh8oGcSX8je6mIIpYFCJqqa+9l3UjSnAhP5Wve/bWYP5zdNLdkx+jgJhBzVZBSOSkq26I7R2toC+NUo6msQv6eIoBaxyPNBbNzpdpgRJ+KuO1yVBLw3QAjdfoiBsARrzYT/DlU9Mhb6Z2HvN7/RfAbb5pUUCyjYaRVEQ9EqrXvKyp42pEsBrLA20nJTnCQHhcLXsOce1SgliV17yEqA3RWScywMTzjji0zZuiqqGLD+eMzdu6UP0+a21SzaeBfJ5UzlrFsvytRv5zDpEdkuRpndeber5E0fMV2tj2b/8Z8ZTEJbFUN2Z+XmBT+EK6KVsFzZdHu2+Kd+7RpXV/RQkbJrJWv/SZpGG5R6hR/F4QpnYbYcNim87mXcqePczeWBYaGZIj9s 9rdQDm7h bfhUCN339aQ3eq69msTyGFV2xHfFVN17/XAT9aWP1qj1lMIz9ZAsQSGKEz0td0Z4m429//LFeX7O8ibTS5Gw/OPbM7hrGYYRZ6bQz4BLiIPKYMG1vxwdDBkti+uXIEmweeaZtxuF6Kokc2xxTtB8uQaEo9pV3a0Drrxcx23KztYsPcLdOQnaVUTzsseIj1LEURJB/4V8ivuEZUBf6t8q4wfJ1ba2S94c28x2/CqLh2Gw4LL9foH71hbFlOEN4OGoEP136b1my/9uUwkYegVB5TNEeua03p5ljO7KIyMwu+QCPjYSzrfMD+kfHfrHyWSv5FmG+GkOJmRDSk7Vpuppxa9sH3J/OZyMiurVkcwq4WgDrrAj/DzXutWHI7oPMk8FxjQEox1WfYKB8pdMJkuI61B1E9nRSv8iNxHb5nPe0YhfQvdQvyDbt7zERMpshypT+ttbdoHviZD6nA14qXEn8zYfzqaO06l3mBUdCtGqhoNl4TIouuBw7ottnvy3zRsk0h20p X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Longterm pinning of FS DAX pages should already be disallowed by various pXX_devmap checks. However a future change will cause these checks to be invalid for FS DAX pages so make folio_is_longterm_pinnable() return false for FS DAX pages. Signed-off-by: Alistair Popple Reviewed-by: John Hubbard Reviewed-by: Dan Williams Acked-by: David Hildenbrand --- include/linux/memremap.h | 11 +++++++++++ include/linux/mm.h | 7 +++++++ 2 files changed, 18 insertions(+) diff --git a/include/linux/memremap.h b/include/linux/memremap.h index 0256a42..4aa1519 100644 --- a/include/linux/memremap.h +++ b/include/linux/memremap.h @@ -187,6 +187,17 @@ static inline bool folio_is_device_coherent(const struct folio *folio) return is_device_coherent_page(&folio->page); } +static inline bool is_fsdax_page(const struct page *page) +{ + return is_zone_device_page(page) && + page_pgmap(page)->type == MEMORY_DEVICE_FS_DAX; +} + +static inline bool folio_is_fsdax(const struct folio *folio) +{ + return is_fsdax_page(&folio->page); +} + #ifdef CONFIG_ZONE_DEVICE void zone_device_page_init(struct page *page); void *memremap_pages(struct dev_pagemap *pgmap, int nid); diff --git a/include/linux/mm.h b/include/linux/mm.h index d1f260d..066aebd 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -2109,6 +2109,13 @@ static inline bool folio_is_longterm_pinnable(struct folio *folio) if (folio_is_device_coherent(folio)) return false; + /* + * Filesystems can only tolerate transient delays to truncate and + * hole-punch operations + */ + if (folio_is_fsdax(folio)) + return false; + /* Otherwise, non-movable zone folios can be pinned. */ return !folio_is_zone_movable(folio); From patchwork Fri Feb 28 03:31:13 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995545 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4DC79C19F32 for ; Fri, 28 Feb 2025 03:35:44 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D367428000F; Thu, 27 Feb 2025 22:35:43 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CBF60280004; Thu, 27 Feb 2025 22:35:43 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AC45728000F; Thu, 27 Feb 2025 22:35:43 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0015.hostedemail.com [216.40.44.15]) by kanga.kvack.org (Postfix) with ESMTP id 8A34E280004 for ; Thu, 27 Feb 2025 22:35:43 -0500 (EST) Received: from smtpin03.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay09.hostedemail.com (Postfix) with ESMTP id 4A668821CF for ; Fri, 28 Feb 2025 03:35:43 +0000 (UTC) X-FDA: 83167938966.03.2EDAAF0 Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2046.outbound.protection.outlook.com [40.107.93.46]) by imf05.hostedemail.com (Postfix) with ESMTP id 86357100004 for ; Fri, 28 Feb 2025 03:35:40 +0000 (UTC) Authentication-Results: imf05.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=tooXzw3h; spf=pass (imf05.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.46 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713740; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=RtBaqYvfVYWwhkk0mbirzaJ8NN/nkFZeQ7cueszAH5c=; b=4fJvONvAfAAWot/UM2dlLHJDWpAE40TPYI+FQlBJ40EkRafOVGU7RErg6AvQeOVD0FWovB hT/ut7bLmiB+wnXs6WddtnC8bY9jo87t/tGrvfGLHzdPOepU880+r9NNh2WAZ/vbH83Dgg agHAUAAY9ycoDv6NMDl/1lBzlVgRvuQ= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713740; a=rsa-sha256; cv=pass; b=zVIP1VNv3WUpBGykiDUxjHBdLL5amMKBMHthwgGo8YrN3bTp7vnxyXsz7XPE6VJKtegKbI rEXqekAX8tENvjaEzT0f7P7syEJJNRal8Dlk/QBbBBmBglMTT6xlhHwgwggtTiaMmHvJIS sBVaU1yjL4LGRPVcKAMrXrSAMq9zcfg= ARC-Authentication-Results: i=2; imf05.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=tooXzw3h; spf=pass (imf05.hostedemail.com: domain of apopple@nvidia.com designates 40.107.93.46 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=Hb16S1Ljf8yKD7N1SgidGVhIG26Om6Zym+Q+u/JgFWjY470kLxx7pVIxWRyOW2k2+b2Xu0lMSYQHjFn9ZxZXkIC7LMUV/jxeD5e0UgZv2qBWPdbJeeUI4+m/lf0bePU6ldeNwdJZa7HtpplZmKRRRfkDgEx/q99VNb4fPFcyLYoheBrr6SJN4t6wauXveJDFPliRikdRYCZcnvheMouQKPyi//Rno1iB1EoDO6dcSpXPvxuOj11EGy5PD3spKgxu5civDRxuiN85XJ4kYwJFDrvpFi9FqQiCQ3T6HC+koRhLIYbAXgxYpwmpIdZWkpChkIuU//cyeZOuhO2bwtWxHQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=RtBaqYvfVYWwhkk0mbirzaJ8NN/nkFZeQ7cueszAH5c=; b=OSNV8Z3C3Id/LhGTjG4FCNSzY2ZZnBYq7unyMrX3xzBrTQlfncjq2Z/3+bz0SKecPR31snvkimhgYGkq+Vsa2ODeLyZYK0RZIbigW8CnCEVwSFENfRw2vkJriLoeHIGlfRriXDzYZYIH1tFvipmhRPP3pWKs/8opEnnLKaXoDF2C430QCA02JC+e8Yp+C26k+TCjOdutj5Qibd9zXIieVWRdVjcbJyvGjmZ8Qz6+uI9mqiO5r1YhTPHwsBAnfLk+JdZoOJWxZIWdfKgBHMx4CmOeRM5atm4Ey27yD+4+c1RCQtBIOMilvrQZA3RxVWDOR1N1+Ac9FZxFKP1Dk6X9Sg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=RtBaqYvfVYWwhkk0mbirzaJ8NN/nkFZeQ7cueszAH5c=; b=tooXzw3hwUI0Fe1wdzvmo0mGUmKMSsS/GwxOeIVQpNcnG1aZL88GTNz4yduFJwpeo+/p/FoaE4QuepLYE5sONFtBQ1kWo8HqjrqkCqeudXQ20thRkeomXfR/JYLFn7s16zBXvtfCcFEaD+Zvn0yr7SHf4TlTZ9x9kBNJyrz+aI5Iy//0xhur7H9TJfnCBbjPX6NN4b3d8XJk6jDF7Ua/p9PCNRBf9DKB1j61oUWzL7nYM8w8WXEEUPNldOyXS2Z9OBRlq/p9qU4aDrfyn+ehZpZpr9OL5jZ0OTBXCjgCtD1304x2ka7EJ5TcGfE/bn7b/3SGXtGeGEvowDzgckVUHw== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:49 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:49 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev, Alexander Gordeev , Heiko Carstens , Vasily Gorbik , Christian Borntraeger , Sven Schnelle Subject: [PATCH v9 18/20] dcssblk: Mark DAX broken, remove FS_DAX_LIMITED support Date: Fri, 28 Feb 2025 14:31:13 +1100 Message-ID: <33eef2379c0d240f40cc15453fad2df1a4ae34c8.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5PR01CA0018.ausprd01.prod.outlook.com (2603:10c6:10:1f9::9) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: aa6f5491-eb67-4c2c-6a5c-08dd57a892f7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: h7ay5QSLmeI7YT+4ZV4qVX6heTeO+TEo/Z7BHuCs7C3NfLH6exwHV4wL6GjujLSUeNcleBppmzdNqtLEjpbvRcm3B0xhEifQvG03+4JbbX/YCpYuHHlNh721oG2YqddRPViR9Xu2+f1DMZCodN14GxPpp6+CsXcasOHhARzKKcvyi/pEV65FbRfwJgZDzjITY4JbWA0iYAwkVZQTo80c3gmdlga4lpiO6sgenusAHUkuvu8wwrCqiVALPvq7edEGoHAfdRXokbJWObsfoJqeauQrkPs5TRbftOb2IfBbyBTfytKkjpOQl1hvbQJp7fT8aA+BhNInDDA2svPL5gYDp4BQulm0QGaIZPcaY5M20TnipA76a6Ummpf4g52GqNr6md7YyN63r9ZiM2edxeM071Ti7niGS7aiA6RKWUsNySrzUajfJulVggvAZh+Gu25CSV2R5u2AYDtFH4aoRBpdfhDKtPao5BgeB/WTfv8BHL6y19XOqgp4C2xqIaArHy5mPlpxQHKHyEUY9BLkoVvwr8NmiDiVhS4XQ0JZFv0kCWrjluGCI6V5BEDJmRsoJb24uQZpclaTzP65MsysqDkTs/KtyZ1ApvPwmV2RwQNAld0VNhUHLcemlxwkZ9xlP/9gb9f4qRe4vQwAYe69XDejGyZZMbG1rDyOeUDJVE6nLaMMB8SuJSr8MPY99DhygQoSwkH9unj8wjrzRJJm02H87qpCIxWSaLwrWQJhTMRtBIg8t+5vP5kyeJJ15vh9lU0XtyWHO6smJ0ICD8DCd7wRQ4ZYWO1bH3s68QzB7bhVl3bCbWUAFxCZehgkjW+WKvJ2p+Fhp0H45VLVUkDqztPZ2buJ0Z4kwoyhcRXt46I7R1IlVNlrnSw+PupG5+ceuhXdmk4SCJKhyB/CMD9+1KbvtElcNaCjWaBBiX90o+knSq81o0/mnI5TPQugs9VjG+BF8luxoDbzOZxsWBh/zylOi5hLSzeVF/bNpTZJEcKfCGRft0s6T2qJvT15svl7eqNh1jAizWh8L8WPMY/mKtyslEYnYJMsOT6veZ6F+qiOn3BMt39vD+RceZhRndjyhXkJiqB7P4etDTEmhDUWewkOBk26LYNO/KbCQW5tuJFdbtn3hKtsckz+g5lPBKIPIZ8JmerelW5XShW9hz/YFEJwZsQGBV8lV9rmhJuj4wQ48fO5FaENVdOW2+6U7CbSzH0BpEZomNskYUoPQPmzL3ZrjP+XwE2ZV0dXhdOu/y/ZaKQhMszsJGGMdLj28+u/a0UbmwqCC3bJe2DLLdOPIcgM8kwiwmcRaFA4wy52zUB29hysA4/OotMV5L3QfsnzdBsF5bR+Nh8wDZqbfsxUAaDMLNe6BXETYoudwEUE4uAv8AqL0whYYW92sxGIEzyWuCsV X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: CND0fBQ8L/d1zldUve59EuaaNXPfI40hSQ3grFH4b4BrGcFsRR10nXninaBb4H9OLZ18748D4Uw+Y146nX6UmmbF9xxnYXp8t+3+IkV8UIm+iYgY/cU27rDAznXa6MHQwU/rdYg8j0bEA/MHf5NxkjJm0XoXOHjJBVECgVHgcj7S/gXHBp99vxDlZWjMFkGTYQLbm6c3b1Ug9ccnGaZ3/2AJwBrbvlRreVo4DUprMHdMeyKT/KOBj6wDSsXIUY6o8fDPTIHNaPIYzFNnDZ9W07KhEoc/9DGcN0BpFXMnzFbX6oCwrDuArV6a3kBdQaRWHo7P4uszLHA/uEnhKKATcVgnq6b0AZjon4qP7a/DR8TIVGuhWRaKMvFgx1YJ5o3BIQkZb8m3B8wXaP3jOmq4mdlNNJnpZnQ5ODwjyxV1CdRSToxmNLST/wpDBTMOXRG4p2AuV5EXZwyUVqxvnF3AwvvxB/278Ig1sbMl+EZml1LZxqVqn7WRbiRFjwTSHNmqoF8EibsXes6NEITaCz6/2b09G0KDhpvOW4T8Np8M9W9zTc2YsO9Q9zwNINbbcK62AP63h7xvERnPagyxtHsa5YbfCwinDwl56VW7dmfESruN2VW0T0QpVsl8UXFtGuVOIbOOISKDfTpWpjZ2vnB+ZatSL7dPU9gAwWlPBrrgl4r2Ogc5HL870Soeat5TqMQQSEEq7vx2vJ6/+XSRgcoWsOxo5HpwoEBzBBlrPVTCLlcOHV+H7KuOArtbNaaKfT3fFzRYVpqT5xAEWEescK6Lq1yjDT9DL5pr+QzTDE2Is/1b6JCGFK6EWoSc2vs34Ld+7TvPXE+kATp4w6r00LjuHSH6IC6qrYdKHl0+SkJIhRe31tv1rImzsz2ZKniaMpDMUTv1Kw6gLDnrqEroXCdl6GTYOjxzz5DEbpMIZ5d2dXEPB9W+GvXHRorz+OQsIuZgaWjrqV5WOCJ/t0hsM6YOYilr/6EreLabbvSK3Pz5Rl6z64sRDkYinw4O8vR3IkSBIooiRuTDLfaqXOJtWaogaCcX3u/B/cqjqh1QKDncPAvAkVUJoi5jtBezKPdr5VoEEJwm4XA2WYRRypatgJA5Ws3bhAklOxZSxdB5PwYUc9YiHGGz+zfLmpursspLCsTlLgnFulpJZsWXvSmKtXblwLsLNOsxduYnrsJX6xRRAC3DDkUCRC3VSnyEj9ADX+l4FGi+LIiQKFB4DKEhSERsoAS3BrnH3eoa4gl9CF/rjMIl3UUIJA+ZWHurf4qV/oTW9tMEcnBXjyZS/O5KiEyCSbMelDItjLeujCUxuFI9SeEGOeeJPh+paUb6u6/VqaBCdY58lnA5OK8Iyi1wsf0i7LpnknWOJROwMm4YcSHVqtpdsWWU/h1j5wzOiSmqVSzhV7MsyY2/o1dXbfmQXWHWl7K7SAl6UiyySAonsM60Me/m+nDlBaCElnYqcCbQy5gMFBBUaD8P01rSv0XD4NLKB9BC/yNhfxIUWxdxdsbW3TuyxOYhMBKrqddWpwLEed9kiSrhH1u+12FSfQW+7U4HwIhI/fSa/bzaak6rhcBTTeLxBSbcN57H+aB7YndAzOec X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: aa6f5491-eb67-4c2c-6a5c-08dd57a892f7 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:49.2190 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 1gu48D8Xkt1oQiym/ArIhc9IDia1HbiFv1k+f705iM4PQ8njlUzEM/378T4g7WNR16aKv2ZH/qcEONBHZlvFZQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Server: rspam02 X-Stat-Signature: 4z6ryxdwegs3y7w95wf6gfzt9ywiup4y X-Rspamd-Queue-Id: 86357100004 X-Rspam-User: X-HE-Tag: 1740713740-112668 X-HE-Meta: U2FsdGVkX1/Ql6TKM6unYoTlbM2uKghPH3lSy2vJs3Vl99GpF72PrrJTIHP5VUdn6uqOLHMrAVZWbm7AioM/UfNc2Hy3VzJ04qThC+LqJl7PIwlPrNYTFKu9ksUg9+oUon9yEfIOBN+8ehvCRhMHmHdyK/VG87C/mA966ntY6FKMCPskdjw0DA8E6DrnRTk9iHh6JBfS0kQMQgqaxOk92PEQZAcbAVxEcFKBG8JVrlfOusqnMI6VV5fAeuexaNjVPB52BB2i/YYUfc7/wxp9L5iUEQcDhhWPRKNwnXvHm2oLfk00m0A8I+Nd+gjmPtFQZjxLxf2KoAnvghQ83l68krQjoYO5rEmbi3b5r/XjPPptdVzijE44QtKhb90kFjhyG+5jkjeBjpXrm9hXPdKuL2YGMjyld4stP2prXbqH4uwsxkjLBMVjKp/wGzr3g/Qj/lvtosw/0Q5pPqognTBOjox4hwXwVrQo55sX9BfKdpEIZzBJNiMZSKMrv44nuW9J3Mx8vId36Gy2sbs3WL6XpQ1YQatqz5JvHeRrb9VpVpLpHrgth0wBAdVQJShe2OXAe7FdqXzQBAGI+J+0OzEeST/RFzDHL8UPy5mom9g5IgRSZKi0qMEGGDyOSdqhj9/Vhu3ym4TvOy4J2jqvU88DUxUK2YzYlbQyFSbj9ziZ33QmX3UNf6LzYVzp+RKNZMjOnrIehHiiOhWMy3L2SlfSojhgWPK7P+KKJ8KXr0QoaeJxcE3yMr20MyfD2QrWU88diE7MBCA7kfOOPIsFKtea+bZapkTtVvp5UACLGL0z2SRuszxf/o99j/xt+s/eLqrJ7wE55z6fK+wN0tX5qnby2FV5NYtbOnqIy6j30EEMfj+RScs/HoX5FPwlfz7u8iZmIvk+KXksK9M2ychmPCu5Nr789jQyUzkc1AJenefykb7UcBGDT4nFXLZO9CBGvz84LRhdJOLJcylLI0viyrU Dv1Ls3Em JiUzScef3LCalLvZHoTTPPhAgLlhRtQWjo3DUfXogmeGUVjt+yC//EQhIqehAbNPu8QVEDbU5jvHtErW67u2FTigxjSu7EbdI2ic5uBA7KAFPDAE8usmAQOgUchxTySPeliOVBn8BUPtlFSIOLQk+byuGiWtP8G7hkocm9Cb/HSV32Ht+wlkykqbvLOuvwWW4Qspt0wqHZHHkRp/gImw5dP6GfiITL+Qez8AOGIpbnMtb7csAd0VTAWjEuNZ2TOiGvz7Wi75Bq8HY0xNsAQQDxbQQz3IARhR8hFQtVpNZ/RX4J2b3BGuNM5mjglnQiLcfXD5V+ufV3l3oqGTASdQuad9JRIvA2CkiruMmd1NEb3oeZf+72e7FrfMYY3FDN11qfKYlgul/Ls/8Axbu+GxWA9Gxk2QijWIan/+0ivLZUmsNs2VCO7TfSa3D0S5FQ1bflxppOCGMCQhr/TMGJY+lWvVkPh+60TDFpu7TX4sF/vWcmgPZEC7IA7DEuiuvw+sDHE+ABaZBbrUuX/jVPogZYv/GuzVcqfrZI5SoisprEm4G5n4YlL+uKEl50PdUYeMi1trxdxwdBsdCy2xXhv6llzqWGiNlybGqpZxWOz4PzxBbFRI= X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: From: Dan Williams The dcssblk driver has long needed special case supoprt to enable limited dax operation, so called CONFIG_FS_DAX_LIMITED. This mode works around the incomplete support for ZONE_DEVICE on s390 by forgoing the ability of dax-mapped pages to support GUP. Now, pending cleanups to fsdax that fix its reference counting [1] depend on the ability of all dax drivers to supply ZONE_DEVICE pages. To allow that work to move forward, dax support needs to be paused for dcssblk until ZONE_DEVICE support arrives. That work has been known for a few years [2], and the removal of "pte_devmap" requirements [3] makes the conversion easier. For now, place the support behind CONFIG_BROKEN, and remove PFN_SPECIAL (dcssblk was the only user). Link: http://lore.kernel.org/cover.9f0e45d52f5cff58807831b6b867084d0b14b61c.1725941415.git-series.apopple@nvidia.com [1] Link: http://lore.kernel.org/20210820210318.187742e8@thinkpad/ [2] Link: http://lore.kernel.org/4511465a4f8429f45e2ac70d2e65dc5e1df1eb47.1725941415.git-series.apopple@nvidia.com [3] Reviewed-by: Gerald Schaefer Tested-by: Alexander Gordeev Acked-by: David Hildenbrand Cc: Heiko Carstens Cc: Vasily Gorbik Cc: Christian Borntraeger Cc: Sven Schnelle Cc: Jan Kara Cc: Matthew Wilcox Cc: Christoph Hellwig Cc: Alistair Popple Signed-off-by: Dan Williams --- Documentation/filesystems/dax.rst | 1 - drivers/s390/block/Kconfig | 12 ++++++++++-- drivers/s390/block/dcssblk.c | 27 +++++++++++++++++---------- 3 files changed, 27 insertions(+), 13 deletions(-) diff --git a/Documentation/filesystems/dax.rst b/Documentation/filesystems/dax.rst index 719e90f..08dd5e2 100644 --- a/Documentation/filesystems/dax.rst +++ b/Documentation/filesystems/dax.rst @@ -207,7 +207,6 @@ implement direct_access. These block devices may be used for inspiration: - brd: RAM backed block device driver -- dcssblk: s390 dcss block device driver - pmem: NVDIMM persistent memory driver diff --git a/drivers/s390/block/Kconfig b/drivers/s390/block/Kconfig index e3710a7..4bfe469 100644 --- a/drivers/s390/block/Kconfig +++ b/drivers/s390/block/Kconfig @@ -4,13 +4,21 @@ comment "S/390 block device drivers" config DCSSBLK def_tristate m - select FS_DAX_LIMITED - select DAX prompt "DCSSBLK support" depends on S390 && BLOCK help Support for dcss block device +config DCSSBLK_DAX + def_bool y + depends on DCSSBLK + # requires S390 ZONE_DEVICE support + depends on BROKEN + select DAX + prompt "DCSSBLK DAX support" + help + Enable DAX operation for the dcss block device + config DASD def_tristate y prompt "Support for DASD devices" diff --git a/drivers/s390/block/dcssblk.c b/drivers/s390/block/dcssblk.c index 0f14d27..7248e54 100644 --- a/drivers/s390/block/dcssblk.c +++ b/drivers/s390/block/dcssblk.c @@ -534,6 +534,21 @@ static const struct attribute_group *dcssblk_dev_attr_groups[] = { NULL, }; +static int dcssblk_setup_dax(struct dcssblk_dev_info *dev_info) +{ + struct dax_device *dax_dev; + + if (!IS_ENABLED(CONFIG_DCSSBLK_DAX)) + return 0; + + dax_dev = alloc_dax(dev_info, &dcssblk_dax_ops); + if (IS_ERR(dax_dev)) + return PTR_ERR(dax_dev); + set_dax_synchronous(dax_dev); + dev_info->dax_dev = dax_dev; + return dax_add_host(dev_info->dax_dev, dev_info->gd); +} + /* * device attribute for adding devices */ @@ -547,7 +562,6 @@ dcssblk_add_store(struct device *dev, struct device_attribute *attr, const char int rc, i, j, num_of_segments; struct dcssblk_dev_info *dev_info; struct segment_info *seg_info, *temp; - struct dax_device *dax_dev; char *local_buf; unsigned long seg_byte_size; @@ -674,14 +688,7 @@ dcssblk_add_store(struct device *dev, struct device_attribute *attr, const char if (rc) goto put_dev; - dax_dev = alloc_dax(dev_info, &dcssblk_dax_ops); - if (IS_ERR(dax_dev)) { - rc = PTR_ERR(dax_dev); - goto put_dev; - } - set_dax_synchronous(dax_dev); - dev_info->dax_dev = dax_dev; - rc = dax_add_host(dev_info->dax_dev, dev_info->gd); + rc = dcssblk_setup_dax(dev_info); if (rc) goto out_dax; @@ -917,7 +924,7 @@ __dcssblk_direct_access(struct dcssblk_dev_info *dev_info, pgoff_t pgoff, *kaddr = __va(dev_info->start + offset); if (pfn) *pfn = __pfn_to_pfn_t(PFN_DOWN(dev_info->start + offset), - PFN_DEV|PFN_SPECIAL); + PFN_DEV); return (dev_sz - offset) / PAGE_SIZE; } From patchwork Fri Feb 28 03:31:14 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995613 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 3B2CAC282C1 for ; Fri, 28 Feb 2025 03:37:12 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id CC246280011; Thu, 27 Feb 2025 22:37:11 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id C4B98280004; Thu, 27 Feb 2025 22:37:11 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id A7749280011; Thu, 27 Feb 2025 22:37:11 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 83795280004 for ; Thu, 27 Feb 2025 22:37:11 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay01.hostedemail.com (Postfix) with ESMTP id 2F4871C6F30 for ; Fri, 28 Feb 2025 03:37:11 +0000 (UTC) X-FDA: 83167942662.02.BCEC31F Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2076.outbound.protection.outlook.com [40.107.244.76]) by imf02.hostedemail.com (Postfix) with ESMTP id 4EFEB80015 for ; Fri, 28 Feb 2025 03:37:08 +0000 (UTC) Authentication-Results: imf02.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=G638jKfa; spf=pass (imf02.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.76 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713828; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=0/1fPSud41Om/5E3Ev1OVsqEH9uR8RyQzS/KEtTOXIM=; b=qC32YisWI0Z6MaMm+XTspaiBUpoAB8kS/OKUmbifkuQQkJb1P6N6p8wtbKt9kQoUhK7FBi Ig3UVV+ch6siQnuM2bTgQHCwfquBG9gA783Ubwx36NsMytggdeOdqN/UVqNJUHSuV1/yn5 R5dQBU/wvREIEOV7iXawj+hrPORQa3w= ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713828; a=rsa-sha256; cv=pass; b=galOricodvJOcJxKKjyI0qTTEJHP7R/hx0tUewCndtwr/G0GBQ4vWkXD+bDs7CkMiAU9aa tDNvPS5wxgScP9BCfQyC8vX2lXjck/ALVOMme6zyQVZYCq+5eVuPQ/QrZn1iP/CPclEFav GLzSXfizo/fZQP+zzRDtpJtHWh2F89g= ARC-Authentication-Results: i=2; imf02.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=G638jKfa; spf=pass (imf02.hostedemail.com: domain of apopple@nvidia.com designates 40.107.244.76 as permitted sender) smtp.mailfrom=apopple@nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); dmarc=pass (policy=reject) header.from=nvidia.com ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=BHVK1DfZZNClBquCoeWBxIMJxZ2Zjhlz/Kj3lhyVd30pwxqKdTlx8jGzuRfHemEPxui8qfOkLDjwu9YPLjVG1RwZ/3BMn345IoIlRse/E/wzYp+eR2nwqLDUXvyGX4yqHwmvSdcRqRRfhAkvx4RiWdiILGfTIztbijIPLgUr7Uh3S7Z1njzyoX0XoW+fV1TL37p5/NrgG1qgAmL0yfqLNxQG2Ue2LKKyOj8Z7gqE7vSxv6QQX5tgsEBkvj/ocHA+VFWS4NOoJXqULc9PP68B2rsVCgjrcsXl2EbjJH2uBqs634rjPXL6YxsbDGLSCOu2f6LJCsitZT6RcW4tfgDvXA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0/1fPSud41Om/5E3Ev1OVsqEH9uR8RyQzS/KEtTOXIM=; b=cBZhwUnSj3xnW4HJyyC66Uelb039i3S8HExBMRy/7qYfyibQAPgjA/wnFzcOUAfTlI7C4Qe5TZL+HPpv8jal+qrE4WbhxRVZUZXUKCHrVWVJfUFhMsXENTq7tCBe5v0bpLymMztuItqMZjgvtZ881Jp+Zemddl9AXJKaCioMDLiDgR/3I5NnOA5BplzS3E/ku57f81lfNNJWiMh2OBbNtzPsH55nqN9swXqLVhyD9sXmTUctcOpNnAzRH/cZXFiMwu38Xx4ySo8fFrvGXMQZ/aq1x+i+nlxh2+NHsTAgAsK/dlC2cL3tZy9zwuplnc1Imt26aylC/1GWSMj1TQK9Rw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0/1fPSud41Om/5E3Ev1OVsqEH9uR8RyQzS/KEtTOXIM=; b=G638jKfaVzspmFsmjZug0QSQ90YrTI1WNrS4UxF/SPztk1VvbOdBqkjGuzz6+X4emJZW9IqRFLO40B0T4RE4FX0GyNckSMlTXpEgFqIEoj40mGeZYB//VPuNx7Xt4Y0b6Um8mZlVjoWsIk8ZSv3Hw2v0QtVTGB9ugNQTAJlAxsJxI/iTF59lgX8sxeFrwA1zYv1iajpU9ij8hg70RqRzZnjxrs6hAqpJTkikE/RylHig5u5MrYBhr/F2fzS3pNoN/9czEtSbpNa/SWq3f7EsItIr5McT8U1AupXRc3LA/kqyXW2lsmlWLBEvyupa7oX//wpkXIqBjMAgEYfl/fMfDw== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:54 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:54 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 19/20] fs/dax: Properly refcount fs dax pages Date: Fri, 28 Feb 2025 14:31:14 +1100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5P282CA0039.AUSP282.PROD.OUTLOOK.COM (2603:10c6:10:206::17) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: 65f1a6ff-62da-4f6f-3b11-08dd57a89613 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: 2/TftW+Aq0dQFZ2mMPWLeSgyEEKwTJhXB76BQRm/zlyUcr4c3qYxTPEy1/E1pgGuMQrcbmj3lytGDrA4tjDMlRqYAMzFsLJkqHfyGlFSa5i6fSz5LeqlItLr539j1PZ7rhdFVW8TQeAQAIE8Dmva1lpTyibG6WyFrBcyr2WnQ5gzYYorOB9VL7jw3aptDl2/cJDqhn+2gh2En6lEBM4ZfLvr2+3UON/rBU9NdMjNI+Fq732hBdjW8iie1bCF2gWJxL4w+zX4MLSRzirH8xpeesqDkRAINeukfZhqw3RGLr/sKa/qBDW1znAV7F1yZTwvKhcQRKSN3P10uXqJSsbY/TIPQ0SwcXO4cvE6s8fboET5uID6jpDlPgFBJuhjwPLlN9oa5ixfCtGDcschwpGQA+mlb0+6f6c/ZeyvpBgfZ45ODpRSEEG6wzYHqYtx95lXwtoyUTMSUQQ2WxOraoPp0HmxeeHagdfFDvbZMC0/D8K3OUsu5p9uvaHrwNNFhnaAypAq7O0aWI7P7bHEaDgigO61ttAK64UhjLMIeuKz7NUuJnA40+1evr1qEMxIQDhYPjgsEmMhRyDsMr1QxI47TCRBugFw5nWw4myRCOQk9kz5MCxltWLxh2H7iZYCebEzyRu2Ms2FOQecUUf87CznrHcyajxC+mV49MAto+bgqr+U+MeS3pgv9/xOye8lH1Yw6vxcczonoabut3DBDDhrvUeIIsEqquqNPGECQY3TO4a/n9+ZzhjpW32voRi/MO/zMm6z8NoTqlQmuti6KqTWnWgVMPJOEPJzEpLEAwcMbRejkN15YqMdQ7yDjOSZlyJMkFQIKFE15u+aVXZwvMr8FgxiUoiYpqp8nLsLzCErr6cncp1hDwhw5MeIonPjcTxYltIA9QF7KBrIU8wVHzlvlJ2oLvF+ybI75KizhxUPxq5Sr+g14+F4hFRFMZkEutfvJj0IID3RtrAFnJONPND62Y98daw5UekEdib4vYt4u/SCqR3QOkLYm5m3hL9dcOGrmes1Q0foKafw9ofy+OPOVvM7zKJ3en24LTdGaR6BSgamdt8cepOL1cZWcXLCkiAjcw7fGzg6FScv18PWaCf9o7r1j8LL/iTgCTgiSueDF/PgUpl4H7MIsrsg0AJMJSJfEVyyJK+l+KHCGzipvbOaRvDWhUF9MoWn7Yln8K+bPlmxlKEJbaKdbce6bLJCkj8ZosrLfd2tAC8aYYYln+Bni5+4GlWeWbCysM66EEWUxfpxDi2Xdx8IxpFhiQnvvPKvZmFChkPSYRwFcD+u5N1WRODjaez/VU4pPM6eO+rLm26YCSOT3o7Y4aDTLks3oUKfsjem/wpf3X3Ee76eR6D9a/Kq054Jq3gRfBBnxCoUchuy3K6/hFzJpV1QX4upKJ+J X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: A0fUXzzws/mwpJyxaat+gbfOVfg7yg3q6Gw+9v2M9qA9YD4FL2xZWzAJ854X3RFVPXdO2jSNrpBwUYyscbqwWqiIMSiB+CQWd0qwQh964zlO5eM3auPtjwMxayLuBW2pUhEjmB4CXhZL0XlD12oISBJVjGaHRhZaYgq03w/CJcMj2SbNqlnhnpPWOBAIuIwnfLCz2Z8TheVWoEynuQBDuuiO4mF9Ac4uy+v/PchdcC4zi72ic3tBn6Pzo7LcVPd51Sc22pe/5T+S6ETO6XnSpanlqG1pPbwM8AxfapJTn4q9S/hZ1e0VSvwb91T6tzf1pj9i/OyHwP9seUKK/AEoBFuhRgeOgttGCd9s2IBwLwWwtKKX+LwyArvT3b02lCq5L4vV8m73aYRLNmE9OY3P/iXcAc9DCHSeWmXGLurQkZN1VhScOVnRpW9nRxW4kTn9Ra/YusPjvmX2PLKuNuOHVMdMsaJNCrof7t/11f8UfPNQZ9rEvLxVS94LpofJJZ/8fTKQGBlX6ZkvrnLhHKNFpleCO7vpJyk9lBD8exTmVmLT+Mk0U9Ip7bYLCb7yRFPVelRHUJSQNWPD43jngYKOKTsFBr6XSZkxudSlNbVyCuGjT2g0ImA3RWcHKuJDJG+iunhq/7mqA5KQs8EWXR9xT7fQ2MjsU1mtYdH7r4lrgxvwsJOrxxRCLyZk38zVd+xu6b+L5yv03HeqW3O9mJDKVYfWy24cNQWLCWACGkgnmd8/yeurQFFxF3Ahphj8mV8x3QVK3/pGBsbLHlIttxbyaQqNiexe5K2heO/2jqnYZbNStAYTqConda0by7PoZQHSBEDzE2fySrY4M9phzZsFUg8EXo9fFVuiZZEjdWQYcfrWHBSvRnWbjech9I28307t3bw7b13G6O7voPP6YtTF/11jLg3asXk9s/sHHgN+I/mPn1ual8fw6P6WiqkafS6AtPchvndf+otcwMKT/hzCMMfO0QVqCMlyBm/WAOdAQGUkg+PodnvQneMBDCAUQwiz2YyITrZOglLp7Dx9ZZ3HRTOh88FH6mJpBoSiqrdCjvboDi3tGzpwy0jWQqNsUACQtgLDMBKuLSEyHAjgZ4cbTXGJqYe7PTl4PyY/5OxZIWmG1osN/JTraoC0DwPffgZKjHbzdghYhI60j/E1/SIocqNEY2rDHktmadJdarsOc2w0PGbNwONAdZrIY2kJhkVKBZWeSh9Lwz6CLTjTc1Iz+TnAmpgEimvn7ADRmLPufoZ7DOrlZTcX5uMlizCpW1EcSSuMvg2oNqFx+2UGM+RHYfjvYhkSuJZhJG8gYnPqOsL+miY8p8cD7bZGQqYnosi6zYNhsWnUuxRm5VMgkVE27sgDNtWQaCnOR7WRFPy6tJt849lZnwo/6MU3SjiEBPRGMDSBQI/+a/4hxKOijk3ow5CjNLqkyFVoYGuvJkHjV42BOqJfdzafUNpBmgvEObFjAdCfRteQyI0qtf+8R9O6oIqmafzukaPCKWkZ+MCSVoyrCv6srdGXLfE7xMfDyh5Ka8wkg7RoJ3KLFZQFkATMC+6A+eiJoZRCtN5bcMGzR93yOok+yQ66u+ZI3jwt3vZQ X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 65f1a6ff-62da-4f6f-3b11-08dd57a89613 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:54.4562 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: acYHFRdKWefV9CizP1ZcU+7jVG4SDvlpDo3Y2sEj9OA5H/NpkT4PHvuteSYaezl+EPEkvl0glx+IQ27//kwtOg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Server: rspam02 X-Stat-Signature: rjg8xxynzhff5tkrezk9x63h1hnn9isq X-Rspamd-Queue-Id: 4EFEB80015 X-Rspam-User: X-HE-Tag: 1740713828-980472 X-HE-Meta: U2FsdGVkX1+dcI8tp5Pzfrxkzsos5fq4Dojx3o+Q1sVFyJrut/PCNS2XqNB2BKuV1b3vuZ/MfRIvEKWTXf92jl6njc8z8mxuL9IkyDQs2gf51hU2FRHDDvhqbAKfc4MWxqP0fzqXI80gVfPv1CtUYqM7QI8mISvXQ5isrMGsvTEPAhezMVCdrVSK34EizHdb5l9HAESpdxTBiLaUqadCgAUWqhSe1dSExo/pE2c2h3XXIclRXFqPaLB5ZWHSVbXB/gMI1pBxZMtxTBmGmDd1d2IM0OzEue5nvXsDTvvTyC2D6nd5WKJk+twp5JrSu2XQ1L4KJ67DkeDwpM5Hh+/nu+rNbrFoXAYqOoVqMYrPKynWa1ua/wqR6M/Z2Bt3MJNeNRxDgHcrGz0fdopteveHPq8InC/hVre+3x8coy/P2MyRS9RkwbDLBBb2qmlsKyOngf6p4FDwbAwecsgFtba+B+mbOtcrOl47JGhM65fsnusFiph14bOVrKCz9Sbd3sZi4XDIIcpeHlQfgIpYdUUND7glnPGrd93lloNvMxqX8hX0sm0eAaNbWcRTxlvXqgKVgOOt8uwG/uUTbs9j9q1hw61A7w0Z+HMClOEfpdXC++bv9DUyCFJhrDtmf38RkPzLCXmfMA9TTm78eCLALqMjuA+UHoHCkF8bQFbMXPKBP458JD5OTsk2v29I/6B3yKQdgNH7cejFEX8GOB2eOw+GsNL9VmMclqloJEOERjIia4URAzJY61AAtGPxuK6C7zZjZiii2e0nQ33op+q1KmTvjHu8tG5uzJiZhKcR0KNz6XKMYzsVpjpTwnbGgRXFdqsIU7ky/hEDdYockt/t8RT93nQAQmPc3SAusv/j/CuxJ+t3lvzU+p71df59Ov7ewU/lthPy0FRvJIat3ebNNablrCFn3k8zSqBY9rxqz/FOiKotryi7RwvVsPw57N8oFS3WJi3+0iPTCW82keSrJtg xLxrYkUY vT0t8yBKIpAgjqkpWAhf6xvpRDSWy6XGc9pm71YULjHTQ4QsLmfBppFd9H2DHjS202ICCtSDDMwXvC2BiY++wDXZWDQPoybXM5F1QRvCjJ/Vcm4Scxfax4dh3Cg8F1gg/NrbLlQnKhCz484r5LMZibiXPWKkGrcXB5SAnVlCsISgZkyj+jqttoeyWFefrKEeCUPRrbAn9ilxiqStsmZbGO+OXSoTnlq4R+F+laMsEAFKl5adVyzdDvzF+DxEhN97Z1B4AuSwGa/ogBCYZdUvvl7+1p9ipLAERUyq6VslYjbMxGhOMrKZquh0j5M17oFY9EhTqBfvMu8DCkDjHaAhNLVjEM/ywDRwr/OoMmetTto0/2fD08Nk6SXZuUTu+SGj20mKkex+iCF4n8brN+HW6Ab6rF9xHyGibehEAqFfhVL/C3jUZtiTRtVifxw== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Currently fs dax pages are considered free when the refcount drops to one and their refcounts are not increased when mapped via PTEs or decreased when unmapped. This requires special logic in mm paths to detect that these pages should not be properly refcounted, and to detect when the refcount drops to one instead of zero. On the other hand get_user_pages(), etc. will properly refcount fs dax pages by taking a reference and dropping it when the page is unpinned. Tracking this special behaviour requires extra PTE bits (eg. pte_devmap) and introduces rules that are potentially confusing and specific to FS DAX pages. To fix this, and to possibly allow removal of the special PTE bits in future, convert the fs dax page refcounts to be zero based and instead take a reference on the page each time it is mapped as is currently the case for normal pages. This may also allow a future clean-up to remove the pgmap refcounting that is currently done in mm/gup.c. Signed-off-by: Alistair Popple Reviewed-by: Dan Williams Acked-by: David Hildenbrand --- Changes for v8: - Rebased on mm-unstable - conflicts with Matthew's earlier changes. - Made dax_folio_put() easier to read thanks to David's suggestions. - Removed a useless WARN_ON_ONCE() Changes for v7: - s/dax_device_folio_init/dax_folio_init/ as suggested by Dan - s/dax_folio_share_put/dax_folio_put/ Changes since v2: Based on some questions from Dan I attempted to have the FS DAX page cache (ie. address space) hold a reference to the folio whilst it was mapped. However I came to the strong conclusion that this was not the right thing to do. If the page refcount == 0 it means the page is: 1. not mapped into user-space 2. not subject to other access via DMA/GUP/etc. Ie. From the core MM perspective the page is not in use. The fact a page may or may not be present in one or more address space mappings is irrelevant for core MM. It just means the page is still in use or valid from the file system perspective, and it's a responsiblity of the file system to remove these mappings if the pfn mapping becomes invalid (along with first making sure the MM state, ie. page->refcount, is idle). So we shouldn't be trying to track that lifetime with MM refcounts. Doing so just makes DMA-idle tracking more complex because there is now another thing (one or more address spaces) which can hold references on a page. And FS DAX can't even keep track of all the address spaces which might contain a reference to the page in the XFS/reflink case anyway. We could do this if we made file systems invalidate all address space mappings prior to calling dax_break_layouts(), but that isn't currently neccessary and would lead to increased faults just so we could do some superfluous refcounting which the file system already does. I have however put the page sharing checks and WARN_ON's back which also turned out to be useful for figuring out when to re-initialising a folio. --- drivers/nvdimm/pmem.c | 4 +- fs/dax.c | 186 ++++++++++++++++++++++++---------------- fs/fuse/virtio_fs.c | 3 +- include/linux/dax.h | 2 +- include/linux/mm.h | 27 +------ include/linux/mm_types.h | 7 +- mm/gup.c | 9 +-- mm/huge_memory.c | 6 +- mm/internal.h | 2 +- mm/memory-failure.c | 6 +- mm/memory.c | 6 +- mm/memremap.c | 47 ++++------ mm/mm_init.c | 9 +-- mm/swap.c | 2 +- 14 files changed, 165 insertions(+), 151 deletions(-) diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c index d81faa9..785b2d2 100644 --- a/drivers/nvdimm/pmem.c +++ b/drivers/nvdimm/pmem.c @@ -513,7 +513,7 @@ static int pmem_attach_disk(struct device *dev, pmem->disk = disk; pmem->pgmap.owner = pmem; - pmem->pfn_flags = PFN_DEV; + pmem->pfn_flags = 0; if (is_nd_pfn(dev)) { pmem->pgmap.type = MEMORY_DEVICE_FS_DAX; pmem->pgmap.ops = &fsdax_pagemap_ops; @@ -522,7 +522,6 @@ static int pmem_attach_disk(struct device *dev, pmem->data_offset = le64_to_cpu(pfn_sb->dataoff); pmem->pfn_pad = resource_size(res) - range_len(&pmem->pgmap.range); - pmem->pfn_flags |= PFN_MAP; bb_range = pmem->pgmap.range; bb_range.start += pmem->data_offset; } else if (pmem_should_map_pages(dev)) { @@ -532,7 +531,6 @@ static int pmem_attach_disk(struct device *dev, pmem->pgmap.type = MEMORY_DEVICE_FS_DAX; pmem->pgmap.ops = &fsdax_pagemap_ops; addr = devm_memremap_pages(dev, &pmem->pgmap); - pmem->pfn_flags |= PFN_MAP; bb_range = pmem->pgmap.range; } else { addr = devm_memremap(dev, pmem->phys_addr, diff --git a/fs/dax.c b/fs/dax.c index 6674540..cf96f3d 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -71,6 +71,11 @@ static unsigned long dax_to_pfn(void *entry) return xa_to_value(entry) >> DAX_SHIFT; } +static struct folio *dax_to_folio(void *entry) +{ + return page_folio(pfn_to_page(dax_to_pfn(entry))); +} + static void *dax_make_entry(pfn_t pfn, unsigned long flags) { return xa_mk_value(flags | (pfn_t_to_pfn(pfn) << DAX_SHIFT)); @@ -338,19 +343,6 @@ static unsigned long dax_entry_size(void *entry) return PAGE_SIZE; } -static unsigned long dax_end_pfn(void *entry) -{ - return dax_to_pfn(entry) + dax_entry_size(entry) / PAGE_SIZE; -} - -/* - * Iterate through all mapped pfns represented by an entry, i.e. skip - * 'empty' and 'zero' entries. - */ -#define for_each_mapped_pfn(entry, pfn) \ - for (pfn = dax_to_pfn(entry); \ - pfn < dax_end_pfn(entry); pfn++) - /* * A DAX folio is considered shared if it has no mapping set and ->share (which * shares the ->index field) is non-zero. Note this may return false even if the @@ -359,7 +351,7 @@ static unsigned long dax_end_pfn(void *entry) */ static inline bool dax_folio_is_shared(struct folio *folio) { - return !folio->mapping && folio->page.share; + return !folio->mapping && folio->share; } /* @@ -384,75 +376,117 @@ static void dax_folio_make_shared(struct folio *folio) * folio has previously been mapped into one address space so set the * share count. */ - folio->page.share = 1; + folio->share = 1; } -static inline unsigned long dax_folio_share_put(struct folio *folio) +static inline unsigned long dax_folio_put(struct folio *folio) { - return --folio->page.share; + unsigned long ref; + int order, i; + + if (!dax_folio_is_shared(folio)) + ref = 0; + else + ref = --folio->share; + + if (ref) + return ref; + + folio->mapping = NULL; + order = folio_order(folio); + if (!order) + return 0; + + for (i = 0; i < (1UL << order); i++) { + struct dev_pagemap *pgmap = page_pgmap(&folio->page); + struct page *page = folio_page(folio, i); + struct folio *new_folio = (struct folio *)page; + + ClearPageHead(page); + clear_compound_head(page); + + new_folio->mapping = NULL; + /* + * Reset pgmap which was over-written by + * prep_compound_page(). + */ + new_folio->pgmap = pgmap; + new_folio->share = 0; + WARN_ON_ONCE(folio_ref_count(new_folio)); + } + + return ref; +} + +static void dax_folio_init(void *entry) +{ + struct folio *folio = dax_to_folio(entry); + int order = dax_entry_order(entry); + + /* + * Folio should have been split back to order-0 pages in + * dax_folio_put() when they were removed from their + * final mapping. + */ + WARN_ON_ONCE(folio_order(folio)); + + if (order > 0) { + prep_compound_page(&folio->page, order); + if (order > 1) + INIT_LIST_HEAD(&folio->_deferred_list); + WARN_ON_ONCE(folio_ref_count(folio)); + } } static void dax_associate_entry(void *entry, struct address_space *mapping, - struct vm_area_struct *vma, unsigned long address, bool shared) + struct vm_area_struct *vma, + unsigned long address, bool shared) { - unsigned long size = dax_entry_size(entry), pfn, index; - int i = 0; + unsigned long size = dax_entry_size(entry), index; + struct folio *folio = dax_to_folio(entry); if (IS_ENABLED(CONFIG_FS_DAX_LIMITED)) return; index = linear_page_index(vma, address & ~(size - 1)); - for_each_mapped_pfn(entry, pfn) { - struct folio *folio = pfn_folio(pfn); - - if (shared && (folio->mapping || folio->page.share)) { - if (folio->mapping) - dax_folio_make_shared(folio); + if (shared && (folio->mapping || dax_folio_is_shared(folio))) { + if (folio->mapping) + dax_folio_make_shared(folio); - WARN_ON_ONCE(!folio->page.share); - folio->page.share++; - } else { - WARN_ON_ONCE(folio->mapping); - folio->mapping = mapping; - folio->index = index + i++; - } + WARN_ON_ONCE(!folio->share); + WARN_ON_ONCE(dax_entry_order(entry) != folio_order(folio)); + folio->share++; + } else { + WARN_ON_ONCE(folio->mapping); + dax_folio_init(entry); + folio = dax_to_folio(entry); + folio->mapping = mapping; + folio->index = index; } } static void dax_disassociate_entry(void *entry, struct address_space *mapping, - bool trunc) + bool trunc) { - unsigned long pfn; + struct folio *folio = dax_to_folio(entry); if (IS_ENABLED(CONFIG_FS_DAX_LIMITED)) return; - for_each_mapped_pfn(entry, pfn) { - struct folio *folio = pfn_folio(pfn); - - WARN_ON_ONCE(trunc && folio_ref_count(folio) > 1); - if (dax_folio_is_shared(folio)) { - /* keep the shared flag if this page is still shared */ - if (dax_folio_share_put(folio) > 0) - continue; - } else - WARN_ON_ONCE(folio->mapping && folio->mapping != mapping); - folio->mapping = NULL; - folio->index = 0; - } + dax_folio_put(folio); } static struct page *dax_busy_page(void *entry) { - unsigned long pfn; + struct folio *folio = dax_to_folio(entry); - for_each_mapped_pfn(entry, pfn) { - struct page *page = pfn_to_page(pfn); + if (dax_is_zero_entry(entry) || dax_is_empty_entry(entry)) + return NULL; - if (page_ref_count(page) > 1) - return page; - } - return NULL; + if (folio_ref_count(folio) - folio_mapcount(folio)) + return &folio->page; + else + return NULL; } /** @@ -785,7 +819,7 @@ struct page *dax_layout_busy_page(struct address_space *mapping) EXPORT_SYMBOL_GPL(dax_layout_busy_page); static int __dax_invalidate_entry(struct address_space *mapping, - pgoff_t index, bool trunc) + pgoff_t index, bool trunc) { XA_STATE(xas, &mapping->i_pages, index); int ret = 0; @@ -953,7 +987,8 @@ void dax_break_layout_final(struct inode *inode) wait_page_idle_uninterruptible(page, inode); } while (true); - dax_delete_mapping_range(inode->i_mapping, 0, LLONG_MAX); + if (!page) + dax_delete_mapping_range(inode->i_mapping, 0, LLONG_MAX); } EXPORT_SYMBOL_GPL(dax_break_layout_final); @@ -1039,8 +1074,10 @@ static void *dax_insert_entry(struct xa_state *xas, struct vm_fault *vmf, void *old; dax_disassociate_entry(entry, mapping, false); - dax_associate_entry(new_entry, mapping, vmf->vma, vmf->address, - shared); + if (!(flags & DAX_ZERO_PAGE)) + dax_associate_entry(new_entry, mapping, vmf->vma, + vmf->address, shared); + /* * Only swap our new entry into the page cache if the current * entry is a zero page or an empty entry. If a normal PTE or @@ -1228,9 +1265,7 @@ static int dax_iomap_direct_access(const struct iomap *iomap, loff_t pos, goto out; if (pfn_t_to_pfn(*pfnp) & (PHYS_PFN(size)-1)) goto out; - /* For larger pages we need devmap */ - if (length > 1 && !pfn_t_devmap(*pfnp)) - goto out; + rc = 0; out_check_addr: @@ -1337,7 +1372,7 @@ static vm_fault_t dax_load_hole(struct xa_state *xas, struct vm_fault *vmf, *entry = dax_insert_entry(xas, vmf, iter, *entry, pfn, DAX_ZERO_PAGE); - ret = vmf_insert_mixed(vmf->vma, vaddr, pfn); + ret = vmf_insert_page_mkwrite(vmf, pfn_t_to_page(pfn), false); trace_dax_load_hole(inode, vmf, ret); return ret; } @@ -1808,7 +1843,8 @@ static vm_fault_t dax_fault_iter(struct vm_fault *vmf, loff_t pos = (loff_t)xas->xa_index << PAGE_SHIFT; bool write = iter->flags & IOMAP_WRITE; unsigned long entry_flags = pmd ? DAX_PMD : 0; - int err = 0; + struct folio *folio; + int ret, err = 0; pfn_t pfn; void *kaddr; @@ -1840,17 +1876,19 @@ static vm_fault_t dax_fault_iter(struct vm_fault *vmf, return dax_fault_return(err); } + folio = dax_to_folio(*entry); if (dax_fault_is_synchronous(iter, vmf->vma)) return dax_fault_synchronous_pfnp(pfnp, pfn); - /* insert PMD pfn */ + folio_ref_inc(folio); if (pmd) - return vmf_insert_pfn_pmd(vmf, pfn, write); + ret = vmf_insert_folio_pmd(vmf, pfn_folio(pfn_t_to_pfn(pfn)), + write); + else + ret = vmf_insert_page_mkwrite(vmf, pfn_t_to_page(pfn), write); + folio_put(folio); - /* insert PTE pfn */ - if (write) - return vmf_insert_mixed_mkwrite(vmf->vma, vmf->address, pfn); - return vmf_insert_mixed(vmf->vma, vmf->address, pfn); + return ret; } static vm_fault_t dax_iomap_pte_fault(struct vm_fault *vmf, pfn_t *pfnp, @@ -2089,6 +2127,7 @@ dax_insert_pfn_mkwrite(struct vm_fault *vmf, pfn_t pfn, unsigned int order) { struct address_space *mapping = vmf->vma->vm_file->f_mapping; XA_STATE_ORDER(xas, &mapping->i_pages, vmf->pgoff, order); + struct folio *folio; void *entry; vm_fault_t ret; @@ -2106,14 +2145,17 @@ dax_insert_pfn_mkwrite(struct vm_fault *vmf, pfn_t pfn, unsigned int order) xas_set_mark(&xas, PAGECACHE_TAG_DIRTY); dax_lock_entry(&xas, entry); xas_unlock_irq(&xas); + folio = pfn_folio(pfn_t_to_pfn(pfn)); + folio_ref_inc(folio); if (order == 0) - ret = vmf_insert_mixed_mkwrite(vmf->vma, vmf->address, pfn); + ret = vmf_insert_page_mkwrite(vmf, &folio->page, true); #ifdef CONFIG_FS_DAX_PMD else if (order == PMD_ORDER) - ret = vmf_insert_pfn_pmd(vmf, pfn, FAULT_FLAG_WRITE); + ret = vmf_insert_folio_pmd(vmf, folio, FAULT_FLAG_WRITE); #endif else ret = VM_FAULT_FALLBACK; + folio_put(folio); dax_unlock_entry(&xas, entry); trace_dax_insert_pfn_mkwrite(mapping->host, vmf, ret); return ret; diff --git a/fs/fuse/virtio_fs.c b/fs/fuse/virtio_fs.c index 82afe78..2c7b24c 100644 --- a/fs/fuse/virtio_fs.c +++ b/fs/fuse/virtio_fs.c @@ -1017,8 +1017,7 @@ static long virtio_fs_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, if (kaddr) *kaddr = fs->window_kaddr + offset; if (pfn) - *pfn = phys_to_pfn_t(fs->window_phys_addr + offset, - PFN_DEV | PFN_MAP); + *pfn = phys_to_pfn_t(fs->window_phys_addr + offset, 0); return nr_pages > max_nr_pages ? max_nr_pages : nr_pages; } diff --git a/include/linux/dax.h b/include/linux/dax.h index 2333c30..dcc9fcd 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -209,7 +209,7 @@ int dax_truncate_page(struct inode *inode, loff_t pos, bool *did_zero, static inline bool dax_page_is_idle(struct page *page) { - return page && page_ref_count(page) == 1; + return page && page_ref_count(page) == 0; } #if IS_ENABLED(CONFIG_DAX) diff --git a/include/linux/mm.h b/include/linux/mm.h index 066aebd..7b21b48 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1192,6 +1192,8 @@ int vma_is_stack_for_current(struct vm_area_struct *vma); struct mmu_gather; struct inode; +extern void prep_compound_page(struct page *page, unsigned int order); + /* * compound_order() can be called without holding a reference, which means * that niceties like page_folio() don't work. These callers should be @@ -1513,25 +1515,6 @@ vm_fault_t finish_fault(struct vm_fault *vmf); * back into memory. */ -#if defined(CONFIG_ZONE_DEVICE) && defined(CONFIG_FS_DAX) -DECLARE_STATIC_KEY_FALSE(devmap_managed_key); - -bool __put_devmap_managed_folio_refs(struct folio *folio, int refs); -static inline bool put_devmap_managed_folio_refs(struct folio *folio, int refs) -{ - if (!static_branch_unlikely(&devmap_managed_key)) - return false; - if (!folio_is_zone_device(folio)) - return false; - return __put_devmap_managed_folio_refs(folio, refs); -} -#else /* CONFIG_ZONE_DEVICE && CONFIG_FS_DAX */ -static inline bool put_devmap_managed_folio_refs(struct folio *folio, int refs) -{ - return false; -} -#endif /* CONFIG_ZONE_DEVICE && CONFIG_FS_DAX */ - /* 127: arbitrary random number, small enough to assemble well */ #define folio_ref_zero_or_close_to_overflow(folio) \ ((unsigned int) folio_ref_count(folio) + 127u <= 127u) @@ -1646,12 +1629,6 @@ static inline void put_page(struct page *page) { struct folio *folio = page_folio(page); - /* - * For some devmap managed pages we need to catch refcount transition - * from 2 to 1: - */ - if (put_devmap_managed_folio_refs(folio, 1)) - return; folio_put(folio); } diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 6f2d6bb..689b2a7 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -296,6 +296,8 @@ typedef struct { * anonymous memory. * @index: Offset within the file, in units of pages. For anonymous memory, * this is the index from the beginning of the mmap. + * @share: number of DAX mappings that reference this folio. See + * dax_associate_entry. * @private: Filesystem per-folio data (see folio_attach_private()). * @swap: Used for swp_entry_t if folio_test_swapcache(). * @_mapcount: Do not access this member directly. Use folio_mapcount() to @@ -345,7 +347,10 @@ struct folio { struct dev_pagemap *pgmap; }; struct address_space *mapping; - pgoff_t index; + union { + pgoff_t index; + unsigned long share; + }; union { void *private; swp_entry_t swap; diff --git a/mm/gup.c b/mm/gup.c index e5d6454..e504065 100644 --- a/mm/gup.c +++ b/mm/gup.c @@ -96,8 +96,7 @@ static inline struct folio *try_get_folio(struct page *page, int refs) * belongs to this folio. */ if (unlikely(page_folio(page) != folio)) { - if (!put_devmap_managed_folio_refs(folio, refs)) - folio_put_refs(folio, refs); + folio_put_refs(folio, refs); goto retry; } @@ -116,8 +115,7 @@ static void gup_put_folio(struct folio *folio, int refs, unsigned int flags) refs *= GUP_PIN_COUNTING_BIAS; } - if (!put_devmap_managed_folio_refs(folio, refs)) - folio_put_refs(folio, refs); + folio_put_refs(folio, refs); } /** @@ -565,8 +563,7 @@ static struct folio *try_grab_folio_fast(struct page *page, int refs, */ if (unlikely((flags & FOLL_LONGTERM) && !folio_is_longterm_pinnable(folio))) { - if (!put_devmap_managed_folio_refs(folio, refs)) - folio_put_refs(folio, refs); + folio_put_refs(folio, refs); return NULL; } diff --git a/mm/huge_memory.c b/mm/huge_memory.c index d189826..1a0d6a8 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -2225,7 +2225,7 @@ int zap_huge_pmd(struct mmu_gather *tlb, struct vm_area_struct *vma, tlb->fullmm); arch_check_zapped_pmd(vma, orig_pmd); tlb_remove_pmd_tlb_entry(tlb, pmd, addr); - if (vma_is_special_huge(vma)) { + if (!vma_is_dax(vma) && vma_is_special_huge(vma)) { if (arch_needs_pgtable_deposit()) zap_deposited_table(tlb->mm, pmd); spin_unlock(ptl); @@ -2882,13 +2882,15 @@ static void __split_huge_pmd_locked(struct vm_area_struct *vma, pmd_t *pmd, */ if (arch_needs_pgtable_deposit()) zap_deposited_table(mm, pmd); - if (vma_is_special_huge(vma)) + if (!vma_is_dax(vma) && vma_is_special_huge(vma)) return; if (unlikely(is_pmd_migration_entry(old_pmd))) { swp_entry_t entry; entry = pmd_to_swp_entry(old_pmd); folio = pfn_swap_entry_folio(entry); + } else if (is_huge_zero_pmd(old_pmd)) { + return; } else { page = pmd_page(old_pmd); folio = page_folio(page); diff --git a/mm/internal.h b/mm/internal.h index 109ef30..db5974b 100644 --- a/mm/internal.h +++ b/mm/internal.h @@ -735,8 +735,6 @@ static inline void prep_compound_tail(struct page *head, int tail_idx) set_page_private(p, 0); } -extern void prep_compound_page(struct page *page, unsigned int order); - void post_alloc_hook(struct page *page, unsigned int order, gfp_t gfp_flags); extern bool free_pages_prepare(struct page *page, unsigned int order); diff --git a/mm/memory-failure.c b/mm/memory-failure.c index 995a15e..8ba3d1d 100644 --- a/mm/memory-failure.c +++ b/mm/memory-failure.c @@ -419,18 +419,18 @@ static unsigned long dev_pagemap_mapping_shift(struct vm_area_struct *vma, pud = pud_offset(p4d, address); if (!pud_present(*pud)) return 0; - if (pud_devmap(*pud)) + if (pud_trans_huge(*pud)) return PUD_SHIFT; pmd = pmd_offset(pud, address); if (!pmd_present(*pmd)) return 0; - if (pmd_devmap(*pmd)) + if (pmd_trans_huge(*pmd)) return PMD_SHIFT; pte = pte_offset_map(pmd, address); if (!pte) return 0; ptent = ptep_get(pte); - if (pte_present(ptent) && pte_devmap(ptent)) + if (pte_present(ptent)) ret = PAGE_SHIFT; pte_unmap(pte); return ret; diff --git a/mm/memory.c b/mm/memory.c index a978b77..1e4424a 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -3827,13 +3827,15 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf) if (vma->vm_flags & (VM_SHARED | VM_MAYSHARE)) { /* * VM_MIXEDMAP !pfn_valid() case, or VM_SOFTDIRTY clear on a - * VM_PFNMAP VMA. + * VM_PFNMAP VMA. FS DAX also wants ops->pfn_mkwrite called. * * We should not cow pages in a shared writeable mapping. * Just mark the pages writable and/or call ops->pfn_mkwrite. */ - if (!vmf->page) + if (!vmf->page || is_fsdax_page(vmf->page)) { + vmf->page = NULL; return wp_pfn_shared(vmf); + } return wp_page_shared(vmf, folio); } diff --git a/mm/memremap.c b/mm/memremap.c index 68099af..9a8879b 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -458,8 +458,13 @@ EXPORT_SYMBOL_GPL(get_dev_pagemap); void free_zone_device_folio(struct folio *folio) { - if (WARN_ON_ONCE(!folio->pgmap->ops || - !folio->pgmap->ops->page_free)) + struct dev_pagemap *pgmap = folio->pgmap; + + if (WARN_ON_ONCE(!pgmap->ops)) + return; + + if (WARN_ON_ONCE(pgmap->type != MEMORY_DEVICE_FS_DAX && + !pgmap->ops->page_free)) return; mem_cgroup_uncharge(folio); @@ -484,26 +489,36 @@ void free_zone_device_folio(struct folio *folio) * For other types of ZONE_DEVICE pages, migration is either * handled differently or not done at all, so there is no need * to clear folio->mapping. + * + * FS DAX pages clear the mapping when the folio->share count hits + * zero which indicating the page has been removed from the file + * system mapping. */ - folio->mapping = NULL; - folio->pgmap->ops->page_free(folio_page(folio, 0)); + if (pgmap->type != MEMORY_DEVICE_FS_DAX) + folio->mapping = NULL; - switch (folio->pgmap->type) { + switch (pgmap->type) { case MEMORY_DEVICE_PRIVATE: case MEMORY_DEVICE_COHERENT: - put_dev_pagemap(folio->pgmap); + pgmap->ops->page_free(folio_page(folio, 0)); + put_dev_pagemap(pgmap); break; - case MEMORY_DEVICE_FS_DAX: case MEMORY_DEVICE_GENERIC: /* * Reset the refcount to 1 to prepare for handing out the page * again. */ + pgmap->ops->page_free(folio_page(folio, 0)); folio_set_count(folio, 1); break; + case MEMORY_DEVICE_FS_DAX: + wake_up_var(&folio->page); + break; + case MEMORY_DEVICE_PCI_P2PDMA: + pgmap->ops->page_free(folio_page(folio, 0)); break; } } @@ -519,21 +534,3 @@ void zone_device_page_init(struct page *page) lock_page(page); } EXPORT_SYMBOL_GPL(zone_device_page_init); - -#ifdef CONFIG_FS_DAX -bool __put_devmap_managed_folio_refs(struct folio *folio, int refs) -{ - if (folio->pgmap->type != MEMORY_DEVICE_FS_DAX) - return false; - - /* - * fsdax page refcounts are 1-based, rather than 0-based: if - * refcount is 1, then the page is free and the refcount is - * stable because nobody holds a reference on the page. - */ - if (folio_ref_sub_return(folio, refs) == 1) - wake_up_var(&folio->_refcount); - return true; -} -EXPORT_SYMBOL(__put_devmap_managed_folio_refs); -#endif /* CONFIG_FS_DAX */ diff --git a/mm/mm_init.c b/mm/mm_init.c index d0b5bef..5793368 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1017,23 +1017,22 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, } /* - * ZONE_DEVICE pages other than MEMORY_TYPE_GENERIC and - * MEMORY_TYPE_FS_DAX pages are released directly to the driver page - * allocator which will set the page count to 1 when allocating the - * page. + * ZONE_DEVICE pages other than MEMORY_TYPE_GENERIC are released + * directly to the driver page allocator which will set the page count + * to 1 when allocating the page. * * MEMORY_TYPE_GENERIC and MEMORY_TYPE_FS_DAX pages automatically have * their refcount reset to one whenever they are freed (ie. after * their refcount drops to 0). */ switch (pgmap->type) { + case MEMORY_DEVICE_FS_DAX: case MEMORY_DEVICE_PRIVATE: case MEMORY_DEVICE_COHERENT: case MEMORY_DEVICE_PCI_P2PDMA: set_page_count(page, 0); break; - case MEMORY_DEVICE_FS_DAX: case MEMORY_DEVICE_GENERIC: break; } diff --git a/mm/swap.c b/mm/swap.c index fc8281e..7523b65 100644 --- a/mm/swap.c +++ b/mm/swap.c @@ -956,8 +956,6 @@ void folios_put_refs(struct folio_batch *folios, unsigned int *refs) unlock_page_lruvec_irqrestore(lruvec, flags); lruvec = NULL; } - if (put_devmap_managed_folio_refs(folio, nr_refs)) - continue; if (folio_ref_sub_and_test(folio, nr_refs)) free_zone_device_folio(folio); continue; From patchwork Fri Feb 28 03:31:15 2025 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13995541 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id DD62FC19F32 for ; Fri, 28 Feb 2025 03:34:02 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 6CCAB28000B; Thu, 27 Feb 2025 22:34:02 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 654D2280004; Thu, 27 Feb 2025 22:34:02 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 459D728000B; Thu, 27 Feb 2025 22:34:02 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0017.hostedemail.com [216.40.44.17]) by kanga.kvack.org (Postfix) with ESMTP id 1F6BB280004 for ; Thu, 27 Feb 2025 22:34:02 -0500 (EST) Received: from smtpin02.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id C459EA399A for ; Fri, 28 Feb 2025 03:34:01 +0000 (UTC) X-FDA: 83167934682.02.5127084 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2083.outbound.protection.outlook.com [40.107.92.83]) by imf28.hostedemail.com (Postfix) with ESMTP id 0F611C0005 for ; Fri, 28 Feb 2025 03:33:58 +0000 (UTC) Authentication-Results: imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="A/IGqJAH"; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf28.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.83 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1740713639; a=rsa-sha256; cv=pass; b=d97QM/d6kE7VvEjpNQrO9C/PrY6I7Az69JbbPXxV29YfJ5xD5KkEKSJOdKxm/h/AnLtcRR MDpLbuHY7sVAPUjNBD3wA/SYHYbMjqZ9uqd0FgH0tKoOW3APPmoX9673MxQnu/V8hP5yz7 VD9c7o6KxjzEjvR446Qrf9P/x6EMCRA= ARC-Authentication-Results: i=2; imf28.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b="A/IGqJAH"; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf28.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.83 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1740713639; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=+u0YnLmYHc+DH3jiR+pkv/UEz5kFOMI/DlwLxrVnEzo=; b=BnZqXfjGQbz+7U43vAQ3L7v+6iGgGGKIf9Qh/xGmTbLEHwmZ9FWVvHZrhGfULO0vuTw69M pd1TN7aXNy0jPzOJr+C1WWpylC7496x0GoqLruguU6vE+L4rkaTx+B7Tt1q6sPFoqebC0w A79Rqme9puX44tvewspnosILoQyFlSM= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=eb3EThmRMp1UMZA0CDkO57LwUJPql6qb9GbDUQ2nPRrmcPQp9B2N1VuOq/F6puakwSLiV5F3+O2uH5Xw+0OPo8tf2bfzLT4y66y15tI7lwKrPUHNBxCYZawH/vcgqNH72SrXCixS4ihgRK6aMS0aXk/I1rv1q3SiTsDCmUMA2o6cpVTh6UCOmRNqDSsAtEpuJugiEmqRsbCkwzgsWye+UWt0tVy2KHtD0sN2yG5gcbx4Ou9HrwHM1anevltPzfdlD/1nBbaT7IEGq8d6IF2pktplOSz8Y5ZSaSoCQuQ8i25j9gvxf8pzmH5Xcc8Gzfnb89bK6OkqdopEcwFwy0x7fA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+u0YnLmYHc+DH3jiR+pkv/UEz5kFOMI/DlwLxrVnEzo=; b=JGbT6pT3zQ1E/MqGxVAPvQCIYJYTbgWeas9/KedR7bX0wkWkfT9x8c7Cmn0n9hiJ66ATO3uwo2GPwBuMGKJ51CWnIKTckmCTlKA/OJFOlhngIjrDfkv0U2jB1Z4TbNNtmjYLtLG7GdHstNKWvJ9lXj5Ux9OHLQcY56QzZSAiWwYquKSKJ9liO5uUJTNAo7sL8uDIhxzYeZ1KNfVhDHt79/inlnaj0RqNETuMqOsWiC/umnfrP2GbE1oZXPW0Axutpgoew+n4rZ+ZiIw6M+dI4M2ZJatFjDUAaWORfmq7JCZ7+Osn4ff9rw9icPHF9W6t+5DTaU2VyfrgmMgPPsVlMg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+u0YnLmYHc+DH3jiR+pkv/UEz5kFOMI/DlwLxrVnEzo=; b=A/IGqJAH/XLQCjKjZ3u8DoDW/As78o4LyTwHd0ubgio5fD35xQLwi83gms+1aAWmCyKlC/AsiTyx4lUHf0jqXwOU5BlhiI2JG9d6eFxPz0xch9WylNXJh6Ig4GLmX3HGB6MwKSA++Z4CP19++wuXwVasm/Rh27qJRB3T03w6o5i6HgI61DsVSLOFDkuXkaP5axgiT+ikMSQZniNmTMBICa/K99/an759KYMu7DcvR9vA927D8FQLG8C7KuD6o0UYD47K5gUcN5TXk5MYBEmGE4gxgRI69kXUMLojrVZpFigZCuZ4SpWBlWlEJkN+HfIUwgoPpWUXf7BcTBrgwVjLFQ== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by SJ2PR12MB7991.namprd12.prod.outlook.com (2603:10b6:a03:4d1::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8489.22; Fri, 28 Feb 2025 03:32:59 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%7]) with mapi id 15.20.8489.018; Fri, 28 Feb 2025 03:32:59 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , Alison Schofield , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com, chenhuacai@kernel.org, kernel@xen0n.name, loongarch@lists.linux.dev Subject: [PATCH v9 20/20] device/dax: Properly refcount device dax pages when mapping Date: Fri, 28 Feb 2025 14:31:15 +1100 Message-ID: <968d3a8e9157e7492e85d065765c027e525f9fc9.1740713401.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5P282CA0175.AUSP282.PROD.OUTLOOK.COM (2603:10c6:10:249::26) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|SJ2PR12MB7991:EE_ X-MS-Office365-Filtering-Correlation-Id: f03f1155-1661-442b-4f90-08dd57a898f8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|7416014|376014|366016|1800799024; X-Microsoft-Antispam-Message-Info: ZboMyZxBpuoKnE/MyLLyscmUBlS/adloX9CLap6AJCrFwYFkTWLwgaUChs+cBI78b9l2ksKykqQpdemLyiDVUGjM1fa5y63aEPJ/kQg8EEiiqHtSExg1ui02+IkFusgGZvDQ8GYg66JoWxd5A1zOd3GHpapJupULK0dWdYWB/VU659JTCl/pz2tbyIxgEPpaREiHtX9GlfHhH6HCLDzx8XuT6MYYDsHS9S6j6jL9hi5liH+KRNU3cmpRrIw2esefXqx9zZR3pmjQgm8uM5jpSYIpN4FaAQE0WrIlfE0z8+OpPaijYVJ3tIHtsMCsFDuO3MBye8HG9T+hcned5qy86zsUouUdwAvtvnxvxacde1GYTnczCrG157aXIFQMi7UB1aW9CL2qLL/tftQHmfsDjSsMih/vDEv+KTlqJF2T5kpBGJ3XkGaxxTUM2GN3Tl8B1FxSDZV/F1S/6vKSNr3bg0cRWD8tA8OHs+FudKxEU9qTGcPuaUPuTFjL4lWM89j47opM0CFII1xbSpliuIHUPpOfvaqaJe1wCFrOhGNnQo4/2nyE7lLc18MMJ72ydp7GlyeiUdrp5AN5z09NDNDKHKQfNw/oElK1P7FlDahE5ofmI3nviz0uWjLSduewLoGe7tXhuRza5b9lDa3AaONPpomtkUsWkG0BDLYwEsfLe0BFD1dfAWrjHFnpHrCZUSeI3oNBASnDXkBoOFhAvJTSIsHrLtPyIrtrQDm7G8VYxrt/Nx5z+duy2WMpBrj2CTCmCoZ+RJkcU6yCadMYUxPUJ7BOC1E4Ewmgl+V/WkZVu5kmdgSqYyF0pArlppAQ9UvR9r3tXLwrpMmgT/FJJz9aGZgHlo8gFqyoKLMiRn+MuaZAr5nq9w5zSfInCIlVvZ/qYIii7+LimhPbQvxhpXdSMaNMYl5ibMMllh+6ZAyFdfjtuJ5jfKeI9kqksmK0vONAQ6KFVh/CdjxtXIkKXu/XYlnv2ixtd2K6h4RDRU8SQlNg4p5XMENqBlAIa9kgtrREbnNdtDiKjaLImYCLDql7V1Hs2R/z5SAj5vBX/ObvJmgG172ye0/gfw36FS5HslyfOf37m7WFtod3gPl2h1WTZOnaU6CFMkEb2MtAYmxc11JajzlY/lBintUOox13HaYaCaTtVliBoa0IMPa1pumGbsA/ahq8Bh0eYJGX/2voXJrHuv23o+wRsSMUG0pFIGlN/vLyztYtEcgAQJDSl03hiMGaEa7lRM0diim91ezgU4E4TktRC+u6HeudUfGjYHLuoULb2138Ec/iEa/EwVdJrgpR38HuyXDYIVHCns0zScUdFPhSzysYXW759JNYTeJzVTCCm7gNzel0d8/aN0+8x7mH8g3yDhCDRvoeefQlJnMUwNWzM9IuPJDPfJgpbu4L X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(7416014)(376014)(366016)(1800799024);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: SsyXbcIJBsCx2zSjk/fTLogxTepGD6GyfDif3hHH/eY9KkEU70vstXl1PPmmW1KvKQI8WwL3mr2fuh+WOfNb3Y6fIVew0zetUw4l062MnClRaBazhLidX0SMmLpHwMJGOHa9k0WjpRfSD8ZUESbbj4w4kklnpQ3AcKuoQfjRR509mQnFyggryOpOfuce0EouiJS9TpE2bs1VtQjr52V6/QIknHzImd4veazr0PI8nM9UNDJHsc3vdLRD7Q7/HDM7Ie0LN+k4I8VTRUJ1BlugVcpfoRCRz6NBJ1JfgDcD47zSVNGZU9aMI/dqRYJtI6VHgnjpvZr+LZ3Am06DH18pucXIm+v5sEwZynGU4CyZrzh1xdWBR96j2BAabnddxLOIo3jzeKd/fISXsnL6RAJQXNiBmx9olVcr+AppYK7/TnWz2jMim2oHEw32Q60pmdxThy9VYCPJjEto25jHsOwG6ZEPOb2b0BVUeQsH5rx0I99Lb52NhSfW/5BGdnGWQ4LY2yIVg2wp0hGp43stVBxSUg+zDu3Cr0ktVTTzi5RFU+395hnVopRk21k1Ik9SRfwQhv2lmZ5TokH2+qufqINUDeN9zB9sxgmXjFclCU5IZnpZSUMAcNcBCMTzqp8hEr5hmJ689evV45Qweea6nMezj/0iw9WwiUNBBQyTDOk/8TWT/kHfsYKW+K1MlopmReXW2nJMcbykmyKgZBgnpkX6AUIB9uq5Y7qM2hb4Q6j8+CHnj1LcuGoXXZCXIkLXlmCiQdwMrHSQr1fbjN2Ixfm3Kn5SIPN08emQh+ausKWykkJnnTE3PNLHFkH0glNnR6RkyvrXx9IS5XU6EXfcPZuuzrbhXx3Ua67hRlq5uKGo12aPBApFDRJA3LQEwl3FRqEiHc9lv34g1KkVhEKasP/5Yv45Bj1W8dE8eVeCWSJEMeXiOPsSPKzoczxRc8L9NA7jzFIS3AROtj7o8RGryV+qu62tGnra1dgudnGxXiICWkfH7GsRagWwTo7I9JXaenWegkEeo/woklHaDqTcls2rUhZUGm76vIkH82h9aY7OKW23vtOgVF+sMCIerXA+XGwo87LKkinhQHvlaP6eBeWscB4o55ysdBHGAMlSqPDquE5yXC7QoSPEibcAhzzYgmtg6D+xqVCTHRGEwepeOPXryfH5p873d2cOo/q9zf++tutgOxt++hmuLJeLhDqTVccG91K8cbh1vQvUNZJ2Lo0K38/mFTQh4HfSRcJVMdGe5gaBEbF/wUrCQvd6CvTEtMZEpKUBx/c4Uz57KRheVoBlBjZ46oyRqjYPXmEfKzDJp5+iTMU+dch39Jq4VJqbdpiqkehpzKDClQYc+f8QfIbM4w4qUn2Xvr178/WfFoRAbLth5LQacHALqXGL53plX0c7iZckLFwDb2sXpIvXr83MLEM/xPXBwBmlW+fNqGLPH7yYzjYIbmAM0a9UrUXeOV7oRLio4fYUo1lVAXDIPWio6qAyB7uDfP5M/xeEImPT/vX5d5oi5qtMs8fNaQ7PrQMtbTiq2I8qOSmk8EE/+reOX7Vxj4MLELj8sMZVpLnh7e592dFnPdIAdnc+/ZAtmIMD X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: f03f1155-1661-442b-4f90-08dd57a898f8 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Feb 2025 03:32:59.2524 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: +gwYvZ9ATlaMB1KHr74HZw3AAUQ/U9ML83qzTvhsIWmHRNJ3H0m5eNhUjZI3PPKViJIfcxx7CENVgiIjHlDlNQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR12MB7991 X-Rspamd-Queue-Id: 0F611C0005 X-Stat-Signature: nke67yww4m66ofjpbm8kef7k9mi54g48 X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1740713638-673449 X-HE-Meta: U2FsdGVkX182EVeWHq4H3y1huzjPGzKllH7LYFjkZanmhVRzxe9s046F5WxgpCDFzLyp83zIUnrZVJTBRxnXk05L6vWG656/KzPkrDz4GJVWM7hI36PSSAN1zHJ81AT51b80eA/PBihgUcU6GUjAf9Hx/Tl1cnkLYYyK95VMWA+vcCUbV7qCtHKaD2SWNmQyW07trm9+CeXujpJ/giCx4wuWhn/Z8DGcJI31Mu/seehuHtjdoKZ3UCYosoVqZEjP4IO75ELK8pvQcmYY/sCVzzcRhBBPSglvo3qqQTAwyVHYXxMEu4fmJ2RAioWApPkpc5AxhbphGtDe5Ucu0KO+l6vybTGXRpQvpEm5ky6B8bfFxbS/+yRcdE+51DQw0OpUBToQY6a/9j3oOJ+wMRYEMf9uC5DjocKu0uBU/RbcEehASN0GMxsGtM7k9A5KNAfIRtPpphkgpL52BGAeWAluRR6Px1ZUW+Zhvt8jDn1Juy75hwAtvooPY9uembjP4Sr0xWo5sTEd6Zowp6W1NCceqfLdzTXJ8VFwPAPUQYcKpKlq4CkMkj6o45k3aA6gRrA/ZjUnX5IDTVHd91fs0deHKJbVkT7JCU5lnqi4QWIMMjLNoFJzGIjStpUhi/LfnNXe0nYzcxBKAF2AZC9HoAx5SOiOJqPw/Ptf97E1DN8PK9geFooA7s05v1dUh8mFBzUx5h0MmLSDD3GXlDs3x8Z2cPjFilEhyUm7q03LeTtPIfKwsVJcTJU8q6T5M6P+SB8mhfmQpbVJqi4fbDXBJpkwJdkAQZThoa4ZKxnpoBwCWHpVPFhAis4OmfttGnKBXT1EyFCV8MZD/+Z/FQpuYjq0ySfa/TZ0oDi7zLyB4hsHhUv1Urxo6cnzHiUEnGdk3DRcVXsaue2mImvs4OaKGleL8AKrETUKWXPly2N880y8497HIBU1ghKwK0s05zHLEYsEVVSPVXxG67/QlEbm+o2 EUMB07tT t3NXwhQMXfFOgjMyEbuQETwiZuKkEp6zgvDDe5pRL4i7T8arl/5NSI3xGC81ytdBvi8UXs4dyK4lhmpw5+zlVLwZ/jkenTa9TdvZaSyM4dWVwXsdWJ/RUIgcm0EM8LpPg9lfZ9t0LGvqdCAQkNqF4ZxYpdoKEpIbI3PhDHYNs1bcMPBTBZ4VrNiWNJ0FjAZJI+fIKD5Fgn3H4oqqrj70Wpeg+odTXoMvKdtX+BF8XfBYkI4LqpZ4z7k2fgUrZizm+gBzvj3qNIu5S3mcIOODMP03j7CiSbxm4LT+6xlJ1vptcdSAFofJUv3jbEr+boc+VUna6lnIKpppNnNHzJh6aGymrx94yfPOW4PUmFwP47jHyI7Z7LDYPbiQBR+2dGaYqJQR36N/d6tQRA9WqOn8QWv84Oi36PiRmylj8 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Device DAX pages are currently not reference counted when mapped, instead relying on the devmap PTE bit to ensure mapping code will not get/put references. This requires special handling in various page table walkers, particularly GUP, to manage references on the underlying pgmap to ensure the pages remain valid. However there is no reason these pages can't be refcounted properly at map time. Doning so eliminates the need for the devmap PTE bit, freeing up a precious PTE bit. It also simplifies GUP as it no longer needs to manage the special pgmap references and can instead just treat the pages normally as defined by vm_normal_page(). Signed-off-by: Alistair Popple Changes for v9: - Fix warnings when pgmap->ops isn't defined for FS_DAX, reported by Gerald Schaefer. --- drivers/dax/device.c | 15 +++++++++------ mm/memremap.c | 14 +++++++------- 2 files changed, 16 insertions(+), 13 deletions(-) diff --git a/drivers/dax/device.c b/drivers/dax/device.c index bc871a3..328231c 100644 --- a/drivers/dax/device.c +++ b/drivers/dax/device.c @@ -125,11 +125,12 @@ static vm_fault_t __dev_dax_pte_fault(struct dev_dax *dev_dax, return VM_FAULT_SIGBUS; } - pfn = phys_to_pfn_t(phys, PFN_DEV|PFN_MAP); + pfn = phys_to_pfn_t(phys, 0); dax_set_mapping(vmf, pfn, fault_size); - return vmf_insert_mixed(vmf->vma, vmf->address, pfn); + return vmf_insert_page_mkwrite(vmf, pfn_t_to_page(pfn), + vmf->flags & FAULT_FLAG_WRITE); } static vm_fault_t __dev_dax_pmd_fault(struct dev_dax *dev_dax, @@ -168,11 +169,12 @@ static vm_fault_t __dev_dax_pmd_fault(struct dev_dax *dev_dax, return VM_FAULT_SIGBUS; } - pfn = phys_to_pfn_t(phys, PFN_DEV|PFN_MAP); + pfn = phys_to_pfn_t(phys, 0); dax_set_mapping(vmf, pfn, fault_size); - return vmf_insert_pfn_pmd(vmf, pfn, vmf->flags & FAULT_FLAG_WRITE); + return vmf_insert_folio_pmd(vmf, page_folio(pfn_t_to_page(pfn)), + vmf->flags & FAULT_FLAG_WRITE); } #ifdef CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD @@ -213,11 +215,12 @@ static vm_fault_t __dev_dax_pud_fault(struct dev_dax *dev_dax, return VM_FAULT_SIGBUS; } - pfn = phys_to_pfn_t(phys, PFN_DEV|PFN_MAP); + pfn = phys_to_pfn_t(phys, 0); dax_set_mapping(vmf, pfn, fault_size); - return vmf_insert_pfn_pud(vmf, pfn, vmf->flags & FAULT_FLAG_WRITE); + return vmf_insert_folio_pud(vmf, page_folio(pfn_t_to_page(pfn)), + vmf->flags & FAULT_FLAG_WRITE); } #else static vm_fault_t __dev_dax_pud_fault(struct dev_dax *dev_dax, diff --git a/mm/memremap.c b/mm/memremap.c index 9a8879b..2aebc1b 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -460,11 +460,7 @@ void free_zone_device_folio(struct folio *folio) { struct dev_pagemap *pgmap = folio->pgmap; - if (WARN_ON_ONCE(!pgmap->ops)) - return; - - if (WARN_ON_ONCE(pgmap->type != MEMORY_DEVICE_FS_DAX && - !pgmap->ops->page_free)) + if (WARN_ON_ONCE(!pgmap)) return; mem_cgroup_uncharge(folio); @@ -494,12 +490,15 @@ void free_zone_device_folio(struct folio *folio) * zero which indicating the page has been removed from the file * system mapping. */ - if (pgmap->type != MEMORY_DEVICE_FS_DAX) + if (pgmap->type != MEMORY_DEVICE_FS_DAX && + pgmap->type != MEMORY_DEVICE_GENERIC) folio->mapping = NULL; switch (pgmap->type) { case MEMORY_DEVICE_PRIVATE: case MEMORY_DEVICE_COHERENT: + if (WARN_ON_ONCE(!pgmap->ops || !pgmap->ops->page_free)) + break; pgmap->ops->page_free(folio_page(folio, 0)); put_dev_pagemap(pgmap); break; @@ -509,7 +508,6 @@ void free_zone_device_folio(struct folio *folio) * Reset the refcount to 1 to prepare for handing out the page * again. */ - pgmap->ops->page_free(folio_page(folio, 0)); folio_set_count(folio, 1); break; @@ -518,6 +516,8 @@ void free_zone_device_folio(struct folio *folio) break; case MEMORY_DEVICE_PCI_P2PDMA: + if (WARN_ON_ONCE(!pgmap->ops || !pgmap->ops->page_free)) + break; pgmap->ops->page_free(folio_page(folio, 0)); break; }