From patchwork Tue Dec 17 05:12:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13911033 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 29028E7717F for ; Tue, 17 Dec 2024 05:15:45 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 1DDEC6B00A8; Tue, 17 Dec 2024 00:15:40 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id 18DA76B00A7; Tue, 17 Dec 2024 00:15:40 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id DEB7D6B00A8; Tue, 17 Dec 2024 00:15:39 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id B9C796B00A6 for ; Tue, 17 Dec 2024 00:15:39 -0500 (EST) Received: from smtpin21.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay07.hostedemail.com (Postfix) with ESMTP id 720AF1603F3 for ; Tue, 17 Dec 2024 05:15:39 +0000 (UTC) X-FDA: 82903287390.21.B7B4C52 Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2050.outbound.protection.outlook.com [40.107.223.50]) by imf08.hostedemail.com (Postfix) with ESMTP id B06DB160004 for ; Tue, 17 Dec 2024 05:15:16 +0000 (UTC) Authentication-Results: imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=JKHNlyXK; spf=pass (imf08.hostedemail.com: domain of apopple@nvidia.com designates 40.107.223.50 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1734412505; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=8p0f5STc3jZ7dypfmVkR93SWOiB4Au8VequleXzSvOU=; b=YiRDt1oHfD6EizXVtqZHflbsGE/AyJBW/tRbPIgZNT23A945y81F9o+JQSah6MfT68EynJ lpgTQU8ua1cLbyyAAPS7u+c0NRKAy4TCcZOlrLGNblFSB6by0bX08eTUXmYz13Crxuq/tO izD9JESgIgGS7fS7ucSWsKXczTIHazk= ARC-Authentication-Results: i=2; imf08.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=JKHNlyXK; spf=pass (imf08.hostedemail.com: domain of apopple@nvidia.com designates 40.107.223.50 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1734412505; a=rsa-sha256; cv=pass; b=TiGYnIYEgumXuLeuE93I+1YwIC7cxXPwa17205DFO0ZVNp79NwMXaae43qaWP7etZ6EtgM tMf7nwFdufUaPSzQk/njS0DyoWlG1nVOAwhJ4UwCZ35wtbDdoG+u5WXaULBekVTyUiuOEm dfUq4cEjwWBpQFaQC7fZuhfrc+RskQI= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=dZeDZTeS+TQgSebGS+9n5lswdsEgyXAH1KEVxk6X6a0z9vnaOknOLGTJrQtDKONakTTRYkyvb4MFb+SY95cUKK2Pd/n981hVZfvC2JcHP9AiFPnRBtFacaG1lwxb6bh6JueZL1kGhfVkFj6WP/rlNdTBptEdrKHZySCORXzhIf1p0MTbx18520MQFhuCsGqjcd8QeDKpDE2o9xmmido1Ram20i1zTZL5sx3qqZq5z21bGHZGcrnvp5h8BJdwffTWVgc8NscEFKQtj0rYsve6y/Zlmxs1SR1Po1hDaclrEwepJ4/WRBjIh4/V+9x6uoWaS6gTnrXbHRLVRss4gHIIVQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=8p0f5STc3jZ7dypfmVkR93SWOiB4Au8VequleXzSvOU=; b=Z1JV//oYA56VWqeOAObWm5/01TuUpsLG1sEp6NQdrNR5xuuKW1NGqDqdsotuGpB5EKxMcODSJi0EnrUrlLtmr33gaGwQtABvhbJQ4COaasyJJq0HBbJTGZWbuBzYM1KHYQZ3IpbiJipWMUWS881jaSKCBfsPRV2ANOqWKSJJyy3B3lvXDNQpN7xLZixEKuDfR50fzLQd3mdcIR7LrLpgfjTeFGg9NxgsJ+vacdT4BAR1tEdfbd5q4FMqcg8XiaSBrR7JRmkm+mQUm9G6IDfHhaj66HCsIjh+cdDlTNYU1yI5FktH5p4sU2iNG8wi5PdLIY28xPhPDgjzbT231hugTg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8p0f5STc3jZ7dypfmVkR93SWOiB4Au8VequleXzSvOU=; b=JKHNlyXKgo4YHc/Fs21PhnRUfoZ9iR9i1iXj1XcmWLNKhM0/SicRohXpSEqBN247dlU32ofSmLd3KGCwLd1VVRrZ79ixfGmYqsnvXF2Df2jLXtuCbhRFaqw/aa6MWUwfDF38ZJ/jP62irVouqW12jeoSySYTAVNjt617W4cxItLsAuw6amRFbwIQnRdPPS4oCPnOStZJoAFa1pvH9Kxsavx6z0n3cMXJE1aQ96o4RCBFDJ7ENqe6j3FcACbYLAMt4OlgsjB/EDbYg9yaAIh1ViPtnq4eX0Gagk3twSdHr+SnqBWLTBPjGAkI55Gay6EbDalEsDySjKiCDHlSAtGP5Q== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by CH3PR12MB8936.namprd12.prod.outlook.com (2603:10b6:610:179::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8251.22; Tue, 17 Dec 2024 05:13:53 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%4]) with mapi id 15.20.8251.015; Tue, 17 Dec 2024 05:13:53 +0000 From: Alistair Popple To: akpm@linux-foundation.org, dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com Subject: [PATCH v4 05/25] fs/dax: Create a common implementation to break DAX layouts Date: Tue, 17 Dec 2024 16:12:48 +1100 Message-ID: X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5PR01CA0051.ausprd01.prod.outlook.com (2603:10c6:10:1fc::11) To DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|CH3PR12MB8936:EE_ X-MS-Office365-Filtering-Correlation-Id: 1609b788-e811-48b8-d423-08dd1e599990 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|376014|1800799024|7416014|366016; X-Microsoft-Antispam-Message-Info: 6uAGNTVtzvJ9GKkP1DgOY/DqIfX9RK/oAWPMjmX1xkKWZPxOr2e78NrGV7yt4rig3GfT6ang1ABW4RxAXMk1K9mKpN6J/lgNpR1kEyL2vqn5Lc7n7lvmh7xAnnwZ+9pyh5HkCha9iFCFyh02a/DvwWWQrw3eDxN9B/pXgGG8zQ9lcyJILjtB9uA4UxRxZP8lEo2MgVYi3w9bg97WyOmuEaBOVpanOIWzd/vfXZocDIEjPHKMKCYZ4VHmgu1jLyabJ9mWCfCmAYNVh7+YzEuiqZv+zza8IgJI+S9x3gSBB8Xqfc+TGcfNo1rLs7YbsGeDFMH2D3luJv/uPbeoY7wDfWbeX5zMLQxUTsUDVofs8e65wWEA2TzZcQ7OW0W3Fd6Qm7OefkyHjRxA5y3xaC4SSUnjNLZ8L/OasB6GPvLRB6FA4kDyGhsXVkqFcUqY4a7TrqKfr4a7J+iG9JqCx/LG4sP/y47QhVUcMnr+1kO2/ZMB79qQ9I2dhP2X7AH6eNxNhxn4YM69xvoe/iXzX2UmmQmCnKvmayNQ4LCyHv509SsNfO+joBdQ47O6wpRk5acTJNAoqWHK7u2XbDCWxMfAvZ35YfE0S7Jn+R/yHGUAOpjafzv95Suy9ky7ErO1CHAL6ZQjtanTpjB2swATW7Qcej4eXz7D4IcqJ93pfN2e5g8WuKw8rml07m1RAbS775/3o0+cinHWruP7sO1NhCmfFMMVFN+EwpIWwHomC3WtnFRlGXDisyNCKAY4baml12wfrIsXdaKg/m88whEnsyZaS2iKcqw3W6wNRmCAyyPYRD5OEjKeGsF+h2KJLUA9y8X0Dbke/V1I0Eq/Y/hqW5VAdCCotBIutE8Uck8ZaFybMnBBMxcUKhtR1qARRGdWtLKEccQm/IuN2fk9EZpyqn/jF7NS/CfisSdEHDpOBoEtCmDTMc7yVORw2UCtXbB6WAxZTie5PW9R8Mdfm9eALJzdjMe0W4TE/oSK/xb0R+gMiQlICB/gzlUt4DEd7O5b/PNNw+uWTTBNDaK5jXXbkZ9Y7AItiWCiFzKxoQl1SkHWTs9puX9C6015Gs07WZCZ84ynWQ1j2V/bXZjZT/VGV3vXREqHsUxWpwEuaJk8VDOwn2O6+qUjzYvCqbIlRbW9i7S2B7vSj9VpwzhtqDBU7x6HL9oRY8su9iHOGU/m/TZXQhufxk3U9uBEIgRAZVB99jJyFuqThLApbZIyWu8nfNXh2z8hQXwe0akOXKU6Oc4IulBIqi3AYb/27qhw9ZpnyAHhj+a7GBbzFLsAVTuC8RGk8aVAn3L5a0o0OWH6V0SvRk1PkFrMTJCdl7w/U1Qjq3jL4M17Zgr6Y94gh4xqZTdaoC983GkPfM811tbAUNUc7kKIeIyMHzZOR0v8dEX9P/dD X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(376014)(1800799024)(7416014)(366016);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: AGHpvSwsBn20BERcghLsY/9mt3Kqc9mST4PmYU28St7rqzgjn79qSZaOZNQ9HbiRpe7Qp3embFgsANj6R3sZA7jcQPTlhh0J3LVppsC9oluyhERrxiC7Def7f75kMFVEWHA7JAateP3zxdZs951fVwAJ0r7oB0Tbr7V0DMn50C7+Z7OLik3evuhce/3xl97hQ8o/UTzsXlLuYxPKrZu0DUWCTYizgMdK0sM8L9JtPwco2usGE3c/CAEKtzEMGsZyDKaHWl9tXsFJGiVkBOFN+5YaWfYHTAmo3sJQOp0QBX1kemqlN8pEyr2UTN6/PgbA8+USZ/n7mbGIyLagfM5DHfoY/Rt6aFRHAu2zNAlZmNpW1MGTMRdDVEcUK8mNrZjFzMhgCvmCPvpyYblybxDDPt/yk4e0XD6iefJp6UTVSS3QEKczWSa1EvaPu8kKwOCmb2Xe9vvddTUpRxasljxYz6h+uUiKkQViW852B8n9GXFYtcM83pYK1V9eSMnt3D4iARI1LVpL9zjJRDDqdnb4d2mh3+EDM0SFLGJAa2Kxx8auQKN+D4fUyNj+2KyDBeMEorPCMLchWn1B05hXUZgLgwhDfrEYmwE5vG2GumFF6V4vuo+4NrwrdPqxZ9xWKp6pTUCIkK/2R7uUbP1SEw/mOcknOaIsnr2NGW0Q/jrEGCdkiWNDDD5s9qZI0AhgPf91FTGKRgju3L8DzPJAKC/jDuAA3DV1YN+E7jdn4ML4LJ3hQkzofnamoZb2E5BFGMA2Ab+zEP3KJDZOF/aYldPM0PfhUxbWo9+qw4vX1iDJixVkl5nhTPL5b3RxFpnCmvHBjzL8RrECd+Fdu0uaeQ5oOADCeucaaR9bLOseSbKwEkd64nLmB+OUPtM3ZwGEYE9HP/nSwAd5sbF8nXC2R1QM59k7/dgKAygGte3fTj00/yWsNOWM5MkNyS8z+dGpKHwxcvd/rOvpWlWZuJO9k6JkCZbrBqXNh2SvC3eeweSkHJ8EisK8Kp1JGoK2W0gsPbgGb9XMjI5IlaTVXNfLen5bE/ja3AlZPlnoSFht4mNPbYAZN7Do7ja2lsXz9VgN+vSHL26gUMhv0YOVNkbPG8c2oh3hppEYP3l6ka6U2qQToqYnFzYQrk1G2Z/Hzl5UvFsKfvw32eaYrc8Z99bNwDWQHptPwJQxyHqzbitcL18kdM3VxX88uZ/TmBtYIEBNk348lzXfeZoO09KVoWwp+OSyH7xJyx/wxnOpVOO4XYQ3tHiVP9OxGZBp679ne9jKdVKisEWW3Jj19hQztRr8zM1TPamT88ZRO3+TAgIymcwv5KHj1DXi6Bv2FGBX7I2tRKa0176Na5lbxTKZ2A+F+WH2kHVbAcMNB7fPUnkQyDzuOEgbVfbE9dGyMJ3zNAETEPe0PLhWVF2ERHIEMV1TL58ZywfslVGsweCsvot3KNmOgagTTT3Qvh320/4F3dZ42yNp51AraXq6+CjADOksPx2Vui6W+ySOzcIhuo/URm0sJkzssuLu7MdPY3fBb3lJB4fWySXAY9ajU+xJTD7XG5gzxoYMFG3lQUEav79hhuxlBo1OtojpLd+hLDM2UVTZpI/G X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1609b788-e811-48b8-d423-08dd1e599990 X-MS-Exchange-CrossTenant-AuthSource: DS0PR12MB7726.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Dec 2024 05:13:53.6819 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: JU0QvJClRJgWGDY/N1ErJEU7VXQWpqrtattWz3zgC3vSAz8Y0o5KM35dV/f+3zRH6MRdypKWGGarZvUH9dwvYg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8936 X-Rspamd-Server: rspam06 X-Rspamd-Queue-Id: B06DB160004 X-Rspam-User: X-Stat-Signature: ytbsq7s5f7tu55txwgk1u9y96mwmyu64 X-HE-Tag: 1734412516-169205 X-HE-Meta: U2FsdGVkX1/dVndqi8d84W12jDDKyzmPK5W9pf0RMLEcUteB0sZYuoA1awujEgedGZypzjqAb8xwoN6xekJq2ZDvLGenvJt76yO0AoKJX0K4XyypVIeEo26/HcrwWmEOzOxlIRWsEIYTvMd0JogSv0ZnOi+wXmJkiFVOx/kuGpEOXqACK70hHrRU/cr3rUYXuITOFUlgpRnPTpxMOHYiR90Y1nUvfU4u3M3/8t3qeL3VGH1ZgVNhMWnEmWtkpmSs/BCzFDKoEOu/sDE78PRQuwV6pSviH1LD+sjbbWHb25Ii3RpYHhSMCxxLmouZTLt2nOoGDSgB8O1JC1AuPE/1tRycLhib8GbPdk5izwqWM6RwwPCQVtZXpCRgPc8pVi0JkhHtIXLY+92SU+6ftOom92VQvOwfEFIS83zq9d4aZmKcdTMKQw8nW84KxGDeyYdgiPh/f0m9Zp8WtzfbzTJOh+i/CpBQyydGnCCYqTyYJSkByRmdh8ociYK6iMFbmizi3jQxCrlXtFUq/03prgEgHkkqLPwShKpLj78VuDmdwfjV1tswwrsLx89N9UYGRBekQLhTECuIQ8u1E+YwHwdoacKlyidMQEhmjBZiNi8GhMJ2UifFX6nTvUBOQscm5mCAug4YsbX1kuHRjMRC7M/dXdnrLlQk+P3YGHcO3WZpWP4N6iwqjS7V5L8MVchg29z63Jh9CXYCsJG3Vr/wYdJhBVU7yrtMcd9/SH/3acg8S3P6+kl+85oJkXE7haeNPF9GmDG02rIQbOG9bDSG14DDtwLbINQ1jL5Rl/wfckw5k7QzShmuiM//30Y9p9qWUwpdoZiIxUUK0wi/hWZdHL+LtB6EKEWplAUDIZmRZnQPtJgM5As4uxOfTDpFAs8Z1uppmRodINzVTHstD8HkcwjOmpWt4fU4rCNla5oY3a/w/5XdF4p6EcKv9snC1tQk8+r/79Ql83LNGZCfTA8VPDJ O6WniWhk MSkt8YTUBJUBOGgI64nvCJwziJazzA1cdFEtV0VLEaVBOpBSW6Z6uab6QSJ8xVFKulGJxE+zmxpKRdJ5lcL3L4dwyNPG8IbKkAQtRPhv1iTLF3FYLFuDpclsee4/ukWAdchxk2aT3WEFBju+7f/EdWhGXrgO6ptAorkItmkHkgJkgS+WKtEJmH8VKWLrMjhOhTkRGPxC/aWZCoqFlcbI0WUsLnfYyZyre0McnFwVvQLmefAbDCPcvaKfSTnkCNWl5JNirtS0cUNuiodQIPQU3YH5YtfcNVJIWGHQPD+fkpwIuNEHPIHKQiT9mov025F4gSJjw/OiUFMcWwSRL/7I5LfqmEAuQtkxd1986BmL0UzmF3bFLA9ApsLmRrD0KuD1jgFlWhzWMwoiWyOZ3W6Yb/HvsP8fWrx3Bh8RE X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: Prior to freeing a block file systems supporting FS DAX must check that the associated pages are both unmapped from user-space and not undergoing DMA or other access from eg. get_user_pages(). This is achieved by unmapping the file range and scanning the FS DAX page-cache to see if any pages within the mapping have an elevated refcount. This is done using two functions - dax_layout_busy_page_range() which returns a page to wait for the refcount to become idle on. Rather than open-code this introduce a common implementation to both unmap and wait for the page to become idle. Signed-off-by: Alistair Popple --- Changes for v4: - Fixed some build breakage due to missing symbol exports reported by John Hubbard (thanks!). --- fs/dax.c | 30 ++++++++++++++++++++++++++++++ fs/ext4/inode.c | 10 +--------- fs/fuse/dax.c | 29 +++++------------------------ fs/xfs/xfs_inode.c | 23 +++++------------------ fs/xfs/xfs_inode.h | 2 +- include/linux/dax.h | 21 +++++++++++++++++++++ mm/madvise.c | 8 ++++---- 7 files changed, 67 insertions(+), 56 deletions(-) diff --git a/fs/dax.c b/fs/dax.c index d010c10..5462d9d 100644 --- a/fs/dax.c +++ b/fs/dax.c @@ -845,6 +845,36 @@ int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index) return ret; } +static int wait_page_idle(struct page *page, + void (cb)(struct inode *), + struct inode *inode) +{ + return ___wait_var_event(page, page_ref_count(page) == 1, + TASK_INTERRUPTIBLE, 0, 0, cb(inode)); +} + +/* + * Unmaps the inode and waits for any DMA to complete prior to deleting the + * DAX mapping entries for the range. + */ +int dax_break_mapping(struct inode *inode, loff_t start, loff_t end, + void (cb)(struct inode *)) +{ + struct page *page; + int error; + + do { + page = dax_layout_busy_page_range(inode->i_mapping, start, end); + if (!page) + break; + + error = wait_page_idle(page, cb, inode); + } while (error == 0); + + return error; +} +EXPORT_SYMBOL_GPL(dax_break_mapping); + /* * Invalidate DAX entry if it is clean. */ diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c index cc1acb1..ee8e83f 100644 --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c @@ -3917,15 +3917,7 @@ int ext4_break_layouts(struct inode *inode) if (WARN_ON_ONCE(!rwsem_is_locked(&inode->i_mapping->invalidate_lock))) return -EINVAL; - do { - page = dax_layout_busy_page(inode->i_mapping); - if (!page) - return 0; - - error = dax_wait_page_idle(page, ext4_wait_dax_page, inode); - } while (error == 0); - - return error; + return dax_break_mapping_inode(inode, ext4_wait_dax_page); } /* diff --git a/fs/fuse/dax.c b/fs/fuse/dax.c index d156c55..48d0652 100644 --- a/fs/fuse/dax.c +++ b/fs/fuse/dax.c @@ -665,38 +665,19 @@ static void fuse_wait_dax_page(struct inode *inode) filemap_invalidate_lock(inode->i_mapping); } -/* Should be called with mapping->invalidate_lock held exclusively */ -static int __fuse_dax_break_layouts(struct inode *inode, bool *retry, - loff_t start, loff_t end) -{ - struct page *page; - - page = dax_layout_busy_page_range(inode->i_mapping, start, end); - if (!page) - return 0; - - *retry = true; - return dax_wait_page_idle(page, fuse_wait_dax_page, inode); -} - -/* dmap_end == 0 leads to unmapping of whole file */ +/* Should be called with mapping->invalidate_lock held exclusively. + * dmap_end == 0 leads to unmapping of whole file. + */ int fuse_dax_break_layouts(struct inode *inode, u64 dmap_start, u64 dmap_end) { - bool retry; - int ret; - - do { - retry = false; - ret = __fuse_dax_break_layouts(inode, &retry, dmap_start, - dmap_end); - } while (ret == 0 && retry); if (!dmap_end) { dmap_start = 0; dmap_end = LLONG_MAX; } - return ret; + return dax_break_mapping(inode, dmap_start, dmap_end, + fuse_wait_dax_page); } ssize_t fuse_dax_read_iter(struct kiocb *iocb, struct iov_iter *to) diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c index 42ea203..295730a 100644 --- a/fs/xfs/xfs_inode.c +++ b/fs/xfs/xfs_inode.c @@ -2715,21 +2715,17 @@ xfs_mmaplock_two_inodes_and_break_dax_layout( struct xfs_inode *ip2) { int error; - bool retry; struct page *page; if (ip1->i_ino > ip2->i_ino) swap(ip1, ip2); again: - retry = false; /* Lock the first inode */ xfs_ilock(ip1, XFS_MMAPLOCK_EXCL); - error = xfs_break_dax_layouts(VFS_I(ip1), &retry); - if (error || retry) { + error = xfs_break_dax_layouts(VFS_I(ip1)); + if (error) { xfs_iunlock(ip1, XFS_MMAPLOCK_EXCL); - if (error == 0 && retry) - goto again; return error; } @@ -2988,19 +2984,11 @@ xfs_wait_dax_page( int xfs_break_dax_layouts( - struct inode *inode, - bool *retry) + struct inode *inode) { - struct page *page; - xfs_assert_ilocked(XFS_I(inode), XFS_MMAPLOCK_EXCL); - page = dax_layout_busy_page(inode->i_mapping); - if (!page) - return 0; - - *retry = true; - return dax_wait_page_idle(page, xfs_wait_dax_page, inode); + return dax_break_mapping_inode(inode, xfs_wait_dax_page); } int @@ -3018,8 +3006,7 @@ xfs_break_layouts( retry = false; switch (reason) { case BREAK_UNMAP: - error = xfs_break_dax_layouts(inode, &retry); - if (error || retry) + if (xfs_break_dax_layouts(inode)) break; fallthrough; case BREAK_WRITE: diff --git a/fs/xfs/xfs_inode.h b/fs/xfs/xfs_inode.h index 1648dc5..c4f03f6 100644 --- a/fs/xfs/xfs_inode.h +++ b/fs/xfs/xfs_inode.h @@ -593,7 +593,7 @@ xfs_itruncate_extents( return xfs_itruncate_extents_flags(tpp, ip, whichfork, new_size, 0); } -int xfs_break_dax_layouts(struct inode *inode, bool *retry); +int xfs_break_dax_layouts(struct inode *inode); int xfs_break_layouts(struct inode *inode, uint *iolock, enum layout_break_reason reason); diff --git a/include/linux/dax.h b/include/linux/dax.h index 9b1ce98..f6583d3 100644 --- a/include/linux/dax.h +++ b/include/linux/dax.h @@ -228,6 +228,20 @@ static inline void dax_read_unlock(int id) { } #endif /* CONFIG_DAX */ + +#if !IS_ENABLED(CONFIG_FS_DAX) +static inline int __must_check dax_break_mapping(struct inode *inode, + loff_t start, loff_t end, void (cb)(struct inode *)) +{ + return 0; +} + +static inline void dax_break_mapping_uninterruptible(struct inode *inode, + void (cb)(struct inode *)) +{ +} +#endif + bool dax_alive(struct dax_device *dax_dev); void *dax_get_private(struct dax_device *dax_dev); long dax_direct_access(struct dax_device *dax_dev, pgoff_t pgoff, long nr_pages, @@ -251,6 +265,13 @@ vm_fault_t dax_finish_sync_fault(struct vm_fault *vmf, int dax_delete_mapping_entry(struct address_space *mapping, pgoff_t index); int dax_invalidate_mapping_entry_sync(struct address_space *mapping, pgoff_t index); +int __must_check dax_break_mapping(struct inode *inode, loff_t start, + loff_t end, void (cb)(struct inode *)); +static inline int __must_check dax_break_mapping_inode(struct inode *inode, + void (cb)(struct inode *)) +{ + return dax_break_mapping(inode, 0, LLONG_MAX, cb); +} int dax_dedupe_file_range_compare(struct inode *src, loff_t srcoff, struct inode *dest, loff_t destoff, loff_t len, bool *is_same, diff --git a/mm/madvise.c b/mm/madvise.c index 49f3a75..1f4c99e 100644 --- a/mm/madvise.c +++ b/mm/madvise.c @@ -1063,7 +1063,7 @@ static int guard_install_pud_entry(pud_t *pud, unsigned long addr, pud_t pudval = pudp_get(pud); /* If huge return >0 so we abort the operation + zap. */ - return pud_trans_huge(pudval) || pud_devmap(pudval); + return pud_trans_huge(pudval); } static int guard_install_pmd_entry(pmd_t *pmd, unsigned long addr, @@ -1072,7 +1072,7 @@ static int guard_install_pmd_entry(pmd_t *pmd, unsigned long addr, pmd_t pmdval = pmdp_get(pmd); /* If huge return >0 so we abort the operation + zap. */ - return pmd_trans_huge(pmdval) || pmd_devmap(pmdval); + return pmd_trans_huge(pmdval); } static int guard_install_pte_entry(pte_t *pte, unsigned long addr, @@ -1183,7 +1183,7 @@ static int guard_remove_pud_entry(pud_t *pud, unsigned long addr, pud_t pudval = pudp_get(pud); /* If huge, cannot have guard pages present, so no-op - skip. */ - if (pud_trans_huge(pudval) || pud_devmap(pudval)) + if (pud_trans_huge(pudval)) walk->action = ACTION_CONTINUE; return 0; @@ -1195,7 +1195,7 @@ static int guard_remove_pmd_entry(pmd_t *pmd, unsigned long addr, pmd_t pmdval = pmdp_get(pmd); /* If huge, cannot have guard pages present, so no-op - skip. */ - if (pmd_trans_huge(pmdval) || pmd_devmap(pmdval)) + if (pmd_trans_huge(pmdval)) walk->action = ACTION_CONTINUE; return 0;