From patchwork Fri Nov 22 01:40:31 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 13882579 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id 511E6E65D10 for ; Fri, 22 Nov 2024 01:43:54 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id D48DF8D0002; Thu, 21 Nov 2024 20:43:53 -0500 (EST) Received: by kanga.kvack.org (Postfix, from userid 40) id CF9706B00AD; Thu, 21 Nov 2024 20:43:53 -0500 (EST) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id AD4F78D0002; Thu, 21 Nov 2024 20:43:53 -0500 (EST) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0016.hostedemail.com [216.40.44.16]) by kanga.kvack.org (Postfix) with ESMTP id 862306B00AC for ; Thu, 21 Nov 2024 20:43:53 -0500 (EST) Received: from smtpin10.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay03.hostedemail.com (Postfix) with ESMTP id 3A79BA0D4B for ; Fri, 22 Nov 2024 01:43:53 +0000 (UTC) X-FDA: 82812033066.10.CCCC035 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2080.outbound.protection.outlook.com [40.107.237.80]) by imf26.hostedemail.com (Postfix) with ESMTP id 03B7214000B for ; Fri, 22 Nov 2024 01:43:09 +0000 (UTC) Authentication-Results: imf26.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Z0DXHulI; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf26.hostedemail.com: domain of apopple@nvidia.com designates 40.107.237.80 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1732239693; a=rsa-sha256; cv=pass; b=6/Vqm+2b5v0gxbPu3mQdgq8zrqgRw7XiENbIrmWC/MhslBVgFMJaQfDOEG14R/BjWSg+dA 9NAlD48oeyQOmLbCnrg6WhoQ/AMRf9EHuIeRJrQ37+43CWspG5aJgUgXrTQce83pe/HoL/ BdhBoqlr1N6IgAVmB8M10GN2prW+cF8= ARC-Authentication-Results: i=2; imf26.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=Z0DXHulI; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector10001:i=1"); spf=pass (imf26.hostedemail.com: domain of apopple@nvidia.com designates 40.107.237.80 as permitted sender) smtp.mailfrom=apopple@nvidia.com ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1732239693; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=ESy7x46loh4IEoR5LJkN4qdt6dpTfnJr5eSjyPduQ/Q=; b=WT/yGActcFaiAc6jH6khO1PctAJTk5NJJMJ8J+/xrF+Zu48FHT3N+0tz4bZhswLcJFKw3I g5kcOruzMLiJKMMT6T4griILBavxL4ovpL+jgvRjB/EyMovjldfXuv6jym1tQNjyZXCByw R0TiJG3DMLrLQBkUK1oi9wbJng0L5O0= ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=IvSsFWVpCDcn3DXD8tYh4PKczkIgdcMaO7OD5+SxO7bBJImgK5If7t5ZXd/M+hXSrDBuoMfSve+Dh97QyYGHSXgrNDqwOD/ElmqIVE7rWwNscotP0CfLLHdQa6goOX820mfTRhCZuC7+gxezsU9LXH2BxrlWxrSUU8v4jGGfq/F3eI3spSnSNtaOaTOfuheKlT/F2P90ECI6izLGQdBU11b1NK4/Am7PA+thltE5d+vYQi2q5SVOAwTLl18k7bvq8l/mEmgm03gh8yEpzXRROiAze3V3zPp8G1uXwoImH+LLLqwlFDfOc4hJnMRjYBYKAXna7e6z6G27obebqkzFzA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ESy7x46loh4IEoR5LJkN4qdt6dpTfnJr5eSjyPduQ/Q=; b=kdTZN3CvNoxTygZDF+0Z3BhZEZq49KDu+TkrfN5voeHRP8BXzcEL6qTfXiIcyDiaSx4k/wR5zPlJII0W0KFoKlcjvTbPGH78+/SH5z8dT44sYmjA7Aqj0LELQSI6qnq86uWLcNb+BtHb8ost7BM2HMPprA1lqDrylJTQJjq1my4YDLaMpJ+J7BUr6Yy8KRJ4x2YVo4dchTludjvcP54Ly/nzuEOIbEMiREA7/aAOdau8Gj5yJSj7H3/g/I1182uTc3QMhrLVtbrnshDLa+sLvyskLaxkzMAAPMDF1EYoX6T/DUzdpeTRUlQY5Hh5XITWHtvGhpf3SxEv1d+NV7AvXw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ESy7x46loh4IEoR5LJkN4qdt6dpTfnJr5eSjyPduQ/Q=; b=Z0DXHulIaXHnocHapL/4RkNKv0vd841JpCLXuYJTP5JiVnH4jU064lFe9X1DcYZKQ0ThJdGLFcwHRXqZJuSzd4eo/+nRz1Zz2c67475OaCbykQGpxcfwQNTz34H1nt7N8yhUbfSAbM7TAqRgZ3WOAGyrhU2nUs7gYa8VH6qsP9lKYA3WsU2b1trD38DZFwUTqC/zAHcM9evlkWmpyeci0urR9iS3dCcYholdtPp6JSLzMnNPYxa7B+Vt1mTBv3gYbWoh7a2VTFUBN5LrS4NCrL93ULyFWP6Rxe0sbND1wHPKLvLE+My18iP3PS32vz+01gGlYCcVdNgljMdQn6xLvA== Received: from DS0PR12MB7726.namprd12.prod.outlook.com (2603:10b6:8:130::6) by IA1PR12MB6305.namprd12.prod.outlook.com (2603:10b6:208:3e7::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.8182.17; Fri, 22 Nov 2024 01:41:55 +0000 Received: from DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe]) by DS0PR12MB7726.namprd12.prod.outlook.com ([fe80::953f:2f80:90c5:67fe%4]) with mapi id 15.20.8182.016; Fri, 22 Nov 2024 01:41:55 +0000 From: Alistair Popple To: dan.j.williams@intel.com, linux-mm@kvack.org Cc: Alistair Popple , lina@asahilina.net, zhang.lyra@gmail.com, gerald.schaefer@linux.ibm.com, vishal.l.verma@intel.com, dave.jiang@intel.com, logang@deltatee.com, bhelgaas@google.com, jack@suse.cz, jgg@ziepe.ca, catalin.marinas@arm.com, will@kernel.org, mpe@ellerman.id.au, npiggin@gmail.com, dave.hansen@linux.intel.com, ira.weiny@intel.com, willy@infradead.org, djwong@kernel.org, tytso@mit.edu, linmiaohe@huawei.com, david@redhat.com, peterx@redhat.com, linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, nvdimm@lists.linux.dev, linux-cxl@vger.kernel.org, linux-fsdevel@vger.kernel.org, linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org, jhubbard@nvidia.com, hch@lst.de, david@fromorbit.com Subject: [PATCH v3 10/25] pci/p2pdma: Don't initialise page refcount to one Date: Fri, 22 Nov 2024 12:40:31 +1100 Message-ID: <27381b50b65a218da99a2448023b774dd75540df.1732239628.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.45.2 In-Reply-To: References: X-ClientProxiedBy: SY5PR01CA0107.ausprd01.prod.outlook.com (2603:10c6:10:246::23) To CY8PR12MB7705.namprd12.prod.outlook.com (2603:10b6:930:84::9) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DS0PR12MB7726:EE_|IA1PR12MB6305:EE_ X-MS-Office365-Filtering-Correlation-Id: ad6702c7-2253-454f-780e-08dd0a96d8a1 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|1800799024|366016|7416014|376014|7053199007; X-Microsoft-Antispam-Message-Info: AAzer2QAd2rCsPipO8VaHOtjoBkhfWVCEAJeqKdWioOKNMg9GfbgNwvi+Y2lKGX5bfcN381QeVg6wRE7ULK/ArbwzEi+e/WZ7OxtjTkr9qxovFvmZWbtxv8pNyt0rqtfz2VmNxey67D5/C0M8N5tXYfwtAzTmqQEYzs4A1oIczpusZ9qwV0kZsuQlwPCmogUrrUEFudCimM7ZTb8gXw7Rulsd7IG8I8wD4mnrbyIw44yPy5vHH/oWBbOnvmMjpDpRRYxBNNb3xjjmiu8RjDk0+h3ANRjoUFzfv8Cs7XpyvC0aIBEn1etYz3qjGT20YQ8r4tyf7KBtYDIiYzE4/YvApUHG/WQDOVqbxSL/xEWxWXafrNBrBNolQkX3G9VCdxtzL4LsoYO02RLGuOngL73JYjVe2lyFogkM8XOvrT0DVM6sZFVsW1Mm4+G3kq2Lzp3Af0iK59Ptoo7NRZponcv46lm1v+KP7pFpSyMbc6WYJ/m+5dmLfV/Wr3ruzAcRfDWgLXEN9PgYUsJiT/tdFzWspicfXpsj1fpp88sG1ed0oloVo3C1F8dZL6RhuUXCqq0Yw+ImKS8bi+f/b3d/x3SkT2d8c/+CJSuSB1KB6SPoDwIhMj/06iZjuK/T/FveUXTYz/1vDosNmQbMwXZzqCh4wdPSJqevthT88s6y2/GpIEZSibDATe0eIi5S3P3WoV6ocKxzjy5ExMxatCYuLcfX5SLbUvrYrNK9dOJgx+C+3ZU/UGpF6Psld4asEU3zcKFBuBRBrqtdpSQgdXhI1TY8hQ2UPaUG2Lv3SHVvRs7lD0sb8Ae81mQS9E4Mct+LprvswsM1jF5M9dFZwIYzrrgSWLbXrU/8BHUzV3ZSGcZMy/SerUsPa8P9vQZE5ZZuePMeSDh5kH+2ycZ+RjoKqtH3qQb90s0Z63TY+uRg6abVHIJsvi8eXqN2tP/v3drDAjAqkf/v3jYW4fLInbDQCqn542yxmVRmXpLnySldJ+TuFnYcbQdQNpHOvGhPBTAUFrgxNWHiMcMZyu8hxXcdqCNNPgPRdGDlv/daNstUyS2Rx6BvkUqyBRAljogoZIjpCakBd89TGkY42dgqHeZeFTZgCTfltGrvM5UJKFLOBRv6JNC7hxLjExKbX4CgCe+JPF3jzVwHVzvKjgjv7V8fYWD0kCQAyvjda1ZtD1rBjH0zOBWz5JEyWqEGfpr0BX6+IEep/aBdI3I6O4CkgBMRc1puGSnst+kLcflnBsZD2PY/VCWEHX4JwukQxBdpg0NSnHnLdZ640cLiy7+T1OSnbwU2U80rwvxITmb0wHRe2euFkI= X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:DS0PR12MB7726.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230040)(1800799024)(366016)(7416014)(376014)(7053199007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: ovEzYvTYrcZpUogth9ZSNVD7lRTF9lR3Nlbfaev1FSK3PsvDX3qpVUs/2VzImFev4QyEZmodhJD3qUc/Gvkghfp733lj8kdPSn47fIFgwLWo0AFjf2rmfCGsA+anCbm4VFLWylQeVJbpQV2hHgwib7X2INBQ1DlQD7C3OVDxKN4pvZELU5Pn/y0APV9x2ux1zAc/M7dlsT0/wpkB5RK0Bm3VSZpzFduYcWOZI8/SC2qlZMQ3tCjQ41UzGCYLYuHMvx9YRfkgvCjJSQY+TlnQMG2AqHeDI1uLcm3gHJ9WNSbuoTEl5Co5XsU7jHfOTnOXFz4qXR92dzSzo2qq6+kmji2dz3e2QGgsRCzlWzeEEp12WW6/7EkjqiAjlVq/A6zn7C2+LdWeQrNpK3KWNIZtuSWdP9P1+jtltlXYCyaCxvkZoNJ8laob6bxzvKnjCumtb4Zq0wy6H3W/25W6jRFS1SRK4z0qP5dbhyIy5O5wHfcsuP7F22G4Utr73dsJL8hSZUl5HtDb0V1wPX1X23RRaqyQW/8SLSTphKP58awxdtntLHfL4FHIgLxF4BOoivslIKON6vZ+axURmRhfBolN+xfDFI0EOcKMy5HLgca9kXbD70eZgEcEa8Dm5Pqex8V14nnYIAGj1jj+fcUo3J6LxPi3QB6BeW9qWS3fBWkTlQwLvI8X70ttM3qAppE/ZAmReFuaonwt060dxNkiiqRDVRkYkJigTihym0JHsdFTcfvlZ9hfzCRgX9dgyWYkAew0QLyHThmFn5q6arX2bJ8u4eQytjDr+q1AnIfW2/27dyI2jSocq24pTk7zgSEWzCqmWNsIaGpDIsQbfAXEwapbKEDhop+p24YQt1pgMTmH7nGswjcTUQbrvwo3srtMLkorBE5GXYWYRWPRwvG/aPowC8eY5xoFKYjPVVzo+gsgOiaUQ/HPNmESP/zbQBy23LnvdHEOweIE9ayzZf6Aa9WeM+6EXN4382Qx7zzMiCY/OOOHgpXiYMbFsozbvvxK0JkEJAxuZglZQWPA6ecozgIDWH9H/1xBrrqohFv9YfI0FbHkXXyVSn85MAphLALzO0KL6A+1SUefiDWWsEtLWel3Ps1YScOXJAUTWC+V+AbXqL7A/lxwqNZle+Y35rFcz83z1w8qx3lp4j9BvrRahfy5WJSBjw3hSCb5/+DTvU+G5vtmj79yA4jmoQTYaPTTA83YL4id+4p1Ejs37Vn2ebkK9FDC/83rieZXvLqnW8VjT85JSt7HJ/Z77bfO5/EjqR+pSoWuDeRXO/NxAdANZZkN0vV6UIT07vGTuomAtZDbaXXDP62AhbxNzDAOymE0c94sQCOrvx/sPSPgZdZeK6ZxdbXfztmxR4kw9v4sJ0jA5Q7GRQD6D/csHTveYtKyxtk1yp3sFFTpTMRkoJrVauVXsxOW4S0IrWcvryMEgCuroAzLaDphMVfi7EOJISm4ErjjoorSbnk9AsJV0lPj7vC7TtsDgKFsqRl7Jo2meINTQEwlqBBVZKnFOXArZQqV0s0gd/qopVO93xE0hA1DzDdSwMDexNg0fR+jw1J4+dxFUkjYysBSl4a5mbPyGUtRboEq X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: ad6702c7-2253-454f-780e-08dd0a96d8a1 X-MS-Exchange-CrossTenant-AuthSource: CY8PR12MB7705.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Nov 2024 01:41:55.7004 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: K7S7vTYjSjo4WhSh/8ji4NUilaH21lz8ZDp+4PyOyCA7vOI7JQlsNI1b6H5L9+3wv6DZi88+bPQLF8Lmin/F0Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6305 X-Rspamd-Queue-Id: 03B7214000B X-Stat-Signature: esz6jhbysj4rp6xago65bha3kzx3bypp X-Rspam-User: X-Rspamd-Server: rspam05 X-HE-Tag: 1732239789-460219 X-HE-Meta: U2FsdGVkX19XGdmGiPWPceIe3GLO3/5jrKN7YFv2Drff+xnIB137OFsQjtouHL2wB9x4GrGi0mvOgsxXQ2JAJpGpPZAqMFUl+Uezs6uNMqe42mvQx8xqYxp3hozyThMjUFApbThol4DtQvyG0pwUPcsxFaJmHW+N6YpWWgyNitDahcJVLIU11WBx+vM6UIw+vFXiKqKz9lrMpGZ30tu1AQFbd+cgz2AAxt0IPfgNACwoFgAZE/hopFxp7iIhoLtWJ96hlDkQBfJ8bOzCkpMp1YA14VivALU4eKjeNIfCvKtJyRdNA+yAe4u/THucVJ6GcCbnjNJ9E3MDU7HLsyeaGrWGhH4h+tM0g4oKM/wDZbciduT0cwVXa1z+uHPa8l9yk7aG+tfoz2sIeTOaD7TyJCZOcBX3yzssxG0qEDzt/QEzJCUJntUND3SRFm16c9OPrHjwgSp/Bv3k/gQjEP0XzPsRgTW16orIgZQjoZLr62KsiugDh5dOx3VJ7Q0qrQ2ngEBl2Uui0NBoRIW/m6m4N1KFmZW6mJNRMwzercWtASMq147y2/4AvIVGkeC/zAm6c/8DVfqL7tcqg86xmCiDIvfykODKf40ZhHncB9WRdKHAaXTFx8Tn3jj+J0QI9ONmufN97jMJro8smvDLlIESF6dnCvLuQeOd5AF5avMtrBl7r4lvKF4XA8d6Z6vyrGnW6r6AFWwfpJvklNFEFbvMN1wzVGTY8XidMZ1ya/xWirHxsABQwyu3CISMS65dGj7Sx7LBbmvyEcDQtxcjFtFeEPynFu3e0BRFJN5iSuFTWXYnkUJQMESJAi5sjtGrpKNkuNS8kZOI+z5pfZy4fTD4sBrksf99o0t+bmWq+VfP9hjbkLzHYoecmmhNkfiLDuk/f5DrgVjZpOKM8cUkhcTpSPsA3H6Pvnl185LI49LK7oxABxjkyBJNOF9nNPdIPF3SJrlUTythKFX8if3okQX 7yTK9ZIY 60TJPapL1Do78/dKlAt7p7ImP8lM9MMmoYY1bQRo6nx1SAB5gYdgYpduL4NEw6HulIAbZvbmszAAtjVkuv8gy4/+MXyM5j+4jIyTPibEKd+pGMiXLl5uDTJ+kfj6bgAxMfbnkAFFhKvj70WTUy/yKsPqDEp540/ncppgRi85GdHtxcN+wSxcH5hUrsc9ZK3sEoxORzZ95ucN/Gqp2s+Q4gQpigHqx9b5kwiDMSqL/7KfACJ7JQ+PqN+NNmoIz/E7WMHsGpsk+i2N5IltDCGl7TjVhNEafPTSjHnphnKVf2RVr1gg+vl6bb+VNOnzb1ZPkqd5c7A0hdbgu8nI4j98GGzOpd5gaHxoN2D/vFsSRNKl5LOjsW/ByhuhBI6+jy304aMtR5GBBf5VrGTNd0nanWvJjw31HeAOMZrA8w2XmM6RdsqbxOPxQeArucQ== X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: List-Subscribe: List-Unsubscribe: The reference counts for ZONE_DEVICE private pages should be initialised by the driver when the page is actually allocated by the driver allocator, not when they are first created. This is currently the case for MEMORY_DEVICE_PRIVATE and MEMORY_DEVICE_COHERENT pages but not MEMORY_DEVICE_PCI_P2PDMA pages so fix that up. Signed-off-by: Alistair Popple Reviewed-by: Dan Williams --- Changes since v2: - Initialise the page refcount for all pages covered by the kaddr --- drivers/pci/p2pdma.c | 13 +++++++++++-- mm/memremap.c | 17 +++++++++++++---- mm/mm_init.c | 22 ++++++++++++++++++---- 3 files changed, 42 insertions(+), 10 deletions(-) diff --git a/drivers/pci/p2pdma.c b/drivers/pci/p2pdma.c index 4f47a13..2c5ac4a 100644 --- a/drivers/pci/p2pdma.c +++ b/drivers/pci/p2pdma.c @@ -140,13 +140,22 @@ static int p2pmem_alloc_mmap(struct file *filp, struct kobject *kobj, rcu_read_unlock(); for (vaddr = vma->vm_start; vaddr < vma->vm_end; vaddr += PAGE_SIZE) { - ret = vm_insert_page(vma, vaddr, virt_to_page(kaddr)); + struct page *page = virt_to_page(kaddr); + + /* + * Initialise the refcount for the freshly allocated page. As + * we have just allocated the page no one else should be + * using it. + */ + VM_WARN_ON_ONCE_PAGE(!page_ref_count(page), page); + set_page_count(page, 1); + ret = vm_insert_page(vma, vaddr, page); if (ret) { gen_pool_free(p2pdma->pool, (uintptr_t)kaddr, len); return ret; } percpu_ref_get(ref); - put_page(virt_to_page(kaddr)); + put_page(page); kaddr += PAGE_SIZE; len -= PAGE_SIZE; } diff --git a/mm/memremap.c b/mm/memremap.c index 40d4547..07bbe0e 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -488,15 +488,24 @@ void free_zone_device_folio(struct folio *folio) folio->mapping = NULL; folio->page.pgmap->ops->page_free(folio_page(folio, 0)); - if (folio->page.pgmap->type != MEMORY_DEVICE_PRIVATE && - folio->page.pgmap->type != MEMORY_DEVICE_COHERENT) + switch (folio->page.pgmap->type) { + case MEMORY_DEVICE_PRIVATE: + case MEMORY_DEVICE_COHERENT: + put_dev_pagemap(folio->page.pgmap); + break; + + case MEMORY_DEVICE_FS_DAX: + case MEMORY_DEVICE_GENERIC: /* * Reset the refcount to 1 to prepare for handing out the page * again. */ folio_set_count(folio, 1); - else - put_dev_pagemap(folio->page.pgmap); + break; + + case MEMORY_DEVICE_PCI_P2PDMA: + break; + } } void zone_device_page_init(struct page *page) diff --git a/mm/mm_init.c b/mm/mm_init.c index 4ba5607..0489820 100644 --- a/mm/mm_init.c +++ b/mm/mm_init.c @@ -1015,12 +1015,26 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, } /* - * ZONE_DEVICE pages are released directly to the driver page allocator - * which will set the page count to 1 when allocating the page. + * ZONE_DEVICE pages other than MEMORY_TYPE_GENERIC and + * MEMORY_TYPE_FS_DAX pages are released directly to the driver page + * allocator which will set the page count to 1 when allocating the + * page. + * + * MEMORY_TYPE_GENERIC and MEMORY_TYPE_FS_DAX pages automatically have + * their refcount reset to one whenever they are freed (ie. after + * their refcount drops to 0). */ - if (pgmap->type == MEMORY_DEVICE_PRIVATE || - pgmap->type == MEMORY_DEVICE_COHERENT) + switch (pgmap->type) { + case MEMORY_DEVICE_PRIVATE: + case MEMORY_DEVICE_COHERENT: + case MEMORY_DEVICE_PCI_P2PDMA: set_page_count(page, 0); + break; + + case MEMORY_DEVICE_FS_DAX: + case MEMORY_DEVICE_GENERIC: + break; + } } /*