From patchwork Mon Sep 26 06:03:06 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alistair Popple X-Patchwork-Id: 12988232 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by smtp.lore.kernel.org (Postfix) with ESMTP id B8479C6FA82 for ; Mon, 26 Sep 2022 06:04:21 +0000 (UTC) Received: by kanga.kvack.org (Postfix) id 51182900009; Mon, 26 Sep 2022 02:04:21 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 4BFFB900002; Mon, 26 Sep 2022 02:04:21 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 33A65900009; Mon, 26 Sep 2022 02:04:21 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from relay.hostedemail.com (smtprelay0011.hostedemail.com [216.40.44.11]) by kanga.kvack.org (Postfix) with ESMTP id 24C29900002 for ; Mon, 26 Sep 2022 02:04:21 -0400 (EDT) Received: from smtpin04.hostedemail.com (a10.router.float.18 [10.200.18.1]) by unirelay04.hostedemail.com (Postfix) with ESMTP id F140D1A0697 for ; Mon, 26 Sep 2022 06:04:20 +0000 (UTC) X-FDA: 79953196680.04.740AFE3 Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2073.outbound.protection.outlook.com [40.107.92.73]) by imf07.hostedemail.com (Postfix) with ESMTP id 79EB840005 for ; Mon, 26 Sep 2022 06:04:20 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Oca6pW+6+KtG/XgCOK0VrJvZ6IhDf/Nyfx/5a8Mmk63oyuThRZP1J+mGiuqYAk/5RhG8aUYgwFYjJ376bIKV/2RaLYxMMGYEWTPzSC6qslwWKWR7h0f8cv30ahReMc3jE4pv+G3oP58CGVRJ3rCIylPVkGKAWSp/nyQUuZRiAtjrwTwIhw0g7Xt8dMyKJZLljNgS8KxMV51bgn6aPuG0Kj/bBLVPlUUmGNYBkO9VR/6e65z4Yen8SiOZBRL9sMdAMfryQ//cDjjuSHx+NC4sUcxtk3Q9TmFRFz8AAqqzJEcNclncMirxISFUSx7EMvaFjUj5nBZlUxGSJC7mUpinaA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=lpUR8gcIPaAR7nD1dKeC+83we5Ll0QdDdKicRyU/2VU=; b=Fhop26GLvxmPg1GIiy6s7+0gK9ChzQqUbXlK7SR5s2IauiEC666hVU2OP0uDZBpyaVWAHBFoimP/Nv5pet9OUaXm4DDYjXihmmjuXSSNLB213PT6urGg0OTuxNSxHXdAY3ETSEQuUms93krHl4W5FlO6GrURiZ+04Y8VM+l2fA1kLGuVgCb1Q531JoKWQrUzpB9j6YMZhSFpko1EPplTZNP3OoW9QsmqlDxcmLJmMotrCK+FGVPX/Zs+CZQuOZ0xs9/t+heTiHGIN4tA/P7hxhDeAlC1IOIp085qhvnYundWbWPSUt5TL5yre60mjigEPobtaBQS5AQrnzaDJWKCNA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=nvidia.com; dmarc=pass action=none header.from=nvidia.com; dkim=pass header.d=nvidia.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=lpUR8gcIPaAR7nD1dKeC+83we5Ll0QdDdKicRyU/2VU=; b=PMHcNX1AcNm68fswb4wqfJoo6IGL+BWzkRMjjMdIa7gj5tXlvdnuBp3XWC1q/p3cvsfuh/l9f4VPo/p2oT3uDedlJ8xugwlEQxBDlanEdSBVw9C4fZZJp9CWAQbmutX53+sg+LmJoHvH2SMSS/qYUBA7aYsZ10kDa+ur0Gr5kFDoxnteawoWHIuIap6SNaJM93GHXoGq+1W3iShdwMXA3WQtxMLA+8NlO5t77uWKM1i6SKlG5plvh2RF/K4txTbw7G11atKzhaTBVPIqdQqYUcXfqEkThG4ASMY/gKvA03kn9ZpqrC0inHhn+AnWhoE0nljFeOW1fZNpIpo9bI3QdA== Received: from BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) by MW4PR12MB7359.namprd12.prod.outlook.com (2603:10b6:303:219::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5654.25; Mon, 26 Sep 2022 06:04:18 +0000 Received: from BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::ed0:a520:ac8e:9966]) by BYAPR12MB3176.namprd12.prod.outlook.com ([fe80::ed0:a520:ac8e:9966%6]) with mapi id 15.20.5654.025; Mon, 26 Sep 2022 06:04:18 +0000 From: Alistair Popple To: linux-mm@kvack.org, Andrew Morton Cc: Michael Ellerman , Nicholas Piggin , Felix Kuehling , Alex Deucher , =?utf-8?q?Christian_K=C3=B6nig?= , "Pan, Xinhui" , David Airlie , Daniel Vetter , Ben Skeggs , Karol Herbst , Lyude Paul , Ralph Campbell , "Matthew Wilcox (Oracle)" , Alex Sierra , John Hubbard , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org, amd-gfx@lists.freedesktop.org, nouveau@lists.freedesktop.org, dri-devel@lists.freedesktop.org, Jason Gunthorpe , Dan Williams , Alistair Popple Subject: [PATCH 2/7] mm: Free device private pages have zero refcount Date: Mon, 26 Sep 2022 16:03:06 +1000 Message-Id: <3d74bb439723c7e46cbe47d1711795308aee4ae3.1664171943.git-series.apopple@nvidia.com> X-Mailer: git-send-email 2.35.1 In-Reply-To: References: X-ClientProxiedBy: SY0PR01CA0006.ausprd01.prod.outlook.com (2603:10c6:10:1bb::22) To BYAPR12MB3176.namprd12.prod.outlook.com (2603:10b6:a03:134::26) MIME-Version: 1.0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: BYAPR12MB3176:EE_|MW4PR12MB7359:EE_ X-MS-Office365-Filtering-Correlation-Id: 42dc79fb-aca4-4410-816b-08da9f84f2bc X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Sg8pk3ObyJPpGQXkXCrSAZrlI2JfuD7xYLEcvcWnSqNBC5l/zr/eIlMA3v5g6cNqFpPIBZk6le8SP+1o/FzBAL4i298q7XU4Kp9UvfXgJAAazaL5Y0LgRpKIT6vGoMrUJ6k83kfQiAfhQLcFQIZ3fdeKxeX9YB1BtbvXFM5Lm02EmMDaNfd/sWVTogWERr+z3BIjIbdaBtCYJP7TkUY/ejuUd3E9huIAFtn8wqeP5VL0egDMcAThpaErJjkr22/RjMXX0C9W298V96AgEpdLVPSQhHN4J6sAs5tjbO1olnfHB4XS7KLZHzA90D8yqDbkZS0RAwW9M1m8+03MajgeToulqBnox/cdDpW4BKDlRInUqrswG9FkTGJS8KYn+lKcHGpzU5FmLnSMOR+690JQiMJM0XDAe6x/8Il11R+32tILcxb+Ixsx5235bOE5DVZDhEhq1sGsAHejmdLZvVLsNyZzDGt37RBfyI34D0qol+lPRmRUC64DXhPG9thIZglx7UWSbk2kRXXYePRu1B0OEPvF7+a2v0n9VsYg84I48jx6H7oSJ6ahBzRofXCsytfXeDWYF3z1INZJJsLOvVBrlNvOmfzzNKfaP0YoYH/j0tmhBprpZi1PJwxHo3H7G1oF8ssOicZY4CBSYRlzlq/MNDSFawTCgGpnaDv6EwDnyMs/hixWEyIXMpuV0vZY9/X/43jPLnYlfV1eXqgcMT1isUQP8//m/8to+Mwhjhyy22+23EH3Sl1JLVgwrZkP6cio X-Forefront-Antispam-Report: CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:BYAPR12MB3176.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230022)(4636009)(39860400002)(136003)(366004)(376002)(396003)(346002)(451199015)(8936002)(86362001)(7416002)(2906002)(5660300002)(83380400001)(6666004)(41300700001)(6512007)(186003)(107886003)(26005)(6506007)(478600001)(6486002)(2616005)(6916009)(66946007)(4326008)(66556008)(66476007)(8676002)(316002)(38100700002)(54906003)(36756003);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: VzBdNO6pyTug0f53OiDET+Rk1JMLTpnAA29eaaVmvQqdoMusNTsQgP71Jb0gK/+grlWM0NnjIFLOFpmEmc0nXC4M/aINMh1iXnNqtZ0fkl/vQJ/CLZNEZGHOcEcy9lJBqmAgQQm0guSUiTuImpDL+dIBBDc8dlGTSWGYE3nDF/Drh0AeTe5Qa6o6zJW6/xatwj20z9g85cAJ2DD3ircz5HkNCT1o3PNRkSUnsan0vWLhQw0OR/lDyzlMOHcPvzmoh2LVozQGRYkvo7POPIa2S3+f6C7ebmDsq1zMtBd1Z4vXZXm4t3PIU5UFw2Ns6AaLDlWZ2mEaK8aQrJBuumLuMr7PIDNkiYLsYYt76aGTES3rxfvsd9QXYKBEHieKzlhXReoe6LGcd0Mw/qRqBPS6znuWKhsD05aKvEkjgNtkS9MlN4ESF1xL7gTo3BBx/M+I40r1GKmoR1yQbMcE+PWXYPpVlqdbDLvkpJSHPYq+G+sA78T4ZprIdTMcgxLjXEo9zjPaDt/RWAMg3uJf5bgBgu8Rp9j3DibaOsmWrSGFZHjpwPsX7KLVl/TXTrDXuiPZW4dHEnRTxQaajUj6GjxQQjbid3LX0j7rFQx9PwaiKf4DGREnvr9R+1DSA1rAwpg7HPdX8Gd+biWDzkKWlKsQrYFh4/638O2jWI46Z2UzpmNNmeHdJZnTNQuavvjjpmtToPeXRXfVQwJOI2eNzXJiefEn4EmmW/BNPYcGevdhJlV7nYQKQ9lOgXxcdPgEc44rMDdgoHgYbM1PSmDvJTRhTVIZVXzRnY1n1SkamRiVH7jI5ZgmylqCR/cykiucHuidM57GmBDS09O8a+0YM6GTeac8VLW5rbooo3hJ7EI/9au/nZ3FpwU5MPYP1K2Pe9/fMgJxoXWZAphp7GrUgUwqn6kHuh3FfuHqquWQIAGSsZx9NMCyOQr4LlgOZC//c4+gwVhx3Rn9cjs4m8kz2x27JPlXUYY4sQ+BqhboYBA2GQ/vxuqJ9gTIYDzsO/4KYkmdgxwPl8N9Mxbb3elxgMsLayt431bNB8gyE7rbB40co4xGZYItMcSsiG2Y3u30DTGocWl1mdlS793oXhWG1VADdqnljn0j05VoIpIi4OlxUz86sphVVDDedQml6IyQTILLDJQrpQdEIyASkcRVVkQIhMhbTAC6Og7O9AhV9ICIMLh4cHSUJ+GuFxqStPeZjJjmD3cmiTNjXmsKDTxYEeomxSsqwYlueEA1Ps3YO0Q6fzhD2t2MLePgr4PprgH4lNHigstCWuEtH7L3DY8yq4VDEMn/f31wpTiIjTjFG/M3TtpW2wmLKFXlx1BD68gk+s0ihbZhs4xgrskwjxsCPVZgBoD2q+5Hz0Bok9/mJLKLpLsVcXzRGsqJ+LFpqPMd6EApW8sjmOKeFalYPA7RFvDowp2eFK9CvBqVr6L5bY+8sD4+/M6BDFvi8k83r5LoPowU95+td4ia1QdCG1tRUy8zTtVIeUwzyDGuU6yp2Sebi+iAFsld31UWJNhkTdpoLhrKLHKxneFtWKLxbpAO4s7hg6AJz7V9MJ3DA7Bl8M57P0Zr4gN9onksrqXmZZoAer1F X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-Network-Message-Id: 42dc79fb-aca4-4410-816b-08da9f84f2bc X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 06:04:18.5160 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: pSJfaFl+AoP2iXV+KdZPR0OQ/OuPImsGV0AlcHXWV5stObXND6Q/pax0LvM2JKiC8tBfzane1jtZP79gVwCvAw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB7359 ARC-Authentication-Results: i=2; imf07.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=PMHcNX1A; spf=pass (imf07.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.73 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") ARC-Seal: i=2; s=arc-20220608; d=hostedemail.com; t=1664172260; a=rsa-sha256; cv=pass; b=4T30QNFkN3KweXIXlcI2c/Plw/c6CxX5lxkOq442owvBlakZUwUNHDJ2l7sV5kkozqTW7z TSI6ny1RzNcOsa2A04sWp+3PO5zm8+sL8MiWk1fpKb0FkWf+hpL2p1ZnKMWz0g8E5i1ntz d4GPIRX/iQxay+R/tyU32cSzKsd6Gx4= ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=hostedemail.com; s=arc-20220608; t=1664172260; h=from:from:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:dkim-signature; bh=lpUR8gcIPaAR7nD1dKeC+83we5Ll0QdDdKicRyU/2VU=; b=ry7dHHMZ6lXCvE/3yJwFQsZ/T7R++JaQSMlJEKnuFJLSc/wccwysJjD73D6ykRwy4SMxOh tL5lEBcpPdv6imUJKHNYAzo+y3iklBIGjUtENlx6UV1K/ZDEoaT5c4A4+ee2J2q+oA9bqK /fW6EB6AfmCZ4SgrkPP70gjR4gvDI4M= X-Rspamd-Server: rspam04 X-Rspamd-Queue-Id: 79EB840005 Authentication-Results: imf07.hostedemail.com; dkim=pass header.d=Nvidia.com header.s=selector2 header.b=PMHcNX1A; spf=pass (imf07.hostedemail.com: domain of apopple@nvidia.com designates 40.107.92.73 as permitted sender) smtp.mailfrom=apopple@nvidia.com; dmarc=pass (policy=reject) header.from=nvidia.com; arc=pass ("microsoft.com:s=arcselector9901:i=1") X-Stat-Signature: ckpjodfit7x6zmu75egsaegrmb7wgex9 X-Rspam-User: X-HE-Tag: 1664172260-936158 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: Since 27674ef6c73f ("mm: remove the extra ZONE_DEVICE struct page refcount") device private pages have no longer had an extra reference count when the page is in use. However before handing them back to the owning device driver we add an extra reference count such that free pages have a reference count of one. This makes it difficult to tell if a page is free or not because both free and in use pages will have a non-zero refcount. Instead we should return pages to the drivers page allocator with a zero reference count. Kernel code can then safely use kernel functions such as get_page_unless_zero(). Signed-off-by: Alistair Popple --- arch/powerpc/kvm/book3s_hv_uvmem.c | 1 + drivers/gpu/drm/amd/amdkfd/kfd_migrate.c | 1 + drivers/gpu/drm/nouveau/nouveau_dmem.c | 1 + lib/test_hmm.c | 1 + mm/memremap.c | 5 ----- mm/page_alloc.c | 6 ++++++ 6 files changed, 10 insertions(+), 5 deletions(-) diff --git a/arch/powerpc/kvm/book3s_hv_uvmem.c b/arch/powerpc/kvm/book3s_hv_uvmem.c index d4eacf4..08d2f7d 100644 --- a/arch/powerpc/kvm/book3s_hv_uvmem.c +++ b/arch/powerpc/kvm/book3s_hv_uvmem.c @@ -718,6 +718,7 @@ static struct page *kvmppc_uvmem_get_page(unsigned long gpa, struct kvm *kvm) dpage = pfn_to_page(uvmem_pfn); dpage->zone_device_data = pvt; + set_page_count(dpage, 1); lock_page(dpage); return dpage; out_clear: diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c index 776448b..05c2f4d 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_migrate.c @@ -223,6 +223,7 @@ svm_migrate_get_vram_page(struct svm_range *prange, unsigned long pfn) page = pfn_to_page(pfn); svm_range_bo_ref(prange->svm_bo); page->zone_device_data = prange->svm_bo; + set_page_count(page, 1); lock_page(page); } diff --git a/drivers/gpu/drm/nouveau/nouveau_dmem.c b/drivers/gpu/drm/nouveau/nouveau_dmem.c index 1635661..f9234ed 100644 --- a/drivers/gpu/drm/nouveau/nouveau_dmem.c +++ b/drivers/gpu/drm/nouveau/nouveau_dmem.c @@ -326,6 +326,7 @@ nouveau_dmem_page_alloc_locked(struct nouveau_drm *drm) return NULL; } + set_page_count(page, 1); lock_page(page); return page; } diff --git a/lib/test_hmm.c b/lib/test_hmm.c index 89463ff..2bd3a67 100644 --- a/lib/test_hmm.c +++ b/lib/test_hmm.c @@ -627,6 +627,7 @@ static struct page *dmirror_devmem_alloc_page(struct dmirror_device *mdevice) goto error; } + set_page_count(dpage, 1); dpage->zone_device_data = rpage; lock_page(dpage); return dpage; diff --git a/mm/memremap.c b/mm/memremap.c index 25029a4..e065171 100644 --- a/mm/memremap.c +++ b/mm/memremap.c @@ -501,11 +501,6 @@ void free_zone_device_page(struct page *page) */ page->mapping = NULL; page->pgmap->ops->page_free(page); - - /* - * Reset the page count to 1 to prepare for handing out the page again. - */ - set_page_count(page, 1); } #ifdef CONFIG_FS_DAX diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 9d49803..67eaab5 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -6744,6 +6744,12 @@ static void __ref __init_zone_device_page(struct page *page, unsigned long pfn, set_pageblock_migratetype(page, MIGRATE_MOVABLE); cond_resched(); } + + /* + * ZONE_DEVICE pages are released directly to the driver page allocator + * which will set the page count to 1 when allocating the page. + */ + set_page_count(page, 0); } /*