From patchwork Fri Nov 18 23:44:17 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13049179 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4F5E6C433FE for ; Fri, 18 Nov 2022 23:45:02 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F1ABE10E823; Fri, 18 Nov 2022 23:44:56 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2050.outbound.protection.outlook.com [40.107.92.50]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9632010E81A; Fri, 18 Nov 2022 23:44:38 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UPW9MQqzvMuZzcAP313Ih/hFvTX2beQOR4zgsWfJbhsQBZPxSk1D8Ljj+XWGChuXLXxXIgv7z3YkuIJ9lSVCW1k/wKh39JsWOMUPXMuHgfCpuInYh/BYm15dVqjmC86xLqUOrHsWNnCU6Y88a/i7ySK0a9iIbDLVC7a8CNNKUM56VdRe/fcnsoRv77HiBDgkNd47sau3P0XwzehWuQXJw8iMKfLoF69xxWnOyay5H3mFni9+v90Zj3iyhG4f4cKAyDEUS+QzPyZ4slRd/1NqJjU0ZmXKknMg4Eeo3HUl4MEEfOUZv5Mja+HQhAayhfUy8vdrMyfXZuqc+h0Ey3M3bA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ftdrtYZn0RwkgwVYpdgwvwzLQ4M3gdgnmZ2cvtwtYWA=; b=M7pF36XLoIEMHiPbqDjyw8z0S0rLRt9N5TPwUusS8sjYLDjl5oAoaDvvHuSWhKTEx7446DaUzkr0B3CksuRez6xuZEdA2V5aRgrCgDdcfqYriSEnGfVH35B2hOy1zqkS7r9ybwSVpe7DnxuqCewCtSMt+11zcBPl0sNn1XmksDZlcmrvIGulRF97U0d0pfVvzxK8+FRYoduZr2f/6yBbQ6DoHhUkqdn0k4BdpGcWb8aGULbiFzqkvC6qfovk2IVgK+4/g2qwvstylR/ErtCNJdnNiIEr4eYshKz4G6QFNQQtTKIMsn9H3I0DDDYVJA++fCA0OkDDJYWIQsE8CngKvQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ftdrtYZn0RwkgwVYpdgwvwzLQ4M3gdgnmZ2cvtwtYWA=; b=Vda03ikUtFRALKNOGbhfPssMeZZDHaKjpD14bhC7rOnH+JLumOI56/rDnwMg3tvnsR3YT6qHYUZU2IyANnwMEdM+sikwFQ18M9u325tW/lNEhIKp4mERq1e2bwmnhlgnyNqJMNQTu7ENrFQFsUWS6cntTvaQ1C1KYV5ofYB3DG4= Received: from DM6PR03CA0056.namprd03.prod.outlook.com (2603:10b6:5:100::33) by DM8PR12MB5494.namprd12.prod.outlook.com (2603:10b6:8:24::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.20; Fri, 18 Nov 2022 23:44:36 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:5:100:cafe::7c) by DM6PR03CA0056.outlook.office365.com (2603:10b6:5:100::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9 via Frontend Transport; Fri, 18 Nov 2022 23:44:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 23:44:36 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 18 Nov 2022 17:44:35 -0600 From: Felix Kuehling To: , Subject: [PATCH 1/6] drm/amdgpu: Generalize KFD dmabuf import Date: Fri, 18 Nov 2022 18:44:17 -0500 Message-ID: <20221118234422.1468806-2-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20221118234422.1468806-1-Felix.Kuehling@amd.com> References: <20221118234422.1468806-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT014:EE_|DM8PR12MB5494:EE_ X-MS-Office365-Filtering-Correlation-Id: 17071b8e-2c09-488d-96c2-08dac9bed9d8 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Vxoba7F0CTgvq3XXErRnrAllSCsdK5btsAdtFBYsZQAG85LPG6brSfJrGixdMVpsVctI2e4m/6rkiIAIW8e6jIn/QNsGNUGfBnEYeITif/QCpUeDkNEQ9uPD2jMxt1fVeavOVPpmOjHKRthWtgEVz1I0EJfGWZqPgjkDluC/mQtrYUlzfyacHq7zB7SHDPsf/Ch5TcoMutgHDSOUnM/a3fbI/KsYBlfb6f2+NFuTAtC288Zr110d7p0QFwfTNuioVXoqKDBUO679t1YPMLz7uc5dhzhdjJgpyZzF1cISV9p5QR1+J1wTgkh3WIYEBQX38ebw3EXPfocPGCHv2UaPJkA0NQOpnWab8bqh7NH4rM5m+P9pnqdMmjP46y+9hg+tFDHl2h4EpSo2fYdGr+UBpbIYuGV9564vvBn4KQuFE70reITPHUOhaN8lUlidsZhtHwv0kzMQi0yPzJkrCpSYHEXR2gxdnWoXKIjWkpf5bPfSuV9r7BTee6kOvrXTPVBIqcnWeL6wRNz3gcvhcB/hVhbt4bBfnY8cRo42e/t5LsmE792X4laJ2SzuKOW6u6p2PgDv4m/fMibmI3SE10bLrjCeYXmcGJvM4HBs+TGgv5Bk262q2kJqY5ix+3zxZr0wZChlvDvhRF26ID7bVA6H8aWiocQpptgp4i92Wtfaqza0kPusL+47miXCQaGblqbb0c3Iu3nxiaKxkta8f4IrtTpiw/gpAz+3vKglIURvzHU= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(346002)(396003)(136003)(376002)(451199015)(40470700004)(36840700001)(46966006)(336012)(7696005)(1076003)(5660300002)(186003)(16526019)(110136005)(316002)(2616005)(40460700003)(26005)(450100002)(36756003)(8936002)(70586007)(70206006)(8676002)(36860700001)(82310400005)(81166007)(356005)(41300700001)(86362001)(82740400003)(83380400001)(2906002)(47076005)(426003)(40480700001)(6666004)(478600001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 23:44:36.1384 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 17071b8e-2c09-488d-96c2-08dac9bed9d8 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5494 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use proper amdgpu_gem_prime_import function to handle all kinds of imports. Remember the dmabuf reference to enable proper multi-GPU attachment to multiple VMs without erroneously re-exporting the underlying BO multiple times. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 38 ++++++++++--------- 1 file changed, 21 insertions(+), 17 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 3a763916a5a1..67c832ae498b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -2183,30 +2183,27 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, struct amdgpu_bo *bo; int ret; - if (dma_buf->ops != &amdgpu_dmabuf_ops) - /* Can't handle non-graphics buffers */ - return -EINVAL; - - obj = dma_buf->priv; - if (drm_to_adev(obj->dev) != adev) - /* Can't handle buffers from other devices */ - return -EINVAL; + obj = amdgpu_gem_prime_import(adev_to_drm(adev), dma_buf); + if (IS_ERR(obj)) + return PTR_ERR(obj); bo = gem_to_amdgpu_bo(obj); if (!(bo->preferred_domains & (AMDGPU_GEM_DOMAIN_VRAM | - AMDGPU_GEM_DOMAIN_GTT))) + AMDGPU_GEM_DOMAIN_GTT))) { /* Only VRAM and GTT BOs are supported */ - return -EINVAL; + ret = -EINVAL; + goto err_put_obj; + } *mem = kzalloc(sizeof(struct kgd_mem), GFP_KERNEL); - if (!*mem) - return -ENOMEM; + if (!*mem) { + ret = -ENOMEM; + goto err_put_obj; + } ret = drm_vma_node_allow(&obj->vma_node, drm_priv); - if (ret) { - kfree(mem); - return ret; - } + if (ret) + goto err_free_mem; if (size) *size = amdgpu_bo_size(bo); @@ -2223,7 +2220,8 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, | KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE | KFD_IOC_ALLOC_MEM_FLAGS_EXECUTABLE; - drm_gem_object_get(&bo->tbo.base); + get_dma_buf(dma_buf); + (*mem)->dmabuf = dma_buf; (*mem)->bo = bo; (*mem)->va = va; (*mem)->domain = (bo->preferred_domains & AMDGPU_GEM_DOMAIN_VRAM) ? @@ -2235,6 +2233,12 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, (*mem)->is_imported = true; return 0; + +err_free_mem: + kfree(mem); +err_put_obj: + drm_gem_object_put(obj); + return ret; } /* Evict a userptr BO by stopping the queues if necessary From patchwork Fri Nov 18 23:44:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13049178 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5200AC433FE for ; Fri, 18 Nov 2022 23:44:57 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0384110E822; Fri, 18 Nov 2022 23:44:54 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on20616.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eab::616]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2108A10E82C; Fri, 18 Nov 2022 23:44:39 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BA3Ns644VBI41Mh5i3uJp3hhcDvlOZ6B+6zmwy//F1/v3tFXdeBx+NOeuOXRsTgMYMaCwNXSiOKvZ9Uau/Hm3EH/6vjSPgpYMINkZIQf7S2ATWg4OKTU4jo+B7DQ1sdQJ2PPtRAoKQEvaft27aBpOiC9VGaxDI2o0Sb3Ls4gyPazk/DzGswhqRjN0ZoCkKi5ACb8L6n93qNMQBN/vx5ibMaysBq3RY+1DbkF403V5QWu7ZBdznzPuru+l9QWwmCysPTjIMCnf6zuQOCxU/nOnwDXzCqGVDjVbv42H5lQNnqC/8SR8SyzsopygWjccsdwF21sWbsHTVW668pTSU7xbg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=wWWHIQilUDJDQ+EKrfhQprbPRtRpjN4wkSwUYlBM6OY=; b=dWZb9pdjUnJllow92zdYA53sEDPYBs/27n3sW5BOVFSNDoGyJO6mGkZWcT+VL0+xEjI7VByxoMcrqYWQpEmMTf0pWa7X5OneyKjLiltzRMFnprprVTobCpl9eibPl8v8kUizCVJcWDZE1YjVAqz7+ug1AqAB0Q6I9tze0yed+u4PMo+LmOeLhJwn27ZWvom6CHsc4wrXLheYQgkDVAtv3ZVgjozDti1Cqya3dqzyU9XR3Jczcd0CAnlOxqZ6s+A4GOsxwwi2aEtMEKhO7MAVdsmR7R58dmYxgWksq146qyLnsq1yIJbPRF3LF65e7tnIHcgJZndTaC71qKC3b5Jsvw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=wWWHIQilUDJDQ+EKrfhQprbPRtRpjN4wkSwUYlBM6OY=; b=e9zD8NTIoe81zYQuDbofKLrmBVuGesi/WiRTeF+ET1G2jUryMsYjkGwl/wS8JgSXnF2G8bK+V35JpEqHe0XYuAJGda5R+HlrvYt7ZfJlwfWBrzWO3sC3gbQ5HAKOqZ5oBK41eNIJHU3EuqXif0aimrSEyebHHBFKrYhemPVUQnM= Received: from DM6PR03CA0045.namprd03.prod.outlook.com (2603:10b6:5:100::22) by CY5PR12MB6034.namprd12.prod.outlook.com (2603:10b6:930:2e::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.17; Fri, 18 Nov 2022 23:44:36 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:5:100:cafe::6d) by DM6PR03CA0045.outlook.office365.com (2603:10b6:5:100::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9 via Frontend Transport; Fri, 18 Nov 2022 23:44:36 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 23:44:36 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 18 Nov 2022 17:44:35 -0600 From: Felix Kuehling To: , Subject: [PATCH 2/6] drm/amdkfd: Implement DMA buf fd export from KFD Date: Fri, 18 Nov 2022 18:44:18 -0500 Message-ID: <20221118234422.1468806-3-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20221118234422.1468806-1-Felix.Kuehling@amd.com> References: <20221118234422.1468806-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT014:EE_|CY5PR12MB6034:EE_ X-MS-Office365-Filtering-Correlation-Id: 751a17fa-8448-4ff6-806d-08dac9beda03 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QeGPWyDMLkHBazVRZzggzVAwc1hBkGttTD1GLIaI338q3t4rbayO08JOfrXi+qDpSUzHhBRwMnMuo2CaagwMQaCAZ4uVzrM1nIlYOfa9E2JUs7T/zv1czg7xmzwig+EBbQby3LhjOiJogF/jRNu7jKKbJuV4XTPt4mPmY6e9TEjmDs0QK3gZ6UoUIYuqsQ0FO/zJhwmysZJyYaNcKcfsphfbIlS0RfejSDorKc4Ut25cMdiMQJyApBn4Trw9ejfuPV8UQ4vGMmjIlI7cDlHGzipKZavjNq7GxplTArvkjfbJi2puwyP1r13ZdI9qGujutSJm8t3Gko0eLTlL3ZZmpxvr6rMnbJLXIkCrS/MQfOEENA3VvRTC6AayFCm847X3p1oC6czGR46+Y5coTlaYj/d8xFqNcc9+nnphT1LNe2ANYQRw979LlxytWx4K4LIqeG6f1hl8MXKOpFZU3gcoagPwsHSD4lgR5YwzmlvfKE214Cb1ttYMRzu1dkI9+pFrGTWeqJCjbUzLKLvTjKB6A4GP3Orl5/HgO56VNdpaDldKmqKpqPi0+jzQpwGCfbrQd9BLSKN2IHVMw7rWRU7Nl07GYbmzTJ6cvIkM0PqPUbnPfejbrKbSIcJsrYsLtaJw4oYHcdM476gZe6RubtNRqabEEddxYnMO/CQvmj0gYhmJ8iYDPfgyM4loYUScqxE+IDz7SVez1iPY0e8grD2yGSBrKVKsP1a2bRTSSgY2G5X6EI7Ah/E2yxTM7VEOxA+xWhBZgpdAGEOZ9EpilSlqqg== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(346002)(376002)(136003)(396003)(451199015)(40470700004)(36840700001)(46966006)(316002)(26005)(7696005)(450100002)(82740400003)(426003)(2616005)(16526019)(47076005)(2906002)(186003)(110136005)(41300700001)(70586007)(8676002)(8936002)(336012)(82310400005)(5660300002)(86362001)(356005)(81166007)(70206006)(1076003)(40460700003)(83380400001)(36756003)(966005)(6666004)(36860700001)(478600001)(40480700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 23:44:36.4197 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 751a17fa-8448-4ff6-806d-08dac9beda03 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6034 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Exports a DMA buf fd of a given KFD buffer handle. This is intended for being able to import KFD BOs into GEM contexts to leverage the amdgpu_bo_va API for more flexible virtual address mappings. It will also be used for the new upstreamable RDMA solution coming to UCX and RCCL. The corresponding user mode change (Thunk API and kfdtest) is here: https://github.com/fxkamd/ROCT-Thunk-Interface/commits/fxkamd/dmabuf Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 + .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 45 +++++++++++---- drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 55 +++++++++++++++++++ include/uapi/linux/kfd_ioctl.h | 14 ++++- 4 files changed, 104 insertions(+), 12 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index f50e3ba4d7a5..f72d25d5dd4f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -301,6 +301,8 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, uint64_t va, void *drm_priv, struct kgd_mem **mem, uint64_t *size, uint64_t *mmap_offset); +int amdgpu_amdkfd_gpuvm_export_dmabuf(struct kgd_mem *mem, + struct dma_buf **dmabuf); int amdgpu_amdkfd_get_tile_config(struct amdgpu_device *adev, struct tile_config *config); void amdgpu_amdkfd_ras_poison_consumption_handler(struct amdgpu_device *adev, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 67c832ae498b..bd36636a9596 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -710,6 +710,21 @@ kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, } } +static int kfd_mem_export_dmabuf(struct kgd_mem *mem) +{ + if (!mem->dmabuf) { + struct dma_buf *ret = amdgpu_gem_prime_export( + &mem->bo->tbo.base, + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DRM_RDWR : 0); + if (IS_ERR(ret)) + return PTR_ERR(ret); + mem->dmabuf = ret; + } + + return 0; +} + static int kfd_mem_attach_dmabuf(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_bo **bo) @@ -717,16 +732,9 @@ kfd_mem_attach_dmabuf(struct amdgpu_device *adev, struct kgd_mem *mem, struct drm_gem_object *gobj; int ret; - if (!mem->dmabuf) { - mem->dmabuf = amdgpu_gem_prime_export(&mem->bo->tbo.base, - mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? - DRM_RDWR : 0); - if (IS_ERR(mem->dmabuf)) { - ret = PTR_ERR(mem->dmabuf); - mem->dmabuf = NULL; - return ret; - } - } + ret = kfd_mem_export_dmabuf(mem); + if (ret) + return ret; gobj = amdgpu_gem_prime_import(adev_to_drm(adev), mem->dmabuf); if (IS_ERR(gobj)) @@ -2241,6 +2249,23 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, return ret; } +int amdgpu_amdkfd_gpuvm_export_dmabuf(struct kgd_mem *mem, + struct dma_buf **dma_buf) +{ + int ret; + + mutex_lock(&mem->lock); + ret = kfd_mem_export_dmabuf(mem); + if (ret) + goto out; + + get_dma_buf(mem->dmabuf); + *dma_buf = mem->dmabuf; +out: + mutex_unlock(&mem->lock); + return ret; +} + /* Evict a userptr BO by stopping the queues if necessary * * Runs in MMU notifier, may be in RECLAIM_FS context. This means it diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c index 6d291aa6386b..b9a9c4c9ec16 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c @@ -1567,6 +1567,58 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, return r; } +static int kfd_ioctl_export_dmabuf(struct file *filep, + struct kfd_process *p, void *data) +{ + struct kfd_ioctl_export_dmabuf_args *args = data; + struct kfd_process_device *pdd; + struct dma_buf *dmabuf; + struct kfd_dev *dev; + void *mem; + int ret = 0; + + dev = kfd_device_by_id(GET_GPU_ID(args->handle)); + if (!dev) + return -EINVAL; + + mutex_lock(&p->mutex); + + pdd = kfd_get_process_device_data(dev, p); + if (!pdd) { + ret = -EINVAL; + goto err_unlock; + } + + mem = kfd_process_device_translate_handle(pdd, + GET_IDR_HANDLE(args->handle)); + if (!mem) { + ret = -EINVAL; + goto err_unlock; + } + + ret = amdgpu_amdkfd_gpuvm_export_dmabuf(mem, &dmabuf); + mutex_unlock(&p->mutex); + if (ret) + goto err_out; + + ret = dma_buf_fd(dmabuf, args->flags); + if (ret < 0) { + dma_buf_put(dmabuf); + goto err_out; + } + /* dma_buf_fd assigns the reference count to the fd, no need to + * put the reference here. + */ + args->dmabuf_fd = ret; + + return 0; + +err_unlock: + mutex_unlock(&p->mutex); +err_out: + return ret; +} + /* Handle requests for watching SMI events */ static int kfd_ioctl_smi_events(struct file *filep, struct kfd_process *p, void *data) @@ -2749,6 +2801,9 @@ static const struct amdkfd_ioctl_desc amdkfd_ioctls[] = { AMDKFD_IOCTL_DEF(AMDKFD_IOC_AVAILABLE_MEMORY, kfd_ioctl_get_available_memory, 0), + + AMDKFD_IOCTL_DEF(AMDKFD_IOC_EXPORT_DMABUF, + kfd_ioctl_export_dmabuf, 0), }; #define AMDKFD_CORE_IOCTL_COUNT ARRAY_SIZE(amdkfd_ioctls) diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h index 42b60198b6c5..2da5c3ad71bd 100644 --- a/include/uapi/linux/kfd_ioctl.h +++ b/include/uapi/linux/kfd_ioctl.h @@ -37,9 +37,10 @@ * - 1.9 - Add available memory ioctl * - 1.10 - Add SMI profiler event log * - 1.11 - Add unified memory for ctx save/restore area + * - 1.12 - Add DMA buf export ioctl */ #define KFD_IOCTL_MAJOR_VERSION 1 -#define KFD_IOCTL_MINOR_VERSION 11 +#define KFD_IOCTL_MINOR_VERSION 12 struct kfd_ioctl_get_version_args { __u32 major_version; /* from KFD */ @@ -463,6 +464,12 @@ struct kfd_ioctl_import_dmabuf_args { __u32 dmabuf_fd; /* to KFD */ }; +struct kfd_ioctl_export_dmabuf_args { + __u64 handle; /* to KFD */ + __u32 flags; /* to KFD */ + __u32 dmabuf_fd; /* from KFD */ +}; + /* * KFD SMI(System Management Interface) events */ @@ -877,7 +884,10 @@ struct kfd_ioctl_set_xnack_mode_args { #define AMDKFD_IOC_AVAILABLE_MEMORY \ AMDKFD_IOWR(0x23, struct kfd_ioctl_get_available_memory_args) +#define AMDKFD_IOC_EXPORT_DMABUF \ + AMDKFD_IOWR(0x24, struct kfd_ioctl_export_dmabuf_args) + #define AMDKFD_COMMAND_START 0x01 -#define AMDKFD_COMMAND_END 0x24 +#define AMDKFD_COMMAND_END 0x25 #endif From patchwork Fri Nov 18 23:44:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13049180 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 10C11C433FE for ; Fri, 18 Nov 2022 23:45:07 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9E71110E82B; Fri, 18 Nov 2022 23:44:58 +0000 (UTC) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2056.outbound.protection.outlook.com [40.107.212.56]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3647910E830; Fri, 18 Nov 2022 23:44:40 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eytDtat5RKZnYKL1Sc4xVOhDGPF/zB5ci32IVls+jGaqgqc0+ULf05VartTML3oBbzGhsWh38qTx0H36aJ3ZJNjYmy5leCDzA/sCVkz3ycslE9HPE4GbbFVFNtngff0a8tEpJo89b0/2n0eDb7pJxXZdyxgjGV6k+CiOQHYUW935oiNanN6k5Vjf2tgL5pWSu72GqjLKzAhNhHnF/UEPz9CVHwYNVYal583PQ9zp5X22Hwbe2hdrgizHXxeE6isHQ8uPsE699KmuY1sRHqbAlhO7f7cUg6d+xn4rfjqZW2t5Dz3PrtiqN1mD9YtaI5r1igSW1ZrdsATEduZCxwnKOA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Bw+N8ccGpl3r9CmoFwme8QjBDbux2qwPzymG3gYVcbU=; b=YhoJBTEVUiXPH4AAOFtBi44kqBe/ioJYtBpyQdYrxoKNg32Jf96ZWq4qHHOK0lavAEF26b1Io8pjfmiICNBd08iVimSuxoH+Low8qlOkQPYLqd/IsTBzGIfYxAo4J8JZaTFcWsun6mdkMLuGZzIVauNHQ3m3b8ID+K6ydbR/FUxpwm5VDhAVF1imcbw6wMtRSkjpjkWaHAcgboxszqM2BzsOaqld+l16I6tr+zpI4XXqSp9+b6YkGYfi7wrNKpRTOUEXgfTjmMBbIKnlPi8hgAq0NEYVHmcG7/HxXJcDkkWDVYouf+X7AdEhmfbPCxg/XUU0iMG2kcZ/MCDndrTV8A== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Bw+N8ccGpl3r9CmoFwme8QjBDbux2qwPzymG3gYVcbU=; b=zZFS8mEx+Ee/MhhUmOV5gjv44zIU0fmKHNTEuIKT9i6+7NNDcKjX7yc7XPcBeyDB5YcCvcwbpdxgs4WkuX/LzGN0GTaU5mRNL3GdIRIz5+CCFvD6qHQjBTW7/tToxK077+HtrbAP6cCR6YY+dQRIxfZjVm8lUMEfhhdmYD+es6E= Received: from DM6PR03CA0045.namprd03.prod.outlook.com (2603:10b6:5:100::22) by SJ0PR12MB8115.namprd12.prod.outlook.com (2603:10b6:a03:4e3::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.17; Fri, 18 Nov 2022 23:44:37 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:5:100:cafe::6d) by DM6PR03CA0045.outlook.office365.com (2603:10b6:5:100::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9 via Frontend Transport; Fri, 18 Nov 2022 23:44:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 23:44:37 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 18 Nov 2022 17:44:36 -0600 From: Felix Kuehling To: , Subject: [PATCH 3/6] drm/amdkfd: Improve amdgpu_vm_handle_moved Date: Fri, 18 Nov 2022 18:44:19 -0500 Message-ID: <20221118234422.1468806-4-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20221118234422.1468806-1-Felix.Kuehling@amd.com> References: <20221118234422.1468806-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT014:EE_|SJ0PR12MB8115:EE_ X-MS-Office365-Filtering-Correlation-Id: 6066de2b-efbd-4802-8d40-08dac9beda73 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +efgX6aM7/sSVniOzl2HBDEpj3CHl5hG6hCFh1VIq6wanNiIEZvRRvXhZKVRDnPdORh8yg2cE/8taWSX08cyU6Xk7u9b11aq1CQ1vxxOQ6Z80yOvOaeqWfsMmzueAGtIAlpXaNEP0WyZegU+A6BaKh3ufppWQNT0nS802hjEP2LNF2oOy/6Tb3K7EPwiHPpt18W0eyk2QEW8rvw1t5dPjEO+dXch8M6VZ2x/tbjSDByX46ccusIWlRktvJMI6IyAc9CJC0Rs1DyPxS4ke5NF8uAwWzCDCT52ZhCp9nA/BuYRQ9N3Z1l085AMOQ6j92EZ+05RC99xjn9m+EhYb6EVBl1wXx51MJtNNXLHeXgSVdU3vt48f+A/vL/Fj2fV8UwUZXDurfBwllTpuT8I8DsSillal3PLCJM+spBnwboqKkLc+4rbsFOpJlPPlbID/EC7ED06G5g/rOKdrh/sqappPuojccqP8YDpGuzMM5dNoq279KsMLjrzMzlQACyuVU/oP2GmALv3Lnr0FLJdRujfMrdOobJaDS3DzwcMOe9HL431KUyl0rOWU3mqZDqt53T62pB8a/FE2/bX4CAaWl5OBUuggTWPGEajY4hsg/Z9YUO+no27cDyqB/akWtgmLYMkvfHpocKsMR35wWUMdPXQRpUFH4zAq9jMToUI3vn+2tPURnqCCo9FSfCRWYs6GHIOSXQEnSg7DtcnzQvyIweDGgguq0CdG35EnTSzvoUm5/k= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230022)(4636009)(136003)(346002)(396003)(39860400002)(376002)(451199015)(36840700001)(46966006)(40470700004)(86362001)(82310400005)(40480700001)(36756003)(41300700001)(40460700003)(1076003)(2616005)(70586007)(70206006)(450100002)(4326008)(8676002)(426003)(66574015)(8936002)(16526019)(186003)(5660300002)(47076005)(336012)(6666004)(478600001)(26005)(316002)(82740400003)(110136005)(356005)(7696005)(81166007)(2906002)(36860700001)(83380400001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 23:44:37.1540 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6066de2b-efbd-4802-8d40-08dac9beda73 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB8115 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Let amdgpu_vm_handle_moved update all BO VA mappings of BOs reserved by the caller. This will be useful for handling extra BO VA mappings in KFD VMs that are managed through the render node API. Signed-off-by: Felix Kuehling Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 6 +++++- drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 18 +++++++++++++----- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 3 ++- 4 files changed, 21 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 275da612cd87..a80d2557edb2 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -1121,6 +1121,10 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p) return r; } + /* TODO: Is this loop still needed, or could this be handled by + * amdgpu_vm_handle_moved, now that it can handle all BOs that are + * reserved under p->ticket? + */ amdgpu_bo_list_for_each_entry(e, p->bo_list) { /* ignore duplicates */ bo = ttm_to_amdgpu_bo(e->tv.bo); @@ -1140,7 +1144,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p) return r; } - r = amdgpu_vm_handle_moved(adev, vm); + r = amdgpu_vm_handle_moved(adev, vm, &p->ticket); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c index 271e30e34d93..23a213e4ab2c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c @@ -404,7 +404,7 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach) r = amdgpu_vm_clear_freed(adev, vm, NULL); if (!r) - r = amdgpu_vm_handle_moved(adev, vm); + r = amdgpu_vm_handle_moved(adev, vm, ticket); if (r && r != -EBUSY) DRM_ERROR("Failed to invalidate VM page tables (%d))\n", diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 003aa9e47085..b61eb2e575d1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -1278,11 +1278,12 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, * PTs have to be reserved! */ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, - struct amdgpu_vm *vm) + struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket) { struct amdgpu_bo_va *bo_va; struct dma_resv *resv; - bool clear; + bool clear, unlock; int r; spin_lock(&vm->status_lock); @@ -1305,17 +1306,24 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, spin_unlock(&vm->status_lock); /* Try to reserve the BO to avoid clearing its ptes */ - if (!amdgpu_vm_debug && dma_resv_trylock(resv)) + if (!amdgpu_vm_debug && dma_resv_trylock(resv)) { clear = false; + unlock = true; + /* The caller is already holding the reservation lock */ + } else if (ticket && dma_resv_locking_ctx(resv) == ticket) { + clear = false; + unlock = false; /* Somebody else is using the BO right now */ - else + } else { clear = true; + unlock = false; + } r = amdgpu_vm_bo_update(adev, bo_va, clear); if (r) return r; - if (!clear) + if (unlock) dma_resv_unlock(resv); spin_lock(&vm->status_lock); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index 6546e786bf00..53d1ac4b69db 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -404,7 +404,8 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, struct amdgpu_vm *vm, struct dma_fence **fence); int amdgpu_vm_handle_moved(struct amdgpu_device *adev, - struct amdgpu_vm *vm); + struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket); void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, struct amdgpu_vm *vm, struct amdgpu_bo *bo); int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm, From patchwork Fri Nov 18 23:44:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13049181 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1AA00C43217 for ; Fri, 18 Nov 2022 23:45:11 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 213E610E828; Fri, 18 Nov 2022 23:44:58 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2082.outbound.protection.outlook.com [40.107.223.82]) by gabe.freedesktop.org (Postfix) with ESMTPS id B9DC110E81A; Fri, 18 Nov 2022 23:44:40 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ldAM371+jgjcQWbyVr4wExCbM/7yaIy7+t0gaLFdq8JoyMgW3/SJsGxYp66SFbuJy0/yCnXNgeu3Yz5QGJQaX33QNBPB3P3YA7GnOFiooz3l7dsNBPl9XtoPe721aDfuX628DX+HSqj0FnigKr6Z5UUheRjDy1jYu9XrwoGquQouWsbM2xKdxdnYgJciXjFVLyCYajYHt9znSMXf+f9b7ZizRX6t+ocaP24E86dGj2vfCfXrMae0cLoJ5GTmn/S1BduhfGI3vGBuySyLCLBwPjF378kzAVJDxkWad1B1mC8tfRy6+aSozSodhpFMP5deFFqItEZt1Ry1ba8n35sVDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=MNa+0jXg3zotebl8DF+7mkWyGFevVD+e6cAVXyH7dmE=; b=VaO/lB4jGBmkRJuJmJXLuXLs5Rkp0xxBgI7Qgq9gylmsu85/DpAx2o5k5g0BtKWd5+2MokWVZu/KHW8Oms3/SQUGyITJXyYNiPL6NOqjKQ8b64ff8sZb7z5uSlOQMQbvmfuT4C715Qt8MnJYIwjG4CMqp+wiPqm+icXOzMI1zxHa7PZr3nso4CCutAaCpyLYpdzq1OaoSnhS6r+I0DuUy18EEXFxE2h+KLYTaE00qjEkYM1NGWx0AxZL/48qPRNUzY/yOBLTYQYIy228OK7kWSDntWhKWXLubH6eUZ9TfPCee5zqfJs8WY/c/UGHJ6Q78cGOzdsqG7nJQFmUEieiQg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=MNa+0jXg3zotebl8DF+7mkWyGFevVD+e6cAVXyH7dmE=; b=WASA9L3y7ZXareBTUrWeYvgeuXId4k2N3G6YCFP+fpdIzerm+lgn7RsQqtij5Dkf/5HazhZXWIJ9tSK8kjdaMNYF3GIxEuzlEfhe86iiavJcQ+HlAZulXzEqmR5k6ChqA6Khpu+Upd/SxSfuxsmP4RmZetXJwjgxdwIy6D6emeo= Received: from DM6PR03CA0050.namprd03.prod.outlook.com (2603:10b6:5:100::27) by MW4PR12MB6900.namprd12.prod.outlook.com (2603:10b6:303:207::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.16; Fri, 18 Nov 2022 23:44:38 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:5:100:cafe::1b) by DM6PR03CA0050.outlook.office365.com (2603:10b6:5:100::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9 via Frontend Transport; Fri, 18 Nov 2022 23:44:37 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 23:44:37 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 18 Nov 2022 17:44:36 -0600 From: Felix Kuehling To: , Subject: [PATCH 4/6] drm/amdgpu: Attach eviction fence on alloc Date: Fri, 18 Nov 2022 18:44:20 -0500 Message-ID: <20221118234422.1468806-5-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20221118234422.1468806-1-Felix.Kuehling@amd.com> References: <20221118234422.1468806-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT014:EE_|MW4PR12MB6900:EE_ X-MS-Office365-Filtering-Correlation-Id: 7054f08b-b194-4271-9a09-08dac9bedae3 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: meXNt2k9T1v7+aEpZAVGPC9BZmxbR1FAyiMQPUSmKo/CNMoS0DNohPnG2q7ECrTlngq5k5RPxhE9aMFWrI0x/li9yJ7TsSCaV+Uk6/UDFn9uD9HZNxmXpww4mlsxcROz1T6yD0/1hW2u4Kgjw7xrvDvLjgUgHXqesQmjlCFLroihWiiHCxIogAta9hCmexxP80ovkN+nR7hHTFNuvHAIeEIcmAu55mh7yzKNtO2dTyRQcR+6aW2UeYx5ar4sLK4nbkOE6XfUj2U3hYWRmFUqnYtv8vQ41CIxWfGDj1AIKhJD3Rue0Z3wRsUZ2eEiwxJGHLkibreQOjnnfnTFVEKXVhjPPG32lrsVhxcrg4g7CfZf44fYum7N6/nFew+E1DP+yh4clqyNqh9qYoXeRnUhKjCqM6Lu1Ci6n3yymNOKV1+Mtcd284Eljuj2aP+tugwIpRCA9xKQtnaH7ib8hHd1ZrNx29HKN1F1mjArurdAztM+afRDJusibW0FqVu5t/xF4AnojCytsUhgCP1pao2bOvZhF2jWD7BdpQyjotBiSQJf+2V8uZCYHm5XuXsxfSItuWCEuz/GNItKrD5YE1wuNv1k2eJEVr9GzHz+U3PRXqOZCePtxIbeeiv3mDK1CQ90H9m5WfK0gjIn0csTcbNR5GkDKX26bc8pld1cdbe6AJaiFaoh029bKGbiAU/UOyI4yhlQa9nbiovSOioToE74v6VwS9CpdR/tt52H/CCNu6U= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230022)(4636009)(346002)(136003)(396003)(376002)(39860400002)(451199015)(36840700001)(40470700004)(46966006)(70206006)(450100002)(2906002)(8936002)(81166007)(36860700001)(70586007)(82740400003)(36756003)(356005)(8676002)(40460700003)(5660300002)(83380400001)(41300700001)(40480700001)(478600001)(110136005)(7696005)(6666004)(26005)(316002)(426003)(86362001)(2616005)(16526019)(47076005)(1076003)(336012)(186003)(82310400005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 23:44:37.8883 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7054f08b-b194-4271-9a09-08dac9bedae3 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB6900 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Instead of attaching the eviction fence when a KFD BO is first mapped, attach it when it is allocated or imported. This in preparation to allow KFD BOs to be mapped using the render node API. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 63 ++++++++++--------- 1 file changed, 32 insertions(+), 31 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index bd36636a9596..e13dff75ca5c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -360,6 +360,24 @@ static int amdgpu_amdkfd_bo_validate(struct amdgpu_bo *bo, uint32_t domain, return ret; } +static int amdgpu_amdkfd_bo_validate_and_fence(struct amdgpu_bo *bo, + uint32_t domain, + struct dma_fence *fence) +{ + int ret = amdgpu_bo_reserve(bo, false); + + if (ret) + return ret; + + ret = amdgpu_amdkfd_bo_validate(bo, domain, true); + if (!ret) + dma_resv_add_fence(bo->tbo.base.resv, fence, + DMA_RESV_USAGE_BOOKKEEP); + amdgpu_bo_unreserve(bo); + + return ret; +} + static int amdgpu_amdkfd_validate_vm_bo(void *_unused, struct amdgpu_bo *bo) { return amdgpu_amdkfd_bo_validate(bo, bo->allowed_domains, false); @@ -1709,6 +1727,11 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( } bo->allowed_domains = AMDGPU_GEM_DOMAIN_GTT; bo->preferred_domains = AMDGPU_GEM_DOMAIN_GTT; + } else { + ret = amdgpu_amdkfd_bo_validate_and_fence(bo, domain, + &avm->process_info->eviction_fence->base); + if (ret) + goto err_validate_bo; } if (offset) @@ -1718,6 +1741,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( allocate_init_user_pages_failed: err_pin_bo: +err_validate_bo: remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); drm_vma_node_revoke(&gobj->vma_node, drm_priv); err_node_allow: @@ -1788,10 +1812,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( if (unlikely(ret)) return ret; - /* The eviction fence should be removed by the last unmap. - * TODO: Log an error condition if the bo still has the eviction fence - * attached - */ amdgpu_amdkfd_remove_eviction_fence(mem->bo, process_info->eviction_fence); pr_debug("Release VA 0x%llx - 0x%llx\n", mem->va, @@ -1905,19 +1925,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( if (unlikely(ret)) goto out_unreserve; - if (mem->mapped_to_gpu_memory == 0 && - !amdgpu_ttm_tt_get_usermm(bo->tbo.ttm)) { - /* Validate BO only once. The eviction fence gets added to BO - * the first time it is mapped. Validate will wait for all - * background evictions to complete. - */ - ret = amdgpu_amdkfd_bo_validate(bo, domain, true); - if (ret) { - pr_debug("Validate failed\n"); - goto out_unreserve; - } - } - list_for_each_entry(entry, &mem->attachments, list) { if (entry->bo_va->base.vm != avm || entry->is_mapped) continue; @@ -1944,10 +1951,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( mem->mapped_to_gpu_memory); } - if (!amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) && !bo->tbo.pin_count) - dma_resv_add_fence(bo->tbo.base.resv, - &avm->process_info->eviction_fence->base, - DMA_RESV_USAGE_BOOKKEEP); ret = unreserve_bo_and_vms(&ctx, false, false); goto out; @@ -1964,7 +1967,6 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv) { struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); - struct amdkfd_process_info *process_info = avm->process_info; unsigned long bo_size = mem->bo->tbo.base.size; struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; @@ -2005,15 +2007,6 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( mem->mapped_to_gpu_memory); } - /* If BO is unmapped from all VMs, unfence it. It can be evicted if - * required. - */ - if (mem->mapped_to_gpu_memory == 0 && - !amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm) && - !mem->bo->tbo.pin_count) - amdgpu_amdkfd_remove_eviction_fence(mem->bo, - process_info->eviction_fence); - unreserve_out: unreserve_bo_and_vms(&ctx, false, false); out: @@ -2240,8 +2233,16 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, amdgpu_sync_create(&(*mem)->sync); (*mem)->is_imported = true; + ret = amdgpu_amdkfd_bo_validate_and_fence(bo, (*mem)->domain, + &avm->process_info->eviction_fence->base); + if (ret) + goto err_remove_mem; + return 0; +err_remove_mem: + remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); + drm_vma_node_revoke(&obj->vma_node, drm_priv); err_free_mem: kfree(mem); err_put_obj: From patchwork Fri Nov 18 23:44:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13049183 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id C6564C433FE for ; Fri, 18 Nov 2022 23:45:26 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8E05010E82D; Fri, 18 Nov 2022 23:45:25 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on20611.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e89::611]) by gabe.freedesktop.org (Postfix) with ESMTPS id E070F10E822; Fri, 18 Nov 2022 23:44:40 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TshNBiUwo0nHro3jy6pHZUXfz10pd71bbuyFQbebycSAqo+UoVGEPh05RTX4T7476VrdSFVudfA7vkSIp3P8BpPVFvn+4Yh2isrdS4Yrnd1t3o4+qOD0kumth9BH84rRB5d+yV7jmSmQsCiCgMij8/u+Du8KPxeU+WcWdBdkG8DP2JF2342md3dtT3xm7PvSeoduPUNht48n7XSZwjv/qTqW+poukrQjUbXGp84APJRyPEYdPMCQlO8UT7MtkXSVq2J574QvazFiGwKR8qoQZ/KslrrErToR2WeYqSUMlDE2CYT7OQ5xR1AKoucXCWU+ZSxDPlMkj6KmnF5H+pqrJw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rHwDlwC5Ppk5MBrNaHtl6E2VCkfhWbOCgEKSpuHgEwQ=; b=XomJeqOyv1I4ttd/G6k3ShFH5othuuDTnIVwdjYmO6HUOiMpLicahZ43WToepzL2KhlzKpkAMCLvKTb09XKVnYx148YiOJhlP2BKbmhn1mUUIhXYDtt2Q9deTk1lgE+ziZURMOkX9JkX4Rz1xp1wutQKdq8Pipla4zNd8UbqZztivbnq00go2A7HMtgSxGMdVVPHqrnDtD1fno5L9BXV86VH2NbxBWMg78OU/gGwoS6YzvyNALDzbTaJ4i9Lr5VPmYkeW3mXLcufUFq6B1jDVCB3aXqM/o3IuLJc92Qk0vF7m7/X+iTsBDXsyDBYFzxgC9p+Q4XdLMgrjDwF5bLniw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rHwDlwC5Ppk5MBrNaHtl6E2VCkfhWbOCgEKSpuHgEwQ=; b=wgprWvoRKkWBz4ODbcawiVIG2Vr8Q/T632ZzcemDtodbY+qr0rX0ecPj7v47X760fMxpyXs0B/ihE4k+nDQPok3VyW2I9I+KU5Z8yUcJtUT7pJ9IPBNQUQCSIjG7ntiBqnLNx8X1JaX/M9GST2GSvbXc+1tToXDLpipFN8ewQ2M= Received: from DM6PR03CA0058.namprd03.prod.outlook.com (2603:10b6:5:100::35) by PH7PR12MB6955.namprd12.prod.outlook.com (2603:10b6:510:1b8::7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5813.18; Fri, 18 Nov 2022 23:44:39 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:5:100:cafe::5) by DM6PR03CA0058.outlook.office365.com (2603:10b6:5:100::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9 via Frontend Transport; Fri, 18 Nov 2022 23:44:38 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 23:44:38 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 18 Nov 2022 17:44:36 -0600 From: Felix Kuehling To: , Subject: [PATCH 5/6] drm/amdgpu: update mappings not managed by KFD Date: Fri, 18 Nov 2022 18:44:21 -0500 Message-ID: <20221118234422.1468806-6-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20221118234422.1468806-1-Felix.Kuehling@amd.com> References: <20221118234422.1468806-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT014:EE_|PH7PR12MB6955:EE_ X-MS-Office365-Filtering-Correlation-Id: b02293d2-142d-47b4-e238-08dac9bedb4e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4iEsZYkEDOQf/5ZIx7PFV5LKpG2lePIVeGGItpxqo2IH1m61fENnXtWBQY2J1y7X4ydn3VifYTMHqaeavGvZ7lmdrUEtuYgzqaAj3mBQcmYEhrHc78PqOAe2erRXI7SnpG3jkcWQTgMZc6CMj2hR80PKzVRBipVz+l/Gf7f4blc6c2OhBwUdXT8I0vP3VT/E4jxV3sGtMWvzuw1LKJW4N9CJa/B4LT1wTLfLxa0mCsqX8zjwgpE5sLTpRuDkbRUipfy4oQkQpeRTwcvcyXpWpWMvX7To24eAoUOrqV6+9O/CWX1VXFDfDq6NJob+ULs7jzYBcYtvhlpbvphvgZ+BdkJmyZ/jBQlUZpLzF+HUjSzBvgmNqB+r4wyfyxZzoEQ0mJ6re/YeA4PG1nhwVosMlbC56q+2/+G5w9gMS+kaaAC38D8KAWoxQfgxkHndc9mIZnKIiHbQQ3C6h3raWP0yQ4ZCPb302XDtc83gZ88pEaljfR0SpApK6Yumeg8b/9QpUgdvMie1Ee2p72y/sDxxQKJJ5BHZkBRXJ+y/dfwkMLRo2FWOb/BAt0aOehJ6EqL81XV91A/0v81hBgFLh9z78HZqQ/81HTZrDW8yevsyRxS4sZrAeoi96xL1rLe3PKqlpSQJAtYYp5CYfTcHCD8h/nHaF2Lok1CL9gZ1vlgy56UYd4fYjOOGDXWX/4k6O564nn5anlg8CAaWiJLiAB2NnfbH06X6qJbndeQE0MZTEPs= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230022)(4636009)(396003)(376002)(346002)(136003)(39860400002)(451199015)(46966006)(36840700001)(40470700004)(356005)(40460700003)(110136005)(41300700001)(82310400005)(316002)(186003)(8676002)(47076005)(70586007)(81166007)(450100002)(5660300002)(70206006)(1076003)(83380400001)(336012)(82740400003)(15650500001)(86362001)(36860700001)(8936002)(2616005)(2906002)(478600001)(426003)(36756003)(40480700001)(7696005)(26005)(16526019)(6666004)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 23:44:38.5914 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b02293d2-142d-47b4-e238-08dac9bedb4e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6955 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" When restoring after an eviction, use amdgpu_vm_handle_moved to update BO VA mappings in KFD VMs that are not managed through the KFD API. This should allow using the render node API to create more flexible memory mappings in KFD VMs. v2: Sync with pd fence after all page table updates v3: Update comments, remove TODOs that are no longer applicable Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 28 +++++++++++++++---- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index e13dff75ca5c..ecf81f27cb4b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -2655,12 +2655,6 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) if (ret) goto validate_map_fail; - ret = process_sync_pds_resv(process_info, &sync_obj); - if (ret) { - pr_debug("Memory eviction: Failed to sync to PD BO moving fence. Try again\n"); - goto validate_map_fail; - } - /* Validate BOs and map them to GPUVM (update VM page tables). */ list_for_each_entry(mem, &process_info->kfd_bo_list, validate_list.head) { @@ -2708,6 +2702,19 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) if (failed_size) pr_debug("0x%lx/0x%lx in system\n", failed_size, total_size); + /* Update mappings not managed by KFD */ + list_for_each_entry(peer_vm, &process_info->vm_list_head, + vm_list_node) { + struct amdgpu_device *adev = amdgpu_ttm_adev( + peer_vm->root.bo->tbo.bdev); + + ret = amdgpu_vm_handle_moved(adev, peer_vm, &ctx.ticket); + if (ret) { + pr_debug("Memory eviction: handle moved failed. Try again\n"); + goto validate_map_fail; + } + } + /* Update page directories */ ret = process_update_pds(process_info, &sync_obj); if (ret) { @@ -2715,6 +2722,15 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) goto validate_map_fail; } + /* Sync with fences on all the page tables. They implicitly depend on any + * move fences from amdgpu_vm_handle_moved above. + */ + ret = process_sync_pds_resv(process_info, &sync_obj); + if (ret) { + pr_debug("Memory eviction: Failed to sync to PD BO moving fence. Try again\n"); + goto validate_map_fail; + } + /* Wait for validate and PT updates to finish */ amdgpu_sync_wait(&sync_obj, false); From patchwork Fri Nov 18 23:44:22 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13049182 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F06B6C4332F for ; Fri, 18 Nov 2022 23:45:14 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 56FE910E82A; Fri, 18 Nov 2022 23:44:58 +0000 (UTC) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by gabe.freedesktop.org (Postfix) with ESMTPS id CAEF810E81A; Fri, 18 Nov 2022 23:44:41 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=h+ooGHSd4RzNv6ABCpqot2xp3nSyDNoL2nfIlIUpHslSdRJP0XaxcwfGysvHePexhzSZJTcSXEdYOtwsTbm3AkSTESk9EC+iMJJxrIAAY6UuzWH/yRNsfC9JJun/U+6s15K1603x7nOJGkEkb6yKWAIlJNNOIh1d5q3glIzGpHoDRZXCgqWaUCysCufpPCAVzUdTIn9SetxjcXNhlhHrgURanb8tSr7q80UEhNM/Sb65xqxqxdCk9hqgrrCHvadK4NuIxFkF+Lz4MqlEmKlwdS1Tv3I5oH9mTwptTBk4iJ38q4Kq+eN60npbbajNqUEDWw3sZP0hV+X2A5JhyMzTfw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=H4yzmwS1Qh4YoaxEWo8BJZa5KmJMGPv/L2JuTUnDtbo=; b=Vtst5ed64TbIADFZV2Kdpse4aun6VrMDsqPd70vOTeJFwsFeHcqqbrZdVhKlYvATry/JSj1cyaW8/Y+M3G7gPWq9P0PB47DG0iD6ZHmZsB1p6+cQaPJwOya0S1BFqC/M3jq9Z5DcF1ZTGjjl8Hqfm/m07oIRt1HbV9mHiUfc7ZBRzkBlYjBrOjcNz9HLZBnfWSCRMWT7DOpqwCNqWtMdtycV3/nfcK2ffic3MjFXr8ffwmVDVHsirpvkvN93CbwPe9Q6QOqJr6ZZwcI0UZ9bWTsoMhVXrFuAOuPChriJ2qXloElq8xo2yAbZLcsV7k/eWBa8FLZZAnR056bw9npE3Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=H4yzmwS1Qh4YoaxEWo8BJZa5KmJMGPv/L2JuTUnDtbo=; b=5rYdmtypV21CA1XD6pMXbCxceRx6ZhVYy0ALMMtC7VdPxaMocc8Ja11+KMt8Z4vHu686UMWgserde2SxcWFBT4EgCKPugEgBbO9ze+pcnZ75TpihNBtC/h9Aop1tnwnLZ8HopNUG6MNefNQRfu1RcjL+qioptWaImUsTiCUyRC8= Received: from DM6PR03CA0057.namprd03.prod.outlook.com (2603:10b6:5:100::34) by MN2PR12MB4173.namprd12.prod.outlook.com (2603:10b6:208:1d8::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9; Fri, 18 Nov 2022 23:44:39 +0000 Received: from DM6NAM11FT014.eop-nam11.prod.protection.outlook.com (2603:10b6:5:100:cafe::3) by DM6PR03CA0057.outlook.office365.com (2603:10b6:5:100::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.5834.9 via Frontend Transport; Fri, 18 Nov 2022 23:44:39 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by DM6NAM11FT014.mail.protection.outlook.com (10.13.173.132) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.5834.8 via Frontend Transport; Fri, 18 Nov 2022 23:44:39 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2375.34; Fri, 18 Nov 2022 17:44:37 -0600 From: Felix Kuehling To: , Subject: [PATCH 6/6] drm/amdgpu: Do bo_va ref counting for KFD BOs Date: Fri, 18 Nov 2022 18:44:22 -0500 Message-ID: <20221118234422.1468806-7-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.32.0 In-Reply-To: <20221118234422.1468806-1-Felix.Kuehling@amd.com> References: <20221118234422.1468806-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: DM6NAM11FT014:EE_|MN2PR12MB4173:EE_ X-MS-Office365-Filtering-Correlation-Id: dbf9a182-dbd3-4d9e-ef02-08dac9bedbbe X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RJJopOItqbTPsk2xtRYWEZ2zOgPgLKkpDm+5hEoHqfcpFEgKHGqcnVa/7t8dwgTNKWSZnZrTzO4al9uLuZAYdQre60XIihOE6illxrtw2nc7c0ea9qMUkqbzBeCcO7srTNKo9HoXouoOBo4SJ9R1tuYG6CME5DlFmxSRvq2M5UhSDWNCMno6y2b8ynthKVu708H/A9crJs/1C2d/J7jXsVlv80oyJ7sf21dm75I/oZhtFB5x0NbZZcg3cNomEXoAnwdphz77MITNCMMOrInkhy86HdVfWonT/XNyaFIkxfsZBJGse1tmLEyrN9mJ1uM68dbNdaZYFXIyTPzEJO/+GjN5tZEhjf99TWEOcae5TF3Bxd5nl78RhaslE1zaOr+D94z8IXNjqpwa+r1Ru5rNbl8dQMazDNiHVcbz5czsYUjBgZxXU7xqdweDGnYcX5bcBo5fUqweSmGVXt+ZeAoacsHvFIKMSLomMF1iE5Lsms3Upfz/pYpp+NiLSlptJ6yMKZoPNWNdhHc84zS5A4omxzzN4tdhjsCVKIKuLg5BPin7drwKlIWuPCoGSjUA7IQwgPEgQZg0oCsDttvxDWhRS4swFkf2dYtZRb4fOEzn8wAr31vQnEI1n9AUeUS5LvPTogeL06WcFYctS/ZiTmJ7i49sI1gwXc19zisiLrS3Z6SGf9ffl9TxAXJ+gTheYQiwRwrSrUW1TXLhUIARfXgTMc5aPaPDryhMoDM/vJ1gSlw= X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230022)(4636009)(39860400002)(346002)(376002)(136003)(396003)(451199015)(40470700004)(46966006)(36840700001)(82310400005)(356005)(82740400003)(316002)(26005)(8676002)(70586007)(86362001)(70206006)(81166007)(36860700001)(83380400001)(336012)(2906002)(1076003)(40480700001)(36756003)(47076005)(2616005)(426003)(450100002)(41300700001)(5660300002)(40460700003)(186003)(16526019)(8936002)(478600001)(7696005)(6666004)(110136005)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Nov 2022 23:44:39.3257 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: dbf9a182-dbd3-4d9e-ef02-08dac9bedbbe X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT014.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4173 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is needed to correctly handle BOs imported into the GEM API, which would otherwise get added twice to the same VM. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 28 +++++++++++++++---- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index ecf81f27cb4b..68e794cad647 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -370,9 +370,17 @@ static int amdgpu_amdkfd_bo_validate_and_fence(struct amdgpu_bo *bo, return ret; ret = amdgpu_amdkfd_bo_validate(bo, domain, true); - if (!ret) - dma_resv_add_fence(bo->tbo.base.resv, fence, - DMA_RESV_USAGE_BOOKKEEP); + if (ret) + goto unreserve_out; + + ret = dma_resv_reserve_fences(bo->tbo.base.resv, 1); + if (ret) + goto unreserve_out; + + dma_resv_add_fence(bo->tbo.base.resv, fence, + DMA_RESV_USAGE_BOOKKEEP); + +unreserve_out: amdgpu_bo_unreserve(bo); return ret; @@ -785,6 +793,7 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, uint64_t va = mem->va; struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; struct amdgpu_bo *bo[2] = {NULL, NULL}; + struct amdgpu_bo_va *bo_va; bool same_hive = false; int i, ret; @@ -871,7 +880,12 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, pr_debug("Unable to reserve BO during memory attach"); goto unwind; } - attachment[i]->bo_va = amdgpu_vm_bo_add(adev, vm, bo[i]); + bo_va = amdgpu_vm_bo_find(vm, bo[i]); + if (!bo_va) + bo_va = amdgpu_vm_bo_add(adev, vm, bo[i]); + else + ++bo_va->ref_count; + attachment[i]->bo_va = bo_va; amdgpu_bo_unreserve(bo[i]); if (unlikely(!attachment[i]->bo_va)) { ret = -ENOMEM; @@ -895,7 +909,8 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, continue; if (attachment[i]->bo_va) { amdgpu_bo_reserve(bo[i], true); - amdgpu_vm_bo_del(adev, attachment[i]->bo_va); + if (--attachment[i]->bo_va->ref_count == 0) + amdgpu_vm_bo_del(adev, attachment[i]->bo_va); amdgpu_bo_unreserve(bo[i]); list_del(&attachment[i]->list); } @@ -912,7 +927,8 @@ static void kfd_mem_detach(struct kfd_mem_attachment *attachment) pr_debug("\t remove VA 0x%llx in entry %p\n", attachment->va, attachment); - amdgpu_vm_bo_del(attachment->adev, attachment->bo_va); + if (--attachment->bo_va->ref_count == 0) + amdgpu_vm_bo_del(attachment->adev, attachment->bo_va); drm_gem_object_put(&bo->tbo.base); list_del(&attachment->list); kfree(attachment);