From patchwork Tue Oct 17 21:13:27 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426133 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B79B4CDB474 for ; Tue, 17 Oct 2023 21:14:35 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CD8C710E332; Tue, 17 Oct 2023 21:14:26 +0000 (UTC) Received: from NAM11-DM6-obe.outbound.protection.outlook.com (mail-dm6nam11on2077.outbound.protection.outlook.com [40.107.223.77]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2DC3810E32B; Tue, 17 Oct 2023 21:14:24 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=E3yDyWFj2+o/ALO6vSuUx7KuFivbCfcbNU7+B3cWO9KFNN8FdRls6HcClnp08J7t8LiHwkPCQ5fZlLfSO4if/u9hPmg/VFbAgXwqIIusx2pHJ7hAKWXPdSHbEcmjsjMXa/fQ9vnOP/T0k/5fcOsy+Vy0nbFe9MiHE/TGpDv0O9wzFgpdrKp8+dMI31T0s6chiwnjwfQZtTT6EUIODyKxIrZLUqGFn5xJZR6Wv/+4eN45cJBKWWCOPxN6N7PJo72aJHXHZNjF5qHGh+2+1SgfqFR11swwoaV2//OBJLwidHx9vhBT0N/FS7qpJAHI4bcEYKKaD68hrfXlt2YJAClXcA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7sZinNPWFmaotSiWfnlXYvYAEgtu3zQtdKvllRSF47s=; b=ZVvd0TDCm2jxbZJrpY2wrUGA6WkzYqDNwNFrp5EliVrq8Jl/8D4wfzQqxZfJIu12yoVFKpFJniHm8zNp4n15JyqDJWLZs9cCKxBubEhHC39t2paSe2Sgl1+gVMcAOkcKcOqYUCpWDAJlF6PXu0pajslH1WpkMXw1hLKWOpYM6/gB6wGjWM0Iccc+2tWYB2fgpx+uNojwKLZ5to+eJk7CT2J48/FQE8HwTBbw4KF+UUlp4DhbHRFWKpGX9Lj295WYjkNwAi833xc5R01iOtfftEXzEaRgkZxFRDDWCi8pYMq0TxA8FSay7Q6utAH1x8ArZNj7JTnbmJxuSdOzCJBdIQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=7sZinNPWFmaotSiWfnlXYvYAEgtu3zQtdKvllRSF47s=; b=Rcf5nD75BwBdjRMcZTVei9a8B3fHQaDRS5MoueDK3gSNi+RSlFSBT0zdIT7w6TurqyhcuvQHJY87zYQr+uTrEWEkUdo5vk8WT9fn4MRIW48BO/pHhO6ZdtkAwelVBQiTMCmOBK3EgP6vR/4P3bueVE4lQupT0dGuBOBdzewPgfw= Received: from BL0PR02CA0025.namprd02.prod.outlook.com (2603:10b6:207:3c::38) by CH2PR12MB4214.namprd12.prod.outlook.com (2603:10b6:610:aa::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6863.37; Tue, 17 Oct 2023 21:14:21 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::f9) by BL0PR02CA0025.outlook.office365.com (2603:10b6:207:3c::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21 via Frontend Transport; Tue, 17 Oct 2023 21:14:21 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:21 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:20 -0500 From: Felix Kuehling To: , Subject: [PATCH 01/11] drm/amdgpu: Fix possible null pointer dereference Date: Tue, 17 Oct 2023 17:13:27 -0400 Message-ID: <20231017211337.1593869-2-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|CH2PR12MB4214:EE_ X-MS-Office365-Filtering-Correlation-Id: a5bad8f0-b11f-4db5-7787-08dbcf56085f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: dOV8ox7xSzNgRcVXVHv3okWHt0mNFjF6dXG+9zQwIzOZEAvzFDxF4l0iLPe7+Ar9fOdOwBiBEt/lQkpQXWNiuTGjkQ4YkMCNYClpb6nZgOQIrc8p8eTD7csAm+O6zKLHNdooj6vKZuf5C0gr3xrbfPx+JIKvacmNu2KRjreTJSzy8/xXowNTnGDhyy6vLWduGGgrRfFGyJKnCgQFsCPXJcKOXbfSp6D+VLXxw4tWPV2pyzwMbJBg3+Bl2axqY61qLpDyI3VN8bLmd3dfAs1z/P9R8wqTpZb/f0bXgac4vs1pQ0Sk0qVDXui09RuWS/TNsfEgaE4jeebP2hFUdyRguMdLpG9p9Sm+jSWiElb6jcQ5/XgZH7JCIxK9V+KpOMJVNYJ9bwS6/NlVwCZXz7qqezGrWMLkudgoIACb7fNB7r87ws0xF3IgLAB4qPxobC79l/y1q3RBjLgY5y0aKn1WSKN9mF1STO2xI+OuU9r+Mon2E/A0qs6UopX6If9FlbeFaTjp2uePpJf1sEG5TOh0dONtcGDnCzCUxSYyLkRX+KKk9gFH2vziijcG5W0il57YIS6wv//7HVfpf3OwXdUJWGeKKdIU4X6VNEeGzbicWoWoB3M70L4EKhZ5g8sqluPWQW5EaQ9cuWG6cJYVcc4DZwY+/WkWXEms+kwW0+bRY5WEKOL0UyNi1jLobTFSJkLOrwc/Zm3bQl4npqiLup5WgIG2q/Pq9DXT34no327zoR7KPZ6Hds+GMoGjVw9w0MgIV4pR82JUuKbA8zbLazGWUw== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(376002)(396003)(346002)(39860400002)(136003)(230922051799003)(64100799003)(82310400011)(1800799009)(186009)(451199024)(46966006)(36840700001)(40470700004)(110136005)(86362001)(8676002)(450100002)(41300700001)(5660300002)(54906003)(8936002)(70586007)(4326008)(316002)(70206006)(40460700003)(4744005)(2616005)(2906002)(36860700001)(47076005)(356005)(40480700001)(82740400003)(83380400001)(81166007)(16526019)(1076003)(26005)(7696005)(478600001)(426003)(336012)(36756003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:21.7198 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: a5bad8f0-b11f-4db5-7787-08dbcf56085f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4214 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, Christian.Koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" abo->tbo.resource may be NULL in amdgpu_vm_bo_update. Fixes: 180253782038 ("drm/ttm: stop allocating dummy resources during BO creation") Signed-off-by: Felix Kuehling Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 46d27c8ffff7..d72daf15662f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -1007,7 +1007,8 @@ int amdgpu_vm_bo_update(struct amdgpu_device *adev, struct amdgpu_bo_va *bo_va, struct drm_gem_object *gobj = dma_buf->priv; struct amdgpu_bo *abo = gem_to_amdgpu_bo(gobj); - if (abo->tbo.resource->mem_type == TTM_PL_VRAM) + if (abo->tbo.resource && + abo->tbo.resource->mem_type == TTM_PL_VRAM) bo = gem_to_amdgpu_bo(gobj); } mem = bo->tbo.resource; From patchwork Tue Oct 17 21:13:28 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426135 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6BB2FCDB482 for ; Tue, 17 Oct 2023 21:14:43 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1442A10E348; Tue, 17 Oct 2023 21:14:30 +0000 (UTC) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2049.outbound.protection.outlook.com [40.107.212.49]) by gabe.freedesktop.org (Postfix) with ESMTPS id 238DE10E32B; Tue, 17 Oct 2023 21:14:25 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MdWRCKg9G5/6Dgz2cg6H/sIzxjaEwgI9vLOKEFi/dN57gUdajFdWEhGABFpEVyuwwcCs7RwMUDebFP186DuK0RVelUxZgXz4cFUAZn03clFfdIt/rm+xyP5ikbmcfSrECxIPczaFb/R+bfJJdf0vPWFSTzsRNdqF3r9rVkUqmRl14POf3VL66tF6U3sve+NJkqhNkQt31bnAaOp2PHr149rCnf3foRnPIBPEzlVWqzVn/3r7thQXFjbgFt0cFoNjTbVGc34HOstakQjFW59VvMH/Handrir8cH7s6kRs1sh+/b3AeGnqk/ScCnQ2mIONql15KU7z7C+K5LB8knAOVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VJVTvrynJqwestsLwPiHCPDP9UZP765r1KhbIPPWoA4=; b=Sp3e1CWQTmKkS0Sz+HiU3HLf1eTB3zXIhn0JT/lsDbk/0YppgVYmkLAAV7JuKyy2Sntj1ptL/+3NSmPigGrqETrgJznGs3y8EEEJYZ70JAEZajhUthBoCzqcnqlvQrwqFtYDDaegbw3nvtQZbvXTYsu79dzTVjvcBzNQs9ENfT4pnLYXfCW+ddJKgLEnpbLWNXO0x+mOP8gN1vQl/OwnVTiUCz8kVwJl1ALG8r8m5C+3qRtFxNsTt8t6kRcjioSKpJQff0Gd3ej8IiMo6toalygqdDD+8BAj/q+eH68RuMls+I/P4HuI2LrsoQQIUYi6R8q/Z9R6HcHcPUFd2XygQQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VJVTvrynJqwestsLwPiHCPDP9UZP765r1KhbIPPWoA4=; b=LevfPT4P/5jmyQbOLAPCncjxLTmek9jkvVCSUnuMPrIft67NRLLd/WBcYzg3isr/VD+wiihxGDjyEZwcxAJXhNnXQRnE/T2+kxeisGEsEc9wOPtG2YNoa5oS8F3SMvv0BiYqcG1kHoUXUaRi0WzqADxwtxxg2R+HXbFRSPSiD1M= Received: from BL0PR02CA0028.namprd02.prod.outlook.com (2603:10b6:207:3c::41) by PH0PR12MB8174.namprd12.prod.outlook.com (2603:10b6:510:298::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.34; Tue, 17 Oct 2023 21:14:22 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::15) by BL0PR02CA0028.outlook.office365.com (2603:10b6:207:3c::41) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21 via Frontend Transport; Tue, 17 Oct 2023 21:14:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:22 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:20 -0500 From: Felix Kuehling To: , Subject: [PATCH 02/11] drm/amdgpu: Reserve fences for VM update Date: Tue, 17 Oct 2023 17:13:28 -0400 Message-ID: <20231017211337.1593869-3-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|PH0PR12MB8174:EE_ X-MS-Office365-Filtering-Correlation-Id: 1df954c5-9204-48c2-7101-08dbcf56089d X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Pyxn5jg72cXIlmSNX5OEAj20vRQjuNMKCUUmxm4OFtZgz+yxtS2lJRWF0IZEVait2L6UwT4Mhzrgfl9mdgtsCv6wt5pqrKx2rsHiYoD1pS80SpPr3O6YhENL/SaQ1lKgUFkM607zgE/1SVcOz5T5i2c0EoHAzrRNw3SPZzNQ2DYZAVlSb9WmpewKZRxt11WeRt4VjRd+OofEnotVxuK3A2NSb4ptApy/eClHy++j6UCNrNIIEn6LVg6j1v0ij94gonDvhjN4AJGB9qIhBuOaGDRlPnHL/67L8mK3nnpyVAujhLHw8EM2lfhAG94eZ/omCtM7CW1jgYjrR7bkoFs0wu2d3dbm7HD8bXh0H9hP9eY+lpbbgzT0gp9SmzBwVz9HEZxMLqG5vUl7dvcmhVDLyBCWQ9U6nL7z1ZFXIOo0y3Y4bwbgX7TKSx+wlaOgv1iC9BGqD8ITG1x2PhzY1HoNKEq6ENkvkwxE9c3HttWvELhVF/146vtsopfeRPKrMFKp3Pb6Ne9icTxpmzyq8yJ14uROggPJi1wxNrqIqq1L455X4Tcq12g/+2ETRRbpV6PZxbVbU3pn9VStjE7imjVJ2lx4KeIYoxAnAkEl3VFt6/xg3XnTZLDQpUOCN3UTyALdrZ1tAz++nJKfqRE8Dh1yQp2h+5S7VmRBhjCr7nObZvs7BhSyCJZGfqHOcz5ih+/Ayv8EHTHW+C0GL3NVcznZNska3McBMlExUiOa5MzWNIn7clNAXcGXV223lk+2Zq/vu9FKfuRTP6usd4GwLzuyIw== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(39860400002)(376002)(346002)(136003)(396003)(230922051799003)(451199024)(64100799003)(82310400011)(1800799009)(186009)(36840700001)(40470700004)(46966006)(40460700003)(36860700001)(70586007)(70206006)(316002)(110136005)(54906003)(2616005)(1076003)(16526019)(478600001)(26005)(82740400003)(81166007)(8936002)(47076005)(8676002)(356005)(66574015)(7696005)(426003)(4326008)(83380400001)(336012)(450100002)(15650500001)(40480700001)(86362001)(5660300002)(2906002)(36756003)(41300700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:22.1261 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1df954c5-9204-48c2-7101-08dbcf56089d X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8174 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, =?utf-8?q?Christian_K?= =?utf-8?q?=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" In amdgpu_dma_buf_move_notify reserve fences for the page table updates in amdgpu_vm_clear_freed and amdgpu_vm_handle_moved. This fixes a BUG_ON in dma_resv_add_fence when using SDMA for page table updates. Signed-off-by: Felix Kuehling Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c index 76b618735dc0..b5e28fa3f414 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c @@ -404,7 +404,10 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach) continue; } - r = amdgpu_vm_clear_freed(adev, vm, NULL); + /* Reserve fences for two SDMA page table updates */ + r = dma_resv_reserve_fences(resv, 2); + if (!r) + r = amdgpu_vm_clear_freed(adev, vm, NULL); if (!r) r = amdgpu_vm_handle_moved(adev, vm); From patchwork Tue Oct 17 21:13:29 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426142 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id EB540CDB474 for ; Tue, 17 Oct 2023 21:15:01 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CEE6A10E349; Tue, 17 Oct 2023 21:14:48 +0000 (UTC) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2040.outbound.protection.outlook.com [40.107.236.40]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2FED210E33A; Tue, 17 Oct 2023 21:14:27 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NXhwOa4P1rT5hbFvPbFFdwrFsbclpkriCkG+3ATUGLObhDtsFe8NjjE+fGIGTSqTsVCx707aAhGsprE0Z5/YYm6tv4r1TrW8uO1FBYv2tGyrqcslqccR0EBHd3mxE6HT29DLTOtBes77+wjzpj7kV8SxHgYiylk43wAgeRBzHH4loaukXdq4hFroH/CwJCLMHxENYLZ61MYIvhGHr7EhRCeEa+E+jQ4qkzs3r/3UrPc3NspccvbJsRjgWIo3BLplZxN5UC/IT992bu47aGrIzun0owVT0SR5agRsctMb8iQu23Y6lm/PA+6ZB6avZjly1kpclQIjc0qbNACX892c3w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=0WEtX89C7DraaGgxUehf/ObJUB3x2Z1jucUJmibLGio=; b=KuiLVh5xjeXzjxJCyr/TfX0t9p46Qzl4wapOjVevxLSzoaxBZZLnp+JRJRxwlmhvTF6YypF7Nd/eT3Fm9ZjxxTuFlY5W9tDzKBMGNCf2YCIStNySZR2hcIP30lovRFMZn76uQE40JvGjI2v9kqG4mqTbQ+JVKrExw/iO4slmvcylB2izr4WhRI4B5xgUDb5Q/Mg5gI1ptGK1sOni+lVzc8+3IDc8w/c2+tp94DRCQjsw4gUEI/SGhwATEwXZ4nz8jwYrdQ+HzUJx2i8UqPc9odXERMdGhHuWWcDOp4Lfj2ugI1bxgXpFi2UzVpzs9sy83jCRkh08/cm+Odq2mdND+Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0WEtX89C7DraaGgxUehf/ObJUB3x2Z1jucUJmibLGio=; b=QkphjTHkJDb+89ClTmkLIZ01LyCoXlIBB4dRB7Sdqc2W4m+ctbWU+lr1Mx7HtbWyhuBqL1bngbaPpYYvck2UGfJc6DCK5lG2fyb2bjDNA9Z82r+sTbTXVUGJl4SfGkJMOmGmvLbgg2K89+HMLw3kBVTgGmeFVTVqG1azhXqSXpU= Received: from BL0PR02CA0008.namprd02.prod.outlook.com (2603:10b6:207:3c::21) by MW4PR12MB5641.namprd12.prod.outlook.com (2603:10b6:303:186::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.34; Tue, 17 Oct 2023 21:14:23 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::f5) by BL0PR02CA0008.outlook.office365.com (2603:10b6:207:3c::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.36 via Frontend Transport; Tue, 17 Oct 2023 21:14:22 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:22 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:21 -0500 From: Felix Kuehling To: , Subject: [PATCH 03/11] drm/amdkfd: Improve amdgpu_vm_handle_moved Date: Tue, 17 Oct 2023 17:13:29 -0400 Message-ID: <20231017211337.1593869-4-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|MW4PR12MB5641:EE_ X-MS-Office365-Filtering-Correlation-Id: 2fa95757-b709-4603-bb2a-08dbcf560908 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: x7vSbJhLWj5J4LWCIQq+aR5Hmk4CDAfRNbGMeMgyG5fQQJoMw7ZmA5/xgV0ThoBQOXlSV8DETQBEXZhedW0UanWEFIrrR9TJty7rh0kE9okC4K3AB/MHIq7OyRpumtSQP+aK4k/k7bHDfvtLv51dp5ZEEgAmHzPwVUdEqv1zTHi6MzQaDqy0DD8ElEVyJOvPDb4m9vvaSKNea/yIsxtuODzKSCL+IpV0+khjJVeBxZzZP7peD9hKDtAN3sHC7bnj+G33I0EERk2X/E+yMikNpgDj1pY+KHVl3CKxdWhDlnnbCC9f1yruxjwU+TqIt1AKyPgcOQVUVnPlkzg0KQapdmjvip+ROnz3Nf8BFPw3fR2ij+DT6QVKFZCRjFfXV/dUie8kIPyRe8TCKNK1vCz5+/ZS8XLPYCwKHiec6YL254B5dW9UhI3srfmX1C7geD9pnoJiey5hk+4g5i0/RP7zyLh32d91tDAIbG7j3H8NZZ+Kr5IkVry6A0+AlTh0IugcwW6Ahc8b4IpV5XLzJUlnsKwNd7fNPJjwTrFWEYHOOOwcYhDKQBf9BUGB5Kru2MtsqKJuKVjlEe2/aJnS118SzBzHmjbwWr5DBQyHYxqUvZTt0MTAhGlva6NsE4+CLF3fRolY1yxMroH6fRm9cCT07SvHciY9VOahYUkuhJOLKGOOS2SsGbi1wE6R4ix/7cMvvWJnNO/9FcY2ZqFx0YhsGZGaQN/0qcC7m1COv3l0CM7TBpraAlf20KAVMqVZSpBZ5YJ3LK8QlfBHKXgHTOQhAw== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(396003)(376002)(346002)(39860400002)(230922051799003)(1800799009)(186009)(451199024)(82310400011)(64100799003)(40470700004)(46966006)(36840700001)(40460700003)(40480700001)(478600001)(70586007)(54906003)(110136005)(70206006)(47076005)(36860700001)(86362001)(82740400003)(356005)(83380400001)(66574015)(7696005)(316002)(336012)(26005)(16526019)(2616005)(1076003)(4326008)(426003)(81166007)(36756003)(41300700001)(2906002)(8936002)(5660300002)(8676002)(450100002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:22.8292 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 2fa95757-b709-4603-bb2a-08dbcf560908 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB5641 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, =?utf-8?q?Christian_K?= =?utf-8?q?=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Let amdgpu_vm_handle_moved update all BO VA mappings of BOs reserved by the caller. This will be useful for handling extra BO VA mappings in KFD VMs that are managed through the render node API. Signed-off-by: Felix Kuehling Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 22 +-------------------- drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 19 +++++++++++++----- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 3 ++- 4 files changed, 18 insertions(+), 28 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 74769afaa33d..c8f2907ebd6f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -1113,7 +1113,6 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p) struct amdgpu_vm *vm = &fpriv->vm; struct amdgpu_bo_list_entry *e; struct amdgpu_bo_va *bo_va; - struct amdgpu_bo *bo; unsigned int i; int r; @@ -1141,26 +1140,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p) return r; } - amdgpu_bo_list_for_each_entry(e, p->bo_list) { - /* ignore duplicates */ - bo = ttm_to_amdgpu_bo(e->tv.bo); - if (!bo) - continue; - - bo_va = e->bo_va; - if (bo_va == NULL) - continue; - - r = amdgpu_vm_bo_update(adev, bo_va, false); - if (r) - return r; - - r = amdgpu_sync_fence(&p->sync, bo_va->last_pt_update); - if (r) - return r; - } - - r = amdgpu_vm_handle_moved(adev, vm); + r = amdgpu_vm_handle_moved(adev, vm, &p->ticket); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c index b5e28fa3f414..e7e87a3b2601 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c @@ -409,7 +409,7 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach) if (!r) r = amdgpu_vm_clear_freed(adev, vm, NULL); if (!r) - r = amdgpu_vm_handle_moved(adev, vm); + r = amdgpu_vm_handle_moved(adev, vm, ticket); if (r && r != -EBUSY) DRM_ERROR("Failed to invalidate VM page tables (%d))\n", diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index d72daf15662f..c586d0e93d75 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -1285,6 +1285,7 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, * * @adev: amdgpu_device pointer * @vm: requested vm + * @ticket: optional reservation ticket used to reserve the VM * * Make sure all BOs which are moved are updated in the PTs. * @@ -1294,11 +1295,12 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, * PTs have to be reserved! */ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, - struct amdgpu_vm *vm) + struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket) { struct amdgpu_bo_va *bo_va; struct dma_resv *resv; - bool clear; + bool clear, unlock; int r; spin_lock(&vm->status_lock); @@ -1321,17 +1323,24 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, spin_unlock(&vm->status_lock); /* Try to reserve the BO to avoid clearing its ptes */ - if (!adev->debug_vm && dma_resv_trylock(resv)) + if (!adev->debug_vm && dma_resv_trylock(resv)) { clear = false; + unlock = true; + /* The caller is already holding the reservation lock */ + } else if (ticket && dma_resv_locking_ctx(resv) == ticket) { + clear = false; + unlock = false; /* Somebody else is using the BO right now */ - else + } else { clear = true; + unlock = false; + } r = amdgpu_vm_bo_update(adev, bo_va, clear); if (r) return r; - if (!clear) + if (unlock) dma_resv_unlock(resv); spin_lock(&vm->status_lock); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index 6e71978db13f..ebcc75132b74 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -432,7 +432,8 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, struct amdgpu_vm *vm, struct dma_fence **fence); int amdgpu_vm_handle_moved(struct amdgpu_device *adev, - struct amdgpu_vm *vm); + struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket); void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, struct amdgpu_vm *vm, struct amdgpu_bo *bo); int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm, From patchwork Tue Oct 17 21:13:30 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426134 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9A9A3CDB474 for ; Tue, 17 Oct 2023 21:14:39 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4B72B10E33E; Tue, 17 Oct 2023 21:14:28 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2061.outbound.protection.outlook.com [40.107.220.61]) by gabe.freedesktop.org (Postfix) with ESMTPS id 40D3F10E332; Tue, 17 Oct 2023 21:14:26 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lS6sFe3DZPDmb0s0LO9KmdVyMUlki8uGfkDrKbt5TTnmRNKjLaejRyZCAS4jDjX1dLBdfcfH56lPPOMnYv7TgL23ed6fzv28o1Dr4SKhLNib+Nec5UJOUEgnAUlGM37SAPWqaUE+n01W4LoqjtUzJujZ+wKr7R7beJ2aMoUhgrU5FOAVUSHdiD8enhMozzF0I5FALL5KsmkyarpOKghOXqpQibgBrJXDeMI+R5NmV8PfcIn5RT0JrL+iWlSKDJziDVCyFEv4vntcGgak/WDZnMUHnM68YfF5N2146xdecsjAIr2oc8Mqb0d9xhytlO1JmpkCp57IcsB/gbg/gmOLXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=c3lDHuqCY7ieEjAggHysdKnqNvMxZgEGftYuBviYuIM=; b=NP2I/TNrKnxKPtmv2Rlnlhhac/pGSLXhAGZgMafgO0tdNBXo2eJfZ6stMAe/wcBQwABesL4REdg3OOGfju+7AzhY0m/uBAml0yRjRx+KJHsNVxrZHuGgPGw1yGobP4/Ljz97tLP9ySgYuHO3VcYlChj36EZpWh7pNhkeUomIaOi54uuvFHhAahO+9mjtipZv4yhSvHpUDdZHwpFm9O7GZGIGDUnTOumnNa00LVzbOVGhMMH+sujRW0W1p4vPR4Wm/13YbzGxWOeL3/LSTipaZgCNFX+yBVHykck2x5ND8YlTjBm55qkrAFvkCYuSRNFRh4WCD3xn5DuKlZT1yOXWMw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=c3lDHuqCY7ieEjAggHysdKnqNvMxZgEGftYuBviYuIM=; b=HYlfNUke8o9Dc5MjZXDYHO1Be0CfshqGluwjNoqb0kSerG+1DXtB0UDNwTlnGgaCTlFct6bapPQ0f+Cn9zMefbWf3ncBZ1GVXyH9tTqPxvtjpXsEzorOLgyuvNaFLkgmCw3DXE15M6D9tb71DDFJjqleoqRThxcP+Qlp2yaeNqs= Received: from BL0PR02CA0025.namprd02.prod.outlook.com (2603:10b6:207:3c::38) by CO6PR12MB5492.namprd12.prod.outlook.com (2603:10b6:5:35d::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.35; Tue, 17 Oct 2023 21:14:24 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::b1) by BL0PR02CA0025.outlook.office365.com (2603:10b6:207:3c::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21 via Frontend Transport; Tue, 17 Oct 2023 21:14:23 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:23 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:22 -0500 From: Felix Kuehling To: , Subject: [PATCH 04/11] drm/amdgpu: Attach eviction fence on alloc Date: Tue, 17 Oct 2023 17:13:30 -0400 Message-ID: <20231017211337.1593869-5-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|CO6PR12MB5492:EE_ X-MS-Office365-Filtering-Correlation-Id: 55d558d7-3a33-4204-c893-08dbcf560997 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5VmCZexjBaJznfKWDRIhFEc6Ea2thRPhVj8j73RYYXyUkaHmGpwhs3+tlrJ1GXwDzIYnelT2r2D+Vi/9v/U8gwOKPIEz0I3RDI4eKvSTc5fZB89TwTd5U8vkRnfdX5NHwGknQkvJIMukvDtey5Dws5+i2oLxCzZFQWpgYVCs6Vrc94J1HrjtpY8pJaSKWsLcYu1Ia3muEDt9rlmJ0k2E6oD+iA/jQFsWLq3gAobGxhcu8c6cLfZRJLE+KHnuDD4TGTyj3/AAIdljO3shVicMMFqyY4LfuayTIx3xlceB2EQFnKO3K9hGX6OOnjF5pg9rDoFyhaLDQN0NZbUvmANvLMfiljc+gW3Ct8eNB5qjBCybEnkTgiLWKrjGqyMw0Z4oQi3dgcB+YYg8DEkxW2uoSwuMeMiPc/J3jSfwYTeJLb1cu4WSF4cBvRnJo59oj6i+FqSNylSaXMjIkReyT2bT5cwTm/KFMxz7YbwYyEjtQOyoajoDeJ3KR5kH/fKn90qSYX6I3Bh6V7H1Q+jGblWzIelTzwcgihnnG26/dweowPukBOXS/vs9bEpCW9ymDC3O2MTmNJ2vlm9zd08Nxik+ZFgRAt1Uz5s18Mn1phYoU1EgYtb2yNR1QloEvoLcaf7ggAVZ8hWfJKp3G6NS7n3BZ3eMdqmmrW0uhvrSMeayUWzI5nTHRp0g/VWcxncWBNMk6qDUJB4ryHf6WXovi1A6CnJ28XgZj0rVlUDwO/T0N6abaF0vz/D5uoD8KTf3vcoDCdQKQdseD6pcEZtYm/vTrA== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(376002)(39860400002)(396003)(346002)(230922051799003)(186009)(64100799003)(1800799009)(82310400011)(451199024)(40470700004)(46966006)(36840700001)(70206006)(1076003)(70586007)(478600001)(2616005)(426003)(316002)(110136005)(66574015)(16526019)(7696005)(2906002)(54906003)(8676002)(4326008)(5660300002)(450100002)(41300700001)(8936002)(26005)(86362001)(82740400003)(47076005)(6666004)(36756003)(83380400001)(36860700001)(81166007)(356005)(336012)(40460700003)(40480700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:23.7823 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 55d558d7-3a33-4204-c893-08dbcf560997 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO6PR12MB5492 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, =?utf-8?q?Christian_K?= =?utf-8?q?=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Instead of attaching the eviction fence when a KFD BO is first mapped, attach it when it is allocated or imported. This in preparation to allow KFD BOs to be mapped using the render node API. Signed-off-by: Felix Kuehling Acked-by: Christian König --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 79 +++++++++++-------- 1 file changed, 48 insertions(+), 31 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 54f31a420229..7c29f6c377a8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -423,6 +423,32 @@ static int amdgpu_amdkfd_bo_validate(struct amdgpu_bo *bo, uint32_t domain, return ret; } +static int amdgpu_amdkfd_bo_validate_and_fence(struct amdgpu_bo *bo, + uint32_t domain, + struct dma_fence *fence) +{ + int ret = amdgpu_bo_reserve(bo, false); + + if (ret) + return ret; + + ret = amdgpu_amdkfd_bo_validate(bo, domain, true); + if (ret) + goto unreserve_out; + + ret = dma_resv_reserve_fences(bo->tbo.base.resv, 1); + if (ret) + goto unreserve_out; + + dma_resv_add_fence(bo->tbo.base.resv, fence, + DMA_RESV_USAGE_BOOKKEEP); + +unreserve_out: + amdgpu_bo_unreserve(bo); + + return ret; +} + static int amdgpu_amdkfd_validate_vm_bo(void *_unused, struct amdgpu_bo *bo) { return amdgpu_amdkfd_bo_validate(bo, bo->allowed_domains, false); @@ -1831,6 +1857,15 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( } bo->allowed_domains = AMDGPU_GEM_DOMAIN_GTT; bo->preferred_domains = AMDGPU_GEM_DOMAIN_GTT; + } else { + mutex_lock(&avm->process_info->lock); + if (avm->process_info->eviction_fence && + !dma_fence_is_signaled(&avm->process_info->eviction_fence->base)) + ret = amdgpu_amdkfd_bo_validate_and_fence(bo, domain, + &avm->process_info->eviction_fence->base); + mutex_unlock(&avm->process_info->lock); + if (ret) + goto err_validate_bo; } if (offset) @@ -1840,6 +1875,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( allocate_init_user_pages_failed: err_pin_bo: +err_validate_bo: remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); drm_vma_node_revoke(&gobj->vma_node, drm_priv); err_node_allow: @@ -1915,10 +1951,6 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( if (unlikely(ret)) return ret; - /* The eviction fence should be removed by the last unmap. - * TODO: Log an error condition if the bo still has the eviction fence - * attached - */ amdgpu_amdkfd_remove_eviction_fence(mem->bo, process_info->eviction_fence); pr_debug("Release VA 0x%llx - 0x%llx\n", mem->va, @@ -2047,19 +2079,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( if (unlikely(ret)) goto out_unreserve; - if (mem->mapped_to_gpu_memory == 0 && - !amdgpu_ttm_tt_get_usermm(bo->tbo.ttm)) { - /* Validate BO only once. The eviction fence gets added to BO - * the first time it is mapped. Validate will wait for all - * background evictions to complete. - */ - ret = amdgpu_amdkfd_bo_validate(bo, domain, true); - if (ret) { - pr_debug("Validate failed\n"); - goto out_unreserve; - } - } - list_for_each_entry(entry, &mem->attachments, list) { if (entry->bo_va->base.vm != avm || entry->is_mapped) continue; @@ -2086,10 +2105,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( mem->mapped_to_gpu_memory); } - if (!amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) && !bo->tbo.pin_count) - dma_resv_add_fence(bo->tbo.base.resv, - &avm->process_info->eviction_fence->base, - DMA_RESV_USAGE_BOOKKEEP); ret = unreserve_bo_and_vms(&ctx, false, false); goto out; @@ -2123,7 +2138,6 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( struct amdgpu_device *adev, struct kgd_mem *mem, void *drm_priv) { struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); - struct amdkfd_process_info *process_info = avm->process_info; unsigned long bo_size = mem->bo->tbo.base.size; struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; @@ -2164,15 +2178,6 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( mem->mapped_to_gpu_memory); } - /* If BO is unmapped from all VMs, unfence it. It can be evicted if - * required. - */ - if (mem->mapped_to_gpu_memory == 0 && - !amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm) && - !mem->bo->tbo.pin_count) - amdgpu_amdkfd_remove_eviction_fence(mem->bo, - process_info->eviction_fence); - unreserve_out: unreserve_bo_and_vms(&ctx, false, false); out: @@ -2400,8 +2405,20 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, amdgpu_sync_create(&(*mem)->sync); (*mem)->is_imported = true; + mutex_lock(&avm->process_info->lock); + if (avm->process_info->eviction_fence && + !dma_fence_is_signaled(&avm->process_info->eviction_fence->base)) + ret = amdgpu_amdkfd_bo_validate_and_fence(bo, (*mem)->domain, + &avm->process_info->eviction_fence->base); + mutex_unlock(&avm->process_info->lock); + if (ret) + goto err_remove_mem; + return 0; +err_remove_mem: + remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); + drm_vma_node_revoke(&obj->vma_node, drm_priv); err_free_mem: kfree(*mem); err_put_obj: From patchwork Tue Oct 17 21:13:31 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426136 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9F53FCDB485 for ; Tue, 17 Oct 2023 21:14:49 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0FD0310E335; Tue, 17 Oct 2023 21:14:37 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam04on2044.outbound.protection.outlook.com [40.107.100.44]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5C57710E341; Tue, 17 Oct 2023 21:14:28 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=C7PRxcX1D0CsfLwN6XsrCBRCU9DFM5kYUVQRIfghkR6PS0ghCVgkVrd49RY+WypnDU4ei+jIilv+JOQGWm+Iku1Sk7T66AD7HKyrfqeCTQNpPbItke1s4AN+2abvxRc4WVizK2x/xZ/aQUQ4VJDjZroBTkyEtCGVGhK+BFBIKeR+fn/tFUanEuA2KLM57lxsEb/4trdNoegYfCsoTwq9/gFwNlNPQwcGcZqEvdl95n670sCCTG4mwA7x7KJq0fKKM6Lzxa11Dhd4ZhQEtqD7CXzVSZihatgpvPGqLl2gtOxYU+y1432dZgFGYtALic0MuSNXbLQe0xqZzJ6+6/CxqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=+kWWqUPGmqJVUgYDSMPAAJ1OfD4ffYOtd8NRANLghYA=; b=f9XOwzvFlZ7f+DWlWk830dbVbyoJCIu8aE9afd6P7GyDMgaTqHlMadqHMwFHK3V2+bT0MHYbppl8mt0xj9uiD7UaBsrS0Iq0x7KSyxfH0YbwDyVEbNZOkFa90spcboy6EgJqdO6n8WUegqymdiMyLZL7Zub/AEQUc6ZqmipirawZ6CDJKZpCo8kyz3buXS0R9Up8tAUVZKAAAdbJzLpF84NcdUI6lk0TaytOFsfkNTD5S7wAYQ5Zq+/mcXvrujwRa2ERvuCOrixnQB7VLYo+VGtP4XN4pv87vfuqedkfy3BT1EN6/PlsNPnksBbtX3pY647O4Rx9saqS7Hd/IZzMhQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=+kWWqUPGmqJVUgYDSMPAAJ1OfD4ffYOtd8NRANLghYA=; b=d8S5qXWyDRYgmOIRyJM4g3MbMQjg1C2rTEF0aYayMStuCvBDsKOXm/nqNcjbv6zNs1iX7CmERVrwHIwx7z3uGQRfBpNChPaQrZE9Vkzw1r4G8aiai+On/crYHKlnRhyrtVVDVKeS8ghzXSvqQPCvliv92BaPt+7spre1AH4reWQ= Received: from BL0PR02CA0004.namprd02.prod.outlook.com (2603:10b6:207:3c::17) by DM4PR12MB5867.namprd12.prod.outlook.com (2603:10b6:8:66::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.34; Tue, 17 Oct 2023 21:14:25 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::c9) by BL0PR02CA0004.outlook.office365.com (2603:10b6:207:3c::17) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.36 via Frontend Transport; Tue, 17 Oct 2023 21:14:24 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:24 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:22 -0500 From: Felix Kuehling To: , Subject: [PATCH 05/11] drm/amdgpu: update mappings not managed by KFD Date: Tue, 17 Oct 2023 17:13:31 -0400 Message-ID: <20231017211337.1593869-6-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|DM4PR12MB5867:EE_ X-MS-Office365-Filtering-Correlation-Id: 4cb45e49-39bd-43c4-9ac6-08dbcf560a2b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ObFmGHleIo1wzHP/lXTn7rx5rW5q3ennUYoLiOXb1QpPJOYipAI4moa/pz43aro1l4Gu41lkY+fuKA8s2ToVkrLJD0YMV7MxA8RKoBz7bf0nR5aMkzmNj2eiJb8+Agw9bYaZbcrcCN/J7HfV0aBQ3q/xXxBuNfK8tw9+H+uSG3E0eZGXx1DEkVEGWEWVq45f0Y5agqKqNKTKGVjlr5porhyAQo8ZxD7y1ok0fID0y/ZrKfeIae/BGtx93pgOxe9t4HvCExpIpC8WahDBV6CtcydUBas7q+CX1I6X65Tg33SpL82g8nq13ATiwfeKCii0Ii1bZSJzjQ0BKdUrhzbURc/z8FhfVzvmEi+ez2XYEaNtNyj8xKQlV6nwFDuZW2DBVw1Yh7ESdIbotzELeT75k5ZrIOZNpDlWsYmhLwl2wQNKr2n0Wtpj9aeO3Zp1pWNXTHjH4MyxCSwvalOeZtL+LfqOBD643QeqcezTauxHkZ3DnBd5Ef0783hnUk/dpfBURc9uax/XtcySicFWRps9llukrO8MoBPuoNobeeXtRoGOjMyzassFM1jQZ+v2iq7HQiSufn9+Iz6V213JInTjQwxDg0uvPIzrpzDdDzxTsI42OjliSpbOkkxGwXNp8kP3xYTS3PgZltnMQ9KosrpBh52PF+U8auOmv1Xv+kiZ6PH0L5sY+Cr/vdiDeLIYmMN6ArmVE9hu6biZvZTWjixc+p5EpjJfnRhOEtirftT/1D2MwBCAe/nNwNuFhuvRSx4eVWhSNIffNk32g3ta9JvkNg== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(346002)(39860400002)(396003)(376002)(230922051799003)(186009)(82310400011)(1800799009)(64100799003)(451199024)(46966006)(40470700004)(36840700001)(40460700003)(70206006)(316002)(70586007)(110136005)(16526019)(1076003)(66574015)(40480700001)(86362001)(426003)(36756003)(83380400001)(82740400003)(356005)(47076005)(81166007)(36860700001)(2616005)(478600001)(6666004)(7696005)(54906003)(41300700001)(5660300002)(15650500001)(26005)(336012)(2906002)(450100002)(8676002)(4326008)(8936002)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:24.7355 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 4cb45e49-39bd-43c4-9ac6-08dbcf560a2b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5867 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, =?utf-8?q?Christian_K?= =?utf-8?q?=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" When restoring after an eviction, use amdgpu_vm_handle_moved to update BO VA mappings in KFD VMs that are not managed through the KFD API. This should allow using the render node API to create more flexible memory mappings in KFD VMs. Signed-off-by: Felix Kuehling Acked-by: Christian König --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 28 +++++++++++++++---- 1 file changed, 22 insertions(+), 6 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 7c29f6c377a8..2e302956a279 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -2892,12 +2892,6 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) if (ret) goto validate_map_fail; - ret = process_sync_pds_resv(process_info, &sync_obj); - if (ret) { - pr_debug("Memory eviction: Failed to sync to PD BO moving fence. Try again\n"); - goto validate_map_fail; - } - /* Validate BOs and map them to GPUVM (update VM page tables). */ list_for_each_entry(mem, &process_info->kfd_bo_list, validate_list.head) { @@ -2948,6 +2942,19 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) if (failed_size) pr_debug("0x%lx/0x%lx in system\n", failed_size, total_size); + /* Update mappings not managed by KFD */ + list_for_each_entry(peer_vm, &process_info->vm_list_head, + vm_list_node) { + struct amdgpu_device *adev = amdgpu_ttm_adev( + peer_vm->root.bo->tbo.bdev); + + ret = amdgpu_vm_handle_moved(adev, peer_vm, &ctx.ticket); + if (ret) { + pr_debug("Memory eviction: handle moved failed. Try again\n"); + goto validate_map_fail; + } + } + /* Update page directories */ ret = process_update_pds(process_info, &sync_obj); if (ret) { @@ -2955,6 +2962,15 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) goto validate_map_fail; } + /* Sync with fences on all the page tables. They implicitly depend on any + * move fences from amdgpu_vm_handle_moved above. + */ + ret = process_sync_pds_resv(process_info, &sync_obj); + if (ret) { + pr_debug("Memory eviction: Failed to sync to PD BO moving fence. Try again\n"); + goto validate_map_fail; + } + /* Wait for validate and PT updates to finish */ amdgpu_sync_wait(&sync_obj, false); From patchwork Tue Oct 17 21:13:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426137 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0E8C3CDB474 for ; Tue, 17 Oct 2023 21:14:52 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CB4ED10E33D; Tue, 17 Oct 2023 21:14:37 +0000 (UTC) Received: from NAM04-DM6-obe.outbound.protection.outlook.com (mail-dm6nam04on2076.outbound.protection.outlook.com [40.107.102.76]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7F7BE10E356; Tue, 17 Oct 2023 21:14:29 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lmYgYR6O/eAuAI97dddKpe+WESrHXNrJbzA40ZirWY0VKY4vGHQNcAQe31flP9pVNp6Ez+KF/uMjWGxdzYDrF1MmntaE+BEAkA+i3877QiMkTv+nWRdHqD/UFMW7UzGy/cc9SCiUg04gsf6bahCW4OUeUiacEwOh0awEedzw1QPDdb0guqfag0Hf4c2fsqbC54Tvlp/BzTcDqoWBxKSu/jfEbGiGqZ5nHc4pFUIcc0VLF2TxfQL4SFGu53h4xKztsq2P5BfilzznURMQ2pi/1LNNTGQ76qfunfKxzmRICF6CAD3Z1BtITNyp/vdNZ0W2GeperWku4PdIUerhso1t4w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=VYHATuho7LcB17bwU2rZ0BhICtqsy2qiH73WbiWrZDY=; b=N0NDDA+b/+gwWB45HNJBR6x/pFpbXatxrHkfm1Q551sqsvivF9iAQhmy6oJ9M/pBruUMEGa1Rzp6rd107RTZMn/DfPdqi5tOwFI4qCyBdwyM7iMNJERfhd/T000S8aj06Qrt2yT9xu3SqwZTQXaaxy+tvlrU921SpQZMbLnj4YQhrqYINKkXhff8rmtnOVdlXFQSPxb64G+1JlqK72x/F4+i/0xm1pmSuzxPHnqiWra/4Ni+/jy2CQto9R5f26CseoDFSxA/Feuu0QJ9cXygSTOun3Jo6OTNPymRMzmX5Ab0vv37szdorMvwwHbLqg4l6ZDfEyuptwIVeoyBQXpxiw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VYHATuho7LcB17bwU2rZ0BhICtqsy2qiH73WbiWrZDY=; b=vq2N4VyZ7R0WlWRXkM8QNB9ZBkiFukBiYCVXGJFOSxSdQ9VVRdzBv/6IuE5Dn6kSSM9gzIG3XU6TR+BjW7eOb5emSW7FQOeUAo/80DLxQdHJZy28W0WCSG+hQzNBUYZTmk628LWOX3uJ/WUtk8iT/NSaOEgqQL7FDU+d6hcGxws= Received: from BL0PR02CA0026.namprd02.prod.outlook.com (2603:10b6:207:3c::39) by MN0PR12MB6198.namprd12.prod.outlook.com (2603:10b6:208:3c5::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21; Tue, 17 Oct 2023 21:14:25 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::46) by BL0PR02CA0026.outlook.office365.com (2603:10b6:207:3c::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21 via Frontend Transport; Tue, 17 Oct 2023 21:14:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:25 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:23 -0500 From: Felix Kuehling To: , Subject: [PATCH 06/11] drm/amdkfd: Move TLB flushing logic into amdgpu Date: Tue, 17 Oct 2023 17:13:32 -0400 Message-ID: <20231017211337.1593869-7-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|MN0PR12MB6198:EE_ X-MS-Office365-Filtering-Correlation-Id: 9c696bf0-c21c-4414-0d2c-08dbcf560a7f X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 9PeK7AuwLT7rr+DmA7UD1eW782axb+rtDToNpzegFgWMG/LuYGwDMeM0gZuHBr7OUWGBuSA6RBFjLVCsEjDPa8uPmLYcLG5laZhgAU4f6aR7qvYW82/hfeetDp8mjH8zNrvps3nxOHvuhYFxa2Jh7K7iqrl/WHJN36Jfk7CfPbynud8GeeHuhSYwhx90y7449DqSzxjOpTCBqTi4/M4hzrL+NndITCnQ42d5QpXxrLlRQdi28j0ezthTx0l77feYgbUikGbemkJKhgK2icOTq1psICVjwQQYsKVeY7HC39sciIO7g2WgIWUSqojcbZGj2makhn4IRVBhnwCcI7H4S6gKRjkaY+WE0u885bUtSVntP6Z2+Z/sn7JjR4Bf/D68ainCM6cwC+zggQS4/32iIElynu98TISIIO22k3LYBDeaZ7sTIO8kGdVeycvW971VRSi1kkv/pJOsDrNNzEgY0/4kt2lVi35iqVoQMjy3lkGm8Pd/PeCR/llRUvRIoxf21fu0ksc2SSXGq8cqDjsPO8XeYj3stgK/mwjsDGc6in6+fL3ZDKrMoh/MZkbQKYHqWbz8feqeiGGx9X11iivDw4BUfmDPTRxXY2VLzE/36fWBJ1ovxCO69gs21j2+c0wEVlLhmGWvNV/urgNwm4yaDz+Akpv7/f3oR18I0dK9SKhOemWj16w9JwROvasrwForn0Tb+6aV/rKGDKTCnpfNztBWmJdqt8WLGlkkdtZR/Iauv6rtSt26Pe9+lYh9El9dkXYW5GxUSWhR67mgM6yipw== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(376002)(39860400002)(346002)(396003)(230922051799003)(451199024)(64100799003)(82310400011)(186009)(1800799009)(40470700004)(46966006)(36840700001)(40460700003)(26005)(2616005)(1076003)(66574015)(7696005)(6666004)(16526019)(426003)(8936002)(316002)(36860700001)(336012)(83380400001)(5660300002)(4326008)(41300700001)(8676002)(2906002)(47076005)(478600001)(70586007)(70206006)(450100002)(54906003)(110136005)(82740400003)(356005)(81166007)(86362001)(40480700001)(36756003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:25.2824 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9c696bf0-c21c-4414-0d2c-08dbcf560a7f X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6198 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, =?utf-8?q?Christian_K?= =?utf-8?q?=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This will make it possible for amdgpu GEM ioctls to flush TLBs on compute VMs. This removes VMID-based TLB flushing and always uses PASID-based flushing. This still works because it scans the VMID-PASID mapping registers to find the right VMID. It's only slightly less efficient. This is not a production use case. Signed-off-by: Felix Kuehling Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 29 -------------- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 5 --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 44 ++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 5 +++ drivers/gpu/drm/amd/amdkfd/kfd_priv.h | 10 ++++- drivers/gpu/drm/amd/amdkfd/kfd_process.c | 31 --------------- 6 files changed, 57 insertions(+), 67 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index b8412202a1b0..6ab17330a6ed 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -710,35 +710,6 @@ bool amdgpu_amdkfd_is_kfd_vmid(struct amdgpu_device *adev, u32 vmid) return false; } -int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct amdgpu_device *adev, - uint16_t vmid) -{ - if (adev->family == AMDGPU_FAMILY_AI) { - int i; - - for_each_set_bit(i, adev->vmhubs_mask, AMDGPU_MAX_VMHUBS) - amdgpu_gmc_flush_gpu_tlb(adev, vmid, i, 0); - } else { - amdgpu_gmc_flush_gpu_tlb(adev, vmid, AMDGPU_GFXHUB(0), 0); - } - - return 0; -} - -int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct amdgpu_device *adev, - uint16_t pasid, - enum TLB_FLUSH_TYPE flush_type, - uint32_t inst) -{ - bool all_hub = false; - - if (adev->family == AMDGPU_FAMILY_AI || - adev->family == AMDGPU_FAMILY_RV) - all_hub = true; - - return amdgpu_gmc_flush_gpu_tlb_pasid(adev, pasid, flush_type, all_hub, inst); -} - bool amdgpu_amdkfd_have_atomics_support(struct amdgpu_device *adev) { return adev->have_atomics_support; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 3ad8dc523b42..fcf8a98ad15e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -163,11 +163,6 @@ int amdgpu_amdkfd_submit_ib(struct amdgpu_device *adev, uint32_t *ib_cmd, uint32_t ib_len); void amdgpu_amdkfd_set_compute_idle(struct amdgpu_device *adev, bool idle); bool amdgpu_amdkfd_have_atomics_support(struct amdgpu_device *adev); -int amdgpu_amdkfd_flush_gpu_tlb_vmid(struct amdgpu_device *adev, - uint16_t vmid); -int amdgpu_amdkfd_flush_gpu_tlb_pasid(struct amdgpu_device *adev, - uint16_t pasid, enum TLB_FLUSH_TYPE flush_type, - uint32_t inst); bool amdgpu_amdkfd_is_kfd_vmid(struct amdgpu_device *adev, u32 vmid); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index c586d0e93d75..3307c5765787 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -1349,6 +1349,50 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, return 0; } +/** + * amdgpu_vm_flush_compute_tlb - Flush TLB on compute VM + * + * @adev: amdgpu_device pointer + * @vm: requested vm + * @flush_type: flush type + * + * Flush TLB if needed for a compute VM. + * + * Returns: + * 0 for success. + */ +int amdgpu_vm_flush_compute_tlb(struct amdgpu_device *adev, + struct amdgpu_vm *vm, + uint32_t flush_type, + uint32_t xcc_mask) +{ + uint64_t tlb_seq = amdgpu_vm_tlb_seq(vm); + bool all_hub = false; + int xcc = 0, r = 0; + + WARN_ON_ONCE(!vm->is_compute_context); + + /* + * It can be that we race and lose here, but that is extremely unlikely + * and the worst thing which could happen is that we flush the changes + * into the TLB once more which is harmless. + */ + if (atomic64_xchg(&vm->kfd_last_flushed_seq, tlb_seq) == tlb_seq) + return 0; + + if (adev->family == AMDGPU_FAMILY_AI || + adev->family == AMDGPU_FAMILY_RV) + all_hub = true; + + for_each_inst(xcc, xcc_mask) { + r = amdgpu_gmc_flush_gpu_tlb_pasid(adev, vm->pasid, flush_type, + all_hub, xcc); + if (r) + break; + } + return r; +} + /** * amdgpu_vm_bo_add - add a bo to a specific vm * diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index ebcc75132b74..577cdb6d1649 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -316,6 +316,7 @@ struct amdgpu_vm { /* Last finished delayed update */ atomic64_t tlb_seq; struct dma_fence *last_tlb_flush; + atomic64_t kfd_last_flushed_seq; /* Last unlocked submission to the scheduler entities */ struct dma_fence *last_unlocked; @@ -434,6 +435,10 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, int amdgpu_vm_handle_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket); +int amdgpu_vm_flush_compute_tlb(struct amdgpu_device *adev, + struct amdgpu_vm *vm, + uint32_t flush_type, + uint32_t xcc_mask); void amdgpu_vm_bo_base_init(struct amdgpu_vm_bo_base *base, struct amdgpu_vm *vm, struct amdgpu_bo *bo); int amdgpu_vm_update_range(struct amdgpu_device *adev, struct amdgpu_vm *vm, diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h index 9cc32f577e38..a40f8cfc6aa5 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h +++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h @@ -748,7 +748,6 @@ struct kfd_process_device { /* VM context for GPUVM allocations */ struct file *drm_file; void *drm_priv; - atomic64_t tlb_seq; /* GPUVM allocations storage */ struct idr alloc_idr; @@ -1462,7 +1461,14 @@ void kfd_signal_reset_event(struct kfd_node *dev); void kfd_signal_poison_consumed_event(struct kfd_node *dev, u32 pasid); -void kfd_flush_tlb(struct kfd_process_device *pdd, enum TLB_FLUSH_TYPE type); +static inline void kfd_flush_tlb(struct kfd_process_device *pdd, + enum TLB_FLUSH_TYPE type) +{ + struct amdgpu_device *adev = pdd->dev->adev; + struct amdgpu_vm *vm = drm_priv_to_vm(pdd->drm_priv); + + amdgpu_vm_flush_compute_tlb(adev, vm, type, pdd->dev->xcc_mask); +} static inline bool kfd_flush_tlb_after_unmap(struct kfd_dev *dev) { diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c index fbf053001af9..cc3c2d024618 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c @@ -1662,7 +1662,6 @@ int kfd_process_device_init_vm(struct kfd_process_device *pdd, return ret; } pdd->drm_priv = drm_file->private_data; - atomic64_set(&pdd->tlb_seq, 0); ret = kfd_process_device_reserve_ib_mem(pdd); if (ret) @@ -2054,36 +2053,6 @@ int kfd_reserved_mem_mmap(struct kfd_node *dev, struct kfd_process *process, KFD_CWSR_TBA_TMA_SIZE, vma->vm_page_prot); } -void kfd_flush_tlb(struct kfd_process_device *pdd, enum TLB_FLUSH_TYPE type) -{ - struct amdgpu_vm *vm = drm_priv_to_vm(pdd->drm_priv); - uint64_t tlb_seq = amdgpu_vm_tlb_seq(vm); - struct kfd_node *dev = pdd->dev; - uint32_t xcc_mask = dev->xcc_mask; - int xcc = 0; - - /* - * It can be that we race and lose here, but that is extremely unlikely - * and the worst thing which could happen is that we flush the changes - * into the TLB once more which is harmless. - */ - if (atomic64_xchg(&pdd->tlb_seq, tlb_seq) == tlb_seq) - return; - - if (dev->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS) { - /* Nothing to flush until a VMID is assigned, which - * only happens when the first queue is created. - */ - if (pdd->qpd.vmid) - amdgpu_amdkfd_flush_gpu_tlb_vmid(dev->adev, - pdd->qpd.vmid); - } else { - for_each_inst(xcc, xcc_mask) - amdgpu_amdkfd_flush_gpu_tlb_pasid( - dev->adev, pdd->process->pasid, type, xcc); - } -} - /* assumes caller holds process lock. */ int kfd_process_drain_interrupts(struct kfd_process_device *pdd) { From patchwork Tue Oct 17 21:13:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426138 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1ECC1CDB482 for ; Tue, 17 Oct 2023 21:14:54 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2906F10E33F; Tue, 17 Oct 2023 21:14:38 +0000 (UTC) Received: from NAM02-BN1-obe.outbound.protection.outlook.com (mail-bn1nam02on2044.outbound.protection.outlook.com [40.107.212.44]) by gabe.freedesktop.org (Postfix) with ESMTPS id DCF4E10E34B; Tue, 17 Oct 2023 21:14:28 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bINwgYcktmYkGp7J0h8ofJ8nrH8rYDpHEdCHhcgVvhyAHDTsrUVODa/bWtBrZm1hnG6mHzAdLznaSUUVKuYaK5lz3RIkpb3HptnvFXXuYHX2+gn825JShNUavkdgrttffLUFf+FwirTiiurREree4D542HprBOirsUUkQi77TFuVqnGRsfME/m55EKcy3uHp+43EAQ78dmDN61tml+crdH+Hl1Kg//JIn3/tpqkNC09DIH54ahN+DjpUWXwBfaA84vZLuKEzsjalHAvgE1Solyf4o5s/vWWNsbb0XMTB9cFO4DywVuMSADMxWncIIjxq03Yg2ca6QVpuloxlglIByw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ybSQ5oCn8Zm6aBUJ8GvK4Sp4Cf/hYG179lNhSGon098=; b=HovVEJWIG65W/a5tagLdw6lBpC7k2Yd6oHbKJCIdYsZ+q2nqnq+SlS/z+uTaf7VtkduLAYEo5kq1vba8dxy5F17l7IAD/eWcKPQjRp9DSzpJExqP65jzsfx+TCh4ZYfhgIAswEiUhcLtYbh14t/9n4mMKiMdUxGEv7V02Be1igmuRZ5uzMvBiXcoKVj+JwyQbOyHuyRNo9UYCWTnUfX6+8UZPmUJv/VRskD8Z8Iy1CY9I+xn09TpXSZSsF+gIUz9KULdOz+edu7omF7ph5tHr+n79kJ88KashP5Ao7W8lDI4zao6EhieT64zLD8vXf3/7DrZERRfLiFJZldPkL2Esw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ybSQ5oCn8Zm6aBUJ8GvK4Sp4Cf/hYG179lNhSGon098=; b=D4f3QwZ5hShX4Ju9DhyEaNquKolScTcbcibAwGZhqa3gbsq3NAqgqly5mKldDUzt94rmJwoiqEuEVpcqWMQFlMhIDOE5FKo7Wg7ONgZBWLHDxukU0r6F9pFEnglnv6gLVmw0Rt//q0w+ljWsBeFDprSddmeQi6uHJLA+uZWcbQk= Received: from BL0PR02CA0029.namprd02.prod.outlook.com (2603:10b6:207:3c::42) by PH0PR12MB8174.namprd12.prod.outlook.com (2603:10b6:510:298::6) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.34; Tue, 17 Oct 2023 21:14:26 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::85) by BL0PR02CA0029.outlook.office365.com (2603:10b6:207:3c::42) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21 via Frontend Transport; Tue, 17 Oct 2023 21:14:25 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:25 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:24 -0500 From: Felix Kuehling To: , Subject: [PATCH 07/11] drm/amdgpu: New VM state for evicted user BOs Date: Tue, 17 Oct 2023 17:13:33 -0400 Message-ID: <20231017211337.1593869-8-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|PH0PR12MB8174:EE_ X-MS-Office365-Filtering-Correlation-Id: 755a6d6d-7cc9-445e-bb03-08dbcf560acd X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JS3vj1BcVCloW0CAj2OHkb09ed8bMDmefVLPRpMt5IqbP58uMJK/8ZKKgqUTkJmzRsMXOH0EDb9Sd264jS0RGTprS52V4Unk1ZACEXwsxm8yANN5V/rQwIT99x5UtHssr+8ZPLikUoZN3LrXP6ney66UMv+UzjACdTIa7Nk68VUf/TAs7OKo6LTXD6krrML3ebsYuWSjwHBhqaFIL+1dgaYsN3PFPO6xMi6DNabTsCFg8LCBzY4fnSE+WQ41fWFdWqu8meDArpWltWzUJ/IDR7V2PdMZz02h3UjTzxi2BCjqC7h5fupMlJVghMvb1OscPLJ7mrapqf9PFak9TiqmDfmrKulOEvBlPvLgtilGn7/pSDGxGAwQ5eB/cDgqtCn+ui7w1MyRZjAtI7rKrBvDD1aQ2CQ4ZAIMRrHPvZ7kE6HDmY8nErfsyfnFuJR5uiydKwg8pJKY4uaY05GTdpTtDw4zydbGeRB/fLUC6KGykoHxQepierVwyzm8t8XQBeqRxg42EJzb77Eh6WSs5GMydeIIdiEkQhhsW3A8efE9vsccY69njOrP9Q8QfrZwNNF4pJ/gihfCJGGzwupJOhYj39nHgBX9x3pKT1qfXkUrxL9h+xNnGg6tJRbpxrx9n77QxleRj2jAnyR1OKfY41B8sMKH2NZ6MfYcgOpF1FGEw9ciJtfEZZHZDcs1NFW01Idx1eAL4QBaSP05Tg/Obc3S9yLn3rNU8LB6zFTKcO19Fu3UT/1r/D3xagQqejl8TnJKVqZyuu2J6qN2Lh0HFmBDNA== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(39860400002)(376002)(346002)(136003)(396003)(230922051799003)(451199024)(64100799003)(82310400011)(1800799009)(186009)(36840700001)(40470700004)(46966006)(40460700003)(6666004)(36860700001)(70586007)(70206006)(316002)(110136005)(54906003)(2616005)(1076003)(16526019)(478600001)(26005)(82740400003)(81166007)(8936002)(47076005)(8676002)(356005)(7696005)(426003)(4326008)(83380400001)(336012)(450100002)(40480700001)(86362001)(5660300002)(2906002)(36756003)(41300700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:25.7980 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 755a6d6d-7cc9-445e-bb03-08dbcf560acd X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR12MB8174 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, Christian.Koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Create a new VM state to track user BOs that are in the system domain. In the next patch this will be used do conditionally re-validate them in amdgpu_vm_handle_moved. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 17 +++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 5 ++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 3307c5765787..76a8a7fd3fde 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -232,6 +232,22 @@ static void amdgpu_vm_bo_invalidated(struct amdgpu_vm_bo_base *vm_bo) spin_unlock(&vm_bo->vm->status_lock); } +/** + * amdgpu_vm_bo_evicted_user - vm_bo is evicted + * + * @vm_bo: vm_bo which is evicted + * + * State for BOs used by user mode queues which are not at the location they + * should be. + */ +static void amdgpu_vm_bo_evicted_user(struct amdgpu_vm_bo_base *vm_bo) +{ + vm_bo->moved = true; + spin_lock(&vm_bo->vm->status_lock); + list_move(&vm_bo->vm_status, &vm_bo->vm->evicted_user); + spin_unlock(&vm_bo->vm->status_lock); +} + /** * amdgpu_vm_bo_relocated - vm_bo is reloacted * @@ -2105,6 +2121,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm, int32_t xcp for (i = 0; i < AMDGPU_MAX_VMHUBS; i++) vm->reserved_vmid[i] = NULL; INIT_LIST_HEAD(&vm->evicted); + INIT_LIST_HEAD(&vm->evicted_user); INIT_LIST_HEAD(&vm->relocated); INIT_LIST_HEAD(&vm->moved); INIT_LIST_HEAD(&vm->idle); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index 577cdb6d1649..914e6753a6d0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -280,9 +280,12 @@ struct amdgpu_vm { /* Lock to protect vm_bo add/del/move on all lists of vm */ spinlock_t status_lock; - /* BOs who needs a validation */ + /* Per VM and PT BOs who needs a validation */ struct list_head evicted; + /* BOs for user mode queues that need a validation */ + struct list_head evicted_user; + /* PT BOs which relocated and their parent need an update */ struct list_head relocated; From patchwork Tue Oct 17 21:13:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426141 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 98475CDB483 for ; Tue, 17 Oct 2023 21:15:00 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 02F3C10E344; Tue, 17 Oct 2023 21:14:47 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2054.outbound.protection.outlook.com [40.107.94.54]) by gabe.freedesktop.org (Postfix) with ESMTPS id A7ADB10E347; Tue, 17 Oct 2023 21:14:28 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GyvRIm5FVLaNyzwddL+qvC/P845GmgCFvhCXlMh9MNlayuisgOVohjl47GlcqLa0a/aZmX4WDnr2kIc8cjKlmxThy/YmQ/C2L5fvkDKqlU3PX32/+2o9y4S8DGC39x9++wCRZwTUTmoHyStmtahgctHb3nyB9s8Uxnjb3NblG73USCQpCNJZKSfv79ZgbtggHK+0oAc33wOiJ0AatN5nHa5RCymp1+mJb7Ens2K7n9QWg/XzCa8Iu0xlEHIhg4y8vTEQTUbaiCVxEUo/rZ3mp/DdVI3fpl+G33pC9l4DbuiyAPJoNXEaSkAD24VdSx8rtFd939riXTN8Of/uh4L9dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=rGff5szKP5NsgfocFUyE1/1xaP+2wL+SBHpXMqfioRM=; b=VFn1fzWvQ4hAtnCgkceEQXRJDSiKMKRchb8ljSICZBCl/DeZa00lCP5+xoqMFIupV/cTZjDyQk1NopRdtSxotmoNMSugIwQW9EPqaFfZ15mqLnEtdF7bwmixXq0Atxt88y0+bD/VGIY6W8nkuPMUClTOJKcOLYeVnZq9W7GeczDSKcmCfKt3vnN9YIWhbmBeAJ//McDEuvqVcLXaTiKQ7cqYrVPxxz9NNmvvzrCo62+6Mml6cdyDJ+LoQZqG3XWuU/X813Y0EUNabGN17F/SFY6ZwGQ0k0O/FK0JN6dLNAKOIRgHteH8gv9gljl56GehSNusGmCEKHGpvJ94pshBVg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=rGff5szKP5NsgfocFUyE1/1xaP+2wL+SBHpXMqfioRM=; b=StXvVaM7a88umK0BLTBBXhyQYv2MkWpkgy3ArLFxcGtkTHy5jHzdMGNBZGqxZZyaoyinB8O44P/nRRAsha8xOzdo+yjZoPIjiS4WxeIUVViUIckEPmWDXtAJNsLhWDc2LRwJFAVLRxOKfAcY0Fz0LlBmYNqfTw+X2JU5NCEb87A= Received: from BL0PR02CA0003.namprd02.prod.outlook.com (2603:10b6:207:3c::16) by DM4PR12MB5961.namprd12.prod.outlook.com (2603:10b6:8:68::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21; Tue, 17 Oct 2023 21:14:26 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::9e) by BL0PR02CA0003.outlook.office365.com (2603:10b6:207:3c::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6907.21 via Frontend Transport; Tue, 17 Oct 2023 21:14:26 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:26 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:24 -0500 From: Felix Kuehling To: , Subject: [PATCH 08/11] drm/amdgpu: Auto-validate DMABuf imports in compute VMs Date: Tue, 17 Oct 2023 17:13:34 -0400 Message-ID: <20231017211337.1593869-9-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|DM4PR12MB5961:EE_ X-MS-Office365-Filtering-Correlation-Id: f74480a0-30f4-4ba4-6755-08dbcf560b3b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SqrqERqLRNdxNdoyIqZVp6A7m35yNJYvnMhvJT2gUMZBs20XaWWWDghNX59jqCinWnq9tUiy4FKqokGBt+T9bCRdbK3doWaqoKpO6+l6gKAZCSG5WTN5RURca5q3nMHdwvvbR/pHEn2uWYl1xaylXoUP7kpY4iD7pl99Ktk5RATCPO7Ne3Bqto38qtkd6xTrp5+zywc2zrlyLSx4z9ayshhpCpnzwauKTW4/HKjuMB6VgKQa0Cyg+zkNDdgfISs9SYXocWAz6H29mA904id4tDxiJYFMpwxSH6TdFOzJkrFD55XI9lIp/nbMhO2FliyQo6dpoLE74s+wbZFR4wKTi0qm/qj844jy/l7I39vBfZTsUGJmWJG+KZIva2UjsqYPkZeO3cc71Gndxub+LcG5j9l22t6DeFDCWE8SlM5x7sv3BTVoR6RfsMCVJZf5ovhfmzipyWCQVSvcRbT56pm+j86293Uxh50rf+mI0ZZQWvR5lHmefHiGcntUhH/fuaBtpQUmGoIBmLqwD7TpVaYxx7p51hKFsGO9MaVUlxoL6pQG7zpmw0s87p3mHgqWre1DZ9yHFPl/5NEMwlzuU2oZovzy6kWvRvzgrHI3tJKYK9EwXwuhh7g0OGRhUJHwb3Hiz9otYhrxgDVF+6r9ZRHtBAr2W4Ygvvzij1E8VlPEcJKaj3k5aITsKGo2ktdHajjyELWU8ebSJJ+75mcTx+xV//hruoNSUv9viPawZkvM+OZxq3ANUWJmp349b/jId7MIfgMbvR1xH3tpcKcds9brGg== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(39860400002)(396003)(346002)(376002)(230922051799003)(64100799003)(186009)(451199024)(1800799009)(82310400011)(36840700001)(40470700004)(46966006)(6666004)(16526019)(336012)(426003)(7696005)(478600001)(47076005)(36860700001)(110136005)(70586007)(70206006)(316002)(36756003)(2616005)(1076003)(83380400001)(26005)(54906003)(450100002)(41300700001)(5660300002)(82740400003)(356005)(81166007)(30864003)(40460700003)(8676002)(8936002)(4326008)(15650500001)(2906002)(40480700001)(86362001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:26.5167 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: f74480a0-30f4-4ba4-6755-08dbcf560b3b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB5961 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, Christian.Koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" DMABuf imports in compute VMs are not wrapped in a kgd_mem object on the process_info->kfd_bo_list. There is no explicit KFD API call to validate them or add eviction fences to them. This patch automatically validates and fences dymanic DMABuf imports when they are added to a compute VM. Revalidation after evictions is handled in the VM code. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 3 + .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 15 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 26 ++++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 117 +++++++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 6 +- 7 files changed, 164 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index fcf8a98ad15e..68d534a89942 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -178,6 +178,9 @@ int amdgpu_queue_mask_bit_to_set_resource_bit(struct amdgpu_device *adev, struct amdgpu_amdkfd_fence *amdgpu_amdkfd_fence_create(u64 context, struct mm_struct *mm, struct svm_range_bo *svm_bo); +int amdgpu_amdkfd_bo_validate_and_fence(struct amdgpu_bo *bo, + uint32_t domain, + struct dma_fence *fence); #if defined(CONFIG_DEBUG_FS) int kfd_debugfs_kfd_mem_limits(struct seq_file *m, void *data); #endif diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 2e302956a279..0c1cb6048259 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -423,9 +423,9 @@ static int amdgpu_amdkfd_bo_validate(struct amdgpu_bo *bo, uint32_t domain, return ret; } -static int amdgpu_amdkfd_bo_validate_and_fence(struct amdgpu_bo *bo, - uint32_t domain, - struct dma_fence *fence) +int amdgpu_amdkfd_bo_validate_and_fence(struct amdgpu_bo *bo, + uint32_t domain, + struct dma_fence *fence) { int ret = amdgpu_bo_reserve(bo, false); @@ -2948,7 +2948,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) struct amdgpu_device *adev = amdgpu_ttm_adev( peer_vm->root.bo->tbo.bdev); - ret = amdgpu_vm_handle_moved(adev, peer_vm, &ctx.ticket); + ret = amdgpu_vm_handle_moved(adev, peer_vm, &ctx.ticket, true); if (ret) { pr_debug("Memory eviction: handle moved failed. Try again\n"); goto validate_map_fail; @@ -3001,7 +3001,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) &process_info->eviction_fence->base, DMA_RESV_USAGE_BOOKKEEP); } - /* Attach eviction fence to PD / PT BOs */ + /* Attach eviction fence to PD / PT BOs and DMABuf imports */ list_for_each_entry(peer_vm, &process_info->vm_list_head, vm_list_node) { struct amdgpu_bo *bo = peer_vm->root.bo; @@ -3009,6 +3009,11 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) dma_resv_add_fence(bo->tbo.base.resv, &process_info->eviction_fence->base, DMA_RESV_USAGE_BOOKKEEP); + + ret = amdgpu_vm_fence_imports(peer_vm, &ctx.ticket, + &process_info->eviction_fence->base); + if (ret) + break; } validate_map_fail: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index c8f2907ebd6f..e6dcd17c3c8e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -1140,7 +1140,7 @@ static int amdgpu_cs_vm_handling(struct amdgpu_cs_parser *p) return r; } - r = amdgpu_vm_handle_moved(adev, vm, &p->ticket); + r = amdgpu_vm_handle_moved(adev, vm, &p->ticket, false); if (r) return r; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c index e7e87a3b2601..234244704f27 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c @@ -373,6 +373,10 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach) struct amdgpu_vm_bo_base *bo_base; int r; + /* FIXME: This should be after the "if", but needs a fix to make sure + * DMABuf imports are initialized in the right VM list. + */ + amdgpu_vm_bo_invalidate(adev, bo, false); if (!bo->tbo.resource || bo->tbo.resource->mem_type == TTM_PL_SYSTEM) return; @@ -409,7 +413,7 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach) if (!r) r = amdgpu_vm_clear_freed(adev, vm, NULL); if (!r) - r = amdgpu_vm_handle_moved(adev, vm, ticket); + r = amdgpu_vm_handle_moved(adev, vm, ticket, false); if (r && r != -EBUSY) DRM_ERROR("Failed to invalidate VM page tables (%d))\n", diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index 849fffbb367d..755cc3c559f5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -186,6 +186,32 @@ static int amdgpu_gem_object_open(struct drm_gem_object *obj, else ++bo_va->ref_count; amdgpu_bo_unreserve(abo); + + /* Validate and add eviction fence to DMABuf imports with dymanic + * attachment in compute VMs. Re-validation will be done by + * amdgpu_vm_handle_moved and the fence will be updated by + * amdgpu_vm_fence_imports in amdgpu_amdkfd_gpuvm_restore_process_bos. + * + * Nested locking below for the case that a GEM object is opened in + * kfd_mem_export_dmabuf. Since the lock below is only taken for imports, + * but not for export, this is a different lock class that cannot lead to + * circular lock dependencies. + */ + if (!vm->is_compute_context || !vm->process_info) + return 0; + if (!obj->import_attach || + !dma_buf_is_dynamic(obj->import_attach->dmabuf)) + return 0; + mutex_lock_nested(&vm->process_info->lock, 1); + if (!WARN_ON(!vm->process_info->eviction_fence)) { + r = amdgpu_amdkfd_bo_validate_and_fence(abo, AMDGPU_GEM_DOMAIN_GTT, + &vm->process_info->eviction_fence->base); + if (r) + dev_warn(adev->dev, "%d: validate_and_fence failed: %d\n", + vm->task_info.pid, r); + } + mutex_unlock(&vm->process_info->lock); + return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 76a8a7fd3fde..872fa62aea23 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -1302,6 +1302,7 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, * @adev: amdgpu_device pointer * @vm: requested vm * @ticket: optional reservation ticket used to reserve the VM + * @valitate: whether to auto-validate invalid DMABuf imports * * Make sure all BOs which are moved are updated in the PTs. * @@ -1312,7 +1313,8 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, */ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm, - struct ww_acquire_ctx *ticket) + struct ww_acquire_ctx *ticket, + bool validate) { struct amdgpu_bo_va *bo_va; struct dma_resv *resv; @@ -1332,6 +1334,12 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, spin_lock(&vm->status_lock); } + /* If we're validating user BOs, splice all evicted user BOs into + * the list of invalid BOs for revalidation + */ + if (validate) + list_splice_init(&vm->evicted_user, &vm->invalidated); + while (!list_empty(&vm->invalidated)) { bo_va = list_first_entry(&vm->invalidated, struct amdgpu_bo_va, base.vm_status); @@ -1352,17 +1360,120 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, unlock = false; } + /* Automatically validate DMABuf imports in compute VMs, if we + * have a reservation, or remember them for later validation. + */ + if (vm->is_compute_context && bo_va->base.bo && + bo_va->base.bo->tbo.base.import_attach && + (!bo_va->base.bo->tbo.resource || + bo_va->base.bo->tbo.resource->mem_type == TTM_PL_SYSTEM)) { + struct ttm_operation_ctx ctx = { true, false }; + struct amdgpu_bo *bo = bo_va->base.bo; + + if (!validate) { + r = amdgpu_vm_bo_update(adev, bo_va, clear); + if (!r) + amdgpu_vm_bo_evicted_user(&bo_va->base); + goto unlock; + } + + if (clear) { + pr_warn_ratelimited("Invalid DMABuf import is busy in pid %d\n", vm->task_info.pid); + r = -EBUSY; + goto unlock; + } + + amdgpu_bo_placement_from_domain(bo, + bo->preferred_domains); + r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + if (r) + goto unlock; + r = amdgpu_bo_sync_wait(bo, AMDGPU_FENCE_OWNER_KFD, + true); + if (r) + goto unlock; + } + r = amdgpu_vm_bo_update(adev, bo_va, clear); +unlock: + if (unlock) + dma_resv_unlock(resv); if (r) return r; + spin_lock(&vm->status_lock); + } + spin_unlock(&vm->status_lock); + + return 0; +} + +/** + * amdgpu_vm_fence_imports - add fence to valid DMABuf imports + * + * @vm: requested vm + * @ticket: optional reservation ticket used to reserve the VM + * @fence: fence to add + * + * Add the specified fence to all dymanic DMABuf imports that are valid. + * + * Returns: + * 0 for success. + */ +int amdgpu_vm_fence_imports(struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket, + struct dma_fence *fence) +{ + struct amdgpu_bo_va *bo_va, *tmp; + struct dma_resv *resv; + LIST_HEAD(imports); + bool unlock; + int ret = 0; + + if (!vm->is_compute_context) + return 0; + + /* Move all the DMABuf imports to a private list so I can reserve + * them while not holding te status_lock. + */ + spin_lock(&vm->status_lock); + list_for_each_entry_safe(bo_va, tmp, &vm->idle, base.vm_status) { + if (bo_va->base.bo && bo_va->base.bo->tbo.base.import_attach && + dma_buf_is_dynamic(bo_va->base.bo->tbo.base.import_attach->dmabuf)) + list_move(&bo_va->base.vm_status, &imports); + } + spin_unlock(&vm->status_lock); + + list_for_each_entry(bo_va, &imports, base.vm_status) { + resv = bo_va->base.bo->tbo.base.resv; + + /* Try to reserve the BO */ + if (dma_resv_trylock(resv)) { + unlock = true; + /* The caller is already holding the reservation lock */ + } else if (ticket && dma_resv_locking_ctx(resv) == ticket) { + unlock = false; + } else { + WARN_ONCE(1, "Failed to reserve DMABuf import"); + ret = -EBUSY; + break; + } + + ret = dma_resv_reserve_fences(resv, 1); + if (!ret) + dma_resv_add_fence(resv, fence, + DMA_RESV_USAGE_BOOKKEEP); if (unlock) dma_resv_unlock(resv); - spin_lock(&vm->status_lock); + if (ret) + break; } + + spin_lock(&vm->status_lock); + list_splice(&imports, &vm->idle); spin_unlock(&vm->status_lock); - return 0; + return ret; } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index 914e6753a6d0..522c89eccac5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -437,7 +437,11 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, struct dma_fence **fence); int amdgpu_vm_handle_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm, - struct ww_acquire_ctx *ticket); + struct ww_acquire_ctx *ticket, + bool validate); +int amdgpu_vm_fence_imports(struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket, + struct dma_fence *fence); int amdgpu_vm_flush_compute_tlb(struct amdgpu_device *adev, struct amdgpu_vm *vm, uint32_t flush_type, From patchwork Tue Oct 17 21:13:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426139 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id AA788CDB482 for ; Tue, 17 Oct 2023 21:14:56 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 43EAF10E340; Tue, 17 Oct 2023 21:14:38 +0000 (UTC) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2071.outbound.protection.outlook.com [40.107.95.71]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1BFE610E356; Tue, 17 Oct 2023 21:14:30 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=lhxnQab07CPLx0MWd1IeUxJHHeBMf/6uCMfQaaW66pbd2/atB8cfI67QdqnvMPkaNtzrwvMr5VtjX3s64lnBulMLXkQEvD8/E0Px6QamUV7OSwxIB21HgL+aGFFbxPCzh2wSQhfDiIpMfM6imvQOVn/uJovOY8kXc+pFZZ+LWaHJBpcsoLAkF/aw4U72VJd88DnmisGg0RZDq/3DZgBftR7KvuuuyLowHO8E1IRaAhDjFzN7bcITdtKk0Z8ZOqCJJHJnR6mwixhm05emUxRTg1EehhqFOYO+TFKyLfe36tQ9YW1OOxOqmILpRSYswk174BZjdlXiLlw/TuP32ryGeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=dom0y6XUtYM7kJpD3mXnBYOs7Zy+QpGkNG6z8LLSnQQ=; b=IF2NDcmL2nEZYHQobIgso2kCZ9uw3qgzl/JL1EDlT+n4nTYQgVUgJQlHpwEwUhgMNhYTqqO453FqbTw3PHhBfxVbs0w2mNhz4X4FEZcNx+1OMNCjWkBo5RaOVGJJdwfPyoAveqJT3rsfh9JsaUUSJiuIUuiNq3/NdFh3pfr3h9LsyyCJD+5OTvytVdgzH5GgUUEt75cp4faaJlcDkztxOFqFFICCGn+vK3VoduhRTTQWGvuuHjX8/URqM6u8lbAfYOg4T3qaDDKR4jOov9Uce9EGd85U+HGc3lggqI6jyZcMEilPilvI7D28OxHk98MRSUd9LSqQL/ceJyYx+QKVHA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=dom0y6XUtYM7kJpD3mXnBYOs7Zy+QpGkNG6z8LLSnQQ=; b=SkivtWvG/fTxakxYTG93pjwlEksTmowOOQkAEEIzF8ekb7mOm9TDdExnC7bV5ktjo+RoYxVcX4agRHjjjFS0fAN3mSf7x9Hfy7TrnQ3Wp1N0easQ7/gWV5+l88o9uxwFp9NrL5/oBvjobEZRNRSHV2z6pphD95EACBiMpeNTW+Y= Received: from BL0PR02CA0030.namprd02.prod.outlook.com (2603:10b6:207:3c::43) by PH7PR12MB8425.namprd12.prod.outlook.com (2603:10b6:510:240::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.35; Tue, 17 Oct 2023 21:14:28 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::65) by BL0PR02CA0030.outlook.office365.com (2603:10b6:207:3c::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.35 via Frontend Transport; Tue, 17 Oct 2023 21:14:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:27 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:25 -0500 From: Felix Kuehling To: , Subject: [PATCH 09/11] drm/amdkfd: Export DMABufs from KFD using GEM handles Date: Tue, 17 Oct 2023 17:13:35 -0400 Message-ID: <20231017211337.1593869-10-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|PH7PR12MB8425:EE_ X-MS-Office365-Filtering-Correlation-Id: d38e1a10-cdde-4744-c031-08dbcf560c16 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3H1/pQUVeG8BCmPCYrSZ3/P3aNL84c461TOIwJT47TTkWbRD8vmm2th3alfw21QwpuESgnOnZMawt9vKj5JMKG2i4tO4bUrEkpn4hEvyfyaxf13PJJTEI9c2QGKODSKprUDHNFamO0FO+eyWzIzHTHMn3Zjt/gVHgvka30DhTD0q7Ty9Kar+FdgnIoKzbz8TynJiSPhPapZQj1up1DV586sBqZXXshkPptIaToAUZiNgl/dMi0AsJDHUKIVUzcBacAVx+MzjGVYqMC2kTQrQc2krdiMcucO6M7hDJB4KH+ZYIzBLTL/1YYurB1q+p2LSZWpFlZi6zvN+66QdxObLBW7bMrfKdpoXOG/gYqTuhpoS2FyKPX2GmxrZnlvXKEjCOL9TsYqE1OOwEo+RzKZFnzhRz4i4gLOZ0dEOmREqrNfyNk9UbP5r7Ic98wCKA0084YETibjEK8+LV2hqVYIhxDebAlnPHGlfPIjibad1j+fe4WrSHJjKQzVu83iL2WfNP8ZS7IZjZ1/EfcENbaspeul+a9LWLxQvki4yZ62OY5Sa+xDcgLFwQ4zKYcxgYNqjNE0ho2j2Di3gJD6AdsuUz4+zI+fVn2X9gZLWHGcoFOw7GNO/amTu5caoxuy7aCPfXr2+anELEEOfAN4qxV7ZU4hrRnvVoPS6klbiNKFB3sQd5c8euxOZUm3f2D1YeKBbqj7VWI7DBhMIhN2MPXGFokwb7LEPlkxsCEh+grO6C0X6EVNwa7njBwCXaSejCdxcamoTQXJQdzTLs3DzKjEBIQ== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(39860400002)(346002)(396003)(136003)(376002)(230922051799003)(64100799003)(1800799009)(186009)(82310400011)(451199024)(46966006)(40470700004)(36840700001)(54906003)(478600001)(110136005)(47076005)(70586007)(70206006)(6666004)(26005)(16526019)(41300700001)(1076003)(336012)(316002)(426003)(7696005)(8936002)(4326008)(2906002)(8676002)(450100002)(5660300002)(36756003)(81166007)(86362001)(83380400001)(36860700001)(356005)(82740400003)(2616005)(40480700001)(40460700003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:27.9543 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d38e1a10-cdde-4744-c031-08dbcf560c16 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB8425 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, Christian.Koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Create GEM handles for exporting DMABufs using GEM-Prime APIs. The GEM handles are created in a drm_client_dev context to avoid exposing them in user mode contexts through a DMABuf import. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 11 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 5 +++ .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 33 +++++++++++++++---- drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 4 +-- 4 files changed, 44 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 6ab17330a6ed..b0a67f16540a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -142,6 +142,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) { int i; int last_valid_bit; + int ret; amdgpu_amdkfd_gpuvm_init_mem_limits(); @@ -160,6 +161,12 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) .enable_mes = adev->enable_mes, }; + ret = drm_client_init(&adev->ddev, &adev->kfd.client, "kfd", NULL); + if (ret) { + dev_err(adev->dev, "Failed to init DRM client: %d\n", ret); + return; + } + /* this is going to have a few of the MSBs set that we need to * clear */ @@ -198,6 +205,10 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) adev->kfd.init_complete = kgd2kfd_device_init(adev->kfd.dev, &gpu_resources); + if (adev->kfd.init_complete) + drm_client_register(&adev->kfd.client); + else + drm_client_release(&adev->kfd.client); amdgpu_amdkfd_total_mem_size += adev->gmc.real_vram_size; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 68d534a89942..4caf8cece028 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -32,6 +32,7 @@ #include #include #include +#include #include #include "amdgpu_sync.h" #include "amdgpu_vm.h" @@ -84,6 +85,7 @@ struct kgd_mem { struct amdgpu_sync sync; + uint32_t gem_handle; bool aql_queue; bool is_imported; }; @@ -106,6 +108,9 @@ struct amdgpu_kfd_dev { /* HMM page migration MEMORY_DEVICE_PRIVATE mapping */ struct dev_pagemap pgmap; + + /* Client for KFD BO GEM handle allocations */ + struct drm_client_dev client; }; enum kgd_engine_type { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 0c1cb6048259..4bb8b5fd7598 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include "amdgpu_object.h" @@ -804,13 +805,22 @@ kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, static int kfd_mem_export_dmabuf(struct kgd_mem *mem) { if (!mem->dmabuf) { - struct dma_buf *ret = amdgpu_gem_prime_export( - &mem->bo->tbo.base, + struct amdgpu_device *bo_adev; + struct dma_buf *dmabuf; + int r, fd; + + bo_adev = amdgpu_ttm_adev(mem->bo->tbo.bdev); + r = drm_gem_prime_handle_to_fd(&bo_adev->ddev, bo_adev->kfd.client.file, + mem->gem_handle, mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? - DRM_RDWR : 0); - if (IS_ERR(ret)) - return PTR_ERR(ret); - mem->dmabuf = ret; + DRM_RDWR : 0, &fd); + if (r) + return r; + dmabuf = dma_buf_get(fd); + close_fd(fd); + if (WARN_ON_ONCE(IS_ERR(dmabuf))) + return PTR_ERR(dmabuf); + mem->dmabuf = dmabuf; } return 0; @@ -1826,6 +1836,9 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( pr_debug("Failed to allow vma node access. ret %d\n", ret); goto err_node_allow; } + ret = drm_gem_handle_create(adev->kfd.client.file, gobj, &(*mem)->gem_handle); + if (ret) + goto err_gem_handle_create; bo = gem_to_amdgpu_bo(gobj); if (bo_type == ttm_bo_type_sg) { bo->tbo.sg = sg; @@ -1877,6 +1890,8 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( err_pin_bo: err_validate_bo: remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); + drm_gem_handle_delete(adev->kfd.client.file, (*mem)->gem_handle); +err_gem_handle_create: drm_vma_node_revoke(&gobj->vma_node, drm_priv); err_node_allow: /* Don't unreserve system mem limit twice */ @@ -1991,8 +2006,12 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( /* Free the BO*/ drm_vma_node_revoke(&mem->bo->tbo.base.vma_node, drm_priv); - if (mem->dmabuf) + if (!mem->is_imported) + drm_gem_handle_delete(adev->kfd.client.file, mem->gem_handle); + if (mem->dmabuf) { dma_buf_put(mem->dmabuf); + mem->dmabuf = NULL; + } mutex_destroy(&mem->lock); /* If this releases the last reference, it will end up calling diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c index 06988cf1db51..4417a9863cd0 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c @@ -1855,8 +1855,8 @@ static uint32_t get_process_num_bos(struct kfd_process *p) return num_of_bos; } -static int criu_get_prime_handle(struct kgd_mem *mem, int flags, - u32 *shared_fd) +static int criu_get_prime_handle(struct kgd_mem *mem, + int flags, u32 *shared_fd) { struct dma_buf *dmabuf; int ret; From patchwork Tue Oct 17 21:13:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426140 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B2742CDB474 for ; Tue, 17 Oct 2023 21:14:58 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CB42910E33B; Tue, 17 Oct 2023 21:14:39 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2056.outbound.protection.outlook.com [40.107.93.56]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8C8BE10E33F; Tue, 17 Oct 2023 21:14:32 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Hiqw2mt89bKBp8tz7qBgT19dm0DGMrZdy8eUMnQK6zvmLrfNyPMxXUW9A4XUDbGM+0QkiTzbuME1PU6pLOpHukMQGVwmCNdaRmS6N7KH0q035pqb4Eaqzxlhwbdc4t/7F8ImEOyzZerTIGAO1NgHJfITO6EpNrPTNnTAbn6UBpBgLxnDggjp8gPMIq1vfFxaPbHUoqvGaCbNOKced/FnqzOTpr/nasV0sw77DDsleIU/ukPzoIKYRhqUpRGMlsU3iITOPuvapQL5yMNAcEGHfMctFKCDUtWf0vCEmWf0DkEgn/o4mezlsi9au9DvqPNdyAR1TArQkEWXuJ91/RVhGA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=y+2oznMHIhZfSt00MUMFTkxyobaIGeG/XJ9ZqRMEJPA=; b=lxjFkGLhmLJHBhK/klHV22LKoItlvDmcN5yKjT2ncBQqPo4c6Xn/PWqp2HYtAE60hgK3fnl9UVpLY2xgiYpFfHV105WRKh9J1WDoBLW7dqbYMoViuBmNrwR2Z5aEJ3e8Wfg4XUsJpuTxjSak1txCQgVIL4XsOWGjBscen93ciCUblzv9Dx8tmP/qi/P7OUvIkPBOAaY0vlTkGR0J5MKSXqj2MSU37frSzxi4uWKpHHLBmL0wRCB2urApMBDCtgsRKjIc2XxYaUOitsm3kSj2yyS8rRQMLXxcaBDx/+hCusLkThCZXjh04/kj3WuoY4sjIMPpFE76rxnvl2kOV3DZJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y+2oznMHIhZfSt00MUMFTkxyobaIGeG/XJ9ZqRMEJPA=; b=fF+Uno89JTI38RUth0hfmhwjVkOp5MSmSblUdkHGYGT7q7qbb252a4lCt7Z9qDjYIOm0H64357oOSpqBH1iaA8LPgmLWLIA6dzi1VJrs1PkSG8+NFSDTGXPmOva6GhIK3z1od6dq0nnODkuAZJmSWEhXGdrP2K98X2O9Qytd1Zo= Received: from BL0PR02CA0002.namprd02.prod.outlook.com (2603:10b6:207:3c::15) by SA1PR12MB6798.namprd12.prod.outlook.com (2603:10b6:806:25a::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.35; Tue, 17 Oct 2023 21:14:28 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::ad) by BL0PR02CA0002.outlook.office365.com (2603:10b6:207:3c::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.36 via Frontend Transport; Tue, 17 Oct 2023 21:14:27 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:27 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:25 -0500 From: Felix Kuehling To: , Subject: [PATCH 10/11] drm/amdkfd: Import DMABufs for interop through DRM Date: Tue, 17 Oct 2023 17:13:36 -0400 Message-ID: <20231017211337.1593869-11-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|SA1PR12MB6798:EE_ X-MS-Office365-Filtering-Correlation-Id: 691ea4a5-b107-4617-c1b3-08dbcf560bf7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: QIiDDYGGBCIPQe/cE0y7Pl+vHpPQbdL5+QS78dpmScLvvMR94401vTQnFoY2xlalrS9qcRWB8Vd1lr0rND2UfSOG1Z2i2fshwQ6b/NlfcytLXyZTbp9HIKUz4vDXIkXHK7HsKR7gvevzZGzRrH4AvS6yMo7J2gG4Bb2IbzNaTozX1ETJvjlQjW6QaDI15sTuU5fTbhmYDTsTPPueMXSVQErnHE/JuNNLOkSdSnw32c/KPQ3yjUDpYS06ROQLsYd4/7IBODXSFOenwMsVLjO8fio2MMS+uzsMuyRjJYXhJbMtzuN0xd7NCwCWa/uaXbHIlAb5kCXhNGWeSmSbmWaHe/M2jUHlhKtmAgnU3a+OXDnXA7ttQMGtdvzKCjPvB/r0OfvPHenB5LHISDXDknksVJ7wPUlkkq0olCPVAlIwFFSvDqOj1EUITEey/NR5fBkIZ0Nfqr0QLXQIDYBJ92zWtk6PH247wcbZDw4Tgfmk6JM6MRo3IgIq/V3HEewYJZngtCznDahg+6fNjXt5GHcd4AVZTAM7IywT8QPkYXYtl8vTixzZvn9qzpXctpZDDpIhg93DNtGOb3X4CxUUHpJ+9+u7JDwe+3lSsQyYcmk3Nt7EfcikFNHLwT5k4LQeyRnDkr0+cK/BuhoydgNp4NhiB/3bf2gDViHEBTqJCt3DS+Mt4Wyx/z/4saGhXV0ifd84kuQ/juPoy0a0viSRWt+aYRJkZ97H9OejRooAX8EmR85JxemNpULtEMD9tn+hyzlO X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(230922051799003)(1800799009)(186009)(82310400011)(451199024)(46966006)(40470700004)(36840700001)(54906003)(110136005)(47076005)(70586007)(70206006)(508600001)(6666004)(26005)(16526019)(1076003)(336012)(316002)(426003)(7696005)(8936002)(4326008)(2906002)(8676002)(450100002)(5660300002)(36756003)(81166007)(86362001)(83380400001)(36860700001)(356005)(2616005)(40460700003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:27.7511 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 691ea4a5-b107-4617-c1b3-08dbcf560bf7 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6798 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, Christian.Koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use drm_gem_prime_fd_to_handle to import DMABufs for interop. This ensures that a GEM handle is created on import and that obj->dma_buf will be set and remain set as long as the object is imported into KFD. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 9 ++- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 64 +++++++++++++------ drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 15 ++--- 3 files changed, 52 insertions(+), 36 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 4caf8cece028..88a0e0734270 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -318,11 +318,10 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *process_info, struct dma_fence **ef); int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct amdgpu_device *adev, struct kfd_vm_fault_info *info); -int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, - struct dma_buf *dmabuf, - uint64_t va, void *drm_priv, - struct kgd_mem **mem, uint64_t *size, - uint64_t *mmap_offset); +int amdgpu_amdkfd_gpuvm_import_dmabuf_fd(struct amdgpu_device *adev, int fd, + uint64_t va, void *drm_priv, + struct kgd_mem **mem, uint64_t *size, + uint64_t *mmap_offset); int amdgpu_amdkfd_gpuvm_export_dmabuf(struct kgd_mem *mem, struct dma_buf **dmabuf); void amdgpu_amdkfd_debug_mem_fence(struct amdgpu_device *adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 4bb8b5fd7598..1077de8bced2 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -2006,8 +2006,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( /* Free the BO*/ drm_vma_node_revoke(&mem->bo->tbo.base.vma_node, drm_priv); - if (!mem->is_imported) - drm_gem_handle_delete(adev->kfd.client.file, mem->gem_handle); + drm_gem_handle_delete(adev->kfd.client.file, mem->gem_handle); if (mem->dmabuf) { dma_buf_put(mem->dmabuf); mem->dmabuf = NULL; @@ -2363,34 +2362,26 @@ int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct amdgpu_device *adev, return 0; } -int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, - struct dma_buf *dma_buf, - uint64_t va, void *drm_priv, - struct kgd_mem **mem, uint64_t *size, - uint64_t *mmap_offset) +static int import_obj_create(struct amdgpu_device *adev, + struct dma_buf *dma_buf, + struct drm_gem_object *obj, + uint64_t va, void *drm_priv, + struct kgd_mem **mem, uint64_t *size, + uint64_t *mmap_offset) { struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); - struct drm_gem_object *obj; struct amdgpu_bo *bo; int ret; - obj = amdgpu_gem_prime_import(adev_to_drm(adev), dma_buf); - if (IS_ERR(obj)) - return PTR_ERR(obj); - bo = gem_to_amdgpu_bo(obj); if (!(bo->preferred_domains & (AMDGPU_GEM_DOMAIN_VRAM | - AMDGPU_GEM_DOMAIN_GTT))) { + AMDGPU_GEM_DOMAIN_GTT))) /* Only VRAM and GTT BOs are supported */ - ret = -EINVAL; - goto err_put_obj; - } + return -EINVAL; *mem = kzalloc(sizeof(struct kgd_mem), GFP_KERNEL); - if (!*mem) { - ret = -ENOMEM; - goto err_put_obj; - } + if (!*mem) + return -ENOMEM; ret = drm_vma_node_allow(&obj->vma_node, drm_priv); if (ret) @@ -2440,8 +2431,41 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, drm_vma_node_revoke(&obj->vma_node, drm_priv); err_free_mem: kfree(*mem); + return ret; +} + +int amdgpu_amdkfd_gpuvm_import_dmabuf_fd(struct amdgpu_device *adev, int fd, + uint64_t va, void *drm_priv, + struct kgd_mem **mem, uint64_t *size, + uint64_t *mmap_offset) +{ + struct drm_gem_object *obj; + uint32_t handle; + int ret; + + ret = drm_gem_prime_fd_to_handle(&adev->ddev, adev->kfd.client.file, fd, + &handle); + if (ret) + return ret; + obj = drm_gem_object_lookup(adev->kfd.client.file, handle); + if (!obj) { + ret = -EINVAL; + goto err_release_handle; + } + + ret = import_obj_create(adev, obj->dma_buf, obj, va, drm_priv, mem, size, + mmap_offset); + if (ret) + goto err_put_obj; + + (*mem)->gem_handle = handle; + + return 0; + err_put_obj: drm_gem_object_put(obj); +err_release_handle: + drm_gem_handle_delete(adev->kfd.client.file, handle); return ret; } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c index 4417a9863cd0..1a2e9f564b7f 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c @@ -1564,16 +1564,11 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, { struct kfd_ioctl_import_dmabuf_args *args = data; struct kfd_process_device *pdd; - struct dma_buf *dmabuf; int idr_handle; uint64_t size; void *mem; int r; - dmabuf = dma_buf_get(args->dmabuf_fd); - if (IS_ERR(dmabuf)) - return PTR_ERR(dmabuf); - mutex_lock(&p->mutex); pdd = kfd_process_device_data_by_id(p, args->gpu_id); if (!pdd) { @@ -1587,10 +1582,10 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, goto err_unlock; } - r = amdgpu_amdkfd_gpuvm_import_dmabuf(pdd->dev->adev, dmabuf, - args->va_addr, pdd->drm_priv, - (struct kgd_mem **)&mem, &size, - NULL); + r = amdgpu_amdkfd_gpuvm_import_dmabuf_fd(pdd->dev->adev, args->dmabuf_fd, + args->va_addr, pdd->drm_priv, + (struct kgd_mem **)&mem, &size, + NULL); if (r) goto err_unlock; @@ -1601,7 +1596,6 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, } mutex_unlock(&p->mutex); - dma_buf_put(dmabuf); args->handle = MAKE_HANDLE(args->gpu_id, idr_handle); @@ -1612,7 +1606,6 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, pdd->drm_priv, NULL); err_unlock: mutex_unlock(&p->mutex); - dma_buf_put(dmabuf); return r; } From patchwork Tue Oct 17 21:13:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13426143 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 29F7DCDB485 for ; Tue, 17 Oct 2023 21:15:03 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8556010E345; Tue, 17 Oct 2023 21:14:48 +0000 (UTC) Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2076.outbound.protection.outlook.com [40.107.237.76]) by gabe.freedesktop.org (Postfix) with ESMTPS id DF11E10E345; Tue, 17 Oct 2023 21:14:31 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=npSQy2tpNGcp86kwlW3PhitNcl2wVrROHyv0jahdtkc4rWkx/Ypp+kecXUlyeIQuEbimSIJYzAIPPCW2+pruKXWhebNMd1T7jsWikymMUw2P81bIcWiAdCFHWuZQFbIkej9pg2U/KiDxF+av0ROdDChdnJz9kk1jRuWwXtY6kmlMlsNHsfN+4adrRFdmfyE8zT6E5whnDELiEIBaTDqJJi0N6eL47SReyVBafW5fX1ZsIogN2cYqINDttUKw17/9hcOUem1R4pikUEH8QVUKxv89+iMtSug71GrF4loUzb3oe/eJ4p716fe9ElB/pTgRI3xe7RzTeJIPFZA6zEIhmw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Aq1H8WCJuAL37SctBUPuPK3Kqv6ruPc+2YmbD5MHcdI=; b=UfK3A17K5nh6FrL3Fm0S5mWMgwMtgVgF52uMEDv2PHy1qYFF8+zsokQC6Yvw1Ys1NjQseNyD2eTcrgb2vHmTTr3S+LJBYs1sUOESWMZJZr8h10e5iZ/S58SFfSYUgNwk/AD/DhRzRAYn7BFtQyQnwFe7HTiGuRa5VhrACfNW7tEv3ynVTxMuvzGWQJXXOt03wwTHVfOaBFnFA7ub8gJyt4tIUuF23I+phB0v9jeUiRM0dlgIj8IlA+EP9lFvoP2svdmoIpqLxZAGwDLvqKPQ/j/vMn2CghiUEjX1S2uI1BL8SD6G86i0VeSjKViHwZe7IWMZIE89tx1GXREF/EzWzg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Aq1H8WCJuAL37SctBUPuPK3Kqv6ruPc+2YmbD5MHcdI=; b=GLdmFlOXlGkB53/P5feuDffiSzLX5yrBCtSRggWW7w+0bPhpMuMLPvgdgkhGmtYNpK/n4UXmRSiQySiAJ3ymws4EeiUE+C451CSnlqG0y4H1XA09W+DSO7r2JAVTDB7PTylgsUkVF5zp8TKQG1ngjOwo0/HySOz+7TOxlrzqqME= Received: from BL0PR02CA0030.namprd02.prod.outlook.com (2603:10b6:207:3c::43) by BY5PR12MB4869.namprd12.prod.outlook.com (2603:10b6:a03:1d9::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.36; Tue, 17 Oct 2023 21:14:29 +0000 Received: from MN1PEPF0000ECD9.namprd02.prod.outlook.com (2603:10b6:207:3c:cafe::4a) by BL0PR02CA0030.outlook.office365.com (2603:10b6:207:3c::43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.6886.35 via Frontend Transport; Tue, 17 Oct 2023 21:14:28 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000ECD9.mail.protection.outlook.com (10.167.242.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.6838.22 via Frontend Transport; Tue, 17 Oct 2023 21:14:28 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.27; Tue, 17 Oct 2023 16:14:26 -0500 From: Felix Kuehling To: , Subject: [PATCH 11/11] drm/amdkfd: Bump KFD ioctl version Date: Tue, 17 Oct 2023 17:13:37 -0400 Message-ID: <20231017211337.1593869-12-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231017211337.1593869-1-Felix.Kuehling@amd.com> References: <20231017211337.1593869-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000ECD9:EE_|BY5PR12MB4869:EE_ X-MS-Office365-Filtering-Correlation-Id: 7785e31e-a189-40a4-e4ae-08dbcf560c5b X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: e6q2sVTLuiqac5uFrPR2NdYAQZRKNbAKOQHFoW+ClSwJllEt4m93WTjJbFX1tekVJ5NoPwp0ltn0uZy69hJBpWHxOvSIf2vGYkPkTHDVhBs26uIffSbFGqavTscMV8k7guGuG2T1cvUP9rVp+wX1+slCNtqA4173Rmegl9MSXUj/smNT183I24sOqyZyPOg2dMZeeC6hH4LhC2h2Rfxvyinr8HosFiYHqiMTjx6fkOVrakSPzNW3WkoYJ4mljVJUKHn/bbm7EtarQB0xIor4P7cezEsGDp4IF3ZGlehmtai4Mz1jXmNmpFJwgkMz1wNv2+D4iw8Y9XnVfe2q5vtxYEH+yHW4eftmNk9bfAAwKcXrcX2xw/T+5Eih8+Hujf63AlTuo1TONq0Gr6T01BoYGC45E4y/tMcKs7cSezp6ySkn7r9eX4BDvCRmyO3oE4GLh+V1tMUA+BX45CR6KFLyjjXQDRdEuI33o6ywDc5EIkTZ0nfm3o/GtXroCmifQsxwu23c0AFSFzORTxnj07Bh/S32jbvxHY6OXoz48O3DeSfvBLc6hK5/5WQ5rPDGo4OMNi/vjAzU8RPgd2wINQZ1+Obf0jvboHLmUCesfQMeMI1Fjg5zlzKbYxIbFnvAYUFA2mMfJR1d644YM92SnWsx9F3nzRHenuPpbVAeVXdcJ1bGv0aAr+kGyJnXTNpId1puDNSvJ2xmaKWgp+N4X3TuqdcBL+QuUjAPp+7OpZIsP5BG7pwzsunZPxd9yAZfWa55x68mJiwPVRg6JV1+FGT9jw== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(39860400002)(396003)(346002)(136003)(376002)(230922051799003)(451199024)(1800799009)(64100799003)(82310400011)(186009)(46966006)(36840700001)(40470700004)(86362001)(5660300002)(36860700001)(41300700001)(7696005)(8676002)(4326008)(83380400001)(8936002)(450100002)(4744005)(36756003)(2906002)(47076005)(110136005)(54906003)(70586007)(70206006)(426003)(316002)(336012)(356005)(1076003)(81166007)(40460700003)(2616005)(40480700001)(82740400003)(26005)(16526019)(478600001)(6666004)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Oct 2023 21:14:28.4074 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 7785e31e-a189-40a4-e4ae-08dbcf560c5b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000ECD9.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB4869 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Xiaogang.Chen@amd.com, Ramesh.Errabolu@amd.com, Christian.Koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is not strictly a change in the IOCTL API. This version bump is meant to indicate to user mode the presence of a number of changes and fixes that enable the management of VA mappings in compute VMs using the GEM_VA ioctl for DMABufs exported from KFD. Signed-off-by: Felix Kuehling --- include/uapi/linux/kfd_ioctl.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h index f0ed68974c54..9ce46edc62a5 100644 --- a/include/uapi/linux/kfd_ioctl.h +++ b/include/uapi/linux/kfd_ioctl.h @@ -40,9 +40,10 @@ * - 1.12 - Add DMA buf export ioctl * - 1.13 - Add debugger API * - 1.14 - Update kfd_event_data + * - 1.15 - Enable managing mappings in compute VMs with GEM_VA ioctl */ #define KFD_IOCTL_MAJOR_VERSION 1 -#define KFD_IOCTL_MINOR_VERSION 14 +#define KFD_IOCTL_MINOR_VERSION 15 struct kfd_ioctl_get_version_args { __u32 major_version; /* from KFD */