From patchwork Fri Dec 1 23:34:33 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13476618 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 85104C07E97 for ; Fri, 1 Dec 2023 23:36:24 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id B9F4110E97D; Fri, 1 Dec 2023 23:36:11 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2048.outbound.protection.outlook.com [40.107.94.48]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7E07910E091; Fri, 1 Dec 2023 23:36:07 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KIVFmUpRVUKYBDBNHWJcZe6CtDpQF/lphcz4WbQQUVqQCKhzzr6IFNI5uL0G8n6n+noMLL58EGYL0OcyBRaN4fwercO10xf7XYMTYMrc/casafWzyjMA29rs9pAVlwhD7OxRsYixyy579dl6fPGiMuujHyqyY1o+6Qlna4B3ZmtsGO9Z3oFyTSMmr5YUUO11qGdEfvq1GYTEjl2MOgiQHAwNgHB1ZDHAWKFEA4FKQpacdzLZArJdvH0P3U4HbTJnZVUsz4u2dGsUPKwyooaDE3Gc/u70xUG/MuJg1sCSBu6YlxiabJ0YHRTEThdUfDHqfYPxbWjdPTwLO8lwwTak2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=j6CxzWbWiiGwC6LdxA4kjtTwvDZzlrhby7ddajjSquI=; b=aeUpliYT2Iv5zrl4ntuofaMVXgYAyXxbyK9AgPBO+E6wnahvp/OL0/0yiplNRP2X7Kaac11nj2VP84IcvTGD4b7Q5Y3GH6GZMeTvYwKQI1bnBsilTanaeIIMw4e6O5X/kLE8r4PhuKjhdL5d/mQP9jct6LXcP36Bm+paxjqy+v2a5O87zhC5/01VRm+WQVaNFjD0J0wT4ELQ1FnWiEg2/+Q8zZJ4dqwaZY7Jw9KOKcHv6fiK3pJGv1kg9R5JMI1LsnrmnHWfOj17yhs9acp0rxfRzAVm1Fyzgef/zQDNDPAjJR8x3PosCgSJ22erPWahkvEZrEiEZcYszow5AwSUzQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=j6CxzWbWiiGwC6LdxA4kjtTwvDZzlrhby7ddajjSquI=; b=jb2B60VyUsH0ATE5ToUHT098Qo/phuQOUNG5frug4fsDVoKa2QJHt3dZeiZDGTtKJUw+Ujxjd9iPZMdd/K0BmO6GecpHMOyVIVePh/Ulkl5t2WJjFDf5fKvy2ISZOIcKeVUz7gcQ96PLiUPKu39vOK81wG7OFB79fkP0QUqhVpE= Received: from BL1PR13CA0027.namprd13.prod.outlook.com (2603:10b6:208:256::32) by DS7PR12MB5719.namprd12.prod.outlook.com (2603:10b6:8:72::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Fri, 1 Dec 2023 23:36:05 +0000 Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com (2603:10b6:208:256:cafe::3d) by BL1PR13CA0027.outlook.office365.com (2603:10b6:208:256::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7068.18 via Frontend Transport; Fri, 1 Dec 2023 23:36:04 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 23:36:04 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 1 Dec 2023 17:36:04 -0600 From: Felix Kuehling To: , Subject: [PATCH 1/6] Revert "drm/prime: Unexport helpers for fd/handle conversion" Date: Fri, 1 Dec 2023 18:34:33 -0500 Message-ID: <20231201233438.1709981-1-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|DS7PR12MB5719:EE_ X-MS-Office365-Filtering-Correlation-Id: e273aa24-406e-4e3b-8e96-08dbf2c6493a X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: O6NYgT+3nbdU7ZulZRViVqhmvC2v/SJQ6wdjnr+cjjw8ur4Je5ZTc25+n+6oWQUa/Z7K4ph5PKff9Aqha+opn1v4w1AtYcJ1s+s7cZj8skWGCmdfVgLS+DhzEigPTr3wCPq9FMzXaY40fjFZdyga10dirvZBJ9Z+X15yXgJUpQjnpWG/fb34RzC+95yelgYewNJ9B8MBvJbR0E16wO7ekW5iJvmBFXmwmgXZJLNf6CGwa8bvFznS8H8g4lpy02/i+oW/l48AJpZ4AiwkN3TsoJA/R5Gxmv3k1tp7qy8ko4uo3+VB+t+jvL+u9q8nXi6N21z+cBKHIfSB4giSr4YBTsJAJUeRW9Fp5BSVpdhEkS3Kk/XIw7rmU5X0CzvMPyvrWdb39HNpIPO2QhycWWxjFCqdSzWVlWkTPEecEH9UHtHt4yH4IIfPXCA2kN2qXpMv6anWKkVUw8BLI8hd+yVGy2mAXVkHXYFQDt7f2gwMBts1CeOmKlhlMYqDuBijdMzSxpXL4fmyBiiSOx+ZqVJ6LIBoDdLUb10ig3E22AB79P6NAv0jeXV6iIc0GZpjT1O1TtZ/G8PWhbBh1AWebdSfdQppxexjK4SczLcgn9rbXmHZIGIIPhnctfgbg4/g/9X4IF7/9NMzVAt/c/DY+grtfqJB1qO9WKtu0Tykz7ulRbYTj93tQk+KERxREMr5wxnuxhyY2WXDxoM55WCLPzNFHUD2J82RVQyEp8m/U9Ey3Pv9F5xzIKx1F8yegtLNy6qi161zvYKKy/ToiYhkfL/E+A== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(376002)(136003)(39860400002)(346002)(396003)(230922051799003)(82310400011)(186009)(451199024)(1800799012)(64100799003)(46966006)(36840700001)(40470700004)(5660300002)(2906002)(54906003)(316002)(70586007)(70206006)(8676002)(4326008)(8936002)(110136005)(40480700001)(16526019)(7696005)(26005)(41300700001)(6666004)(478600001)(2616005)(1076003)(83380400001)(426003)(336012)(66574015)(40460700003)(47076005)(36860700001)(82740400003)(356005)(81166007)(36756003)(86362001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 23:36:04.8609 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e273aa24-406e-4e3b-8e96-08dbf2c6493a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5719 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: =?utf-8?q?Christian_K=C3=B6nig?= , Thomas Zimmermann Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This reverts commit 71a7974ac7019afeec105a54447ae1dc7216cbb3. These helper functions are needed for KFD to export and import DMABufs the right way without duplicating the tracking of DMABufs associated with GEM objects while ensuring that move notifier callbacks are working as intended. Acked-by: Christian König Acked-by: Thomas Zimmermann Acked-by: Daniel Vetter Signed-off-by: Felix Kuehling --- drivers/gpu/drm/drm_prime.c | 33 ++++++++++++++++++--------------- include/drm/drm_prime.h | 7 +++++++ 2 files changed, 25 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/drm_prime.c b/drivers/gpu/drm/drm_prime.c index 63b709a67471..834a5e28abbe 100644 --- a/drivers/gpu/drm/drm_prime.c +++ b/drivers/gpu/drm/drm_prime.c @@ -278,7 +278,7 @@ void drm_gem_dmabuf_release(struct dma_buf *dma_buf) } EXPORT_SYMBOL(drm_gem_dmabuf_release); -/* +/** * drm_gem_prime_fd_to_handle - PRIME import function for GEM drivers * @dev: drm_device to import into * @file_priv: drm file-private structure @@ -292,9 +292,9 @@ EXPORT_SYMBOL(drm_gem_dmabuf_release); * * Returns 0 on success or a negative error code on failure. */ -static int drm_gem_prime_fd_to_handle(struct drm_device *dev, - struct drm_file *file_priv, int prime_fd, - uint32_t *handle) +int drm_gem_prime_fd_to_handle(struct drm_device *dev, + struct drm_file *file_priv, int prime_fd, + uint32_t *handle) { struct dma_buf *dma_buf; struct drm_gem_object *obj; @@ -360,6 +360,7 @@ static int drm_gem_prime_fd_to_handle(struct drm_device *dev, dma_buf_put(dma_buf); return ret; } +EXPORT_SYMBOL(drm_gem_prime_fd_to_handle); int drm_prime_fd_to_handle_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv) @@ -408,7 +409,7 @@ static struct dma_buf *export_and_register_object(struct drm_device *dev, return dmabuf; } -/* +/** * drm_gem_prime_handle_to_fd - PRIME export function for GEM drivers * @dev: dev to export the buffer from * @file_priv: drm file-private structure @@ -421,10 +422,10 @@ static struct dma_buf *export_and_register_object(struct drm_device *dev, * The actual exporting from GEM object to a dma-buf is done through the * &drm_gem_object_funcs.export callback. */ -static int drm_gem_prime_handle_to_fd(struct drm_device *dev, - struct drm_file *file_priv, uint32_t handle, - uint32_t flags, - int *prime_fd) +int drm_gem_prime_handle_to_fd(struct drm_device *dev, + struct drm_file *file_priv, uint32_t handle, + uint32_t flags, + int *prime_fd) { struct drm_gem_object *obj; int ret = 0; @@ -506,6 +507,7 @@ static int drm_gem_prime_handle_to_fd(struct drm_device *dev, return ret; } +EXPORT_SYMBOL(drm_gem_prime_handle_to_fd); int drm_prime_handle_to_fd_ioctl(struct drm_device *dev, void *data, struct drm_file *file_priv) @@ -864,9 +866,9 @@ EXPORT_SYMBOL(drm_prime_get_contiguous_size); * @obj: GEM object to export * @flags: flags like DRM_CLOEXEC and DRM_RDWR * - * This is the implementation of the &drm_gem_object_funcs.export functions - * for GEM drivers using the PRIME helpers. It is used as the default for - * drivers that do not set their own. + * This is the implementation of the &drm_gem_object_funcs.export functions for GEM drivers + * using the PRIME helpers. It is used as the default in + * drm_gem_prime_handle_to_fd(). */ struct dma_buf *drm_gem_prime_export(struct drm_gem_object *obj, int flags) @@ -962,9 +964,10 @@ EXPORT_SYMBOL(drm_gem_prime_import_dev); * @dev: drm_device to import into * @dma_buf: dma-buf object to import * - * This is the implementation of the gem_prime_import functions for GEM - * drivers using the PRIME helpers. It is the default for drivers that do - * not set their own &drm_driver.gem_prime_import. + * This is the implementation of the gem_prime_import functions for GEM drivers + * using the PRIME helpers. Drivers can use this as their + * &drm_driver.gem_prime_import implementation. It is used as the default + * implementation in drm_gem_prime_fd_to_handle(). * * Drivers must arrange to call drm_prime_gem_destroy() from their * &drm_gem_object_funcs.free hook when using this function. diff --git a/include/drm/drm_prime.h b/include/drm/drm_prime.h index a7abf9f3e697..2a1d01e5b56b 100644 --- a/include/drm/drm_prime.h +++ b/include/drm/drm_prime.h @@ -60,12 +60,19 @@ enum dma_data_direction; struct drm_device; struct drm_gem_object; +struct drm_file; /* core prime functions */ struct dma_buf *drm_gem_dmabuf_export(struct drm_device *dev, struct dma_buf_export_info *exp_info); void drm_gem_dmabuf_release(struct dma_buf *dma_buf); +int drm_gem_prime_fd_to_handle(struct drm_device *dev, + struct drm_file *file_priv, int prime_fd, uint32_t *handle); +int drm_gem_prime_handle_to_fd(struct drm_device *dev, + struct drm_file *file_priv, uint32_t handle, uint32_t flags, + int *prime_fd); + /* helper functions for exporting */ int drm_gem_map_attach(struct dma_buf *dma_buf, struct dma_buf_attachment *attach); From patchwork Fri Dec 1 23:34:34 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13476622 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 55701C4167B for ; Fri, 1 Dec 2023 23:36:38 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BB71710E997; Fri, 1 Dec 2023 23:36:36 +0000 (UTC) Received: from NAM02-DM3-obe.outbound.protection.outlook.com (mail-dm3nam02on2060b.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e83::60b]) by gabe.freedesktop.org (Postfix) with ESMTPS id D001910E991; Fri, 1 Dec 2023 23:36:07 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jUEwnIiXyBQnz8OFB5fkxHFnLPCK/HMVMXreqqb6K4DiO8kVwOwQV2pQJh62Xmw6LG1yUu6OC/blgfS3wHy8+tg8xvXlVAMPGDj8YULDs0VIvBPtBikZqYX644wuKx/J5Cx2h8oGfGeNWELMbveFt9YiCFutAPIe1MQX2EzqBHZjBxJI61j/Ult2VsNB8mPETFbW9D/CQZNfFEwCM6Ew9z6tF3/ivfXE0V1CRGeGSJy2QtrOiCTq5PUqBHj78LHnD35TEyoLdqOCnSXj9YQ8c8mgak+4J220B/UOJiKpwU8FhqFEb++p3YHnUGBCb+QYX7jAISq6RCe8UOcsV8rDtw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=EuQcIXAD77PqqGXWeSxVtuRuqX1xgjVLkWJjITP79HM=; b=lrn8JTjxP3EW7mFBAZgWvlLnYiRbfYlE7z1L7BcBi5L3ubOPJ2hiFZniP/XSTSDFVdUYXbvhbT0Cyc3QtiC98b/0B15it+UDgIrsFTsg+WmvubvYnBNtExvPwmJlcFLKuwyaHvRWBaSLgB/wWHFH+A+WDFheObVkag/RNa9P0Jgy7GTkJbKaSlKtKtwT9st/b3C/Sco0hIgoLWzU4tvEpsCBRIIea5r/zXeD3dJgU402KABeD86NbUWSmHT/ly8qwbWJS9hUR5nJMbJZQAM4lQGeuImRaoBELqBEFsVFiC+sTZDw12jFeyukz9ZtttsovAqXhj9f923GefPxTNj12w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=EuQcIXAD77PqqGXWeSxVtuRuqX1xgjVLkWJjITP79HM=; b=RFZfOLC5uWmk1I73nlL42MvT9HG+xyLP2l3x+jqKDA18u3EyQQvq/0GoEmYOdT9dO5Q3ka41KR1jY17nuRAGhmPnIKy3nB7Iylusz9CLmSyfa6Fg/fkUyGqVi9f7k9P9YMgGVp8QuAJMU9u8Wm7IBMVDZSzBYohkfI1Ei7UHvGA= Received: from BL1PR13CA0005.namprd13.prod.outlook.com (2603:10b6:208:256::10) by IA0PR12MB8694.namprd12.prod.outlook.com (2603:10b6:208:488::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Fri, 1 Dec 2023 23:36:05 +0000 Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com (2603:10b6:208:256:cafe::2) by BL1PR13CA0005.outlook.office365.com (2603:10b6:208:256::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7068.8 via Frontend Transport; Fri, 1 Dec 2023 23:36:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 23:36:05 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 1 Dec 2023 17:36:04 -0600 From: Felix Kuehling To: , Subject: [PATCH 2/6] drm/amdkfd: Export DMABufs from KFD using GEM handles Date: Fri, 1 Dec 2023 18:34:34 -0500 Message-ID: <20231201233438.1709981-2-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231201233438.1709981-1-Felix.Kuehling@amd.com> References: <20231201233438.1709981-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|IA0PR12MB8694:EE_ X-MS-Office365-Filtering-Correlation-Id: 6909402d-3193-4868-b1c2-08dbf2c6499e X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: G5SSVyyqKSUKP989H3zoUZ+CdyuG5mjuwpPA/rV+bxEUxryfhmnbvjUvFIKDR+YjIN2OGKOCPXQnwOBze0AWO887jgLIJkml9W23+S/WRPeyVL14DrKHcbW5ftZOAI/cyOnKwfFt07S6JCP2C139JyOky8vDF+LyC9Y9rThkcPCVs0k1HT6N/r+9ers41OWeVgV+zf75QOEI+jZ0J6JadwxFFZyZ4+CZa8NU2juI7p+qbMpjJtStN+68nexnIzv4Izj3NYLnjzCoS0G+GMFltYRUZ2468L/7nYNE59RngaqarTRBfozke3CwEw+odNfaeKIQDxb4wgpTjewkpzgfAYyNK/FZAJLhBEEgc3Fm7vYGhbA/Hy+6x2nIzDLpakUlAIsch1efGLGgljDw/pdI3+UU5g3VXwGA5ta/JbD8zl+jGqCbGi5+04t6TdO9ufF/Ci5k2HGw7uQAD9o+WylT4FRRXmPY32G+iOZRel0AKQlE/OTXFhsHH3Lx9H1uiT2C2pzdowBh67WSqoozPph2U7nIkb/i4bUQMTQoFWNX6gE3EOugCsnKNGpQmZLbSzyTUvQOcAq0ghShtOruO3IUPBkpOa2hoC5SEV9I90ZwRL9PY+JOs9L28Zf9sDJKl387PBeUaiIwZFbYhfhPoYUJ6DlBGKFFTNhYJHz8pp7OFdLkvIesRoKYGi25FYnBIGqw128A4C4+01STST0v6INPwLtyqu3G53I31/mYXtDVQQ3pX9fq2BCU96GpMDfkeWhTenTjw/8SmCuKYRpp6UNxsg== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(376002)(396003)(39860400002)(346002)(136003)(230922051799003)(1800799012)(186009)(82310400011)(64100799003)(451199024)(36840700001)(40470700004)(46966006)(450100002)(8936002)(4326008)(110136005)(26005)(1076003)(2616005)(86362001)(478600001)(8676002)(6666004)(70206006)(70586007)(16526019)(7696005)(426003)(83380400001)(316002)(40480700001)(336012)(47076005)(5660300002)(36860700001)(81166007)(2906002)(40460700003)(82740400003)(356005)(41300700001)(36756003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 23:36:05.5171 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 6909402d-3193-4868-b1c2-08dbf2c6499e X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8694 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Ramesh Errabolu Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Create GEM handles for exporting DMABufs using GEM-Prime APIs. The GEM handles are created in a drm_client_dev context to avoid exposing them in user mode contexts through a DMABuf import. Signed-off-by: Felix Kuehling Reviewed-by: Ramesh Errabolu --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 11 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 5 +++ .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 33 +++++++++++++++---- drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 4 +-- 4 files changed, 44 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c index 2d22f7d45512..067690ba7bff 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c @@ -142,6 +142,7 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) { int i; int last_valid_bit; + int ret; amdgpu_amdkfd_gpuvm_init_mem_limits(); @@ -160,6 +161,12 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) .enable_mes = adev->enable_mes, }; + ret = drm_client_init(&adev->ddev, &adev->kfd.client, "kfd", NULL); + if (ret) { + dev_err(adev->dev, "Failed to init DRM client: %d\n", ret); + return; + } + /* this is going to have a few of the MSBs set that we need to * clear */ @@ -198,6 +205,10 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev) adev->kfd.init_complete = kgd2kfd_device_init(adev->kfd.dev, &gpu_resources); + if (adev->kfd.init_complete) + drm_client_register(&adev->kfd.client); + else + drm_client_release(&adev->kfd.client); amdgpu_amdkfd_total_mem_size += adev->gmc.real_vram_size; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 16794c2eea35..02973f5c8caf 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -33,6 +33,7 @@ #include #include #include +#include #include "amdgpu_sync.h" #include "amdgpu_vm.h" #include "amdgpu_xcp.h" @@ -83,6 +84,7 @@ struct kgd_mem { struct amdgpu_sync sync; + uint32_t gem_handle; bool aql_queue; bool is_imported; }; @@ -105,6 +107,9 @@ struct amdgpu_kfd_dev { /* HMM page migration MEMORY_DEVICE_PRIVATE mapping */ struct dev_pagemap pgmap; + + /* Client for KFD BO GEM handle allocations */ + struct drm_client_dev client; }; enum kgd_engine_type { diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 73288f9ccaf8..ae7dfaf59159 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -25,6 +25,7 @@ #include #include #include +#include #include #include @@ -806,13 +807,22 @@ kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, static int kfd_mem_export_dmabuf(struct kgd_mem *mem) { if (!mem->dmabuf) { - struct dma_buf *ret = amdgpu_gem_prime_export( - &mem->bo->tbo.base, + struct amdgpu_device *bo_adev; + struct dma_buf *dmabuf; + int r, fd; + + bo_adev = amdgpu_ttm_adev(mem->bo->tbo.bdev); + r = drm_gem_prime_handle_to_fd(&bo_adev->ddev, bo_adev->kfd.client.file, + mem->gem_handle, mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? - DRM_RDWR : 0); - if (IS_ERR(ret)) - return PTR_ERR(ret); - mem->dmabuf = ret; + DRM_RDWR : 0, &fd); + if (r) + return r; + dmabuf = dma_buf_get(fd); + close_fd(fd); + if (WARN_ON_ONCE(IS_ERR(dmabuf))) + return PTR_ERR(dmabuf); + mem->dmabuf = dmabuf; } return 0; @@ -1778,6 +1788,9 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( pr_debug("Failed to allow vma node access. ret %d\n", ret); goto err_node_allow; } + ret = drm_gem_handle_create(adev->kfd.client.file, gobj, &(*mem)->gem_handle); + if (ret) + goto err_gem_handle_create; bo = gem_to_amdgpu_bo(gobj); if (bo_type == ttm_bo_type_sg) { bo->tbo.sg = sg; @@ -1829,6 +1842,8 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( err_pin_bo: err_validate_bo: remove_kgd_mem_from_kfd_bo_list(*mem, avm->process_info); + drm_gem_handle_delete(adev->kfd.client.file, (*mem)->gem_handle); +err_gem_handle_create: drm_vma_node_revoke(&gobj->vma_node, drm_priv); err_node_allow: /* Don't unreserve system mem limit twice */ @@ -1941,8 +1956,12 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( /* Free the BO*/ drm_vma_node_revoke(&mem->bo->tbo.base.vma_node, drm_priv); - if (mem->dmabuf) + if (!mem->is_imported) + drm_gem_handle_delete(adev->kfd.client.file, mem->gem_handle); + if (mem->dmabuf) { dma_buf_put(mem->dmabuf); + mem->dmabuf = NULL; + } mutex_destroy(&mem->lock); /* If this releases the last reference, it will end up calling diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c index f6d4748c1980..b4cb907f80c6 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c @@ -1855,8 +1855,8 @@ static uint32_t get_process_num_bos(struct kfd_process *p) return num_of_bos; } -static int criu_get_prime_handle(struct kgd_mem *mem, int flags, - u32 *shared_fd) +static int criu_get_prime_handle(struct kgd_mem *mem, + int flags, u32 *shared_fd) { struct dma_buf *dmabuf; int ret; From patchwork Fri Dec 1 23:34:35 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13476623 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2541DC10DC1 for ; Fri, 1 Dec 2023 23:36:47 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6FC0D10E99D; Fri, 1 Dec 2023 23:36:46 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062e.outbound.protection.outlook.com [IPv6:2a01:111:f400:7e88::62e]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5E70510E992; Fri, 1 Dec 2023 23:36:08 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GYqyQvJ5llhnMgc5+pN/pcwVpY6o898PZ6nL+ncRFMrVY0sm8sgJOpGjBfo3I8Nf2PU9hjstv952L+lgc/ApgUVILGgDx3YekTlAZF0pf6teuuwbi+5mFi53SaL7lhghBknELR+K9xOi45HTQyBIvncD0nMQICxSeGTTsEGYDdzTfcclhXLwQCcJ/lwyb2CDA7FOPRHZDtnOGtJTCKMaf0INJPYpYmM4qbbQxwnrIo2UTeH5RvY/glzmbZFwo8280sawqyA/SMT8MbXMyErR1sS2f8GEfyqiDi366176/rQZNf9e9scXRXZanpmS5w8vvBt8uxjH74XFWFNEoDmv2Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Z508SUl6xfK1SOuacFeVcUT7xIaThZXv4GiJA/LkZAM=; b=fN5si1RM8ToW4rwwmZ/vGylRC6lwmvLlDyksgNPaVyaoUcRrKU3tCk7JIO6n540BMV6SzxUAbns9qDT7Ci/fwM0zQpb2Y8YxhYhahXYKvyFMgIKZsOQ22OLw3yk6KhnGeY+CDIaxCGvBlayhEG8O9BEawIF8/nhhvkF7AX0nYckeOvFMDoPP2tC8fN9eUbJ0iwqVzJP+OKKYjevZ9OwzNLrMB68GRTaumCvKaAgbG2YtopsC1Q1k3apaFCxa/ODYyvhDhRcePBnUqD7g79UVcZCm+02xFElFxCV+oLmeBzDz+ZDcgb5WcAren2yNuA0ryfutJRaAdN1BXerL6EMj5Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Z508SUl6xfK1SOuacFeVcUT7xIaThZXv4GiJA/LkZAM=; b=Y/jLrUPOZXekGQjvr2i+o0fdB2dPRrvivU8HjPkQJObCjcmKGB5Jbz/Xj/CE0dKZS6QZLBqqniClQf9qYKaLU6ecDFWYHbaPg6F8I34SWVgDaAYWJS+4RcBwDdnsBduaGm2H0B0jk1Cp8ZZkMsS+4KgCMtaSB+rPUepS8HWP3KM= Received: from BL1PR13CA0024.namprd13.prod.outlook.com (2603:10b6:208:256::29) by MN0PR12MB6056.namprd12.prod.outlook.com (2603:10b6:208:3cc::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Fri, 1 Dec 2023 23:36:05 +0000 Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com (2603:10b6:208:256:cafe::5a) by BL1PR13CA0024.outlook.office365.com (2603:10b6:208:256::29) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7068.8 via Frontend Transport; Fri, 1 Dec 2023 23:36:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 23:36:05 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 1 Dec 2023 17:36:05 -0600 From: Felix Kuehling To: , Subject: [PATCH 3/6] drm/amdkfd: Import DMABufs for interop through DRM Date: Fri, 1 Dec 2023 18:34:35 -0500 Message-ID: <20231201233438.1709981-3-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231201233438.1709981-1-Felix.Kuehling@amd.com> References: <20231201233438.1709981-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|MN0PR12MB6056:EE_ X-MS-Office365-Filtering-Correlation-Id: 09590712-9bfd-4628-d7c1-08dbf2c649c4 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hFIHzVanEFV0B7GIHw0z4wRIfmHTi/o36fWgIJdEP6x8Fl48S3doaJsG+Ef4VLP3dw+qiNZE0TsEjRDrR7mQR05n5AAJv3fAOsP9ZUBCOROeMe/UzHTtnXolJ2kwav1qLQN7AfJWd/fpZZeZ1ruFfFwJ/O96Ir24U0IP2d82+KLxk8x2WxI6LmomphZgB6o61e77PwFzc+ZThFIHhNTO/ZyQjh8nuJr/eJ7asfQB3B42uEysnk8w0H5zh3zDpNNDAWoeUU9tvh/R/yWxEVXZE+PvvU4mZFc/5CREQ0BDunAyk0fJaMUGgCXL2o81NlByn8g6xpnZwFcOABDrqZcspoPKw8lE+WdaEoMjhtkgUZXovS1bDbb+LgQWBlaMMeB3oetPz2fIlkP93bT0Coib6sVCejUbK7Jp+Kgw1azpdcaNwP/Ou4K/v3wJ0cmiiCJAxR4SuEsAVl2rmpgutp+paOYobOvO3VmXakOTJqcNGJZnE95z2LGVAuTKPT3l4wCYkcJRq7SGDkUudhvQ+vuVpQH7z7iMHkirkJa++MUUt40fYKKvtw01r333C7J9fYcQwamhSS0ChjEt54JoAO4IAMAsXcV9kr7QYv8RnTRr8JQYhIrEGTDOEPqHzAzGzHZQZUREe0M1WEC3oGmiU8z+gtvF4f++CQ0iUs56aSwSxPywF9YseZMnRnpEKQV/4oakQOtsZOv4Ea6Iw7qCrQ6b76a5yPyKOF5xxVMGXW5aprtimqHnWem8lRc8etRTK2HJ8orS1JwpRzAeZVVJhViciA== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(376002)(346002)(396003)(136003)(39860400002)(230922051799003)(82310400011)(186009)(451199024)(1800799012)(64100799003)(46966006)(40470700004)(36840700001)(5660300002)(2906002)(54906003)(8936002)(4326008)(450100002)(316002)(8676002)(70206006)(110136005)(70586007)(40480700001)(16526019)(7696005)(26005)(41300700001)(6666004)(478600001)(2616005)(1076003)(83380400001)(426003)(336012)(66574015)(40460700003)(47076005)(36860700001)(82740400003)(81166007)(356005)(86362001)(36756003)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 23:36:05.7671 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 09590712-9bfd-4628-d7c1-08dbf2c649c4 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6056 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: "Xiaogang . Chen" , Ramesh Errabolu , =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use drm_gem_prime_fd_to_handle to import DMABufs for interop. This ensures that a GEM handle is created on import and that obj->dma_buf will be set and remain set as long as the object is imported into KFD. Signed-off-by: Felix Kuehling Reviewed-by: Ramesh Errabolu Reviewed-by: Xiaogang.Chen Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 9 ++- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 64 +++++++++++++------ drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 15 ++--- 3 files changed, 52 insertions(+), 36 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 02973f5c8caf..cf6ed5fce291 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -314,11 +314,10 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *process_info, struct dma_fence **ef); int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct amdgpu_device *adev, struct kfd_vm_fault_info *info); -int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, - struct dma_buf *dmabuf, - uint64_t va, void *drm_priv, - struct kgd_mem **mem, uint64_t *size, - uint64_t *mmap_offset); +int amdgpu_amdkfd_gpuvm_import_dmabuf_fd(struct amdgpu_device *adev, int fd, + uint64_t va, void *drm_priv, + struct kgd_mem **mem, uint64_t *size, + uint64_t *mmap_offset); int amdgpu_amdkfd_gpuvm_export_dmabuf(struct kgd_mem *mem, struct dma_buf **dmabuf); void amdgpu_amdkfd_debug_mem_fence(struct amdgpu_device *adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index ae7dfaf59159..48697b789342 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -1956,8 +1956,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( /* Free the BO*/ drm_vma_node_revoke(&mem->bo->tbo.base.vma_node, drm_priv); - if (!mem->is_imported) - drm_gem_handle_delete(adev->kfd.client.file, mem->gem_handle); + drm_gem_handle_delete(adev->kfd.client.file, mem->gem_handle); if (mem->dmabuf) { dma_buf_put(mem->dmabuf); mem->dmabuf = NULL; @@ -2313,34 +2312,26 @@ int amdgpu_amdkfd_gpuvm_get_vm_fault_info(struct amdgpu_device *adev, return 0; } -int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, - struct dma_buf *dma_buf, - uint64_t va, void *drm_priv, - struct kgd_mem **mem, uint64_t *size, - uint64_t *mmap_offset) +static int import_obj_create(struct amdgpu_device *adev, + struct dma_buf *dma_buf, + struct drm_gem_object *obj, + uint64_t va, void *drm_priv, + struct kgd_mem **mem, uint64_t *size, + uint64_t *mmap_offset) { struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); - struct drm_gem_object *obj; struct amdgpu_bo *bo; int ret; - obj = amdgpu_gem_prime_import(adev_to_drm(adev), dma_buf); - if (IS_ERR(obj)) - return PTR_ERR(obj); - bo = gem_to_amdgpu_bo(obj); if (!(bo->preferred_domains & (AMDGPU_GEM_DOMAIN_VRAM | - AMDGPU_GEM_DOMAIN_GTT))) { + AMDGPU_GEM_DOMAIN_GTT))) /* Only VRAM and GTT BOs are supported */ - ret = -EINVAL; - goto err_put_obj; - } + return -EINVAL; *mem = kzalloc(sizeof(struct kgd_mem), GFP_KERNEL); - if (!*mem) { - ret = -ENOMEM; - goto err_put_obj; - } + if (!*mem) + return -ENOMEM; ret = drm_vma_node_allow(&obj->vma_node, drm_priv); if (ret) @@ -2390,8 +2381,41 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct amdgpu_device *adev, drm_vma_node_revoke(&obj->vma_node, drm_priv); err_free_mem: kfree(*mem); + return ret; +} + +int amdgpu_amdkfd_gpuvm_import_dmabuf_fd(struct amdgpu_device *adev, int fd, + uint64_t va, void *drm_priv, + struct kgd_mem **mem, uint64_t *size, + uint64_t *mmap_offset) +{ + struct drm_gem_object *obj; + uint32_t handle; + int ret; + + ret = drm_gem_prime_fd_to_handle(&adev->ddev, adev->kfd.client.file, fd, + &handle); + if (ret) + return ret; + obj = drm_gem_object_lookup(adev->kfd.client.file, handle); + if (!obj) { + ret = -EINVAL; + goto err_release_handle; + } + + ret = import_obj_create(adev, obj->dma_buf, obj, va, drm_priv, mem, size, + mmap_offset); + if (ret) + goto err_put_obj; + + (*mem)->gem_handle = handle; + + return 0; + err_put_obj: drm_gem_object_put(obj); +err_release_handle: + drm_gem_handle_delete(adev->kfd.client.file, handle); return ret; } diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c index b4cb907f80c6..ce4c52ec34d8 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c @@ -1564,16 +1564,11 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, { struct kfd_ioctl_import_dmabuf_args *args = data; struct kfd_process_device *pdd; - struct dma_buf *dmabuf; int idr_handle; uint64_t size; void *mem; int r; - dmabuf = dma_buf_get(args->dmabuf_fd); - if (IS_ERR(dmabuf)) - return PTR_ERR(dmabuf); - mutex_lock(&p->mutex); pdd = kfd_process_device_data_by_id(p, args->gpu_id); if (!pdd) { @@ -1587,10 +1582,10 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, goto err_unlock; } - r = amdgpu_amdkfd_gpuvm_import_dmabuf(pdd->dev->adev, dmabuf, - args->va_addr, pdd->drm_priv, - (struct kgd_mem **)&mem, &size, - NULL); + r = amdgpu_amdkfd_gpuvm_import_dmabuf_fd(pdd->dev->adev, args->dmabuf_fd, + args->va_addr, pdd->drm_priv, + (struct kgd_mem **)&mem, &size, + NULL); if (r) goto err_unlock; @@ -1601,7 +1596,6 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, } mutex_unlock(&p->mutex); - dma_buf_put(dmabuf); args->handle = MAKE_HANDLE(args->gpu_id, idr_handle); @@ -1612,7 +1606,6 @@ static int kfd_ioctl_import_dmabuf(struct file *filep, pdd->drm_priv, NULL); err_unlock: mutex_unlock(&p->mutex); - dma_buf_put(dmabuf); return r; } From patchwork Fri Dec 1 23:34:36 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13476619 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A7646C4167B for ; Fri, 1 Dec 2023 23:36:29 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id AFAD610E992; Fri, 1 Dec 2023 23:36:25 +0000 (UTC) Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2085.outbound.protection.outlook.com [40.107.94.85]) by gabe.freedesktop.org (Postfix) with ESMTPS id 98A0D10E97D; Fri, 1 Dec 2023 23:36:07 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=D6ilWp12fkMwQ9EPLImImbawu6xx9FA9HaWZbWOotJgk1zAhXfkfwoMXgEhfXBy+z9GXkEwwjoc+9n3SwI/Y1/kjXJ4fh8i1csFFtcC5aYvZ2qpC1c74hZVYbIGWkh3NXjxFDqpMFaNKflhfEbrruHBl2HrkmXEWZYfVVW2l8IgbfYt4CZpQfvs48zSDCgSJgM/4aW/0AmtuCmy53iL5Er8c+7wXQeotH+AHL8iv6g2BtnfrhObEC9avOAI3uzPdfOZtBoIRb67lcYKYPW0JNU77SMn16APMo/acs2fFsjwIb8HurLfpMxm14akPAT39rLinLiOZaCow4IlpeuNOxQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=CEw+B9AhJqQoV5SyDpWCqRekxSl3Mz6MaOfwtjHai44=; b=XwZYP00L1UTl8PAFskBKn2JLbConyRmqC3VAT2MrIgDzi/CnpqKlpWgEYr9W8rKxbB3UoMi6C+Fed6UQIxBDf8yYCe6R/Og9Y7Hv2bZEx3Tewe9ztYKf+dxUo9qm4R0LgVDruFqcoiYZuei5LteTJVQCT9O+jG6dZqAYXSuj/Hp8uZGtOFpKPAtLF8GnFMMEj8sHM8owJPUyyam/j9bsRcYETbjgwZZYTnhASMjT9DXYmGUO4JZuT9C5yP86QbWjhLMburPS92sFN2N13W+L4kgQO8ar31mEDt+Nagyw9c1gd0yJUyOs6a0LH3otUr+m63H2FF6sKhIKqxxw2SNWfQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=CEw+B9AhJqQoV5SyDpWCqRekxSl3Mz6MaOfwtjHai44=; b=n1bht0caSt56dsxv8dDxFLQv1g5SMuqkAxhpWyAvbx01i20yp5vHXtCTsEk+GEeQdutU9m4HTF4B1jJg6eSaKXsMkwIXsdXvdPzNzuIVN4gVYgMNsWtR3YBdlKO14vGDI1Zcgl52K0Ew9VdciWTSlQXX02Ji8hvEvg8IAXxNN0w= Received: from BL1PR13CA0029.namprd13.prod.outlook.com (2603:10b6:208:256::34) by IA1PR12MB8310.namprd12.prod.outlook.com (2603:10b6:208:3ff::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Fri, 1 Dec 2023 23:36:06 +0000 Received: from MN1PEPF0000F0E5.namprd04.prod.outlook.com (2603:10b6:208:256:cafe::b1) by BL1PR13CA0029.outlook.office365.com (2603:10b6:208:256::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7068.13 via Frontend Transport; Fri, 1 Dec 2023 23:36:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E5.mail.protection.outlook.com (10.167.242.43) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 23:36:06 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 1 Dec 2023 17:36:05 -0600 From: Felix Kuehling To: , Subject: [PATCH 4/6] drm/amdgpu: New VM state for evicted user BOs Date: Fri, 1 Dec 2023 18:34:36 -0500 Message-ID: <20231201233438.1709981-4-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231201233438.1709981-1-Felix.Kuehling@amd.com> References: <20231201233438.1709981-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E5:EE_|IA1PR12MB8310:EE_ X-MS-Office365-Filtering-Correlation-Id: e9ed97a0-ad5e-4026-ad9a-08dbf2c649fb X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: bU+m0sK+ChkPs1LDb0N8ezIZf9JBr81eiQefzOOnQtQCd8N435x3ETiAjQ6oPJrH1mlIbRR6rt4zCsk5QEuH2RKZUw7m+DO9SuhnZ7O9ycwRMlTnf376l0/9UDpAGmQxiz0ZRAIVsnFHmrrEQpedPlkCf24Ok+zL2TUsJsO8UH9F2EXuCRTdIKkjOJiR+07dhJRHxPrI9POxEJxiNyiV09PCS7qkp+M4X2xvnCfzp7cHh/bCl0k5W2apZv76nO/xBgiESwuANif+hlHRbIk4Se15qEG+ppyA2ZmO1lLTFCu0oh92HHkscFBilzpRCJezsiajCLQbh+lplpI6tYgdMAastWzgMuBD29rMPligDyNPjHXg+n9bxU/7aJhogOVwrMHKqTr/E62K/94H6mQaNYikrTu4CpBuJusQKnJzgizs5SZCrIP39vsemBEC4HCECCyrOm+5Y8Ok+IsT9Ygj/oyIGJN6I9BIEoe+qGhTC73dOIsQWNuV1rf9D9owSRBdmRM/7idccKhsFuF/LciE9+IzMWpPyfxuyE5yeEU9o3N4nHtGqRMzIY9QVLx/G8ck609+ApmDGWyvNlBsAJ0dzrSsXF+slZNm8OwKdDbpD+xAo3GN8g3LLDLkb7zUVW0zlnTgkFvsFLkFux0UwQE1WXvEBu7NPWJMPqyV6BDL4/6/+o/qckciqRHAVNPbkQArkV54mlVKQQAEVGUT+G0Q23DBMbz4Rmt8u3InYCtADkj42I35KyKpZXlPiHtayCXGzS13mVq7GhA+kA6sP+8iGw== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(396003)(39860400002)(376002)(346002)(230922051799003)(1800799012)(451199024)(64100799003)(82310400011)(186009)(36840700001)(46966006)(40470700004)(40460700003)(336012)(83380400001)(426003)(82740400003)(356005)(36756003)(86362001)(81166007)(47076005)(36860700001)(8676002)(110136005)(70586007)(70206006)(8936002)(450100002)(316002)(40480700001)(5660300002)(1076003)(2906002)(2616005)(16526019)(478600001)(7696005)(26005)(6666004)(41300700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 23:36:06.1265 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e9ed97a0-ad5e-4026-ad9a-08dbf2c649fb X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E5.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB8310 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Create a new VM state to track user BOs that are in the system domain. In the next patch this will be used do conditionally re-validate them in amdgpu_vm_handle_moved. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 17 +++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 5 ++++- 2 files changed, 21 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 7da71b6a9dc6..a748e17ff031 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -233,6 +233,22 @@ static void amdgpu_vm_bo_invalidated(struct amdgpu_vm_bo_base *vm_bo) spin_unlock(&vm_bo->vm->status_lock); } +/** + * amdgpu_vm_bo_evicted_user - vm_bo is evicted + * + * @vm_bo: vm_bo which is evicted + * + * State for BOs used by user mode queues which are not at the location they + * should be. + */ +static void amdgpu_vm_bo_evicted_user(struct amdgpu_vm_bo_base *vm_bo) +{ + vm_bo->moved = true; + spin_lock(&vm_bo->vm->status_lock); + list_move(&vm_bo->vm_status, &vm_bo->vm->evicted_user); + spin_unlock(&vm_bo->vm->status_lock); +} + /** * amdgpu_vm_bo_relocated - vm_bo is reloacted * @@ -2195,6 +2211,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm, for (i = 0; i < AMDGPU_MAX_VMHUBS; i++) vm->reserved_vmid[i] = NULL; INIT_LIST_HEAD(&vm->evicted); + INIT_LIST_HEAD(&vm->evicted_user); INIT_LIST_HEAD(&vm->relocated); INIT_LIST_HEAD(&vm->moved); INIT_LIST_HEAD(&vm->idle); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index b6cd565562ad..9156ed22abe7 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -288,9 +288,12 @@ struct amdgpu_vm { /* Lock to protect vm_bo add/del/move on all lists of vm */ spinlock_t status_lock; - /* BOs who needs a validation */ + /* Per VM and PT BOs who needs a validation */ struct list_head evicted; + /* BOs for user mode queues that need a validation */ + struct list_head evicted_user; + /* PT BOs which relocated and their parent need an update */ struct list_head relocated; From patchwork Fri Dec 1 23:34:37 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13476621 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 767CDC10DC1 for ; Fri, 1 Dec 2023 23:36:36 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6721010E996; Fri, 1 Dec 2023 23:36:35 +0000 (UTC) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on20601.outbound.protection.outlook.com [IPv6:2a01:111:f400:7eae::601]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8A85610E091; Fri, 1 Dec 2023 23:36:11 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PRBXuAkHL2wuowt5+YgFiwVO8TmamyeSSYbDZxjKHhzdTfDRzVRUzSoxm5cLHCUiBPSbPmAk1eDxVf+kJG5jqLsv8+AdkdNM0fANLR6gsA8WwEK54UVLBckCH+/kSD1Kl1dV45DY8l4mIYnHkHSSBGjE31xEUW5yh73tXBTyr/kgbQ3WT3ftat7JTHG8pL9gK22mB6sTXN1l2Aes7zjseXv/IUsFk6BAGlx0k0ildeOVdzdya4spVQqdDdRVKzClPJTqJhUcLQ/e9gj2I71oX8tfJ57m1LSj7h402PGUn9PSkRYKyzfwyyGFm6VqjYQ5DaU1Nxf6r+Y4mLzqb92U0w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=z/xifRR7RGDhixuRhO+IyWYQzdIbRg1Rtcp1JQK82g4=; b=duf6lZNWQwlQv2FYXqZMJ+3RAx9qqhytYuuj5MjqVVqPNJGceW6I5lZGsdLG5XELvAvBOKY2w5qhqq3WM2PXqkSgOZknpXB1UP8kIziSP4NeHTNY/zNC+vwc3/USJfjZMYFf6UK/AvJ9/GAzXDS03GzBEeeiHqnENo/37K5AldWkf0aMRiKhPymPA8JgnGl50uvy/33ICbIBMTg5M3VVpj+UIeO16z6oXE093Rf3LiRx0GcCqkvR17s4VJ8Tysw+5YJiIjWy8/QJ0KUkr3czEy16xxo4kBDSUkSFiRWDCricyigA53d1Nl99TVQVBfiN7ATMicjNCak6TuHLxHEoCw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=z/xifRR7RGDhixuRhO+IyWYQzdIbRg1Rtcp1JQK82g4=; b=aYl7R4M7Mb/pPO4voyVEHzelpB/ekqQimO6gJ1fYCtrSXsI7fFqtR7jWfcNTJGeq9JuoPuq0fwss7sl7ex106qW90H2842mv+d9nSfhXNrRXIerTTanZOEzwZqQhQ40L+1Gmo8Up1W/YNTffOdFw+h4umEPHvPYmXBKL0/NjkbU= Received: from BLAPR03CA0099.namprd03.prod.outlook.com (2603:10b6:208:32a::14) by DM8PR12MB5431.namprd12.prod.outlook.com (2603:10b6:8:34::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27; Fri, 1 Dec 2023 23:36:06 +0000 Received: from MN1PEPF0000F0E2.namprd04.prod.outlook.com (2603:10b6:208:32a:cafe::e7) by BLAPR03CA0099.outlook.office365.com (2603:10b6:208:32a::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27 via Frontend Transport; Fri, 1 Dec 2023 23:36:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E2.mail.protection.outlook.com (10.167.242.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 23:36:06 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 1 Dec 2023 17:36:05 -0600 From: Felix Kuehling To: , Subject: [PATCH 5/6] drm/amdgpu: Auto-validate DMABuf imports in compute VMs Date: Fri, 1 Dec 2023 18:34:37 -0500 Message-ID: <20231201233438.1709981-5-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231201233438.1709981-1-Felix.Kuehling@amd.com> References: <20231201233438.1709981-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E2:EE_|DM8PR12MB5431:EE_ X-MS-Office365-Filtering-Correlation-Id: 54b88950-6bcb-48e3-35b1-08dbf2c64a37 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ruffgY/9S4ADENEGzwvd62EHyK6Umh2jXIJWKtecNLl0ruBChXaEUQL5Ba0rkxs0XoEKUpKABw2xgtlOtzLappWmIyIF1h2u7JTU5b2WSu1+MMpiL2TjpuyNg0kcbx67FuEoq152gago4AJQs6q36V/Yw6Xp3+0LTogKxguzGIP/Wj3b+o+cWFGvJ5hWY/R3/SniExpvsxOykpEeKNhzVzUOov0GtwkBe9NHY3WL0vBvb0XhtuIKZjHBXghWw4YYV/iDpWcjnY1MOr+SrNJhOp77ElJIy9k9WiqJEdAfI0y157weR3tgHmxDp3DleIJGoJObrvPIKcW9ws86ouzWpNBYvq7yKfzr9xR4fw8bCb/VGNRAwSTO21rDqeJ3Mm9CloH2a656Y46GyaBqXFSbdsOOjXrIdizX+3dqoKbMmuU9Icjg9s8I+CDoWmcfgHaW9yk6PSP4r1pqXhLFzoTZrZWMB+TdWyHp0kMH5l9OksYeOCznod9sHfNTG/DtKHLBRrk+JkPOe8F+BeP6WehvH7A2yWGJ/6E/hulDdwgTXv1doAGArqNG47YvBRox0znKi4X626mU6KMLoJUTYq2ZbtKx5Fsh7BXtckqhT5BUlf41o1dUr9VEi4cA1gyCFguUB/lygLNFVfE+w7lEj9N2UUXHn0ZDVfYIL3HfY3SF24SYy5DosnOPHxfDArzFDdhEsvKWZKixlfsKcc58dtDaMJP/d4lT24b0PlABSilBqgYcyovg2KcLDKkozKpyKqrA+j+m6NlhqVW7t5OV3kxrCQ== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(136003)(396003)(39860400002)(376002)(346002)(230922051799003)(1800799012)(451199024)(64100799003)(82310400011)(186009)(36840700001)(46966006)(40470700004)(40460700003)(336012)(83380400001)(426003)(82740400003)(356005)(36756003)(86362001)(81166007)(47076005)(36860700001)(8676002)(110136005)(70586007)(70206006)(8936002)(450100002)(316002)(40480700001)(15650500001)(5660300002)(1076003)(30864003)(2906002)(2616005)(16526019)(478600001)(7696005)(26005)(6666004)(41300700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 23:36:06.5166 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 54b88950-6bcb-48e3-35b1-08dbf2c64a37 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E2.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5431 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" DMABuf imports in compute VMs are not wrapped in a kgd_mem object on the process_info->kfd_bo_list. There is no explicit KFD API call to validate them or add eviction fences to them. This patch automatically validates and fences dymanic DMABuf imports when they are added to a compute VM. Revalidation after evictions is handled in the VM code. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 3 + .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 45 ++++--- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 4 + drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 26 ++++ drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 122 +++++++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 12 +- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 12 +- 8 files changed, 196 insertions(+), 34 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index cf6ed5fce291..f2e920734c98 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -182,6 +182,9 @@ int amdgpu_queue_mask_bit_to_set_resource_bit(struct amdgpu_device *adev, struct amdgpu_amdkfd_fence *amdgpu_amdkfd_fence_create(u64 context, struct mm_struct *mm, struct svm_range_bo *svm_bo); +int amdgpu_amdkfd_bo_validate_and_fence(struct amdgpu_bo *bo, + uint32_t domain, + struct dma_fence *fence); #if defined(CONFIG_DEBUG_FS) int kfd_debugfs_kfd_mem_limits(struct seq_file *m, void *data); #endif diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 48697b789342..7d91f99acb59 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -426,9 +426,9 @@ static int amdgpu_amdkfd_bo_validate(struct amdgpu_bo *bo, uint32_t domain, return ret; } -static int amdgpu_amdkfd_bo_validate_and_fence(struct amdgpu_bo *bo, - uint32_t domain, - struct dma_fence *fence) +int amdgpu_amdkfd_bo_validate_and_fence(struct amdgpu_bo *bo, + uint32_t domain, + struct dma_fence *fence) { int ret = amdgpu_bo_reserve(bo, false); @@ -464,13 +464,16 @@ static int amdgpu_amdkfd_validate_vm_bo(void *_unused, struct amdgpu_bo *bo) * again. Page directories are only updated after updating page * tables. */ -static int vm_validate_pt_pd_bos(struct amdgpu_vm *vm) +static int vm_validate_pt_pd_bos(struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket) { struct amdgpu_bo *pd = vm->root.bo; struct amdgpu_device *adev = amdgpu_ttm_adev(pd->tbo.bdev); int ret; - ret = amdgpu_vm_validate_pt_bos(adev, vm, amdgpu_amdkfd_validate_vm_bo, NULL); + ret = amdgpu_vm_validate_evicted_bos(adev, vm, ticket, + amdgpu_amdkfd_validate_vm_bo, + NULL); if (ret) { pr_err("failed to validate PT BOs\n"); return ret; @@ -1310,14 +1313,15 @@ static int map_bo_to_gpuvm(struct kgd_mem *mem, return ret; } -static int process_validate_vms(struct amdkfd_process_info *process_info) +static int process_validate_vms(struct amdkfd_process_info *process_info, + struct ww_acquire_ctx *ticket) { struct amdgpu_vm *peer_vm; int ret; list_for_each_entry(peer_vm, &process_info->vm_list_head, vm_list_node) { - ret = vm_validate_pt_pd_bos(peer_vm); + ret = vm_validate_pt_pd_bos(peer_vm, ticket); if (ret) return ret; } @@ -1402,7 +1406,7 @@ static int init_kfd_vm(struct amdgpu_vm *vm, void **process_info, ret = amdgpu_bo_reserve(vm->root.bo, true); if (ret) goto reserve_pd_fail; - ret = vm_validate_pt_pd_bos(vm); + ret = vm_validate_pt_pd_bos(vm, NULL); if (ret) { pr_err("validate_pt_pd_bos() failed\n"); goto validate_pd_fail; @@ -2043,7 +2047,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( bo->tbo.resource->mem_type == TTM_PL_SYSTEM) is_invalid_userptr = true; - ret = vm_validate_pt_pd_bos(avm); + ret = vm_validate_pt_pd_bos(avm, NULL); if (unlikely(ret)) goto out_unreserve; @@ -2122,7 +2126,7 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( goto unreserve_out; } - ret = vm_validate_pt_pd_bos(avm); + ret = vm_validate_pt_pd_bos(avm, NULL); if (unlikely(ret)) goto unreserve_out; @@ -2620,7 +2624,7 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) } } - ret = process_validate_vms(process_info); + ret = process_validate_vms(process_info, NULL); if (ret) goto unreserve_out; @@ -2880,11 +2884,6 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) amdgpu_sync_create(&sync_obj); - /* Validate PDs and PTs */ - ret = process_validate_vms(process_info); - if (ret) - goto validate_map_fail; - /* Validate BOs and map them to GPUVM (update VM page tables). */ list_for_each_entry(mem, &process_info->kfd_bo_list, validate_list) { @@ -2935,6 +2934,13 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) if (failed_size) pr_debug("0x%lx/0x%lx in system\n", failed_size, total_size); + /* Validate PDs, PTs and evicted DMABuf imports last. Otherwise BO + * validations above would invalidate DMABuf imports again. + */ + ret = process_validate_vms(process_info, &exec.ticket); + if (ret) + goto validate_map_fail; + /* Update mappings not managed by KFD */ list_for_each_entry(peer_vm, &process_info->vm_list_head, vm_list_node) { @@ -3006,7 +3012,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) &process_info->eviction_fence->base, DMA_RESV_USAGE_BOOKKEEP); } - /* Attach eviction fence to PD / PT BOs */ + /* Attach eviction fence to PD / PT BOs and DMABuf imports */ list_for_each_entry(peer_vm, &process_info->vm_list_head, vm_list_node) { struct amdgpu_bo *bo = peer_vm->root.bo; @@ -3014,6 +3020,11 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) dma_resv_add_fence(bo->tbo.base.resv, &process_info->eviction_fence->base, DMA_RESV_USAGE_BOOKKEEP); + + ret = amdgpu_vm_fence_imports(peer_vm, &exec.ticket, + &process_info->eviction_fence->base); + if (ret) + break; } validate_map_fail: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index e50be6500030..d7ba1fd1e3d4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -952,10 +952,10 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, p->bytes_moved = 0; p->bytes_moved_vis = 0; - r = amdgpu_vm_validate_pt_bos(p->adev, &fpriv->vm, - amdgpu_cs_bo_validate, p); + r = amdgpu_vm_validate_evicted_bos(p->adev, &fpriv->vm, NULL, + amdgpu_cs_bo_validate, p); if (r) { - DRM_ERROR("amdgpu_vm_validate_pt_bos() failed.\n"); + DRM_ERROR("amdgpu_vm_validate_evicted_bos() failed.\n"); goto out_free_user_pages; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c index decbbe3d4f06..055ba2ea4c12 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c @@ -377,6 +377,10 @@ amdgpu_dma_buf_move_notify(struct dma_buf_attachment *attach) struct amdgpu_vm_bo_base *bo_base; int r; + /* FIXME: This should be after the "if", but needs a fix to make sure + * DMABuf imports are initialized in the right VM list. + */ + amdgpu_vm_bo_invalidate(adev, bo, false); if (!bo->tbo.resource || bo->tbo.resource->mem_type == TTM_PL_SYSTEM) return; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index b1ce22a9b186..c29653cbde26 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -187,6 +187,32 @@ static int amdgpu_gem_object_open(struct drm_gem_object *obj, else ++bo_va->ref_count; amdgpu_bo_unreserve(abo); + + /* Validate and add eviction fence to DMABuf imports with dynamic + * attachment in compute VMs. Re-validation will be done by + * amdgpu_vm_validate_evicted_bos and the fence will be updated by + * amdgpu_vm_fence_imports in amdgpu_amdkfd_gpuvm_restore_process_bos. + * + * Nested locking below for the case that a GEM object is opened in + * kfd_mem_export_dmabuf. Since the lock below is only taken for imports, + * but not for export, this is a different lock class that cannot lead to + * circular lock dependencies. + */ + if (!vm->is_compute_context || !vm->process_info) + return 0; + if (!obj->import_attach || + !dma_buf_is_dynamic(obj->import_attach->dmabuf)) + return 0; + mutex_lock_nested(&vm->process_info->lock, 1); + if (!WARN_ON(!vm->process_info->eviction_fence)) { + r = amdgpu_amdkfd_bo_validate_and_fence(abo, AMDGPU_GEM_DOMAIN_GTT, + &vm->process_info->eviction_fence->base); + if (r) + dev_warn(adev->dev, "%d: validate_and_fence failed: %d\n", + vm->task_info.pid, r); + } + mutex_unlock(&vm->process_info->lock); + return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index a748e17ff031..2b57281e899f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -442,24 +442,31 @@ uint64_t amdgpu_vm_generation(struct amdgpu_device *adev, struct amdgpu_vm *vm) } /** - * amdgpu_vm_validate_pt_bos - validate the page table BOs + * amdgpu_vm_validate_evicted_bos - validate evicted BOs tracked in the VM * * @adev: amdgpu device pointer * @vm: vm providing the BOs + * @ticket: optional reservation ticket used to reserve the VM * @validate: callback to do the validation * @param: parameter for the validation callback * - * Validate the page table BOs on command submission if neccessary. + * Validate the page table BOs and per-VM BOs on command submission if + * necessary. If a ticket is given, also try to validate evicted user queue + * BOs. They must already be reserved with the given ticket. * * Returns: * Validation result. */ -int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, - int (*validate)(void *p, struct amdgpu_bo *bo), - void *param) +int amdgpu_vm_validate_evicted_bos(struct amdgpu_device *adev, + struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket, + int (*validate)(void *p, + struct amdgpu_bo *bo), + void *param) { struct amdgpu_vm_bo_base *bo_base; struct amdgpu_bo *shadow; + struct dma_resv *resv; struct amdgpu_bo *bo; int r; @@ -499,6 +506,28 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, } spin_lock(&vm->status_lock); } + while (ticket && !list_empty(&vm->evicted_user)) { + bo_base = list_first_entry(&vm->evicted_user, + struct amdgpu_vm_bo_base, + vm_status); + spin_unlock(&vm->status_lock); + + bo = bo_base->bo; + resv = bo->tbo.base.resv; + + if (dma_resv_locking_ctx(resv) != ticket) { + pr_warn_ratelimited("Evicted user BO is not reserved in pid %d\n", + vm->task_info.pid); + return -EINVAL; + } + + r = validate(param, bo); + if (r) + return r; + + amdgpu_vm_bo_invalidated(bo_base); + spin_lock(&vm->status_lock); + } spin_unlock(&vm->status_lock); amdgpu_vm_eviction_lock(vm); @@ -1441,11 +1470,21 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, } r = amdgpu_vm_bo_update(adev, bo_va, clear); - if (r) - return r; if (unlock) dma_resv_unlock(resv); + if (r) + return r; + + /* Remember evicted DMABuf imports in compute VMs for later + * validation + */ + if (vm->is_compute_context && bo_va->base.bo && + bo_va->base.bo->tbo.base.import_attach && + (!bo_va->base.bo->tbo.resource || + bo_va->base.bo->tbo.resource->mem_type == TTM_PL_SYSTEM)) + amdgpu_vm_bo_evicted_user(&bo_va->base); + spin_lock(&vm->status_lock); } spin_unlock(&vm->status_lock); @@ -1453,6 +1492,75 @@ int amdgpu_vm_handle_moved(struct amdgpu_device *adev, return 0; } +/** + * amdgpu_vm_fence_imports - add fence to valid DMABuf imports + * + * @vm: requested vm + * @ticket: optional reservation ticket used to reserve the VM + * @fence: fence to add + * + * Add the specified fence to all dynamic DMABuf imports that are valid. + * + * Returns: + * 0 for success. + */ +int amdgpu_vm_fence_imports(struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket, + struct dma_fence *fence) +{ + struct amdgpu_bo_va *bo_va, *tmp; + struct dma_resv *resv; + LIST_HEAD(imports); + bool unlock; + int ret = 0; + + if (!vm->is_compute_context) + return 0; + + /* Move all the DMABuf imports to a private list so I can reserve + * them while not holding te status_lock. + */ + spin_lock(&vm->status_lock); + list_for_each_entry_safe(bo_va, tmp, &vm->idle, base.vm_status) { + if (bo_va->base.bo && bo_va->base.bo->tbo.base.import_attach && + dma_buf_is_dynamic(bo_va->base.bo->tbo.base.import_attach->dmabuf)) + list_move(&bo_va->base.vm_status, &imports); + } + spin_unlock(&vm->status_lock); + + list_for_each_entry(bo_va, &imports, base.vm_status) { + resv = bo_va->base.bo->tbo.base.resv; + + /* Try to reserve the BO */ + if (dma_resv_trylock(resv)) { + unlock = true; + /* The caller is already holding the reservation lock */ + } else if (ticket && dma_resv_locking_ctx(resv) == ticket) { + unlock = false; + } else { + WARN_ONCE(1, "Failed to reserve DMABuf import"); + ret = -EBUSY; + break; + } + + ret = dma_resv_reserve_fences(resv, 1); + if (!ret) + dma_resv_add_fence(resv, fence, + DMA_RESV_USAGE_BOOKKEEP); + + if (unlock) + dma_resv_unlock(resv); + if (ret) + break; + } + + spin_lock(&vm->status_lock); + list_splice(&imports, &vm->idle); + spin_unlock(&vm->status_lock); + + return ret; +} + /** * amdgpu_vm_flush_compute_tlb - Flush TLB on compute VM * diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index 9156ed22abe7..5de67c57197c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -437,9 +437,12 @@ int amdgpu_vm_lock_pd(struct amdgpu_vm *vm, struct drm_exec *exec, unsigned int num_fences); bool amdgpu_vm_ready(struct amdgpu_vm *vm); uint64_t amdgpu_vm_generation(struct amdgpu_device *adev, struct amdgpu_vm *vm); -int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm, - int (*callback)(void *p, struct amdgpu_bo *bo), - void *param); +int amdgpu_vm_validate_evicted_bos(struct amdgpu_device *adev, + struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket, + int (*callback)(void *p, + struct amdgpu_bo *bo), + void *param); int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job, bool need_pipe_sync); int amdgpu_vm_update_pdes(struct amdgpu_device *adev, struct amdgpu_vm *vm, bool immediate); @@ -449,6 +452,9 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev, int amdgpu_vm_handle_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm, struct ww_acquire_ctx *ticket); +int amdgpu_vm_fence_imports(struct amdgpu_vm *vm, + struct ww_acquire_ctx *ticket, + struct dma_fence *fence); int amdgpu_vm_flush_compute_tlb(struct amdgpu_device *adev, struct amdgpu_vm *vm, uint32_t flush_type, diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c index b8f0c0763c7f..17a2fff50227 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c @@ -1557,9 +1557,11 @@ static int svm_range_reserve_bos(struct svm_validate_context *ctx, bool intr) goto unreserve_out; } - r = amdgpu_vm_validate_pt_bos(pdd->dev->adev, - drm_priv_to_vm(pdd->drm_priv), - svm_range_bo_validate, NULL); + r = amdgpu_vm_validate_evicted_bos(pdd->dev->adev, + drm_priv_to_vm(pdd->drm_priv), + NULL, + svm_range_bo_validate, + NULL); if (r) { pr_debug("failed %d validate pt bos\n", r); goto unreserve_out; @@ -1676,7 +1678,9 @@ static int svm_range_validate_and_map(struct mm_struct *mm, goto free_ctx; } - svm_range_reserve_bos(ctx, intr); + r = svm_range_reserve_bos(ctx, intr); + if (r) + goto free_ctx; p = container_of(prange->svms, struct kfd_process, svms); owner = kfd_svm_page_owner(p, find_first_bit(ctx->bitmap, From patchwork Fri Dec 1 23:34:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 13476620 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 73083C4167B for ; Fri, 1 Dec 2023 23:36:33 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9001D10E993; Fri, 1 Dec 2023 23:36:32 +0000 (UTC) Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2041.outbound.protection.outlook.com [40.107.236.41]) by gabe.freedesktop.org (Postfix) with ESMTPS id 839DF10E091; Fri, 1 Dec 2023 23:36:09 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mhbrJy82AOGTRVJ8IQmY73dWgMKOyYOXIvkC0yGNr8Ff51+Zmmk3teQtSodHTI2+PCtoM20sDlC0RuPLcGkcvLiL0xo9LzPf0nQy48Af8kOfer/BhebltRArvFvo7BhdwRJT0M1bWIsi8JgR+W4TcL8Lp/V+eYYaxbelFEXFoX+Oz0nyd8diUb30ehJtN9aj6Rngqy5YyfZEc539lMSs6qPvyBjm3ekf3P/P6SPG8rdQGeiUixclY92qkzK33FZLxKnIIUkjBOvLzBSWuFKAUJHkONy+vOfUVqEiYsS/+t1SBiUuGogtRf3+wMfn8CQNCmqjcfZGeGTVSMVTo8oQ4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=Aq1H8WCJuAL37SctBUPuPK3Kqv6ruPc+2YmbD5MHcdI=; b=VxCfwEwC9ASAJzOIBVBW+XnuIdblmZJxCkVkAMIAobhQ8klUopeq6tjjxZOw0mz80DXSh8IjXoWsv/ra3VGLA4qMCleSZnEjq5UaduVVQIzJezJkSs5whgdHM0CqniCQzypXtimxgqI6QcfFWWHhf5192N+Mg2/r/P4zR9p8mZ95gxvzxjloAbuxP2trMWnxDcANATH6h++WlyiVFcUNFYHAuTOqmLCUPGSyDFvhATaIeiFXm8eLYFju+H95GeBfNOMkU+8uuECFhSH4n2/PpEt1ZnADLxzBwi3+9Cj//YMK2A0Hh0td8CFinStLpReQMCINm2B5dGgNnjRiU7MMtQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=pass (p=quarantine sp=quarantine pct=100) action=none header.from=amd.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Aq1H8WCJuAL37SctBUPuPK3Kqv6ruPc+2YmbD5MHcdI=; b=nQtDCLhd1LHY/k2SYVaxxBT40QrdMpjt7i95HuXZ/vMWsoN1Dvo3GF+eaKZdxezxRp8dHqgpl7rsp+efJ6eFSCr8vJAC1IH9FkEP8qj+EGX1yKnVvyL5ZRUL+BqI5AgaFXiYtpu6ey6S2l5jY5lwmPfOD8VB6GXcx/kn1HSQ8eM= Received: from BLAPR03CA0100.namprd03.prod.outlook.com (2603:10b6:208:32a::15) by CH3PR12MB8075.namprd12.prod.outlook.com (2603:10b6:610:122::5) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.24; Fri, 1 Dec 2023 23:36:07 +0000 Received: from MN1PEPF0000F0E2.namprd04.prod.outlook.com (2603:10b6:208:32a:cafe::69) by BLAPR03CA0100.outlook.office365.com (2603:10b6:208:32a::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7046.27 via Frontend Transport; Fri, 1 Dec 2023 23:36:07 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=amd.com; Received-SPF: Pass (protection.outlook.com: domain of amd.com designates 165.204.84.17 as permitted sender) receiver=protection.outlook.com; client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C Received: from SATLEXMB04.amd.com (165.204.84.17) by MN1PEPF0000F0E2.mail.protection.outlook.com (10.167.242.40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.7046.17 via Frontend Transport; Fri, 1 Dec 2023 23:36:07 +0000 Received: from Harpoon.amd.com (10.180.168.240) by SATLEXMB04.amd.com (10.181.40.145) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2507.34; Fri, 1 Dec 2023 17:36:06 -0600 From: Felix Kuehling To: , Subject: [PATCH 6/6] drm/amdkfd: Bump KFD ioctl version Date: Fri, 1 Dec 2023 18:34:38 -0500 Message-ID: <20231201233438.1709981-6-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20231201233438.1709981-1-Felix.Kuehling@amd.com> References: <20231201233438.1709981-1-Felix.Kuehling@amd.com> MIME-Version: 1.0 X-Originating-IP: [10.180.168.240] X-ClientProxiedBy: SATLEXMB03.amd.com (10.181.40.144) To SATLEXMB04.amd.com (10.181.40.145) X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MN1PEPF0000F0E2:EE_|CH3PR12MB8075:EE_ X-MS-Office365-Filtering-Correlation-Id: 1379b0b8-bcd6-4e42-e371-08dbf2c64a85 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: O/DzbpfPqu8MAvm05IrbnlDNzHJDr6e4XnN/UDd77oJ0nMO8Lb9+dG4Uy7z47mbbB5EeCJSmn+g7pOVt3+No2Mqte59tk1OK0SgDDQ+5c1SX3FocITZEOSobD8sYpALMcLreKWg6HaeV3VNC8KnsB8KcQU1byhig0Sx6GdaYM3BLLWP5ABO9dfI/T6CGKrKBT1sRA5a6JQs2hcMD4v7oberEktSRkz1lplkofAg4hf++khdehoqelTOMPs8ytDwHKV+PMir4SW8mPHT1QVk668VU4Fa3dnWKscygsHvZM6BtlCZAJ4WYrIvLYg4ULt/84LPIy+B+s/0G/5rIJOpPH78vPSO8qdV4pTFFtRv2dDh2Fyy+u/PD+LcH3uvi3Y8E2n3SwdLKFFThLu/FI00lPKcEee1ArcrapV887O00U7zOebyEK1xEuqyf9sobbsRzVufofQfy65IK9tu+Tswotk1pKZc7JR5MNYcoqSjLn68uVAcOgiRcufbt2iATAGTIwrKlpyguj8TFWEiQDflJH+cjPU3q/T7KWMlcFkYxTgwpn0X5/tszxvbPxRCGMLK0GKiizb1Grubhe3hU9uYRLQZBeutEZfuSFlGCecX14yTBscNr/XfIIXMgTiOA560o7ifHygGUNZh3KXs1Agebure0w2I1X1RBPGWfkWQ6fc2REnZRt0Z994YFONoXFtpTDTl9pV5wfEQRf+0XFIhyWu+6IgI1nDXJVYvwkVrDuySStD30et4v80VI9WSo7CPIeXcG8EqUhV45NNleii10PA== X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:SATLEXMB04.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFS:(13230031)(4636009)(39860400002)(346002)(376002)(136003)(396003)(230922051799003)(1800799012)(82310400011)(64100799003)(451199024)(186009)(36840700001)(40470700004)(46966006)(40460700003)(83380400001)(86362001)(26005)(1076003)(6666004)(7696005)(2616005)(47076005)(36860700001)(41300700001)(426003)(110136005)(5660300002)(8936002)(336012)(8676002)(478600001)(4744005)(450100002)(16526019)(2906002)(316002)(81166007)(70586007)(70206006)(356005)(36756003)(82740400003)(40480700001)(36900700001); DIR:OUT; SFP:1101; X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Dec 2023 23:36:07.0323 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1379b0b8-bcd6-4e42-e371-08dbf2c64a85 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB04.amd.com] X-MS-Exchange-CrossTenant-AuthSource: MN1PEPF0000F0E2.namprd04.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8075 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is not strictly a change in the IOCTL API. This version bump is meant to indicate to user mode the presence of a number of changes and fixes that enable the management of VA mappings in compute VMs using the GEM_VA ioctl for DMABufs exported from KFD. Signed-off-by: Felix Kuehling --- include/uapi/linux/kfd_ioctl.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/include/uapi/linux/kfd_ioctl.h b/include/uapi/linux/kfd_ioctl.h index f0ed68974c54..9ce46edc62a5 100644 --- a/include/uapi/linux/kfd_ioctl.h +++ b/include/uapi/linux/kfd_ioctl.h @@ -40,9 +40,10 @@ * - 1.12 - Add DMA buf export ioctl * - 1.13 - Add debugger API * - 1.14 - Update kfd_event_data + * - 1.15 - Enable managing mappings in compute VMs with GEM_VA ioctl */ #define KFD_IOCTL_MAJOR_VERSION 1 -#define KFD_IOCTL_MINOR_VERSION 14 +#define KFD_IOCTL_MINOR_VERSION 15 struct kfd_ioctl_get_version_args { __u32 major_version; /* from KFD */