From patchwork Thu Apr 22 01:30:49 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217281 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.5 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, UPPERCASE_50_75,USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2B3C4C433ED for ; Thu, 22 Apr 2021 01:31:38 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CCA63613E0 for ; Thu, 22 Apr 2021 01:31:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CCA63613E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5F2A36E98A; Thu, 22 Apr 2021 01:31:23 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062.outbound.protection.outlook.com [40.107.93.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id F30446E0DE; Thu, 22 Apr 2021 01:31:21 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=HB940/s1wVS+mshpvZ2iPWQv+nUrHsvjeRelV2uzyp+O4UMEui/OcssSoZwvljJT+ZPfch07vsP6+cSULZYYbQyjTjrMQV8rwB52xwK+Aejdx2gtOMjy34Y8wCp0tGhWnOg3CFMNRkfvZ8aRYHwAauTT1AgN4UP5TM8wMldLrkzF1y60/RqBpdAZE5CyVJFUFWdSD4WLEojt0K/vX6meiDvmjPjaerz+fuqBV6GGL3TZnbqRpfjbd+WOxJTT5FORbfF+eB/aTx9rD10r/jtUu1h7Iqh4Ghfyie7Httgn3tLyUJnGEXmxGsFN4XEdfTtVgl9mqrUD/OiQ6wJoSRygag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uiGDNPyB9sauXjIoK3sHx2ycVrVYGwVdzXautwbXQvo=; b=jau+5/PbKig7aEehQHgPhVDYP0eZGzhfa1poFHv47uUYeFoU4KlkF60CKJ2B+8ZQcLv5BU2uCJ0RcWYoK9xrgiqnPuT1U4NkkJrGRWnp63Aw0x3IjFfRTqqfl61m59I4CfgcxSkWELZ+8soFsUzE7YMnVQHjM2YwRiD3PWzwo1G7TLjDK1qsN32sdwqiVs54Ns9rT8hW4IWVtvUuPtS5X1swQDmYhfm4XFt8tM2eHCd620XJkOT9tjQymId8QPNT9sI+Yd+0/arkRL1Sv5usolv4BdZ+QJwlg4Y2+RO3U7ZTcQ7ovm5jtfkeHo6Y+r2TJeQkZ4/sxvgWTikce4hKpA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=uiGDNPyB9sauXjIoK3sHx2ycVrVYGwVdzXautwbXQvo=; b=s4VN6mcy/7guGAByigoGKvoL22rg/fG//e4LG1kTN5S8F1cZ6P35VEyIamgsRgWpUJvPrOh4dpOXex7K3XHcacdAPGKn/T10I6ydi4Nhx3eM2pEW1kfH4VA4sT/8dS1cZxC1K2zDD/Me9aUQ/gP0yGDV3L68efXuOkfgaP60Dqo= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:15 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:15 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 01/10] rock-dbg_defconfig: Enable Intel IOMMU Date: Wed, 21 Apr 2021 21:30:49 -0400 Message-Id: <20210422013058.6305-2-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:14 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 503c7d7b-8c36-40f9-5d38-08d9052e51f0 X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4125; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: w7eGUaxuvg9E3eV6zlDSImjjjE1q/tHATFmyGhrTQulo/jOQX+OdZxaML0z2gC6BEGViYXpSch4XhZppkQI49QgEvXcfIpF9C6jKNg5EViOvqwZ2AwRkcBM4no8p1J8TEkPPj0K+OKcEwAzLUnlpV7apBZ08SrhlAWQfY4ea265l1Flh5B8FCu7y/JdnbqRkYONaavPiSlEiaaS3r7X1sOKZK1HDWX16AShBfwF+67k5XIouqDrS+Z6gwi7bVREKU10Wo2YWISQBwk9gITxdaqwWioWZV9IPrTwrZLoaw5J8+gp9LbAyPlZaojQEnMhc38VfB8vPIX2Uh4ubuqm0KhwGMk2fii2+Mnwahm++35nfb7aBVS/OvEKZdYqKTVFk2+Bet31mCyfK7bK69nxeMkOfL7ww3hEKYpwCVSzNfrHM7xLMzLbLxcUxRH/fn9ba1Nap6pR5yQD4YN8btAkd2pn82YLJ0OzPdhBFu8p9dDrrRWFEnwmYCILdzDbXSx9r+8C0xeL70JXopzV+OqzbP6w+G1iaGXdBoZqxQNkwCP4HHAhhyLk+J1SbdPwcK1W39dJVhBJQPR5mojjCInv76YhXBvdcB0/NjLYdn71aO4YQFGnG9DDiO45eQkTmZrA4Mq8QF+kMW0iRjb+LGZ4PKvFNR2YX0bxUfXXBZltib7Q= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: LYokfIyxEAjN5zPzrydrywBOaIDYyilHeLvjyZv3ifKFc93w19+8ppFQo7JBlmgiwk6mZYEI6CyLXH6gyDe/ezUq/WHawU/QlcqJzJFn16krwF91f+Q73RtXbggIh4IWhLRbYu3NP+GLYVEIUy/TekN0y+K8Npenumq2W4VIHMUEtxrFiYTXYWWZdSXXeMd8u+m8o8xL9IVcZ7snkxNFS18bg89Zs2ti/C6fTYmeF+tYWYkYzhROB7m5VIYg8iUUAWxKYL+/zdOltvOqO5sOMuOFqtPtPxns3/cwM6I8tXqwNpuVKdxyWFPI66LSH+0aHZc9XYcIHSzH6m8xCqftoSR1VEtsVQAQZOcP9JmdAafHtKilqxvrSXiMU2oK4dWQcwnYyL9U+JIrXT5L5nh/G+0QsyL5jg+Ua6U41hdLDIgwRmoCofnu9SiNWnK3SiFxOVnpno74/Il8LXKfsGZNw+nCxJCeRVExz3+gOEQ/tvOBuzYVqdvalIRMtEKXmtfzSr3JKZSk1ZY0hpp9ZKJdZehSDpY3DzeZ+d46xFsIqhR9iQXhFkEdKnfgInC3uhExCgj79gfgO//GqPILYkhwhET9BxiscdIGNZqXbzyJuNMoPXI72ag/hHR95wnp5bX4PpwKIr2r1Umcf7TDndWYiZ4t/Z8N/6Dbbop8eEzgXYFhD5tbP/G0XUZJ8kYVTU59IAz5N2yWJEeWST1kkrBZsW6QJPp6dnh4ciyxrwzP1XGTmy/s+lLZn8Nbdlh6UoDXT58ANuoLiS2pwaDJ7LwiFppIAMfAhzVr8/JYaOmlRn+lJwvhlNvfxQ8TjCK181t7HQh0v/ecm4pDwNcejUdzI2gVPK5SJ9tA6nZdvAG9CtpaAPViD8xslw1yB262uFbkdcmtAfBgBg8RuzATdugJvUGoGk/BYFx4mCNnhhMEBOQvLQrDSUlrX/gyXRux80f+aIsJSisKUsQCT0sOyEIfgXmcKQ3gLQGBvyhTKVUJIHcx2fhwRq9XRAEQrxq5cJYCda+kp52jRG/BM6NHQfxY3hujccR5eud6edj5uA6qHHMNMVb9ywrjDWdkKva9Ns7uZHZmF8+PjcBLY3WEFr9P7VCLJwmsprlCxiSsA0//c2PCkIGdW9GFh695t8/ool9RuyZ/U0Z97MEWNy/7HLwZEKK+UvjdShlK0Xs0cUbA8TkYPrsRUrSO9vCCdTECD9WQq2k/cTq5mHkj0J0SfOYHRwvj2ManoK4fBD1WmVoHZ1vHyXp31ZMJgQqmBhoYiYsq1DjNXb+87c/M5m0SCneLKpYqdXbrvgLBpHEWEAhIqaXfzg785WndPl66zxH+kKiF X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 503c7d7b-8c36-40f9-5d38-08d9052e51f0 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:15.3665 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Vkt1hAvYVsUyno8D8iDgxTSsOUUjpwSQKBphD19PUFvPwrD6WsKRsqfB8UWNclcVtaM4hkGTL1522UaCI+dw5g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Enable the Intel IOMMU driver in the rock-dbg_defconfig. This enables testing of DMA mappings on systems with an Intel IOMMU. Signed-off-by: Felix Kuehling --- arch/x86/configs/rock-dbg_defconfig | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/arch/x86/configs/rock-dbg_defconfig b/arch/x86/configs/rock-dbg_defconfig index 54688993d6e2..9f7d93307754 100644 --- a/arch/x86/configs/rock-dbg_defconfig +++ b/arch/x86/configs/rock-dbg_defconfig @@ -296,6 +296,7 @@ CONFIG_ARCH_SUSPEND_POSSIBLE=y CONFIG_ARCH_WANT_GENERAL_HUGETLB=y CONFIG_ZONE_DMA32=y CONFIG_AUDIT_ARCH=y +CONFIG_HAVE_INTEL_TXT=y CONFIG_X86_64_SMP=y CONFIG_ARCH_SUPPORTS_UPROBES=y CONFIG_FIX_EARLYCON_MEM=y @@ -3112,6 +3113,7 @@ CONFIG_DRM_AMD_DC_DCN=y # end of Display Engine Configuration CONFIG_HSA_AMD=y +CONFIG_HSA_AMD_SVM=y # CONFIG_DRM_NOUVEAU is not set # CONFIG_DRM_I915 is not set # CONFIG_DRM_VGEM is not set @@ -3770,6 +3772,7 @@ CONFIG_MAILBOX=y CONFIG_PCC=y # CONFIG_ALTERA_MBOX is not set CONFIG_IOMMU_IOVA=y +CONFIG_IOASID=y CONFIG_IOMMU_API=y CONFIG_IOMMU_SUPPORT=y @@ -3783,7 +3786,12 @@ CONFIG_IOMMU_SUPPORT=y CONFIG_IOMMU_DMA=y CONFIG_AMD_IOMMU=y CONFIG_AMD_IOMMU_V2=m -# CONFIG_INTEL_IOMMU is not set +CONFIG_DMAR_TABLE=y +CONFIG_INTEL_IOMMU=y +# CONFIG_INTEL_IOMMU_SVM is not set +CONFIG_INTEL_IOMMU_DEFAULT_ON=y +CONFIG_INTEL_IOMMU_FLOPPY_WA=y +# CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON is not set # CONFIG_IRQ_REMAP is not set # @@ -4184,6 +4192,7 @@ CONFIG_SECURITY_NETWORK=y CONFIG_PAGE_TABLE_ISOLATION=y # CONFIG_SECURITY_NETWORK_XFRM is not set # CONFIG_SECURITY_PATH is not set +# CONFIG_INTEL_TXT is not set CONFIG_LSM_MMAP_MIN_ADDR=65536 CONFIG_HAVE_HARDENED_USERCOPY_ALLOCATOR=y # CONFIG_HARDENED_USERCOPY is not set From patchwork Thu Apr 22 01:30:50 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217293 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 843E5C433ED for ; Thu, 22 Apr 2021 01:31:50 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 3DCEE613B0 for ; Thu, 22 Apr 2021 01:31:50 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 3DCEE613B0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0F69B6EA29; Thu, 22 Apr 2021 01:31:27 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062.outbound.protection.outlook.com [40.107.93.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id EB8E46E9B9; Thu, 22 Apr 2021 01:31:22 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BMfjgD+c4/jLJ3PGF99L0WzlseGHc/N0Pa+bv/fjnowDnGBPviZ0GvyFDhBA53wqYCSgBufftHyJ2vIseCg50RTyRa8vRPZOUDgJzsmvXW4qy0bCA6xYJvwu616EhynBD1uaAKQ4/HhkJ3HLAj68yD1QKZlTR3N8xgm4lxXdbMiUueeSzH85/14d6zUj19Hjg9SQQEcuF2vG1Vdy7SVlTbkOItODRu9PZfmJjQIrHb0ZQJAOLgDLgzrEupqF6eqIPLgUjC1necZ1RbQQXY2uxNATxpJXa33b+OYjI9FpBKGqF4QwxijZX9hzp7i/YN0j38pBcf6QILYwbMnv88YryQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/OSNqCVUPE4iCtBCcI4bMlwtQqGaY6BGBy/DPxKZQRg=; b=J4nnhbHOQNF0eYz4mrQ8YbuS/KrMXXdwzyz7Bhtcca4GJgWryLPrRzqgiAOoNWPUdf/ZegB0VcLrP+Eua1nMU56jg1cX0yuIJf8vA8t28R533SKKhQMRTmmo8DVwT3F6/fa2/bNJqEuCG0RCsMff4ihQF5QPTG0K4wp1YaMx0XpseSh3FMmzPLF8WemCmhB10PsYc8o8Nom1f9OMRpnLFlhc4xWykx6Exct7zkGt1UBDXwmj2XCKJ28hX7LTpndcTpyabc9eMW1jyv69YRgfQsUyVezJUOKqkuWeUrcxllt/7hb6lfrY08+tJ3RvTds694JjbpnvTWuGSm/CUBN8/Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/OSNqCVUPE4iCtBCcI4bMlwtQqGaY6BGBy/DPxKZQRg=; b=gNhjN9z+HiMN+jO5PpH2eLsxmJ23Ct3V9Gm31drUk+xoqWNAOwPrFR0k5YM7T15lYFxo4/IHH0TLapal7ONQqgd5DX8ihcrUtUHCVIpEeap+z9KpZ3il5qrsJclZ1XOTmvsCbhBcL/BZT6hJuN/IJDZEws7GJPO/2DSbF5rpZ6k= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:15 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:15 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 02/10] drm/amdgpu: Rename kfd_bo_va_list to kfd_mem_attachment Date: Wed, 21 Apr 2021 21:30:50 -0400 Message-Id: <20210422013058.6305-3-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:15 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0debc9d2-a24a-4991-baec-08d9052e5240 X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8fN9TQOvk06sY5piT97zvl29h9WbvhcP0YXQr82koOxJk5pNsmTu4NZVNVkfWXh1feV4K+rjoWJX2sE+TThnZYSNC3mNciN87NgkSShoaQbM+7qADVgGZprGqcLeOVEINKMQswmVYKSMcEyKWnoWC/k7IzQ1voh1KFsOasH3yFVzqHO22Wer3YIsqV4Z9/fxtFTHOaPVnpbgbknOAHjj78Xzror7MPJruQRPC+AE+ZX6avYX5GLOqfqGPakIkMGSyzUMUxAeIX1ax8bSrm+oTPi+BdqAsYtDr8uB2IcGilsZMC34HTAlBdD8+JLc1a8NrR/Tb86ZsLsQe6eqErJbv51WrA4gxlhfTRtzcNyA0KuArFjv5caM/VvG9EGhMO+VHPoT8gyBEGfxZcK7TsX7xUfBB3I8Lq5w8zrvyFzQLUdRGXdGto5Gt9rEUIuLZ5ZaQgSe4KHYhLwSy8DKyi0i6vSxa4qI0Xfn/+5GkQxMltqIgoWSvQ7IqfwhxapuMDael10n58Ah9cfdllorRYZjTROwJE5fW9Ew0b2/Xpvjk8Fj6CzqMl1CjpNqhdMNJ5H86Bsd8x6YcvELA2psZjGLHz7c2AA/Slx5zC6Q6SdP4KNY84abv3UXE5DpaQrmxgFHEdu0+uYncTWES2bC5gTR28KD2YDMRvjZs3jhkYBkEL4= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(30864003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: UXesu930t6fjQYMYfYUcjkF8uk8sNwne7GCVMO6iURCPbmkgA6UQB+XEJ/28BiaQchsP4b99+puMDlktqx1UF4fN3TBMDKy6jt8O4/3/KEdUrGykg+jAjyB4KwGCh/ozhtJpjN3hyOMeoJ05Q5ONwuXCb94Job5KYHIsDbd5wAvf5Ah3O5wcabFKvvSbHQOskx354nT0imcfa1Vk+q7xqsk5Gt9uROTDveA0JAknepZxCQb0VIlKPIebSTt1feahvZd8dY5jPtL4kITLcnyqWwjsnUPfErc7uo8fUTVrvMvHs4hPq5D0KhL2NeWQiz7hcW+GIZyi1s3y0MLN2Kzjqfe52gOJH1i3T44aAb0HLOiNQLS3dSopi0nwTEv3hEv5iyMrxnhXnR11DljKTQE43ShX9+AVFYLXMtCC0wDxs7B364KLs5X+Dbu3N0E/Z6UTKs0sEqHpdLcAs3GAo9s/EVu9KJUOPwWmmBiRZKrGpSi2hN/jNglPhjA9XtYv2pZpi13lZx9u/ir4aEynmOrS43mdqC3C9OcSYTSEhcYITYOhLDSj208lEeXUp107YAxRtUyPGFcdR167ZBOV2Z7JIMDDN24Ulr63qnhsO55rl/aUjae9CuVZv7DKZseFgD8+SAkGeTIfTT7kmtwbTMVcQKqpTuOIT+Uvyx98ca7KmBbhvaR+UVF9cSKrZe+GTbbX5TfQ3pGfYlv9gKQ10nrZCcfug82uRDSxHl2UxoWoLGkrN1FXIVm+CRKcQW+nTvtAUC22LdzZrZrmtk87Q1dNCZOPN82hAN+kzmSdt+azHAPknHlN7z+J4qkYK/h2TP2PJdh0orU3erlvyUfn17siYLfK1GwU6PBU8dH49qqPEM7rNAxW5QRSe2rN5u2oNKfh5kcYypVQkic5mX3S0GGMFXmAKYs5WpZxtlQwgnMJ921fetDG51zjbfirIsna6J89xLVdRsg3TIr67YfjHTwsDJztMTWBuVuCPzpLdyOLVcHEhQ3LpXGSaONO45Gitc9MtBQFqsiZIilwKsqvOuwHXA0ftlQajuz5Ab6Z9KUOA1vamhE2zlf7E7HF0z3ORHBnTqvrLe63ZTjvaRxsGMR4sPy+7Ibfl8sfSbFQOr+RUK8/GkJxZUKh+Qpa70CwUpLEQ5TJ68qFDu6QOLXEs9UoE+XcoP2izgPF1/EL/ZxZ8Ze/GHGr/9X41FeOCO5F2Ugvl0Pbw7TzsXj8cNoCN1iMnY7r8h1m/ej+ViE2sP/40upfs3B7yeSHiJMQAb2VZ7d7S8nn+s/Dgl7GG/duoBSsrcfV8i+/sBrBPYdhMeyGh6qMKOdIz6yuXr4gkKk4kUV4 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0debc9d2-a24a-4991-baec-08d9052e5240 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:15.7578 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RbumvE046H6hZoEu6RN5ycpQJHgd6ugsTD6VEX+HpQIwF4Tt7OJatHOsmSQ6pmm2aUr7M0chI4zFzhyPEgLoag== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This name is more fitting, especially for the changes coming next to support multi-GPU systems with proper DMA mappings. Cleaned up the code and renamed some related functions and variables to improve readability. Signed-off-by: Felix Kuehling Acked-by: Ramesh Errabolu Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 8 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 209 +++++++++--------- 2 files changed, 104 insertions(+), 113 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 313ee49b9f17..c24b2478f445 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -38,10 +38,10 @@ extern uint64_t amdgpu_amdkfd_total_mem_size; struct amdgpu_device; -struct kfd_bo_va_list { - struct list_head bo_list; +struct kfd_mem_attachment { + struct list_head list; struct amdgpu_bo_va *bo_va; - void *kgd_dev; + struct amdgpu_device *adev; bool is_mapped; uint64_t va; uint64_t pte_flags; @@ -50,7 +50,7 @@ struct kfd_bo_va_list { struct kgd_mem { struct mutex lock; struct amdgpu_bo *bo; - struct list_head bo_va_list; + struct list_head attachments; /* protected by amdkfd_process_info.lock */ struct ttm_validate_buffer validate_list; struct ttm_validate_buffer resv_list; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index dfa025d694f8..fee4c64dd051 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -72,16 +72,16 @@ static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) return (struct amdgpu_device *)kgd; } -static bool check_if_add_bo_to_vm(struct amdgpu_vm *avm, +static bool kfd_mem_is_attached(struct amdgpu_vm *avm, struct kgd_mem *mem) { - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; - list_for_each_entry(entry, &mem->bo_va_list, bo_list) + list_for_each_entry(entry, &mem->attachments, list) if (entry->bo_va->base.vm == avm) - return false; + return true; - return true; + return false; } /* Set memory usage limits. Current, limits are @@ -473,7 +473,7 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) return pte_flags; } -/* add_bo_to_vm - Add a BO to a VM +/* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added * to a VM. It can later be mapped and unmapped many times without @@ -485,15 +485,14 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) * 4. Alloc page tables and directories if needed * 4a. Validate new page tables and directories */ -static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, +static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_vm *vm, bool is_aql, - struct kfd_bo_va_list **p_bo_va_entry) + struct kfd_mem_attachment **p_attachment) { int ret; - struct kfd_bo_va_list *bo_va_entry; + struct kfd_mem_attachment *attachment; struct amdgpu_bo *bo = mem->bo; uint64_t va = mem->va; - struct list_head *list_bo_va = &mem->bo_va_list; unsigned long bo_size = bo->tbo.base.size; if (!va) { @@ -504,29 +503,29 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, if (is_aql) va += bo_size; - bo_va_entry = kzalloc(sizeof(*bo_va_entry), GFP_KERNEL); - if (!bo_va_entry) + attachment = kzalloc(sizeof(*attachment), GFP_KERNEL); + if (!attachment) return -ENOMEM; pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, va + bo_size, vm); /* Add BO to VM internal data structures*/ - bo_va_entry->bo_va = amdgpu_vm_bo_add(adev, vm, bo); - if (!bo_va_entry->bo_va) { + attachment->bo_va = amdgpu_vm_bo_add(adev, vm, bo); + if (!attachment->bo_va) { ret = -EINVAL; pr_err("Failed to add BO object to VM. ret == %d\n", ret); goto err_vmadd; } - bo_va_entry->va = va; - bo_va_entry->pte_flags = get_pte_flags(adev, mem); - bo_va_entry->kgd_dev = (void *)adev; - list_add(&bo_va_entry->bo_list, list_bo_va); + attachment->va = va; + attachment->pte_flags = get_pte_flags(adev, mem); + attachment->adev = adev; + list_add(&attachment->list, &mem->attachments); - if (p_bo_va_entry) - *p_bo_va_entry = bo_va_entry; + if (p_attachment) + *p_attachment = attachment; /* Allocate validate page tables if needed */ ret = vm_validate_pt_pd_bos(vm); @@ -538,22 +537,20 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, return 0; err_alloc_pts: - amdgpu_vm_bo_rmv(adev, bo_va_entry->bo_va); - list_del(&bo_va_entry->bo_list); + amdgpu_vm_bo_rmv(adev, attachment->bo_va); + list_del(&attachment->list); err_vmadd: - kfree(bo_va_entry); + kfree(attachment); return ret; } -static void remove_bo_from_vm(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, unsigned long size) +static void kfd_mem_detach(struct kfd_mem_attachment *attachment) { - pr_debug("\t remove VA 0x%llx - 0x%llx in entry %p\n", - entry->va, - entry->va + size, entry); - amdgpu_vm_bo_rmv(adev, entry->bo_va); - list_del(&entry->bo_list); - kfree(entry); + pr_debug("\t remove VA 0x%llx in entry %p\n", + attachment->va, attachment); + amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); + list_del(&attachment->list); + kfree(attachment); } static void add_kgd_mem_to_kfd_bo_list(struct kgd_mem *mem, @@ -728,7 +725,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, struct bo_vm_reservation_context *ctx) { struct amdgpu_bo *bo = mem->bo; - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; unsigned int i; int ret; @@ -740,7 +737,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, INIT_LIST_HEAD(&ctx->list); INIT_LIST_HEAD(&ctx->duplicates); - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { + list_for_each_entry(entry, &mem->attachments, list) { if ((vm && vm != entry->bo_va->base.vm) || (entry->is_mapped != map_type && map_type != BO_VM_ALL)) @@ -762,7 +759,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, list_add(&ctx->kfd_bo.tv.head, &ctx->list); i = 0; - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { + list_for_each_entry(entry, &mem->attachments, list) { if ((vm && vm != entry->bo_va->base.vm) || (entry->is_mapped != map_type && map_type != BO_VM_ALL)) @@ -817,7 +814,7 @@ static int unreserve_bo_and_vms(struct bo_vm_reservation_context *ctx, } static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, + struct kfd_mem_attachment *entry, struct amdgpu_sync *sync) { struct amdgpu_bo_va *bo_va = entry->bo_va; @@ -833,7 +830,7 @@ static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, } static int update_gpuvm_pte(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, + struct kfd_mem_attachment *entry, struct amdgpu_sync *sync) { int ret; @@ -850,7 +847,7 @@ static int update_gpuvm_pte(struct amdgpu_device *adev, } static int map_bo_to_gpuvm(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, struct amdgpu_sync *sync, + struct kfd_mem_attachment *entry, struct amdgpu_sync *sync, bool no_update_pte) { int ret; @@ -1194,7 +1191,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( ret = -ENOMEM; goto err; } - INIT_LIST_HEAD(&(*mem)->bo_va_list); + INIT_LIST_HEAD(&(*mem)->attachments); mutex_init(&(*mem)->lock); (*mem)->aql_queue = !!(flags & KFD_IOC_ALLOC_MEM_FLAGS_AQL_QUEUE_MEM); @@ -1283,7 +1280,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( { struct amdkfd_process_info *process_info = mem->process_info; unsigned long bo_size = mem->bo->tbo.base.size; - struct kfd_bo_va_list *entry, *tmp; + struct kfd_mem_attachment *entry, *tmp; struct bo_vm_reservation_context ctx; struct ttm_validate_buffer *bo_list_entry; unsigned int mapped_to_gpu_memory; @@ -1327,9 +1324,8 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( mem->va + bo_size * (1 + mem->aql_queue)); /* Remove from VM internal data structures */ - list_for_each_entry_safe(entry, tmp, &mem->bo_va_list, bo_list) - remove_bo_from_vm((struct amdgpu_device *)entry->kgd_dev, - entry, bo_size); + list_for_each_entry_safe(entry, tmp, &mem->attachments, list) + kfd_mem_detach(entry); ret = unreserve_bo_and_vms(&ctx, false, false); @@ -1372,10 +1368,10 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( int ret; struct amdgpu_bo *bo; uint32_t domain; - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; - struct kfd_bo_va_list *bo_va_entry = NULL; - struct kfd_bo_va_list *bo_va_entry_aql = NULL; + struct kfd_mem_attachment *attachment = NULL; + struct kfd_mem_attachment *attachment_aql = NULL; unsigned long bo_size; bool is_invalid_userptr = false; @@ -1424,21 +1420,20 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( bo->tbo.mem.mem_type == TTM_PL_SYSTEM) is_invalid_userptr = true; - if (check_if_add_bo_to_vm(avm, mem)) { - ret = add_bo_to_vm(adev, mem, avm, false, - &bo_va_entry); + if (!kfd_mem_is_attached(avm, mem)) { + ret = kfd_mem_attach(adev, mem, avm, false, &attachment); if (ret) - goto add_bo_to_vm_failed; + goto attach_failed; if (mem->aql_queue) { - ret = add_bo_to_vm(adev, mem, avm, - true, &bo_va_entry_aql); + ret = kfd_mem_attach(adev, mem, avm, true, + &attachment_aql); if (ret) - goto add_bo_to_vm_failed_aql; + goto attach_failed_aql; } } else { ret = vm_validate_pt_pd_bos(avm); if (unlikely(ret)) - goto add_bo_to_vm_failed; + goto attach_failed; } if (mem->mapped_to_gpu_memory == 0 && @@ -1454,30 +1449,30 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( } } - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { - if (entry->bo_va->base.vm == avm && !entry->is_mapped) { - pr_debug("\t map VA 0x%llx - 0x%llx in entry %p\n", - entry->va, entry->va + bo_size, - entry); + list_for_each_entry(entry, &mem->attachments, list) { + if (entry->bo_va->base.vm != avm || entry->is_mapped) + continue; - ret = map_bo_to_gpuvm(adev, entry, ctx.sync, - is_invalid_userptr); - if (ret) { - pr_err("Failed to map bo to gpuvm\n"); - goto map_bo_to_gpuvm_failed; - } + pr_debug("\t map VA 0x%llx - 0x%llx in entry %p\n", + entry->va, entry->va + bo_size, entry); - ret = vm_update_pds(avm, ctx.sync); - if (ret) { - pr_err("Failed to update page directories\n"); - goto map_bo_to_gpuvm_failed; - } + ret = map_bo_to_gpuvm(adev, entry, ctx.sync, + is_invalid_userptr); + if (ret) { + pr_err("Failed to map bo to gpuvm\n"); + goto map_bo_to_gpuvm_failed; + } - entry->is_mapped = true; - mem->mapped_to_gpu_memory++; - pr_debug("\t INC mapping count %d\n", - mem->mapped_to_gpu_memory); + ret = vm_update_pds(avm, ctx.sync); + if (ret) { + pr_err("Failed to update page directories\n"); + goto map_bo_to_gpuvm_failed; } + + entry->is_mapped = true; + mem->mapped_to_gpu_memory++; + pr_debug("\t INC mapping count %d\n", + mem->mapped_to_gpu_memory); } if (!amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) && !bo->tbo.pin_count) @@ -1489,12 +1484,12 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( goto out; map_bo_to_gpuvm_failed: - if (bo_va_entry_aql) - remove_bo_from_vm(adev, bo_va_entry_aql, bo_size); -add_bo_to_vm_failed_aql: - if (bo_va_entry) - remove_bo_from_vm(adev, bo_va_entry, bo_size); -add_bo_to_vm_failed: + if (attachment_aql) + kfd_mem_detach(attachment_aql); +attach_failed_aql: + if (attachment) + kfd_mem_detach(attachment); +attach_failed: unreserve_bo_and_vms(&ctx, false, false); out: mutex_unlock(&mem->process_info->lock); @@ -1509,7 +1504,7 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); struct amdkfd_process_info *process_info = avm->process_info; unsigned long bo_size = mem->bo->tbo.base.size; - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; int ret; @@ -1533,26 +1528,24 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( mem->va + bo_size * (1 + mem->aql_queue), avm); - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { - if (entry->bo_va->base.vm == avm && entry->is_mapped) { - pr_debug("\t unmap VA 0x%llx - 0x%llx from entry %p\n", - entry->va, - entry->va + bo_size, - entry); + list_for_each_entry(entry, &mem->attachments, list) { + if (entry->bo_va->base.vm != avm || !entry->is_mapped) + continue; - ret = unmap_bo_from_gpuvm(adev, entry, ctx.sync); - if (ret == 0) { - entry->is_mapped = false; - } else { - pr_err("failed to unmap VA 0x%llx\n", - mem->va); - goto unreserve_out; - } + pr_debug("\t unmap VA 0x%llx - 0x%llx from entry %p\n", + entry->va, entry->va + bo_size, entry); - mem->mapped_to_gpu_memory--; - pr_debug("\t DEC mapping count %d\n", - mem->mapped_to_gpu_memory); + ret = unmap_bo_from_gpuvm(adev, entry, ctx.sync); + if (ret == 0) { + entry->is_mapped = false; + } else { + pr_err("failed to unmap VA 0x%llx\n", mem->va); + goto unreserve_out; } + + mem->mapped_to_gpu_memory--; + pr_debug("\t DEC mapping count %d\n", + mem->mapped_to_gpu_memory); } /* If BO is unmapped from all VMs, unfence it. It can be evicted if @@ -1701,7 +1694,7 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct kgd_dev *kgd, if (mmap_offset) *mmap_offset = amdgpu_bo_mmap_offset(bo); - INIT_LIST_HEAD(&(*mem)->bo_va_list); + INIT_LIST_HEAD(&(*mem)->attachments); mutex_init(&(*mem)->lock); (*mem)->alloc_flags = @@ -1898,7 +1891,7 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) list_for_each_entry_safe(mem, tmp_mem, &process_info->userptr_inval_list, validate_list.head) { - struct kfd_bo_va_list *bo_va_entry; + struct kfd_mem_attachment *attachment; bo = mem->bo; @@ -1921,13 +1914,13 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) * VM faults if the GPU tries to access the invalid * memory. */ - list_for_each_entry(bo_va_entry, &mem->bo_va_list, bo_list) { - if (!bo_va_entry->is_mapped) + list_for_each_entry(attachment, &mem->attachments, list) { + if (!attachment->is_mapped) continue; ret = update_gpuvm_pte((struct amdgpu_device *) - bo_va_entry->kgd_dev, - bo_va_entry, &sync); + attachment->adev, + attachment, &sync); if (ret) { pr_err("%s: update PTE failed\n", __func__); /* make sure this gets validated again */ @@ -2108,7 +2101,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) struct amdgpu_bo *bo = mem->bo; uint32_t domain = mem->domain; - struct kfd_bo_va_list *bo_va_entry; + struct kfd_mem_attachment *attachment; total_size += amdgpu_bo_size(bo); @@ -2128,11 +2121,9 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) pr_debug("Memory eviction: Sync BO fence failed. Try again\n"); goto validate_map_fail; } - list_for_each_entry(bo_va_entry, &mem->bo_va_list, - bo_list) { + list_for_each_entry(attachment, &mem->attachments, list) { ret = update_gpuvm_pte((struct amdgpu_device *) - bo_va_entry->kgd_dev, - bo_va_entry, + attachment->adev, attachment, &sync_obj); if (ret) { pr_debug("Memory eviction: update PTE failed. Try again\n"); @@ -2208,7 +2199,7 @@ int amdgpu_amdkfd_add_gws_to_process(void *info, void *gws, struct kgd_mem **mem return -ENOMEM; mutex_init(&(*mem)->lock); - INIT_LIST_HEAD(&(*mem)->bo_va_list); + INIT_LIST_HEAD(&(*mem)->attachments); (*mem)->bo = amdgpu_bo_ref(gws_bo); (*mem)->domain = AMDGPU_GEM_DOMAIN_GWS; (*mem)->process_info = process_info; From patchwork Thu Apr 22 01:30:51 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217277 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4342CC433B4 for ; Thu, 22 Apr 2021 01:31:33 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C9CDB613B0 for ; Thu, 22 Apr 2021 01:31:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C9CDB613B0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1DFC46EA15; Thu, 22 Apr 2021 01:31:23 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062.outbound.protection.outlook.com [40.107.93.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4BCDF6E9B2; Thu, 22 Apr 2021 01:31:22 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=N5BgQYpQL3td/7Y9g5qloOunnf9P+xrXA3CZL2cfLGuH8LhXpnrjOuc0/uRljcO5euUvHMMQ1FFo1c4OtMzCHlcNRGkXUBHlOLBXflIZpc9Lz/Eppvw5Y6XPwbNVVWg9YIsTDheIsEAGnt1V1gUF5rtWbH74R+brbH0yJMqMXGfeseZ11r8gAEPq9rLvMSMbV2twdmFEzbREVA8kyjtIcA/TffaryOPncyWG6O8ZN9Lnwqb+nWKvV0htSI67aahkNR46F/fe7OpboH+IHt9k9md0+UJ6KPC/Ca0QTULtLp05wG+gNYO7Y+EmbhOPKRHNTiCqH/elVgzj7rmvplGFVw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZYIDgGFiL2SILe5/JJc6WjFNa2ip7Eqnut8kecUOJNs=; b=XddOd5mpUe1sxRwrDAfvPNB3umwAXN4pjxZBsCLX3NeYVGti08Cy9PIOJLEcDhcJE3tVisDbmf6kCL8m3xUn75Hyd0T/1oKOTSug/a1kAms10PELLaTgZ2rT+RcKT7puK3XiQdixCapWRgGJC7vfCOlH77vyh9W8E84ADMma3JE0cX4L6MkGE2AQWB3IIlsPWfb+bgaObOycOK16Ll9wfv7Xeb53gpj4WunFQLM0JZ/BvWmeB73JTsP3mjP7ui9owuP0lbCVxmOcVPfFDsZB041B8L6QMxthkmrvbw1SvEx+kgJ1huJ9MXFwqkWQhmWN8QsK4hp3UFXu2BC9Ww2Zsw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZYIDgGFiL2SILe5/JJc6WjFNa2ip7Eqnut8kecUOJNs=; b=xsRje/gS/lYHRjsnQKZEKpdChWdbNHDBwnhRyjHAMlckjhIyEEqJO/nVhHoernR0WczJpRqaX/6WHvd4/0/LGxoaLylOl5a6sg4nZ76M0YJwCrrRQHZS/MUdeXo6bHPJAXeEA+W7TmB+17/kcts8IE1Bu9j8+uEMbHF1mUSu+/g= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:16 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:16 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 03/10] drm/amdgpu: Keep a bo-reference per-attachment Date: Wed, 21 Apr 2021 21:30:51 -0400 Message-Id: <20210422013058.6305-4-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:15 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 63eab476-0c5a-4d08-c394-08d9052e527a X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1xq3HZLxUyIe27ToZeajY465Cnom/eYdrOyhf2HFmDb5uSyBTYDJHIWpiX7JzJ6bu63kKoN7j2YGmmAU9tB2v/63T8M8GRpaL9ND+RnGThDz63c9jEU0xlcBf++VKaKc9sjA6l4X0Zig0fxBMwGeSBHHtlmAAOq2iO3+G+mApbq/WvqEVKqogGYIimZgmNs0F3X3REcTO9Pgk94nG+6I+k0nD41sCvTxjp1NIDTJolCtfxe1NlD15S+PwNNbtT1j/cof6R7lzyf5LDdRseHSBh/UFWubPeyw0J5YqPTMs45Dkl7XlduY6Ro8MLJJwVmlEi8CtxLcCi4gD6JD1cnAdPjqtT7k8NdI27FZfVkeLF8zVWg4zFa3Evxh27+fXGHcPnHDZXwtogaIJ17UvVX+XWyfUi7SfaiDEyJHa//7wsO07BJhnbMLA307aFRFv4OuWVQijwhGmVSwXTT1tocgf/PIYntf9/E4iaZWcMWkRBegky7vg9rX0TdgskytWL26lxEPVAarrOkywa5fuv1ZDt76Zf/TswSlzV9uZUvfu7GUBMzn7PT8HofYv7pfj2mvkEfVRzOoRO07FucCbQXX+gPhTOwvgM2ZmqrRMW+O2V9E9agDTErAObo3MA3vjcJGZ1Bz1IL4jRI1rnrsRUksykmvfrgYzWWovgDqaep/7Ok= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: llQLzvbISg47H8ME/U4cKTcb6FYon3FavuleKQn1CZtNCdfJ7aWZxfI3hynE2ybnKZMUY64Ce3T/N0B5OAJ+jdtD4mF258PemFNvbySV03H99VwKOtjkmZMGUXljRpah8F+AkzfCnTBSDoXTSlHOH89v+0DnWJCCj1ad+3xkJkU3Vyvg1K0YsXxInsdXxKiM2YQmEpgW9JOiaU3h/Fhh7ke1pU2ASKG7iOh6RrGOCd+xS0oVGs9hI8fDjLj5fevf8MsqvPgd0R94pfkXAd21+NiUxd/yqldPHoel9wWrc7lu+oWGyQOGzvOQRLyPC3yvHnFzO7L1/WbHswZBeo1baPVraVPXBj8QKxn5o2/Oi02qxS+xwiwAz/ej662bKKB7Vm0ivawLtYfEGBi1UHedOgm5cXT9wlTGBJ0PEtUl6bDOT6G6HjFL2uRbWpHs6Tz1DfnrXUnXF+WwjXz3Zfs/mDnWGHXgZOU+wXiXsjV4YkxoyQrXQNgRipL7Q8iu/X+/O0DHflzlNZK6vXa85dW6KCIaNlRzFVcThE0Fs4y55nm35AtjjjANpqNc/6pv+q4sEdiG0q12KUEHmyJxKFNK+oMd0QvB71pt7J/VpEzF+RcdPhrJn3+nHZltiuonKnfRKIUxPLcBroa/ohH3qUnfESrGzDA2GCh95Zl4GZTdjGo0VEm0m5osSs33r8adffU4tUKoY/Z0ZlJkx6cC/f2ItNUVxpp9wLHX8+cDZNk3riIcD9NJvNc5Si0Ph9aW548+fVnVIJ6xXk3hOE6cJhQDa5f/hTbat847Zejf+ANCkhkott3Sb+lj4zcxgXe0bNXgDBJX+3Zbf+9QydFk4j/xoJnRUNM+1G8anh8Gp8IzhXcy03in4r6/UU2tXL68KjqKhceKVVKQEw8wLSLOkXRDxdQk08udZ9HXwnlpzZcL6dqKh18gKRHP+UluU63zhpLC7kFfnYTwJTszO8Qp6Y+eHwgF2LmH+ZhkvU8zbwAEQdl8k68TksT2fZcyQYcogKe0UowyFWE9kQe7l6/BzzUDRU40YRltMQoGafqrl7H3ISj9pe3bjS1h/lTE9/2Y/c44ULBG+Vbj0x9l7haOh3hh5+eWL3RHkuzpHokav7N30VyY43MZ+OQL7LIuP1aKyPFa1vxVKI9aoe6Yvnkkx8xtow2wjKj8iCcxX9DEpkJ66reb74kCZ0euxIjgS4uyJoERjDXZpx9VFi0GmtWKPWYq5PIcS4e0mhtPUJ3lJj4q7NcOmOHpIF58OGAUm9CCaMixiqF3SBCHhk9C5+u5+vJuc1lwBP10k68xgkFSt7YJuCGAUxMo1W/VtlGeoUZCYSWy X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 63eab476-0c5a-4d08-c394-08d9052e527a X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:16.1740 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TKdR6Aby+OxP0HvBXpEULjXNcYfPUaLu60HaeFMRrI8BeKHtgm1xnsUs0K//ohg6oKfU1kX0cUZo1f3wDNjQRg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" For now they all reference the same BO. For correct DMA mappings they will refer to different BOs per-GPU. Signed-off-by: Felix Kuehling Acked-by: Ramesh Errabolu Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index fee4c64dd051..34c9a2d0028e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -489,11 +489,11 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_vm *vm, bool is_aql, struct kfd_mem_attachment **p_attachment) { - int ret; - struct kfd_mem_attachment *attachment; - struct amdgpu_bo *bo = mem->bo; + unsigned long bo_size = mem->bo->tbo.base.size; uint64_t va = mem->va; - unsigned long bo_size = bo->tbo.base.size; + struct kfd_mem_attachment *attachment; + struct amdgpu_bo *bo; + int ret; if (!va) { pr_err("Invalid VA when adding BO to VM\n"); @@ -510,6 +510,14 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, va + bo_size, vm); + /* FIXME: For now all attachments use the same BO. This is incorrect + * because one BO can only have one DMA mapping for one GPU. We need + * one BO per GPU, e.g. a DMABuf import with dynamic attachment. This + * will be addressed one BO-type at a time in subsequent patches. + */ + bo = mem->bo; + drm_gem_object_get(&bo->tbo.base); + /* Add BO to VM internal data structures*/ attachment->bo_va = amdgpu_vm_bo_add(adev, vm, bo); if (!attachment->bo_va) { @@ -529,7 +537,7 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, /* Allocate validate page tables if needed */ ret = vm_validate_pt_pd_bos(vm); - if (ret) { + if (unlikely(ret)) { pr_err("validate_pt_pd_bos() failed\n"); goto err_alloc_pts; } @@ -540,15 +548,19 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, amdgpu_vm_bo_rmv(adev, attachment->bo_va); list_del(&attachment->list); err_vmadd: + drm_gem_object_put(&bo->tbo.base); kfree(attachment); return ret; } static void kfd_mem_detach(struct kfd_mem_attachment *attachment) { + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + pr_debug("\t remove VA 0x%llx in entry %p\n", attachment->va, attachment); amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); + drm_gem_object_put(&bo->tbo.base); list_del(&attachment->list); kfree(attachment); } From patchwork Thu Apr 22 01:30:52 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217289 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 4AFCFC4360C for ; Thu, 22 Apr 2021 01:31:46 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EA9AD613B0 for ; Thu, 22 Apr 2021 01:31:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EA9AD613B0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 921DF6EA21; Thu, 22 Apr 2021 01:31:25 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062.outbound.protection.outlook.com [40.107.93.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id C5DF06E135; Thu, 22 Apr 2021 01:31:22 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JWA//eHX+eZukdUUR04e4BRZwreZU2jrX/CAYl1zpp/FkxCzkACg5LqCq7xC7Ow4Yojh1uDkf8IpBCl8iuziJWUlzwiKT1i89SqzjMvKQk3aPIOZz+Nw0urgYR/2Uysn5pKJaHeiqGqdyf4jm0+KHF2H/L4AZdasi93r+lASKqTd11cS2UaGquLoePkUruZ+1ffRAg3+Q9XxGjBb0SKjreQiJkB11BazP0YmvYMNtJBV0GY7ZvfZrQ5glW8L2PdQ9L5UauEydmmtF8auBqvPWweVTFZpys92xDtmkKD6jRfNX8tqK66SoCS3cw7LFym77W3bLfUruWlFVsdSp5w43A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W+aNnSE7j41FV5zInva4P9RYXfntrXBFV/m7CzWUFBw=; b=iQATpMCuWev6vpT9NibAqMHAhY98YpTtfN563NATJf6ymEQ7juircTQkEySE4JShTlIx+5yUiFp0xKVuHThsknQsg/2GMcWy7CuCLU17dNEplyKzcBBPgLUptQDykbvZbKbCJCwCQUS7s7af6KKAu9xNQKxP0Z8VUuUu7aacNKaE6K2JJfbxlyaGB2lWhnMlQCTrN/fdsivfRb0Ds6eXmK+baZ/zj4jlfLVSxtCHk/N3dPO1L3U5kd9xUkqlGRiYYlOqqfdy3SWJy6UkUVYxX0UMZbplREWrrr5XFrET1oBQX8BnuMVYzgMR4KAWx/+WnJH3A22XjtQSMLcnBNnzHw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=W+aNnSE7j41FV5zInva4P9RYXfntrXBFV/m7CzWUFBw=; b=T4a5mGYe6ZXF7U2lt4Hupwceui2aP5nMK37jizrl3UyERahcuZCm2Osqz00afmryZ9c5H5gBUCTs6ZD+euR4Kx2de12y+LvIJv5wrIJbacSl3+NdgZ3/FZrSeCF1EWm3A/xHUN5gG6iN8+HHGlza913AcEFOeIZZs/MnIFDuK6E= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:16 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:16 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 04/10] drm/amdgpu: Simplify AQL queue mapping Date: Wed, 21 Apr 2021 21:30:52 -0400 Message-Id: <20210422013058.6305-5-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:16 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a9a9e45f-eebd-4c6d-9bde-08d9052e52bd X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: N/hG2PlrbaaxRB2ivcaRg7qh5a5vN0iQPRds9v8mOBwk9Uh+JquxkmbOI7v7M8oXq8QQ+gQIIbbnZ0DwxMT1zPYJTlPMfNxH1J47O+V0Skrxz2lIQN2oWuHvZRD65GCsN/CV4PxKSSoN5lQIIuNFXQslcxpomfIquXn4PNiDUIHfxqjnB1exUWffO9aM6lGAteNeYuZ0tPfDblq7VM6EeQQswr6S8LVwNs/uukbQYVxmm2DF82lRGS0nhnj5U4+6kJAMnXwLjVQ30pPwvRDrI9NJHkhR3zJc88gGoyXssyouD4SQ2UcopdaADKxkCR8RPJZHTyy/E9/Eov0s3xJuYEkFcoSSRcp1mWCJBk/ljZlmchtGjgE0uHpnOkRiGZjMIUIy1tBERcq+JEqmGkPcjPKtqS6pt5oqjuVw9ieWyAAcxdYYaBAgcGorILKUTiTQ+0NwnluRUzgJvklrdv3YIbcVoL3/lmGfDSe08jVzGO1oVqHQE64c4KVrxeNrmE7nSbtBgG8z1Ru1inwWce/5GDlFNsV6xtwFgT+UEsHcYWZcB/A/4fw7mHHgd8Fe7Vgky4H0c15BzLQCwENuiGxvi7Dqeo0MY/NElGdydorb4lT46oXzyLPo53s5khom9yzAnLBoZzB6uM5xaixZXkR/enw7cpJLvAjigv3Pko/xOO4= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: bY8Kc9rodm0m+J1sHZVqmvUybZnmw1ZSU8thXT1sO8MfuA0p2CWmgR1CrDZgsvjqWwGX0P6dU9OwBDJFeXoKwmAMR9wJutEgkSgmLF6uTCpqs85t3oEdccsFsfsRl7JpfVIiODRy84OWmU3gOUY0s5/V5RNPLSmCTrrQoKdxHtX5Wj3Zaws7mjguei0N6DUihwzefYQd4Uxn4LnBo4317XRm1ZhahHzgqjIUlE4Rgj1l6LF1eG3bgC33s4HD/lUaipHK90uu/EUhqUYkTgRUwt/9NOwdAMBPnhg7L9s3wyXacqidVQIbol0EZWyNJADs/snD3b5ZxbRt8KJP5XvAGilTYjzqAKKGBYAMJbivo2FPtoEUBf04La3GLmtv8QaGE5dCyUED3XEozyevClvSEYIxDLrB7x/ch2IYbdq+VFOdYCGAHdnTzTFCSTteOe7Q9SiFgNC/zW3vw9MLphipFK3rQQKTRZpgGZocbxUPwCxekM7kLU5HjGmyTZ5vURgpGyQwl3NCSudTtotPhJnF6YE7fI09fMnCx0EPAOn9Xvb+wEtTyoySY24r5maxWMdo05SouVcVJYtRz+XHDWaelDfeCosuwiBKWRh8pHgJM39SzOBUJBUt1b4LyrBBrmQO1JakiodVm5pMsxdafR5FwT3Zk+xxV3siiYUKDGnhdyos0dkQkDufrL8QlaEl5lHKw4xBGrRzdocd+8jkLyDUepNvT8ifAAVUXvVtoAxdMqtVMHaOICShWVF0ExlZVEpzLxztJQONqfgjCCGtRs8dcUgM4nDkug3Gk7Si/carrZHbUZt2PzYgielerWq4VW3wvzc7UxpBpDlWXHcSFElkjWW/IvF0iuQCyBhLud6GbnaoZzivokQPGCMuYG2o0I07zcQwxnH5jPK2Co7GaZPSgOUVtyY4dOOBKGtZRB2ZwyL9Hmf7ZDDUYZOHt2gLoVbBSCu+rM5lh5eIMrGdx/E8Od4PFw+HVaIKWAHCp0VgS7OxlUvNZw0n0zYHi3jT5gP3YNy//cU0XEEcV6WuIFhL73OxSxXfVVE9Zz6qsGdGS/uh0stkRVQ/k4P+fpElSzTsoMhtq/gjcfSI5PPleyeCE04vRQA/JVHSUA7mPiMcbm6TuDuOufTmz+UMYv5mDKE941UiBRdnnhKGGlrBv7YWtXR6znv4+5FQdUc1QsWYWHNDi9iNdFouEHMtF1CSjvzIAXfM7QSrH5tLdIaEyZ2Ms0vFDPB2uHwj6D8vCBG6Au2/PN5u6HAs1g7WWpCaSdANULQvtl6r+g8sexm7UxQYeu24h4wMA7Ubx0CjNM4ETyRbT/cPpXGjgz2bsy/GNxXL X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: a9a9e45f-eebd-4c6d-9bde-08d9052e52bd X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:16.5881 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Y/cEdNRS59xVwpVqnJd3DUKeVyz7AJGLNmk+JKn1mWCGguFbKd1ffO4bis41zz9tQwb/3WAZEfzyIF9Q6WWRtw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Do AQL queue double-mapping with a single attach call. That will make it easier to create per-GPU BOs later, to be shared between the two BO VA mappings on the same GPU. Freeing the attachments is not necessary if map_to_gpu fails. These will be cleaned up when the kdg_mem object is destroyed in amdgpu_amdkfd_gpuvm_free_memory_of_gpu. Signed-off-by: Felix Kuehling Acked-by: Ramesh Errabolu --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 103 ++++++++---------- 1 file changed, 48 insertions(+), 55 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 34c9a2d0028e..fbd7e786b54e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -486,70 +486,76 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) * 4a. Validate new page tables and directories */ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, - struct amdgpu_vm *vm, bool is_aql, - struct kfd_mem_attachment **p_attachment) + struct amdgpu_vm *vm, bool is_aql) { unsigned long bo_size = mem->bo->tbo.base.size; uint64_t va = mem->va; - struct kfd_mem_attachment *attachment; - struct amdgpu_bo *bo; - int ret; + struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; + struct amdgpu_bo *bo[2] = {NULL, NULL}; + int i, ret; if (!va) { pr_err("Invalid VA when adding BO to VM\n"); return -EINVAL; } - if (is_aql) - va += bo_size; - - attachment = kzalloc(sizeof(*attachment), GFP_KERNEL); - if (!attachment) - return -ENOMEM; + for (i = 0; i <= is_aql; i++) { + attachment[i] = kzalloc(sizeof(*attachment[i]), GFP_KERNEL); + if (unlikely(!attachment[i])) { + ret = -ENOMEM; + goto unwind; + } - pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, - va + bo_size, vm); + pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, + va + bo_size, vm); - /* FIXME: For now all attachments use the same BO. This is incorrect - * because one BO can only have one DMA mapping for one GPU. We need - * one BO per GPU, e.g. a DMABuf import with dynamic attachment. This - * will be addressed one BO-type at a time in subsequent patches. - */ - bo = mem->bo; - drm_gem_object_get(&bo->tbo.base); + /* FIXME: For now all attachments use the same BO. This is + * incorrect because one BO can only have one DMA mapping + * for one GPU. We need one BO per GPU, e.g. a DMABuf + * import with dynamic attachment. This will be addressed + * one BO-type at a time in subsequent patches. + */ + bo[i] = mem->bo; + drm_gem_object_get(&bo[i]->tbo.base); - /* Add BO to VM internal data structures*/ - attachment->bo_va = amdgpu_vm_bo_add(adev, vm, bo); - if (!attachment->bo_va) { - ret = -EINVAL; - pr_err("Failed to add BO object to VM. ret == %d\n", - ret); - goto err_vmadd; - } + /* Add BO to VM internal data structures */ + attachment[i]->bo_va = amdgpu_vm_bo_add(adev, vm, bo[i]); + if (unlikely(!attachment[i]->bo_va)) { + ret = -ENOMEM; + pr_err("Failed to add BO object to VM. ret == %d\n", + ret); + goto unwind; + } - attachment->va = va; - attachment->pte_flags = get_pte_flags(adev, mem); - attachment->adev = adev; - list_add(&attachment->list, &mem->attachments); + attachment[i]->va = va; + attachment[i]->pte_flags = get_pte_flags(adev, mem); + attachment[i]->adev = adev; + list_add(&attachment[i]->list, &mem->attachments); - if (p_attachment) - *p_attachment = attachment; + va += bo_size; + } /* Allocate validate page tables if needed */ ret = vm_validate_pt_pd_bos(vm); if (unlikely(ret)) { pr_err("validate_pt_pd_bos() failed\n"); - goto err_alloc_pts; + goto unwind; } return 0; -err_alloc_pts: - amdgpu_vm_bo_rmv(adev, attachment->bo_va); - list_del(&attachment->list); -err_vmadd: - drm_gem_object_put(&bo->tbo.base); - kfree(attachment); +unwind: + for (; i >= 0; i--) { + if (!attachment[i]) + continue; + if (attachment[i]->bo_va) { + amdgpu_vm_bo_rmv(adev, attachment[i]->bo_va); + list_del(&attachment[i]->list); + } + if (bo[i]) + drm_gem_object_put(&bo[i]->tbo.base); + kfree(attachment[i]); + } return ret; } @@ -1382,8 +1388,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( uint32_t domain; struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; - struct kfd_mem_attachment *attachment = NULL; - struct kfd_mem_attachment *attachment_aql = NULL; unsigned long bo_size; bool is_invalid_userptr = false; @@ -1433,15 +1437,9 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( is_invalid_userptr = true; if (!kfd_mem_is_attached(avm, mem)) { - ret = kfd_mem_attach(adev, mem, avm, false, &attachment); + ret = kfd_mem_attach(adev, mem, avm, mem->aql_queue); if (ret) goto attach_failed; - if (mem->aql_queue) { - ret = kfd_mem_attach(adev, mem, avm, true, - &attachment_aql); - if (ret) - goto attach_failed_aql; - } } else { ret = vm_validate_pt_pd_bos(avm); if (unlikely(ret)) @@ -1496,11 +1494,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( goto out; map_bo_to_gpuvm_failed: - if (attachment_aql) - kfd_mem_detach(attachment_aql); -attach_failed_aql: - if (attachment) - kfd_mem_detach(attachment); attach_failed: unreserve_bo_and_vms(&ctx, false, false); out: From patchwork Thu Apr 22 01:30:53 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217279 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E9DD5C433B4 for ; Thu, 22 Apr 2021 01:31:35 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9E8F8613E0 for ; Thu, 22 Apr 2021 01:31:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E8F8613E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3D2B46E135; Thu, 22 Apr 2021 01:31:23 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062.outbound.protection.outlook.com [40.107.93.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id A697A6E0DE; Thu, 22 Apr 2021 01:31:21 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XW0/TA6sQzOQqDVMsFgv8NsdwRdUd19b+cVfj4gp5F8ikVZBEnqUWqjw3zb4JKMXtKF2iqWtEn/pJaPj2UWdZgy2nDHruQaSiFyJTjZKroi0cUkPeqTYCJKoidSTwIIvlhnd6lKMkFZyFT907jYiayWs8oU8ir2pOGcPmpygIsNFsK7/ntfjw2bzFvYdqs/NCbyXAsFz97bpjPvihWZ6rthYFAuWieC4qaQL3lJGSYaVcPgibCp6EUtiK+O8p5XN27Ta6x6bE7Lc10gCsNxpfznVsPefmi3Bm2An/OEyKhUSQ0UvmIQjBGpr95k6a6QZSWO2prFF4+85YE3DseG7sQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=igF2IJxGaibEBWeZ2QJ4P6MWb6FQIFHlo8eM57cbMWo=; b=NX+qIIo7lhMgKSrEvoWP/PFybrUTmXuV1CSOWWy2epCCXtg4RPCEEqBxwqdDjPPD453qwj9bt/hoQsaPxCeD03N0tIHxgYYNDjCqgXoECJ6mmsYub6jfMYTTpkmcX1ZqI4M9McFnzHoU/k059lfC4TVQIaTUEyMI+3yax00QZ/m0Aalbj+XrlmVrB1VnjOrdkIGuQ4uL124CHEe5VdTaYnIAd8FF+Tv8AoDq+okhDgtU65zCbh9UyBy/WnweFj2TH6bggdnyj/nTfXRE3lWDQ74OZYWTOdRtLPl7yDxogkhGoQrVZJ11gL26az7yGQuFNuhNje/m9znriYjwgNQeGw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=igF2IJxGaibEBWeZ2QJ4P6MWb6FQIFHlo8eM57cbMWo=; b=JTO1nAdPz/fU9CkOupN5ECb5dhlQn80nv5xvcaSpHD4qFhQjlpAk1uIxtfkV+zacyjT4lvNAO7WRQVJMWSg4VYT2VChs5HBZfa6CnP3aUPMp8VoTu0oZVIATLL3Co7nR81IopIBIZXVP6tOSciWDTHG9wpBd5EfaxYKGqV/cADQ= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:17 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:17 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 05/10] drm/amdgpu: Add multi-GPU DMA mapping helpers Date: Wed, 21 Apr 2021 21:30:53 -0400 Message-Id: <20210422013058.6305-6-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:16 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6fdfc313-1f67-46f9-ae63-08d9052e52fa X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6430; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oCWw7d5PhmrAM36+NGA4LTIVKk4Vl+Nm0sCuLjGX/H8hFx1qi6JMQ+83fUjY0k+Tni0Q4+DW2txBlv0JFzr3tYejd0ZQmiKYy+lZ09wU4RHGOaDt53+lcbBKRH9YtsnOhyEQwqlQf++mWfXNCn44QI3xzH0qL9tiavh7iMFlr+9jsBiBA3RL7o1cdTUKj83PV5Ur4JJG29+AWl/WM1n6nHVa+qY9CgyzQinhVL5LgdMx41H4FhyqlhUULCHX90WxMUdhlqDBjSDodrE+w3+tHbhnWiVB0dGaKT9YkIi7C9kriiQl0Z0kXXARPVLw0dFVTOYQmFSC9XBLGIqwVkrIhxNlU32/VEemf5B9Vtf/Eqf0YiWZn4TuoIeA6FRTBx+GzdoJMWFGIYNU1Wc5rmF0rydRPoR1SsvmmEwdddGaEpuJLxv6++l5gLVu2scZd2f0ce+Q5gUJXYg3nnSC5Fxx7PIlxt7ydG1OSTJ3TBXTUwbMT5CJ0sUvaO3rYH2cJsYt6s3clEqs8iI7babmYEtqtZ9ohEdGoTictp0SzDtMULUcTjP9oELzdtr1IoiE1ni0L0RWjpeduwE7dwcdJY72D4nqal3Q+kJKEhAjemenpmr/ceKsUtrUj5hRj5KeFndw+CqGogTBIqQVeizAvveIgXkNKT7luoPSU2tlHrCx6ck= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 6kqoVvra5ad2Vdu21wZVvO/EoXk2BwouwafaTMoDyIAAGRWl7yYJ08Cb2mXb4Dio6rBC7/nm/Z1UI1avREotudLUUChRDMb9La+OT1br5YRBEQEV61tsZMmLx/RC8POOJnaYH2h0FBIpxHvFYsFdmHqF+8m96rRarvK1gqnFIuYgjq82qD2PPHQo//o9k1miGeScjh/amLtnXk16z9M97mTDDE/DbI1+/YcAU4tP8G/efcvwj1f1pLrGPFt6TiectRmsbkKO24nz2WIv0LtOQUdWZwZ40jGlSf0HrkoHoXSDnHS9prKqCfoFUPaOhY8MkUTVy+im/NTGfLsGWM9vQRnD/00GafQ07DoZbMGjuYDzRJXLI21NipLBZdDOUK96wbkRfNDcFPg5vtd1f6SfkbNijQKDRf1N1SCR8FEsfE635nB+eMpSjJlTCuFWBHE4lbDfl/4GkhUEfJqe5My3etkUCt1xwIbiFuxa7bVLznNpzJe86KMyahJPWMpw+qjgtVDAfxzo1XCNUfj1MTOUQQSLhfS/K4Par0EOXdicxbnr5kJfPcwTbTX2EraqJ18bSf2O+jCO7IYGQ8WqLQHAXJ/WETv9ww/ZlOECj6JB88qkT3VAi7EKcg9kau/J+6Z8T1U4OOUIO3IwayYdOJcsPTCBxfX8cafGJfov9z4j7gup5mNVilEjCEfjebjJOCPCcB5t0lZ2+uvu1lHqdZJkuT9mg5chuW4Da/k1pyBquuYBVKtof3AdGn+L1FKtxDGPiwx6f6ZnAK3UNwbpkO+88cShPctgRPDnPCmDV1w7/ABT/9JWP+dLiYK6IWYicIggITI/PzmCLFIkJ6mqTrUtHcuSH8nRDuC9OrzxwPW4ODkZICC65b1fqJsXElfEqqfZB6HbGDqJ2sPsxXac/3d6HD8xnLlSZpQPcN07H9tzgJs1A4IiDMG1GHyNYszgYvZadld1j8l8tt9Vy/cxB7kOVghesgeg1QIownGkgU/8sLd6Vj5he9AQB7/t5/24KbVSO15KmwTliuQnvM6dyJcL1ljMyORLeElRat2Q/fNomHTyvHduOqP1ZuP4iE2m9MlG0wwtA/73OqJ3KRTi01pFCanohk9LROI7xkpxd59mcn6HcGqEaVTnr5yFZ8hJ9E8XSyhnvSyxgVhAGGqB99Y3B287oMIBU5QhdryxE4bOsjJWotaOB5OHl1RRJdORx2a8Mo+2hC315cpBGS9LNB0D/0WSQYsYpUd6XFFj+JdcEsQu4VRFH5N0K/KnUaoPEkSfg8tP8mmhnsp6E+hUW4KrcldgvK6Uj8Xv5lL/WNzrIM7HB8xGCd4rEhoPReLdpWfM X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6fdfc313-1f67-46f9-ae63-08d9052e52fa X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:17.0501 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 81sxl+h26dL+hJNAeUiacw1SxgTeSg2cyojCApxuifz7JhepmtvCsIlNwsz3pgDNB9W68F3bpWspv0cKiPdyBw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add BO-type specific helpers functions to DMA-map and unmap kfd_mem_attachments. Implement this functionality for userptrs by creating one SG BO per GPU and filling it with a DMA mapping of the pages from the original mem->bo. Signed-off-by: Felix Kuehling Acked-by: Ramesh Errabolu --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 8 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 146 +++++++++++++++++- 2 files changed, 145 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index c24b2478f445..63668433f5a6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -38,11 +38,17 @@ extern uint64_t amdgpu_amdkfd_total_mem_size; struct amdgpu_device; +enum kfd_mem_attachment_type { + KFD_MEM_ATT_SHARED, /* Share kgd_mem->bo or another attachment's */ + KFD_MEM_ATT_USERPTR, /* SG bo to DMA map pages from a userptr bo */ +}; + struct kfd_mem_attachment { struct list_head list; + enum kfd_mem_attachment_type type; + bool is_mapped; struct amdgpu_bo_va *bo_va; struct amdgpu_device *adev; - bool is_mapped; uint64_t va; uint64_t pte_flags; }; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index fbd7e786b54e..49d1af4aa5f1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -473,12 +473,117 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) return pte_flags; } +static int +kfd_mem_dmamap_userptr(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + enum dma_data_direction direction = + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DMA_BIDIRECTIONAL : DMA_TO_DEVICE; + struct ttm_operation_ctx ctx = {.interruptible = true}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + struct amdgpu_device *adev = attachment->adev; + struct ttm_tt *src_ttm = mem->bo->tbo.ttm; + struct ttm_tt *ttm = bo->tbo.ttm; + int ret; + + ttm->sg = kmalloc(sizeof(*ttm->sg), GFP_KERNEL); + if (unlikely(!ttm->sg)) + return -ENOMEM; + + if (WARN_ON(ttm->num_pages != src_ttm->num_pages)) + return -EINVAL; + + /* Same sequence as in amdgpu_ttm_tt_pin_userptr */ + ret = sg_alloc_table_from_pages(ttm->sg, src_ttm->pages, + ttm->num_pages, 0, + (u64)ttm->num_pages << PAGE_SHIFT, + GFP_KERNEL); + if (unlikely(ret)) + goto release_sg; + + ret = dma_map_sgtable(adev->dev, ttm->sg, direction, 0); + if (unlikely(ret)) + goto release_sg; + + drm_prime_sg_to_dma_addr_array(ttm->sg, ttm->dma_address, + ttm->num_pages); + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT); + ret = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + if (ret) + goto release_sg; + + return 0; + +release_sg: + pr_err("DMA map userptr failed: %d\n", ret); + sg_free_table(ttm->sg); + kfree(ttm->sg); + ttm->sg = NULL; + return ret; +} + +static int +kfd_mem_dmamap_attachment(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + switch (attachment->type) { + case KFD_MEM_ATT_SHARED: + return 0; + case KFD_MEM_ATT_USERPTR: + return kfd_mem_dmamap_userptr(mem, attachment); + default: + WARN_ON_ONCE(1); + } + return -EINVAL; +} + +static void +kfd_mem_dmaunmap_userptr(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + enum dma_data_direction direction = + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DMA_BIDIRECTIONAL : DMA_TO_DEVICE; + struct ttm_operation_ctx ctx = {.interruptible = false}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + struct amdgpu_device *adev = attachment->adev; + struct ttm_tt *ttm = bo->tbo.ttm; + + if (unlikely(!ttm->sg)) + return; + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU); + ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + + dma_unmap_sgtable(adev->dev, ttm->sg, direction, 0); + sg_free_table(ttm->sg); + ttm->sg = NULL; +} + +static void +kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + switch (attachment->type) { + case KFD_MEM_ATT_SHARED: + break; + case KFD_MEM_ATT_USERPTR: + kfd_mem_dmaunmap_userptr(mem, attachment); + break; + default: + WARN_ON_ONCE(1); + } +} + /* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added * to a VM. It can later be mapped and unmapped many times without * repeating these steps. * + * 0. Create BO for DMA mapping, if needed * 1. Allocate and initialize BO VA entry data structure * 2. Add BO to the VM * 3. Determine ASIC-specific PTE flags @@ -488,10 +593,12 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_vm *vm, bool is_aql) { + struct amdgpu_device *bo_adev = amdgpu_ttm_adev(mem->bo->tbo.bdev); unsigned long bo_size = mem->bo->tbo.base.size; uint64_t va = mem->va; struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; struct amdgpu_bo *bo[2] = {NULL, NULL}; + struct drm_gem_object *gobj; int i, ret; if (!va) { @@ -509,14 +616,37 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, va + bo_size, vm); - /* FIXME: For now all attachments use the same BO. This is - * incorrect because one BO can only have one DMA mapping - * for one GPU. We need one BO per GPU, e.g. a DMABuf - * import with dynamic attachment. This will be addressed - * one BO-type at a time in subsequent patches. - */ - bo[i] = mem->bo; - drm_gem_object_get(&bo[i]->tbo.base); + if (adev == bo_adev || (mem->domain == AMDGPU_GEM_DOMAIN_VRAM && + amdgpu_xgmi_same_hive(adev, bo_adev))) { + /* Mappings on the local GPU and VRAM mappings in the + * local hive share the original BO + */ + attachment[i]->type = KFD_MEM_ATT_SHARED; + bo[i] = mem->bo; + drm_gem_object_get(&bo[i]->tbo.base); + } else if (i > 0) { + /* Multiple mappings on the same GPU share the BO */ + attachment[i]->type = KFD_MEM_ATT_SHARED; + bo[i] = bo[0]; + drm_gem_object_get(&bo[i]->tbo.base); + } else if (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm)) { + /* Create an SG BO to DMA-map userptrs on other GPUs */ + attachment[i]->type = KFD_MEM_ATT_USERPTR; + ret = amdgpu_gem_object_create(adev, bo_size, 1, + AMDGPU_GEM_DOMAIN_CPU, + 0, ttm_bo_type_sg, + mem->bo->tbo.base.resv, + &gobj); + if (ret) + goto unwind; + bo[i] = gem_to_amdgpu_bo(gobj); + bo[i]->parent = amdgpu_bo_ref(mem->bo); + } else { + /* FIXME: Need to DMA-map other BO types */ + attachment[i]->type = KFD_MEM_ATT_SHARED; + bo[i] = mem->bo; + drm_gem_object_get(&bo[i]->tbo.base); + } /* Add BO to VM internal data structures */ attachment[i]->bo_va = amdgpu_vm_bo_add(adev, vm, bo[i]); From patchwork Thu Apr 22 01:30:54 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217283 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A3BABC43461 for ; Thu, 22 Apr 2021 01:31:40 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 52FFC613E0 for ; Thu, 22 Apr 2021 01:31:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 52FFC613E0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 400BF6EA17; Thu, 22 Apr 2021 01:31:24 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062.outbound.protection.outlook.com [40.107.93.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7186E6E135; Thu, 22 Apr 2021 01:31:22 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ir7+JoeDXGUpIbf9OtmYqipBbkROSasKJGSjEkqHLspq8uzU9nzlS+ReQ9cepJhYKDk7xmum7EXKm1kl6d/9rDVTog/xler2lZhm4EuZkeYpbnm/kwZC+mL5ArQYq4oVNrzIKDCj9zrMCVp1T75dpJ3jSiOTINQy61xdX8ed38FuwJX48RKLuhRsXcYzCtWd27nj1Sxq1hEEwd9oBp66XHdSksVToIxLODN6Z0SFaRuwwU3NJzoP7XU63HX7ubacLSF/46dtIpGWs/h7HatjSA0yz90YYX3Gb30t3KNP8YljsE0DgivjKkQ/FUydCjWBaf2oPhW1a3t1WTHPEwjslA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WZgQCeJ+/+U0LFHKWLBcpQ9cqefTDiNUJe36r8rTMCg=; b=OzifDq+itdpPBdSz6dhGsy4luN6f18aZ2VMNY9NI0t5eghMXBWid2OkdFWohBOOhYHb2PtUJDMKXQjpsJyzUO3Yj9UGYRtKE1lUznVBa182BzCSWMAXsUqa4Uv1tjG+MeXVe6/szyHZ//bJjMMkVqGJTtTiIBQrlTjPBnweyDSQ6NJkso5X0KcquqKCDELWwbaHnjUxyO0ldg9lpIM/v6t1t1Q1K6raa/EEAnyQiK0ad2nrq/02fF0XDJdDNRtzmI6BiQu6zLwkstl05rUHu6q/cpFkgIztmy2J0MQAGKY9yWus9Zj4M68GqsE2tNlyg1yQXMJLZsZEagqvZe07hCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WZgQCeJ+/+U0LFHKWLBcpQ9cqefTDiNUJe36r8rTMCg=; b=MxpXEhvmA5KUODLmExUWRKZqc/gsmub4fIlbnYX0ffdRZKZpni9OAMgz6erq8LThR2+SnRj+PgfiGKQ8iJyfsvbCYWUyS0+sNpbd+s3oumgSxbmWVviXz4ljULRpDrEMZd5FxGV4Cnj8WPzbyH6OTAQTJtsmLUhDt7jkTtym150= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:17 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:17 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 06/10] drm/amdgpu: DMA map/unmap when updating GPU mappings Date: Wed, 21 Apr 2021 21:30:54 -0400 Message-Id: <20210422013058.6305-7-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:17 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4efa65e6-c5d3-4819-bade-08d9052e5345 X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: KakXUSc2idtO5i1g24Ue0Sm0AUqCLhZujKpNkVUvMVVJz0iXfTkWJII9ATSBVB3p6NgWyz/pUmFU8fE3RAYMso4Y/3sGfmeiakSjORdBAxtzNvRIgRp08iENLnJ0evZ9GEnnAucAXo9NQxYqZxHvXRSPIVuY0A7prMJX6j7N3wCHhfI+Io5JxbXjYdWu3fX9rBgk3vJ80RO1QUU6ZgFB6SSm3TuvoBYtEbg9eduTHqDYDxF8zT8qbkcHZhKdpKQfo0j7R34VndAP38QsAS24eiS4/X5hWC6rkX4nWnBk2KWnANE2Vc/UClDOgvajNjUfbChgFghoZ6rWQ4PO07peJY31fo4hySjoeolz+jJXADAnDG/IvQJJUjkbi3x5tHXyShZFlFGS4OguFxqmiVdHpRy2a3U0wCKRBujBC1/p9+l/i7aDhnJUS8+GT7nSrFNLdjB24ZsLjg+LjLtksEpekWuKlelmqhp/Sy7SxPKD90owa6trfK/PmVwp4HG5MckGhv5v2iFttjQbFKnT4hnhO7JPiDwPIZO/tIM/0aruI2IS2WThrQ3ylFPaqCRM81k13UEfB2bjnBKn2TCSveTA1EMDM8Y4l5ywAoqOKBqxmY4f+5naWibTiDHgnptZZRc3H7LEXi6t4UkI9PXGi29kulzC6OKe6Ljl60qtVLCG8d0= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: fsk+YkPSxODgSN5GwAC2UdKRP3T7m/qaR838d4NcxyPJxk/BHCtphfohjRoUmYAMnxX/ctdKDt66ead8ZmHETjbI1e+tv6noP6wCCttyn0bT49BHlayqObpOK3nGxvk+Yh9GG7U+/lCvrsDjlMco2AzwzSOWvJiWMAv1iqrl5+9wdwHKX61MMXR06VRRBmrn9FpA/+Y/oYp9LVs/A/X3qAlxgbZ1jcjM4G6mz3j4mP6KeLmVHvY/Dz2G7hrYEil2vLjuyxxw5ndGlRKTTGy9PmqPlR4TAD9LSEl9lnfTUr4fU34LbQdIsVtf4VcEgZWtTDj1FNVn13eHtZa/OSY/Ee0mTAuyQAnX6hiUBJufk8+WIXVf8NFinhKQCtV64WWk5H9ovgJO9yJm77aP7HzY/k/2MLxPUnQ6onmb8s3CIq/jwa8vkr1EnS4D0etHiCv3d81OtkakcdPgEH+FB8W5hPxlR9LruNlRw5BhweiUYXbY4WAb2NDIOGYbgGJ64XM9ngDZP+zTdENFPw5feX8QvDvzNjM0HHD5uK5aNrzn15Pu2MmlPCeeprIEFG8jXuGZEFJhluXglYVG2PbnGAyOWAzZqIIKGuKEacQpOZiLOX+Yd5PBlMYKWDf5lRO+6tBGihDNlCHWovHrfVCHeeiu56ZruSJD6esB6B1NdWUDJQr5OFciTJnaWCjwif3t5BEeiOq0s7O5cFPPLbjVh/MU8OtBKS2hW61XNBCTB+5PS+vuncDg9e2s7KM8OPr34uGxWrxZtjZnb+QFjMQt5X+4Ni+ziU4ftH9JFCu9qhy40xKZFih59oiHaAfHbFLcuV5PpxU+Is5WhVblPN8uyAcIIjWesWrlIVpW0/4vL8g2aYvk5kUIaLmy6JYp/U3SgYREFg5WAcffXKYnLWT9LcFI0mhA+2oFHlg50PUyPp5rCKazYbJarQ3qaRZwFCbzpucWY6kB8g0svidgjL0KjGrobIgVCVvfym4eNLEX7CW2YvbLogCbKFqP5yCMvAlHjARbj5CJxVVMXGq+32n0IBByAvSyulS9I4n5v+Ff4o2TGiQoVc3hRwi4OTODaGEefiOzNtwCOLmYPDb5vPBz5le9fgv4sevDRwRU0QZosmfq01QkiGPgtEOD2ySyDbJZRdVMycMpCtJIpi4i9jKiWh3yBTUMgQqRDDnQQLiFxWFd6XmiHH4TAaubnthenDSak18eeYO8f4RsQLcj+a1muHb4wetJZTOA7IySn73snEfb+xvS9g2XXocSQATw26WXG6qUDySvQk3d5IO9OuHauuZvMaiGzmctyuOJc6QxGm2o5TztVmbyEP+iu9J92knRjx+M X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4efa65e6-c5d3-4819-bade-08d9052e5345 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:17.4991 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: PslE1jCIpa9S1g51S8OOk4VJ7UNY5T/mUWZadWsByXBrVW+DZoUhUu3a8SmJdDKBuEMmsdcEn+RilHOPNknyFw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" DMA map kfd_mem_attachments in update_gpuvm_pte. This function is called with the BO and page tables reserved, so we can safely update the DMA mapping. DMA unmap when a BO is unmapped from a GPU and before updating mappings in restore workers. Signed-off-by: Felix Kuehling Acked-by: Ramesh Errabolu --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 56 ++++++++++--------- 1 file changed, 29 insertions(+), 27 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 49d1af4aa5f1..7d25d886b98c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -961,11 +961,12 @@ static int unreserve_bo_and_vms(struct bo_vm_reservation_context *ctx, return ret; } -static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, +static void unmap_bo_from_gpuvm(struct kgd_mem *mem, struct kfd_mem_attachment *entry, struct amdgpu_sync *sync) { struct amdgpu_bo_va *bo_va = entry->bo_va; + struct amdgpu_device *adev = entry->adev; struct amdgpu_vm *vm = bo_va->base.vm; amdgpu_vm_bo_unmap(adev, bo_va, entry->va); @@ -974,15 +975,20 @@ static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, amdgpu_sync_fence(sync, bo_va->last_pt_update); - return 0; + kfd_mem_dmaunmap_attachment(mem, entry); } -static int update_gpuvm_pte(struct amdgpu_device *adev, - struct kfd_mem_attachment *entry, - struct amdgpu_sync *sync) +static int update_gpuvm_pte(struct kgd_mem *mem, + struct kfd_mem_attachment *entry, + struct amdgpu_sync *sync) { - int ret; struct amdgpu_bo_va *bo_va = entry->bo_va; + struct amdgpu_device *adev = entry->adev; + int ret; + + ret = kfd_mem_dmamap_attachment(mem, entry); + if (ret) + return ret; /* Update the page tables */ ret = amdgpu_vm_bo_update(adev, bo_va, false); @@ -994,14 +1000,15 @@ static int update_gpuvm_pte(struct amdgpu_device *adev, return amdgpu_sync_fence(sync, bo_va->last_pt_update); } -static int map_bo_to_gpuvm(struct amdgpu_device *adev, - struct kfd_mem_attachment *entry, struct amdgpu_sync *sync, - bool no_update_pte) +static int map_bo_to_gpuvm(struct kgd_mem *mem, + struct kfd_mem_attachment *entry, + struct amdgpu_sync *sync, + bool no_update_pte) { int ret; /* Set virtual address for the allocation */ - ret = amdgpu_vm_bo_map(adev, entry->bo_va, entry->va, 0, + ret = amdgpu_vm_bo_map(entry->adev, entry->bo_va, entry->va, 0, amdgpu_bo_size(entry->bo_va->base.bo), entry->pte_flags); if (ret) { @@ -1013,7 +1020,7 @@ static int map_bo_to_gpuvm(struct amdgpu_device *adev, if (no_update_pte) return 0; - ret = update_gpuvm_pte(adev, entry, sync); + ret = update_gpuvm_pte(mem, entry, sync); if (ret) { pr_err("update_gpuvm_pte() failed\n"); goto update_gpuvm_pte_failed; @@ -1022,7 +1029,7 @@ static int map_bo_to_gpuvm(struct amdgpu_device *adev, return 0; update_gpuvm_pte_failed: - unmap_bo_from_gpuvm(adev, entry, sync); + unmap_bo_from_gpuvm(mem, entry, sync); return ret; } @@ -1596,7 +1603,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( pr_debug("\t map VA 0x%llx - 0x%llx in entry %p\n", entry->va, entry->va + bo_size, entry); - ret = map_bo_to_gpuvm(adev, entry, ctx.sync, + ret = map_bo_to_gpuvm(mem, entry, ctx.sync, is_invalid_userptr); if (ret) { pr_err("Failed to map bo to gpuvm\n"); @@ -1635,7 +1642,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( struct kgd_dev *kgd, struct kgd_mem *mem, void *drm_priv) { - struct amdgpu_device *adev = get_amdgpu_device(kgd); struct amdgpu_vm *avm = drm_priv_to_vm(drm_priv); struct amdkfd_process_info *process_info = avm->process_info; unsigned long bo_size = mem->bo->tbo.base.size; @@ -1670,13 +1676,8 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( pr_debug("\t unmap VA 0x%llx - 0x%llx from entry %p\n", entry->va, entry->va + bo_size, entry); - ret = unmap_bo_from_gpuvm(adev, entry, ctx.sync); - if (ret == 0) { - entry->is_mapped = false; - } else { - pr_err("failed to unmap VA 0x%llx\n", mem->va); - goto unreserve_out; - } + unmap_bo_from_gpuvm(mem, entry, ctx.sync); + entry->is_mapped = false; mem->mapped_to_gpu_memory--; pr_debug("\t DEC mapping count %d\n", @@ -2053,9 +2054,8 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) if (!attachment->is_mapped) continue; - ret = update_gpuvm_pte((struct amdgpu_device *) - attachment->adev, - attachment, &sync); + kfd_mem_dmaunmap_attachment(mem, attachment); + ret = update_gpuvm_pte(mem, attachment, &sync); if (ret) { pr_err("%s: update PTE failed\n", __func__); /* make sure this gets validated again */ @@ -2257,9 +2257,11 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) goto validate_map_fail; } list_for_each_entry(attachment, &mem->attachments, list) { - ret = update_gpuvm_pte((struct amdgpu_device *) - attachment->adev, attachment, - &sync_obj); + if (!attachment->is_mapped) + continue; + + kfd_mem_dmaunmap_attachment(mem, attachment); + ret = update_gpuvm_pte(mem, attachment, &sync_obj); if (ret) { pr_debug("Memory eviction: update PTE failed. Try again\n"); goto validate_map_fail; From patchwork Thu Apr 22 01:30:55 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217295 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A9C0AC43470 for ; Thu, 22 Apr 2021 01:31:51 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 634F4613B0 for ; Thu, 22 Apr 2021 01:31:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 634F4613B0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4D3106EA2C; Thu, 22 Apr 2021 01:31:27 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062.outbound.protection.outlook.com [40.107.93.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id CCC196E027; Thu, 22 Apr 2021 01:31:21 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ofTlIK7aDHLed+XQxu3udt8Z+N/j7qIwSLZ9Am/DfoB5LJ2GY8SD4M5QmRQYiQUZQzjHOHkfM3az10iunXDF036Ef1vMDrhSsv10zy4ZJtOnO5mwoRTdA7furikqJM1hShvtmkAkfeKUETbEoc1nmWFyfZqiEwTw/k+vDCWEKntB200XSD1eeaZ+zZEtXsN0f1sacobpRvXcxTIwTXq19+6ZGNN/Z58dugFXVa+1/QsYp3vzZqFsz2Ye24FhLZ6/mPdI2i9L37arvTp6WX0TiszgnjcQUGk7lpIEQNbKi3cGoVK3zbgvkIdPYpdgjM5Uz/9/JxibwdgCJ1ANxTTMfA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DcvBxQQN3XxmKl/jkoPLrdRLUs7jOhXovQIxMWHgDl4=; b=kXiOUzWALUAytJVWGmkDXGi+Ihl7lK6ecAn2oYagZBQWJXejpEuz1IiGLJ9N3XapHh2U2ykWXvHwRytX0Iy+DP4gFVuWAUsRYGGe1vAG6SbhKjuVuj/ESaYiSD315f/UKALVwsGYMazRMPgmtvhmwKIZ0NannVWUoEV2lDz1qzyParRlJQLQxWyMFclri+gnYkaJNGbcOyd6IGakqif681SXPa65GM82587aojP3faX23AzjOpBJnURpRkmcKjz0zNAff8M27RTaddfZN3gcB/IvSVjfnokUcfxrGD4FShsH5RHpuhbLqT3Vv5L/OUappb3runpnW+ZJN2vEAohKwQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=DcvBxQQN3XxmKl/jkoPLrdRLUs7jOhXovQIxMWHgDl4=; b=p1EBMb1l4pVAn/64msbhjQwXDpKXN2k5hEgO6ffs8JEjWkc45pAgBtXkieNVEIKsyE6Jmvxkb2Pr2NjlSS4L0Esfk9HUAJAPK5nmNljxUQ1OT/N9o3Dh/8h2Hp6R+lgypQBJVwe4k5hw+cLwKiDOsrzkrX6S+UJ33a2x9aUdURg= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:18 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:18 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 07/10] drm/amdgpu: Move kfd_mem_attach outside reservation Date: Wed, 21 Apr 2021 21:30:55 -0400 Message-Id: <20210422013058.6305-8-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:17 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 36387260-bb42-4eb0-0e4f-08d9052e5387 X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: Tg01OENGkBelaA5rRdYIvvJ3iR0YUPPzNwqOv5AMgdyJG5hXPRCw3x/8fORXVBFMrl+A6UgeqTdNy6S20CCYKFMS1J6xS2g579HUGw/HKQDtdarBvLvT+P0kQCMKNWViTXK/04dyaPNMVVKWP4N3g3tHwXLA/SOaEbkzlUsQvMDG9hTBllo4X/ik9ArTOcGr+rBahWWH2FhYJkztlCUzOmo4ppZ6nHrAxwbYyTCmXLPBR//5EXeAWlCj8NzAAUPxuCC6Z1mOJLP3ZrsU77jORPVUm+eObHmIiMsCPURVRgAMfEYMc7Am4qXbieyH8xfh3TED14eCmPsmoT88mwJ8K/ZC6MY4vZ98Me0Co1zfofYF4yMRSuouGAPy/tWi7gqK+mwcr/jdY+l+8ryTb/9IkkHaMcaTlif/zpuNeL2fGr/dSfG/MyQqQqCBUFJ1eR9P+7VFE6SqoUZGp/mSJDyt8jg7GwUU5gOskWs0vNUN87GIJ0bq3MSpWpmXMbnzai26oTtU3pi4H5TZSKN0RxGqUYEGtCrhgFZ5TUyw3Z7F7iX5M5PZ2SzRT4zoczCA3aRcyA0btKFf03S5BuscakptPejwZ3A4v8XiAjHRudhfZWx1ZmKJ4IDIRCFbQU9saE8hB4aTh2i81gWF4ZytarLmvQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: vG6RE2VOmesT3r7n++QsAUh1Wn+uN8B4QG6agOGzeAV8STdHVFFJHurBbflxm0eObz5CeUliVcZ9hEdzRn8qtCTdsjBX3X5tZX0SS8TrHk0lgk4sD9fyh71eV0vAA+kEHgOfxYr3fhSTH8hWaV3UD8AWezmR1BFVRkU2At7u1+YY0puWlqEBIu9pDPCCtkvK9EgHXcsskhMoSV4qKn3WBoixxTqtn2GIA7axRygaB4UNRsNY6wMVWQgLvVYtg8NsQtnlRlXI2QRjnZnOPKeKTBHGkABZHs1EL7UYZ7kdA2a2gm0fxqiK9Sf4hBbpEbGrk8jpWSNg2/+D3mxdGbasSjnWWHrgSKdiiFxIkxWL0GLk+U/yP+EfJzeQ9/hQLnD+RT1yr/cAe7aBJLaNFTNtavf/cPOe5LKf31LaR25holHbODdupEwVhPBeGJWICJsYSZCuBcBfbFyUYnRPdX8+aG/IJAo2bc3hwYjHgste57vocMAW0q+y+iQ0MqjI/MBQByQ0Wm1UaZepUqtKHdWsj+CWS4FSD00JQm98mctQMjPGr0/mR8SBxxKE6z1i/zT6g2DNfY4kM+Byw5dR/ca444L+TbrP8C1HKHLid4LGwgmcBGZLlXyU3HnxH4BGLNXFLZrfJ3mEJ/GtLqozTVNOb5sivFcf+aWZEDM6tIAHP56l7SSZif5Uug/6+BqWwCqZPvwSQm0NyNg0pMIo7C+7CUt4Zq0fadLIT3LFiLgNa6IgN9w99x+eQ+uMPfpFlSyjj3Tpow0NAL+hKcdqHTEym1u+2cfRMguWmNSxyxerzcdNl2fxROE3u/hsDrau5tWlsBrrqVcbJIr4onjHdtXmOW09rtQqxB62XAFnZAGurd0acBQJ0f+c2O3sWrWYRSBdCOYnojBwWESXpSGVFdTnAoPeM3MhxdNxfpQtpoBkJJDD3jDXq62+iDzb4fyaUQ9LyadZ4LI5kqOWajQd/2Vih+c1lDFb+048QXB9mrlCTABylirUCMoBIkneODsp1dPkvfelZw4y1drn1BNo6szlKHv7PD0eArnaB4nKfGyV306NTqBA4Y/28IEH+2mByS0JR4m7gIuvUysFyL9C2X8J5lGdRtU2eXDvhbp0cHfwr5/X88KjdfdAUp5xP3R/zdR+ZnIbZgzoshdgMrQ4SrhxonFRGvW4Lc0U2u1Zz0DqvxjyJHzjBOfBeETlPMXX87rchuhQwm+zwdfRdCDBAx5F9uI6cec7sxqmOPUskJtYYgzdfETbyZxtOyAAag/h48UD5j7j/hXuAdeqMO6ELSQp2McYDBiA00DsHU2b10luwkpJd3O7fk9rK/wFzVt2T13x X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 36387260-bb42-4eb0-0e4f-08d9052e5387 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:17.9502 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: ju13sVNDXgWKimULig8WmqnwEyz7B4s00Dm/HhYiuGuvuWva9mov8K5XtEV9rJHUbhTVx2mhCLK9HJ6XPsNisA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is needed to avoid deadlocks with DMA buf import in the next patch. Also move PT/PD validation out of kfd_mem_attach, that way the caller can bo this unconditionally. Signed-off-by: Felix Kuehling Acked-by: Ramesh Errabolu Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 75 +++++++++++-------- 1 file changed, 44 insertions(+), 31 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 7d25d886b98c..9eeedd0c7920 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -577,6 +577,34 @@ kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, } } +static int +kfd_mem_attach_userptr(struct amdgpu_device *adev, struct kgd_mem *mem, + struct amdgpu_bo **bo) +{ + unsigned long bo_size = mem->bo->tbo.base.size; + struct drm_gem_object *gobj; + int ret; + + ret = amdgpu_bo_reserve(mem->bo, false); + if (ret) + return ret; + + ret = amdgpu_gem_object_create(adev, bo_size, 1, + AMDGPU_GEM_DOMAIN_CPU, + 0, ttm_bo_type_sg, + mem->bo->tbo.base.resv, + &gobj); + if (ret) + return ret; + + amdgpu_bo_unreserve(mem->bo); + + *bo = gem_to_amdgpu_bo(gobj); + (*bo)->parent = amdgpu_bo_ref(mem->bo); + + return 0; +} + /* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added @@ -598,7 +626,6 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, uint64_t va = mem->va; struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; struct amdgpu_bo *bo[2] = {NULL, NULL}; - struct drm_gem_object *gobj; int i, ret; if (!va) { @@ -632,15 +659,9 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, } else if (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm)) { /* Create an SG BO to DMA-map userptrs on other GPUs */ attachment[i]->type = KFD_MEM_ATT_USERPTR; - ret = amdgpu_gem_object_create(adev, bo_size, 1, - AMDGPU_GEM_DOMAIN_CPU, - 0, ttm_bo_type_sg, - mem->bo->tbo.base.resv, - &gobj); + ret = kfd_mem_attach_userptr(adev, mem, &bo[i]); if (ret) goto unwind; - bo[i] = gem_to_amdgpu_bo(gobj); - bo[i]->parent = amdgpu_bo_ref(mem->bo); } else { /* FIXME: Need to DMA-map other BO types */ attachment[i]->type = KFD_MEM_ATT_SHARED; @@ -665,13 +686,6 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, va += bo_size; } - /* Allocate validate page tables if needed */ - ret = vm_validate_pt_pd_bos(vm); - if (unlikely(ret)) { - pr_err("validate_pt_pd_bos() failed\n"); - goto unwind; - } - return 0; unwind: @@ -1478,12 +1492,12 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( pr_debug("Release VA 0x%llx - 0x%llx\n", mem->va, mem->va + bo_size * (1 + mem->aql_queue)); + ret = unreserve_bo_and_vms(&ctx, false, false); + /* Remove from VM internal data structures */ list_for_each_entry_safe(entry, tmp, &mem->attachments, list) kfd_mem_detach(entry); - ret = unreserve_bo_and_vms(&ctx, false, false); - /* Free the sync object */ amdgpu_sync_free(&mem->sync); @@ -1560,6 +1574,12 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( mem->va + bo_size * (1 + mem->aql_queue), avm, domain_string(domain)); + if (!kfd_mem_is_attached(avm, mem)) { + ret = kfd_mem_attach(adev, mem, avm, mem->aql_queue); + if (ret) + goto out; + } + ret = reserve_bo_and_vm(mem, avm, &ctx); if (unlikely(ret)) goto out; @@ -1573,15 +1593,9 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( bo->tbo.mem.mem_type == TTM_PL_SYSTEM) is_invalid_userptr = true; - if (!kfd_mem_is_attached(avm, mem)) { - ret = kfd_mem_attach(adev, mem, avm, mem->aql_queue); - if (ret) - goto attach_failed; - } else { - ret = vm_validate_pt_pd_bos(avm); - if (unlikely(ret)) - goto attach_failed; - } + ret = vm_validate_pt_pd_bos(avm); + if (unlikely(ret)) + goto out_unreserve; if (mem->mapped_to_gpu_memory == 0 && !amdgpu_ttm_tt_get_usermm(bo->tbo.ttm)) { @@ -1592,7 +1606,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( ret = amdgpu_amdkfd_bo_validate(bo, domain, true); if (ret) { pr_debug("Validate failed\n"); - goto map_bo_to_gpuvm_failed; + goto out_unreserve; } } @@ -1607,13 +1621,13 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( is_invalid_userptr); if (ret) { pr_err("Failed to map bo to gpuvm\n"); - goto map_bo_to_gpuvm_failed; + goto out_unreserve; } ret = vm_update_pds(avm, ctx.sync); if (ret) { pr_err("Failed to update page directories\n"); - goto map_bo_to_gpuvm_failed; + goto out_unreserve; } entry->is_mapped = true; @@ -1630,8 +1644,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( goto out; -map_bo_to_gpuvm_failed: -attach_failed: +out_unreserve: unreserve_bo_and_vms(&ctx, false, false); out: mutex_unlock(&mem->process_info->lock); From patchwork Thu Apr 22 01:30:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217287 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 73E51C433B4 for ; Thu, 22 Apr 2021 01:31:47 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 2F03B613B0 for ; Thu, 22 Apr 2021 01:31:47 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 2F03B613B0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id ED6F66EA18; Thu, 22 Apr 2021 01:31:25 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062.outbound.protection.outlook.com [40.107.93.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id 258EF6E135; Thu, 22 Apr 2021 01:31:22 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=UTAGDNUmnbAkEEJFGSAcDmY607qrWIh37vBc0gTgRuzc0V8u8M54mYYz+leZsWhvBmeU5g4vehUzy3ghE4dnch9tfdWBa5wsyThaCtlUb/Wyq7O5HY7wQd7VVucqu3ZMpzMbk6RkIC7F051ezZ0OU84ZGv7lByQXTM9Ilvt1i9AA9wNzP1MdfwuNZiQaXbQr2OV3j//4zvUpeAn1JYkfbNxOeTLZwvFr9ia0JFlIMu0XDMXV+P/V0k/0OpPuT8J7rKkQyMNExhWEuwm2SAgUAnNmnxbqvMar42atPtXnApyRuNScY4/izeGjFmSSHYqGDT7bMvpSI9HMwLKHX+OA0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0AH8vwQB/hMPZGMhmIfyXkt08cCoOdvghm8HU4NJz4s=; b=CWEh2WkxkREKKyO6DspqUKoPIP0mLAGdWFnGYTJ0+9BtaVJPLo06Lrtn/HZVRHPtj9u0V6VRWBBiThz1eSWb7tyP/Mlb9TcLym7a1teYpAhTlKqg35o/0ahGQI+AQiDrbTTzoFrJsP1zz7ThqSg2OBd1o8fiwvAJkOVGw7CzQ8BD/tj3tx2GiqnvFEcFhQs+0zDKv8kbLyJkv/e6et53Xt42ISECRmvQGWagjv3gkJqmZCKrsmUXEd1MEtJ0uhfAhxY/5JvU1zSgWi4jA3ZOYLhJLe12/pM26GU+N9quBChvQHOSul4BH7jJns0uOIIZPdGO8epHfSliERWj6xd1ew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0AH8vwQB/hMPZGMhmIfyXkt08cCoOdvghm8HU4NJz4s=; b=Vsi0AFmRd6CrjNSrNtHpad0IsAk+GNUiFrBeb4U2tB2BUBlvkWTdkJXzyWD4+bMUhWhOKu0/qVPYp2hAmSw8w9EerrzCUzeyL3c/KXtWsi3UsAPfZkEns9jty98VT5oRJjNFn8bpSf7dzdrxEU0SREUyneVuDb+35vFwfnqRYVw= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:18 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:18 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 08/10] drm/amdgpu: Add DMA mapping of GTT BOs Date: Wed, 21 Apr 2021 21:30:56 -0400 Message-Id: <20210422013058.6305-9-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:18 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f7260c95-6272-43f8-7bd5-08d9052e53cc X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:608; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: oX7kOJNQ5kOo0J29o+uubYnPuoh+q9jvy+vDnTyhNZDvOReRIRSSKKVK6lB8KJvpA38i/OeRU22zU9VDKAhiOf+b0YryHrrcV3cG58S4e3Ir53+5wZaB/4Gub9jRM9HuKZ07GyGTyjLN3eAfp4hjsg4QUlBRvqPgJrzg/vqNVUlxkNxpvzJXP9+idJssoVjdwbN2dwAlXj3cTF4kRqLjiRrl6/FnBgiaCktHfJaCobnv0m5knH5gT6wpCakjn+ACKpmlloMTj+bpgMLa4IxaklJ0HTpJ5RAiZUb3aapTXwkpwDcizUh0HFld+J51I+mjvkHIdm113aPTd1xMYQyJ2WBIhT7EuIS/CowwzukYC2A9rL9p8PnyRBfMAKie/DnNHN2Z2ri440DDgrbjyuBWzWlh8bXltiWYuOZ1VPObdpyYaomLkXVpyVAx/qBA1Qqn+c036XXEAaJlctHbFYXcqBIAix17bvSZfg2+Ao6HUQ6knphzc+oID6FZfhwIjc6kEfn0xR8G++4ZqNYt82bvjtFPgctiY1Pi5v9YLgt2jg35hhkrhPOATRPwFHO0/oU6wEJ5EyCywFWUCPFKuP+OrTDsoYwpaYCOshvOM5wE8sLs9NtZE2IjnsEXtjfQgk3ccYWtZICNxnbMsBv5uh2rhLMoGEaS0+hWMSMf1bhIONw= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: jA5hJlBMRo6D6dMQ8SFetooubqUUURIoe6uI/SXpU1r+lCT8xK9Yf/t+eo91QOvRvRoeJiG5hvUpswY7uVVTqauxChJQlCtJ51rdhaFQRR6+QKxhOlVW1Zi1k5I1S7kwobMEmIt3cuGudV6ZFaFNp8ne2C8KW7kmEGDLFcn7BEC7CqDmNpNecsS1APs7dvOCg/onjczJj31Az5IXOn0hz9zon9ARIEnLiBhs8R//y7oH2qb9NglBAL/IV7Oy0Mv9Qq1n45A26tqpeJPs1zWaVghzrbkCba4iXAEbpLjhFvU8pABEOwmI2F8jjfQuxIMx4tqWTicABHCXDopX/KoNkleVlURrISu53Jj3mE091fD8h1MxPD0GQVtkvhcKAwjGJXQ0ITB/kMvBm+eqqNmL8TgTn8685SCVMdmIiT6zJUo3y+o7cwL+Xq2lVLetO42aOYC1degk/4PV7aUMBoMmdOQSuxeCsTZwYV55sfXRLPAdoVTfxB7Xb4XKetdLm9EeLLl/7JY4MAAPSe75oQN1XG9BGzu9uJyloTtTCEp2Za67LdiaOjpOuBRmPEqCcdLi00EM5gkdAX965ZWS/I6y2x03BOHzSQdCtgHv1mQyT/x5+U5/3bXp6Q1CtyNmIy2343Lq0pxQB/qzeNwnI1qTAw/JzyL4TeXYoLU1aHL9D5+6JoPGqntTCNHxgh18LX25MUcPj+O0dhvUPOi1Nqt34X/eEetiWyANZAWmS+YudRhyWWjr6uTLkXGUT4VHKIMeWzV/AC/kx83rpgK9BF+pe63YKXn+kI5/Xons6rocWIlu9EIJAhhPtu7/ljENrhNC9VUdK3h8P0bjySW/2hqwkrXVgUYri1DVx8b7eMqzdEHYH+0IpPm8vWrxKdUhUVAU4MwxQkhg196Afy4dZEIhiFf0TVkQL1Qfv3oOtTxa8UXnMlfAoTLWwXmK8qlCS069/9m8bzp97LUybSMtefaA8TJ2+Ib6baeR/ZjTobGOnRViBB1f6JlLSFY4J4kl8lxK5l+Kqr8R7kRaJKBhfpfbCRg1d+WgMBiwR3RW2K3V7K5FP9GtRYUg8Q6aqy4JfCLPF9NoFjkEHruJYV7x/IijmunS3ka4nlh5FUzONGEiKpUenULfASNT6HxZjfz/KZXsQp1pbAsvKuLqwddnl5r+uZentYuY2C2DBMkV1Te/tBGIv0cfIDRH/I6mn4FV8pD5azHAKKsI/mhxAxXuAnT5/ZzOGqsXtU63ORYFPruSfuuQQvIlvuWqzelTNNXtnbsopNQxiacfxSL8NbsPg7smzuKRpUowlTq+Fd0Ui6am3778KUw+qoa12iuzQmJJMzcC X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: f7260c95-6272-43f8-7bd5-08d9052e53cc X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:18.4271 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 809ADPzee2yF/McJUQAWsG9WVB1QF8OoeJ8N93dZOltLfiKJ0kvXtgQkkui5+QEljkiEQE65kX5E2dXddHY2Tw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use DMABufs with dynamic attachment to DMA-map GTT BOs on other GPUs. Signed-off-by: Felix Kuehling Acked-by: Ramesh Errabolu --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 + .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 76 ++++++++++++++++++- 2 files changed, 77 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 63668433f5a6..b706e5a54782 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -41,6 +41,7 @@ struct amdgpu_device; enum kfd_mem_attachment_type { KFD_MEM_ATT_SHARED, /* Share kgd_mem->bo or another attachment's */ KFD_MEM_ATT_USERPTR, /* SG bo to DMA map pages from a userptr bo */ + KFD_MEM_ATT_DMABUF, /* DMAbuf to DMA map TTM BOs */ }; struct kfd_mem_attachment { @@ -56,6 +57,7 @@ struct kfd_mem_attachment { struct kgd_mem { struct mutex lock; struct amdgpu_bo *bo; + struct dma_buf *dmabuf; struct list_head attachments; /* protected by amdkfd_process_info.lock */ struct ttm_validate_buffer validate_list; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 9eeedd0c7920..18a1f9222a59 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -524,6 +524,16 @@ kfd_mem_dmamap_userptr(struct kgd_mem *mem, return ret; } +static int +kfd_mem_dmamap_dmabuf(struct kfd_mem_attachment *attachment) +{ + struct ttm_operation_ctx ctx = {.interruptible = true}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT); + return ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); +} + static int kfd_mem_dmamap_attachment(struct kgd_mem *mem, struct kfd_mem_attachment *attachment) @@ -533,6 +543,8 @@ kfd_mem_dmamap_attachment(struct kgd_mem *mem, return 0; case KFD_MEM_ATT_USERPTR: return kfd_mem_dmamap_userptr(mem, attachment); + case KFD_MEM_ATT_DMABUF: + return kfd_mem_dmamap_dmabuf(attachment); default: WARN_ON_ONCE(1); } @@ -562,6 +574,19 @@ kfd_mem_dmaunmap_userptr(struct kgd_mem *mem, ttm->sg = NULL; } +static void +kfd_mem_dmaunmap_dmabuf(struct kfd_mem_attachment *attachment) +{ + struct ttm_operation_ctx ctx = {.interruptible = true}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU); + ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + /* FIXME: This does not guarantee that amdgpu_ttm_tt_unpopulate is + * called + */ +} + static void kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, struct kfd_mem_attachment *attachment) @@ -572,6 +597,9 @@ kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, case KFD_MEM_ATT_USERPTR: kfd_mem_dmaunmap_userptr(mem, attachment); break; + case KFD_MEM_ATT_DMABUF: + kfd_mem_dmaunmap_dmabuf(attachment); + break; default: WARN_ON_ONCE(1); } @@ -605,6 +633,38 @@ kfd_mem_attach_userptr(struct amdgpu_device *adev, struct kgd_mem *mem, return 0; } +static int +kfd_mem_attach_dmabuf(struct amdgpu_device *adev, struct kgd_mem *mem, + struct amdgpu_bo **bo) +{ + struct drm_gem_object *gobj; + + if (!mem->dmabuf) { + mem->dmabuf = amdgpu_gem_prime_export(&mem->bo->tbo.base, + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DRM_RDWR : 0); + if (IS_ERR(mem->dmabuf)) { + mem->dmabuf = NULL; + return PTR_ERR(mem->dmabuf); + } + } + + gobj = amdgpu_gem_prime_import(&adev->ddev, mem->dmabuf); + if (IS_ERR(gobj)) + return PTR_ERR(gobj); + + /* Import takes an extra reference on the dmabuf. Drop it now to + * avoid leaking it. We only need the one reference in + * kgd_mem->dmabuf. + */ + dma_buf_put(mem->dmabuf); + + *bo = gem_to_amdgpu_bo(gobj); + (*bo)->parent = amdgpu_bo_ref(mem->bo); + + return 0; +} + /* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added @@ -662,8 +722,20 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, ret = kfd_mem_attach_userptr(adev, mem, &bo[i]); if (ret) goto unwind; + } else if (mem->domain == AMDGPU_GEM_DOMAIN_GTT && + mem->bo->tbo.type != ttm_bo_type_sg) { + /* GTT BOs use DMA-mapping ability of dynamic-attach + * DMA bufs. TODO: The same should work for VRAM on + * large-BAR GPUs. + */ + attachment[i]->type = KFD_MEM_ATT_DMABUF; + ret = kfd_mem_attach_dmabuf(adev, mem, &bo[i]); + if (ret) + goto unwind; } else { - /* FIXME: Need to DMA-map other BO types */ + /* FIXME: Need to DMA-map other BO types: + * large-BAR VRAM, doorbells, MMIO remap + */ attachment[i]->type = KFD_MEM_ATT_SHARED; bo[i] = mem->bo; drm_gem_object_get(&bo[i]->tbo.base); @@ -1522,6 +1594,8 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( /* Free the BO*/ drm_vma_node_revoke(&mem->bo->tbo.base.vma_node, drm_priv); + if (mem->dmabuf) + dma_buf_put(mem->dmabuf); drm_gem_object_put(&mem->bo->tbo.base); mutex_destroy(&mem->lock); kfree(mem); From patchwork Thu Apr 22 01:30:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 324C7C43462 for ; Thu, 22 Apr 2021 01:31:49 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id CC1F4613B0 for ; Thu, 22 Apr 2021 01:31:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org CC1F4613B0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5E4656EA1A; Thu, 22 Apr 2021 01:31:26 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2045.outbound.protection.outlook.com [40.107.93.45]) by gabe.freedesktop.org (Postfix) with ESMTPS id AD3E36E9B2; Thu, 22 Apr 2021 01:31:23 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=M6tZbmwbTpxrtHtZEVVGmt8tEAJjZjPUToHEvqH8zoVFCrMylsta846SfGwUz/h0zoXABbtEYxz5bkf4lsNx5zujeVQDOBq5NVPox0OeHUicF3Tqyy7WPQshaUr56vSZe54KXPtQ0w71lh1jmYwE0RhzmR9b7eEov0SLoLqGEZKthbryoxh6sZ13ug92PC/msXxFuvr0L0SNlKXzHTR/mfx1xuaJdhov+OZgp0mPeOqr78ZAe5B0heaS9xkzXX8o9BV3I7SF/o5ZVo4iQ9i17effCQ/BACx/wvZK4Z+WQgDjssMEq3C2g2FGwI4HFeil8Xspvnx7celxVLpSa+I4vA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eN2Yd6c53oyb+YR1k7QtiRCenmta+Oh7ACloZoJ7IWE=; b=UDHh7nWkx6MMozIwOHyzkpvdFta4hP5JmIhAIvui9RjEhIV4bfQyT87yRPuHLbZzUklfpTqujasIqWzics8uyQGByMLAz3v/HMBbPs/gOKFTIwOZYgci/xJMSBGS94ck1WUpv1mtuGdzRwB/wN/95Dt5J9JhiQcf5f5Z2CDhCfIL54TqgAZ/WLRoq+Yx2HKfbEF7yE3nPsETlGJq5CyRf5DnfxyUxpqaWbtzr0Fc0DIR0D+zOu62PQ8BiiF4eJSmKyFvuFqFcOpgp+qplBlAyM4ccy2Nj8dzNmPOuK+vENaC1kVALWD9oNNHC+38D30FS2JE5qESn178TCggPis4Xg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eN2Yd6c53oyb+YR1k7QtiRCenmta+Oh7ACloZoJ7IWE=; b=MVXRybiV09+6nNDjaaTMITFDJLV3ff1pW8j0GCEuYm/K1dAlQVGdePXnOe0CmbZvbl/Rt6vXCB/XKaEroc4WXD5xbLyTPvKJymnaJ4ffuQXfDE13n4DWg+dCRj9o+58dgw6Jo1GN/HUBTO9qXJ72kBaEHYn/EuGnzOv30OXVqA8= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:19 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:19 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 09/10] drm/ttm: Don't count pages in SG BOs against pages_limit Date: Wed, 21 Apr 2021 21:30:57 -0400 Message-Id: <20210422013058.6305-10-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:18 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4431fdc8-ad74-4329-b599-08d9052e5415 X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2512; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: NRB9rdImk3OoQiH0MLYOEUqY7TZ7GLaHYDSlCBS5VsumnR3B5InrzasK6Dr/j/8T8lUZNHfy3hgeLUlcm4DjAG1PkO5GmOamfzDizpcIsa8Ateb7MxuLpD3aW+3XsFeMAD2xeaIg2SXoI+G1QY2NQJweFaOHNjALsgskNn7UvWYMNfC9uyhrKGPxfl/xCC32nLUYSp7yckTBehAKNWvQEu7w4MuR0gHZ9bRc1cc9iXI5m4iIDJ/FZs1aU0aHwSzsRwQki0AReQeipvpwg/XFGkPxj0YSqgx2EtDZ4fx4UafcpS72iZT2foA3NwKCJirwsh4fStKathsFaLzkV7MXYw5aA02t2HSmbS6vhUM00xPBh5Lu0R22ZfagQ4Kdd6nHpeJV9SqqP7s/btgHkZzjhAkX+H+Aae9hTKUXkoCi0eSVhNWOKPtt3hhE9P/S9DQNoHpUF3vfNXuUztnhj6obqjTfT8LRTDVxZRbs0isajVL97zx7dZ96aLTLqeD702eO3ZNBtuVbeBIpN0MfZUeSD5z30i3Mf2DmVOdqflKFGn03+vavb/7P4uBzPXBrsQPck0Rf3d3paujvOIHBufaaAJ5o69P9rb1y/IGtYQmRFTyAUEv4TSRbYCEd1dHzh9Q0kTkr1V5JJLQWgW4tfZKakEtxbSl9DH7kYKQC165y2N8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: flEpaH/6b1HJ9cy2vlM7rrTB+akhCrORn3QfkkwnOUcS90w8Wd5KYNvuBnoRHKNCLjgGTvC7IoKHt2ldESYugNxhNbR2qlMUeW0I4esFxQv526BYi1UAE1t1XQSvnXRE3KMjY90kjj/wDJjc135A4shJB5FF3C2xxQEqn6aqQZ6a3eKMlizbhEC7b1Roa2JJLmGXL7KxQPVrPtqWn99MDOhtl91J5GymyELHsVML61UqPf5jplkAndw/k2AngpUEGltbQrqwQ8atRTOTk3fVouP5K76c+HWmZMmoPStjS6mDgalaSbMsAlyPvCxSKa8iX3R3U+/o8AgjdCc/2EAUd7/U6IHkOu0MHsVLL9Q4G6X16E0QqRpq1bZk/EUvsNV3zu00kHv8WVTx7fUZJox5pplul+aGTPly42mySCN6srekz4waTCBWX09iP7QBXPaHQi/FpMglQN5XZgGIGizaFXTjDsEZrPnQtyb7I2pon7Q2Ex1yDDZ91jlnv/uQXjuW2+jfsxoFpw6VGDPTpMKrXhaGDjIwmcWFr/EAdUFh83kgcE2NcEsUJsHQCLWyHI1L8QXfRjyxEQR9viMGBI2OENF73zRYO1CBWB5XJx7eiRK0uW6iBeBGT6C5YxNLriEalLz71UtgtlNeEoN6MZFiPQwggB/yvrF1d7rB7ZM1CUGYfCSIkb/9XiHvRQZYmfZigbrEb/tUBcVDuf0nC/AD4/npn6z+X3OmhR59VNDDTNVzBe9OD0yW0mEjZ+kT8FY1bSYktWZOpZWNHCgfbYDMftEraKJSOfUd9tp3XtL5GFHy6E9R2mGdr/rg7z6DzbVrpnEDGh2yFdfqwlyWXobGbsnvMCvVoCz0iXngB00NSOXXtTsWhVpCQsB9JWcAil4LwRNSvcM991lBnUEL7JLVun6kthswwBB2EfZl6yGaz4tU5J4j+wcNDODkFCj+oKngQZeQDmC05sa032U0nUYjdGcp019Uvxvu/22fnjxkfkvl4eE+qDVohBWiEUKgrFjXRzOEtFiSxI2o5wT6A2E7dF5fVUlsrC/HgzdL+TUfPMmRchAEhga6vEDTDnxVjsub/1DA75d1TS7qQIywe6VOzL9bsr+NoBllpHuFtiCpBMRS8BThO1g095Np/nzBbYHw1xjJz+IfBiKQ2zPulS2eLKAML0KiVPMhXHG4cSjPbHCNQptBEZl5RMG0IZsQVAzA/hnBydAXi7+uag2+5K78+yiV/UQqBarDy92yT8EFmzbRnXKhcQOcUlt3JHKXuGW+02//nMVuvl6VqdWLgKqkw5DBiIhTTpIxTotawZGuXlysq/OBtvzrEo+JL69orMjF X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4431fdc8-ad74-4329-b599-08d9052e5415 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:18.8562 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: RRP0AkD6NwsysZng9xsBURwF0Y1dPkRorBc4XXIUK+I09IOQFyuMf6fkXZklJFXZuczXJPOmGta1U131u5wefA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Pages in SG BOs were not allocated by TTM. So don't count them against TTM's pages limit. Signed-off-by: Felix Kuehling Acked-by: Ramesh Errabolu Signed-off-by: Felix Kuehling --- drivers/gpu/drm/ttm/ttm_tt.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 5d8820725b75..e8b8c3257392 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -317,9 +317,12 @@ int ttm_tt_populate(struct ttm_device *bdev, if (ttm_tt_is_populated(ttm)) return 0; - atomic_long_add(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_add(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_add(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_add(ttm->num_pages, + &ttm_dma32_pages_allocated); + } while (atomic_long_read(&ttm_pages_allocated) > ttm_pages_limit || atomic_long_read(&ttm_dma32_pages_allocated) > @@ -350,9 +353,12 @@ int ttm_tt_populate(struct ttm_device *bdev, return 0; error: - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, + &ttm_dma32_pages_allocated); + } return ret; } EXPORT_SYMBOL(ttm_tt_populate); @@ -382,9 +388,12 @@ void ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm) else ttm_pool_free(&bdev->pool, ttm); - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, + &ttm_dma32_pages_allocated); + } ttm->page_flags &= ~TTM_PAGE_FLAG_PRIV_POPULATED; } From patchwork Thu Apr 22 01:30:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12217275 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2EF69C433ED for ; Thu, 22 Apr 2021 01:31:29 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D790F613B0 for ; Thu, 22 Apr 2021 01:31:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D790F613B0 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 499336E9AB; Thu, 22 Apr 2021 01:31:22 +0000 (UTC) Received: from NAM10-DM6-obe.outbound.protection.outlook.com (mail-dm6nam10on2062.outbound.protection.outlook.com [40.107.93.62]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7FC726E027; Thu, 22 Apr 2021 01:31:21 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=ay9LPdfXhN9LD1vJIi7rpw+98nYsbDwgoe5u9ZiqqxEDrqe+F2m3eynWw5sG5/LLGJ095Nx3lhFXcC8bE8S47ysBfmgSJE//dq+TT9/F92V/Z3ycmwcL+eAdBdKB3glE4YzWPH5IaAonVEaxbg1UHvsGyxhbOZkf1tTMNCjVGFe8F4finlHxjlzZrNgUzEW9aiBkCqFVNyioE6nN0piNcopumww5RDQn9bdFrsVfo+NoGXiNDPhIGK1eseL3vMCsEk82CbO2tjcq8D5MGjZG9LIDiUEkLxvK7uED1p4h937dgwYBRUy/AjeBxR3oXyBsNojIXgrTAu4bUx7J/jRLIg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VkZjhC/1rnhhkSi5MTUeQLMCe5BWmL9UkuDzzaGAfpA=; b=dHyV5lLgIW82wNd/4VMyX+BgWoa16lb69AXa2PPb02gnygNPc54fyGtsNzYXBf8tbL7DGvYgHgRcO65Q9dRQzR/cuymxPsnMP52SAmPalPhGYdKEWhLhF2D9LX9XcUA1pxvEhU5r+k4kk8u1nDUTegzmqY86iKncBWvdH84q1wuIvZadG++fY2RRzqFF4CRM3LbZYmOXCn9C7pNn05shg7kYEE/DUf6VBnRrU6+UmqUqzEMkNom4rRL7jD+r5od3eFvNaKALThcLZ19aUhWBUV9cY5vPQgfaFFofxm+w3XtBmJEGOWaq4vhcQzfHjtARsJS6G6GMZ9Wv6LiSERBu8g== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=VkZjhC/1rnhhkSi5MTUeQLMCe5BWmL9UkuDzzaGAfpA=; b=j4Q1SqpFkjHhvceCyKmDaWU1sAov04tvCZdukHC5FWCwFmPhA+evbJrN9Eihk11NRa4bmupNQmBCyIOgNGTgN1iMtlq8wu+Uv59U4NsjEZRfcK4+N/ouH8VTmt8h05c/LQKVDEgDPrwgl0VRnnTzyoIqol1UuGvVGIENM3GrQGA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4438.namprd12.prod.outlook.com (2603:10b6:208:267::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4065.21; Thu, 22 Apr 2021 01:31:19 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Thu, 22 Apr 2021 01:31:19 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH v2 10/10] drm/amdgpu: Move dmabuf attach/detach to backend_(un)bind Date: Wed, 21 Apr 2021 21:30:58 -0400 Message-Id: <20210422013058.6305-11-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210422013058.6305-1-Felix.Kuehling@amd.com> References: <20210422013058.6305-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0027.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::40) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4042.21 via Frontend Transport; Thu, 22 Apr 2021 01:31:18 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 83e12367-74a5-4e7c-c1ac-08d9052e5454 X-MS-TrafficTypeDiagnostic: MN2PR12MB4438: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5236; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: a7QGYK8PFErh7FnGiiU8TqXqPDGsvrjuabLBcQE+onSSLCPwVHjDuqeP2kjvNTSVKjO6rDkvHztDDfZHn40Zse+aCz6RyqRqafSCNnG0MQtGSpUBcej0qwbCSSjfMbcquZ7w+TOlwjOxT+5zXBITbsrcI8iGtKX7WtDXHd49jiTOwZGa60aAbhtU8ln2XddQZcOCP7cWjiXPp926Ep/yOoUBsMuUPMYh272zAL+50srODiCfQ97uVGxEpRK2p50M5lwg0JIz7B8x3y+zCa/TN0kowCSoZt6DiDQjgdodDZOqJZme5CRYzrvuN6XRVIDYtzLNcOyus94ZBNNbomGPPcTElKdwu9mrFu2+lEzXHiwkLsdE5TI5Afm81p/BX/dm9Ri8sIkYm/1bQbptOBC+ucECUHfeFafCo1jKUmTcWD2rTSq5AMZXg/R573mYl466jUEN0YeVNWsv1ffc30XQvq6Hktsn36gcP6rDmBC6mLcWvOtVSRMLL10DhzK0JaqGGt2IDE9JnyyMQW6ukPUwScykp9UkTuithsgfuOzI7gUjypHruUHGd+aBa02rPNSr0+AsBCaW602SU/ROuCcimBq9hO6b6wrHdAJMMFE/kM0lx+SaV7dCN3y8G3XoUHgOs1tdzD6bnlL0vs7OV49GUDlhA/Tkj6EzMFY5cN8YGpU= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(136003)(376002)(396003)(366004)(39860400002)(8676002)(16526019)(450100002)(186003)(66556008)(36756003)(66946007)(8936002)(52116002)(956004)(2906002)(26005)(2616005)(478600001)(6486002)(38350700002)(38100700002)(83380400001)(86362001)(316002)(7696005)(5660300002)(6666004)(1076003)(66476007); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 8lNE5KPRIWySUzwIJ759xjZiFNPWjX4lDUpe1DLtR+wiolWlMhIGm1BINZHedyeFFjs4uOF95hOsErF1DM2AWKwzXll9nz/uo5IsVflnk7+na7oOx7WHDSsO7nkqz6bT+PNh0Vn24WazrvuKuWqgnXHsVUaDNhIz1M5eI0w53x2Q2YezW1nAY9kl3DGLIm8sf4xk+9dJavsJqpLrkPHe1o6C9En7UkwbBTSL+AzP7ULgCXZq/wdkDOUYJEhgKhPc0t+QsHne5gkYn6JSDOMtIMzOSYWt18Re0EjcCy1/TFrY4qJSbnt8+kgnSU6vKm+YNDV0DwjFUMIJ5qYTbFQo/6umjQHSZtSBBDYFjCRp5T9rj/nCSWZxlSb+mwWZ/jR2ZvsbI5fmrcxn9Ek2ZkfGFyaS7vbn3tyfM7Q72yORjm7PUbbRjvfW2NAAma/68gQpydmgxtVikDOkkbDIekviBMV1aYiXdBCHmskRTGXohGP/gSW6aWYPX7l7WG2h/4Ldb+xlrSSF6nGxjcMZpyiG4r5weHLas6PnY0gLACtGoeApShgl2E3Ij2Ev6DUyQX8jYNWvKX/vbMQLXiA86cSUUC6lsK276U6sWDbsqWgt4uatovK3eulsMFIdxnT2UyKmCI7lzB8kRZnc3iboPmSWwsrbZrHstKruOmBSnSNaUbC2qgVwtlbdvjslqE1IHuUvC+WLsldyKdMfcq3pFIWWLN66xNjSOo4voJH56Oiyt/QNTDqj/g9SNuXi2HojVvBE24w0hVVCL0/9fvgmr6YOcJd4NvEA8q3sPLyIhN2ijGNg1SdbCwh6Wbfg09dQdE0mV7hlZNLItchhO/0ox94NtTj+2EGMK6jnDHW1e8yrVFR6xxsizvViVPRZVjhYHTG6kpYs1Es+HfEtivoLaOcuKvsC0giPpI+0a5d8iHh2qIThXYKXebmeoR5PSRqbdsuENyR48YuseAeX3OVAsugHkP6FmJUPtotMLdlr+94YMq/A8o5nSB2P07VdyuesRgJ1ADgbcpAwYftljn/jZ5g2xDi7Y/UunlNxVMRIaMQ1VChEG7YJbPMLdpb0krp+2VlXSUySpQKdWJu8K36kgqRzd/x/eHvTqp5xz0Z02ypX0BVQbwtB6MtGCSP+ZfLR6ipUHA9Q7h0T6ctijOdikSHZZuKnei+pGdUtlimfgIQN9p9TuWiiiazhTTX9I1AEoQvVqjuABatYrFVD7jtcTLt7ZsXkeOVu++n+2uOqd9XmGQIs5DSyPa2JzVylsUgFAX/Bks/ltDKW4CqGaPpSDm5fn+HDdvYRylXa+fX02D8SR10Ao0TOQDOW85hLqIP1s2s0 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 83e12367-74a5-4e7c-c1ac-08d9052e5454 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2021 01:31:19.2544 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 92QLOwWDYQ72c6yNLzybmKNhEGK/qlWxw6mYbjagb0+VemMgPQNnK80AU/i6lPWMeW9eF8W6WrQqqsfx4JxFBw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4438 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The dmabuf attachment should be updated by moving the SG BO to DOMAIN_CPU and back to DOMAIN_GTT. This does not necessarily invoke the populate/unpopulate callbacks. Do this in backend_bind/unbind instead. Signed-off-by: Felix Kuehling Reviewed-by: Christian König Acked-by: Ramesh Errabolu Reviewed-by: Christian König --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 3 -- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 51 +++++++++---------- 2 files changed, 25 insertions(+), 29 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 18a1f9222a59..68e6ce8dcf33 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -582,9 +582,6 @@ kfd_mem_dmaunmap_dmabuf(struct kfd_mem_attachment *attachment) amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU); ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); - /* FIXME: This does not guarantee that amdgpu_ttm_tt_unpopulate is - * called - */ } static void diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 7e7d8330d64b..fc2a8d681dbc 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -910,7 +910,23 @@ static int amdgpu_ttm_backend_bind(struct ttm_device *bdev, DRM_ERROR("failed to pin userptr\n"); return r; } + } else if (ttm->page_flags & TTM_PAGE_FLAG_SG) { + if (!ttm->sg) { + struct dma_buf_attachment *attach; + struct sg_table *sgt; + + attach = gtt->gobj->import_attach; + sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL); + if (IS_ERR(sgt)) + return PTR_ERR(sgt); + + ttm->sg = sgt; + } + + drm_prime_sg_to_dma_addr_array(ttm->sg, gtt->ttm.dma_address, + ttm->num_pages); } + if (!ttm->num_pages) { WARN(1, "nothing to bind %u pages for mreg %p back %p!\n", ttm->num_pages, bo_mem, ttm); @@ -1037,8 +1053,15 @@ static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev, int r; /* if the pages have userptr pinning then clear that first */ - if (gtt->userptr) + if (gtt->userptr) { amdgpu_ttm_tt_unpin_userptr(bdev, ttm); + } else if (ttm->sg && gtt->gobj->import_attach) { + struct dma_buf_attachment *attach; + + attach = gtt->gobj->import_attach; + dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL); + ttm->sg = NULL; + } if (!gtt->bound) return; @@ -1125,23 +1148,8 @@ static int amdgpu_ttm_tt_populate(struct ttm_device *bdev, return 0; } - if (ttm->page_flags & TTM_PAGE_FLAG_SG) { - if (!ttm->sg) { - struct dma_buf_attachment *attach; - struct sg_table *sgt; - - attach = gtt->gobj->import_attach; - sgt = dma_buf_map_attachment(attach, DMA_BIDIRECTIONAL); - if (IS_ERR(sgt)) - return PTR_ERR(sgt); - - ttm->sg = sgt; - } - - drm_prime_sg_to_dma_addr_array(ttm->sg, gtt->ttm.dma_address, - ttm->num_pages); + if (ttm->page_flags & TTM_PAGE_FLAG_SG) return 0; - } return ttm_pool_alloc(&adev->mman.bdev.pool, ttm, ctx); } @@ -1165,15 +1173,6 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev, return; } - if (ttm->sg && gtt->gobj->import_attach) { - struct dma_buf_attachment *attach; - - attach = gtt->gobj->import_attach; - dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL); - ttm->sg = NULL; - return; - } - if (ttm->page_flags & TTM_PAGE_FLAG_SG) return;