From patchwork Wed Apr 14 06:47:56 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12202003 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9B515C433ED for ; Wed, 14 Apr 2021 06:48:29 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 68A356101B for ; Wed, 14 Apr 2021 06:48:29 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 68A356101B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3CD1E6E8DE; Wed, 14 Apr 2021 06:48:25 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2079.outbound.protection.outlook.com [40.107.92.79]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2157A6E8D0; Wed, 14 Apr 2021 06:48:21 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=FyjgpyffqfzoYKDm8wjeju/KbSUvMHDRxZTvoY9wI1l0hmdgVVAy/cFdD6hx+qA5FdGN0vBsZiN8uAKILkJeUKH5dHlZSfZGSmj6bm6S/D40qbG15ySry4SySNZ8wAyjP4+KsLF8md3ynTOgg2TwEFwTXSe8HYmmkiBosiTyNXnxscGt/RRfRfofziiv8WVVGVUs4n3dZizWj7xg30P74gQp2iIl65+ZS6lOzlLDFN4d1jDpi/WjTK3uXmvRHz+dt5+XYaYujxORteMFx3E+Yi6zycaxkEzo6K3y1KCUuStisKRlF49Trj+8FdThFe5+toFI9WcL34ZXQCDgaj58wg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2GUTVJD2QQLOcysdAZoVYPo8qqJJSUCAvbgkijjckEI=; b=ERLRdHzWpNpCmZ8UuKXG+4I/6DXGeeBgEmPRdkYw3I1aEotLfvMZ2c+8vZlduAsFFfDQxenrT9WiaFT0i0xf3uJx8RseTp+zJFHPbxlNo/DoiLZvakmz0qN9uNxnWVZpNEMm5lOKka+KRh6qatHypLsHJJU2FXWg601Npoj0tpDXtbbH7XQvQnXD3+kdj6FwCxGKTgjIzg6HQf3MkK1ZO2T7TGTlBqjNyyGUt5Hxn+SSeTMNu302/pOefBdJ7ToNLUVEtVpmUxhzPn4DbPoEypq4w3qLBdxsllmYkHkhNPjbVc1+Qh4jCXJWeEtOyMj8znMzHtWJmKnvK984TKxFPg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2GUTVJD2QQLOcysdAZoVYPo8qqJJSUCAvbgkijjckEI=; b=m5iiE0YWY2URU2BFkVn/R4F9hUKQcol/dJfTuruwXcjejMcynMH7/khyYWDGFbjQXDNU5+ps04vdoAYKihqsk2ZCs5x5hnfqoJkIdsD8hmXtOmnexuSuDzcrE5f4KJtp0pttlFLkqPO9KK3z/9Nrfh2NFsxuCiVsoORfnHNokdI= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:48:19 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:48:19 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 1/9] drm/amdgpu: Rename kfd_bo_va_list to kfd_mem_attachment Date: Wed, 14 Apr 2021 02:47:56 -0400 Message-Id: <20210414064804.29356-2-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064804.29356-1-Felix.Kuehling@amd.com> References: <20210414064804.29356-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Wed, 14 Apr 2021 06:48:18 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 93dcbb30-e949-4cca-aa94-08d8ff114975 X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: /aGqvQrBI4oDhRuxxhZvLLOQ1J6FoDKSn3Gg4/hprhUE4TY/MP3yIySAToUPRtuwdY1EkSMNAzGejqZWD7qUr/1nb3A7lz9wz8AUoodzQJ8s5yXHMUBHNZNYRZtFOgb001qrpwbBf1ieuQKcK/cQDJgu79zgE8VR8xmUbxDb4S1o1qsLfLFqSoouqoccxWpucMOqtFo1prNqUCLffbRPMMJXaVo2eEvEkWDpJaxE/nwdVUzMFkg7Sye9EleoQt04Zlhs4W3T/TiA4o1JOmI+jP/LTt3CzsflD38zpX8RwtEKjv16hEEmCb9vwxB/2aw7/1R8jdrainQ/5UabUA3a/uCeqW8IyYjTyGQl5/Cr52z/Ax+SJeCGcYT8KFtWTCZ6IJ/OYeiH1p+FbKApV6ymitVQF3OQj0TpnRDiVkf1uNqYFZz3mUKMd1SUnq4K6j21Hr1ejXa1EwK9ZnGFjOeYCAZ15IMu3f8f1F8ouOGStniHNoo116WwIIr2tEtjK9tgks4MkcCyQdCrFsjFZKN/0L9NKQ6fh0bpNSod4oyo4M+/hltqhdfMe6tpKGfNYS7pb6IhWMruWeudvPrZOofH20JIjzGmyPxrMH3xS42AVgO/x9xvkMYtIrT3w/0HXjAUZB6k/7PNAClCBeefr3RK3G5Pa8jyh5zX/AsPHeAgpus= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(8936002)(86362001)(52116002)(38350700002)(66556008)(7696005)(6486002)(2906002)(1076003)(38100700002)(2616005)(36756003)(956004)(66476007)(26005)(316002)(478600001)(6666004)(450100002)(5660300002)(66946007)(4326008)(83380400001)(8676002)(186003)(30864003)(16526019); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: HXoCLqBg8TGFsyURicvynHeJJSDfOpOzo1sGW2kcw8MqNaK/nkZJY1vmABYlLQSiwBIdJCUzyXxs5UHcItgMbJJoYHtQke5woOyy8Krz910c91h51VkkypIf1mK/U7dNFCGt+4ZsAUNr/XwMki4V7thVZDgkkMXaj2j2JfHrwXiIQeGvd/SBh2nBZmMbONkxcv0KTnhqzY1ZM1ps+GgXraRiqvVeDjbPjVouQ7PelE/wjqUkcl8YqztT33C9pjpvFqvfgiyM+k1WyI3LHSo1Ugn3yp00zFS7ACzqmgMxv6N26e87XoCz8Wdz8T397aFiiU46L29K1FSE6EmuJBMZ24l683ug/FZ9E3CWshODO1yKUK+Z4y73JBzyZb1mrjoMkRD3KrECZdM+XGiUFDghDfRdYjj7ViCLh2CskgvCSxmQaMDxr1I7DokU/EJMOy9vUHbPdCp075hU0ugjPP2BdQ+NY1lG62/1CwXuNvx1cTKZCRDEjx5Z9VpJd8PktfFqU8md6fxsm8lIYzhLC71ylYjC5ALUODNvQ0sEojT/gf0NQ9pXGFmoBIfX6Ga803IBErxK1cgpWND6QJkDhVmNYEwk5IWL+/OF6Q7fPPzTzKuleZvnDMYOo6ijOGAQfgcPiBNt8YlQtgPNyXnN+tE6ib1mNSPsnhxlVNvCWUO69wfyVOOMT/+UxQ6NIGXo/jMuIy2t0suvjZuP4+TA5yc20gLKydVPzrJ5VZjdFAxtWN9+tIpItv8LV7dZyczWVk2hVLR3hNX++hBaxobWWI8expEVmFw3EOPG6wByIZhCpu8hnQZ0wQmFmqcv8jQH8y9lNeIoz9W9RXXHXUZphKRzoGQWsMPvALKpjTkB9u5Xs7rHl6KNy7io67o0SWUeQrjeX1wBRR8bGwgfvqSymqraEZ6XO1Mtjb/iClH35ymT2XvqCRLA9rEO6yiv9yApDjzqVBuWvBgqS7M9jpLzcEv4TBTHThAoYZGMR5IZWIlTSE0pVTS8p6HvmokefgP7G+vFCX7OwM+zmuNRgRYbmemw8Y1SaLnd3fxBRuNObyE+T2HkiLJZVe+e04a74gJPw3aHJ929v3aLebmeDcE3oUdmIC7r+Auq/bgXCefR+eFSnRlLugaEZOTaLx4sM0Yt4g/VY1BUvauKt9FG4kp1aTOzivTN8dJwUJWsh6QNip8pGtXh9iTmS3DdeOXk8J2IOT3P2dGVOvglqrnArM6I5x6nr/kj+mCWE80CH567QakLX2m5RBp6WlCnkkKHT0sS+3jQmv+uqU3SKbO3ahj696TIlVXwV5IrkQNjqAvm8gGzULO97is1FwXF2FQGNlZhJqzO X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 93dcbb30-e949-4cca-aa94-08d8ff114975 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:48:18.6353 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 19U2s/sNQfI30rupw+PwmSKqwz3LX0jdxSTYg0HKP69RHtz6b6h+PGm8qUHK7DO+KEsJpADODGfTMnt/2E5HBg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This name is more fitting, especially for the changes coming next to support multi-GPU systems with proper DMA mappings. Cleaned up the code and renamed some related functions and variables to improve readability. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 8 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 209 +++++++++--------- 2 files changed, 104 insertions(+), 113 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 14f68c028126..910c50956e16 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -38,10 +38,10 @@ extern uint64_t amdgpu_amdkfd_total_mem_size; struct amdgpu_device; -struct kfd_bo_va_list { - struct list_head bo_list; +struct kfd_mem_attachment { + struct list_head list; struct amdgpu_bo_va *bo_va; - void *kgd_dev; + struct amdgpu_device *adev; bool is_mapped; uint64_t va; uint64_t pte_flags; @@ -50,7 +50,7 @@ struct kfd_bo_va_list { struct kgd_mem { struct mutex lock; struct amdgpu_bo *bo; - struct list_head bo_va_list; + struct list_head attachments; /* protected by amdkfd_process_info.lock */ struct ttm_validate_buffer validate_list; struct ttm_validate_buffer resv_list; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 6622695a5eed..d021152314ad 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -75,16 +75,16 @@ static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) return (struct amdgpu_device *)kgd; } -static bool check_if_add_bo_to_vm(struct amdgpu_vm *avm, +static bool kfd_mem_is_attached(struct amdgpu_vm *avm, struct kgd_mem *mem) { - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; - list_for_each_entry(entry, &mem->bo_va_list, bo_list) + list_for_each_entry(entry, &mem->attachments, list) if (entry->bo_va->base.vm == avm) - return false; + return true; - return true; + return false; } /* Set memory usage limits. Current, limits are @@ -471,7 +471,7 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) return pte_flags; } -/* add_bo_to_vm - Add a BO to a VM +/* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added * to a VM. It can later be mapped and unmapped many times without @@ -483,15 +483,14 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) * 4. Alloc page tables and directories if needed * 4a. Validate new page tables and directories */ -static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, +static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_vm *vm, bool is_aql, - struct kfd_bo_va_list **p_bo_va_entry) + struct kfd_mem_attachment **p_attachment) { int ret; - struct kfd_bo_va_list *bo_va_entry; + struct kfd_mem_attachment *attachment; struct amdgpu_bo *bo = mem->bo; uint64_t va = mem->va; - struct list_head *list_bo_va = &mem->bo_va_list; unsigned long bo_size = bo->tbo.base.size; if (!va) { @@ -502,29 +501,29 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, if (is_aql) va += bo_size; - bo_va_entry = kzalloc(sizeof(*bo_va_entry), GFP_KERNEL); - if (!bo_va_entry) + attachment = kzalloc(sizeof(*attachment), GFP_KERNEL); + if (!attachment) return -ENOMEM; pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, va + bo_size, vm); /* Add BO to VM internal data structures*/ - bo_va_entry->bo_va = amdgpu_vm_bo_add(adev, vm, bo); - if (!bo_va_entry->bo_va) { + attachment->bo_va = amdgpu_vm_bo_add(adev, vm, bo); + if (!attachment->bo_va) { ret = -EINVAL; pr_err("Failed to add BO object to VM. ret == %d\n", ret); goto err_vmadd; } - bo_va_entry->va = va; - bo_va_entry->pte_flags = get_pte_flags(adev, mem); - bo_va_entry->kgd_dev = (void *)adev; - list_add(&bo_va_entry->bo_list, list_bo_va); + attachment->va = va; + attachment->pte_flags = get_pte_flags(adev, mem); + attachment->adev = adev; + list_add(&attachment->list, &mem->attachments); - if (p_bo_va_entry) - *p_bo_va_entry = bo_va_entry; + if (p_attachment) + *p_attachment = attachment; /* Allocate validate page tables if needed */ ret = vm_validate_pt_pd_bos(vm); @@ -536,22 +535,20 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, return 0; err_alloc_pts: - amdgpu_vm_bo_rmv(adev, bo_va_entry->bo_va); - list_del(&bo_va_entry->bo_list); + amdgpu_vm_bo_rmv(adev, attachment->bo_va); + list_del(&attachment->list); err_vmadd: - kfree(bo_va_entry); + kfree(attachment); return ret; } -static void remove_bo_from_vm(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, unsigned long size) +static void kfd_mem_detach(struct kfd_mem_attachment *attachment) { - pr_debug("\t remove VA 0x%llx - 0x%llx in entry %p\n", - entry->va, - entry->va + size, entry); - amdgpu_vm_bo_rmv(adev, entry->bo_va); - list_del(&entry->bo_list); - kfree(entry); + pr_debug("\t remove VA 0x%llx in entry %p\n", + attachment->va, attachment); + amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); + list_del(&attachment->list); + kfree(attachment); } static void add_kgd_mem_to_kfd_bo_list(struct kgd_mem *mem, @@ -726,7 +723,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, struct bo_vm_reservation_context *ctx) { struct amdgpu_bo *bo = mem->bo; - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; unsigned int i; int ret; @@ -738,7 +735,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, INIT_LIST_HEAD(&ctx->list); INIT_LIST_HEAD(&ctx->duplicates); - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { + list_for_each_entry(entry, &mem->attachments, list) { if ((vm && vm != entry->bo_va->base.vm) || (entry->is_mapped != map_type && map_type != BO_VM_ALL)) @@ -760,7 +757,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, list_add(&ctx->kfd_bo.tv.head, &ctx->list); i = 0; - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { + list_for_each_entry(entry, &mem->attachments, list) { if ((vm && vm != entry->bo_va->base.vm) || (entry->is_mapped != map_type && map_type != BO_VM_ALL)) @@ -815,7 +812,7 @@ static int unreserve_bo_and_vms(struct bo_vm_reservation_context *ctx, } static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, + struct kfd_mem_attachment *entry, struct amdgpu_sync *sync) { struct amdgpu_bo_va *bo_va = entry->bo_va; @@ -831,7 +828,7 @@ static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, } static int update_gpuvm_pte(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, + struct kfd_mem_attachment *entry, struct amdgpu_sync *sync) { int ret; @@ -848,7 +845,7 @@ static int update_gpuvm_pte(struct amdgpu_device *adev, } static int map_bo_to_gpuvm(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, struct amdgpu_sync *sync, + struct kfd_mem_attachment *entry, struct amdgpu_sync *sync, bool no_update_pte) { int ret; @@ -1235,7 +1232,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( ret = -ENOMEM; goto err; } - INIT_LIST_HEAD(&(*mem)->bo_va_list); + INIT_LIST_HEAD(&(*mem)->attachments); mutex_init(&(*mem)->lock); (*mem)->aql_queue = !!(flags & KFD_IOC_ALLOC_MEM_FLAGS_AQL_QUEUE_MEM); @@ -1316,7 +1313,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( { struct amdkfd_process_info *process_info = mem->process_info; unsigned long bo_size = mem->bo->tbo.base.size; - struct kfd_bo_va_list *entry, *tmp; + struct kfd_mem_attachment *entry, *tmp; struct bo_vm_reservation_context ctx; struct ttm_validate_buffer *bo_list_entry; unsigned int mapped_to_gpu_memory; @@ -1360,9 +1357,8 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( mem->va + bo_size * (1 + mem->aql_queue)); /* Remove from VM internal data structures */ - list_for_each_entry_safe(entry, tmp, &mem->bo_va_list, bo_list) - remove_bo_from_vm((struct amdgpu_device *)entry->kgd_dev, - entry, bo_size); + list_for_each_entry_safe(entry, tmp, &mem->attachments, list) + kfd_mem_detach(entry); ret = unreserve_bo_and_vms(&ctx, false, false); @@ -1404,10 +1400,10 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( int ret; struct amdgpu_bo *bo; uint32_t domain; - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; - struct kfd_bo_va_list *bo_va_entry = NULL; - struct kfd_bo_va_list *bo_va_entry_aql = NULL; + struct kfd_mem_attachment *attachment = NULL; + struct kfd_mem_attachment *attachment_aql = NULL; unsigned long bo_size; bool is_invalid_userptr = false; @@ -1456,21 +1452,20 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( bo->tbo.mem.mem_type == TTM_PL_SYSTEM) is_invalid_userptr = true; - if (check_if_add_bo_to_vm(avm, mem)) { - ret = add_bo_to_vm(adev, mem, avm, false, - &bo_va_entry); + if (!kfd_mem_is_attached(avm, mem)) { + ret = kfd_mem_attach(adev, mem, avm, false, &attachment); if (ret) - goto add_bo_to_vm_failed; + goto attach_failed; if (mem->aql_queue) { - ret = add_bo_to_vm(adev, mem, avm, - true, &bo_va_entry_aql); + ret = kfd_mem_attach(adev, mem, avm, true, + &attachment_aql); if (ret) - goto add_bo_to_vm_failed_aql; + goto attach_failed_aql; } } else { ret = vm_validate_pt_pd_bos(avm); if (unlikely(ret)) - goto add_bo_to_vm_failed; + goto attach_failed; } if (mem->mapped_to_gpu_memory == 0 && @@ -1486,30 +1481,30 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( } } - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { - if (entry->bo_va->base.vm == vm && !entry->is_mapped) { - pr_debug("\t map VA 0x%llx - 0x%llx in entry %p\n", - entry->va, entry->va + bo_size, - entry); + list_for_each_entry(entry, &mem->attachments, list) { + if (entry->bo_va->base.vm != vm || entry->is_mapped) + continue; - ret = map_bo_to_gpuvm(adev, entry, ctx.sync, - is_invalid_userptr); - if (ret) { - pr_err("Failed to map bo to gpuvm\n"); - goto map_bo_to_gpuvm_failed; - } + pr_debug("\t map VA 0x%llx - 0x%llx in entry %p\n", + entry->va, entry->va + bo_size, entry); - ret = vm_update_pds(vm, ctx.sync); - if (ret) { - pr_err("Failed to update page directories\n"); - goto map_bo_to_gpuvm_failed; - } + ret = map_bo_to_gpuvm(adev, entry, ctx.sync, + is_invalid_userptr); + if (ret) { + pr_err("Failed to map bo to gpuvm\n"); + goto map_bo_to_gpuvm_failed; + } - entry->is_mapped = true; - mem->mapped_to_gpu_memory++; - pr_debug("\t INC mapping count %d\n", - mem->mapped_to_gpu_memory); + ret = vm_update_pds(vm, ctx.sync); + if (ret) { + pr_err("Failed to update page directories\n"); + goto map_bo_to_gpuvm_failed; } + + entry->is_mapped = true; + mem->mapped_to_gpu_memory++; + pr_debug("\t INC mapping count %d\n", + mem->mapped_to_gpu_memory); } if (!amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) && !bo->tbo.pin_count) @@ -1521,12 +1516,12 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( goto out; map_bo_to_gpuvm_failed: - if (bo_va_entry_aql) - remove_bo_from_vm(adev, bo_va_entry_aql, bo_size); -add_bo_to_vm_failed_aql: - if (bo_va_entry) - remove_bo_from_vm(adev, bo_va_entry, bo_size); -add_bo_to_vm_failed: + if (attachment_aql) + kfd_mem_detach(attachment_aql); +attach_failed_aql: + if (attachment) + kfd_mem_detach(attachment); +attach_failed: unreserve_bo_and_vms(&ctx, false, false); out: mutex_unlock(&mem->process_info->lock); @@ -1541,7 +1536,7 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( struct amdkfd_process_info *process_info = ((struct amdgpu_vm *)vm)->process_info; unsigned long bo_size = mem->bo->tbo.base.size; - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; int ret; @@ -1565,26 +1560,24 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( mem->va + bo_size * (1 + mem->aql_queue), vm); - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { - if (entry->bo_va->base.vm == vm && entry->is_mapped) { - pr_debug("\t unmap VA 0x%llx - 0x%llx from entry %p\n", - entry->va, - entry->va + bo_size, - entry); + list_for_each_entry(entry, &mem->attachments, list) { + if (entry->bo_va->base.vm != vm || !entry->is_mapped) + continue; - ret = unmap_bo_from_gpuvm(adev, entry, ctx.sync); - if (ret == 0) { - entry->is_mapped = false; - } else { - pr_err("failed to unmap VA 0x%llx\n", - mem->va); - goto unreserve_out; - } + pr_debug("\t unmap VA 0x%llx - 0x%llx from entry %p\n", + entry->va, entry->va + bo_size, entry); - mem->mapped_to_gpu_memory--; - pr_debug("\t DEC mapping count %d\n", - mem->mapped_to_gpu_memory); + ret = unmap_bo_from_gpuvm(adev, entry, ctx.sync); + if (ret == 0) { + entry->is_mapped = false; + } else { + pr_err("failed to unmap VA 0x%llx\n", mem->va); + goto unreserve_out; } + + mem->mapped_to_gpu_memory--; + pr_debug("\t DEC mapping count %d\n", + mem->mapped_to_gpu_memory); } /* If BO is unmapped from all VMs, unfence it. It can be evicted if @@ -1726,7 +1719,7 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct kgd_dev *kgd, if (mmap_offset) *mmap_offset = amdgpu_bo_mmap_offset(bo); - INIT_LIST_HEAD(&(*mem)->bo_va_list); + INIT_LIST_HEAD(&(*mem)->attachments); mutex_init(&(*mem)->lock); (*mem)->alloc_flags = @@ -1923,7 +1916,7 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) list_for_each_entry_safe(mem, tmp_mem, &process_info->userptr_inval_list, validate_list.head) { - struct kfd_bo_va_list *bo_va_entry; + struct kfd_mem_attachment *attachment; bo = mem->bo; @@ -1946,13 +1939,13 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) * VM faults if the GPU tries to access the invalid * memory. */ - list_for_each_entry(bo_va_entry, &mem->bo_va_list, bo_list) { - if (!bo_va_entry->is_mapped) + list_for_each_entry(attachment, &mem->attachments, list) { + if (!attachment->is_mapped) continue; ret = update_gpuvm_pte((struct amdgpu_device *) - bo_va_entry->kgd_dev, - bo_va_entry, &sync); + attachment->adev, + attachment, &sync); if (ret) { pr_err("%s: update PTE failed\n", __func__); /* make sure this gets validated again */ @@ -2133,7 +2126,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) struct amdgpu_bo *bo = mem->bo; uint32_t domain = mem->domain; - struct kfd_bo_va_list *bo_va_entry; + struct kfd_mem_attachment *attachment; total_size += amdgpu_bo_size(bo); @@ -2153,11 +2146,9 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) pr_debug("Memory eviction: Sync BO fence failed. Try again\n"); goto validate_map_fail; } - list_for_each_entry(bo_va_entry, &mem->bo_va_list, - bo_list) { + list_for_each_entry(attachment, &mem->attachments, list) { ret = update_gpuvm_pte((struct amdgpu_device *) - bo_va_entry->kgd_dev, - bo_va_entry, + attachment->adev, attachment, &sync_obj); if (ret) { pr_debug("Memory eviction: update PTE failed. Try again\n"); @@ -2232,7 +2223,7 @@ int amdgpu_amdkfd_add_gws_to_process(void *info, void *gws, struct kgd_mem **mem return -ENOMEM; mutex_init(&(*mem)->lock); - INIT_LIST_HEAD(&(*mem)->bo_va_list); + INIT_LIST_HEAD(&(*mem)->attachments); (*mem)->bo = amdgpu_bo_ref(gws_bo); (*mem)->domain = AMDGPU_GEM_DOMAIN_GWS; (*mem)->process_info = process_info; From patchwork Wed Apr 14 06:47:57 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201999 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 52593C433B4 for ; Wed, 14 Apr 2021 06:48:26 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 0FB68611F2 for ; Wed, 14 Apr 2021 06:48:26 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 0FB68611F2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 89E256E8D3; Wed, 14 Apr 2021 06:48:22 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2079.outbound.protection.outlook.com [40.107.92.79]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4BAB86E8CD; Wed, 14 Apr 2021 06:48:21 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jRzLj/0fDX6IWbd3TlZp3Y701jVBl2vdbmOhnYDX+0PvHzHG7ZqKyPkWUNyh6oyx9ksQvPBmkQPm14UPx90rNbfQg/ifjZGV2VEoAzhn2hDlP1X8MiPwChA3oV5BYkAYar8oxzah/vxPXEw+R+sJPBblitTS9cVsFBAckWErbjZzGqHFMmYisyViBem7L4mhxSKCSBUSEAVfpM4m60Mudk5RjLJzbVGN4z/9JW25MBBww3/4lHWe13HHGMipqvUnS0MHOlh0d9H/jcfxg7R77Fnt8BUn9n2qMrk3al930xxKxWy8/X3c2PKBfAhdo+j6zpXrL6CIGXmMOCtozXdngw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XUcSThGOmgW2TkEvH03AiiupJoZbHEnbE0vI9a2WclU=; b=RfP9xMNiC8ab92iF4Ri14pzs3utzICLdD69FHGqZeJdF3ktIEfgORUJDBfiK9nhmDyTUEeJUZj+WLS+Gxr7IGyrMmxFxXroFKss+eeG4hfgpTSmuxEOqAAalhQmSuGmbkIjYLt/mbhs1K4FhdfzFx4xjUiB3LrI+Smq4So6U7eM7aW1gzIwt6FZxAwUoHUyYSh8P2m8swaltH1/xFiMvap+/GRKXmGaH/HONfzO0YSjg6mQSotwlsM3rGB8khnTWBfR8vcuEn/xHI9maGUX7sTT8A9AENZqGKWxgMeenWD6v7IAUVZMXCvN8nxk+4ULwEVa4GhsF3Wpo2ooQD5kiOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XUcSThGOmgW2TkEvH03AiiupJoZbHEnbE0vI9a2WclU=; b=RlR9cNODKLp2n9UOVLoySHyavTvDepJxs3Ahn7ebfaf+xrFm6aZ9wd10EK/xghPIffa8bV/+25tGjur3TK9bFFEFcjX3580EzQ00FzxwZC20UkRkeZ9jiZZ3G/0HI/6S3zd/7lE82xah2TjGZ0V3nQAKcPROZVHy0GYZCFHr23k= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:48:20 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:48:19 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 2/9] drm/amdgpu: Keep a bo-reference per-attachment Date: Wed, 14 Apr 2021 02:47:57 -0400 Message-Id: <20210414064804.29356-3-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064804.29356-1-Felix.Kuehling@amd.com> References: <20210414064804.29356-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Wed, 14 Apr 2021 06:48:18 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: ff9beb1f-7af3-4d1f-ca58-08d8ff1149ba X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1XtAJtyIKELNbyPgjgmh9Y4NfSG0sJvenM9ODBqelGVCvoGM6vO6JpdKPA5j4jql0Yz8YJxRQw/1hE20Np0l8CT2suu6frEs4PjRVG96alW31r7uN4QFZ3HmGtu4ks6prDF/PBXcX/9ydgHGGjEyfZI987fYtkb+hFuU/V83iZUnFb2GFZV6Qe/dzc8anfLgaP6sDrnNFCt55O9m9rYxA2X5g+c9/RJO3ZvY80AAyS58WNHX/LK/p3ByMmZJUSH4JfkkG531bcqJ7xdlOu1PARQS8rv3x8c4n7k6DFPmPsa6nZxzcMZPdEkjWpytDfGSWCmOzvw1JDnal5o9LQQ/MTwOUtWmXFcrqYV8/UhBz21wxupX7l1RkRQe8rj+C7uACNh6EMboS9q9W5YTd79PAId36jM6ZWcPAkiNBmKcUViyhoc/S4U6adC2xiNrbaoIbkfjyTI3UNSTVaMjDticUVVpv0baPu/jag4oT47k5LiYH6/KyDoBunmOcJcnuByTts03ZH5nuITf2dXJn6zqP0kC+fBJ2oFfhF6FbOsjTwc3yNXKbukg7rnsC9InviCRZXa9yjMCOoKvyjY9CiVSJwA1+252bSY/8I2O2LL+jcYM6uuI7xlUcRm9N4sCMWPuPJLOmgp0fbjG4H4WvBdF+zfiDnCMNue9PchG94SGNeE= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(8936002)(86362001)(52116002)(38350700002)(66556008)(7696005)(6486002)(2906002)(1076003)(38100700002)(2616005)(36756003)(956004)(66476007)(26005)(316002)(478600001)(6666004)(450100002)(5660300002)(66946007)(4326008)(83380400001)(8676002)(186003)(16526019); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: SgMi5TUUxXNb7XcKdQvVgY15NeRDYRVgdY6ZjOIg8jI34AzAWewnnjgkQrdY9Aq4APQT9mfj+RsmGGLa9jKioI02siXbfmsw4kFdd/VJtq/ggpAhTFZvAl1OP5UwsuFiE4PAHgt8tEbFU92oQ5kms+ZA0sBSfDAGQ/HaWy6CKLHOgSwn7JRe2MNvSRdRG3Fq9OakSdF6IcGkWPDkgcLGxaLqB0Mgh+wRZ/9z3ubJhC4qsa5SrS5eLuFI9UVT0lknBoR5jXtvMrsuc3ZimWpMGJsrUOnoeL9fX96siaGdrZpYLSZ7GmaJbVatXuCnMDUe9XP2CdLJHxz/x73wG7rtDh5r++TPPINmII8qMA1Aio8uydKkJ+SDUKjRhZTzA1F7r36wt8hTFygvS2Wbwv3v74UBPxY+jIfm+5475f+KXAJVVoqoR4WmhFQ021lYdRiklPG5i2BALNbLBQG2xWrnpq/2QvdK22T7lDAKJ23yLSNe/Ljbhc2b9pjwbqcGO4TgaSo/m6GmnqvguwHJ/3Yetb5FIjQU6Rst+cHX49MSrp76SX62fvoA1AXK/u7EX3snZDk6VOhe+O0V//1MfKR2t41lZjMazxvBxFSqgQYuBdFTD5llwKVtYwvJosnUcj2+O/Ly8PZEIDTXh7gcop/jyiOjnARAk4XBvGF4HPYv+oe6OKB7fTxVUePkGpyiqD07Zp5LweAPHwa662np+aPU0gY5GDRndDJhHYl/StCPTd660xTRXMvbGdbjB+ttnSsb6WuN5UTT+b/obdLNhyNX8CPkE68adNpT2XTDuStDGwnPmt5jh1rG42d2HUl8xIXFQqzg+PxqV6DMaDJ3/gtuTUHzJ0VUGrvqtOU/0N/aDGYzAuCpvGZQGZFXKU5qsOQhH8iGfq86Xy2ohcG+7nOLSoe5ZN8TM1r3z5iCZSkIIMjps4YFgVM6BlQjY3i43hpQgyVmHZcXLDx2enn/AE4mQHluGSFvhZAJJLHhEhCJjgqs2oIRUsjL/2gOt7Zz7w9OqXlpGkE6sj6eAABpLuK9D2VIYcGwXDQrEtwoMXcpV9Ho58Ns1MlJM9CTLeIdAf4WflQhc8hWio71Nyi89bkdO09N8ALd0KqqOjgKTk9XG+xjc9xV0TcX81IZTjkBPMI4vLrTPebIKUnzkQnL5O7Z7Rs6wT4zpkMON/CYEonCvQj26J3+h/KRmpCG2cFVUkF5cYxKFmPaMYKTvjJ9CT2GEtkhzH3POrMbU5UIczCsGPK9QC8wawZOp2loUve/xnFHRmFOMCAh3RYeenNByAlqLnoibkl5iKebT96IfFjT4QaOSTTg1tlHcAbzNQ1dAp62 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: ff9beb1f-7af3-4d1f-ca58-08d8ff1149ba X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:48:19.1401 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: TJRSruS/awX2YWuxnAKewM3WQfOtqzChRWo3EftBc2DeXURn5l2qZLriHQXXfzvUI5rKhEWh0a8daC5J/whTgw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" For now they all reference the same BO. For correct DMA mappings they will refer to different BOs per-GPU. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index d021152314ad..40a296ca37b9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -487,11 +487,11 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_vm *vm, bool is_aql, struct kfd_mem_attachment **p_attachment) { - int ret; - struct kfd_mem_attachment *attachment; - struct amdgpu_bo *bo = mem->bo; + unsigned long bo_size = mem->bo->tbo.base.size; uint64_t va = mem->va; - unsigned long bo_size = bo->tbo.base.size; + struct kfd_mem_attachment *attachment; + struct amdgpu_bo *bo; + int ret; if (!va) { pr_err("Invalid VA when adding BO to VM\n"); @@ -508,6 +508,14 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, va + bo_size, vm); + /* FIXME: For now all attachments use the same BO. This is incorrect + * because one BO can only have one DMA mapping for one GPU. We need + * one BO per GPU, e.g. a DMABuf import with dynamic attachment. This + * will be addressed one BO-type at a time in subsequent patches. + */ + bo = mem->bo; + drm_gem_object_get(&bo->tbo.base); + /* Add BO to VM internal data structures*/ attachment->bo_va = amdgpu_vm_bo_add(adev, vm, bo); if (!attachment->bo_va) { @@ -527,7 +535,7 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, /* Allocate validate page tables if needed */ ret = vm_validate_pt_pd_bos(vm); - if (ret) { + if (unlikely(ret)) { pr_err("validate_pt_pd_bos() failed\n"); goto err_alloc_pts; } @@ -538,15 +546,19 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, amdgpu_vm_bo_rmv(adev, attachment->bo_va); list_del(&attachment->list); err_vmadd: + drm_gem_object_put(&bo->tbo.base); kfree(attachment); return ret; } static void kfd_mem_detach(struct kfd_mem_attachment *attachment) { + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + pr_debug("\t remove VA 0x%llx in entry %p\n", attachment->va, attachment); amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); + drm_gem_object_put(&bo->tbo.base); list_del(&attachment->list); kfree(attachment); } From patchwork Wed Apr 14 06:47:58 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12202007 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CB218C43460 for ; Wed, 14 Apr 2021 06:48:33 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 912A56120E for ; Wed, 14 Apr 2021 06:48:33 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 912A56120E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 991076E8E5; Wed, 14 Apr 2021 06:48:28 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2079.outbound.protection.outlook.com [40.107.92.79]) by gabe.freedesktop.org (Postfix) with ESMTPS id 753A26E8D0; Wed, 14 Apr 2021 06:48:21 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=X6LNx7KrquNOsEfcC410Mv4/n7Cz/eJ0mGlZsQPIBsGDSI46T16l3/d8ITWUR9xIbiS9aEgN3JLrhDx6u1lZsdMFdpGezC5xn9aB9xStkoM+l1CcpM81Pgtr8va5yU/jHgLWCZvMLC3wBSNyP7+w0BI1kNCXZsPYvS9Zr0/dRChEgnmvHxx4aHUnBXa9CzzdFdnZYZDp/jSBpyhmb3131OyKGYujt9txFrpOU1msi/b/0bR9jAyiSC45nd2O/n26oonmaKtxpfsL4bMA/D86KlKiGPwizc60a+VGxI8OuRDrppF9vb6haOmZIw6EdhRMQGsSWGWfh+hV/DUCSA40yQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J4f6qiDMF3EvXZF3caFnVdL4tzzrONpDOHzq7h2Z3dM=; b=Kb6XsiUcE+9ugkP1TvuG9VG4h2eKRosHEIIMQEfWScaskUGsJMTs4hI4v41nB0PW/Ipk7vxhju1sLA8tfwYyDyq8HPj3LjmwD7krRXON51adDlIUPXH43qHmNV61upt+Ll/DHaYpW4LI+gPzJHLXaegfFJCQuq2GRLFhk8DuGfiMPJHoHbZX/uWMa5oWooH4UblezeuPqChk74HN0lKB+OQNMnXQWy5k8yuSrbxQUAn7rM1c4C5qV9h42/Nq+rgMm51CyR2tlAn+zYhEW2fzR/9+C76TpBpt+c88ddOkeSoz36tJwQXm7l2lHbVyBZXnd+mtaj2N3AQ2Roinds1Vwg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J4f6qiDMF3EvXZF3caFnVdL4tzzrONpDOHzq7h2Z3dM=; b=RJUYOFxHNOlNYxz2YyShWG27lT+1UIxBPFN71o4pl2Zj2guuLP89ftMj8MECDOBEF5PdXlnazSMw+ISdUJyUGxWiCdOhpBD4bimmUF42b8ag1IutVGfV3GbcZzQEaaCIEh5xOK9tnWVFmJrOjBwjrLHNRXMD0MnTfPsis3lXpfc= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:48:20 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:48:20 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 3/9] drm/amdgpu: Simplify AQL queue mapping Date: Wed, 14 Apr 2021 02:47:58 -0400 Message-Id: <20210414064804.29356-4-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064804.29356-1-Felix.Kuehling@amd.com> References: <20210414064804.29356-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Wed, 14 Apr 2021 06:48:19 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 40e2b926-03a3-42cf-6f10-08d8ff114a01 X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: EIA3pJ5uKKflpxayyStuISReRdeQ9sBI2Ziv+kMRRtFWuR6iChUoIVamqCfVURwUY0zdz1CUqljHsuvEIxHZLiGWMwroreirx5/RPtM6B7PDfMpmqUDv4TaFSIugThtNThtVGybYjP+TrclqE+2DlLrT8aaCRw1QWHARi0AzpAxx8kZYdQEglFJtnabSudfV8+RCVIluuENzbQ1HExII0caaECuxu4UBDN9Qjq1IyJomgl1E5MLwbNz+SNoB+6S3/ImVFT0Z4AiCxYVxqepxnVBUAq5gK4fK7UPVB1PfB4KdWukqGpM5O/Tz5iCAjmFu8bnzQrqTwIAzWEmaYvjaM+az3cyjMo3nSJ4lp7QRZiFDOwWJuiOJmZe3pTMFd0IDEs5xHyHMAjpnOy72McIjUte2g47IkZlD3jOA4+eEz1XimcUF4GyQo+Pj6nhu0oFHH1vJzzth7vv/5ShRONAR1Wi2zFQwpH4dxTypXSva/ANEMT0GVR29+rQ/s8LHNimROYzgtEtCYbQ1S4IFvjCDXlDWse0qfiQkAsCoib4AlyY5+PihbQYGp/Y5UesZGr4y+JYn0av9xIrxB077Pq5p/O/UTnfTdpec/zhTl3LoissevMFmbqd4kzW1IymxZkMxqIl4cGVaEmVMlZqXUtUK5M1ZsoTmD+4wFfoJEtPducc= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(8936002)(86362001)(52116002)(38350700002)(66556008)(7696005)(6486002)(2906002)(1076003)(38100700002)(2616005)(36756003)(956004)(66476007)(26005)(316002)(478600001)(6666004)(450100002)(5660300002)(66946007)(4326008)(83380400001)(8676002)(186003)(16526019); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: PspmwOYmyqYs0vH0f8OQFWsj6pELUjE2lcrt+qy9FfZRKd+AHbf3U8aE/l58xaVyhw+rh6eFiDsIsB8qt0BYM7GWtTlEmdzYa8WRi3j1A7ueSnh116rQQZScxG51PiS/ZUWwU1TmYXFKosEHynR9PN6SkyiPvFq48bIvEFWT1I3NIoSls/cU8Mf7MbTKdRMPnZJfxVoQuGq1HIoTy0wvQamIoO0uPdv5DLmNc9RtHKkYI17o3O+vGdwjVdcj/xMqm4j8yNBy+c11wjY315JRsgIllpnKvBFfu16vSNxzWNNv+UbZ8J3VAfFpaL5mIQJicJfj9edXo5Kwce4Dq1qW0OH2PvzGbW3J0717CuhcFk3wNmZdznjfF+IyYRRDgsfZ17btHDZaDIVeVPTX5/wDj/4i0t2DYdyWKSkMC1QrRBP8ycG885eIBWfhjIjUm62PlmhZO5sBwWwN9+nbAMTMPRHAUDq6q1hAcyG8fjOv5/TYeTjWIOXe9QFGHyxPzR5bQ5Cs1R2qdSJU5/iVKb425WgDRcvYS7MtMtHPDOpOX7UHlu6qjPX2Z1tSbLM6co/ek09vxyRqQXX66e5Erqop4Jj2z3t/jCfp6xlId0OdUMyxLrlgzCCvbVLsvnG7e9Z0lKLQ+fg3Fz6kvRoA5PnwyLcIlvXO8VssNVOWbMWayjbgYUJHm7AAnBKYoX5PVBcAB4c7SnWE/YWor6wAwU3e1ibRjhkjXKHTjg8qDr15r7suKRes7CM9KxjvjPYWhrrGfGzTzNPYgqwlhCBjXA6zBMYzhnPNBbgxH+3dM74bo1J4WdfMbqHgfj98kqWsHMbYptVmFwg1swJjruVv8/csnJHPDqGSclCqZb3Sd8FdswMc3DQh/dKaZQ5nNJKb7iekOvbMNar/Xp+Y6HpXBqcsR0JFxRUe36V2qupVyYhFNkNlBrCRP5UtPbRniVuX4x9sVnC7oCeDOG8cpcSwpeE+rAzp73Q1hTD3dG6lfahGDLqu9Sx1fXYPB90nZmCad4/Q5V4I7WZ2fPYCaxhbB0p117aYI8/6RiQGoSIKxWraKimyF2W+YOvTjzCbOJcVWT1AiFa1t2A5DyMgYc9Y9F6WbJnTPxWSjilqGQRCggJTQNMSCxhqLgFv/j63EUpZjX4Mj4uLmWVy9Y/dyk9jt0DjqyyPFBAFwBqMBdDMpifVjp7WCoLvmFZm7DTT6EsN/3fVRUSkATFugWXbCTLSeI82AYaSqBMiNPIlO9+h3RmF1bZCD8Cah6dYIj2Wo5yCE9tE58p/s+XsZGVHwVidippmupCYTg71ny+QJlmAQbKcwQuVzg9phYtSTwS0gDwKggDO X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 40e2b926-03a3-42cf-6f10-08d8ff114a01 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:48:19.5533 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 0+dRdSQ7ZO/rqYYFWVaHWdDXZ2Ge20bs7qcznj3x8ZtUdQp+IlMAOnZOcGqYfPOxjWTBLfO0cyeFyT+opcuNKA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Do AQL queue double-mapping with a single attach call. That will make it easier to create per-GPU BOs later, to be shared between the two BO VA mappings on the same GPU. Freeing the attachments is not necessary if map_to_gpu fails. These will be cleaned up when the kdg_mem object is destroyed in amdgpu_amdkfd_gpuvm_free_memory_of_gpu. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 103 ++++++++---------- 1 file changed, 48 insertions(+), 55 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 40a296ca37b9..114fbf508707 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -484,70 +484,76 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) * 4a. Validate new page tables and directories */ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, - struct amdgpu_vm *vm, bool is_aql, - struct kfd_mem_attachment **p_attachment) + struct amdgpu_vm *vm, bool is_aql) { unsigned long bo_size = mem->bo->tbo.base.size; uint64_t va = mem->va; - struct kfd_mem_attachment *attachment; - struct amdgpu_bo *bo; - int ret; + struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; + struct amdgpu_bo *bo[2] = {NULL, NULL}; + int i, ret; if (!va) { pr_err("Invalid VA when adding BO to VM\n"); return -EINVAL; } - if (is_aql) - va += bo_size; - - attachment = kzalloc(sizeof(*attachment), GFP_KERNEL); - if (!attachment) - return -ENOMEM; + for (i = 0; i <= is_aql; i++) { + attachment[i] = kzalloc(sizeof(*attachment[i]), GFP_KERNEL); + if (unlikely(!attachment[i])) { + ret = -ENOMEM; + goto unwind; + } - pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, - va + bo_size, vm); + pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, + va + bo_size, vm); - /* FIXME: For now all attachments use the same BO. This is incorrect - * because one BO can only have one DMA mapping for one GPU. We need - * one BO per GPU, e.g. a DMABuf import with dynamic attachment. This - * will be addressed one BO-type at a time in subsequent patches. - */ - bo = mem->bo; - drm_gem_object_get(&bo->tbo.base); + /* FIXME: For now all attachments use the same BO. This is + * incorrect because one BO can only have one DMA mapping + * for one GPU. We need one BO per GPU, e.g. a DMABuf + * import with dynamic attachment. This will be addressed + * one BO-type at a time in subsequent patches. + */ + bo[i] = mem->bo; + drm_gem_object_get(&bo[i]->tbo.base); - /* Add BO to VM internal data structures*/ - attachment->bo_va = amdgpu_vm_bo_add(adev, vm, bo); - if (!attachment->bo_va) { - ret = -EINVAL; - pr_err("Failed to add BO object to VM. ret == %d\n", - ret); - goto err_vmadd; - } + /* Add BO to VM internal data structures */ + attachment[i]->bo_va = amdgpu_vm_bo_add(adev, vm, bo[i]); + if (unlikely(!attachment[i]->bo_va)) { + ret = -ENOMEM; + pr_err("Failed to add BO object to VM. ret == %d\n", + ret); + goto unwind; + } - attachment->va = va; - attachment->pte_flags = get_pte_flags(adev, mem); - attachment->adev = adev; - list_add(&attachment->list, &mem->attachments); + attachment[i]->va = va; + attachment[i]->pte_flags = get_pte_flags(adev, mem); + attachment[i]->adev = adev; + list_add(&attachment[i]->list, &mem->attachments); - if (p_attachment) - *p_attachment = attachment; + va += bo_size; + } /* Allocate validate page tables if needed */ ret = vm_validate_pt_pd_bos(vm); if (unlikely(ret)) { pr_err("validate_pt_pd_bos() failed\n"); - goto err_alloc_pts; + goto unwind; } return 0; -err_alloc_pts: - amdgpu_vm_bo_rmv(adev, attachment->bo_va); - list_del(&attachment->list); -err_vmadd: - drm_gem_object_put(&bo->tbo.base); - kfree(attachment); +unwind: + for (; i >= 0; i--) { + if (!attachment[i]) + continue; + if (attachment[i]->bo_va) { + amdgpu_vm_bo_rmv(adev, attachment[i]->bo_va); + list_del(&attachment[i]->list); + } + if (bo[i]) + drm_gem_object_put(&bo[i]->tbo.base); + kfree(attachment[i]); + } return ret; } @@ -1414,8 +1420,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( uint32_t domain; struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; - struct kfd_mem_attachment *attachment = NULL; - struct kfd_mem_attachment *attachment_aql = NULL; unsigned long bo_size; bool is_invalid_userptr = false; @@ -1465,15 +1469,9 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( is_invalid_userptr = true; if (!kfd_mem_is_attached(avm, mem)) { - ret = kfd_mem_attach(adev, mem, avm, false, &attachment); + ret = kfd_mem_attach(adev, mem, avm, mem->aql_queue); if (ret) goto attach_failed; - if (mem->aql_queue) { - ret = kfd_mem_attach(adev, mem, avm, true, - &attachment_aql); - if (ret) - goto attach_failed_aql; - } } else { ret = vm_validate_pt_pd_bos(avm); if (unlikely(ret)) @@ -1528,11 +1526,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( goto out; map_bo_to_gpuvm_failed: - if (attachment_aql) - kfd_mem_detach(attachment_aql); -attach_failed_aql: - if (attachment) - kfd_mem_detach(attachment); attach_failed: unreserve_bo_and_vms(&ctx, false, false); out: From patchwork Wed Apr 14 06:47:59 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12202013 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C4F7EC433ED for ; Wed, 14 Apr 2021 06:48:38 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 86D2D6101B for ; Wed, 14 Apr 2021 06:48:38 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 86D2D6101B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5C5CB6E8DA; Wed, 14 Apr 2021 06:48:33 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2079.outbound.protection.outlook.com [40.107.92.79]) by gabe.freedesktop.org (Postfix) with ESMTPS id A347D6E8CD; Wed, 14 Apr 2021 06:48:21 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AZ5SFqyIM+CQk43f+ZIP2r0tvb8WqggOrOr7d+yao2IgN37yHcMBL+4GvpcrLRMdw5ieKMwan2hMyweel+VaeJhoQifEPpElFcZoOBoYvaAkE32yA0FxB+s5ILzp/vzz680HajgAY/KVyTC9H6/+jsI5y0zjGgOGv4HP0HL2QH3Rm7SFbNqz/pFebFmxUyHOrpgke2Bfa0EI+aygqR7d+9KMDnwJAPYdbRUxd2Drlsfzrs3NvV6W75v1A4DBJpxfGk98NcSjW+Vbup/JcJhDLKYjSBp3wpXpNKP7OYqHmm7pFmdsHDs89ym0Pj+krlDRX0gw74hHWi1OlnJgh6hCGg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TK/o7AidCakKRfTtzZvV0HJAVQwbNXls9+EfNohUvJk=; b=CRsE73VIqpdxbsuvxsxhEZz4eFIiY/V9sEfKgdQR0mieR/LD04yHjc1U7H/qqDS54IEFYaEPrqHCxX2Ll3561K+18uY8r8qrcAWXIiRr//eaHOmm8KQGqSzrLCvmZBVFqtlbpJRInoAx18O8YqzarQMBxshnLg12vEiakSvZpWeagttkJprs25cxKunktuNYikkdTl74pSZoHVoT5T89rtrKtWxL2h35rgGpa4xsIxDL4iTOwm7gULxH3CzBAKSdxDtYaRqU8qaZVC5sWh/QJfuKWx0MI39rnm9GG7+l3xqJ1USirOG3KYUhD0WdP/Lq9fEhprNvwlux2eJ55li1ug== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TK/o7AidCakKRfTtzZvV0HJAVQwbNXls9+EfNohUvJk=; b=TnHbU8altQeoU8bxJXzhpmoqyYP+wBsjDH67A54Oo2C4LGz0N6zdDiax45sqIIiwXUTx7kwNfwklWOP1dmD/HrGeWHFx6hQcpqd77iTSfCGqPPxn4NMwdiZcPeay1i5XGEsiS1v7XSSWQQdc8tqf+qSBVuTZjx/FP6w58gZwF7w= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:48:20 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:48:20 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 4/9] drm/amdgpu: Add multi-GPU DMA mapping helpers Date: Wed, 14 Apr 2021 02:47:59 -0400 Message-Id: <20210414064804.29356-5-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064804.29356-1-Felix.Kuehling@amd.com> References: <20210414064804.29356-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Wed, 14 Apr 2021 06:48:19 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f09728d2-af67-44c0-d2a9-08d8ff114a43 X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: iNnpFX/4LSs9fEr5T3C/SiVMih3e4hiGCzTaQMrIaQCpaW1yC+TJxPcR4t760rmFcYhH7gNnkJse+cOqSSQqFzr2x+6YM3eETWKxrLNlm/DDrLPPobJeIL7SoE0lIMTN/prVCsRNjOhiqwxbJn8C5+oH53c7hjTy5R1hZ3v3VA+3Xg/Radcoi7KJ3rhWPSHdh77/cl4DRRm4RbJoYtSRaHIgxdf/XISXeYqlOlp+MSuIMF42hWIhVcZk2cLsXOObGh9kqQsEnn0Han2Ki+uhlkrBmiNU51LaweiUoel8/5Q/I7hM1OHkqkLw+6J6oDCQY33vLTZo7bp7v370AgkXhAydrkyqL9KBD+5CWI6ikhSyi/3JCAUfxeV0Q+TJIzzJQSHC8yIPMegwqdeZ8iQgXntUulsTMjzcAeTMFYuF27KyFRRdD9XcMI3U4TLh4u/COHjeEK8xSJxtEkZ1ArXK0UHCK5b/W7GmXp9WqlHoNCYxjq5Y5UZAGhDLSNQmoihIsYhKQ1pyHqIwRIAYl9aBfBJGCPCddbAB7rXj1h20d6QgPwRmI7+9cWn3OjUTH374uTGkJ0tR9pzjvNZsILAxoa4saErLbJA+vWADaKTMKRCvdZct0ysNUwts2D3BAhnTVMoQ1HgLbsUzIMHwL4efbSjRvgZ3i9OXHohkSPKUADY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(8936002)(86362001)(52116002)(38350700002)(66556008)(7696005)(6486002)(2906002)(1076003)(38100700002)(2616005)(36756003)(956004)(66476007)(26005)(316002)(478600001)(6666004)(450100002)(5660300002)(66946007)(4326008)(83380400001)(8676002)(186003)(16526019); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: Wzcj+BQfsJO8bexCzZHLN0reDNJ+X2fImMYciOz0dVkH5aRf+I1ayDAS7QcgeQ5A7wewWteSwDWdDfZehMWl0EgzuwGM6zlKWZyQF2zVaLnoJmlIA392c0f5bAid1ds6QEqO3+owrxIcnNTO7EAd7MZAdAD/4HxdJXv0QXJ0IBCufQV3DV+J5FFmwgOP8aMybz27hF94KMsC+TbzCSvmdYbMi2S+qgMjdf2w2lkniLALiYzt193Sd8ra1nZFb3SIx9qYwaIrJDvI2A79uRhBc8UHVpt/Ku9sPvGSUCQCwpj+S2fWerAtBeb9xTpBS62UWJEOsrFRJ7zZBmyiSx74jeWGIHXRv2Lia0BC+HuzdIz7sc2fd8U3q7slx6Mu9BX6Qm6ajwo7xqIm7hSlZhso+6rvNnOlp9dolm7ODDPJxzJ7ChW2zyztrfAG2vrgxzBMsYXf5G70SWaGSl7ZHUW0jM5FqmBGZ8bBxuiHBKxIjWezoyKeja12H15m2wjRQerb1yMwFO+LE1qXqwOLFOlvgA11pv5q2RZlaZkDvBVzrxrw9jzVjrdXNkqcB0JlzQEWuO0er1sSeGQgPKOSdUax1SYScTqVbCOIyral1WD2t/gOuO8x7n47haR8ikO65IKSLH1NY02bMxzVNppi5bk6aSWvP60oUtHTecKHcpdd+Borj8RiDaVmDc/ndfVwsH3LnlBfHQi3XRhfmUwTNl4R6+ORwIp7VxNulJ4M7ZPALfKk0rSeLa3Bj2iOritpPvcC/TGXBnLk9Sq1LfQcGdauE5r2T+NjDpvwWWeg7P/pI/6KxSCFww6XfcUfiHYjhBYu04B3epYFPjafHY70uk5ksI6pmHURwCk84nWP5PyYQ7nISlIOgsWhxM30et3R6qW4V4Glgp/WfOUQx3/6yWUYsr+drRMqqiJPVKhoJSkZkx1SPIJ2cgxTfeqCIyPeQQqZSQoLx8zjpwUw1HBWOBnOodMDbKK8shn51OkFBW37z/4IJPxnPUKs0z5vC8/ZXDCQUdAdDOqk6AaazNIz/U4rYhgR+xw3C8iJ8qz+Bs7jlD2T1UUQlNroDdrRzGqmUgQCEjiFbdUgiUKNZJz7NBq9YQoyPMgBvw0VSI/Y8NNScVj3WpEX9PdZGy75tEkryfKyFQv6goU5GvNLgSEKCy+sViamgYUkgeUu+J7HqkBznsQJcHCJJ8nMr1hpgc36PJnrNKkbmMTrmN4Q8mClocVkQla6gqwiJchedmJc3W+vcf9auB2eykgFpbsvLpk9lwfxh5rrshYaNa/FRNzn5Hb1cX5fIAqIK6Ra5oY2gapR6bw34g42ogLw/PIxrUhPcjOt X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: f09728d2-af67-44c0-d2a9-08d8ff114a43 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:48:19.9605 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: WpnQkwuXY/9dvo0MQaDG/fyscirveShA9eEfhnnp5JYEh0OOccj/mfzJWoV5fDZF4pusiCQ7O8mRMWGlo9fu0Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add BO-type specific helpers functions to DMA-map and unmap kfd_mem_attachments. Implement this functionality for userptrs by creating one SG BO per GPU and filling it with a DMA mapping of the pages from the original mem->bo. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 8 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 155 ++++++++++++++++-- 2 files changed, 152 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 910c50956e16..fc3514ed1b74 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -38,11 +38,17 @@ extern uint64_t amdgpu_amdkfd_total_mem_size; struct amdgpu_device; +enum kfd_mem_attachment_type { + KFD_MEM_ATT_SHARED, /* Share kgd_mem->bo or another attachment's */ + KFD_MEM_ATT_USERPTR, /* SG bo to DMA map pages from a userptr bo */ +}; + struct kfd_mem_attachment { struct list_head list; + enum kfd_mem_attachment_type type; + bool is_mapped; struct amdgpu_bo_va *bo_va; struct amdgpu_device *adev; - bool is_mapped; uint64_t va; uint64_t pte_flags; }; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 114fbf508707..51502a07fc1d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -471,12 +471,117 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) return pte_flags; } +static int +kfd_mem_dmamap_userptr(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + enum dma_data_direction direction = + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DMA_BIDIRECTIONAL : DMA_TO_DEVICE; + struct ttm_operation_ctx ctx = {.interruptible = true}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + struct amdgpu_device *adev = attachment->adev; + struct ttm_tt *src_ttm = mem->bo->tbo.ttm; + struct ttm_tt *ttm = bo->tbo.ttm; + int ret; + + ttm->sg = kmalloc(sizeof(*ttm->sg), GFP_KERNEL); + if (unlikely(!ttm->sg)) + return -ENOMEM; + + if (WARN_ON(ttm->num_pages != src_ttm->num_pages)) + return -EINVAL; + + /* Same sequence as in amdgpu_ttm_tt_pin_userptr */ + ret = sg_alloc_table_from_pages(ttm->sg, src_ttm->pages, + ttm->num_pages, 0, + (u64)ttm->num_pages << PAGE_SHIFT, + GFP_KERNEL); + if (unlikely(ret)) + goto release_sg; + + ret = dma_map_sgtable(adev->dev, ttm->sg, direction, 0); + if (unlikely(ret)) + goto release_sg; + + drm_prime_sg_to_dma_addr_array(ttm->sg, ttm->dma_address, + ttm->num_pages); + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT); + ret = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + if (ret) + goto release_sg; + + return 0; + +release_sg: + pr_err("DMA map userptr failed: %d\n", ret); + sg_free_table(ttm->sg); + kfree(ttm->sg); + ttm->sg = NULL; + return ret; +} + +static int +kfd_mem_dmamap_attachment(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + switch (attachment->type) { + case KFD_MEM_ATT_SHARED: + return 0; + case KFD_MEM_ATT_USERPTR: + return kfd_mem_dmamap_userptr(mem, attachment); + default: + WARN_ON_ONCE(1); + } + return -EINVAL; +} + +static void +kfd_mem_dmaunmap_userptr(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + enum dma_data_direction direction = + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DMA_BIDIRECTIONAL : DMA_TO_DEVICE; + struct ttm_operation_ctx ctx = {.interruptible = false}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + struct amdgpu_device *adev = attachment->adev; + struct ttm_tt *ttm = bo->tbo.ttm; + + if (unlikely(!ttm->sg)) + return; + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU); + ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + + dma_unmap_sgtable(adev->dev, ttm->sg, direction, 0); + sg_free_table(ttm->sg); + ttm->sg = NULL; +} + +static void +kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + switch (attachment->type) { + case KFD_MEM_ATT_SHARED: + break; + case KFD_MEM_ATT_USERPTR: + kfd_mem_dmaunmap_userptr(mem, attachment); + break; + default: + WARN_ON_ONCE(1); + } +} + /* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added * to a VM. It can later be mapped and unmapped many times without * repeating these steps. * + * 0. Create BO for DMA mapping, if needed * 1. Allocate and initialize BO VA entry data structure * 2. Add BO to the VM * 3. Determine ASIC-specific PTE flags @@ -486,10 +591,12 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_vm *vm, bool is_aql) { + struct amdgpu_device *bo_adev = amdgpu_ttm_adev(mem->bo->tbo.bdev); unsigned long bo_size = mem->bo->tbo.base.size; uint64_t va = mem->va; struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; struct amdgpu_bo *bo[2] = {NULL, NULL}; + struct drm_gem_object *gobj; int i, ret; if (!va) { @@ -507,14 +614,36 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, va + bo_size, vm); - /* FIXME: For now all attachments use the same BO. This is - * incorrect because one BO can only have one DMA mapping - * for one GPU. We need one BO per GPU, e.g. a DMABuf - * import with dynamic attachment. This will be addressed - * one BO-type at a time in subsequent patches. - */ - bo[i] = mem->bo; - drm_gem_object_get(&bo[i]->tbo.base); + if (adev == bo_adev || (mem->domain == AMDGPU_GEM_DOMAIN_VRAM && + amdgpu_xgmi_same_hive(adev, bo_adev))) { + /* Mappings on the local GPU and VRAM mappings in the + * local hive share the original BO + */ + attachment[i]->type = KFD_MEM_ATT_SHARED; + bo[i] = mem->bo; + drm_gem_object_get(&bo[i]->tbo.base); + } else if (i > 0) { + /* Multiple mappings on the same GPU share the BO */ + attachment[i]->type = KFD_MEM_ATT_SHARED; + bo[i] = bo[0]; + drm_gem_object_get(&bo[i]->tbo.base); + } else if (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm)) { + /* Create an SG BO to DMA-map userptrs on other GPUs */ + attachment[i]->type = KFD_MEM_ATT_USERPTR; + ret = amdgpu_gem_object_create(adev, bo_size, 1, + AMDGPU_GEM_DOMAIN_CPU, + 0, ttm_bo_type_sg, + mem->bo->tbo.base.resv, + &gobj); + if (ret) + goto unwind; + bo[i] = gem_to_amdgpu_bo(gobj); + } else { + /* FIXME: Need to DMA-map other BO types */ + attachment[i]->type = KFD_MEM_ATT_SHARED; + bo[i] = mem->bo; + drm_gem_object_get(&bo[i]->tbo.base); + } /* Add BO to VM internal data structures */ attachment[i]->bo_va = amdgpu_vm_bo_add(adev, vm, bo[i]); @@ -557,13 +686,19 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, return ret; } -static void kfd_mem_detach(struct kfd_mem_attachment *attachment) +static void +kfd_mem_detach(struct kgd_mem *mem, struct kfd_mem_attachment *attachment) { struct amdgpu_bo *bo = attachment->bo_va->base.bo; pr_debug("\t remove VA 0x%llx in entry %p\n", attachment->va, attachment); amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); + /* FIXME: For some reason SG BOs don't get individualized. Do this + * now manually. This is probably not the right place to do this. + */ + if (bo != mem->bo) + bo->tbo.base.resv = &bo->tbo.base._resv; drm_gem_object_put(&bo->tbo.base); list_del(&attachment->list); kfree(attachment); @@ -1376,7 +1511,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( /* Remove from VM internal data structures */ list_for_each_entry_safe(entry, tmp, &mem->attachments, list) - kfd_mem_detach(entry); + kfd_mem_detach(mem, entry); ret = unreserve_bo_and_vms(&ctx, false, false); From patchwork Wed Apr 14 06:48:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12202005 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B5B07C433B4 for ; Wed, 14 Apr 2021 06:48:31 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5C91F611F2 for ; Wed, 14 Apr 2021 06:48:31 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5C91F611F2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 2E33D6E8DD; Wed, 14 Apr 2021 06:48:25 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2088.outbound.protection.outlook.com [40.107.92.88]) by gabe.freedesktop.org (Postfix) with ESMTPS id E6BF26E8D6; Wed, 14 Apr 2021 06:48:22 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=KKh5ShnohYB6xPenNGPwP8mWI8wVekOhbHYNcJNpu9Z4eIVsEntpyf+xe/j1j7al/+uaUNMjYGE8qwFbiiK1PDstofJb9zQJSz/74sf8XzaxU/HeXg1v0jmKS+GXL/n33a9tC2wpxtOmCfewsTz1cOOEP+CRzS39vhWmeTJs4NJUt3v1RYO0J24NOh0U2SHOPTG+5yG5V74U2vP1j1ltzK58ugbp0wBfcp43apiHIJ9yANEkrVQZseTS82ZOutvfR4N6BqOCuyG58T78hTm81Tr0uOFL/ofZUkZZGK7DozNKuWRlOQFNdt1Sv9ZAxaX2TLet9GU9mYruBUCRYyi+iw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cb8YxsUBmYMVnZl2DgiAN61Zr0J9HEePKeYL5KfZrdQ=; b=nNjY+NcRH8k9m4uAeGYHZs+08DY36cvzdWtdbzSh3ipatTDemHMPtBLg/Wo0oUiMPDpwQ4kayurba+4I/mpkn4W+mh4N6UUCeEaMydk4rUIhSuhTdAb7zA9UN3Lfp04Wznp1MXC/jrX+vykUKux36U87aHbUtgPMfkvNwJmwNmgGDAhrm2htgjQXtA52Iiqansg2479M8lWcJdwc9vyzz8onFL4hJkznehg1TsL43tE26s4sksmgLN40YwDCaWuGNE6OzAH5TiKBkKdyUDEAc/eXvLSP67lDY1bk5OoKtDtqC92P3RKZZdbvIPG4m2JfWjPdGY6X0TyPobUq72oaow== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cb8YxsUBmYMVnZl2DgiAN61Zr0J9HEePKeYL5KfZrdQ=; b=qbsrCO80mf57CrM3lkmK8PK2czKFmkjDs1NqBdxPZLbmx3lWwJ/q+xzrbEPeyk1anqYhTVBrYdEcb8vOYQPfvs9Sv71kvQ/miOFrZ70bBJ4czDBrV95xQmLRYmkGsqQX39EQsCXrwRdZ4IeRVyWoal244765KJzbK/R0VjDuCQg= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:48:21 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:48:21 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 5/9] drm/amdgpu: DMA map/unmap when updating GPU mappings Date: Wed, 14 Apr 2021 02:48:00 -0400 Message-Id: <20210414064804.29356-6-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064804.29356-1-Felix.Kuehling@amd.com> References: <20210414064804.29356-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Wed, 14 Apr 2021 06:48:20 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: bd64d84b-ead0-473d-76e0-08d8ff114a7e X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 1GExj7Rqh92zne9tm2m/nxE+KG59xx0bWgld1cVBeqTiyXmboIcIGG3m/1Ph6uGqWoT/gi6TYDtzYxFxOVDcp6PF1k+V/HKnwkZfc0SgCSR3uYSRsMyGjL510eGtHYC6RqHsmWcEr7fgUpaRxQvD8eAtTNZBAO9u4x1q2h7NWm8J8FCp0XXKtED0F2nuGUw6IDNmoLkZmHdOlEb/x5+prpGne+imineaPT7vUfBrylT4/wFyEVxOJsAsl3b8rn7I6uYs8YdkUYyJTq6PesbbVM5CacMUDBAw2+UXU5HoHzzGf32SSIDNdnW25GvAbyYtVKPbZkT7t77iV+sMz/TWLbguibjGG1eio52UYnKglcyGGT37nuAa5eEFEIVnWD2ZgyK66STzmmq0PDa0oR9kWyhAihYfzquk4rDbWaR5GBZqLDuIOOKtQ+L/2FoBNOiH1MTYxn6K/1wbIrmgArrH2mzp7hRKfVeWoyYqwolKOKO1hD03l7eREK3NkstifnuK90Ru2LbfgYXEifcjHpr9FDvcUjqirf3947ei3L9hptOEpJBPKpnbGyCGP1W+Oz+6Bcvuro+zJqdH9lEqmGePMUF+K1h0j24ufzftqncKEm4uhwU93ojxT5eWf/9EP0kG+CA1YxlxHxgbBdZY+BS+vcBb00YnApGdzH+1djwCOWI= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(8936002)(86362001)(52116002)(38350700002)(66556008)(7696005)(6486002)(2906002)(1076003)(38100700002)(2616005)(36756003)(956004)(66476007)(26005)(316002)(478600001)(6666004)(450100002)(5660300002)(66946007)(4326008)(83380400001)(8676002)(186003)(16526019); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 5MUhUT1psH+7wrhsOTTMdcxH9AF64vR9235L4WD04/XRyouZ8+mcTmHxV1r6vj87bCwdTrpHU8k52J7mXH3Fo2qad7v1Lm+0sgyOEUvCN5dSKAoY5KR97P5ONb9v8USRGZwzx3/WeDNeTSOloP/HJsa4XklCO7Bg5pIuUAJybpuFyh4/37zH6SSFGZuEz/cFRs+LOUYWtLmnu52gFunKoGOZ4ZGtY6CeYO5OAMoSLxwIlnASXmdZnh6hAa0o/1KAS4XZ79Otwb0FMPEUxV1o6HSoOWKMKQKnyLcOJdoj6c5KLrLe5XDhwk/clVHH5BjvNcHpmRII5NR04mks1JowSC6bUBRnrZAgqxybZxZ9IAxeIsMIA7sCntFw/HsOkxqAmnB3FZpm0YKaUAmTHnRzYE/lv0aFsrMkNgLPUMLyWueqlVnkUw1OX9DxdyYUQ7PecxUQCa99ImpjLuozOExg11xk8U8qO1qtS+92UQzTell4sYO+ZiA2J2RpcGqiArdy7GkoWYVIC5Is5oje9rPc0ue+J1TmfyvQeHWRfQCwiFKAyI3em3J7of51LO9hiWWRnqEFemNUyKKJypRkc+tfQWIe1gGzqVL/zwXPvwx262fFK/qu+r/c8u7XNCsvXUDT0Arj0TlpzNA238a3Iyk6n9a4NRCjHRChQVOKz+t0NYoqBKlLMcEe3YXQKOQn8xTycPYjBwDUj8oT9ZBGGxHvgpwfLguu6xFmvMF+ShJq7zA1fkbkWmRvgES5nmmSE15/G680idAR4QmNXpaNRUHV+DFlelt+fZFD8xAqDlOaLQxcJvow6R0L9IkIjnvelwGtupCyAzIRov4j2LuKdZMwtMrXru3cnerhpA85XghKnAknVWR169jjMl1dgVbIZo9ZY4cLe7FCBmK/TA9aZx0gVKpQ83cnEipFbTuvtmmX580ES1uCSvi5+kQxx2KGx7EVux3LUwo3shPvZoRScsh+68xZZKwGIlcQ5wJc5o5kAP2Y8wXqiRf5FIVcqaCps8rg9gdWBV4UUFjHAV+ekWNNp9n5BKg+QFs8VKpwsHV3pV4zkmhDEP/CJEygwmYDd4f88WDueWkwcDcODHmNI6flhl3KVgKXPXsCxMTyokUw8WyY2hpPj2F2x8ey/B7RvMYrcK7USmjeFvJ4DtOQ6I5Y4GMJQaXRWery9PH58rE16aISooniwQp8+LJCDbdK70woTBjl5dQ+88PbDgWcinE/Fu0DUVqt5hTZejAj0sflQgLB7+P1c7U/acCXq7OtdExKtdIYG9Nu0TFKu9aCcMt5yAfVjLryAAGLGVkZYrbUWX4kfhGxoSMWFoiwnhbbmrG3 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: bd64d84b-ead0-473d-76e0-08d8ff114a7e X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:48:20.3428 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: SzzOFxhsgszkzemYoEXgKzWATeQrJB1r9aL8LUy9pLKBP4UrS9/iMlEokNhuvj/xsOtorBQl5uV+heKtFSeU1Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" DMA map kfd_mem_attachments in update_gpuvm_pte. This function is called with the BO and page tables reserved, so we can safely update the DMA mapping. DMA unmap when a BO is unmapped from a GPU and before updating mappings in restore workers. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 56 ++++++++++--------- 1 file changed, 29 insertions(+), 27 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 51502a07fc1d..3bb2ae185bbb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -964,11 +964,12 @@ static int unreserve_bo_and_vms(struct bo_vm_reservation_context *ctx, return ret; } -static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, +static void unmap_bo_from_gpuvm(struct kgd_mem *mem, struct kfd_mem_attachment *entry, struct amdgpu_sync *sync) { struct amdgpu_bo_va *bo_va = entry->bo_va; + struct amdgpu_device *adev = entry->adev; struct amdgpu_vm *vm = bo_va->base.vm; amdgpu_vm_bo_unmap(adev, bo_va, entry->va); @@ -977,15 +978,20 @@ static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, amdgpu_sync_fence(sync, bo_va->last_pt_update); - return 0; + kfd_mem_dmaunmap_attachment(mem, entry); } -static int update_gpuvm_pte(struct amdgpu_device *adev, - struct kfd_mem_attachment *entry, - struct amdgpu_sync *sync) +static int update_gpuvm_pte(struct kgd_mem *mem, + struct kfd_mem_attachment *entry, + struct amdgpu_sync *sync) { - int ret; struct amdgpu_bo_va *bo_va = entry->bo_va; + struct amdgpu_device *adev = entry->adev; + int ret; + + ret = kfd_mem_dmamap_attachment(mem, entry); + if (ret) + return ret; /* Update the page tables */ ret = amdgpu_vm_bo_update(adev, bo_va, false); @@ -997,14 +1003,15 @@ static int update_gpuvm_pte(struct amdgpu_device *adev, return amdgpu_sync_fence(sync, bo_va->last_pt_update); } -static int map_bo_to_gpuvm(struct amdgpu_device *adev, - struct kfd_mem_attachment *entry, struct amdgpu_sync *sync, - bool no_update_pte) +static int map_bo_to_gpuvm(struct kgd_mem *mem, + struct kfd_mem_attachment *entry, + struct amdgpu_sync *sync, + bool no_update_pte) { int ret; /* Set virtual address for the allocation */ - ret = amdgpu_vm_bo_map(adev, entry->bo_va, entry->va, 0, + ret = amdgpu_vm_bo_map(entry->adev, entry->bo_va, entry->va, 0, amdgpu_bo_size(entry->bo_va->base.bo), entry->pte_flags); if (ret) { @@ -1016,7 +1023,7 @@ static int map_bo_to_gpuvm(struct amdgpu_device *adev, if (no_update_pte) return 0; - ret = update_gpuvm_pte(adev, entry, sync); + ret = update_gpuvm_pte(mem, entry, sync); if (ret) { pr_err("update_gpuvm_pte() failed\n"); goto update_gpuvm_pte_failed; @@ -1025,7 +1032,7 @@ static int map_bo_to_gpuvm(struct amdgpu_device *adev, return 0; update_gpuvm_pte_failed: - unmap_bo_from_gpuvm(adev, entry, sync); + unmap_bo_from_gpuvm(mem, entry, sync); return ret; } @@ -1633,7 +1640,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( pr_debug("\t map VA 0x%llx - 0x%llx in entry %p\n", entry->va, entry->va + bo_size, entry); - ret = map_bo_to_gpuvm(adev, entry, ctx.sync, + ret = map_bo_to_gpuvm(mem, entry, ctx.sync, is_invalid_userptr); if (ret) { pr_err("Failed to map bo to gpuvm\n"); @@ -1672,7 +1679,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( struct kgd_dev *kgd, struct kgd_mem *mem, void *vm) { - struct amdgpu_device *adev = get_amdgpu_device(kgd); struct amdkfd_process_info *process_info = ((struct amdgpu_vm *)vm)->process_info; unsigned long bo_size = mem->bo->tbo.base.size; @@ -1707,13 +1713,8 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( pr_debug("\t unmap VA 0x%llx - 0x%llx from entry %p\n", entry->va, entry->va + bo_size, entry); - ret = unmap_bo_from_gpuvm(adev, entry, ctx.sync); - if (ret == 0) { - entry->is_mapped = false; - } else { - pr_err("failed to unmap VA 0x%llx\n", mem->va); - goto unreserve_out; - } + unmap_bo_from_gpuvm(mem, entry, ctx.sync); + entry->is_mapped = false; mem->mapped_to_gpu_memory--; pr_debug("\t DEC mapping count %d\n", @@ -2083,9 +2084,8 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) if (!attachment->is_mapped) continue; - ret = update_gpuvm_pte((struct amdgpu_device *) - attachment->adev, - attachment, &sync); + kfd_mem_dmaunmap_attachment(mem, attachment); + ret = update_gpuvm_pte(mem, attachment, &sync); if (ret) { pr_err("%s: update PTE failed\n", __func__); /* make sure this gets validated again */ @@ -2287,9 +2287,11 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) goto validate_map_fail; } list_for_each_entry(attachment, &mem->attachments, list) { - ret = update_gpuvm_pte((struct amdgpu_device *) - attachment->adev, attachment, - &sync_obj); + if (!attachment->is_mapped) + continue; + + kfd_mem_dmaunmap_attachment(mem, attachment); + ret = update_gpuvm_pte(mem, attachment, &sync_obj); if (ret) { pr_debug("Memory eviction: update PTE failed. Try again\n"); goto validate_map_fail; From patchwork Wed Apr 14 06:48:01 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12202001 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 02574C43460 for ; Wed, 14 Apr 2021 06:48:29 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A0B566120E for ; Wed, 14 Apr 2021 06:48:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A0B566120E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 91A036E8DB; Wed, 14 Apr 2021 06:48:24 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2088.outbound.protection.outlook.com [40.107.92.88]) by gabe.freedesktop.org (Postfix) with ESMTPS id 21ED36E8D7; Wed, 14 Apr 2021 06:48:23 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=J2Frpw5K3TSv3Ec3xxVj8Uffy+WNfHqERNK/ZTDZDRRmKsV0uRNWC3L9mcxUwdi+OMBo4KTiXvmVhH2C0XLjiIl9Q0pnN9caT4N/PEJkJ0JUmRbHpcQnDd/TkPiFLBWO7MSISa+ubTGkptwYuUeu2VJWvoQi5jAk95h0zAOTHAe11J4vi1o5e9KzNaf/puuQQZpAm1VSqKIlJGtkY4qGwRwqFlXOGd2oQK6qTj7xqboPEEDhpjxuIwxiwv8pWmtnxJMiDnSembmUTbf7TmkkEiKuykxCZFg9O8TCdZ3dA+xzT/GUZRmJHL/Qg+WFtCZZr8hXjEE5BdyB1LVNlnYnHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yQut1xXkpwdjAD1BditEYBDjILo3RVAJf1/Yf2Aq5Us=; b=aBXEZj1C8E17bhJy8RH6AKP4rvtTbPr69Fac8YqPuOmVlG+BB+cl/57CBSEEOJdzfq6bFeIDcZfZbhJgd9miMCtHz6omF9ui6f82T91ywKkPhNrW+i/DUUax8F2yC18Fv75YrRpptfwitoK8vZgg/KIL/q88l4N1kUyXZqf6a/+Klr/21KdbbD5dDwpTpGmh1S7dd2d0vJYYZN7C2ir23cTPcUWsLV5Y0jkSJELpnGxqgi12ewZV6IS/FJ6KGsWrWOc0yiUDFeizOUobamWej9aNauoOADP39x02AR1w3hkEI0MZ4s1Ux1XjOMubzNBHUPnlrS6LVU4qRKipcgeIVw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yQut1xXkpwdjAD1BditEYBDjILo3RVAJf1/Yf2Aq5Us=; b=Sg21ZqsgdC7At4dNt5BYfPG+T5sF2TtoDyRwFLqIPAyf1i3v81cq8GnZ+lM6XOqgAJWqsJ0ateyQaE3IOz7TVIU6eSm4fw4cCGt3RVFtRuz+hZ9qByMp0Kjfrhv8/y6qXJBn6lz5MnBqSN0UN0lwHCV287tWxZ5AoGz+xmHjx3M= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:48:21 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:48:21 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 6/9] drm/amdgpu: Move kfd_mem_attach outside reservation Date: Wed, 14 Apr 2021 02:48:01 -0400 Message-Id: <20210414064804.29356-7-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064804.29356-1-Felix.Kuehling@amd.com> References: <20210414064804.29356-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Wed, 14 Apr 2021 06:48:20 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 4490978d-a5cc-4908-7c8a-08d8ff114abf X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4heHkAiCHQo9qiZ3Jh7nY/MzOsv5Cn09xPve7N9iv1w1754a9ddXYxDG+ytk4qkAzDfnuMXRVAETTmxbVk0/pvToDbAkOh3F53DtuJmQSYymnF4U7sVZs6CT+UbZj1+et6QGQZzuUEjH2lK+ct8lM6dZLJUEeySmn4gEYz9774h/i0mYgrGH+PlnrJ5wFP9g8lytWgjNuZLRi0EUNfFyI6bkf/o9CGZ9CEZmqx39whgIurmJcQSWnlsdpu8UIVPpxXDSph1gvL//s053CDQME9xajoxGnPMwRil2gYTta+mWpHNowWAoZOt5fifZaLqW2Q1rfZ04EVbWtcRXmRuc1eIJsgVMyeunBO+imn83aYGv0ISJUtw7g3tWeO0E+vPtd/cv1XiDqGGhZcG0U+PA4ajelIn1RAXHImigfbZv/1Ym334u0gaSET7g1gj6CH+AaPzmWhQCwrsPeSD+OboYdHR75Ec9Nq+8aXDdb/7TGMUSkbSsXtPYjjoqXpv/kpfSuy2wG/CmfYhes2j529kC+4eB0IzU8MUSlPovsVTisUf5gqQdh153bA+ru6RdFjepWy/enEX/fKF7hSM1hqjNejB5q9SL3TF058gmnvLX/FOihEKIoMAzG10ZD3BLK6YMU8hal7fFYeTvpEHAvPWyc3hzNwuoMj8RrYF2e/Xd3jM= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(8936002)(86362001)(52116002)(38350700002)(66556008)(7696005)(6486002)(2906002)(1076003)(38100700002)(2616005)(36756003)(956004)(66476007)(26005)(316002)(478600001)(6666004)(450100002)(5660300002)(66946007)(4326008)(83380400001)(8676002)(186003)(16526019); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: YAZVlu7smxY1sbtUk28v8PxTWd9MVV0iKO6kj2aqBD4IIlEFZmG3nj3Vs4B+ilrUGZed3RF4dJx7piF/hih6RaaVtx3Etvt/3DboNFS8+QtrZYGy0WMl0o9eFMb+ACuKCSYWARHxs0yrkAodQ4af/v8+dBlIp9EoLJaFSrnQ5jlaLePmeH+HT+sgzvKcMZ+cYvbJGFbSQkSZEQHSOfe9Jk1lnsVMSBQyz8xTZvnqcW9hjF0fURrQfTcHixxVuX6AFeLYAHH6XbtU0OTO4AoU2sk2MxC0APQpXhyGMKmAVaBawb0XMWA/VQ3l9T5JBjl3P7WTaEfTIJyKr++UaM1NV8W1Q5SUKBES02VAXUEgUncHCH9qX7KOXrNbQPnch1yZpExEPHlTErWEgjY1SdKQgIv9UFpckEoCN1JTfw9VrpXAjr1urkoCa64MJ8evh4hXD+tOvrjmf7FC6wE37itBJ1tcy3yLk3DUhPm2ZaBa4XqIebSd0O7GuMCU3gDEwEfgOdio5g+t4PH7uUR5fKFsxwb1FpHEFIW5LsJGOrmXgLacvNwOaWMA0jlcjK2sVkarlxN8YhHnoqXDyYIQdYVycJfLHUFr5aquwFCNTzuxSz0DG7mc1OWzN2NN8eZQEKxvMRmJ+Xp4ERe2G7/sXUQKB3EoMNvKxwmph2ROc/bTwRZrmhDdfb+mAjK2WOm85fVQcsR9DtTeRxKQkm7aCLNn3CrOyE2vMpKqyqKq/GM1P/eKd4qYJgC6RSZsErwgF6OSQpVzt8l/XeXjxeaS3jHI79VUY2PuJHLXBUZan7yL/RC6ekOMtN23nkvYTe0J2Hnhj+yZovspsQmSHpYThFy9dN4lVluqg7Y9mFm2TuB1zBwfEskhYI/zbA0E3oFIjZ8WRtNw6pQq3cbVlQUrf5XeCOV5Ti2blZf6qk+liJaZiIb+6LSkUwe7j9DssjGQw2y33Rt0afwvlrcVxb1iLh4/tIrupcoh5/wTAJmnSLIux0Sqb5i4OTjuqObc73hm01lXXOvwC+iXJgE0ZXBwc8FkEOxY/6JHUOxROgSgK1kXw8UWQq+DWuvfsvgALweoxTdvgxpjxJn05kM+3S/sNHllrrVAVKYWOd7vFpAMD5uTZ2FLWQiGtuFOJFLMOkQPxrheiYd3f5jq9LJHQmsm48x4tmN+OyXQHNqduRduFltOj+Kfnb39JQs8XHCsP1Zk35pAEz4IrXz2nQbdCMMnX2x2ofvoTIATLoBbbPYlWL1DTS6RzdUEubaLpX9oS8SNoCMMFgpQehVIJNy8SbOjfz3hSFX5/fFd/NhXW7wlelxCxICFsYzHrY1tt+dIxcH6IJQ1 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 4490978d-a5cc-4908-7c8a-08d8ff114abf X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:48:20.7968 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VYExOLpeSVYtvYmkB1WcwRia5dhj5z8vXlHnff2GB6Fq3Xvke7ZJewu/5jusFJELeg6LozKY4Rztd9TxmxbeaA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is needed to avoid deadlocks with DMA buf import in the next patch. Also move PT/PD validation out of kfd_mem_attach, that way the caller can bo this unconditionally. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 72 +++++++++++-------- 1 file changed, 42 insertions(+), 30 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 3bb2ae185bbb..1416f3c03f1d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -575,6 +575,32 @@ kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, } } +static int +kfd_mem_attach_userptr(struct amdgpu_device *adev, struct kgd_mem *mem, + struct amdgpu_bo **bo) +{ + unsigned long bo_size = mem->bo->tbo.base.size; + struct drm_gem_object *gobj; + int ret; + + ret = amdgpu_bo_reserve(mem->bo, false); + if (ret) + return ret; + + ret = amdgpu_gem_object_create(adev, bo_size, 1, + AMDGPU_GEM_DOMAIN_CPU, + 0, ttm_bo_type_sg, + mem->bo->tbo.base.resv, + &gobj); + if (ret) + return ret; + + amdgpu_bo_unreserve(mem->bo); + + *bo = gem_to_amdgpu_bo(gobj); + return 0; +} + /* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added @@ -596,7 +622,6 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, uint64_t va = mem->va; struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; struct amdgpu_bo *bo[2] = {NULL, NULL}; - struct drm_gem_object *gobj; int i, ret; if (!va) { @@ -630,14 +655,9 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, } else if (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm)) { /* Create an SG BO to DMA-map userptrs on other GPUs */ attachment[i]->type = KFD_MEM_ATT_USERPTR; - ret = amdgpu_gem_object_create(adev, bo_size, 1, - AMDGPU_GEM_DOMAIN_CPU, - 0, ttm_bo_type_sg, - mem->bo->tbo.base.resv, - &gobj); + ret = kfd_mem_attach_userptr(adev, mem, &bo[i]); if (ret) goto unwind; - bo[i] = gem_to_amdgpu_bo(gobj); } else { /* FIXME: Need to DMA-map other BO types */ attachment[i]->type = KFD_MEM_ATT_SHARED; @@ -662,13 +682,6 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, va += bo_size; } - /* Allocate validate page tables if needed */ - ret = vm_validate_pt_pd_bos(vm); - if (unlikely(ret)) { - pr_err("validate_pt_pd_bos() failed\n"); - goto unwind; - } - return 0; unwind: @@ -1516,12 +1529,12 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( pr_debug("Release VA 0x%llx - 0x%llx\n", mem->va, mem->va + bo_size * (1 + mem->aql_queue)); + ret = unreserve_bo_and_vms(&ctx, false, false); + /* Remove from VM internal data structures */ list_for_each_entry_safe(entry, tmp, &mem->attachments, list) kfd_mem_detach(mem, entry); - ret = unreserve_bo_and_vms(&ctx, false, false); - /* Free the sync object */ amdgpu_sync_free(&mem->sync); @@ -1597,6 +1610,12 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( mem->va + bo_size * (1 + mem->aql_queue), vm, domain_string(domain)); + if (!kfd_mem_is_attached(avm, mem)) { + ret = kfd_mem_attach(adev, mem, avm, mem->aql_queue); + if (ret) + goto out; + } + ret = reserve_bo_and_vm(mem, vm, &ctx); if (unlikely(ret)) goto out; @@ -1610,15 +1629,9 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( bo->tbo.mem.mem_type == TTM_PL_SYSTEM) is_invalid_userptr = true; - if (!kfd_mem_is_attached(avm, mem)) { - ret = kfd_mem_attach(adev, mem, avm, mem->aql_queue); - if (ret) - goto attach_failed; - } else { - ret = vm_validate_pt_pd_bos(avm); - if (unlikely(ret)) - goto attach_failed; - } + ret = vm_validate_pt_pd_bos(avm); + if (unlikely(ret)) + goto out_unreserve; if (mem->mapped_to_gpu_memory == 0 && !amdgpu_ttm_tt_get_usermm(bo->tbo.ttm)) { @@ -1629,7 +1642,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( ret = amdgpu_amdkfd_bo_validate(bo, domain, true); if (ret) { pr_debug("Validate failed\n"); - goto map_bo_to_gpuvm_failed; + goto out_unreserve; } } @@ -1644,13 +1657,13 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( is_invalid_userptr); if (ret) { pr_err("Failed to map bo to gpuvm\n"); - goto map_bo_to_gpuvm_failed; + goto out_unreserve; } ret = vm_update_pds(vm, ctx.sync); if (ret) { pr_err("Failed to update page directories\n"); - goto map_bo_to_gpuvm_failed; + goto out_unreserve; } entry->is_mapped = true; @@ -1667,8 +1680,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( goto out; -map_bo_to_gpuvm_failed: -attach_failed: +out_unreserve: unreserve_bo_and_vms(&ctx, false, false); out: mutex_unlock(&mem->process_info->lock); From patchwork Wed Apr 14 06:48:02 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12202011 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 65DC5C43600 for ; Wed, 14 Apr 2021 06:48:37 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 27C9E6101B for ; Wed, 14 Apr 2021 06:48:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 27C9E6101B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 608626E8CD; Wed, 14 Apr 2021 06:48:30 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2088.outbound.protection.outlook.com [40.107.92.88]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4C20D6E8D9; Wed, 14 Apr 2021 06:48:23 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JAyJNIVIULons+xytz/Pwfdu1CM78nfCAdeMqZWeGCKH9nEb1i84hZej80XzHIfreVMl6lHzv9HmF3IB7xdbKHHEHkiHNejg5f/6/sKWEWj6BFK1HBIAqXbVyZ8Ct4mPNsNpDv8rJCmFJucaCCFmG8X9/BY7yuDi3Oc0LqSCdRr4+tuNeCBLZ6MYjmGP+ZuO5kV6qm8l6XqtwbRsf/B5H4F+lF8n2nhFv0+rJipWVO6K0u9RBq2IbNpmxih7vhW/S+p1VcG9D5261XgAB/98CkeAMdtiM3nJt34FpmuQU37I1uUYfvYKJ/zFv01tXcMcKtjPQZ7Porf/B4lwqKpKiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U1Z7N+TokxJateGZpU1pIpnTcEkYQ9R8vQvQEi4r2h4=; b=PGhAaa4i4kNXMRfocMnm4RfZGgtNJrXgo6zcdpPfBlMfXJIb9QAaJ4DqhIQqkd+28RtSYbu+BnMf/3p+1gKQaMYbtgzXQZ+jrCZeGFnE+1SQKvIgTd5qGZ5w9zhBUu0+T2MEH+1fIcshk6UryME/fOFYiplkS83uApbWd/f+u65KmIJE19xkoVJrout4kNpnCsgM7vQUaJP5HF5uAAmcJkbsEvTJOz5E635CnNNeZ+e4o1Q0BBfbxLl7T6NkZ5ZlMgaq5W79H/MVHplKoIDU8IXWg8FwNRNYDsqfoC+m8UweuMpV0SXXgTitqQLyrX4utp6lrqFYWBOm6wjrd5uNfA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U1Z7N+TokxJateGZpU1pIpnTcEkYQ9R8vQvQEi4r2h4=; b=k/W80tQplDV53AQJh3k4WuId8xz/xM3hj7I3GjHB+ePxTNhhumRCZRPR0R72+Qe8jUavmFVReLQrX8OdFI9s0fO5jbZiOkE14nnbk118PWnCGh6r0pbYdUKgVk0lBSEKp5fHAJ9o25Talih9bWIEKEyDKvQ6q3q5Tim7FCZ06vU= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:48:22 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:48:22 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 7/9] drm/amdgpu: Add DMA mapping of GTT BOs Date: Wed, 14 Apr 2021 02:48:02 -0400 Message-Id: <20210414064804.29356-8-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064804.29356-1-Felix.Kuehling@amd.com> References: <20210414064804.29356-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Wed, 14 Apr 2021 06:48:20 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c2042c79-6d37-4b0d-61b8-08d8ff114afe X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:608; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: V1GReXIBTOvFuZiEYYg53p1m31l94A50p9y6pGX8+qaFd2iyCO2tyw3W8IzraS8TQzAFMpGgJ/QCkOaiYKUhNhVzJ9Kqv5tRGSM2w25MVANCh0KakpzmBrpJmvXCC7/Gek1hujSb9DMy4t5ZVoJgXfUEApvV3+VljgRw+aOQ9n3QomeOlKEGTKlSSYCvk0LhR9k3LhkycYFIbDeVS6nNAipCDHzjZ5sa85N2u56dtK7G/vc1Iwy4X+OYpGngvcTU7h8EtbosGYdsh1wX/wa5T77sXwS/Hgq/T0wbrs7QLC9ijqAkXLlmatTLn9WlLdkKCuP0tJPyBnhn0yEPYCkN6uR6ET/nobtXDywcL++PN6eI6Z5wEkhPpS0R5AlRVL1gE2FldoS+Yzfy/swcIE8SgpJXyMWffNWuEmIi+0uwidbEhxh5QRQQmbcNVRsXi9jTMJW1UOLrpKCyzgIJI6x2N8fMV9vJcB5ILSruQOgg/2zsVf628HhIRn/q01EQpApSBmJPlwOvl5dZLbTQIif7I5+pdxMgJScfXsnQvpYQcnuDNNH78YpCRsxADUJ39dPTRUd4Zfi9KELtvpgEX5UO5UKh2OV9+FKvHotP8PGf7qKI96nVpDkOwpETd12U6M963U5wS5krx1ALG+Vaj4Q/P+EHggEpJpfGcImCDi3zOxc= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(8936002)(86362001)(52116002)(38350700002)(66556008)(7696005)(6486002)(2906002)(1076003)(38100700002)(2616005)(36756003)(956004)(66476007)(26005)(316002)(478600001)(6666004)(450100002)(5660300002)(66946007)(4326008)(83380400001)(8676002)(186003)(16526019); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 1Cxwpt/uRrq1jKcnqnGOMm31J5HBzZQRhFUZ2+4rTRI8PwC2ikEMemQRSsYU0MBiGIktJAXSPdZLFMjJmCDGck2tpF6BgGzkDPBMWIDmoos3kmgx5qkXNRjfkzel6WefgCAUUZB4KxJ3+pAaEB0pxO9ldZcnS5F+vOnjR6HugM/ggRYpPYlWLZUd6EAbxHPBy0xfKgziCKQl/qXLLcJF6PHQ+zbT0eXotdJuWG6rI5ZdrcprEBBHjANZFjNE7MvJxjvGDt7KV7R+kcHFk9Z/QWa20WonN5hXJITq7uT1MVV1JCTL3PhTHViHPO4zrBijSaDjh9oJO8BgZuJRh5Hn5+s65KHJE7imKhovNIA37YaxpSKA+hKiOT2rP8tkS3Lnqg3cNCeN7LWbvXmvIHFycacWBv7d6DJG9NnAm6qu+AAdbmonf0hlcqHRvTvsck2ngLnl0v1tp+HI5TsX2Q1dAjyS1x7tJJrYfZjtZoyb/gyeIYOYNReDk08Tkp88fcXDeihVHPpYs++Yrg6DXPO6J/H6U1n/yz3CQgTxOFmS8Xa3O/aolchim7eVvdwLrtATB16tWZlld25wi2H81RgJ9S2incfQS4VGmPvDB80zRPu8b+Axpdv6cdor3aJN9Fzm5bVNF6SbFfM2Jubo0/t6xzZpVnZYfsa6UV8/YKGYCRyguMUGUC2brLPOWd2MEvieawh6yGXOIWAhgy1fGEgTmZlDWwPl488xhD+vbTnt4LXVvJQr+DOQIgqw4aMwT7LyWEXhgGZ/JMcZ9/T/ZiOdZLtK3TLsnQpooGx5Zdd1hJpbGqCpoDcyHgmO4P8EuV2k+WDrzc2LSc4ChCDDkMApGDY+B29/WEtcSqy/b4IUyQqKfvNoS6xKqp3YhdQmUOJIkd1ivQzto1GaWK2Wh8CuKBplKjNb106y/lFRfw0hUZgbzGPxBNMCmCR0/e9khHrfLTaBf4BP5uW/UwqhHlRWmau7rZAI4yVRKWRnil8ITnN4snfMmJ7flk7HGU4RIdPimSin8CzhFrcntM2ZAxwytH54oDSqqpWIOz+PZtlOuNVCoxCdOcf4cLWGW4/XIzFf5RBRD0YcypK6BrG1lEsWIsZ0S5GfihcEkPbXn3ps5DHXCYCfaftsOE5YNrL8mXaC+NufbjbNg/iHBO+tT7jxn5x0f8cj8P4oOeqh58TODT0SElv8d6OUiUfqYw3lJisr79nI+7BErXxybPCaOKokFICzrZPFaG/LeBQqnwH4TEVQuYGQm3lJoFz1mDIwZF+9KRI8maDM5gGPxUm4mxRNY4Y/8H9Ss7wOqlr+TWIjLlbE9BdFyId+6jmypmMI3Qaz X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: c2042c79-6d37-4b0d-61b8-08d8ff114afe X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:48:21.2359 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Y1TyPt0GhRQhVOXdvK31ApXjF23gsLEB/W1FPz6YT7wiHVXOwvNexI1ErrII9VgmgdObexSeRbEiSE+4TV7zHQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use DMABufs with dynamic attachment to DMA-map GTT BOs on other GPUs. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 + .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 74 ++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index fc3514ed1b74..3ea51982b720 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -41,6 +41,7 @@ struct amdgpu_device; enum kfd_mem_attachment_type { KFD_MEM_ATT_SHARED, /* Share kgd_mem->bo or another attachment's */ KFD_MEM_ATT_USERPTR, /* SG bo to DMA map pages from a userptr bo */ + KFD_MEM_ATT_DMABUF, /* DMAbuf to DMA map TTM BOs */ }; struct kfd_mem_attachment { @@ -56,6 +57,7 @@ struct kfd_mem_attachment { struct kgd_mem { struct mutex lock; struct amdgpu_bo *bo; + struct dma_buf *dmabuf; struct list_head attachments; /* protected by amdkfd_process_info.lock */ struct ttm_validate_buffer validate_list; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 1416f3c03f1d..bb3a96ab8f20 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -522,6 +522,16 @@ kfd_mem_dmamap_userptr(struct kgd_mem *mem, return ret; } +static int +kfd_mem_dmamap_dmabuf(struct kfd_mem_attachment *attachment) +{ + struct ttm_operation_ctx ctx = {.interruptible = true}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT); + return ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); +} + static int kfd_mem_dmamap_attachment(struct kgd_mem *mem, struct kfd_mem_attachment *attachment) @@ -531,6 +541,8 @@ kfd_mem_dmamap_attachment(struct kgd_mem *mem, return 0; case KFD_MEM_ATT_USERPTR: return kfd_mem_dmamap_userptr(mem, attachment); + case KFD_MEM_ATT_DMABUF: + return kfd_mem_dmamap_dmabuf(attachment); default: WARN_ON_ONCE(1); } @@ -560,6 +572,19 @@ kfd_mem_dmaunmap_userptr(struct kgd_mem *mem, ttm->sg = NULL; } +static void +kfd_mem_dmaunmap_dmabuf(struct kfd_mem_attachment *attachment) +{ + struct ttm_operation_ctx ctx = {.interruptible = true}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU); + ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + /* FIXME: This does not guarantee that amdgpu_ttm_tt_unpopulate is + * called + */ +} + static void kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, struct kfd_mem_attachment *attachment) @@ -570,6 +595,9 @@ kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, case KFD_MEM_ATT_USERPTR: kfd_mem_dmaunmap_userptr(mem, attachment); break; + case KFD_MEM_ATT_DMABUF: + kfd_mem_dmaunmap_dmabuf(attachment); + break; default: WARN_ON_ONCE(1); } @@ -601,6 +629,36 @@ kfd_mem_attach_userptr(struct amdgpu_device *adev, struct kgd_mem *mem, return 0; } +static int +kfd_mem_attach_dmabuf(struct amdgpu_device *adev, struct kgd_mem *mem, + struct amdgpu_bo **bo) +{ + struct drm_gem_object *gobj; + + if (!mem->dmabuf) { + mem->dmabuf = amdgpu_gem_prime_export(&mem->bo->tbo.base, + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DRM_RDWR : 0); + if (IS_ERR(mem->dmabuf)) { + mem->dmabuf = NULL; + return PTR_ERR(mem->dmabuf); + } + } + + gobj = amdgpu_gem_prime_import(&adev->ddev, mem->dmabuf); + if (IS_ERR(gobj)) + return PTR_ERR(gobj); + + /* Import takes an extra reference on the dmabuf. Drop it now to + * avoid leaking it. We only need the one reference in + * kgd_mem->dmabuf. + */ + dma_buf_put(mem->dmabuf); + + *bo = gem_to_amdgpu_bo(gobj); + return 0; +} + /* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added @@ -658,8 +716,20 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, ret = kfd_mem_attach_userptr(adev, mem, &bo[i]); if (ret) goto unwind; + } else if (mem->domain == AMDGPU_GEM_DOMAIN_GTT && + mem->bo->tbo.type != ttm_bo_type_sg) { + /* GTT BOs use DMA-mapping ability of dynamic-attach + * DMA bufs. TODO: The same should work for VRAM on + * large-BAR GPUs. + */ + attachment[i]->type = KFD_MEM_ATT_DMABUF; + ret = kfd_mem_attach_dmabuf(adev, mem, &bo[i]); + if (ret) + goto unwind; } else { - /* FIXME: Need to DMA-map other BO types */ + /* FIXME: Need to DMA-map other BO types: + * large-BAR VRAM, doorbells, MMIO remap + */ attachment[i]->type = KFD_MEM_ATT_SHARED; bo[i] = mem->bo; drm_gem_object_get(&bo[i]->tbo.base); @@ -1558,6 +1628,8 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( } /* Free the BO*/ + if (mem->dmabuf) + dma_buf_put(mem->dmabuf); drm_gem_object_put(&mem->bo->tbo.base); mutex_destroy(&mem->lock); kfree(mem); From patchwork Wed Apr 14 06:48:03 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12202015 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07D1CC43461 for ; Wed, 14 Apr 2021 06:48:40 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C3994611F2 for ; Wed, 14 Apr 2021 06:48:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C3994611F2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 92BDE6E8E1; Wed, 14 Apr 2021 06:48:36 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2088.outbound.protection.outlook.com [40.107.92.88]) by gabe.freedesktop.org (Postfix) with ESMTPS id 7797E6E8DA; Wed, 14 Apr 2021 06:48:23 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=P1RvIhRi1FSIsDsro/TTRykaJ9+tIYSqNLXp/F7UfUleslHbREfyxpBsGS3ON+WBrRdxN69D5EVzJIeKLhZnRFpUmY2WEz1IPwpECVNAXnwZ6Uqw04sn8Iyo54gb7YveCusaCkx4RJjqf/i2LDHwnZ/CcGKnEVZabnLT7cTD6XQJN9Vz8vLVmofhDxLiwoY21Z3sEhYBU/sEZdaHx1CdT27cC9YVYZf/AG8/JXAgUpCSh2fSmVi1mJjsBWrBVkLo7Wx4gyAHZf5Fx6NVL2LUBYo4XEmQHqOB+2fycB7xU1q32LPa4h0IMnoti4eFxdJgB1z+Ih/y0DO/pXBqOO6Irg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eN2Yd6c53oyb+YR1k7QtiRCenmta+Oh7ACloZoJ7IWE=; b=oDzDpsOmyU47rUqhBhD1H+z/F5Jx2+WaBPEAqw6wznKNOagd6UXq5/iWN9TiEq6Wof2Gt04BxnG+uBJI5v7C1PNk5Hgzc8VufkmIBUgVh84iRs0DC12+i8VmLkSVxIgnnfUBxUPLjxiK3PdEfyB05RW/iSGgvYmKo1lyQ7Tv+fncqPIMvwK89G/wBDWvJcjVQmKOIckoEJzxEUDCO6/KNzoGwPN1WCmVMotAAnrTABBV5q5S1/fwXUItqekqV7WXePB4NP1+tygdzLDOVCikSVhaUJNGwdgbZIGNjsWUwYdaBz4NWRytJOo2OGLtQv6xRqyJ892Mg2aFHfnMlaUcUw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eN2Yd6c53oyb+YR1k7QtiRCenmta+Oh7ACloZoJ7IWE=; b=qU9+XYCgWDhd4rVb/dRnA9sV2P7duhKcTZUHKlsLOf9qJQ9qrX1uhW2BOq3tJ65L09+ldOphsaj9n5xCdQj5UJLiMsK2q5qUWXr0wdW+R2VVg/UMvbM7lEb1CLBSDS6+xft983uV02zR4ED3XKij2tG2QArr16e8jAbRbV/KZZ8= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:48:22 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:48:22 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 8/9] drm/ttm: Don't count pages in SG BOs against pages_limit Date: Wed, 14 Apr 2021 02:48:03 -0400 Message-Id: <20210414064804.29356-9-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064804.29356-1-Felix.Kuehling@amd.com> References: <20210414064804.29356-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Wed, 14 Apr 2021 06:48:21 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 49dc1e52-f6c3-4708-aec4-08d8ff114b41 X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2512; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 4W0aRDGcefFotVlrGoN+AYvP3gvCOMrqJH2lJkpHvww03MYLINFeJ6JHd8Rmi4s7/hEjzhYM7MIHO/GYgws8bXI+GbMIxe7ZSGIa6Jm6Dra27bIl/V4ny2uOCN1rQpM4TV/Mk8Fw3tvQG0+m1KhaoGMSREjBPrlR11ma7NwX7N2C/088ofT55Qh5bYeH+zVL7ZApmlq7dWS/KApM0PnJEiNEG5RIMG3eNwe4ZEZTo59vJQOT8pOvfJQehc7aJlNJnCoRQypJZJG/eCunnsm2MrIiQ4zYnesgam9NBhWr3sLRWa54BBie7zI0FST5s2U4FbV2cdEkyjnanmDuyO05iNabWqxNJabvqqptT1R8qVleCMHo9KyJnMj89hPmT8SlEB4cRukpfoxSe9HKsxerWffzfkMys5SbM1atHg8fzfPZM/wQl5DhiBsNa8f4RRyiDoXNJg2wPxzwGV6zQvT9P5wttVGSPnQRis2RPAbanYqXgGJIke7m7cP6/Dwhw+EeT0ru1eLe6hKSILa6LybPAZ+q8Ow1d3zBK+eyTYBkRHZy4E7dHT5kjwIB64UmU7JxqUMSeRdgIycB333Xg/f6W99X9PCy9aQe8XnLk9MbwrfjHGe5H7UlHdu9VzM3Odlp9SnvgMTNevNXHQNPo9DjPjhWiBDo3QEm0AZjXC6C1Lc= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(8936002)(86362001)(52116002)(38350700002)(66556008)(7696005)(6486002)(2906002)(1076003)(38100700002)(2616005)(36756003)(956004)(66476007)(26005)(316002)(478600001)(6666004)(450100002)(5660300002)(66946007)(4326008)(83380400001)(8676002)(186003)(16526019); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: FUg9EAuIl+2wdQckIE6azMSzbtBwTAvAp/PEPPgz6dBQgvolAiiVZ0Jj1lpSxIluwt63ziSczLTof2r7aEYXiBdvboebafDxj8KwGtjGnIzyGYvayB0zM5DdBxbFgTUdPobQO7HAKrd8gvk9BqGlkxp6NcTo+FWvCshLmgZWapTzHRmh7XTpO65hVxv7FD7/54BttgsxcPtlOAN4Plu30eH8rqEk1zQEbNU6JWBsi5XSBIX2Ialc3DPa67tk2R7X5pVmipe/D2MOH4XBXmVhd0STUBm8LIAKsNl7lZPCL9abH9EFp60oLb4vxvBgzxWzrlwYen+H/ps37XTaBn92UH7l07UeiqlNNiBkZ0GB5y2BW23LYJxAZovdXXo+Ni5jRWxgv8zBWmDFIrrBi5EVLqcG4HkAFUpKmTXsITlM7uwCXTYMHlUgb0Nl+78sKLfpF1c0ivny7QpzhwuJeeCCx0ieXIjpq8I19FG1mX/lHGj8Ldk5TlhS9oIPu5ajCho1bDqR36wH/5xJVgw+3HnnrTDjsqSjfmQXveLU1OIgzDJXKL58QEXyhWp/GDRWBbQ096DsKg75V8Y94K6WzxGKokPnYoE8WVsoNoNriqXRTCMcbzELed25d6kleqODfzTo741SGo9F6vLlVIY/aCqqgFs2SNua+WAKtocEsz1er+cYeFSXptRtrhKxxYD2qXdELUTQygqRXXDz43/6eTCEv9AzkiurAsjwIM3vkFsGrDhphFVK7V1GEl2hvDEHkzf6+Ghko5p3djEUXvWFPSruFqX/BgAMcZbWLx+qSWTQjtOlPC/e1eSFCly31Qyz+NuY1SYtHaiw8k82bqxzfxCGNHd2biLI6Dzzf3xC9oZ0IExGQCZmFw9msFcqV/9tNszTc6JaEwq2Ia8/Pho/g6/IujXI7w/RBHSqBbF5ULhPbHVN3uuC3uJfT9rbKLnafpHuGlRZ5IyvRKd5nsuTciZDl6+s4yFjcW1/3Dp+fjNPNPDokLV8zR8eCD8LoafEJvPI9OsDjcn0jyUziyVnDdnxjrgfU1CGk9yOCZ4shNucdOYqONImHQY0RsiPjVrNsjEf27/PyNkt7lD+8oL2/EKx/ataMRPpV+x1ZcL+WDiSsI3wVqFF/2fp8yRtr8tEr0IBy4AsWah4gH16ltnSqETBcwT/PRfkFHJw1gd4NXPtxQFYCFsPDavgHZa9Dt1lI8ciOvSHZ2RODaCbHqW+mxcxlZvpuxu4eicAcoYZn/Bb2JKQvNVWy8yCFeMO/DgXoHj7zKAeldPumzhzpXZDNpoa4nTqTpdRd3ufQQy2ueWBbQ/uUwxuN1UTme7bt8Qxg+6f X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 49dc1e52-f6c3-4708-aec4-08d8ff114b41 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:48:21.6680 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YLzqpZruHABrUbDKls/OzxrVlWCuDgJrOajk1e2hSI/c3/g8Ge7xNN7UbuQ3/lyxUTQZQ0LlK1JlSexP8ND5bg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Pages in SG BOs were not allocated by TTM. So don't count them against TTM's pages limit. Signed-off-by: Felix Kuehling Reviewed-by: Christian König --- drivers/gpu/drm/ttm/ttm_tt.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 5d8820725b75..e8b8c3257392 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -317,9 +317,12 @@ int ttm_tt_populate(struct ttm_device *bdev, if (ttm_tt_is_populated(ttm)) return 0; - atomic_long_add(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_add(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_add(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_add(ttm->num_pages, + &ttm_dma32_pages_allocated); + } while (atomic_long_read(&ttm_pages_allocated) > ttm_pages_limit || atomic_long_read(&ttm_dma32_pages_allocated) > @@ -350,9 +353,12 @@ int ttm_tt_populate(struct ttm_device *bdev, return 0; error: - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, + &ttm_dma32_pages_allocated); + } return ret; } EXPORT_SYMBOL(ttm_tt_populate); @@ -382,9 +388,12 @@ void ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm) else ttm_pool_free(&bdev->pool, ttm); - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, + &ttm_dma32_pages_allocated); + } ttm->page_flags &= ~TTM_PAGE_FLAG_PRIV_POPULATED; } From patchwork Wed Apr 14 06:48:04 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12202009 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 593E3C43470 for ; Wed, 14 Apr 2021 06:48:35 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 062A76101B for ; Wed, 14 Apr 2021 06:48:34 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 062A76101B Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8954C6E8E3; Wed, 14 Apr 2021 06:48:28 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2041.outbound.protection.outlook.com [40.107.92.41]) by gabe.freedesktop.org (Postfix) with ESMTPS id 53B916E8DF; Wed, 14 Apr 2021 06:48:25 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Kur9UMaaelJLOL7EzVR6RPynbpiO+MocKT7YqD28ID+JxNFQgQM3LTTDG7lNnkDB3ycnRkQeocQiLPyi9XAyPI3hFOLXa6CE88wqGh9VoOkorNt/5sqGEjf1LqZbhNrtKWEpRHyDMIMIO5T9pmUtRsauUW9hr6hzlEQBenDeC469Oz5IcmYoniHKqHBQak+wOqrGtEla3x1/cVQTmExYCcX5VV2UpoGANhbg0d7hIx9z8Kq4KsziEa1wRW4L5+F0MFuM2sYeuH2TDDjKhRxMJd65gU5PvvybkO5pW9SlAqQp7xojx22/Ro/9OO80pRwxR7jK5Avvf6t1ASEn2Gr22w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h6cYmo3dLmpw5Ddu0zE/rP1cYO7LSD0YBQ2wvpnQCJg=; b=AdAaxMZWvan3646yFPk0I9EbYetfkv7wOE+r1C5JuWAS3FQ5e+5DI7RxIv+dZufu/cIiNtPC2NTetOw74sKgGEr84Nwa7wvky1tG3sp9RCwiq0D9BSZq7jnnaWvty+pdxRQJjRVRI3SSS/hOHtGLFigHXVjK6E0DFgrYdPL0aHnZ4fPm6YlxdQ9HqgthGsEK/+nOkly0bevfzJzxIYLi8lhezHpLe3iLR8LC5k85p0LbcntVZLEu5lOnBe/e2VEQoej2eZLnhGSOKVEwqbvpj82BRmt4UJ24NPtvhDIVfm9jQz1XMXcpUE4FP1eMf8/5peIQaQ/RpKMjewaMEIHJbg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h6cYmo3dLmpw5Ddu0zE/rP1cYO7LSD0YBQ2wvpnQCJg=; b=K0Po7AWwV8wJ2KIfZPxu2yRC52B1iaHXtry8zkwNkfKDqgRGUhsbV5BQu94QrQzcr3WQifLCq++3MNLxjrJYL+Z/Qo73bOJ0HEIG9Nu7oc9zjaTE7Dl5ZOXKlgRMlMmEHeyijS+yk60WsoAxPDLdLBxAbTQPm25QDGpSVrzVids= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:48:23 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:48:23 +0000 From: Felix Kuehling To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 9/9] drm/amdgpu: Lock the attached dmabuf in unpopulate Date: Wed, 14 Apr 2021 02:48:04 -0400 Message-Id: <20210414064804.29356-10-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064804.29356-1-Felix.Kuehling@amd.com> References: <20210414064804.29356-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0073.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01:2d::12) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.22 via Frontend Transport; Wed, 14 Apr 2021 06:48:21 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9666222d-d848-4a6d-975e-08d8ff114b85 X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 8DBx3lVy86EW/6xeMzQ27ufKtB0VBpOmajuOpizLgkk0v1iiKiaipS0DnPaNBhPJi7Nlw10om0NNsXmMUmjAHHF3gF21hCOYNVojw5KEQqZBUJiDYOlxYHnDJebqg5T8KO0C9x+oWom6brNCrPq3TGRrcA0jRxX7M+k9LcdhvhNW1uzzxD6EB1gapho5e+KWOcON2kWOfZE5rXhIN8VH9EeGUkofR4BdhighwSTyaE8IDI1hcT2ZePfK904KGuoAm/cSiycdyFHv/tzJwirQJtndnho1nLFGgXaqiSe7Px1W0a7IIJ4QsRIVRF5mljUksoamllhBYkDRNWL3jZ/1JC9RwXzm/QQXgUubkoOJocfCHCsVJkFs+8lEqdizTlP0zhB3TPDvt6Ss+X4Sxkb5LGX+uN+uehZoYuptBrLCzDFwVsxphJjbrhqvuVyPyzLx8DVgUZlz5l8n+wcoHTZtI1ODQexZUxZsjtRCMzH5GsDIpq9HG6Rtx1HgbxK0MkQLl2o28fULHSsveHZ4BOQv5pRyYIzLadCAjuJyWNnqnkZ7uGAzUvF0UFhg/+ShIUvJNlJ7GjqbzYerYseG3PD5j0v+AQ0V55VO9Z578LZeY/G6nHWbA0DA05oa9+nUyoIj6IHLsqTmHzLiX8cvt691zQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(366004)(136003)(39860400002)(346002)(8936002)(86362001)(52116002)(38350700002)(66556008)(7696005)(6486002)(2906002)(1076003)(38100700002)(2616005)(36756003)(956004)(66476007)(26005)(316002)(478600001)(6666004)(450100002)(5660300002)(66946007)(4326008)(83380400001)(8676002)(186003)(16526019); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: pX8NAogUsKXEy5/lnD7ppwzTHnrHQakmr4mopnZRPUJq1MjogfgguYObENWOSqAInQHmfx2wTNs47eqPDhqpQ0oKDeWQjFWUhdDyMH1WDO67cHq0lPHLXUlp465aWstLw/QoJQ4x6iQAb53p7ExjsTQAWk9U/ezsgI0V3cB4nABRsgbrkHXWR64XmIQWfUCgDjUa8wZcnJjuMlbNkFHhR47QnmlqJhn7RtX6lHB4+17b5tzlqOvMcqjMdgq56yLZyI7PW0HgY4JCLmGBdbxSAiWgQ2G7cNZTTFjYFc5aGL8jluDNJknOsa7IX0gQyEkJch0vnGR6m4xc3JhLVtK81rIrSOUuXzxq85rWr8dw87DTMFj+1xPT/YiGBktlxD+GN/XNrO9CRbOPJ7HraULE9O8/PguibqqVIW34cJwN7Bjo6ms8kOhPYNRf30qa3PI7WEtyYC9Uk1ETlf7i1zK6+1EdD/GFnhzHQTn0bSByTI8W7lAUS2AxrDAJFDysMBH0CfCQhog+tab9BIYAk2NrMrTr2A7OpMxnYGlfN/4B/6jGsEIG+bWJu6TwSDsb8bwU/+1j0z7CPZQUHyek0iUqaHjvpTXdmNB2Yhiq7LhHnptuLUPZIRzt66Nu/MPeKYgwT/7ofRTAxdN9BBw7L2LJ4wkpzP1yb+UTjka2zZvVXAorqAmypGJqWQWV1AEkjXczGT73Zt1o1cRha3mMFi7meRz2lCEwDt8xDzfb81dp3wHpd4jBYWCL4UTv2LTU3MaMPDB/jKlqe48QJs7mQarNKY9ucEvRaJ4R3FhOoy5nCRX7zpi+GFdwXnWe0vSWu5TTA3Bfs1sKelTRt+6uqCMdSZhsx6Y785qH6It7QVYLj9r2X/hBwGgCcq+oV1veL5U/oZicCWarCQV2EU1SyalV+lUji9zd2TTPxtBUqBjEi+2Ly246XhEtgK5i7JnPNUI8EUL4yyT2qH4wupf8sxLhPEKzSahaiai3SVvLSfixGzvIk0gmaydA7q42C5M/GHKILC3Zz8kmgMdhVEPDnH5dCLb+NP5Gq8wA1RnMwxv/ZFiVaLBUo+38FxSESSreGN9n35Jtoel1PR2SeoUS18N0un/7TorVwZSxjii00BCxcOTMt50G6EBQklhO50cU0R5O+2UzNWybjBf7KJdkk4+nEXvqTqd0O4poe2fyvHlepfcxXVJ23VQkicxEsH/lAD7iUmut3FX9xTUlLiWd7SvApvs3tKgqvQLZBMxLpbINGq6Et7JeSROx/ITQKuWH5vCRJheb3xK/awmhibk8w0+ghFQIKViT0x2ViW5HHA0qEn8BwVDSTcv30FXvTIOQLK/d X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 9666222d-d848-4a6d-975e-08d8ff114b85 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:48:22.0484 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fB74wqi00tE3/HavHK9abDWbbcl20V14cwn5MWpiiLfi0bt4olojPF2q9zCoOIqF6Ie/WB2UlyqKFc5QKAhC9Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" amdgpu_ttm_tt_unpopulate can be called during bo_destroy. The dmabuf->resv must not be held by the caller or dma_buf_detach will deadlock. This is probably not the right fix. I get a recursive lock warning with the reservation held in ttm_bo_release. Should unmap_attachment move to backend_unbind instead? Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 936b3cfdde55..257750921eed 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -1216,9 +1216,22 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev, if (ttm->sg && gtt->gobj->import_attach) { struct dma_buf_attachment *attach; + bool locked; attach = gtt->gobj->import_attach; + /* FIXME: unpopulate can be called during bo_destroy. + * The dmabuf->resv must not be held by the caller or + * dma_buf_detach will deadlock. This is probably not + * the right fix. I get a recursive lock warning with the + * reservation held in ttm_bo_releas.. Should + * unmap_attachment move to backend_unbind instead? + */ + locked = dma_resv_is_locked(attach->dmabuf->resv); + if (!locked) + dma_resv_lock(attach->dmabuf->resv, NULL); dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL); + if (!locked) + dma_resv_unlock(attach->dmabuf->resv); ttm->sg = NULL; return; }