From patchwork Wed Apr 14 06:46:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201975 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E283BC433B4 for ; Wed, 14 Apr 2021 06:46:53 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9E0D66117A for ; Wed, 14 Apr 2021 06:46:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9E0D66117A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C8D0E6E8C1; Wed, 14 Apr 2021 06:46:52 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2074.outbound.protection.outlook.com [40.107.92.74]) by gabe.freedesktop.org (Postfix) with ESMTPS id 47D546E8C1 for ; Wed, 14 Apr 2021 06:46:51 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XeE9iv5Uq4PWlRSh5R+4TDRJUsM+Ndrg5Fx6LQnBlodM3zrl8n8LkHp0RRV3Gc1kWwp9hJj4arJaM4u7S3xabrSC8GvfyFZU6DsrnNK3mPnS68U123ydwB94OpsyDAlUFCzS6zUBsadz7YzS29onTDlTabGuovl3b8h8yzb7I/YmuJClwtWBcEWjsLR1+OONdeXvfIwetScl1U4hMrTaQH5l69ujlt1mOoK2Pp38mgz6t3oc8QKx2RIhgPgGcywaoQ2UqK3utj8BXGW1dCsDB9nv7GTJoc8pwNlbDznct+gZw02NlE3wK9oAtoH3JmRcCRI4kTtnP6uyGwe1HvsX6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2GUTVJD2QQLOcysdAZoVYPo8qqJJSUCAvbgkijjckEI=; b=MKN6T+XwbxnemNtCDYb4bcRnOHZL1UgpIJLyBh0t3Ip62Wfa6pwtKN6X5LDf0fBgs5K+w40IRuH01YljdWnwWjcqJbZLYOabcTEN0SQ93+89CPEctiHHCW4JylW4hlTOUyLtuZ0wSxLImUgMr1rGgKUpKltD8KifRw6+j54CVE8EWR6Fbc8HG6BUpMk5uEVTRkSGYbTviy0vsBHezkzVarfqWyXtMWQdnL/x0gOQerhZrNeEE0Z6Nyctkp4DrNWakprrK04fJ+7jZiGRzlvwb/M4uUrtroNa6eSp3GXHIJycsrLYf3CbUzY8BbS1KMK1G0p1BK8Yle4eOWixcjrzlg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2GUTVJD2QQLOcysdAZoVYPo8qqJJSUCAvbgkijjckEI=; b=T5/HYvC+MJioym0iUQOZCmqf+my34ZmeYSZAIobwdazy0cv305oVJJXVoUxpxtzW6+BfAh+NP7W/ns6TbqdsSzY3nkgXg+laY8Kuq2EXiuhJEFFcS1uAVzOpBYsUdNq5M+XcBd3p8aw4OInNh0iUyO7sQhbd118PbQGyVIXz/vI= Authentication-Results: lists-freedesktop.org; dkim=none (message not signed) header.d=none; lists-freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:46:49 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:46:49 +0000 From: Felix Kuehling To: amd-gfx@lists-freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 1/9] drm/amdgpu: Rename kfd_bo_va_list to kfd_mem_attachment Date: Wed, 14 Apr 2021 02:46:13 -0400 Message-Id: <20210414064621.29273-2-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064621.29273-1-Felix.Kuehling@amd.com> References: <20210414064621.29273-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Wed, 14 Apr 2021 06:46:48 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: c720ca02-ed50-408c-da5e-08d8ff1113f5 X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4714; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: MBN0NPKKxlm3LHQXjcWc5g0VuPsXTfnOiy43BacrxsnkRK5pqfR5FgZW0PwWfUQH/2YsIgFSbetLs+dwGvlACOxA6NbZ1bpxrqNhfcbgmbO1cOa567YXA4R3WLjgJfjH0Jo/5l3Yqym6rpng0PFcu9oho0gaGv88A8yPuXFwHGhmzAcG7UoqEPLuAKwot/EYHU+7w+jr0v7cnw2HqeDW9B1betSbblIU5xq3SqceegxNCaavKHdfDJz7i3XTU3YviJM0Vh0eo/AARx0zsgZLBiZHz7w2L9dQ9asxdz2AQWtVLkUZiL1luEYtUx+lJyy5Z7maVJEd71ZOVRjJjbdAS55Nbfgc6ILmCKRvbBkFRjQztxrV6JR/rTutirZTrVwhOx0gU3SU1y73Qmjfg0SHdr2U1NO//Ut7CTBB5v/LoA56tz6qUFR1OAOAAUHHQFZIQk/ZIjtD+A0m7FDMSng0A/LfKaPdbYpsi81xQSdo9nqRTiiCY24zp2QFjCKAPJtJNYdXElEnDRtlGRjRIoZRBtYChmTg8L/bPRcIwdu1T1IQToT6Wx/l8k7pBag7/uXl2LuMBaNzBWYdVpM3xofiJRcF+8H0J9/DB7myfh8OAOU0/Qkab8yN4K8Q/a9/6W977vHdYKNTDM4MKMNoBlzf7jopQIKw7uuPhtBXIIK9l6E= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(39860400002)(136003)(366004)(396003)(376002)(5660300002)(478600001)(6666004)(316002)(186003)(30864003)(16526019)(66946007)(8676002)(83380400001)(4326008)(6486002)(2906002)(66556008)(52116002)(38350700002)(8936002)(86362001)(7696005)(26005)(66476007)(36756003)(956004)(1076003)(38100700002)(2616005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: /m78RmUsnsi5y+PHBiFenJmftJePnIyjH446OArGb7pSEsfHDKskC51H/UsPhWS779xdNbWgWjuw23eOiP0Ib7hfEuc5kQLDw2o+z05BDodfqyEIsuSrvhIGsmvWM3iE0/Wew6QI7IvgR90EtRCtwftY19gKIMPzDjNWoajP0EvSxaZHhgxF109jRugVkirMTqWPCW2Z1MP7V06CfGqjKoqqv3NIw0JCXr60yHflC8+CvX91hJWI68HIlR7KONcPq+/Bt37KHnndOaOrxhb0sq/Sn7UNq5NV+QyGw8a8nKJ+MXxRYJMsp8DTJl0WV0RM7AmeM0nDjkayOzxN3JtziKzgm81yeTX0HRO7Z0n1BB9KqS0UiFFYouO63XsvTTM4et7WpCvDxhE0ag0Ja+OnDVPEKCDVZ4LcibiJv1VGNkn+WxQP3UsX9qTae0kXnfq7li1Xa1XGSrHm67LVX3uDdV/3v12lMfcqkoia6sy2yBmkREi5F8kk64xEpFOIG58wKo+BIPLBdV78GQe7QmlnOh25c8CTbgcH2TReqqRn2WX5POQ87qEM36TPhk3RVpxmL3nQ5QJ9cXhIqaEr2nHKifeH5EHzHa9rS8CM27LW9bWcBEUmp9wz/tU0zOSzQdVMxiwE3P1lmjnZ6JAH3I2ypI/De7GErz7v2FEBHPsdlaqt9TtuXr+TpWe20puzieZXhEt975ZtvKHJRm7fa+qDYqxO73NWZktkhsXxbRmvk2LQOfukc7ihhJr/ggy1n0Vs4w75gs9v9h+S28+DAdxxdyvRv/crWpMdSSP8XhFD2vOndaeEgAcARf4VfcVgoqPHEReFa8p03y8qLCruv3OyS7mjwTWs4827f60hmwD9Y2dg7hsm6uaaNCjONIKmB/cxygk7vRmmf6eqZr4VF9SDUpOQmiNM9cxyiGPBANKoNymQLBZaUSOnoewcIjQfptvHBmcIAEJQnGQuCyuo0gMLhL/tQgrxiEg3CgRJb05nBxwcHV+QE/Fb5PnPDc/dLJGgITyC8tiLt8c6n90NNMR31Ji/odmfv0eIHSEmt8M7P7O67VKF+FJ5WuuUvnH+xiyxz7qNndCba+eDYkQqmgFCd4LYcLW8K4VCC3F507dXY7pscwRnXC2KV4ZRfxR5VobeXkwIrxWqG8clXDodJ6tW1sJ1NbRKlocujjM4+NdGgdkhZ+lm4Lyjp5he1nDJzayt0T3yEd0FzsrQP4QrS6thYeOsLKMUh+RTRUDfHZ94cPeij+l4o3JhMfbciufMAtLnS0yRAT8E690TconC0G3rPLR4PPvUGLxPP7Fv7T1SYdOaj0031FnypPNps83mi3zw X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: c720ca02-ed50-408c-da5e-08d8ff1113f5 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:46:48.8645 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: wPfpi+tr1UZKGEuLNKteko/yarzBRBU3Nn21k7Jhog+esK3uVyPygJ5kPQlWDrVUwEugHV4vbTIXmUI5Aakr3Q== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This name is more fitting, especially for the changes coming next to support multi-GPU systems with proper DMA mappings. Cleaned up the code and renamed some related functions and variables to improve readability. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 8 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 209 +++++++++--------- 2 files changed, 104 insertions(+), 113 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 14f68c028126..910c50956e16 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -38,10 +38,10 @@ extern uint64_t amdgpu_amdkfd_total_mem_size; struct amdgpu_device; -struct kfd_bo_va_list { - struct list_head bo_list; +struct kfd_mem_attachment { + struct list_head list; struct amdgpu_bo_va *bo_va; - void *kgd_dev; + struct amdgpu_device *adev; bool is_mapped; uint64_t va; uint64_t pte_flags; @@ -50,7 +50,7 @@ struct kfd_bo_va_list { struct kgd_mem { struct mutex lock; struct amdgpu_bo *bo; - struct list_head bo_va_list; + struct list_head attachments; /* protected by amdkfd_process_info.lock */ struct ttm_validate_buffer validate_list; struct ttm_validate_buffer resv_list; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 6622695a5eed..d021152314ad 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -75,16 +75,16 @@ static inline struct amdgpu_device *get_amdgpu_device(struct kgd_dev *kgd) return (struct amdgpu_device *)kgd; } -static bool check_if_add_bo_to_vm(struct amdgpu_vm *avm, +static bool kfd_mem_is_attached(struct amdgpu_vm *avm, struct kgd_mem *mem) { - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; - list_for_each_entry(entry, &mem->bo_va_list, bo_list) + list_for_each_entry(entry, &mem->attachments, list) if (entry->bo_va->base.vm == avm) - return false; + return true; - return true; + return false; } /* Set memory usage limits. Current, limits are @@ -471,7 +471,7 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) return pte_flags; } -/* add_bo_to_vm - Add a BO to a VM +/* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added * to a VM. It can later be mapped and unmapped many times without @@ -483,15 +483,14 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) * 4. Alloc page tables and directories if needed * 4a. Validate new page tables and directories */ -static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, +static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_vm *vm, bool is_aql, - struct kfd_bo_va_list **p_bo_va_entry) + struct kfd_mem_attachment **p_attachment) { int ret; - struct kfd_bo_va_list *bo_va_entry; + struct kfd_mem_attachment *attachment; struct amdgpu_bo *bo = mem->bo; uint64_t va = mem->va; - struct list_head *list_bo_va = &mem->bo_va_list; unsigned long bo_size = bo->tbo.base.size; if (!va) { @@ -502,29 +501,29 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, if (is_aql) va += bo_size; - bo_va_entry = kzalloc(sizeof(*bo_va_entry), GFP_KERNEL); - if (!bo_va_entry) + attachment = kzalloc(sizeof(*attachment), GFP_KERNEL); + if (!attachment) return -ENOMEM; pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, va + bo_size, vm); /* Add BO to VM internal data structures*/ - bo_va_entry->bo_va = amdgpu_vm_bo_add(adev, vm, bo); - if (!bo_va_entry->bo_va) { + attachment->bo_va = amdgpu_vm_bo_add(adev, vm, bo); + if (!attachment->bo_va) { ret = -EINVAL; pr_err("Failed to add BO object to VM. ret == %d\n", ret); goto err_vmadd; } - bo_va_entry->va = va; - bo_va_entry->pte_flags = get_pte_flags(adev, mem); - bo_va_entry->kgd_dev = (void *)adev; - list_add(&bo_va_entry->bo_list, list_bo_va); + attachment->va = va; + attachment->pte_flags = get_pte_flags(adev, mem); + attachment->adev = adev; + list_add(&attachment->list, &mem->attachments); - if (p_bo_va_entry) - *p_bo_va_entry = bo_va_entry; + if (p_attachment) + *p_attachment = attachment; /* Allocate validate page tables if needed */ ret = vm_validate_pt_pd_bos(vm); @@ -536,22 +535,20 @@ static int add_bo_to_vm(struct amdgpu_device *adev, struct kgd_mem *mem, return 0; err_alloc_pts: - amdgpu_vm_bo_rmv(adev, bo_va_entry->bo_va); - list_del(&bo_va_entry->bo_list); + amdgpu_vm_bo_rmv(adev, attachment->bo_va); + list_del(&attachment->list); err_vmadd: - kfree(bo_va_entry); + kfree(attachment); return ret; } -static void remove_bo_from_vm(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, unsigned long size) +static void kfd_mem_detach(struct kfd_mem_attachment *attachment) { - pr_debug("\t remove VA 0x%llx - 0x%llx in entry %p\n", - entry->va, - entry->va + size, entry); - amdgpu_vm_bo_rmv(adev, entry->bo_va); - list_del(&entry->bo_list); - kfree(entry); + pr_debug("\t remove VA 0x%llx in entry %p\n", + attachment->va, attachment); + amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); + list_del(&attachment->list); + kfree(attachment); } static void add_kgd_mem_to_kfd_bo_list(struct kgd_mem *mem, @@ -726,7 +723,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, struct bo_vm_reservation_context *ctx) { struct amdgpu_bo *bo = mem->bo; - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; unsigned int i; int ret; @@ -738,7 +735,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, INIT_LIST_HEAD(&ctx->list); INIT_LIST_HEAD(&ctx->duplicates); - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { + list_for_each_entry(entry, &mem->attachments, list) { if ((vm && vm != entry->bo_va->base.vm) || (entry->is_mapped != map_type && map_type != BO_VM_ALL)) @@ -760,7 +757,7 @@ static int reserve_bo_and_cond_vms(struct kgd_mem *mem, list_add(&ctx->kfd_bo.tv.head, &ctx->list); i = 0; - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { + list_for_each_entry(entry, &mem->attachments, list) { if ((vm && vm != entry->bo_va->base.vm) || (entry->is_mapped != map_type && map_type != BO_VM_ALL)) @@ -815,7 +812,7 @@ static int unreserve_bo_and_vms(struct bo_vm_reservation_context *ctx, } static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, + struct kfd_mem_attachment *entry, struct amdgpu_sync *sync) { struct amdgpu_bo_va *bo_va = entry->bo_va; @@ -831,7 +828,7 @@ static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, } static int update_gpuvm_pte(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, + struct kfd_mem_attachment *entry, struct amdgpu_sync *sync) { int ret; @@ -848,7 +845,7 @@ static int update_gpuvm_pte(struct amdgpu_device *adev, } static int map_bo_to_gpuvm(struct amdgpu_device *adev, - struct kfd_bo_va_list *entry, struct amdgpu_sync *sync, + struct kfd_mem_attachment *entry, struct amdgpu_sync *sync, bool no_update_pte) { int ret; @@ -1235,7 +1232,7 @@ int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu( ret = -ENOMEM; goto err; } - INIT_LIST_HEAD(&(*mem)->bo_va_list); + INIT_LIST_HEAD(&(*mem)->attachments); mutex_init(&(*mem)->lock); (*mem)->aql_queue = !!(flags & KFD_IOC_ALLOC_MEM_FLAGS_AQL_QUEUE_MEM); @@ -1316,7 +1313,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( { struct amdkfd_process_info *process_info = mem->process_info; unsigned long bo_size = mem->bo->tbo.base.size; - struct kfd_bo_va_list *entry, *tmp; + struct kfd_mem_attachment *entry, *tmp; struct bo_vm_reservation_context ctx; struct ttm_validate_buffer *bo_list_entry; unsigned int mapped_to_gpu_memory; @@ -1360,9 +1357,8 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( mem->va + bo_size * (1 + mem->aql_queue)); /* Remove from VM internal data structures */ - list_for_each_entry_safe(entry, tmp, &mem->bo_va_list, bo_list) - remove_bo_from_vm((struct amdgpu_device *)entry->kgd_dev, - entry, bo_size); + list_for_each_entry_safe(entry, tmp, &mem->attachments, list) + kfd_mem_detach(entry); ret = unreserve_bo_and_vms(&ctx, false, false); @@ -1404,10 +1400,10 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( int ret; struct amdgpu_bo *bo; uint32_t domain; - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; - struct kfd_bo_va_list *bo_va_entry = NULL; - struct kfd_bo_va_list *bo_va_entry_aql = NULL; + struct kfd_mem_attachment *attachment = NULL; + struct kfd_mem_attachment *attachment_aql = NULL; unsigned long bo_size; bool is_invalid_userptr = false; @@ -1456,21 +1452,20 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( bo->tbo.mem.mem_type == TTM_PL_SYSTEM) is_invalid_userptr = true; - if (check_if_add_bo_to_vm(avm, mem)) { - ret = add_bo_to_vm(adev, mem, avm, false, - &bo_va_entry); + if (!kfd_mem_is_attached(avm, mem)) { + ret = kfd_mem_attach(adev, mem, avm, false, &attachment); if (ret) - goto add_bo_to_vm_failed; + goto attach_failed; if (mem->aql_queue) { - ret = add_bo_to_vm(adev, mem, avm, - true, &bo_va_entry_aql); + ret = kfd_mem_attach(adev, mem, avm, true, + &attachment_aql); if (ret) - goto add_bo_to_vm_failed_aql; + goto attach_failed_aql; } } else { ret = vm_validate_pt_pd_bos(avm); if (unlikely(ret)) - goto add_bo_to_vm_failed; + goto attach_failed; } if (mem->mapped_to_gpu_memory == 0 && @@ -1486,30 +1481,30 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( } } - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { - if (entry->bo_va->base.vm == vm && !entry->is_mapped) { - pr_debug("\t map VA 0x%llx - 0x%llx in entry %p\n", - entry->va, entry->va + bo_size, - entry); + list_for_each_entry(entry, &mem->attachments, list) { + if (entry->bo_va->base.vm != vm || entry->is_mapped) + continue; - ret = map_bo_to_gpuvm(adev, entry, ctx.sync, - is_invalid_userptr); - if (ret) { - pr_err("Failed to map bo to gpuvm\n"); - goto map_bo_to_gpuvm_failed; - } + pr_debug("\t map VA 0x%llx - 0x%llx in entry %p\n", + entry->va, entry->va + bo_size, entry); - ret = vm_update_pds(vm, ctx.sync); - if (ret) { - pr_err("Failed to update page directories\n"); - goto map_bo_to_gpuvm_failed; - } + ret = map_bo_to_gpuvm(adev, entry, ctx.sync, + is_invalid_userptr); + if (ret) { + pr_err("Failed to map bo to gpuvm\n"); + goto map_bo_to_gpuvm_failed; + } - entry->is_mapped = true; - mem->mapped_to_gpu_memory++; - pr_debug("\t INC mapping count %d\n", - mem->mapped_to_gpu_memory); + ret = vm_update_pds(vm, ctx.sync); + if (ret) { + pr_err("Failed to update page directories\n"); + goto map_bo_to_gpuvm_failed; } + + entry->is_mapped = true; + mem->mapped_to_gpu_memory++; + pr_debug("\t INC mapping count %d\n", + mem->mapped_to_gpu_memory); } if (!amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) && !bo->tbo.pin_count) @@ -1521,12 +1516,12 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( goto out; map_bo_to_gpuvm_failed: - if (bo_va_entry_aql) - remove_bo_from_vm(adev, bo_va_entry_aql, bo_size); -add_bo_to_vm_failed_aql: - if (bo_va_entry) - remove_bo_from_vm(adev, bo_va_entry, bo_size); -add_bo_to_vm_failed: + if (attachment_aql) + kfd_mem_detach(attachment_aql); +attach_failed_aql: + if (attachment) + kfd_mem_detach(attachment); +attach_failed: unreserve_bo_and_vms(&ctx, false, false); out: mutex_unlock(&mem->process_info->lock); @@ -1541,7 +1536,7 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( struct amdkfd_process_info *process_info = ((struct amdgpu_vm *)vm)->process_info; unsigned long bo_size = mem->bo->tbo.base.size; - struct kfd_bo_va_list *entry; + struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; int ret; @@ -1565,26 +1560,24 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( mem->va + bo_size * (1 + mem->aql_queue), vm); - list_for_each_entry(entry, &mem->bo_va_list, bo_list) { - if (entry->bo_va->base.vm == vm && entry->is_mapped) { - pr_debug("\t unmap VA 0x%llx - 0x%llx from entry %p\n", - entry->va, - entry->va + bo_size, - entry); + list_for_each_entry(entry, &mem->attachments, list) { + if (entry->bo_va->base.vm != vm || !entry->is_mapped) + continue; - ret = unmap_bo_from_gpuvm(adev, entry, ctx.sync); - if (ret == 0) { - entry->is_mapped = false; - } else { - pr_err("failed to unmap VA 0x%llx\n", - mem->va); - goto unreserve_out; - } + pr_debug("\t unmap VA 0x%llx - 0x%llx from entry %p\n", + entry->va, entry->va + bo_size, entry); - mem->mapped_to_gpu_memory--; - pr_debug("\t DEC mapping count %d\n", - mem->mapped_to_gpu_memory); + ret = unmap_bo_from_gpuvm(adev, entry, ctx.sync); + if (ret == 0) { + entry->is_mapped = false; + } else { + pr_err("failed to unmap VA 0x%llx\n", mem->va); + goto unreserve_out; } + + mem->mapped_to_gpu_memory--; + pr_debug("\t DEC mapping count %d\n", + mem->mapped_to_gpu_memory); } /* If BO is unmapped from all VMs, unfence it. It can be evicted if @@ -1726,7 +1719,7 @@ int amdgpu_amdkfd_gpuvm_import_dmabuf(struct kgd_dev *kgd, if (mmap_offset) *mmap_offset = amdgpu_bo_mmap_offset(bo); - INIT_LIST_HEAD(&(*mem)->bo_va_list); + INIT_LIST_HEAD(&(*mem)->attachments); mutex_init(&(*mem)->lock); (*mem)->alloc_flags = @@ -1923,7 +1916,7 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) list_for_each_entry_safe(mem, tmp_mem, &process_info->userptr_inval_list, validate_list.head) { - struct kfd_bo_va_list *bo_va_entry; + struct kfd_mem_attachment *attachment; bo = mem->bo; @@ -1946,13 +1939,13 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) * VM faults if the GPU tries to access the invalid * memory. */ - list_for_each_entry(bo_va_entry, &mem->bo_va_list, bo_list) { - if (!bo_va_entry->is_mapped) + list_for_each_entry(attachment, &mem->attachments, list) { + if (!attachment->is_mapped) continue; ret = update_gpuvm_pte((struct amdgpu_device *) - bo_va_entry->kgd_dev, - bo_va_entry, &sync); + attachment->adev, + attachment, &sync); if (ret) { pr_err("%s: update PTE failed\n", __func__); /* make sure this gets validated again */ @@ -2133,7 +2126,7 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) struct amdgpu_bo *bo = mem->bo; uint32_t domain = mem->domain; - struct kfd_bo_va_list *bo_va_entry; + struct kfd_mem_attachment *attachment; total_size += amdgpu_bo_size(bo); @@ -2153,11 +2146,9 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) pr_debug("Memory eviction: Sync BO fence failed. Try again\n"); goto validate_map_fail; } - list_for_each_entry(bo_va_entry, &mem->bo_va_list, - bo_list) { + list_for_each_entry(attachment, &mem->attachments, list) { ret = update_gpuvm_pte((struct amdgpu_device *) - bo_va_entry->kgd_dev, - bo_va_entry, + attachment->adev, attachment, &sync_obj); if (ret) { pr_debug("Memory eviction: update PTE failed. Try again\n"); @@ -2232,7 +2223,7 @@ int amdgpu_amdkfd_add_gws_to_process(void *info, void *gws, struct kgd_mem **mem return -ENOMEM; mutex_init(&(*mem)->lock); - INIT_LIST_HEAD(&(*mem)->bo_va_list); + INIT_LIST_HEAD(&(*mem)->attachments); (*mem)->bo = amdgpu_bo_ref(gws_bo); (*mem)->domain = AMDGPU_GEM_DOMAIN_GWS; (*mem)->process_info = process_info; From patchwork Wed Apr 14 06:46:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201977 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9615CC433ED for ; Wed, 14 Apr 2021 06:46:56 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 51BB4613C1 for ; Wed, 14 Apr 2021 06:46:56 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 51BB4613C1 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0AE296E8C0; Wed, 14 Apr 2021 06:46:53 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2074.outbound.protection.outlook.com [40.107.92.74]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1D5416E8C0 for ; Wed, 14 Apr 2021 06:46:51 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=JcDIwxHEBbbUdN8EyYG+Gt3zLpD9jLOug5pRUlHnFiZQv7AX+q1FfZVImmbOb61l9Q7I8Otwxx3DZBX0Sply8JHRnUtyK1/Vt8Ke7s+qSvttum5Hp3tLiZXBLlIDr3y3E6eQBSKQpR0NsPxM7MWOwd4C+BeBYH4+LOzpCCDXu/AvX31vv8Ov1FqXsSbMZ+Ar8Wn+yR6vit10N5G/OrJnO5d4n5wgpiqU7WRyVo/hhZnuIqdaNMhi243VLDQOyCxg5rpgm0/Pn6vGJdTeJBSMsa02wgIz0eyybR7gAwwSqj50okR3kZ5gBocrcJMj0Gala5paSgLCShWznEXyb4daJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XUcSThGOmgW2TkEvH03AiiupJoZbHEnbE0vI9a2WclU=; b=ILeyx4xuflXgIamAATIiseQ/qFD8oqjcWX6knuSPJLDnwxszW9HQFL//PS+d1wE8YZCauQRuM50ttr70/phiUYMHMmBQfCJiEjMdB5QjoXJzrZTJiGL5iYrjYltPM+fzi8pYt5JjSLPSe2hOy3dgNT5YoH7EVVjzK52f5UJctZpfnh7M5sOIqYg1A1uJskXmxrLJVrNTCUXlXbUqw2+qNQnt9WSPsRADYhVb5+rQQ8xLMBG0jgd6sgw+Zlf4x1KOSIvCLyYXG2y9eovc64o2Rv2HMIL0eFqZT1PcOaIrYoHrQGQD6zDINPSZdbS772J8iR5BRxLMAW2x/XkNXtSsjg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XUcSThGOmgW2TkEvH03AiiupJoZbHEnbE0vI9a2WclU=; b=HngMhq/qri/nhAf89Q6CnZIYVYSP3RPr/UV6EwWhu7o/2kiWaMvakw52KdN5K4v2/Jhg04P8SgKlfacNeE34YB/m5E6EvhguVyQmtrTYDeoaLGj2WIE7gOjndpc51Or0LGAgEHLbGVgVkEjO6j348xuUIOQmRJkuYge/cQFDYFI= Authentication-Results: lists-freedesktop.org; dkim=none (message not signed) header.d=none; lists-freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:46:49 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:46:49 +0000 From: Felix Kuehling To: amd-gfx@lists-freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 2/9] drm/amdgpu: Keep a bo-reference per-attachment Date: Wed, 14 Apr 2021 02:46:14 -0400 Message-Id: <20210414064621.29273-3-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064621.29273-1-Felix.Kuehling@amd.com> References: <20210414064621.29273-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Wed, 14 Apr 2021 06:46:49 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0e39de6d-5ee3-4ac4-8f40-08d8ff11143d X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: S+H8L41uGw0UBCoFEkJVGW6O3KJ2B6ZSu3rXDX8/z9w5aQSq1mRPv8/NS6IQSkONNQ0fXfHdebWTzQMbhgf976dPZVe3q+s7Z2gWo6qh+Z99pelw7xtZVbNXKbKx/+GLBZNJz9Bp5vRG5JH5BZL3MkoSdHDrJsA5QRXrnStaWLnQTVkiJg2B5DeaSPT++Z+k650gNoJGNTP7QGTUfdoJOyoEwUMKlVFV3p8xNYdRR2mtzb5lDS++dvUTZ0OEJTUQdJebZViQXIgHTxNkPyJN7mqG0DrUYBDdC7/N2W4/PIGTALsqzk2YmMo9wE3btxUhl095ZNf87Y9+vpVdQX0GP3ERr6+ZlhpvGWRFF8PEgHEm+ErKEVz6o7CKc32aq965ZRrlxir0a/QQftXKNjTe4drE28hpfBmJ/uiBLgLjzQm7SKsDIhhDBww3OGroafWZX4uJMEiqxdciZ/2EhvcY9jKEj/KfjRfUZrNMK3BOoMUQiob5z3vA84i356PbhyamQPKZf1CeV9rHzp8mDZp+TXdK+CCEaN65/uIHaATrmJ/fAZvjeCZcgymxBVbrl9C8ovteOWZCQFggPxeoZONNrlW2SKEm+lUNFnsXi++LRXBLrHvCPDXP1eh8pzU4C+TbffvQQtO0aZiv9ohW6wQljVJLPuS9m37QklL/AVc35k4= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(39860400002)(136003)(366004)(396003)(376002)(5660300002)(478600001)(6666004)(316002)(186003)(16526019)(66946007)(8676002)(83380400001)(4326008)(6486002)(2906002)(66556008)(52116002)(38350700002)(8936002)(86362001)(7696005)(26005)(66476007)(36756003)(956004)(1076003)(38100700002)(2616005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: f+sj+FsCrxXnWyyhkR7Oe0SNepqfiyxeRbr+o0j2wv6REAizNB6LilCIfWY8SY26H6aFVovGaWx39/k0JJZJxoCz1Y1qdjW+a22yNSH8H4QqBSXGt5kZOWOwjp+Hie+zjR13gK2hgo1vrgzAHj8kUb727FnpUF0uuhbnYY1ij49dlUJ6IpmkFCJAZ3IAKQfn9inLc1k70pe0uvUPYq8icWTIoLbCWXyHeeHmswPYL3bhRQbr6Q2wYPSkM5yRzimZ9mD93bbm3zGfLmXcIE/AMN9fCGdgqtJSWFVdmDBy2e4zOd8Meb7jwff+p+cZxEwUJJg5TPCoHE1g/v+1i2c/1LX/Gxkj7YhGoCGWx5wq9PDj/FZGh52HYp2Fd75zAJ6/gRHTUL8A8z3q0ZH956cvarSx4N1PYlh7uPmyGL9gORXyp7Mc88QQ/2OZJVsY2xvB776blu8Dk7oYffuaXUYt7vxOF4Sw7b0uA4rIEDLK4CtRpKXkmv5XEspemqRs1ytu2RAUNkK0tq+PChB/SeY+ieh54eMHn1pMg07940p8hhVbrx0XQ82syZgTBYlNVm8edamSyFI+AOfgAik+f+LkLXg5+Cr2lexVt0mL0kmJp82PUFebdeIZynKbaULpUTHhdsbomEGCCls8k0g3vaoQes32dSoCi3EnzNIudBHkQ3nQT/oicTjWNFbV9Y8oLtnC68alx2n5uSjoO/HER/RfpH9KtEiVaUHpHZLFaROIzbDaW83nLus5goX/sZyLDULEde/71Tpw8V1VI3aVQSVvplyHtAmjiNSVwqVseCJFPcYiMibP9rCfyuD7tj98Eiwj3Q0hPvl4io3UIuEjSLUL2WdAIE4QFZCy3xP6fRPyKTxm1t565Nm6Yz9gZmklD2yMcS5LAJ42TfndwqwmVNxWLjNze7MG0mNa5yWCF0W9X/1cYc+N+AXxFiW7znC8vhwfLg3VTfXP2kdYwflwbEuOyOe7T5COeULL65CBTzg1YV8Gtq9+MUw9uDvCXabPLBT605aL4FDQGk3VXqwbwL5pZU7h1696dgV87tQLbVwaJ1wCgPvF42bjxNgdYvosG3gOoNpuVS4NNzGY4ngZ4L8qKnrKrHm0CWisJs+2EReFZYwe25Ah1bwwt79LcYVEGGedhcpXbMjOLkOGnZ/hngcUcDd0bC6iYOGYgCz+orYRYzBePvWjs1gJRYs8Exw6W225SKVDxiKBGsxsvi8Tl2iOE+J2io7QL+qjPh3ZZovMbMB5vDv72aZX7eEt/sIgsgtUEfs+IYQTyA9NSBRIPm13z+dtUD8kud9jCPsJfm8PNrzPQw/PuY2L/893hCswt0zg X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0e39de6d-5ee3-4ac4-8f40-08d8ff11143d X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:46:49.3096 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: x1TdZt9TZbYzOc9WA+NeTlHcOGBW3mp/hb18wAoPXsVxAnfJC8RAH2rbA3g/+P3hp+MZROQ5mhhvi/mWCmbXrw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" For now they all reference the same BO. For correct DMA mappings they will refer to different BOs per-GPU. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 22 ++++++++++++++----- 1 file changed, 17 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index d021152314ad..40a296ca37b9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -487,11 +487,11 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_vm *vm, bool is_aql, struct kfd_mem_attachment **p_attachment) { - int ret; - struct kfd_mem_attachment *attachment; - struct amdgpu_bo *bo = mem->bo; + unsigned long bo_size = mem->bo->tbo.base.size; uint64_t va = mem->va; - unsigned long bo_size = bo->tbo.base.size; + struct kfd_mem_attachment *attachment; + struct amdgpu_bo *bo; + int ret; if (!va) { pr_err("Invalid VA when adding BO to VM\n"); @@ -508,6 +508,14 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, va + bo_size, vm); + /* FIXME: For now all attachments use the same BO. This is incorrect + * because one BO can only have one DMA mapping for one GPU. We need + * one BO per GPU, e.g. a DMABuf import with dynamic attachment. This + * will be addressed one BO-type at a time in subsequent patches. + */ + bo = mem->bo; + drm_gem_object_get(&bo->tbo.base); + /* Add BO to VM internal data structures*/ attachment->bo_va = amdgpu_vm_bo_add(adev, vm, bo); if (!attachment->bo_va) { @@ -527,7 +535,7 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, /* Allocate validate page tables if needed */ ret = vm_validate_pt_pd_bos(vm); - if (ret) { + if (unlikely(ret)) { pr_err("validate_pt_pd_bos() failed\n"); goto err_alloc_pts; } @@ -538,15 +546,19 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, amdgpu_vm_bo_rmv(adev, attachment->bo_va); list_del(&attachment->list); err_vmadd: + drm_gem_object_put(&bo->tbo.base); kfree(attachment); return ret; } static void kfd_mem_detach(struct kfd_mem_attachment *attachment) { + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + pr_debug("\t remove VA 0x%llx in entry %p\n", attachment->va, attachment); amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); + drm_gem_object_put(&bo->tbo.base); list_del(&attachment->list); kfree(attachment); } From patchwork Wed Apr 14 06:46:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201981 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id C862BC433B4 for ; Wed, 14 Apr 2021 06:47:01 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8C3896117A for ; Wed, 14 Apr 2021 06:47:01 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8C3896117A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 4EC186E8C2; Wed, 14 Apr 2021 06:46:55 +0000 (UTC) Received: from NAM10-BN7-obe.outbound.protection.outlook.com (mail-bn7nam10on2074.outbound.protection.outlook.com [40.107.92.74]) by gabe.freedesktop.org (Postfix) with ESMTPS id A19426E8C1 for ; Wed, 14 Apr 2021 06:46:51 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=bwXCNcyhBGCtSfkxO5/TCbG+zn/QiCnm7NXCovp+ePFnSbO5j8ogqcerqJdaucuNHJEAOrGSZB3Aj/ZCvcgZJWvhprLVpwzyjY+YDyGOGAD5WBzEFrBM+5JYZa2YRpoV3Ld6r6LaKj0tZyC5u+vLYDKxpw2aR7RJxcSj0NimoK/oX/MgPf2b0xyjspTDDH10qTs/un1/IdRTiZEb8olnk/1P0aBwxP4gta3/vW2FrYAh21CnvUtROADglGZqukXVv94I0tjcfTBbgcSX9u2WYVoJpsYDzzprulBxC3wI6ZcSbAgVcV2arNcYBhOHIGt1jV3rN1U5YNd2AOdxAy6AHg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J4f6qiDMF3EvXZF3caFnVdL4tzzrONpDOHzq7h2Z3dM=; b=gah2OkSe1GEG27Y+qkXADW/ASeloE/uIx6Gm1YhUJkckw9KtVk4d6+PJs72gaFrBSHco0CrLUCxakQQWDLwLG1iQsUEUes6pi+8uKSCVefpKVDiAgCxoDM2OlLiZGVd1mXYfq/fEYmhf8m4YTA//LYyHTIKMQ9Nl1ZwkQdMRM/Ki5VVAbD/WLSwz9NCaLwkK4AdlTNSCuSEpZgc3RFQHvvHNAkqhfqivTQ/+WO9RGwt/W7bNFsM1faasO2I0V+wMM3N6swRlneAiwS7dwtETcdEX6ISlpmcY20SJ9ok9vyTfemnXRBm7rTtpGCX9YCQJNPY7/eAvUe0zbAoQwt/t4w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=J4f6qiDMF3EvXZF3caFnVdL4tzzrONpDOHzq7h2Z3dM=; b=1b8g2BUo2eDF1xbDPgbeH6uzsJAhUMrpPZTyiDy2KmQlQH/vTZ89izqFwWC/AZ08EvffWqoumhT3AQhtpp8bj8+R5GgHhQj8bgEb1vb7Qffwu6H+K+7Fd9y4tcgpxGr1RmbdMj5pipyoefCERJb5IQixbtApUw8cm5kCTkrHhmw= Authentication-Results: lists-freedesktop.org; dkim=none (message not signed) header.d=none; lists-freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by MN2PR12MB4125.namprd12.prod.outlook.com (2603:10b6:208:1d9::9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21; Wed, 14 Apr 2021 06:46:49 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:46:49 +0000 From: Felix Kuehling To: amd-gfx@lists-freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 3/9] drm/amdgpu: Simplify AQL queue mapping Date: Wed, 14 Apr 2021 02:46:15 -0400 Message-Id: <20210414064621.29273-4-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064621.29273-1-Felix.Kuehling@amd.com> References: <20210414064621.29273-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Wed, 14 Apr 2021 06:46:49 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 06985e79-7024-4a8f-4a20-08d8ff111477 X-MS-TrafficTypeDiagnostic: MN2PR12MB4125: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eMAr7EYtA6UaAkIgbCjSWVo7uyZdRGQGVDMWoaHU8GvZ+XWLo1skKE9lO9abLVdRNQ7FMmlOg+TneNVb+dluf6EFQEPbm5J8m5CLcgKTiSaT6KDeF9x+ehQbsFLFgsz3s2qJRTI7jomcs3NSZHGpwD1iJRB4YzeQ/NjQImg9FTeQCe/+OVjmKwV0CD+9WtywVMscaQc7M9rWtNXEt2LsaYkxOJR7sNnySV9gaMJjIU8v/DkI6TdNQxu7X5G3t2Wr4ne22ZS/WKPv5oovAnVHRgdmit1VBLo33iVF344O+kyADI2RXu083sMfMvUjhVbYhoHv6bGmGkJXNzqWcCT2v+KDLImXs4t2LHCr1SYhuba5xCm1QWcTPhh4O2AdVwDF2Rj7iHvCKp7t6J3tbUUwYdOsdZe019jPwzKYNXf65WGsoBO1sEfWRCT4N6AKpjt1b7K+MFEXaHM3nMgbVemYKZlqCqqdpNa5MRPmPOmAJNIqjbvSa1Ot5oO4DA2CTok4v/7JqLL8f2dBW+NnX3pDVQcIajQLBcbQW+1U0Wqcr+LEBOQE9H/oU2ALK5ljtEu4RDNMeTqBo6z9e4Tja1T+W16Z8iQqegZbYpzY+6+bsVWENb6GY+Ww6wo7qImnRUWXrsfYGeBTuEVgQZq36jeM0G1Vwd2P9Q6DBR1o3Uz1Tig= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(39860400002)(136003)(366004)(396003)(376002)(5660300002)(478600001)(6666004)(316002)(186003)(16526019)(66946007)(8676002)(83380400001)(4326008)(6486002)(2906002)(66556008)(52116002)(38350700002)(8936002)(86362001)(7696005)(26005)(66476007)(36756003)(956004)(1076003)(38100700002)(2616005); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: bOeP5WztcsTMndz6DDaa+DFLa+bpHBrSgjAPRuRH25q8F4ld1+iNb+CRneBhr4UB5DKq/AZuKR6mFfnrNRhikGks4RSlIj61+zHMSRcHgIrkfJX2JzD8DM8V+QKqcG/xH0+BBLd5BS3ZW/L4Nf4o669Vaqlz+GyWuNYsHcq+4e/+wqPnvM8sPH/KwVIvIBa04GkkgLkt5aI6dGiOvAzN6+eH5d8ilHDnzg2HbH3E4bUS8E0dyAZ7mDJOCLMMbKAfhUQptCVmrMPQIS2XZleo++lU6wsZYwiXlzI2rD2iD++p+q30CxEzZY9ENczeUUpaww45+X4ZVbzi9Qv9XCTmCAMuUfASF/k0taUL+XBQ602V8KsqwtaU0gEfG1p1TfLYr2S+2mR/qVLuhSeKXL0uvyJGS1KVxGSPAUlh+NVtL1algGr1NGotVqgQprh6nKDcF2YSyA8LEHaufmeh6gIyB6BjH7dYZDegk36ehMVF+02Bd5qAqJoW/prVsK1MzonOw3Ed3R/T0QYDMPZcIb2IAKKaRqjhl2LYgnwwWu09EV7vp8n8qrcgRHLUCa7Vy6LZKWRprmdTuWT5vxPBKgpfU9fChEp8flyj/TWzmbqqKKxtkDhqxnc9yTfdbj/UTiOdE/pY5pajnWwkH1Rd7rYkporYQ9Efck3ZeZXl57RipheUIU2WwoQhgN5PqxbNt2B2OKwC16v8VK/gXZanqJQV4q8JD08zLbJHr2r9a/ilLvi8S6o7fXf37KrGKYZiXhP8DmKHoxr5PHjg8S7VeTbFzQhQ/qDTvFNxxMWI4N5NsR+U2oHYgut0+7Xzt2rSHw3q8CxXkA61rk243TeHdDnp7jUO1jiiT2n5yZ0ROfx2t1it/Lxfxx/o9WPyyIQ8FrWhEDV7t66RLITaE09rQulPDbGPW7kbjD+lXLIr6N0eJcFhhuBu2PTOCUnE2Xosi0W4ewLbOjJi7VEwFQx4dUlYgoBo/d+COE75WxFbohKDYytjaPyISL0FJl+ZdHsdR7QU+19tJ2mhEJS/lq+4QaBPF5b2EvPtpWVqIyccu+6SWYiWr+H60wMq8lrj6q/3yv6n2QKB4ErZ23BIxKJm4YRRb0BffDBHGXbxUgtRAkKY5SEvCEeShZ882uo8AbPOuYWWsInundcYnP7jwVyCYuLUoyJ65RlkJ1lDCZemv9v+42lY9yCUu6z1nHo89GCloabxxXc4M+VHOGuHJENV/NU36AKwr2q6B++QgZAblU3H+wDO/lGFTvsvKFubfcPa717j6YwqjJAamhxa2S0aZgS42/i2g9NwUAbmJMsjO2uCZHFxuDc9aOcjaRt0XcVf+ODl X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 06985e79-7024-4a8f-4a20-08d8ff111477 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:46:49.6889 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: m5Yapaky+EbBNH8D6LDGaP3J8AJN+Q5w+x/PzJ9vqN9+VEM4GE/QPhnVQ9RGijXt6bPq3VO3BbDs5uGSFmmEVA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4125 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Do AQL queue double-mapping with a single attach call. That will make it easier to create per-GPU BOs later, to be shared between the two BO VA mappings on the same GPU. Freeing the attachments is not necessary if map_to_gpu fails. These will be cleaned up when the kdg_mem object is destroyed in amdgpu_amdkfd_gpuvm_free_memory_of_gpu. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 103 ++++++++---------- 1 file changed, 48 insertions(+), 55 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 40a296ca37b9..114fbf508707 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -484,70 +484,76 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) * 4a. Validate new page tables and directories */ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, - struct amdgpu_vm *vm, bool is_aql, - struct kfd_mem_attachment **p_attachment) + struct amdgpu_vm *vm, bool is_aql) { unsigned long bo_size = mem->bo->tbo.base.size; uint64_t va = mem->va; - struct kfd_mem_attachment *attachment; - struct amdgpu_bo *bo; - int ret; + struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; + struct amdgpu_bo *bo[2] = {NULL, NULL}; + int i, ret; if (!va) { pr_err("Invalid VA when adding BO to VM\n"); return -EINVAL; } - if (is_aql) - va += bo_size; - - attachment = kzalloc(sizeof(*attachment), GFP_KERNEL); - if (!attachment) - return -ENOMEM; + for (i = 0; i <= is_aql; i++) { + attachment[i] = kzalloc(sizeof(*attachment[i]), GFP_KERNEL); + if (unlikely(!attachment[i])) { + ret = -ENOMEM; + goto unwind; + } - pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, - va + bo_size, vm); + pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, + va + bo_size, vm); - /* FIXME: For now all attachments use the same BO. This is incorrect - * because one BO can only have one DMA mapping for one GPU. We need - * one BO per GPU, e.g. a DMABuf import with dynamic attachment. This - * will be addressed one BO-type at a time in subsequent patches. - */ - bo = mem->bo; - drm_gem_object_get(&bo->tbo.base); + /* FIXME: For now all attachments use the same BO. This is + * incorrect because one BO can only have one DMA mapping + * for one GPU. We need one BO per GPU, e.g. a DMABuf + * import with dynamic attachment. This will be addressed + * one BO-type at a time in subsequent patches. + */ + bo[i] = mem->bo; + drm_gem_object_get(&bo[i]->tbo.base); - /* Add BO to VM internal data structures*/ - attachment->bo_va = amdgpu_vm_bo_add(adev, vm, bo); - if (!attachment->bo_va) { - ret = -EINVAL; - pr_err("Failed to add BO object to VM. ret == %d\n", - ret); - goto err_vmadd; - } + /* Add BO to VM internal data structures */ + attachment[i]->bo_va = amdgpu_vm_bo_add(adev, vm, bo[i]); + if (unlikely(!attachment[i]->bo_va)) { + ret = -ENOMEM; + pr_err("Failed to add BO object to VM. ret == %d\n", + ret); + goto unwind; + } - attachment->va = va; - attachment->pte_flags = get_pte_flags(adev, mem); - attachment->adev = adev; - list_add(&attachment->list, &mem->attachments); + attachment[i]->va = va; + attachment[i]->pte_flags = get_pte_flags(adev, mem); + attachment[i]->adev = adev; + list_add(&attachment[i]->list, &mem->attachments); - if (p_attachment) - *p_attachment = attachment; + va += bo_size; + } /* Allocate validate page tables if needed */ ret = vm_validate_pt_pd_bos(vm); if (unlikely(ret)) { pr_err("validate_pt_pd_bos() failed\n"); - goto err_alloc_pts; + goto unwind; } return 0; -err_alloc_pts: - amdgpu_vm_bo_rmv(adev, attachment->bo_va); - list_del(&attachment->list); -err_vmadd: - drm_gem_object_put(&bo->tbo.base); - kfree(attachment); +unwind: + for (; i >= 0; i--) { + if (!attachment[i]) + continue; + if (attachment[i]->bo_va) { + amdgpu_vm_bo_rmv(adev, attachment[i]->bo_va); + list_del(&attachment[i]->list); + } + if (bo[i]) + drm_gem_object_put(&bo[i]->tbo.base); + kfree(attachment[i]); + } return ret; } @@ -1414,8 +1420,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( uint32_t domain; struct kfd_mem_attachment *entry; struct bo_vm_reservation_context ctx; - struct kfd_mem_attachment *attachment = NULL; - struct kfd_mem_attachment *attachment_aql = NULL; unsigned long bo_size; bool is_invalid_userptr = false; @@ -1465,15 +1469,9 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( is_invalid_userptr = true; if (!kfd_mem_is_attached(avm, mem)) { - ret = kfd_mem_attach(adev, mem, avm, false, &attachment); + ret = kfd_mem_attach(adev, mem, avm, mem->aql_queue); if (ret) goto attach_failed; - if (mem->aql_queue) { - ret = kfd_mem_attach(adev, mem, avm, true, - &attachment_aql); - if (ret) - goto attach_failed_aql; - } } else { ret = vm_validate_pt_pd_bos(avm); if (unlikely(ret)) @@ -1528,11 +1526,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( goto out; map_bo_to_gpuvm_failed: - if (attachment_aql) - kfd_mem_detach(attachment_aql); -attach_failed_aql: - if (attachment) - kfd_mem_detach(attachment); attach_failed: unreserve_bo_and_vms(&ctx, false, false); out: From patchwork Wed Apr 14 06:46:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201985 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 31137C433ED for ; Wed, 14 Apr 2021 06:47:06 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id ED1256117A for ; Wed, 14 Apr 2021 06:47:05 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org ED1256117A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D29D16E8C8; Wed, 14 Apr 2021 06:46:58 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2084.outbound.protection.outlook.com [40.107.100.84]) by gabe.freedesktop.org (Postfix) with ESMTPS id 99FEE6E8C0 for ; Wed, 14 Apr 2021 06:46:52 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=AtnCfGTSAqzUmKbeKs+FknA1Ojxa5RH5z1B/3ZYvtyHThzljNnf8M0BlqjoOxXBzccR63bQd0Dx4yl59Jz7FUHfY91XN6TuVwSGyAAYy3xv1CqNsbYoIXUIaEWUZFyDcLtZ1uBZSuVRngBH6OmtwQPBQdoNxep8VubYJU1BO1VlEhD+56gAl0iefVSZMloDre5wxLqdQIW6OaF6OllJL+NdJEqpkLL/9NjYjbSnck+u3QBadrc8zKzqmdnd0om58px2cXgii84alRxaYeSVWCYt884tRSksPcKQRUVBsc5SQIAHmBavya1r5uGitDCz/9XmaaUXFTdhJa47MseqF2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TK/o7AidCakKRfTtzZvV0HJAVQwbNXls9+EfNohUvJk=; b=ZwpxrWT6yx45e72KcMQgipvP9NNVX5VFS6K+23G8V52p4Mtse6/k60zqPASpPWX033T0vfoZ5Y/NdH2yAQGrNzcNd+CN05rlPZWZMrT+Jp8d07dFR54EMPkHYaOAzuVGguDFlCX4JEkbAhlKY/b+zY6Y/dc2mozs6XeTYM1lCFZwMlfsIB96b0sryqOxnnW6KiHs83Ay/xtnY533a2FECPjr/EuKnKRUXZ/FUdwoOCpzDdqGWK8WMKkprHi/nZOLOEKg8JK2GLzz266uMqbl7TQ2vEfe+cTlwnMyc7AUCu4E6J3sbEiiElPS/nGqe4kcygc9eEwr7omfzyudwIYipQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=TK/o7AidCakKRfTtzZvV0HJAVQwbNXls9+EfNohUvJk=; b=mIxXLp2E3embkKzCwKQghqB+5j7c/alMQ3yxl568Z1Of4Z8/xLqq8vUuYq/mnuEZTE9QbqhF5LIv+H9vwWl81A3Hzq2g5l1QvQ51aQ8t8VDMpHW4HbRpG3tbUs95APnn6a6I7lJ439AyWytDbe/NVurHTz7oVGems2EZ/gkPduw= Authentication-Results: lists-freedesktop.org; dkim=none (message not signed) header.d=none; lists-freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by BL0PR12MB4689.namprd12.prod.outlook.com (2603:10b6:208:8f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.18; Wed, 14 Apr 2021 06:46:50 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:46:50 +0000 From: Felix Kuehling To: amd-gfx@lists-freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 4/9] drm/amdgpu: Add multi-GPU DMA mapping helpers Date: Wed, 14 Apr 2021 02:46:16 -0400 Message-Id: <20210414064621.29273-5-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064621.29273-1-Felix.Kuehling@amd.com> References: <20210414064621.29273-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Wed, 14 Apr 2021 06:46:49 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a5e7ae56-ce60-475a-9d8f-08d8ff1114b8 X-MS-TrafficTypeDiagnostic: BL0PR12MB4689: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6790; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 66hUke5qMA5BDDYO9syvDdWjv+QEugsDHfQQ/beIkc1nEyh9rtFxbM9SbF3ZE+mNflkUkT8+HWVWPTrXbSvIgN0TVJuWnCGasfCjMdnnN4Frxu+zmN+mBnIxQTTnAluRn/9YsYv1EhV4sGHVMUsDY0jLv+PXf4QFZ2nhOp7ZyPnCD6pqhUGpefAtEFvtR7oBDGZ+M7aRXbqHEjIZPv1rOYXbM0br4B8dbUjzDtIK37qyb1p7vq8dJo18VfEhsFtXu2AxpXbqzrwRMKcRC+cFXdgIwUoRnZvi7mBP4omn00IL8v9h/b3+GKazV8XsWvaenU2PFIDx5KsXe5L8GQHHUZeZA07MHoD1VfuydhXX/DcPP45+cGenVuXxTcYrHRe3Nk47Fx9CgSJ548ZHOsS6neBQQUBfL1B60aJPV3/rsAtas7G1XGcyXMY6MX7zPK6RZvKqYB44+fazktNXRUNyGG3y/Xuy5kyuR4gTZVMTLwnN+zFIGt5pgm3w5zg+a5yCqvHgtGTyNBCa7Fa8P2wUIq4PzAP8SOX7eBSz7a0kcd/Ebn0ffuCdAHeBn6xpqTO1S43/4WAknttVyWzaoKgQziQ2Gr4bjZO1IbeQYL94KWK4QWEJirYyDj0YD3mhwvZr7MRqW/wauVd+1ncwxXwiBvPIjOJvyH09Y3mVPCHsx+A= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(136003)(39860400002)(396003)(376002)(6486002)(86362001)(2906002)(2616005)(38100700002)(83380400001)(5660300002)(478600001)(7696005)(1076003)(956004)(316002)(26005)(52116002)(66556008)(6666004)(16526019)(8936002)(66476007)(186003)(66946007)(38350700002)(8676002)(4326008)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: bhDOYdf9pnvp5geTItBW72J7PTQcU2ZacDnB+X9gzVQDqXFrYni6Idf7GmtkA5bIQyZ8V9RFCW71neHOFB3861gmdjzZnfIEHa8MZahPaQ5XnyhRoKBRXRVPb9unjnmPqeO0SKYpAslzjJ5VHDuRGJeU5TMMAbQkLwCWaV2LQgz2UY5rRvDsP7cImSRYjLtf4HmZZ/Xm4sl0mXR4923dHNqii8kfrYtc1e6326g0D31Z+UXckt6BZPvINe9QR4WUsKzapYNns+eZYIQ3jdfoZYMghI+L9qF2b5yiMjsLPgC0alL0WxmYT9NARtILfoFoApzhrLh5dErlHaVhUAAbv4V4LUMgVfa0phQJQiC0jU2Rakfk+0lGcf2gNVWEcyMNHsngeMHsMVgAnZh6MRBFIb5tAssQg+dUiQG+xq0ijak6DAFl6MQ64jn2HjBiY25EVrqRlSr2jN02/pYk2JTC3SCKga3tnuiXtnt1BtaKu15F95B07HX1LQNqWiPSQ2yVNtwG++dnufbDYSpM/RPomUvi4p4Nq42Mlpr0WHGnWvoJSw8h4xkPU9kyfRZ5+Ls6e3gwGF28jk100+pqjpzrJssX4TeRJdVcPLcIxKRg3EV6AZJx09D8l0yhdLT34SKJcyuDuLORW6CaPpukDv5p8dMfSavX+xG/2EaPawILSqaXNYVQu+7atAbf9G7kIDgo9F0jEap1iW1FMn+b9LiZMlYDt18R2aVNQkcrsbhMtHkNhhQEEki7U05msBpd2XPQURsJqwMXTVycw/9x65JdHlTtmzmsVbRxmsaHMNj6JolCHYwzqKVfZ8cLXUGlV5d803RGnWxAjF/L2/tkFZGde/iIeRtqKU98fSvqoB9L46/ThnSd/ZTIZLmr1yPaYmV7leBBfsrM0Ve45DPfsfEood0xeIGHnZnEAar2Zu1JuZHsOdKvpa7vAgigiMvjDbblQXNHclEn6SdbtMgmlvid+9FCu6TQ5J7DCZ3yuRUXtknuoFFuIZcCM0b/+Zsx6Al764PnpXuZpw7ktxmu1asdelCLZFTjuOuS7EDPTWPJyMNn4eaHV/e0nE842hWVUUV8KQKU+RlzkiDj5hdZd9RL03QOxKDw9srLy/wyoOifFUUocIIezrmOZDtTY/EIUpUmuBnju52GlNLFS9D3XxW4h6T7/+4P7wdKrcfhEFIr9qktLjILzla4M0NuWWiI5M4vndMMio1JtV5w8U/Anqfi3OTgEAN1pWSmb/4TDk9mgtqVR/aB7eG+Ze1oxQvs1pzQc7gIHFC0xBgHZNSPUibFEo+rzDTxHzMIAFOOkdp7BoLBp3aoDQ+We4YRmDrcdL1Q X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: a5e7ae56-ce60-475a-9d8f-08d8ff1114b8 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:46:50.1130 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: Etozg09eRUhqQPChEkVcNJ//KfL/FDjlfy99emQpPVi9nUPVnOB5OSC2jXxsR3Lmj+fG3jSvKT3VxHhQ8r2GtQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4689 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add BO-type specific helpers functions to DMA-map and unmap kfd_mem_attachments. Implement this functionality for userptrs by creating one SG BO per GPU and filling it with a DMA mapping of the pages from the original mem->bo. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 8 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 155 ++++++++++++++++-- 2 files changed, 152 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index 910c50956e16..fc3514ed1b74 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -38,11 +38,17 @@ extern uint64_t amdgpu_amdkfd_total_mem_size; struct amdgpu_device; +enum kfd_mem_attachment_type { + KFD_MEM_ATT_SHARED, /* Share kgd_mem->bo or another attachment's */ + KFD_MEM_ATT_USERPTR, /* SG bo to DMA map pages from a userptr bo */ +}; + struct kfd_mem_attachment { struct list_head list; + enum kfd_mem_attachment_type type; + bool is_mapped; struct amdgpu_bo_va *bo_va; struct amdgpu_device *adev; - bool is_mapped; uint64_t va; uint64_t pte_flags; }; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 114fbf508707..51502a07fc1d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -471,12 +471,117 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) return pte_flags; } +static int +kfd_mem_dmamap_userptr(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + enum dma_data_direction direction = + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DMA_BIDIRECTIONAL : DMA_TO_DEVICE; + struct ttm_operation_ctx ctx = {.interruptible = true}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + struct amdgpu_device *adev = attachment->adev; + struct ttm_tt *src_ttm = mem->bo->tbo.ttm; + struct ttm_tt *ttm = bo->tbo.ttm; + int ret; + + ttm->sg = kmalloc(sizeof(*ttm->sg), GFP_KERNEL); + if (unlikely(!ttm->sg)) + return -ENOMEM; + + if (WARN_ON(ttm->num_pages != src_ttm->num_pages)) + return -EINVAL; + + /* Same sequence as in amdgpu_ttm_tt_pin_userptr */ + ret = sg_alloc_table_from_pages(ttm->sg, src_ttm->pages, + ttm->num_pages, 0, + (u64)ttm->num_pages << PAGE_SHIFT, + GFP_KERNEL); + if (unlikely(ret)) + goto release_sg; + + ret = dma_map_sgtable(adev->dev, ttm->sg, direction, 0); + if (unlikely(ret)) + goto release_sg; + + drm_prime_sg_to_dma_addr_array(ttm->sg, ttm->dma_address, + ttm->num_pages); + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT); + ret = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + if (ret) + goto release_sg; + + return 0; + +release_sg: + pr_err("DMA map userptr failed: %d\n", ret); + sg_free_table(ttm->sg); + kfree(ttm->sg); + ttm->sg = NULL; + return ret; +} + +static int +kfd_mem_dmamap_attachment(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + switch (attachment->type) { + case KFD_MEM_ATT_SHARED: + return 0; + case KFD_MEM_ATT_USERPTR: + return kfd_mem_dmamap_userptr(mem, attachment); + default: + WARN_ON_ONCE(1); + } + return -EINVAL; +} + +static void +kfd_mem_dmaunmap_userptr(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + enum dma_data_direction direction = + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DMA_BIDIRECTIONAL : DMA_TO_DEVICE; + struct ttm_operation_ctx ctx = {.interruptible = false}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + struct amdgpu_device *adev = attachment->adev; + struct ttm_tt *ttm = bo->tbo.ttm; + + if (unlikely(!ttm->sg)) + return; + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU); + ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + + dma_unmap_sgtable(adev->dev, ttm->sg, direction, 0); + sg_free_table(ttm->sg); + ttm->sg = NULL; +} + +static void +kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, + struct kfd_mem_attachment *attachment) +{ + switch (attachment->type) { + case KFD_MEM_ATT_SHARED: + break; + case KFD_MEM_ATT_USERPTR: + kfd_mem_dmaunmap_userptr(mem, attachment); + break; + default: + WARN_ON_ONCE(1); + } +} + /* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added * to a VM. It can later be mapped and unmapped many times without * repeating these steps. * + * 0. Create BO for DMA mapping, if needed * 1. Allocate and initialize BO VA entry data structure * 2. Add BO to the VM * 3. Determine ASIC-specific PTE flags @@ -486,10 +591,12 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem) static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, struct amdgpu_vm *vm, bool is_aql) { + struct amdgpu_device *bo_adev = amdgpu_ttm_adev(mem->bo->tbo.bdev); unsigned long bo_size = mem->bo->tbo.base.size; uint64_t va = mem->va; struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; struct amdgpu_bo *bo[2] = {NULL, NULL}; + struct drm_gem_object *gobj; int i, ret; if (!va) { @@ -507,14 +614,36 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, pr_debug("\t add VA 0x%llx - 0x%llx to vm %p\n", va, va + bo_size, vm); - /* FIXME: For now all attachments use the same BO. This is - * incorrect because one BO can only have one DMA mapping - * for one GPU. We need one BO per GPU, e.g. a DMABuf - * import with dynamic attachment. This will be addressed - * one BO-type at a time in subsequent patches. - */ - bo[i] = mem->bo; - drm_gem_object_get(&bo[i]->tbo.base); + if (adev == bo_adev || (mem->domain == AMDGPU_GEM_DOMAIN_VRAM && + amdgpu_xgmi_same_hive(adev, bo_adev))) { + /* Mappings on the local GPU and VRAM mappings in the + * local hive share the original BO + */ + attachment[i]->type = KFD_MEM_ATT_SHARED; + bo[i] = mem->bo; + drm_gem_object_get(&bo[i]->tbo.base); + } else if (i > 0) { + /* Multiple mappings on the same GPU share the BO */ + attachment[i]->type = KFD_MEM_ATT_SHARED; + bo[i] = bo[0]; + drm_gem_object_get(&bo[i]->tbo.base); + } else if (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm)) { + /* Create an SG BO to DMA-map userptrs on other GPUs */ + attachment[i]->type = KFD_MEM_ATT_USERPTR; + ret = amdgpu_gem_object_create(adev, bo_size, 1, + AMDGPU_GEM_DOMAIN_CPU, + 0, ttm_bo_type_sg, + mem->bo->tbo.base.resv, + &gobj); + if (ret) + goto unwind; + bo[i] = gem_to_amdgpu_bo(gobj); + } else { + /* FIXME: Need to DMA-map other BO types */ + attachment[i]->type = KFD_MEM_ATT_SHARED; + bo[i] = mem->bo; + drm_gem_object_get(&bo[i]->tbo.base); + } /* Add BO to VM internal data structures */ attachment[i]->bo_va = amdgpu_vm_bo_add(adev, vm, bo[i]); @@ -557,13 +686,19 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, return ret; } -static void kfd_mem_detach(struct kfd_mem_attachment *attachment) +static void +kfd_mem_detach(struct kgd_mem *mem, struct kfd_mem_attachment *attachment) { struct amdgpu_bo *bo = attachment->bo_va->base.bo; pr_debug("\t remove VA 0x%llx in entry %p\n", attachment->va, attachment); amdgpu_vm_bo_rmv(attachment->adev, attachment->bo_va); + /* FIXME: For some reason SG BOs don't get individualized. Do this + * now manually. This is probably not the right place to do this. + */ + if (bo != mem->bo) + bo->tbo.base.resv = &bo->tbo.base._resv; drm_gem_object_put(&bo->tbo.base); list_del(&attachment->list); kfree(attachment); @@ -1376,7 +1511,7 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( /* Remove from VM internal data structures */ list_for_each_entry_safe(entry, tmp, &mem->attachments, list) - kfd_mem_detach(entry); + kfd_mem_detach(mem, entry); ret = unreserve_bo_and_vms(&ctx, false, false); From patchwork Wed Apr 14 06:46:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201983 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id F30F1C43460 for ; Wed, 14 Apr 2021 06:47:03 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id B3EF96117A for ; Wed, 14 Apr 2021 06:47:03 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org B3EF96117A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 9C9426E8C7; Wed, 14 Apr 2021 06:46:55 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2084.outbound.protection.outlook.com [40.107.100.84]) by gabe.freedesktop.org (Postfix) with ESMTPS id D2D466E8C2 for ; Wed, 14 Apr 2021 06:46:52 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=WEwNkFQsVEyrkEJt437t2hkh76H5BGyqb/M6+CXTDYxzbdkZ9Gbw6amA6z1ljGmPLF5rLrKijlht4RC+LIda0Tsy687gYv3WagRQMcybF+7kaAOsh9T1zJ+6j6qe7A5LQ1YzdtrHwh7M1oBfVuMiTCjGH22hurfgXYsJP3Ma9DrjMoSVh8RHtTihNsmF/oOt+Gj5wo+P7XEOneR/i1RKJLt+NxbOh/CCnLaDyJU05cpJa5AEKzVf1+wanfjl2CqKIVIuU6V/r4boVnrUX0pO+m/wvJ+2BDGCDv+Q7EAqSmuDNkLck9jH6i8a4oteVGlcRbBgOxdIIkev0SPSZjp9Ng== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cb8YxsUBmYMVnZl2DgiAN61Zr0J9HEePKeYL5KfZrdQ=; b=DWI7CdXqFC9cJwzLEMqzdEcMZB28SpmXrP2U+dgKcGeicPMw/1JpRqCL4tHJp/W8tNtZ3Gw24fYtJ0wFP4ajWatUD/paTgnD1bvHGwd0YAefg53r+eLsdB8fwA7VhzTR0Mhq2i/t60v6lBslOySfjkPcxbzhwzpW20OJHMkQ/t2xjPfe1yOFsFdSukqewp7dg5P4/4WfilT+uFwNSaGQIIQbeeFeQKTjzDd+B4I2H833+bY+3+6IlHCnChBzzRINBn8+CHOD1BoTiLmRa/MLNISvO0mXaDxjlKTcwjCFV++Ltgaa4G2EQ7pH/8Z+pS2qHlfg5kmawTNgwB+4efLTOg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=Cb8YxsUBmYMVnZl2DgiAN61Zr0J9HEePKeYL5KfZrdQ=; b=vaONb3rIedjwvfTrN1LQZjdMbP7yUkz33CrXpM3hcLYsztqwZygjeJNSs5tkxX+YHDSXeg0TeyWo9PnWtuxEp/qT45leKAE6wTZB+mJurYZqj/Vkur4b8n3jLJlLlsIqqyrsmjC4mGjSAocIVD0C51U490HUzc++SSru44ty29w= Authentication-Results: lists-freedesktop.org; dkim=none (message not signed) header.d=none; lists-freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by BL0PR12MB4689.namprd12.prod.outlook.com (2603:10b6:208:8f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.18; Wed, 14 Apr 2021 06:46:51 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:46:50 +0000 From: Felix Kuehling To: amd-gfx@lists-freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 5/9] drm/amdgpu: DMA map/unmap when updating GPU mappings Date: Wed, 14 Apr 2021 02:46:17 -0400 Message-Id: <20210414064621.29273-6-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064621.29273-1-Felix.Kuehling@amd.com> References: <20210414064621.29273-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Wed, 14 Apr 2021 06:46:50 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d4e473ec-c92c-4986-0aa8-08d8ff1114f2 X-MS-TrafficTypeDiagnostic: BL0PR12MB4689: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:7691; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: ukoxCNJlgQO8AV8hmx9t5NgLeiLiXQWZc/cRvTkHC9repHRG11aN1wS3L4R8JrmvzBJuVaBxadtq10+oqozyjgsnJ1f+YMfLw8q09eYYt2/+iEiTs5iH8Shx9QqQ3/9WuEEOx//weaCz2VAdKEzAxT/pWWy095l/nO4kYaej9b99YLnVAfkJxeA92cGbBtGNlY0+Z7NeipX9/19Po1wR8LkTxvg/xjFrlS6YLKaV48ptWdA8gUfGPd+CCzb2EDjPJzZSSdBTyL9Gzv3Xzk8K2SU2vpDxNdZrs1etjI32AIp5neNkZ1w3+kG3jV7UpmzHTDdBl5NioDoNDzUGigkfyL4GfYAEMd2uTUSusatcegrJwWK9bBBN2Iw/AuFbRMA+4xrITjKWlH9NSdJe6kFOoMTDxkvKsT8bwTsX+dMo8RXXdtDHqnwU39O/pRAeKiZSqPXlueqNxvMhXJ8X5uzOmFgyIOFSE1zs/De1WbdNkpYxb86CzkmoFFHMrMXkU6IX48WleTujFLgdt0Cek+FTqXCPcWigxVRvXt1OflZlQhQxhNINwmTJdbycwVVJnM0IVvnnzIxrTJJPQwkEZOKvlrzml6OKhAEmCaQAtuGtUIWwCKvHEeDzqMZ9pF40yFkxF+h0OmUmJtlat1N2Qy/NG2jeqZAjz+BG6aQFBgjuWtY= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(136003)(39860400002)(396003)(376002)(6486002)(86362001)(2906002)(2616005)(38100700002)(83380400001)(5660300002)(478600001)(7696005)(1076003)(956004)(316002)(26005)(52116002)(66556008)(6666004)(16526019)(8936002)(66476007)(186003)(66946007)(38350700002)(8676002)(4326008)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: KxK2/oI1ncuAzY99GPpEULG8xadP02O4c7DMoMJkmp4gI9eopgVedFxMBzvWKfh8zGgyk8t496mxxF2TkK3DVs3qLWiyjDbW19P4n+GBbL4CsntuyI0/3HMQTlbn8MTwLdeyZ+A8ttgJw0hpMKNzr3yhSS5N+xaqyZO9KNwtNFv4xCXTo5o7CIf9Ao+T8hi6Cya8rUzR9bb4Oy67pGlJFHpeU0QCJDjey455bocdhUUobCsUhh41iK/6EsCajLrDQ3ygCZ7IMOknjAaee0DPWO5DzW13sT3bNOV5oXAo+gBqEyr7Ksw9+C+KiEhq6fOb1ppMEtCDtcAqZqJ/aQL+NYT5imIu4TXapqiaLqXsbZT50dUlzaNVMzQQc8Jfr95Q+wnUv/WEk59ZttB+YgP3qIB8EN2sV2JKP0jYzSjOKCRIIDSkx26tTGm7Roj/2EdiuYlWVJcoMUM7ZqEFgeIKIDyyqGNfoP86KCy/VRUTBQphhGv745a4TIH0YkRkzXoC/7ZnAjooQ2mkyeAqk2ES+EUz5Eyp1TSQGrfdEu1RkZVqWsrR76DhyNgs/SfbwK3Iz5lDmCM26bDmDdTLMhL3q18SR2tkeORVP/1n9DQD56qS5Ixs7JpVVub7BOthSAk2dlPzzjl/JUscQ/WNt67dJcqrSFbMLzsWdHZotmBKh5d5LWFwbZkBZWyu1LNHBR/65LC0+prNggQFSge3HkNT5+YETVRcBO6KmEbHiz5JGkOd6AAfn2PtuMcrxhdUbXyVUjl6RZrK+B8TioK0nhmxkFXcJ5oagtfNXZZullNFocKJiuy5S68vy5GNQqSYUxUHlgwr9m5S4daE7R1YavcpoICSJHMjWpg89dFN7AqviKuCO2Kzp0XSiuepOIQp+Tciudfnu0cxf2TivUvMpZMeKhOd1lkIBcqNQfZeil4XwtYZiGZVwQt5LTWi6n79ehfVvD8DSZI8ugWuEoKaTfQ43mjBpk1AvH5gDkkpBZH00BRNnFyThwaLOtYp8I0wCMy4AExfzNsCTuhSl7KCFmjPWTF0voGMYjWUcckSJyVEPo5RRyDCcVH1tNTyvjQdvm117ndYpMjO/1ZHoqTikPQBDU0LnJPQ5VfGr7hWvVSZ/Rvkbs8EODiveg3QLkxvU+Fflabsi/KWVSwV6AL5bRrhv1ziEZRoaGWViJ3IHKEcC73AUZiUdgYihdn1ssNdNzDu+Fpg3wWeivz8+UoKH7IWQwQqCKXo+hf092dog3a3ynfJ1tZqYWu6aKKdXXGK1HKQVyNbtLAgWv8qfONED/K78T39Ra8/QvmIyCPogDimCer9xcnhyJiWl1I+1wQF5OGW X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: d4e473ec-c92c-4986-0aa8-08d8ff1114f2 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:46:50.5451 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: VXHjQ1bMmvQXtp+/BGNe8WjOoeoP2RYY8Wq8ljq3/6xi38S2WGz3TzDQwzJhcEuNx0Q3jDaAJj4Rcu/63mpbrA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4689 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" DMA map kfd_mem_attachments in update_gpuvm_pte. This function is called with the BO and page tables reserved, so we can safely update the DMA mapping. DMA unmap when a BO is unmapped from a GPU and before updating mappings in restore workers. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 56 ++++++++++--------- 1 file changed, 29 insertions(+), 27 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 51502a07fc1d..3bb2ae185bbb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -964,11 +964,12 @@ static int unreserve_bo_and_vms(struct bo_vm_reservation_context *ctx, return ret; } -static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, +static void unmap_bo_from_gpuvm(struct kgd_mem *mem, struct kfd_mem_attachment *entry, struct amdgpu_sync *sync) { struct amdgpu_bo_va *bo_va = entry->bo_va; + struct amdgpu_device *adev = entry->adev; struct amdgpu_vm *vm = bo_va->base.vm; amdgpu_vm_bo_unmap(adev, bo_va, entry->va); @@ -977,15 +978,20 @@ static int unmap_bo_from_gpuvm(struct amdgpu_device *adev, amdgpu_sync_fence(sync, bo_va->last_pt_update); - return 0; + kfd_mem_dmaunmap_attachment(mem, entry); } -static int update_gpuvm_pte(struct amdgpu_device *adev, - struct kfd_mem_attachment *entry, - struct amdgpu_sync *sync) +static int update_gpuvm_pte(struct kgd_mem *mem, + struct kfd_mem_attachment *entry, + struct amdgpu_sync *sync) { - int ret; struct amdgpu_bo_va *bo_va = entry->bo_va; + struct amdgpu_device *adev = entry->adev; + int ret; + + ret = kfd_mem_dmamap_attachment(mem, entry); + if (ret) + return ret; /* Update the page tables */ ret = amdgpu_vm_bo_update(adev, bo_va, false); @@ -997,14 +1003,15 @@ static int update_gpuvm_pte(struct amdgpu_device *adev, return amdgpu_sync_fence(sync, bo_va->last_pt_update); } -static int map_bo_to_gpuvm(struct amdgpu_device *adev, - struct kfd_mem_attachment *entry, struct amdgpu_sync *sync, - bool no_update_pte) +static int map_bo_to_gpuvm(struct kgd_mem *mem, + struct kfd_mem_attachment *entry, + struct amdgpu_sync *sync, + bool no_update_pte) { int ret; /* Set virtual address for the allocation */ - ret = amdgpu_vm_bo_map(adev, entry->bo_va, entry->va, 0, + ret = amdgpu_vm_bo_map(entry->adev, entry->bo_va, entry->va, 0, amdgpu_bo_size(entry->bo_va->base.bo), entry->pte_flags); if (ret) { @@ -1016,7 +1023,7 @@ static int map_bo_to_gpuvm(struct amdgpu_device *adev, if (no_update_pte) return 0; - ret = update_gpuvm_pte(adev, entry, sync); + ret = update_gpuvm_pte(mem, entry, sync); if (ret) { pr_err("update_gpuvm_pte() failed\n"); goto update_gpuvm_pte_failed; @@ -1025,7 +1032,7 @@ static int map_bo_to_gpuvm(struct amdgpu_device *adev, return 0; update_gpuvm_pte_failed: - unmap_bo_from_gpuvm(adev, entry, sync); + unmap_bo_from_gpuvm(mem, entry, sync); return ret; } @@ -1633,7 +1640,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( pr_debug("\t map VA 0x%llx - 0x%llx in entry %p\n", entry->va, entry->va + bo_size, entry); - ret = map_bo_to_gpuvm(adev, entry, ctx.sync, + ret = map_bo_to_gpuvm(mem, entry, ctx.sync, is_invalid_userptr); if (ret) { pr_err("Failed to map bo to gpuvm\n"); @@ -1672,7 +1679,6 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( struct kgd_dev *kgd, struct kgd_mem *mem, void *vm) { - struct amdgpu_device *adev = get_amdgpu_device(kgd); struct amdkfd_process_info *process_info = ((struct amdgpu_vm *)vm)->process_info; unsigned long bo_size = mem->bo->tbo.base.size; @@ -1707,13 +1713,8 @@ int amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu( pr_debug("\t unmap VA 0x%llx - 0x%llx from entry %p\n", entry->va, entry->va + bo_size, entry); - ret = unmap_bo_from_gpuvm(adev, entry, ctx.sync); - if (ret == 0) { - entry->is_mapped = false; - } else { - pr_err("failed to unmap VA 0x%llx\n", mem->va); - goto unreserve_out; - } + unmap_bo_from_gpuvm(mem, entry, ctx.sync); + entry->is_mapped = false; mem->mapped_to_gpu_memory--; pr_debug("\t DEC mapping count %d\n", @@ -2083,9 +2084,8 @@ static int validate_invalid_user_pages(struct amdkfd_process_info *process_info) if (!attachment->is_mapped) continue; - ret = update_gpuvm_pte((struct amdgpu_device *) - attachment->adev, - attachment, &sync); + kfd_mem_dmaunmap_attachment(mem, attachment); + ret = update_gpuvm_pte(mem, attachment, &sync); if (ret) { pr_err("%s: update PTE failed\n", __func__); /* make sure this gets validated again */ @@ -2287,9 +2287,11 @@ int amdgpu_amdkfd_gpuvm_restore_process_bos(void *info, struct dma_fence **ef) goto validate_map_fail; } list_for_each_entry(attachment, &mem->attachments, list) { - ret = update_gpuvm_pte((struct amdgpu_device *) - attachment->adev, attachment, - &sync_obj); + if (!attachment->is_mapped) + continue; + + kfd_mem_dmaunmap_attachment(mem, attachment); + ret = update_gpuvm_pte(mem, attachment, &sync_obj); if (ret) { pr_debug("Memory eviction: update PTE failed. Try again\n"); goto validate_map_fail; From patchwork Wed Apr 14 06:46:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201987 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1D575C433B4 for ; Wed, 14 Apr 2021 06:47:08 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id D50236117A for ; Wed, 14 Apr 2021 06:47:07 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org D50236117A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id A2C8B6E8C6; Wed, 14 Apr 2021 06:46:58 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2084.outbound.protection.outlook.com [40.107.100.84]) by gabe.freedesktop.org (Postfix) with ESMTPS id 59E6A6E8C2 for ; Wed, 14 Apr 2021 06:46:54 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YVzwxlcLbnF6PPvhTsGVyhzmAUeKojFSbSOyTy0a/EZbyjbHyC7yFCPJXw7lsU+fM2KMq5a3RBX0QlyQqHuGhbaV7qqTM6xpfdV8mpt/eExzxjN0dFWujJC2/FldoHJo4dNJXZgyd1CQ283qaRiL53nzS9CXLb983JqkTjf3JTWG/WYa98RGr9YELaQaUGaQLpXrpYd6/uwoK14VLq5EJxBLzVVXSPgUGFxCDl63VwiF3DGmwjFXEU5qauYMzbwklSv95mamvcPAQpDDZdBns4xn+mc2GKGKe51MDugcmHycmHNgwQ6IlVW1sdtiQit/xOIo2+r5kXcuv2CVDuT7hw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yQut1xXkpwdjAD1BditEYBDjILo3RVAJf1/Yf2Aq5Us=; b=lC6O1ydlkacJg8/a4WnFLx1p5lcxlfDARUkE8paXJgX9rjR3eCSF7UCIf3oWbG7XjveJ8VWlQlebDwIN0R4uiV2AQxHe9WkoyLFVJomH6wINwDVo6UPJoRq5jqsmeoIVuhDDjIoc+nzjkAiVL164jZwbz6oDhX5L8RlBGMxqvx2u/xdkd6Iv9w2XjBMjYtqNUXw3+1LgtpvztsHeP/faP5G2+k72rjXQW794YHGFz6im4qSLDxKtzVUqICxPmvxfUFs4/enuDqaYo74/8adgxNh1NPJinmoej3AFlKKgIFzAU/7Md7+j8Xf6qXYVu6yLlF707GuUd5f1HdpLCguTEQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yQut1xXkpwdjAD1BditEYBDjILo3RVAJf1/Yf2Aq5Us=; b=Egya3pUYsrAHm62bvlfJavfC/zDL7EcsrT+ylrCZeFgfImO0JxOC6bL/E0JhxDw2KhFDEJ0ala2B2XLFjZ/hD6cy9hosDrvUlvonr+x4tpFY4sJO2F4ZYcb2jeJZP+Hm7a8vGwYGHByulBYXFXcuvXxXteHWVehIwuPuVRIPg8M= Authentication-Results: lists-freedesktop.org; dkim=none (message not signed) header.d=none; lists-freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by BL0PR12MB4689.namprd12.prod.outlook.com (2603:10b6:208:8f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.18; Wed, 14 Apr 2021 06:46:51 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:46:51 +0000 From: Felix Kuehling To: amd-gfx@lists-freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 6/9] drm/amdgpu: Move kfd_mem_attach outside reservation Date: Wed, 14 Apr 2021 02:46:18 -0400 Message-Id: <20210414064621.29273-7-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064621.29273-1-Felix.Kuehling@amd.com> References: <20210414064621.29273-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Wed, 14 Apr 2021 06:46:50 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: a4ea2180-46df-4d11-cc5f-08d8ff111535 X-MS-TrafficTypeDiagnostic: BL0PR12MB4689: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hASnQUtFu2ew24VnOq2Rd4BIT+Ya5GDigpzV42HM/o+TjsU+rWm0tP5blFPIBXv/3CZGMUBXpS3NZfIeYlHyudS99HsQot0Xo5tE+b8v1QBkJYVugLaQ652/P/ixly36bqzs+IaqV1tXuwxVfNtffKs927P30r2skr12byNqa50s13Jtesl5doxczQN0vlcI5I0uF84SpCca8L43EVDpF5KwEuWVm7e7/xDHIw6QnzxABRMbZLOo/mmKHD3IPlqTK5c05QKqAX9FQrRfGSaGaWjQntJvzzSluOmbFrFpDKOPjHvnpERH40yXtPJVRawcRWSGU+G/lDPMDJB1PgDCcPnu89a2r6lIrHcjzFoi9oiktkwbcqEDn/8BMwW811PV2NMKa4obVPV3TjoWVRY+/cD0BROHwTBF8a7vYkXGiCF90Monf+IAYF9HnPcAf6scKSAhDqdCS6KGSwxtF6jyjVMGQ1ji3TXFrhfOIEraqEddOpXHnyBJ6/aTQZOIDHJrQrwn1AYQMdwdhXAM6C/xnGX3AdjKfvGN4UnEYQCtr2hP0pM67bYp7WwweoN3tKhBwWL396iT5o0mUtTA8FA/ZFwxanrP4N5OKaYCqVHvwNvYv3RcqUCZU5Qq289yCU6xN4S2bDPn65m91KyJqX6hb3H3CE82yqIvEe4fm6XEqP8= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(136003)(39860400002)(396003)(376002)(6486002)(86362001)(2906002)(2616005)(38100700002)(83380400001)(5660300002)(478600001)(7696005)(1076003)(956004)(316002)(26005)(52116002)(66556008)(6666004)(16526019)(8936002)(66476007)(186003)(66946007)(38350700002)(8676002)(4326008)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: sNaoOtl8GoFR0xXIS9AbYhAKBzS0csybi6aVIvOKvHH1PNrumBKYpEvYbPQdmTpzvBtOwkc6IOJTN1Saa3z4Lgihv+yk5vtrG/I6eNru8gz7AMIXbA868h98TRmrJwJxLb+0/1CxbjDC9Ao92vpBS6WnPWWSXcaJkn6/7PSE6YI20aPc92SV9UoEJ64tr5/triQ1zL71dasD16f0Bb4ZEu7CrxjwcbbMfeWSWbEQTFYShbEXSpJaQrp7GbTYUREMbABMVARIl6P2Jcnu8FHdFhkTBq3Xdy5OMv2g/IzOODU0N4fFq6RCN2pVM1Yw1xaZ8vWIMJ3ddHMwMDvQ7R5QJc1cO976YFCZD+VIuP0+d1vAFT4w1VHn9UIVMdI6dgfJvcLaauSQqipuycMbeVm3Baj9PYin/nA7qwfDqsGgSGGcOkLwoylCe6Bb+xb8JYJXT/ujXmBTt80lP/CvzcXAEpjJX5182/0xaUaHXF0PsHkya6151EA3iOhl/X7ycNoQfDqwefi5oLiBOPzQVP9QP41SKBUMZFOdHppftmXucueadITvbnn/dNkebU49LpcsPJo6lYfSQBx0K/uvg9uqGSfSn9w7C1AcXvOz6aw1bNU07Z0ZMeIqj3Z2uZJ464jt6wtZ+Eiae0nfGGEXMht+vAkYTjkTw4HNlsfmFFCER9XcxjDXtlJISu63+T1IpLj+ei3M/oLeySGfsFaQA9WTjXOjiOEad5akL6DQZQ/i+KJ+9aPoB1RRakr193RrG5wQrV29BbLw1KS1gsA0LZsnCXi0W4QjKOFT0TC6jX7Sxp+f07Nqf+nC95CkG8yrRQaeo4fmh3BhY0AVWk2PHSmQBshU4VCLwnxkONXxeCwz0eT4CfP2rW70SnzEzBbam/RVG/LT8IMFXO1uLSbUkaYaPIaQ6xPixDqRAze/d6mDXrU46faVykHwMtkEMUd0GE75oh5fSOoEfkZkJOPQSXm6Oka91e42WKNpAjXaI2Rwk2JdStEw43lqqTPc5r7y2de/zgjUDuLu+cutQNfYuZgMsNu1TimgukhSOnpkq+6KOzEwEOr3h7y8oGcdlf9hKN7et/lN1snqb7UwJRxaGGa4ldxPMTAbqNrT/OJ4ADg3Itw4Gm8iheISXNgq3qBruphbJTZDYQGqk/Bdjoq1lMonEnfRC3homVcV1nZJmGozoUklaUkikm0iBDPRyMhcePnM4+vig4eTRZAqvbjI9iHCJYjVDN1Kh4+HzEfXYvkaPV1x8Gz9AV1K4orNhQrRSt7vSEyIFo1VaXc2xB7XAla9uCrNujF5M8iOgzs8kFcJrDGE6iubSE983qNVapfF+46m X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: a4ea2180-46df-4d11-cc5f-08d8ff111535 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:46:50.9942 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8WUuljiUSkV4Uxarb4W9lX84cfw/ywd35ZjQVuepV2GQvBUEBlkSPLhEpyFRnDWLTGAoXWHSg0QFiNzL6thAzQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4689 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This is needed to avoid deadlocks with DMA buf import in the next patch. Also move PT/PD validation out of kfd_mem_attach, that way the caller can bo this unconditionally. Signed-off-by: Felix Kuehling --- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 72 +++++++++++-------- 1 file changed, 42 insertions(+), 30 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 3bb2ae185bbb..1416f3c03f1d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -575,6 +575,32 @@ kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, } } +static int +kfd_mem_attach_userptr(struct amdgpu_device *adev, struct kgd_mem *mem, + struct amdgpu_bo **bo) +{ + unsigned long bo_size = mem->bo->tbo.base.size; + struct drm_gem_object *gobj; + int ret; + + ret = amdgpu_bo_reserve(mem->bo, false); + if (ret) + return ret; + + ret = amdgpu_gem_object_create(adev, bo_size, 1, + AMDGPU_GEM_DOMAIN_CPU, + 0, ttm_bo_type_sg, + mem->bo->tbo.base.resv, + &gobj); + if (ret) + return ret; + + amdgpu_bo_unreserve(mem->bo); + + *bo = gem_to_amdgpu_bo(gobj); + return 0; +} + /* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added @@ -596,7 +622,6 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, uint64_t va = mem->va; struct kfd_mem_attachment *attachment[2] = {NULL, NULL}; struct amdgpu_bo *bo[2] = {NULL, NULL}; - struct drm_gem_object *gobj; int i, ret; if (!va) { @@ -630,14 +655,9 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, } else if (amdgpu_ttm_tt_get_usermm(mem->bo->tbo.ttm)) { /* Create an SG BO to DMA-map userptrs on other GPUs */ attachment[i]->type = KFD_MEM_ATT_USERPTR; - ret = amdgpu_gem_object_create(adev, bo_size, 1, - AMDGPU_GEM_DOMAIN_CPU, - 0, ttm_bo_type_sg, - mem->bo->tbo.base.resv, - &gobj); + ret = kfd_mem_attach_userptr(adev, mem, &bo[i]); if (ret) goto unwind; - bo[i] = gem_to_amdgpu_bo(gobj); } else { /* FIXME: Need to DMA-map other BO types */ attachment[i]->type = KFD_MEM_ATT_SHARED; @@ -662,13 +682,6 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, va += bo_size; } - /* Allocate validate page tables if needed */ - ret = vm_validate_pt_pd_bos(vm); - if (unlikely(ret)) { - pr_err("validate_pt_pd_bos() failed\n"); - goto unwind; - } - return 0; unwind: @@ -1516,12 +1529,12 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( pr_debug("Release VA 0x%llx - 0x%llx\n", mem->va, mem->va + bo_size * (1 + mem->aql_queue)); + ret = unreserve_bo_and_vms(&ctx, false, false); + /* Remove from VM internal data structures */ list_for_each_entry_safe(entry, tmp, &mem->attachments, list) kfd_mem_detach(mem, entry); - ret = unreserve_bo_and_vms(&ctx, false, false); - /* Free the sync object */ amdgpu_sync_free(&mem->sync); @@ -1597,6 +1610,12 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( mem->va + bo_size * (1 + mem->aql_queue), vm, domain_string(domain)); + if (!kfd_mem_is_attached(avm, mem)) { + ret = kfd_mem_attach(adev, mem, avm, mem->aql_queue); + if (ret) + goto out; + } + ret = reserve_bo_and_vm(mem, vm, &ctx); if (unlikely(ret)) goto out; @@ -1610,15 +1629,9 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( bo->tbo.mem.mem_type == TTM_PL_SYSTEM) is_invalid_userptr = true; - if (!kfd_mem_is_attached(avm, mem)) { - ret = kfd_mem_attach(adev, mem, avm, mem->aql_queue); - if (ret) - goto attach_failed; - } else { - ret = vm_validate_pt_pd_bos(avm); - if (unlikely(ret)) - goto attach_failed; - } + ret = vm_validate_pt_pd_bos(avm); + if (unlikely(ret)) + goto out_unreserve; if (mem->mapped_to_gpu_memory == 0 && !amdgpu_ttm_tt_get_usermm(bo->tbo.ttm)) { @@ -1629,7 +1642,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( ret = amdgpu_amdkfd_bo_validate(bo, domain, true); if (ret) { pr_debug("Validate failed\n"); - goto map_bo_to_gpuvm_failed; + goto out_unreserve; } } @@ -1644,13 +1657,13 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( is_invalid_userptr); if (ret) { pr_err("Failed to map bo to gpuvm\n"); - goto map_bo_to_gpuvm_failed; + goto out_unreserve; } ret = vm_update_pds(vm, ctx.sync); if (ret) { pr_err("Failed to update page directories\n"); - goto map_bo_to_gpuvm_failed; + goto out_unreserve; } entry->is_mapped = true; @@ -1667,8 +1680,7 @@ int amdgpu_amdkfd_gpuvm_map_memory_to_gpu( goto out; -map_bo_to_gpuvm_failed: -attach_failed: +out_unreserve: unreserve_bo_and_vms(&ctx, false, false); out: mutex_unlock(&mem->process_info->lock); From patchwork Wed Apr 14 06:46:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201993 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A7DD0C43460 for ; Wed, 14 Apr 2021 06:47:12 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 71839611F2 for ; Wed, 14 Apr 2021 06:47:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 71839611F2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id ED52C6E8C9; Wed, 14 Apr 2021 06:47:00 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2084.outbound.protection.outlook.com [40.107.100.84]) by gabe.freedesktop.org (Postfix) with ESMTPS id 84A926E8C3 for ; Wed, 14 Apr 2021 06:46:54 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PGNlsc5+4GRRzyTdLii1E3QwGjutf2DnnBnnUGBZCxbXFUTbZXJhoU15XDKVuCHp+90qB39oBvi89ky5shP4ZzTzj28yYCQXgzcjk+utnBEHK9dQ3kojVhtYMXeuQOu63gxC+AfL+Rj3xfdWBkHqWb355+lLVxu1TuB/MAYPYLPhyJeoQDZ5650r+1AudobELesmHxNa2z88deYr75CzdERxl52uicq1+kT4liq+zBOduhWnSbfLuDfA/Uop0xoe0f5GCkFwz29wQEGJoQfaH6LXvGCJzq7EN1gvrkjANNLrXMyjgbHhxATbD5K/J5w2I8RhJXip3HaYzcfb4f6m9Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U1Z7N+TokxJateGZpU1pIpnTcEkYQ9R8vQvQEi4r2h4=; b=asM7KImf4h9nFpekLOyBayg/S8n11HIPmOuNyY3Ed1Nqt7iJE8jke1lQFAGhw9Z5EDfqKWQVkfojpFPfQgyVXuDuRdiL2xDX2uGe8mO7bXqFXpVAyJ3D93CK+u5TAoDLWqpOTROG37BGgyQdNFIFZMUFZ8UPjPy6NgHGxdg4lAK7kmgWFCWnpjlflfF5D5HZp8oMDKjViIYWlgeUxFJHvc1UyO50/EFHf1rLNNiH+RXTHJCwzhlApJbNjXuxvyMgiDYw30dMYEqGHiDVn0ETD4lLuwxqz+g/iWr808cYpEAWm181kGZAhlaDdeCKFfMMsnWuTyIX3lVfqcx2jf1jIw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=U1Z7N+TokxJateGZpU1pIpnTcEkYQ9R8vQvQEi4r2h4=; b=si6OG2x30NmcaoaAKHcFfcQkbMnf7WUwDej/1/f5JK+ZhMwV5j73ETbonzy9WUr6cyxGr5ly+gY6NSWUusfwYHMgelhCwTGRBmvmc7+RtFVbRwZkSRGLbsNtQOcWE3+j+g0omFjgyjNvv/P4+K4fVpcMa9+tOIG4//zDTAKYNOc= Authentication-Results: lists-freedesktop.org; dkim=none (message not signed) header.d=none; lists-freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by BL0PR12MB4689.namprd12.prod.outlook.com (2603:10b6:208:8f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.18; Wed, 14 Apr 2021 06:46:52 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:46:52 +0000 From: Felix Kuehling To: amd-gfx@lists-freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 7/9] drm/amdgpu: Add DMA mapping of GTT BOs Date: Wed, 14 Apr 2021 02:46:19 -0400 Message-Id: <20210414064621.29273-8-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064621.29273-1-Felix.Kuehling@amd.com> References: <20210414064621.29273-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Wed, 14 Apr 2021 06:46:51 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 3aadd970-5b5f-463b-092f-08d8ff111578 X-MS-TrafficTypeDiagnostic: BL0PR12MB4689: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:608; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: y76yqm8YNsAxNprx7qhehBVVcO0V7HwGebQhGjWxRo6IzNepne285eBWwPxwh0ZPoH23xv46xYKPl9nO5KuhW/4GkN/+XRRFW/wSg/4SqSdP/X7JYEldL23OMa1oh1guR+9/d/hpBdEY3C4xa4CH14/Fe6LArGXraKikDFFA9FNlLdWpkZDbNAJ4jYUNNJki6+2RWUod1FbTb+mtaO6L2rL8CphNbt3eUQQCnJL6DRVSsALd0njlX+vok1Dg+EyHDMXw0hPDfiFVoXJaP0niSNu2p7Hf4NQ07zmvDlajVDla+gMKH5uHzAcQWyvhc+6rKAYVOcOw/UV0QeHHYalrAPLyBOU0ClNtEbJ5rNMmS4qVl6ZEL1B2GASED7W36x6obFYXXXgyuq+lY7NWnWJ9zXjpl9l3y0LJOF8qLBxQvo0GSBrTRSJEqw7k9gGrOV6HO3ifWYGTTKdEdI+pP4c201xZJ2U72EEWHnCKWAXzi6jAtY01Dqd1P0K+LjCxuYJWQG6qyPjMRHiU8kUGWjcG15YTQQriFRKKGDiWPknVgEFLhCiQzn6AUeq9AoW9xKWnArNhep4MKOO2cNh2nIV1/mSGjvCy4WKsa7NpleecgKGyX83iA+r4PIzMCn+JD1vrYgAflplUjiekwG8QXjAK8lmsH3CCNxmqMH9cJIDwycE= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(136003)(39860400002)(396003)(376002)(6486002)(86362001)(2906002)(2616005)(38100700002)(83380400001)(5660300002)(478600001)(7696005)(1076003)(956004)(316002)(26005)(52116002)(66556008)(6666004)(16526019)(8936002)(66476007)(186003)(66946007)(38350700002)(8676002)(4326008)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: lHjs6/Dueo5we1KHlykd8rT95d9EItusWIQl5k9SU6WovByiD//urbigM5hOf4zsz/NABR5VUd0Hop2mswZqBDFWe8WKwX+iZvJ1x97XUrIriWHZvvnMjdxAHmCCsGHx3zcsxtrHRBKcDpz4Vmwp53PLrQ5INQgOz6HrzKwTJFy7nj2WW8EX9Mhb8GNYNsnNnGneR9+/jxdjNzzSY6v23uR5uBLPO3zYJdX4PC0kQ5ptdOxmmI0fRkc4OHL6lVoc0L9kRcmslssFIJjJdggOCwVkQukm+HLmFAeTusIcFl/qtlQ8KbX/eo33TqOIWf87L5SMhGQ0QIa9ikp8BTExdciN6l0SDexQCfFecNcQ/YrINyrwNUOqCq7nz9rZcFy6Vk3u4JphbtW2ZVuepkTHgST5CVdvmusmjRcf/bkToxofEXu7jWkzFqym5sUi8OOaJWt2X1jhU9vwmEwA8aND7PpGmLEu0naNPinMqziKWzDWxVfg4ackXAdhsvCFU8KQ0WpRB1c30ExFFTEiGhvHAbu7bOlBGfXVf6P49FNL6ZEVOp+6eCdFr/3BoNV0sGoDi1sJVT1hKED2/7so4hbLIGXO9hewwqo1fGx5OHBLcVkBb8Eh14iipF4Z7I5aMdirmvafDDhfftQFdwuG6SXKIk6rtFKpb+cuh8U5hhoUk6GN34tyDV/xUZH15pGY/Vwlh940vNbBAtO9cKeju3TeSArBDQFSqRsUACGuwbiMp/Z/k7AtB10vUULYxuQ3fJi79LxNx5AxcUMECFARzkdRNBD4Oqr5yl6hbmpDpFnuGd+hownMV6+VTxABFW7ah5SN1nIORtEzuRnI5Pl57g84R6C7Zgi3y42vfO15SD3Otjf8ipAG/eHK2LcVAGY5vX9UmAs4sWeJVOWVcesNnJmwvB/PFL4iagsW4m9t5iay1TxWCmwsaiv2RErDh4qQAhIogARwepH3r+4jL+5Sb+5hOfifXc/4LvsSefiL82Tp/0bNWJXuEwQcAN++21cG1fUsNNxqp3iL/ot6aHKTdeJaVZjfJSRY8188GDyFx1Gc0ke4aaUzKMZDRutOJhxlZzxCdPeTMJIcy9etlohL/CcdAD2etr9MdbKEfw+7z8ArLmFvx4ND3MsEBDgtRmNLZKCy/i3qOZKmObDElTZJl8IUv7PSIzF+1c5WI6Ez/xz1UnIe7XQhuALpzpcmcuLhEq5fUfBCU4G+QSbVSbxIw1WoiQsarsRxz7pn0lv9ds/KU7MUyDEOB+yWMhxS7vF1CYg90iWE6tCsre/Aw248x8po4JS60aPgbbm3ZTTlV2IQ7qNWfygYJEMgI20H5KpUFYdk X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 3aadd970-5b5f-463b-092f-08d8ff111578 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:46:51.4541 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: FgirISMgHegd7+2SHOGw5DoFPgxRIEnFtiMMo+LXfGR78rIyH2xuJtpj5fU2DJn4mYwXJ7DrQoIWImISMFFWcw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4689 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use DMABufs with dynamic attachment to DMA-map GTT BOs on other GPUs. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h | 2 + .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 74 ++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h index fc3514ed1b74..3ea51982b720 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h @@ -41,6 +41,7 @@ struct amdgpu_device; enum kfd_mem_attachment_type { KFD_MEM_ATT_SHARED, /* Share kgd_mem->bo or another attachment's */ KFD_MEM_ATT_USERPTR, /* SG bo to DMA map pages from a userptr bo */ + KFD_MEM_ATT_DMABUF, /* DMAbuf to DMA map TTM BOs */ }; struct kfd_mem_attachment { @@ -56,6 +57,7 @@ struct kfd_mem_attachment { struct kgd_mem { struct mutex lock; struct amdgpu_bo *bo; + struct dma_buf *dmabuf; struct list_head attachments; /* protected by amdkfd_process_info.lock */ struct ttm_validate_buffer validate_list; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 1416f3c03f1d..bb3a96ab8f20 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -522,6 +522,16 @@ kfd_mem_dmamap_userptr(struct kgd_mem *mem, return ret; } +static int +kfd_mem_dmamap_dmabuf(struct kfd_mem_attachment *attachment) +{ + struct ttm_operation_ctx ctx = {.interruptible = true}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_GTT); + return ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); +} + static int kfd_mem_dmamap_attachment(struct kgd_mem *mem, struct kfd_mem_attachment *attachment) @@ -531,6 +541,8 @@ kfd_mem_dmamap_attachment(struct kgd_mem *mem, return 0; case KFD_MEM_ATT_USERPTR: return kfd_mem_dmamap_userptr(mem, attachment); + case KFD_MEM_ATT_DMABUF: + return kfd_mem_dmamap_dmabuf(attachment); default: WARN_ON_ONCE(1); } @@ -560,6 +572,19 @@ kfd_mem_dmaunmap_userptr(struct kgd_mem *mem, ttm->sg = NULL; } +static void +kfd_mem_dmaunmap_dmabuf(struct kfd_mem_attachment *attachment) +{ + struct ttm_operation_ctx ctx = {.interruptible = true}; + struct amdgpu_bo *bo = attachment->bo_va->base.bo; + + amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU); + ttm_bo_validate(&bo->tbo, &bo->placement, &ctx); + /* FIXME: This does not guarantee that amdgpu_ttm_tt_unpopulate is + * called + */ +} + static void kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, struct kfd_mem_attachment *attachment) @@ -570,6 +595,9 @@ kfd_mem_dmaunmap_attachment(struct kgd_mem *mem, case KFD_MEM_ATT_USERPTR: kfd_mem_dmaunmap_userptr(mem, attachment); break; + case KFD_MEM_ATT_DMABUF: + kfd_mem_dmaunmap_dmabuf(attachment); + break; default: WARN_ON_ONCE(1); } @@ -601,6 +629,36 @@ kfd_mem_attach_userptr(struct amdgpu_device *adev, struct kgd_mem *mem, return 0; } +static int +kfd_mem_attach_dmabuf(struct amdgpu_device *adev, struct kgd_mem *mem, + struct amdgpu_bo **bo) +{ + struct drm_gem_object *gobj; + + if (!mem->dmabuf) { + mem->dmabuf = amdgpu_gem_prime_export(&mem->bo->tbo.base, + mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE ? + DRM_RDWR : 0); + if (IS_ERR(mem->dmabuf)) { + mem->dmabuf = NULL; + return PTR_ERR(mem->dmabuf); + } + } + + gobj = amdgpu_gem_prime_import(&adev->ddev, mem->dmabuf); + if (IS_ERR(gobj)) + return PTR_ERR(gobj); + + /* Import takes an extra reference on the dmabuf. Drop it now to + * avoid leaking it. We only need the one reference in + * kgd_mem->dmabuf. + */ + dma_buf_put(mem->dmabuf); + + *bo = gem_to_amdgpu_bo(gobj); + return 0; +} + /* kfd_mem_attach - Add a BO to a VM * * Everything that needs to bo done only once when a BO is first added @@ -658,8 +716,20 @@ static int kfd_mem_attach(struct amdgpu_device *adev, struct kgd_mem *mem, ret = kfd_mem_attach_userptr(adev, mem, &bo[i]); if (ret) goto unwind; + } else if (mem->domain == AMDGPU_GEM_DOMAIN_GTT && + mem->bo->tbo.type != ttm_bo_type_sg) { + /* GTT BOs use DMA-mapping ability of dynamic-attach + * DMA bufs. TODO: The same should work for VRAM on + * large-BAR GPUs. + */ + attachment[i]->type = KFD_MEM_ATT_DMABUF; + ret = kfd_mem_attach_dmabuf(adev, mem, &bo[i]); + if (ret) + goto unwind; } else { - /* FIXME: Need to DMA-map other BO types */ + /* FIXME: Need to DMA-map other BO types: + * large-BAR VRAM, doorbells, MMIO remap + */ attachment[i]->type = KFD_MEM_ATT_SHARED; bo[i] = mem->bo; drm_gem_object_get(&bo[i]->tbo.base); @@ -1558,6 +1628,8 @@ int amdgpu_amdkfd_gpuvm_free_memory_of_gpu( } /* Free the BO*/ + if (mem->dmabuf) + dma_buf_put(mem->dmabuf); drm_gem_object_put(&mem->bo->tbo.base); mutex_destroy(&mem->lock); kfree(mem); From patchwork Wed Apr 14 06:46:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201989 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id E434DC43460 for ; Wed, 14 Apr 2021 06:47:09 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 92C9C611F2 for ; Wed, 14 Apr 2021 06:47:09 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 92C9C611F2 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 979636E8C4; Wed, 14 Apr 2021 06:46:58 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2084.outbound.protection.outlook.com [40.107.100.84]) by gabe.freedesktop.org (Postfix) with ESMTPS id AC2E76E8C2 for ; Wed, 14 Apr 2021 06:46:54 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=P7YP8sccZiYAycQ+FCqBNdYqQexD/B8nUzEcbwdoGjOn+g9HqaXGhh5dHmhTYOGvJOkGVuuUjyNOFdMnWS6IOSZMW5UtcadO4raLDWekoXFp+Ssaj0harqG0DnTt8Zkj354Y32tAzeBWhTiNLyn60cXmHfiwe1QG3KqAy7PfylsWQyisSlSgZV8vzb5McmKUK2/WCY3wny49YRTxyjamtyva88mBvoygHt1FZvoN1orIVeXxg8pbWFK3Jll+Ah8VHvimvsnDRu48BzJYVdQwdPgluq+NmtnwsRfaw8bqau5FGAQJaw1JBSEoQk1tDFBBKc/YhmvTesIMOjS3xmfxSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eN2Yd6c53oyb+YR1k7QtiRCenmta+Oh7ACloZoJ7IWE=; b=LXHFW7hRrpB/tuSUqPmtvHHAV6qyhJ0OllEXbOEUhtaV63okbmEVRolkhbXxVmJSe+FcipZwMX1Tm/9j9HkUdZ3yXvi/2Zca0dmsxd/CrM48WxCnlSACJTR+3dp5Yjw4cuQDYDWde5JgbjMTrLnOx/uPBbg6Q5vXTU7G5LH+czpBFsAd4sET95t/2tP9xKUa8w1iFORtuxhgnm9rYCKmjNEQkJDxXkpN53D4p68nUcq6/NFJB92Z25gK1J6bJIVnbwzy40R8KPaEVeSo0o4psd9hB23Y1DrBTYAt3Yg8mFDDGCj0DHr8JjGIVjgdADzZiidojITlf3xzU0Cpm0Nyag== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eN2Yd6c53oyb+YR1k7QtiRCenmta+Oh7ACloZoJ7IWE=; b=p5Ztl68bdo8tPsPW1UF31bfiR0em/DYvl+ZGfCn3fKON0Td1zDrsVX8/8fKAczAaQI7xXLXSapItdSk6nktEnzfxLATn0P+YJQB5Yq0F5ORYewa6zOjiOpLCFxo9BxDI8dkahneQYIARJM2USNgf5H7iRtN6+C/lpbEDNd58AKE= Authentication-Results: lists-freedesktop.org; dkim=none (message not signed) header.d=none; lists-freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by BL0PR12MB4689.namprd12.prod.outlook.com (2603:10b6:208:8f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.18; Wed, 14 Apr 2021 06:46:52 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:46:52 +0000 From: Felix Kuehling To: amd-gfx@lists-freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 8/9] drm/ttm: Don't count pages in SG BOs against pages_limit Date: Wed, 14 Apr 2021 02:46:20 -0400 Message-Id: <20210414064621.29273-9-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064621.29273-1-Felix.Kuehling@amd.com> References: <20210414064621.29273-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Wed, 14 Apr 2021 06:46:51 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 318bf6ac-fab8-4ded-3c96-08d8ff1115c3 X-MS-TrafficTypeDiagnostic: BL0PR12MB4689: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2512; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YSL3QiG0KIvzILR88B7CAbNem3VDpFdXItl3dG9yXx4X1D7RsAJFMTlQ+/bM0ytWOHfxRp6+oA917KY5SowcUnbnRP4K2UaIBisSylldTVHU/jBEEVr/2qIr7jLvnMHgt5ZoWN5LP4o9/d89GlbppW7pmNahgPmkad7FW7Alh73LXHPnjGaeYwYjPK3IbqZ2D/mb/GlmJockbn4Wozo0dkwgBgONHufCn7wErBDFnpB+dA5fTFaG38OmhsJHEYcjECd0oxOdcLA2Kt9CLxgj9+JCCLyqb5pxSIqgIVVpJhyBg9G3b0C/zFjFBZ/nrfHCodlx6lN9tE0pQbUuyDiTxs1tSBsX2jEe6YnONZbSUCHFODVUkpfCB36JjKO7yG9730CX6vFEsneIiqJXxilwkjiluedIYr3ipT8H/KzhxFA6Y0Dk746G+nfiMlJuNtDdIjcOexBQsKG1+IXm3CpWXxarjgLOLiJWoyb25i+0KnzZ1JpSmJDpojfvCMUD+HPX60I+rMj5WgKNWqCPIn8bnxJTRepCW+ces67P2/QrMGsBGemzFKmB95IV4H6QwIkCzFFzQxnN5k572hUUJYQiD5ZL9VwwVAoEuqkssp0V3F5DlBuzBjqBi/wJfrFIXjqVqQ94Ohrzv12J8PZaN9KyFofGnFzuQa+afrn7SZMPTIs= X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(136003)(39860400002)(396003)(376002)(6486002)(86362001)(2906002)(2616005)(38100700002)(83380400001)(5660300002)(478600001)(7696005)(1076003)(956004)(316002)(26005)(52116002)(66556008)(6666004)(16526019)(8936002)(66476007)(186003)(66946007)(38350700002)(8676002)(4326008)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 3nOaXm+OTMa/ew5zzRiDyMGE8HNz724i+lX1h1+/1Y1dpkCViY+SxMY2B3txkujvPF6vrrygt7q6uhKtwAxOX4TwHXUEHRPfdvApA6QGjA+7dZ8+RJL9pBl0+B8rifZOm5+tImvGgeyk4uOTgdrhKIgWs7CRjdxayyIrJVjCvNBuimcEqKjvpbmeCHtw+ngfxRplR06n+ERKR6JAizDb6aifb81jlcag9VFIXluJn9tlYMq/YlJU3mgj+IRQn8cGa6x9MYT9/5OVAozYxrl3tFqHrnIT7UMTdD3T/DjNOmKUBPq2j8aPBwWsNMB0hweKE8AQWCQ4jWyy9SpOrI28nEvNUXoMs+/zsWEu8q5Qp7i2C0d/6bz43tiwiHjwWL9r4O0/gyrhL31DdMyaDRCbIJ1jYcgZhAWnGbb2dwcirthyHMdfA/PRESYevIdF5n1IjuvbrnUbCYnYHE1kEm+VA+MLJYK1hygOItA10pAMRYxaOrOQDVXHQbW4eOJOT47R4DkhMbtTPPM0syYsED+JPEBVK0yoEFSdtHOcqlW0bmgE1Rhu4fbUvG2Q0b+7jMWVkCw5VTq9uc37IdgGEIh11Vl+2Rez4geu4OHyb38gAhDpYOFTeuxmjHHKWHC15O48m9WmTXRiy5UBQ5bFPqAiz49Sgzj9zgARrglMkuOifsYhcn2LMr3k9Uj6iLs0VqIDFZMNiCH8OR9t5Tmy7RroPChw2kP2RAQ82rkCnFZoTYXRc/7zgirewPTfqo220KAKMzG4UUZr08Y1pvta0nBjU5TQCUlB5n/5uC3/PUG7Z1pdDPMHbznZHhXxjfRRD+pFJZtdsREYeuqbejBv6aJfj/75+1PjvOtpApItJSBxEa1B4+C4FKbfz5RB9zrrPUjwkh4HHxETipHA9k8km9U1C05nBKL0DIOHjvUs+cEv0q6tzQWN2uTy/aqgJPn+J0NsbZE9Fn+PkXDZDX4S9GGYrqFvIHIwcEjw3uv6E0frLNvG+CsEAcHCyYHTSYDiKGWxjIbgTynWJw2dCkfy8dXMWdWl1M2y2PLRXvBmvBcLWux3ylWP8GVWP0LH8CQ5zGcFsYATmfAKTUesQ5FTRSFRz4kbpIz8mTbCeYLssoKK9XfGYyEp3dV0lyx9i/Sc6Dz16v0Jq+1uJbfjmX3xnXMbhgDMATnKi1r0IDpVFA6tVmio6L4ogp11/QAnHPjPNiJqOWwM6bDP5MXlOaYidibvxxhNYyaI5aCia0ClxQ0c4dduZHKKzKjWOhLY2B7qDfp6XJesiFmOSSYneRKV2LaiRbU0HLHNvHVBOKvROar1/CN2RkxNmY5SiHeI8vzGAHB/ X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 318bf6ac-fab8-4ded-3c96-08d8ff1115c3 X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:46:51.9002 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 8U9lzIj6kUcsWlJtB+g4hAw/kqCsyqZkC9KJb1+hw3TV8GFRzcHEyvvTh4//thf4XKpSi3e4r14rjTe/DTXjSw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4689 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Pages in SG BOs were not allocated by TTM. So don't count them against TTM's pages limit. Signed-off-by: Felix Kuehling --- drivers/gpu/drm/ttm/ttm_tt.c | 27 ++++++++++++++++++--------- 1 file changed, 18 insertions(+), 9 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 5d8820725b75..e8b8c3257392 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -317,9 +317,12 @@ int ttm_tt_populate(struct ttm_device *bdev, if (ttm_tt_is_populated(ttm)) return 0; - atomic_long_add(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_add(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_add(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_add(ttm->num_pages, + &ttm_dma32_pages_allocated); + } while (atomic_long_read(&ttm_pages_allocated) > ttm_pages_limit || atomic_long_read(&ttm_dma32_pages_allocated) > @@ -350,9 +353,12 @@ int ttm_tt_populate(struct ttm_device *bdev, return 0; error: - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, + &ttm_dma32_pages_allocated); + } return ret; } EXPORT_SYMBOL(ttm_tt_populate); @@ -382,9 +388,12 @@ void ttm_tt_unpopulate(struct ttm_device *bdev, struct ttm_tt *ttm) else ttm_pool_free(&bdev->pool, ttm); - atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); - if (bdev->pool.use_dma32) - atomic_long_sub(ttm->num_pages, &ttm_dma32_pages_allocated); + if (!(ttm->page_flags & TTM_PAGE_FLAG_SG)) { + atomic_long_sub(ttm->num_pages, &ttm_pages_allocated); + if (bdev->pool.use_dma32) + atomic_long_sub(ttm->num_pages, + &ttm_dma32_pages_allocated); + } ttm->page_flags &= ~TTM_PAGE_FLAG_PRIV_POPULATED; } From patchwork Wed Apr 14 06:46:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Felix Kuehling X-Patchwork-Id: 12201991 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 84F8CC433ED for ; Wed, 14 Apr 2021 06:47:11 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 39D716117A for ; Wed, 14 Apr 2021 06:47:11 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 39D716117A Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 8917D6E8CC; Wed, 14 Apr 2021 06:47:00 +0000 (UTC) Received: from NAM04-BN8-obe.outbound.protection.outlook.com (mail-bn8nam08on2081.outbound.protection.outlook.com [40.107.100.81]) by gabe.freedesktop.org (Postfix) with ESMTPS id B22726E8C8 for ; Wed, 14 Apr 2021 06:46:55 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=NZwvw9Wm1VdvGs1y2IAvH9qMzCs2VzFzeheHpdfaULjQ0obMLvFRahiLwOcuyf077EskmH5yesHblONpNRy1odRlWxPaI6S1qev0qAqn8C1NExw0AEP1kj4zNLyF2hE5DEGvxezJORtwY6WlsYM719J0qGu+gGWw1HGiW66PMeDD9lXcL27UGIvvzz9wMkLqKAR7EaUZM9sHO27ylBvWsamCgbm95Ffq6maWZIYsOlyLzbQwC3g57TKgKrGi7EhhCZi/ZIXjjjdNe6q2YOiBPtnI2Rm1OZreHQNRXcqFi2yHoxfjIq7eN1dS9jNS5PslHPzARF7IRN+GPR5sMz4W8Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h6cYmo3dLmpw5Ddu0zE/rP1cYO7LSD0YBQ2wvpnQCJg=; b=iIYfWwrXl7vPtoh8Lfuog7F1jqvLmz/6RYrqr9MX5gUM6SpBvxSSYEV2xeo8sh8A6X5odqA9CtlwAjrPl7ePLzG/KgfPJ+DB532X4UxdtTOZTAp4CQWCdZrzqCv7qVm3o68tNwUkauaninkb3e3ebKlTeQ4J73mr6p0Q3JUQmc2l0YCtvqSAq3Ls3ss8odAdMPG7pTW4B5WxxJiYHPyxIqGelhaVvi5ECqGnmGNm9j3gYWlJOz9NOv8JDa9gnxwR7vgXzfUNQubDtji3gA+hM6kiHcmT10sgjqlI4d+OiM1NubTVPyAHAQbvvujiGQRHCkG4BbAGr5CXkP5eGh9U0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=h6cYmo3dLmpw5Ddu0zE/rP1cYO7LSD0YBQ2wvpnQCJg=; b=l/pRtc5OTXKLB4+9voBZCa4IJ0yNRpMGyrRc2g9naBxN9ngl3x/AHQAhTtPRhnnHDHw23qc1ZsLrfGidhZxwvI3NBDCi0E6cmj1Vi2KY0Qgl4l84U5CvPhzIl1rZ0mx5Dwu//g+7eW9eioUOFs63EQ3AWw+OEjRAEchKMnJK7KE= Authentication-Results: lists-freedesktop.org; dkim=none (message not signed) header.d=none; lists-freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) by BL0PR12MB4689.namprd12.prod.outlook.com (2603:10b6:208:8f::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.18; Wed, 14 Apr 2021 06:46:54 +0000 Received: from BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033]) by BL0PR12MB4948.namprd12.prod.outlook.com ([fe80::70f5:99ed:65a1:c033%7]) with mapi id 15.20.3933.040; Wed, 14 Apr 2021 06:46:53 +0000 From: Felix Kuehling To: amd-gfx@lists-freedesktop.org, dri-devel@lists.freedesktop.org Subject: [PATCH 9/9] drm/amdgpu: Lock the attached dmabuf in unpopulate Date: Wed, 14 Apr 2021 02:46:21 -0400 Message-Id: <20210414064621.29273-10-Felix.Kuehling@amd.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210414064621.29273-1-Felix.Kuehling@amd.com> References: <20210414064621.29273-1-Felix.Kuehling@amd.com> X-Originating-IP: [165.204.55.251] X-ClientProxiedBy: YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) To BL0PR12MB4948.namprd12.prod.outlook.com (2603:10b6:208:1cc::20) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from Harpoon.amd.com (165.204.55.251) by YT1PR01CA0026.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b01::39) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4020.21 via Frontend Transport; Wed, 14 Apr 2021 06:46:52 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 6b79af60-56e6-4ec3-a3ad-08d8ff11160a X-MS-TrafficTypeDiagnostic: BL0PR12MB4689: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4502; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: sASXfb9ZNWGJy3vGPx/+eZg9T9zzwpIgG1aJBaABEx8ZPz+QkfFmw2oI5Hq5pTDfaDlve9CUDrFs319xuHj+V7T1ivB146fRCJAoUKK9s3y3TkgrVWjKAY2uw9CeGmC53F+H/wv3vi/oPlWCScr+zy74YcWdvp5Tk48yt9S+zDks/L/YnDfTS0RD2/bnAdRclbL0Ay1qGYws/EFMOwUlCPH/79/NCO9ES3w3fqSNGEOEphz3cbk8Ce6Ha/5y/rGl4529udFz99D7oC51XW/OjmKHraouKG8mwFYb6c8lBnX+SeEPavfvzB4rQ9DkYpR3Cn/pRTo0/1nvLiKViI15jCv6LiD9/y6WLH9yJWwtXzMcq4vXDoauEaWmKrQkUgITkjyocaGqPZZP9uWE9pByFvoh5XxenqP9v3OhhXQIzo3wrid7SxFHm/Fg1v81OJFh5m6MB63h+XmIJr512AYFPJVNMRaHCNpWeSYvEAdk9aCi5LxHsyUl7no1BNIjoB6O26r62c+OBm1ws8Nj/VESlcHidLVMx+49T9RYX/AZs7TxoKfwoaWA3ZFoG+at090hpdPg4S+TMktIPZkW1ZIWMp9EF0pf0COyjjjFSA3bGz9x6lfK8cM8GmtmkXheXZrkSHzSnwcx3JWtOTIctzrJmQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:BL0PR12MB4948.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(346002)(366004)(136003)(39860400002)(396003)(376002)(6486002)(86362001)(2906002)(2616005)(38100700002)(83380400001)(5660300002)(478600001)(7696005)(1076003)(956004)(316002)(26005)(52116002)(66556008)(6666004)(16526019)(8936002)(66476007)(186003)(66946007)(38350700002)(8676002)(4326008)(36756003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: EFKoOSDMpozgUjRYBfACVn6jPtZKYlkYNeSnqfa/Q5XrmcfBQRZWBZsRrOKQZ4kEl0MISJwUKac2iefUbrFVTzAQrA9YhkU6PLMAY3krn4hUd0LXj4X7T4P8mBD+xbwXtly0U+4JgaFr1xFpqx2bUvgeWLCrT2AUz7/U6nmqSqdHZee/F8B2qV6R3VBvsax7oUgH54pzeLh7WK5X8TGDYrkFKhSl9LBpvDMi43PMleaJmwWGmQyA0uf8BwLIM/gMfPBe0QJLfz1HPezkBXqkxrfn9GLfv1ZiCwPyMEspReWtJ9cA5APnucgntD+I6eHmo/sLarKixwQu38lWTFQqNd6U68jbiBNNRL47ExQV2ctt8pnEu74ZPJUeSWgBAp1p2wuahBWgqH2un0u96vY9m5I3IgJpNvsJtVeRjisSLA4ReTOY+4DSCmsalV34aOoJnBOGUnfIPwkTOeZxUMNU+Dlp9EcTEPRBay6HOHVBAC/TDyl5enV2/zAZmxvZX1ppvT0PaDSqG0S2X1NiiIfO8iTknJkMJ5i86jtLXFrV6r+zI4zL9f7cACGhHx6BFntEx5AbjiBnCe3fdxRF3ippYekiwDd4XRU42oNr2O35MZV7ZKE+qRl9fYecPAcMLfaxvfYL0gzK2rvczgFJKnx/Xc6SGsA6kEv/ZD9Te5kFKBk0jCl5j/W5KArCtcGuZ8xw/NI7YdulzYc6qppMSzJJFV3QThkWI/T2q5o+dgJ0rPqrA/Rd7Up6IxZQKH2ve+prdiZ3/x12C6c/NY5ykOpYcHvGBFdd8EJsxJ66KPikpXAqRHcTdNYCPToMZ+0HWrn5R+LYHDj+3+kghgh5EezM7CGIxVA9aSueNnuMCJjbUklPVzvJWeSfI7vuCGq+MTOBN9PHn4B+zFg6qdGYxb4NvVQH1aQdg4B5dgSvREM1jCcnCwMFH1oEL2YUpe3ueFfMQYo2AOb/ubDPmwd7epNUvc8PcKWuWlsVhpJfXtlzxR7bZUIlCDQWaoA1UJ6rPC5UNDLB7YF9nRyRQ5RIps4ACv4s0I7um6b8lcKu7OcL7YqQnczjgQyFNbRK+wRJcdR+ILWlVW9E6dpJ3JM0HmVGtVYa8SdTYMBRpRw0DGCRUW0CBpMK/jCqctXE/ar3vzdV2eT9JpQBPuYpqUHYV+pJsztEysuK6xoIBPNHdKEqC3oa7hyRnIu52rr1e2sAzJnKydHWQQqETNcjzMLT7So0rxAG5ac20dtEBf7rwv0vGFtqSFJD2n5S3Qh3PM8r6oG50Bon9q9jFvOcaVwvapkwv+EhKOEnvUkRp6J84v2bq6hvt00NtR8XOvpfQCDQdNNi X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 6b79af60-56e6-4ec3-a3ad-08d8ff11160a X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB4948.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Apr 2021 06:46:52.3651 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 79uiDwUwOxuyXU4YSPoTo1l1oSgBYWJWtIqMuBRq5TYcsTx3JM/E/QSq61qTJs3vcFfLuvyFGLLRrR7xrqGThQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB4689 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: christian.koenig@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" amdgpu_ttm_tt_unpopulate can be called during bo_destroy. The dmabuf->resv must not be held by the caller or dma_buf_detach will deadlock. This is probably not the right fix. I get a recursive lock warning with the reservation held in ttm_bo_release. Should unmap_attachment move to backend_unbind instead? Signed-off-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 936b3cfdde55..257750921eed 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -1216,9 +1216,22 @@ static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev, if (ttm->sg && gtt->gobj->import_attach) { struct dma_buf_attachment *attach; + bool locked; attach = gtt->gobj->import_attach; + /* FIXME: unpopulate can be called during bo_destroy. + * The dmabuf->resv must not be held by the caller or + * dma_buf_detach will deadlock. This is probably not + * the right fix. I get a recursive lock warning with the + * reservation held in ttm_bo_releas.. Should + * unmap_attachment move to backend_unbind instead? + */ + locked = dma_resv_is_locked(attach->dmabuf->resv); + if (!locked) + dma_resv_lock(attach->dmabuf->resv, NULL); dma_buf_unmap_attachment(attach, ttm->sg, DMA_BIDIRECTIONAL); + if (!locked) + dma_resv_unlock(attach->dmabuf->resv); ttm->sg = NULL; return; }