From patchwork Mon Jan 18 21:01:10 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028239 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EB616C433DB for ; Mon, 18 Jan 2021 21:02:25 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id A97FE22C9E for ; Mon, 18 Jan 2021 21:02:25 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org A97FE22C9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1AF536E595; Mon, 18 Jan 2021 21:02:24 +0000 (UTC) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-eopbgr750053.outbound.protection.outlook.com [40.107.75.53]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8A2A86E59F; Mon, 18 Jan 2021 21:02:22 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XMVEe0RessHdabBffNSuTR86ehksXME9n7dQ3SjpAfnzG8s3zuNWuV9b4ODTKgnUD8L1rDe8LvaQ80r9wfVLFXbQhw5N6ZywLH4ZUsBt/pob5RY1YeasyQ+GeB6oR8Mj5p2YKJ2EV14cjKcRSpj6kMUMFVS1Qe3WBL/qLxWBxqdbFqC1oyUw2caL26Z1sXZ43wbEDVLz3WrFUbKZktJHuNsu5hZpcPsACx5hUax2pYmRHhxg6olZ7Not+9ybLxiXNKgLAwGD3GvcXPA9jdPBSaXsrM14JERyoqW2xXGAqEd9SgwfFACYuqnGCcb25jO/KKbwT/ycBuf5sB8YsMw/xA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BP/54bMQ4sgO6D23Tqmmn+/xWCR7dSOOdaZ01gId4mA=; b=S9OiVC7JdtZHknpEQbkuHqVikcqoGr0iVgpzFnf+V5TwyOPhnxjFVWkv5PbubRoKz9RzX5brTrkvldAPFbQV2Yvof+xUVQeHIIZK7fuMyLQtI/KT4ghPJATaA81gZzmxaW1/BVCGjai9RRqovLrz5CI6bYA9yc6Z06RnlYP6TMwUF5A2v/vNWGAVEsMYFjktAoIbHt7qtFB84oO2qXifdsC8ldYXB1Vk870WNEGs4ekwWsJdvNhFW5U4SVzElv8L/4ewCkWAvnMFxPtqzi2sb6pvDXHCMATwhas9wNKE86jatuBnkO64fVKqcD7cUjFleK2WA9+9DS72kUY1LCXZFA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=BP/54bMQ4sgO6D23Tqmmn+/xWCR7dSOOdaZ01gId4mA=; b=jzXL31G1kgNyQYFjOmZCAvPJD1EfaD095ud7aMufFPVExj3IOPWvzSYE/qDPPZi0F1ItYCYET169bKEipvubcUv78v4B4t8Qcs0/yY8kp8BWpGgHcH/D+DolHCebYxFiz4ytnPHK/yGSPqMVuxIblIN/OfGhwegy9a9O46vuuzA= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:20 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:20 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 01/14] drm/ttm: Remap all page faults to per process dummy page. Date: Mon, 18 Jan 2021 16:01:10 -0500 Message-Id: <1611003683-3534-2-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:19 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: ab8e0dd2-61bc-439b-407c-08d8bbf45881 X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: deM01CxxidvlgVQHRVklo1QG6scbCkldLLXyvcgjZmbSTRxRSft1rHYq6e3QwPKZjuoR2Ys2UDDIRGz6DSYFN+zF8nwlOnPhhE6vXN0jjlaQV9tv/l8D1WQ/Uk8O37nDHEVLDN0MnyhhGDRCGdxQgTuWx0R58Vy8VLOj+m8VIiL6C5D5Ncxysmf+Hu4hEK/CXxDa3HyG/4oGDIYIjZ5SGfuRzsUI9rJlPAwJcRXJ7RxWwvVo88QK9xOU9e3U+G0oSsHUm5ieaTpDdh8z/cAR7N+LrCfmldfL9cX505geUAEz6ytFenDgH168yDAAX2WlcwFc95Cq67oxjvNX2nzEpY4EfQ1EXDt/qfSkysCqDBy7NbM/wmvqMDn4OBc5d3D54eEgPznxvvXf8OgK4Prq7g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: Lp7C78WlsbpW+7kcchh39gKKIBCCMK51KAcmEKzew69qPt/n9Hh9+/ivTpOs/0tak89twzYG1Up/1iz++3254KIsw2ByTTa1ZcEKXs9ta/NvImXZuPNpr8npLl6/ty77k07iNT0V5dR78xFc3kCML3cjUC6BOkZ2Utlo0S6i0jFUeXpPbyVgabzO5YFAiu68UmJpemw3HnXRmvY2jKTfxLZ8L+j4EVMn4dmplxVxn+f2enis49GEY0uFSaUH1V09i9aWo7ewf5QJw3kH5fUIpva2WgmR0Kbp2Nh8ZJwywkArwcgmm7vwRXLT934WGjbHZurC1Dqn6GzrYaGxpjRXr2V0qgi8sYDu7dbUcQWJUloSyCBEVRqryXh7Be+yyP6vNMLVUP0Vkeawj6fisqSLE8JMgEkWCWuZ72lyBZ0sxSbtTpJPgURJzLfEV8ZwDje24RL8hTA6GYJwFLKPdGW+lU32cBJZsDnxdfLK1AGjjKRMiAyuyfL123vmi1s2Jqb+b1myAkWcVjM6NIzKmAsyAAKzeTmvfrALZt+j7SKIXxGIyiLMWzR9oaCg34Vk1ZR5WvOFRo0DELVJRXOAwNIvoYrNckQ1Gw8jJFn9AOjd/V728F27V2HVCNkZ0C/VTo6sJjP3RcximJD+oRxO0jDN03RRb5dVbHDgeshIutEuSwq34a+Jb7rAbGxisdMRvd225SOfrCex9c3CZJ5aIo+vAmoeWt+ou++3X3dFXNulcQ14z2Me3oJVT5JwdjACJ42YLpYrK4GtR7GvfuR3KYfRGSfsMMStKh5gANh0clRgzbwFyb1uD5NQRCkMk39t5cTApsmn/ao5ZhNk6BFWUMsbNqglR2O5vtnbnSzGclgdIIt79vhLy5/EBYRw3d93P2tIYvTK10rv8F7HGEXRu0CC88lvLwnV/aCMX3mTUWtQz7QJYdX7DZrKaWfRFkP7qhNGMhzEh75MU6nT2dcRgYqP1MgCV4VCQbDJ/Y2tbJcf2wswzWa38TQcec6AjpPrRojhaERLzi6LB4yLUuk+EnJIJZCtv7OXbZGCxlqrZyx7fCmvnlJRNZ4FdI9fha4bKf9R3aCQeA5qUn3IZfzmiwSQvVxNnCpLfV8kXWERR2T13QOvl/aTrsI7IPU2M5lwHr++aiK3oXpoLMpAfSWNcKMoQQ== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: ab8e0dd2-61bc-439b-407c-08d8bbf45881 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:20.5894 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: swFV+ztCJshTvpFlYIgt6kSx2Kq6Sf7ZkMrCFNFY+4uwKsSCHrZL1rjOImuDh0WTGDbAzPmNNCnqjbok+MOi2g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On device removal reroute all CPU mappings to dummy page. v3: Remove loop to find DRM file and instead access it by vma->vm_file->private_data. Move dummy page installation into a separate function. v4: Map the entire BOs VA space into on demand allocated dummy page on the first fault for that BO. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 82 ++++++++++++++++++++++++++++++++++++++++- include/drm/ttm/ttm_bo_api.h | 2 + 2 files changed, 83 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 6dc96cf..ed89da3 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -34,6 +34,8 @@ #include #include #include +#include +#include #include #include #include @@ -380,25 +382,103 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, } EXPORT_SYMBOL(ttm_bo_vm_fault_reserved); +static void ttm_bo_release_dummy_page(struct drm_device *dev, void *res) +{ + struct page *dummy_page = (struct page *)res; + + __free_page(dummy_page); +} + +vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot) +{ + struct vm_area_struct *vma = vmf->vma; + struct ttm_buffer_object *bo = vma->vm_private_data; + struct ttm_bo_device *bdev = bo->bdev; + struct drm_device *ddev = bo->base.dev; + vm_fault_t ret = VM_FAULT_NOPAGE; + unsigned long address = vma->vm_start; + unsigned long num_prefault = (vma->vm_end - vma->vm_start) >> PAGE_SHIFT; + unsigned long pfn; + struct page *page; + int i; + + /* + * Wait for buffer data in transit, due to a pipelined + * move. + */ + ret = ttm_bo_vm_fault_idle(bo, vmf); + if (unlikely(ret != 0)) + return ret; + + /* Allocate new dummy page to map all the VA range in this VMA to it*/ + page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!page) + return VM_FAULT_OOM; + + pfn = page_to_pfn(page); + + /* + * Prefault the entire VMA range right away to avoid further faults + */ + for (i = 0; i < num_prefault; ++i) { + + if (unlikely(address >= vma->vm_end)) + break; + + if (vma->vm_flags & VM_MIXEDMAP) + ret = vmf_insert_mixed_prot(vma, address, + __pfn_to_pfn_t(pfn, PFN_DEV), + prot); + else + ret = vmf_insert_pfn_prot(vma, address, pfn, prot); + + /* Never error on prefaulted PTEs */ + if (unlikely((ret & VM_FAULT_ERROR))) { + if (i == 0) + return VM_FAULT_NOPAGE; + else + break; + } + + address += PAGE_SIZE; + } + + /* Set the page to be freed using drmm release action */ + if (drmm_add_action_or_reset(ddev, ttm_bo_release_dummy_page, page)) + return VM_FAULT_OOM; + + return ret; +} +EXPORT_SYMBOL(ttm_bo_vm_dummy_page); + vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf) { struct vm_area_struct *vma = vmf->vma; pgprot_t prot; struct ttm_buffer_object *bo = vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx; ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret; prot = vma->vm_page_prot; - ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + if (drm_dev_enter(ddev, &idx)) { + ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT, 1); + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret; dma_resv_unlock(bo->base.resv); return ret; + + return ret; } EXPORT_SYMBOL(ttm_bo_vm_fault); diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index e17be32..12fb240 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -643,4 +643,6 @@ void ttm_bo_vm_close(struct vm_area_struct *vma); int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write); +vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot); + #endif From patchwork Mon Jan 18 21:01:11 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028237 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7E832C433E9 for ; Mon, 18 Jan 2021 21:02:29 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 35D6722C9E for ; Mon, 18 Jan 2021 21:02:28 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 35D6722C9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 646226E5A1; Mon, 18 Jan 2021 21:02:26 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2049.outbound.protection.outlook.com [40.107.220.49]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1B7616E5A1; Mon, 18 Jan 2021 21:02:25 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=mqvImLXT8iwWBhE3kFUPW0KPyBCVGbsa7fPKBvFykJfOINO0lojKfn+ICtA9YG4r9UqMmnBYN49fQtasGb8+VdBqJxf9oOYh6RDfkT9RdVU56RQLtaC8ZSCJQNWuaG/EMMA88OVxyxwgj0AD21HOtQSbohRNDyQ6vdzDHMfvTXkRBMlJbxWQKnza/Qil8gzC3oVA4FGcUmF0z176zFyizLWdjT5PGjHkcThT6mbL3gFQI2lmku3T0ymEopyFlQLDtxEvnPvmC8WEctH4Rkkjtf+XKofoncZKcZn6PxhKa1uerspQSusnJ+sk+NebpFSgG53bRTIqe4riCBcE4H5uag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eP+CWFFP1/158PxaoXh8nWaz8kcCjPtiaThEI9ZA/Vc=; b=KX/ygTOCPv4Tad0eYqzSMzSlKn5tDfYDKVvT7RXaelKPy3egDKf9k8KQFg8o/7ZRIpqszyx8cmR6Wh5Wr9fDDaDu+ZwM64xdTvgQTMjoNGKFFlgzBym7YCYKuxyK++0fHL/dRgC0qYcNcWk6v2MXylhCTWIxpU0aNbgbjtkuViXi8h/m7m8CYrd1x+jrpFwEvIbK98yOyyWvqiz0xC/z3iuxMbI37djxt8ntiBX3BTM7iCQ5F2X5XKQSOmmKlJ27MwMULkbO10VzNBIUgBnEpVzIU7MlGQxoOWPoPrPEL6yn/xvkbU//26iEetHV+DADB6szu959ycbHjyuiuF4fCA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=eP+CWFFP1/158PxaoXh8nWaz8kcCjPtiaThEI9ZA/Vc=; b=2B/qZp2wAnzl2FjCN2TesIuaf0u0C4rn5AowQbnSsqvT8CYCLjVB5zpYbSxoGbfJhO0h9uzwxTJ68OjggQ4Qpr9ggKqRYKEr+uaU5RHc++9oHBcTYvp5Kc3MhRaGwLaKCIuoStbBaOF0teGp6Ps1w2Gv65oDC8hm2FtXhFEXM2c= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:23 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:23 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 02/14] drm: Unamp the entire device address space on device unplug Date: Mon, 18 Jan 2021 16:01:11 -0500 Message-Id: <1611003683-3534-3-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:21 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 5bfaa15e-8cef-47bd-ac83-08d8bbf45a42 X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:773; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hXTeKLN9X5KcSvCdgiNOFlRAB+onmHJtLBHvth3ThC7htiaf2EjCmx2SEkf7FlllqYb1jW6yGbY1nRwrwu3TNpt0feU5+26/lGfYYiiDky5Lny7jXqQzr09TaAOd8FN67gOyn9FvG9UIl/wef+8pI5yr+A3cBCBxITSvJ6izHZhoHGzw2qwHz+KZARHwpLhCwi2bKH54gca9l85kXZ6DnvD21CO+A46tlEYpIvwdBj+tJJ0U8hpRxxREA64InaC16PPBiaweSiF0zp1d17IEqYedGIgdS1qdXyX/pew5zIGsFhZgnHqXNBzrQZ/DbEZHuQhwkyN5pmx240vyQ8MetuOVwnwXhwEwPkJ+p0Oty+N1o+NkPvJ5VIfHWB7hN/v85kOx0BCnw2yzFIb/r9Am/A== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(4744005)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?HlEITUQbfbMQRLioUa6liswKXmFis7?= =?utf-8?q?bZjOx25jMfCfLFkLwtU6uPzDPVWcBxYBG1GPSc/NtvwxNBddGKS4qB7NC56uq/24V?= =?utf-8?q?vU0dfxfb6S3eq4+rM4yNTA+tcFKltDsnrtPPST7S0NUzs72/Vadq/6aJBU9e31Dp6?= =?utf-8?q?YN+K2zxZWYW+smi0i6mePH7gL5u/GuuPe9dIsiP8KmnkVY4LxAFkekW4dauscLBsR?= =?utf-8?q?ENi3TljmkQOV4oDo8rvV32scZD1ZMv6n3aWgEeqpgKbm9sOMlvuxgg3ley731ENga?= =?utf-8?q?hvmxkVEEMks9qZFM3aSfGearJowyNSKCucdmq4NPTl4rVpDwJnkrk9P6EUBQ1vQaL?= =?utf-8?q?4lsROkApS96SHo1J8xGt4KDNz8zmLoYC7cTDo2Nvs96xGezviCZ1O9bYn1MuGXpy5?= =?utf-8?q?kajBVDNbO0n4QdzPkhsCyypn3j5yLjteoh6LrNNSpju54gfZkZGX+B59i9xHO1gMw?= =?utf-8?q?hhHAIS1QCLRFtmrQ57O6XfXi5DDwBgzWf5bxvO+AwKfzNTEf86jeeulmKWKoVZLRv?= =?utf-8?q?LsWl7PVAmfEpAZTICd26blpuWnbnib69eMAYlQcNosxfOWPcWurNW/SAb/VKB3SIa?= =?utf-8?q?yzAq5fUz4HQem7RWn64gBTNvWrMgfF37WgQXT8KqjfJR56TUGUr/J2uF/01jLY3z6?= =?utf-8?q?EERsbyhVLx5025kfRKaGFYC8VI6Zc5KAY3fLIav6xlGy717yQco4RqqGhCFL7KVG0?= =?utf-8?q?Z8VlUAc+YhP7gua6bzfultNooovI7pYCC8kpYCTKZq1rDgP3OIUCmfDwNd9g3NWi4?= =?utf-8?q?9gImjQHbmb7gdTkfvcAUJUkJtRxjDvXM/hFb0lu2izrRFc8ssK5EVh3MSzauUd0Nx?= =?utf-8?q?L1zI73qyGuyYhnm9HirFL9ZOfQXzi47o4VL21Fku0PbP67GtBYwnhOVvR8dFx6NIT?= =?utf-8?q?mk74q4+AE3AUJb8xEjQt3EBnI7XSYydZNGChMIoklPZV8V0fna05ag1lFcJrPF27l?= =?utf-8?q?V7oCv+hbXqzgL0duqihCJXsDpcLgSMNC3ld0ydtQ8a1C6VrxAlGkzJ1kK2p8ahaPB?= =?utf-8?q?ETi4VmmvMYn7lL4Pok0tcqZgwv/UYXFooHPRWgNW3nQqlJN+Od/9A+id/D1AjzcRN?= =?utf-8?q?kHTVa5s+Wn6hTzXR6wof1nGJ1Qa0/nL1q6Tfwvw=3D=3D?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 5bfaa15e-8cef-47bd-ac83-08d8bbf45a42 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:23.5007 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: i01vlg1oywQwip2gGt6ME7CvwOG8u9VvMHPLq21aXL7iuukFTnnukScf9nE5Qf9/EghHEMz9w5bztwKfBiHUbg== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Invalidate all BOs CPU mappings once device is removed. v3: Move the code from TTM into drm_dev_unplug Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König Reviewed-by: Daniel Vetter --- drivers/gpu/drm/drm_drv.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/gpu/drm/drm_drv.c b/drivers/gpu/drm/drm_drv.c index d384a5b..20d22e4 100644 --- a/drivers/gpu/drm/drm_drv.c +++ b/drivers/gpu/drm/drm_drv.c @@ -469,6 +469,9 @@ void drm_dev_unplug(struct drm_device *dev) synchronize_srcu(&drm_unplug_srcu); drm_dev_unregister(dev); + + /* Clear all CPU mappings pointing to this device */ + unmap_mapping_range(dev->anon_inode->i_mapping, 0, 0, 1); } EXPORT_SYMBOL(drm_dev_unplug); From patchwork Mon Jan 18 21:01:12 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028241 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id D5EF9C43381 for ; Mon, 18 Jan 2021 21:02:32 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 9C1BA22C9E for ; Mon, 18 Jan 2021 21:02:32 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 9C1BA22C9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0526A6E5A4; Mon, 18 Jan 2021 21:02:30 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2076.outbound.protection.outlook.com [40.107.220.76]) by gabe.freedesktop.org (Postfix) with ESMTPS id A7A236E59F; Mon, 18 Jan 2021 21:02:26 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OpEViCvYz/30yjfnFvdyZoWWV8ZiQQMlmSkfBdbiq5KKqvL5BOlQsGA7ZIqx9NxR2yOQJDzK3JrAOljCuQ+Vx4iIDpgxvwW7n+S0mlq9mxmles0jXXxI0Ee1Q+CjTjlssfYTkAAyHUYo//5/Nauu+Z2+kQ01Gume+tj4N1sEgA8BY3YTqHqmVlJduxATuTGiBKuOOtgGy8WJGF6NY9qJiCouQACajYzd3jhd1vyOBu+XBGTz048nRolffkg9mvlJzYhFke7Ryv2/Ibj2vLfoU4pqxAwa9Qkl+3fN1sDhgtC0nzAtx9UCYCBzYhFEqoykcSbDXRCOA08KZwqQurFQ1Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aNxQ6p8R5S29VKWbdWbRF0nw4BHEli44FYyNd4XyeIY=; b=Mev1K7Vi5d17D+yYC1sOI1+m1Yq04zyFJMO23tnPKZSYQ71GXVWlhSpKAkuNF9B4FWninx6oiHB2HCTAmCBafhB6l0LNRp/NB7R5W3KWVvOgRkUY3TO1v0WT83PkNjtpqci9GOdBxvO3mMyDIRf13Dm6J+B5OOy5btlmdhVmTm/OyH6LOYKq2sDctO9sGAVF8OU7TuPqgPGvrYBx8q3M9mtKeOQrNw+hjtVlirv1jaZBqSYkXDpjXXSO5ShRNsMzRcqlBZNUVvZJXOy4H7lah02WasObCf7T23aA8CDyR6dL2tEg84K+et4tv8aCwtKVMKG8wQQgBMFKVfrDj5aEzg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=aNxQ6p8R5S29VKWbdWbRF0nw4BHEli44FYyNd4XyeIY=; b=S8ia0NMzEWAyoF8wwOgu9fV2bBnEYSJcKJb0P/4jfOcZgACiq5GWilmf9K5uJVWahSs/DoNOEpycTBeSIeqQWNKQehHUykbXxZM8F1HLcuQ5Pq18PudrHPHQLVSs3gg0nMzZFflkz1MrztUdCFR5tERezAfpuiE7LT4LMgcbutg= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:25 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:25 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 03/14] drm/ttm: Expose ttm_tt_unpopulate for driver use Date: Mon, 18 Jan 2021 16:01:12 -0500 Message-Id: <1611003683-3534-4-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:24 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: ddca2c0f-6aa6-4485-bbf4-08d8bbf45b5b X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:404; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: c7o4qajIy8c324wU4n0ykWvApMdu17hP1bAaB17n0kirJmHA15+n8rHlVAPA6LPkOSjuGH2Qf2Cg61vVksNmi76B4d6S8+v9eN8qm2PPggQzShA/lNtBtx0WcKjAhRGClf3O2TTmDwI1laqehmpbxBIQ0zrLooodJuBJs/gP7g2zRwp2dA4JjlI44BSu0zdDkYt5b9O8WTkPcZqI8M5QQnF0CoZ3T3ITw14sFeAnBX/dRJyGioljYAKH0vSqPcuQF797qeqbrrMi/2Z0VU7fpd5m7FKXgFrtLzmNDEi1PEGoMY15kLFfyRgqj73JBYyjOxXGC7QzF5duUabz/zxMiVF7qD5RTI1kWXyyTpxq7nH3Sl9vYDv9kzSXgU9qzC8vcuTdamYLbPe5YJAj7RmfEg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(4744005)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: r/9CbIsv7CsJlaQvY5AhwZ0gujoJM7nuJy/gM4Furvq0fnA/5TXSoQpuY2CXxxgpS9GRcsA4PeLTw7PfMS1oQdancBfl+0YuasXQtO/F6Bl4XgBw9NxRXSFifSejlkVaBfOkoHHtnIqVgzpuXtHfZR2aa/x8Ee2n8nzVZOyhMeDiWnNC2uCOk9o1V7J65R4ZgTbhGXYsWtKKKbX1UXiJhNXRFdjr4zReZ4i07P3FHLcgKMe0xn5VIir1v9PqGSOfWpMPEI0qDXSJYAtXSHcDQ3USo2Zy1zOlEL91+qAms98TmaGglsnTD2tJzAaSSvTTaJn+anqU4aGOxY1H2qrXVbDy/2jeIktRgRSKMwYSS7GOp1Pm9SYTLwVH6vwlBoFejslCsqrp4w4775piaNt7dArLgIRYf6jBqWSZzyOOixh4DKumYTRj/S4td0vhPgAytjl2IlP3lZYkNY2Zba0XUHlV66e5Yu3ol1y+aBxgiZCjLE3uKrPzZJNEPK51aea1Y7xVXhNebbTKjn0gY5FKeGIg2tJGUD8eiptmp7tdutoryJnVH7pRjdAH6Hyx0kDh4t3sCO7oIwBZTStLjXjwNYTahEit3fjLPyJFOdsmkT+554BLJ+tEBZGcRcEdJw/IYyvO9roQsFo1gNfvB6nPy6oKPWDreHQOladWWYFHg7OkPjAatN7rZU8FDIO9bnKG3CpDcpko3eiKOTzy/J0q6es4P1cO67agyXWAPjk70fSB3YOjd4LOSfmTbqm22XaJh8O7t8n9GGsS07tEgP0ZfukSt+k1y70Urey86idzOPw+EkVSD43KueKFFQeOgrXePHQ4cIqrbwuDp/PhvzTSH+eqI8Eb5DVpgEyxl+NPQWXUFE01kqUNVd76It+3xV4qxS+mgny+SXadtDo7w+aTYkahYGEtARm6cGF+/L9uLIimLvLQ6o6ehrvF9iBpG/dY8DyNPMUX4VpOZt2f2aeVWzMXyVbYx9NG36yuriDZA9T9/SId8j81p9aeNExp42pHEnAmdDvaKSeXu7WOMIBveCnZ8OGvtSbJ6O7P8EJnGVk3SDd68zAx1xA9pkN0fGTlF04sq8kZI425/FcvroTzrC5wM5JoVpZPeCLRG87/+YpR+pqivioZMl8pyl/o63Ua X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: ddca2c0f-6aa6-4485-bbf4-08d8bbf45b5b X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:25.4546 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: namgSRCff0nEWyUEVyWfECjmukZvS+2qzhX4La+hqqff/h0lyg7ybX1SEMRK0FLBQhPgC8+b01g4fLGx8zMa6w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" It's needed to drop iommu backed pages on device unplug before device's IOMMU group is released. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/ttm/ttm_tt.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index 7f75a13..f9e0b0d 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -341,3 +341,4 @@ void ttm_tt_unpopulate(struct ttm_bo_device *bdev, ttm_pool_free(&bdev->pool, ttm); ttm->page_flags &= ~TTM_PAGE_FLAG_PRIV_POPULATED; } +EXPORT_SYMBOL(ttm_tt_unpopulate); From patchwork Mon Jan 18 21:01:13 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028243 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 736F1C433E6 for ; Mon, 18 Jan 2021 21:02:35 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 344DF22CAD for ; Mon, 18 Jan 2021 21:02:35 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 344DF22CAD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0930B6E5A3; Mon, 18 Jan 2021 21:02:32 +0000 (UTC) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-eopbgr750052.outbound.protection.outlook.com [40.107.75.52]) by gabe.freedesktop.org (Postfix) with ESMTPS id 188E26E5A2; Mon, 18 Jan 2021 21:02:29 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Qdhq6+T8nA0nqrU5YHxIZI3hZ+h0eIpGEVx9KHf6FM+sg6Rxdab4CayRf2vHcllBmH0ik76yIktUJhNnKrFxsTItjpiqhDu8NJjrlGlDJJrIW0LAtQyatzzN/8ReMKhinMOaA84P7uazeXgGzIU9CYUrOHFizZpGUdB01Sz514Yf62DvVWdsh33hJJO8S51IKlBF7vctNSqyuQKdpE8PnfLun9Yt3JVyNjKxiono6Btrq8IHasIooEp/kkU+ZMief1cB74ctEN3LwkmJ1SpTbJDpMUKuAH132vHQr+XwL6tSi1z+uSB+GPiUD6c855ZEllFNXFmVG0BNMJDdonWxpQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8vLdzE+IJs46N8vQKS6qBAcqDAEa3YfT3MnnF6Zcy0Y=; b=m7L2Weeu1nAC2j+op5xtimQ1E/4IU5jDqBMuRLup2XkblxfRlfIc/pfbdg06JDStJssr0YIa32AkfwZsia/L6DSHUx0jN37bLeXdAj49BCexLqe73FnHyen4oq/cajwwh4hzAWVLqx5EFTwKsl97ogLrO6mYV+utmbxV2+yEbsyCM2Qu7fIGzhIXsX0vc/WoC4jdcHPLxEMlEgWkg5ZXUqFXv9ansG6WxRNyDmTDK813N1sN3ufbdeeQvbU6mb8MxplBLK8srrTMxc81O9yLwWcydlRCtqm30vPHkyCz8ux5lHaOnD0QWFt65McaYFi9ZlJtLHTR4++jYUUvVo0BEw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8vLdzE+IJs46N8vQKS6qBAcqDAEa3YfT3MnnF6Zcy0Y=; b=BH6FX0k0AOf9Hqv8bB1+Hc85jF4daVHd3ZMgEBicAuzYgi1qNWRQMZA+CpVrNGUjMblMCvu/UXvTU7FH6VbRL6TPwsHiD2+RoSdSUcUg+78IKa1WYtjEuymdRxr1A1yrqPNmG0QLL70LdWuR6is3+WoLZ12wQnL6qYPOm2D8hek= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:27 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:27 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 04/14] drm/sched: Cancel and flush all oustatdning jobs before finish. Date: Mon, 18 Jan 2021 16:01:13 -0500 Message-Id: <1611003683-3534-5-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:26 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: b0535b1a-2401-4ac1-db6e-08d8bbf45c87 X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:318; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: rt0G0pYTcdm19yrYE2/xUpIzQTeKlcGNZ5pP+sb646ZF7NR1gCU9IuIKcawVNKfOsqyS9JGZAztHWWrt5T9+8JQNmwTRYFqQFQevOgFlWhqPNqj85USyMm7iQJKq8w0I8ExD4lUjv4loTjN5u+KVcZ3iiLmffo/sCOKFWdJtCgMi4pUohJhYMu0jM4zlmPYoEl9mKKrUWA9XLdqen00M2E4SNBk5qB4i7MKFhid2DA2/KCSC4Jd0ChxdGNZYBHH7IzoRMPjRqwlFjowjA9LwCIdQq+eMlft0tGnzM8FTPJTh+BX57VlpTB+G2L94C+3DQUP0UbO3p0jVUpSYco/pEqiFy2/aQAgzEkdL0tFl2haic9ME8f8nd28qoj6LDcBJdeg+vKW+Hr5uJTXWJwVb5g== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(66574015)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(4744005)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?/FKLo+HrNXFiTWvsn4X8BkbZhadWVi?= =?utf-8?q?B70wWU7tPlrsCUpBw8iBUubbHVhbTTOr3kR8qjFHD4dFZvWuzwreAmvBEPdDR4e5p?= =?utf-8?q?oBD3u+qO7UQRlp/xygGT8xItw1sXR1NSJ+QGaqN80ahNJfgyrU6ylJjRwkDbL0GT7?= =?utf-8?q?BKEAdrvIfl/w+kHWzsv3AXkZAiq9BxwCsh1PuUCcu228pwVe7MJhvp2kSDMSGPTpJ?= =?utf-8?q?RjwXUMNTu7uNc8iBa6HZQSyVQXV40lUy8o69cY/bCzsPGcsi/IkIZmdkwWBaT7e8E?= =?utf-8?q?w7chstGT4akeR+2l9GmJakhd7vJI+hJ02+RsRU6oD8+aAOpuSzWwJOhrkqse5B7bM?= =?utf-8?q?APiLuNZPF0PfhtYoFT/p6xDff0G0LSqR2OgTrgbtYnFDsnYU/bxSwcahcB0Sc4QIw?= =?utf-8?q?sK/o2pOsazFzcCLZvP7JOCZr1rgB4KHNN7C12+g275btjf6N2ZXuGUpJpNKanG09u?= =?utf-8?q?ARpVsCseAGr4Jotxmm3/XDts6pjzLtf9F247O+9Y4Xf7yIKXMw4XMubuCk8x6NTQj?= =?utf-8?q?Lyk+k9QStdskQ927x7ve68NLtWKC7SEdSON6v0OY3kIqmy66/EwTsNqU/MbQ0dp3r?= =?utf-8?q?3sztgDF3RbZGG4QTHU6O0yQQzNmnZVgeSjB0ZZHc+MJCSPm73r7+KN1NTbDvotMnH?= =?utf-8?q?Vu1O50lvTsUWLHgnmpztTnxEIPM6YSUCClzImt+TVNVa/O7gfvXwFog0vdXyRLtNf?= =?utf-8?q?xDR5uYP/skdYPWrCSWRb7J72i8JyI/SFuPvARcXuy+5kn6a6qkFf6yZxlwkr9qbMJ?= =?utf-8?q?VnH/CcgFRIlHI1Qs4AxdJHjg/Gl267yHz9RWVK9SBHaQ56AkjJ3our/hBx4Y0HDvM?= =?utf-8?q?31hPu13uVnbnL2sbOCgwdE7+Zw1rV2Ev6qNv5/9ZKg0fuDgitjZvgEuix84AlbVU3?= =?utf-8?q?/tgg42IqaKgkZ/l99o3b7An3h5KQgHMukb79ztwoqxgmhmgP/ZqBm1ZgTEvj8LnHT?= =?utf-8?q?+AJD3ddzDNi/AmxVQTEge216DiD1bhhkEimOVJrWjsKcU54LTaa+l4+Ovi1Cpi1vZ?= =?utf-8?q?dI7J4NyFQxxfDwsiwSQ97ocDz0U6IsbQ2/WDNHlzp607Y3zh3+hkyjt2bEBHE3p/P?= =?utf-8?q?13W2kD9ie3dQTaGMe?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: b0535b1a-2401-4ac1-db6e-08d8bbf45c87 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:27.3965 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dLTK5EarHXw1Lct1o/bmZCpW+OTmkckEuJ9Qz/BKMjCJ0IIyfWcAiFvl1TTqdg7nnmAX/Cun/ll/g0Tf3XWQcQ== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" To avoid any possible use after free. Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/scheduler/sched_main.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 997aa15..92637b7 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -899,6 +899,9 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched) if (sched->thread) kthread_stop(sched->thread); + /* Confirm no work left behind accessing device structures */ + cancel_delayed_work_sync(&sched->work_tdr); + sched->ready = false; } EXPORT_SYMBOL(drm_sched_fini); From patchwork Mon Jan 18 21:01:14 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028247 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 88161C433E9 for ; Mon, 18 Jan 2021 21:02:39 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5509B22C9E for ; Mon, 18 Jan 2021 21:02:39 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5509B22C9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 32E136E5A9; Mon, 18 Jan 2021 21:02:35 +0000 (UTC) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-eopbgr750070.outbound.protection.outlook.com [40.107.75.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5195C6E5AE; Mon, 18 Jan 2021 21:02:31 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=iC0T/3g86MgugFxFReElKPeiKvyzI/ak0afpe7HJ6BtsAEfVIju75UuK9h++aBJC3yi1/L1sa+vQYHJamL858dRXRAXzuL7lnEAjVgF7HpAM0yZ81Umhvd1XLda5d5oXW3yfQ5ZsANEyzl2J7cfrMQssnP2t0UYZb66MzQKW4wc/L7PIhk6c2a+f1FB2Do+JqBjOh/iAc2QZ0ukCIQre6bKAZGsbp+n8tnym2cWGhsEOu0LEiUYCCs3AfoCtZQcM6jxa9fodbcJzxYml7OS7NmFXej6eboesjO0HP2MfT8/7e4V0s2+siRy/8TbVfMorWEWn0qUSXGxuyTw6ugr6wA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WPNvRvOuYfpxDobhAFMQosfBhPUt43d2TbuLSB3GNcw=; b=P+D/QxPFSN0UQ6m3WrBXIAVIXArUoypvpDvfBObiroxLAU29xLgB4EZ+EGB3ivz4WMBE0sKTF6azHKgFHJRbXcKMxlbJ5JCdgW0MLdxjH1CuoNVWn4AVCPNxxsVSfKPTWbPkO+OKZdkDNSqPJCpYcwE23ExHwN5Fig/G9EKAU9+RrccbwBWrw8q6W4JELzGBEkAaXeLWklOmHoUOw4oIgtMpJtvVc1bj0gD1AmblCoJJuS63FLTDAGbJTORet63mCv5OEU8FtvZj/X9+nDtBuylzkUk+hlEGN90y7eL4LcDYNtQxuoi9vKqyDDxHVPvp8VJPhfcfnfVq2jAOXkjyGA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=WPNvRvOuYfpxDobhAFMQosfBhPUt43d2TbuLSB3GNcw=; b=oAsVk0YOs+U4wi9iW7uyp7fkZRJkd/HCPNRkfCHc69TQV/QruSxhbmorsffbow05t9/jS8q8JXa1U+8wCBE6of6PTOGDftH/U+Vr5TlT6AAbI8Z7TRyO+oaN4sWbDWbfM+OO2tqWIpfmiqqzMmpnx2hrPfTWRWWuKb0pHAHHQSg= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:29 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:29 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 05/14] drm/amdgpu: Split amdgpu_device_fini into early and late Date: Mon, 18 Jan 2021 16:01:14 -0500 Message-Id: <1611003683-3534-6-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:28 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 56a92fb7-e8fc-4cb7-40a8-08d8bbf45daf X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:862; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: jSeYC2PFKJG9gHCDPGrZlYXN1sp5Qtqp1T7D3Jani6+sEqKGltTtXGwbCHoOy6XB1K7uMlA9j9Dzv5aWQ4Hkn35VpHd4L9uvPsWVlqMZW2bCgqViZCi22kr8aARp4zYwQG92irCQuBKTFPe0fEEaUyt/PgS0WI8PjCfELwoLsIqmsSA9+jW5crE/N3Xmb6Q2HzVzFEXdJcvVr8PqgAIx5Sprjg/pDCPM1SavhN0U/HLDE+d8XKZlyyafZRxU9n+lfNLqt8fhbR/20PNudOya0YuWWgyPufwPblPXYuGqJPnFH73ZTCvW6scAodbeejtPf8yJRi4DUXlZDHswVDHCcF9kQAXzO0XfJIIMflf62+tTqCIP29R/wuLJofFAynd4VaRrvZNBFtIvYM1gHdv7vQ== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(30864003)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: wZycfGio+tpLS4gcia8nn6N9Mpv65c/POOLAqyAGhY1R8QGr45DvrUCBpOJwA9wHiJAk6erS6KwW5n6UnGsrfm2XcUX4VmhkfjbmqnNeTKjDbxnP6ER9mfYalmqOUM6JCdTzmvUYYnZDzsbm5+Sa25gn0kmMPds/U2es3EZ3hJZGm1r9kjCLmxyLDGWIWy17cr3T4/29y2mR5Is+hmCx18TB+TLPW+lmvO2V3ggSdDHRGvVurHXAnCNSL0WP72jqPRHngd0Zss5d2TRechSuc3xqU87yjgRsUYRedTUrQ0pJt1AslyL8BrSjKxmZhnExCu+wD/VYCZ5NXZmZn4/588ATfEU+gNQO7PpBxsIwxrZcIFDvmtGOfVPp4LCPAJEobSZ6fGYIR9qCgd9GuOFCapu34qkLn77GeavVYiwm0uFTPbq4oJKbgZguoCybk5i+Uk7fFN4I+jQp6mphF8P1V/by6HkHREFhptydB1yVzvbu6ZeuVac2sFdKmlA5S3WlbaI9tGjoEoWF4NYFqq1REcU9kDQCV4okV2NeuXJlO/ApsyqUSk7EEvxXwBGy0B+LgBUx3ULAW8SzGEhRDgLl9LOBxPGZqTs+rN3aG1FvnLUgzDYRTv78bStAruUBMf6Vir4urnAxbaWLzjy7jXZwtFwtsZdU7k6RctfyrzHU41N5AlFy/KujuoGE9T5qRS/DyLqLyXZZ1VoQPuWkgDReqG+mAilsoWyybUJhur56p73joNm0zeusO6+lNfe5ILlePq5bD1tUq9rcVuew+5kiSEoRFRlNEqsDMoGOKFHcS6hkXKHgMseg4Iuf+RujbtNFThNFiQwSPbp3pM3AzysHzUCVK1mkajJQTEelev7VdvKTjcLcBpmfpk2g90mkShI8K0ugj/f+/c/FOVF4xQWMUZupPaovfm+DQavrlJZ/lGSqImx8htPuK5tHO8E7102Wr3geO0x7pQUYkG/RntB6tmO/+kg8YXM4hJ988lvyYDvSqcgI4h7KRLrUhBb4Zej1byQYExTUmRpjqS1+4NJ7G/M+qceu97AMN57HZ1WXu0k2YBoWVvs/gXmlBLe62PxBjN7UW96XLMmn6pRmrbK1s7AHLKcnRekhH/ihFZyIF/GeaLt/JYsV7rCEyYZUyuf8 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 56a92fb7-e8fc-4cb7-40a8-08d8bbf45daf X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:29.3094 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: fBhRGIcjINthvdSvoFipKemcN5oSN/KY4QfyHgzBJUiSwHKJG9M/24Yk1NFfl3l6c+P92OtLnuinsGd6Ms9BAw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped. v4: Change functions prefix early->hw and late->sw Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 +++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 ++++++++++++++++++-------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 7 ++----- drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 ++++++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 26 ++++++++++++++++---------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 12 +++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++- drivers/gpu/drm/amd/amdgpu/cik_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/iceland_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +- drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +- 16 files changed, 78 insertions(+), 35 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index f77443c..478a7d8 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1060,7 +1060,9 @@ static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_bo_device *bdev) int amdgpu_device_init(struct amdgpu_device *adev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_hw(struct amdgpu_device *adev); +void amdgpu_device_fini_sw(struct amdgpu_device *adev); + int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev); void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, @@ -1273,6 +1275,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev); + int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 348ac67..90c8353 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3579,14 +3579,12 @@ int amdgpu_device_init(struct amdgpu_device *adev, * Tear down the driver info (all asics). * Called at driver shutdown. */ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_hw(struct amdgpu_device *adev) { dev_info(adev->dev, "amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); adev->shutdown = true; - kfree(adev->pci_state); - /* make sure IB test finished before entering exclusive mode * to avoid preemption on IB test * */ @@ -3603,11 +3601,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev) else drm_atomic_helper_shutdown(adev_to_drm(adev)); } - amdgpu_fence_driver_fini(adev); + amdgpu_fence_driver_fini_hw(adev); + if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); + if (adev->ucode_sysfs_en) + amdgpu_ucode_sysfs_fini(adev); + sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); + + amdgpu_fbdev_fini(adev); + + amdgpu_irq_fini_hw(adev); +} + +void amdgpu_device_fini_sw(struct amdgpu_device *adev) +{ amdgpu_device_ip_fini(adev); + amdgpu_fence_driver_fini_sw(adev); release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; adev->accel_working = false; @@ -3636,14 +3647,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev); - if (adev->ucode_sysfs_en) - amdgpu_ucode_sysfs_fini(adev); - - sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (adev->mman.discovery_bin) amdgpu_discovery_fini(adev); + + kfree(adev->pci_state); + } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 72efd57..9c0cd00 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1238,14 +1238,10 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev); -#ifdef MODULE - if (THIS_MODULE->state != MODULE_STATE_GOING) -#endif - DRM_ERROR("Hotplug removal is not supported\n"); drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + pci_disable_device(pdev); - pci_set_drvdata(pdev, NULL); } static void @@ -1569,6 +1565,7 @@ static const struct drm_driver amdgpu_kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops, + .release = &amdgpu_driver_release_kms, .prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c index d56f402..e19b74c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c @@ -523,7 +523,7 @@ int amdgpu_fence_driver_init(struct amdgpu_device *adev) * * Tear down the fence driver for all possible rings (all asics). */ -void amdgpu_fence_driver_fini(struct amdgpu_device *adev) +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev) { unsigned i, j; int r; @@ -544,6 +544,19 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev) if (!ring->no_scheduler) drm_sched_fini(&ring->sched); del_timer_sync(&ring->fence_drv.fallback_timer); + } +} + +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev) +{ + unsigned int i, j; + + for (i = 0; i < AMDGPU_MAX_RINGS; i++) { + struct amdgpu_ring *ring = adev->rings[i]; + + if (!ring || !ring->fence_drv.initialized) + continue; + for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j) dma_fence_put(ring->fence_drv.fences[j]); kfree(ring->fence_drv.fences); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index bea57e8..2f1cfc5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -313,6 +314,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; } + +void amdgpu_irq_fini_hw(struct amdgpu_device *adev) +{ + if (adev->irq.installed) { + drm_irq_uninstall(&adev->ddev); + adev->irq.installed = false; + if (adev->irq.msi_enabled) + pci_free_irq_vectors(adev->pdev); + + if (!amdgpu_device_has_dc_support(adev)) + flush_work(&adev->hotplug_work); + } +} + /** * amdgpu_irq_fini - shut down interrupt handling * @@ -322,19 +337,10 @@ int amdgpu_irq_init(struct amdgpu_device *adev) * functionality, shuts down vblank, hotplug and reset interrupt handling, * turns off interrupts from all sources (all ASICs). */ -void amdgpu_irq_fini(struct amdgpu_device *adev) +void amdgpu_irq_fini_sw(struct amdgpu_device *adev) { unsigned i, j; - if (adev->irq.installed) { - drm_irq_uninstall(adev_to_drm(adev)); - adev->irq.installed = false; - if (adev->irq.msi_enabled) - pci_free_irq_vectors(adev->pdev); - if (!amdgpu_device_has_dc_support(adev)) - flush_work(&adev->hotplug_work); - } - for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index ac527e5..392a732 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -104,7 +104,8 @@ void amdgpu_irq_disable_all(struct amdgpu_device *adev); irqreturn_t amdgpu_irq_handler(int irq, void *arg); int amdgpu_irq_init(struct amdgpu_device *adev); -void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_sw(struct amdgpu_device *adev); +void amdgpu_irq_fini_hw(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index b16b327..fee95d3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -29,6 +29,7 @@ #include "amdgpu.h" #include #include +#include #include "amdgpu_uvd.h" #include "amdgpu_vce.h" #include "atom.h" @@ -93,7 +94,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) } amdgpu_acpi_fini(adev); - amdgpu_device_fini(adev); + amdgpu_device_fini_hw(adev); } void amdgpu_register_gpu_instance(struct amdgpu_device *adev) @@ -1153,6 +1154,15 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); } + +void amdgpu_driver_release_kms(struct drm_device *dev) +{ + struct amdgpu_device *adev = drm_to_adev(dev); + + amdgpu_device_fini_sw(adev); + pci_set_drvdata(adev->pdev, NULL); +} + /* * VBlank related functions. */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index c136bd4..87eaf13 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2142,6 +2142,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) if (!con) return 0; + /* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index 7112137..accb243 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -107,7 +107,8 @@ struct amdgpu_fence_driver { }; int amdgpu_fence_driver_init(struct amdgpu_device *adev); -void amdgpu_fence_driver_fini(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev); +void amdgpu_fence_driver_fini_sw(struct amdgpu_device *adev); void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring); int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring, diff --git a/drivers/gpu/drm/amd/amdgpu/cik_ih.c b/drivers/gpu/drm/amd/amdgpu/cik_ih.c index d374571..183d44a 100644 --- a/drivers/gpu/drm/amd/amdgpu/cik_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cik_ih.c @@ -309,7 +309,7 @@ static int cik_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/cz_ih.c b/drivers/gpu/drm/amd/amdgpu/cz_ih.c index da37f8a..ee824d7 100644 --- a/drivers/gpu/drm/amd/amdgpu/cz_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/cz_ih.c @@ -290,7 +290,7 @@ static int cz_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c index 37d8b6c..b24f6fb 100644 --- a/drivers/gpu/drm/amd/amdgpu/iceland_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/iceland_ih.c @@ -290,7 +290,7 @@ static int iceland_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c index 7ba229e..c191410 100644 --- a/drivers/gpu/drm/amd/amdgpu/navi10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/navi10_ih.c @@ -716,7 +716,7 @@ static int navi10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); amdgpu_ih_ring_fini(adev, &adev->irq.ih); diff --git a/drivers/gpu/drm/amd/amdgpu/si_ih.c b/drivers/gpu/drm/amd/amdgpu/si_ih.c index 51880f6..751307f 100644 --- a/drivers/gpu/drm/amd/amdgpu/si_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/si_ih.c @@ -175,7 +175,7 @@ static int si_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); return 0; diff --git a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c index ce33199..729aaaa 100644 --- a/drivers/gpu/drm/amd/amdgpu/tonga_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/tonga_ih.c @@ -301,7 +301,7 @@ static int tonga_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih); amdgpu_irq_remove_domain(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c index e5ae31e..a342406 100644 --- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c +++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c @@ -627,7 +627,7 @@ static int vega10_ih_sw_fini(void *handle) { struct amdgpu_device *adev = (struct amdgpu_device *)handle; - amdgpu_irq_fini(adev); + amdgpu_irq_fini_sw(adev); amdgpu_ih_ring_fini(adev, &adev->irq.ih2); amdgpu_ih_ring_fini(adev, &adev->irq.ih1); amdgpu_ih_ring_fini(adev, &adev->irq.ih); From patchwork Mon Jan 18 21:01:15 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028245 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 934A8C433DB for ; Mon, 18 Jan 2021 21:02:37 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5544522CAD for ; Mon, 18 Jan 2021 21:02:37 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5544522CAD Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 39BDC6E5AE; Mon, 18 Jan 2021 21:02:34 +0000 (UTC) Received: from NAM02-BL2-obe.outbound.protection.outlook.com (mail-eopbgr750044.outbound.protection.outlook.com [40.107.75.44]) by gabe.freedesktop.org (Postfix) with ESMTPS id DA4B66E5AB; Mon, 18 Jan 2021 21:02:32 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=eRzlpz+nQ60WgztlPl4NeNoIcRGx5tE6y49l/wofBoY0bwZ3PK2Ax2xrAG3uB2oZo2MwSghyFs04c/vHe8HkXQYIrvS5thf5VBg1ZhwGqGXl6sj5TEugb0g0l+nAfQlzJewIS107eIb7h5hka0+6WV02MhxW4jfwx96+Pk+Advq92hhuMVXHFPCZeMdrrp9lbzMRblTFCLY/M+YiBzRCt+Uv2dB3972ivjZ9P7dKoWrbBlkpqXGUO/3tq0AnzX3r5spjMQy7Mjm0VGn2Fc3tcTpVn5MQQq6KZFYcs5GZgdoXcIQ5bLLJ7DodJXV7ntE/IxHuhzAqAuC/LVAOwEVaeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LU/u1XRXyCv7y9tC+X+LdprijnnjsfvBr5Voq81hyEo=; b=hohpmb5e8AQmEbf0SHnxDSO0Puz4TIZr2KuaOKAj5F4EB1f307c8dx7UexHzU6jOrXlSZtDj3ddFlrOETd01kXFnJxEjBD0ykZHkBVsgPYZ4NcnJ/jUZd7TEBX9DR1W/XQT8sKjGeTYbp0+CRJBUbVZZX44rWbhO8zdes3LWkEl11ndmmsgbkHPJlBkN6llu9ju/rOpiA47VbYzymSJD88Gb6ZY2fk9kKgJBtSWc6/wwOMOU0VNMNtysz0DEi1R/jLAufaYiV9/OG5fAYLwQwGyAmqwdq+/ZZa/ZJIl/YiSiiOUprFZLMSyE8k4H6z5bCs4n+X4+HfXPq9Myg/p49w== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=LU/u1XRXyCv7y9tC+X+LdprijnnjsfvBr5Voq81hyEo=; b=I7Lp4MJ1n7CZVySUHv+R2c1C1VT4kVYF738ohyRtrcQRMF3eB9diEkRazvqvLgjnuDBAL1ufiaC+bsxGzQIjB1qfRlpYlbnG+Pn1gSKl6lo9/XUW2ddWZ6JDzK++G7RyfzaXKdI/6HYa9AzdAgqYF+IbPBp41+t+NAzb0picSGc= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:31 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:31 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 06/14] drm/amdgpu: Add early fini callback Date: Mon, 18 Jan 2021 16:01:15 -0500 Message-Id: <1611003683-3534-7-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:30 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 1757c35b-5af8-420c-0e38-08d8bbf45ed4 X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:4303; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: hTaH+HSWt147dOlwzW2H5RT2NeLSug8cHEhCJtN4MFzXuK7gibbqinKaozsbB6bqcBmNKlbmxxx3uPIR5aSC/JO8UeYuJxtlsD4W+WJTELfEJ2lU/Lqewi2xAbRDiW5BOC9NDq18VuNNTJbRGnmewIkpe9GJfg9doe8XflJy20VCjwugxK9w5zkM+7ml5deO+DFKcGeJV0NITqjG+CBMyDknrJ3G7hTBkk8znvo8D4bwVOTQ8NVHjJI3+jkfJ2xPErST4UH6uBYxyZJ93pnGSzPphJbt6N/21dH3ZygmcJnlD2IPGQm2Ut2TJ9TMH+e+i8rACX6ohR9vhTMIk9x0SB705TRl4bf4Zy5TQaI7LhMSDRVvMaczY9topWOrEQjv56nX6jDie8GqBDsXmRjfpA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: i76KWX+3rmBQqjfgu3lN8boYaJEcH5r6wlsGpDtXe/Aw0MFo9PPQqkKTCDbptyokGZesDTjQ6pEFp2XGB82yP/8M2MOkeoNT+bHtAntPL1vZZM/jYE6FkMWaamuuCWmw2EB2/it5vPo0HqENrmBqGV1iRPoHx29xMEg6mqgONS9PrNSRgzMSMkxksH7bhqqr2iUPUFYRRAA12UYPLkgOene12YsoAk4LP3Y4YJoGam/zRPP+sMOH7NuiL8c70jNPQlt/NRj+DO9amMNoA/qYTG+CAwOwTHxLk5AKFz6KIwa5y2n7+87a1NDvD6LRF9Ah9uqx/sahJZlcmfRJJipv4hNEu83WZnEhY/1yM2Q9VBejeucktdolnmv9hkoLydM40gdASR1DxyYoH3/VoQJtXbXBubDdcDOnI32IX/vufHRqV4v2OLVLPEkfGtRrkwk6GpFoRP0S6w4VuNcV/ZmogqBlHc3N5uziIj9DThuRxgWpp4gG0UoJJcjecVRmD0eYAj8+Kcf4nOjBrqUNkf7PcLbUfRSo4QEAA9/KPbmJYW/l0s4Endj/oJXdgknQ8omSPbdbosEvViDHreXyc5IHgTUY3d58mIoqfRJ7DDxIDNLcYSkczNV2m0rKLzRkoAP681yHuRvF3XSPrzj1kTBGVV6TlRbQA4VAKiOq6dItgy/n/I1+wwbUnxISCezqyNs01VqktX//9CJEL5CwgwrPCEISAeUWqTiaBEU4ZalG9fGBN2kBkNlTFRavxpoxdt82WgyXJzCP0y0rY5cUxwXk8qj6Gq2ZpFyb0fNQcY4+cv5cOHxzwqvtiBV5Qj2UlAaGjKEb/6AUDaL9g8Q+y2eJWuiKK6+imyvQICRy9jSg3dVK3EIWgkiYe7jJZMwp+H5KzH7Td6N0alGDByhX0VfwAFxHnWmMc9oerS2rP8A9STr/nVZZAFSXamojlcT0zxrLvRE+qb4qXp+gowChO9wk4rY5RjQ/TMSltOf2wDvbxwGblMuvsXTa2gJ1YQFfYxAdP5JelenEax++kD5wxLkYzktvSdjRqTrbeSqxrmqUaKHpbpx41TPV/K+UG5725Tk9UNRVFbSGBSsZZ5PYGAaVaJE+mvHrjV+GGUwHbB6LW5VzEohMtnYETf7Qmo2w/Ak9 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1757c35b-5af8-420c-0e38-08d8bbf45ed4 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:31.2273 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: hePAWrkL/sXQf4tTHy+jdjqMUf5HdQJd3I55aoZVRaQH5c0A1IyNkxgVlyNN7+NSO8erAXlyzapK9gTRelvfDw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use it to call disply code dependent on device->drv_data before it's set to NULL on device unplug Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 20 ++++++++++++++++++++ drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 12 ++++++++++-- drivers/gpu/drm/amd/include/amd_shared.h | 2 ++ 3 files changed, 32 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 90c8353..45e23e3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2529,6 +2529,24 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev) return 0; } +static int amdgpu_device_ip_fini_early(struct amdgpu_device *adev) +{ + int i, r; + + for (i = 0; i < adev->num_ip_blocks; i++) { + if (!adev->ip_blocks[i].version->funcs->early_fini) + continue; + + r = adev->ip_blocks[i].version->funcs->early_fini((void *)adev); + if (r) { + DRM_DEBUG("early_fini of IP block <%s> failed %d\n", + adev->ip_blocks[i].version->funcs->name, r); + } + } + + return 0; +} + /** * amdgpu_device_ip_fini - run fini for hardware IPs * @@ -3613,6 +3631,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_fbdev_fini(adev); amdgpu_irq_fini_hw(adev); + + amdgpu_device_ip_fini_early(adev); } void amdgpu_device_fini_sw(struct amdgpu_device *adev) diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 86c2b2c..9b24f3e 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -1156,6 +1156,15 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) return -EINVAL; } +static int amdgpu_dm_early_fini(void *handle) +{ + struct amdgpu_device *adev = (struct amdgpu_device *)handle; + + amdgpu_dm_audio_fini(adev); + + return 0; +} + static void amdgpu_dm_fini(struct amdgpu_device *adev) { int i; @@ -1164,8 +1173,6 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev) drm_encoder_cleanup(&adev->dm.mst_encoders[i].base); } - amdgpu_dm_audio_fini(adev); - amdgpu_dm_destroy_drm_device(&adev->dm); #ifdef CONFIG_DRM_AMD_DC_HDCP @@ -2175,6 +2182,7 @@ static const struct amd_ip_funcs amdgpu_dm_funcs = { .late_init = dm_late_init, .sw_init = dm_sw_init, .sw_fini = dm_sw_fini, + .early_fini = amdgpu_dm_early_fini, .hw_init = dm_hw_init, .hw_fini = dm_hw_fini, .suspend = dm_suspend, diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h index 9676016..63bb846 100644 --- a/drivers/gpu/drm/amd/include/amd_shared.h +++ b/drivers/gpu/drm/amd/include/amd_shared.h @@ -239,6 +239,7 @@ enum amd_dpm_forced_level; * @late_init: sets up late driver/hw state (post hw_init) - Optional * @sw_init: sets up driver state, does not configure hw * @sw_fini: tears down driver state, does not configure hw + * @early_fini: tears down stuff before dev detached from driver * @hw_init: sets up the hw state * @hw_fini: tears down the hw state * @late_fini: final cleanup @@ -267,6 +268,7 @@ struct amd_ip_funcs { int (*late_init)(void *handle); int (*sw_init)(void *handle); int (*sw_fini)(void *handle); + int (*early_fini)(void *handle); int (*hw_init)(void *handle); int (*hw_fini)(void *handle); void (*late_fini)(void *handle); From patchwork Mon Jan 18 21:01:16 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028249 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 24E35C433DB for ; Mon, 18 Jan 2021 21:02:41 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id DDAEA207C4 for ; Mon, 18 Jan 2021 21:02:40 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DDAEA207C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1E69F6E5B9; Mon, 18 Jan 2021 21:02:36 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2063.outbound.protection.outlook.com [40.107.220.63]) by gabe.freedesktop.org (Postfix) with ESMTPS id ACC916E5B2; Mon, 18 Jan 2021 21:02:34 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PfvZnEOebs1JaZBIQyQ7PDRE48RD3RHsE0EOPqI1JDFyn53hJOBM0dy0qKyIPEHpqDiJNHfljC5f2LDW2hgMIEUKoqf2ezIIqwYFcPyYa3zePdedEmMinBaSHIfwjgTbDI+fUBiMA1zOpC5MxooHnFoIPuRRiSU2D6hsun7Sy9z5sgUcIIrdtAv7ldX7+mgFuBanjVv7r758YsjncWkxdaLElrU4ILmXZvVPPousjebdktDtjEhqPkVMBeEBXZLRJspnLdZHN4Oi86dgv5Lsl6J68mT2etRC9yk7mwgqQ6An2uOG00N7uyRhV4XSPuZVGPFpZ8DVETQBX0OHX80iDA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/PfpSublj3Y8be2ajDdC2ctToz/Q9I2bjgB+BMJxNH4=; b=byFl/JPzmyeZgbDKsNmFiU0trh7Ayobd4mIyUOh2Z4ej/SA1ZFJ5IXZZdwPpkbVA3lwWxDJOJO2KBYaey4Z6TBYAGmRIjTcm1QJHiKk/DHfO7Q+dYqT4qA9kqiW+FBvy/THPA3tJEZsJGwj+wrZ8LfO+qxe/YYAfdl8maa9jRXt3KzvJu046ZDF8Yo1Q8CGDwv78VQGLtDwaqKa2RQ/9ZAN1WIsS1CnuAii8TCnzJxMajzEjOJsTdwxdmZPlRxnp4QCnwSxCBPp0JaY2Xm/gfjS8sJyobphhaH6EFjDseMAxOSX4MCbQjMq7elQUv+Bs+EOWA1bWQGakx6EEsTb/CQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=/PfpSublj3Y8be2ajDdC2ctToz/Q9I2bjgB+BMJxNH4=; b=w8Fp69HhRbpmJd7fia1Tf7n7QftxzkaSPP9qDGiWYpCaCyoBknDmjtJV5Z6YM4k6+JjvWmLjaDZhqXlRpHs9987iWzMcLfzDZJpnBLexaeH/gsJmL2ldbu/jP9G03gsq1TDjLP+B/bH4W5UMtCKZ7wojSyiD6XljKU5oYVPS/ck= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:33 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:33 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 07/14] drm/amdgpu: Register IOMMU topology notifier per device. Date: Mon, 18 Jan 2021 16:01:16 -0500 Message-Id: <1611003683-3534-8-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:32 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: dabe95af-f14f-4b55-d368-08d8bbf45ff3 X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3383; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 2YqEXsmnlUw0Cc/m4u8GGhc9fOuRZj1o36PGuMREQZDDevFqNplj6XbYCftrHxUkd4XtBusFdtRkArWZ/WVdNekQpkUKOqg/kKNchDekFCB7AAxDIDo8YpAn2w71/5E4Q5FuB/ftHlhZkEqj8BULTES8lhhUXm/uT/IznSSJZV0MdIXzNI0MPzEJEeM0/5pxY9WAfPGf+mcsYb3AJTmalwo89reX+3PHSS7WVuBOILtTb49YNClLxvIyEoGfRThcw7F3oQ+h08ciTKI8mCWjK4vBRcWVd1v5UcdeQRZaJbLZ9Wc9l6Ug3bFviUK30icZ21oimTIlPlnV6n09XhXH7tQ3fGGbBVmPdirMDkw0RmlH/Kf7LXoxmMGL0cFBYblT5JQHjRdt7tYqqw0c5mGuMg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: p55WdWnf3CyJrxza/vPha4vlCTVW3WnwXBkBNGII+zR4OaurFSF2d5MjiLwH3mEk78HVlpd9SpVnWJt4h0Q4A2UvAX8vkYU3UVgw5DlkZrRfDyDUrcEFTq0H8ZcY0akLQMyxDo1XNKRYHGwextJmX+jKxOkb/2/J+XXLJ/zg8XnFskRJJoTaZaQLIm3pvZwgb7xpTJ5hfm8D+DaFuoavoSF+vYECCHl2azicGRIw43HpXU2t8KYgHLIgGWk/Ao0z8SM3lTsoonw150LtMMIjSHrci2iGxPSvUsK38qf0wxgG55SOVMbK1epZV2mSt8fSb9mPz8lBQX92qRATIoKjPNs5NVTQzhGYct182PEOEkyDmkbLz+3e2+ptuLMEHR/4k90GI75X/oyAB4AxvBDD3ZdVdd0B1UlAclGqjN9+NjjvSGUPwkivKeKFd1SZV+iOSImgR8jyv3gllvOAAL2rdHob9TGzoUmu3RH+9eqhFJ0SeCVQapmSVUffe7TUEQ20TAJZ/mXchZF6O/qGslbJIqviLK4TMXq4uG06WTCG83WIxS3yhSW9i155QhF9IpA9PWGBYqSo9IIRtcgmezRWOFUKDuqnM6YXNS6zC4WFhHz5XtEDq5v7Sp34cNs6xnoRFjTX4W1OePI3Stepd6Nn5XLmtIjDUATvlQLFh6OlIUqwNL/lVLH6fqsVkq7+4C2Ssc6lj59FR3yecv8CVy/HjdD7/ZNq2BYuMc2SLxYM63f36KhDJN750cIDl/l8ccvZi+YH57+02oL/h5R3pCF8slPO+D/+N4KHcD6MPrJz8W2uj9j1eDihujcOLGHAVP1oaT5XZke/Lq4/zsxf4imke0f3l9Qr1IwllsUwp90ggKS3BFgOIrulUfOdezWXsqxmYfFm8CqG4cQojGZrviBK+xjzlBfyztyrPeLkIZNZyhMwdr/kOIYoCJpJT5OWBhgf2v/LNBSoH7pLNOPal3GePMK7h9EOj/qGhxhr52GV07jtun15Zfz4A7GScs4niQiPW8l7pFnO/B5ChoK3LK9vU2Z8Xxb19K5bzmJk00VNrcz4L9YR86DrcXaKnWpDeu5lfbDXU5+jHh6YeBGwVEi+KmByG7zYRRmL6sWcOT4iHekQ4U6yECJUbHSJgrq9vke8 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: dabe95af-f14f-4b55-d368-08d8bbf45ff3 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:33.1543 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: C8UcuFLycGT8JJjlaPjlggulrO+wDRCEAYXmY11JxYzpgpoUAypFCczzHS0vxKT4eTqu6YP4ulzXRmivAJSwPw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Handle all DMA IOMMU gropup related dependencies before the group is removed. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 ++++ drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 46 ++++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c | 2 +- drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 10 +++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.h | 2 ++ 6 files changed, 65 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 478a7d8..2953420 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -51,6 +51,7 @@ #include #include #include +#include #include #include @@ -1041,6 +1042,10 @@ struct amdgpu_device { bool in_pci_err_recovery; struct pci_saved_state *pci_state; + + struct notifier_block nb; + struct blocking_notifier_head notifier; + struct list_head device_bo_list; }; static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 45e23e3..e99f4f1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -70,6 +70,8 @@ #include #include +#include + MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -3200,6 +3202,39 @@ static const struct attribute *amdgpu_dev_attributes[] = { }; +static int amdgpu_iommu_group_notifier(struct notifier_block *nb, + unsigned long action, void *data) +{ + struct amdgpu_device *adev = container_of(nb, struct amdgpu_device, nb); + struct amdgpu_bo *bo = NULL; + + /* + * Following is a set of IOMMU group dependencies taken care of before + * device's IOMMU group is removed + */ + if (action == IOMMU_GROUP_NOTIFY_DEL_DEVICE) { + + spin_lock(&ttm_bo_glob.lru_lock); + list_for_each_entry(bo, &adev->device_bo_list, bo) { + if (bo->tbo.ttm) + ttm_tt_unpopulate(bo->tbo.bdev, bo->tbo.ttm); + } + spin_unlock(&ttm_bo_glob.lru_lock); + + if (adev->irq.ih.use_bus_addr) + amdgpu_ih_ring_fini(adev, &adev->irq.ih); + if (adev->irq.ih1.use_bus_addr) + amdgpu_ih_ring_fini(adev, &adev->irq.ih1); + if (adev->irq.ih2.use_bus_addr) + amdgpu_ih_ring_fini(adev, &adev->irq.ih2); + + amdgpu_gart_dummy_page_fini(adev); + } + + return NOTIFY_OK; +} + + /** * amdgpu_device_init - initialize the driver * @@ -3304,6 +3339,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_WORK(&adev->xgmi_reset_work, amdgpu_device_xgmi_reset_func); + INIT_LIST_HEAD(&adev->device_bo_list); + adev->gfx.gfx_off_req_count = 1; adev->pm.ac_power = power_supply_is_system_supplied() > 0; @@ -3575,6 +3612,15 @@ int amdgpu_device_init(struct amdgpu_device *adev, if (amdgpu_device_cache_pci_state(adev->pdev)) pci_restore_state(pdev); + BLOCKING_INIT_NOTIFIER_HEAD(&adev->notifier); + adev->nb.notifier_call = amdgpu_iommu_group_notifier; + + if (adev->dev->iommu_group) { + r = iommu_group_register_notifier(adev->dev->iommu_group, &adev->nb); + if (r) + goto failed; + } + return 0; failed: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c index 0db9330..486ad6d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c @@ -92,7 +92,7 @@ static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev) * * Frees the dummy page used by the driver (all asics). */ -static void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev) { if (!adev->dummy_page_addr) return; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h index afa2e28..5678d9c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h @@ -61,6 +61,7 @@ int amdgpu_gart_table_vram_pin(struct amdgpu_device *adev); void amdgpu_gart_table_vram_unpin(struct amdgpu_device *adev); int amdgpu_gart_init(struct amdgpu_device *adev); void amdgpu_gart_fini(struct amdgpu_device *adev); +void amdgpu_gart_dummy_page_fini(struct amdgpu_device *adev); int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset, int pages); int amdgpu_gart_map(struct amdgpu_device *adev, uint64_t offset, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 6cc9919..4a1de69 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -94,6 +94,10 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo) } amdgpu_bo_unref(&bo->parent); + spin_lock(&ttm_bo_glob.lru_lock); + list_del(&bo->bo); + spin_unlock(&ttm_bo_glob.lru_lock); + kfree(bo->metadata); kfree(bo); } @@ -613,6 +617,12 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev, if (bp->type == ttm_bo_type_device) bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED; + INIT_LIST_HEAD(&bo->bo); + + spin_lock(&ttm_bo_glob.lru_lock); + list_add_tail(&bo->bo, &adev->device_bo_list); + spin_unlock(&ttm_bo_glob.lru_lock); + return 0; fail_unreserve: diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h index 9ac3756..5ae8555 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h @@ -110,6 +110,8 @@ struct amdgpu_bo { struct list_head shadow_list; struct kgd_mem *kfd_bo; + + struct list_head bo; }; static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo) From patchwork Mon Jan 18 21:01:17 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id EDFEDC433E0 for ; Mon, 18 Jan 2021 21:02:44 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AB99B22C9E for ; Mon, 18 Jan 2021 21:02:44 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AB99B22C9E Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1825D6E5BB; Mon, 18 Jan 2021 21:02:42 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2085.outbound.protection.outlook.com [40.107.220.85]) by gabe.freedesktop.org (Postfix) with ESMTPS id 707306E5B4; Mon, 18 Jan 2021 21:02:36 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Kh/tm237gPkJ/fUenhVmIZFpEaG6zBLT9zb3oC1GG38I/gKkG2txAZKFLbz2g4NHmtBS3x4LypUDi05OvETOExdGFxrOVQ0zdSwwEE3Ova42c43gQxEv1rQkuYmGqfjeqHWXr6/1Sr/PnVtbotjE1lfaRwkfdjD5FzAnC/leDZzxKzLiBw1fGyR+rQGi+HnIcfXKgR8drNXq2Gj4cWR51IvT5jf3tyrDi17RlPF75T6ZZvwdmR0Id2r+qaDo1NXdOuCBaNCQ/SEXALngzKvJ34yQ/gdLGiOnoit0KuJKTcL99StTPrzLNyFZBUKvYKZ2VD3vk8wBNN1tWJX4Zh8uKw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X+r4VsynykAXVhAXMWzkWfYRx8xc119kunjkr3AZ3Rg=; b=LqJ4o/vF2Rh+H6JgaRZ1g2Evv6XxuXL2F+OJOYDYMlgOkeafsJRCvF5IV8HIB4uZOcQ/IEQ99kSk8HuzCvlM6VOiqysU5bmIEzCU9A6UXQIJPN+i3qG2Cj1bhSz/gJJBvJh6QwrVKMTnrVkvGSYiMBfjhavqA7IxliZKskFiF+xrJ1T7tPVOJXkxw9/wRXd6LY0v/n/K4Cn42uquhP6b2gD6VQNm28v396Dmx5aFdkviLryfPCpcgOQ+dEzxRdWYmb1lyggJ+NT5mu6EEbeCk5/LbMcPJQ8VbHfwddG74lr+q22YbMir2PQvbMLGKh4IPnullHutn+7oGPyRJmBhHg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=X+r4VsynykAXVhAXMWzkWfYRx8xc119kunjkr3AZ3Rg=; b=TUQNSWXL0IxfQ0FGOTryPFlw8QDfsw6ghXHkJp57KQsG9bh6f39LQmvx/1r5EbYMekuWmcro0InaWL7IGfNcVxdPFSK/BzEpEjBRbp5NYR0E2/7le2ek6HrW9/kx+olt1cnND7J5zuy8YZiuJD3zdl9+0gcD3ax+b6gcKMdQO1A= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:35 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:35 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 08/14] drm/amdgpu: Fix a bunch of sdma code crash post device unplug Date: Mon, 18 Jan 2021 16:01:17 -0500 Message-Id: <1611003683-3534-9-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:33 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: f05136ca-1d2b-46c1-2763-08d8bbf4611a X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:346; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: LIrW+crzhdtkmpwuu1PHBmzD3dCx60Wg0gvquIDwYtFd0i1Xwr7ds/eHMv8U61j4rqlLdjEZWUaVgmN2P3zHinZL7rrWPTZA0rVHvqGSNLJ5reBvjl4Zbj0jPlNrbg0JRIvHqhPAivtretLORo5hG/kUm7iXnzKaPih/q4YTtbqe3rPCxxaHmB4oMwut5RSZGiaIJA0Y69N2i/0IZ4Ui3lMGFu4Jix3+0lhbURPiRciKzDR6U8cqvD+uGXIf6zAUIVvDduJF4NqcDVDnrdRXNz7qoAPl2UP/XtGvC6dvA24/CTy7QyyNGgBpR2FJuMQsUiNcD1dY+UnXYjIjFD36Ls05c6GcgLALzaKISilP1UBPfIDFDBaclwC1DMNmR7AKffSJF/oae0VTBF6udimNFA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: k1/PSckwlMx47+46ivmw9OecHgNq8eox2PGwN6Tz/eyUCjTh5S9gOCmpQbjT9L+CoTpbVe69GlKn38wzdeZHAtQOGA7Gq57rS5dqT26eH+nRwupu5zl+eBCQmQYig6O3Wsxk+0yEBmantUIJXM49DZM45BhYe2BV/0BnOBU04ibEsu56bsdJtmrHnfZ/Dk4j0Y+Em+8Jyu3Lut1v3kxPeMMcF3hAyU2SX/NEnk2/Ih/sxCuOSTbrM5Jg3oSgCHPqUESg6qT/MzKHx5vYOipTYY1UXRlu3AuZX9MoebL+xW3BmKrwADsfHW0VlW4z6jcmr3uK91fjVeNf/y0WvWMpIXvAZh+soMfjC4rWNlgBUtOZ2EBvmkEqbEMhSiLrsU0JE20sueIFzGRadx/EW98752u/KgPNjKCJuofNSty6IxQQ1loCb/P/quvVcqeaxJiWgYxCOZcmwwJvgWfXB8Pdsv66cJ0R1tPP68osSwU2VrPzaxS3nzncl6TDDpe7IqQzPrStCERDCOeBjq48Dkg7yEIJ+VEsVciJMqdKts9Fgtm5qdp73Z8bsROyFG/n8khEUlSGDnVb+E6fScV8pRTwWkqQOD2YxRpxvs2LU3xKvyouKW19t6mGQ0i6ZUHUPV6LtVxsE9vva+EQRZpNiUzbzDvCEOzO7tMohXJdy1xMQC/4Ocd3kxVZ5kFxlhheIUvIRoe3X8EN7MmMAe4wHnaWEQS4i/jnQb/0pwlI0tqYqLQ9hIWEodp3u76ER75eaBZfkX9/AXO0X4Nf6c8eU4UXriVjNGu5XmKIBlniiJxjKOU8S60SOoSTDpfXGCCLEVVX31K/Q90AjsZgUy2qNRCDWgMYi9wtCjpxpubtw9eLxlmHU4rfubOImJHSLIYtV0vogWFt8TuZaR+IuzVX2/HTD66h+kcynpNeXE7qPVQKYzyTLUQMzHwc4A0TTcYZoRWPp8U/qETuCDPf/GpniS49rnMZanbJ+Xh7LQDj2XUCha9avLWsoRD48zniXODbCI7oyIkI23lL5ke/gl14xrbTZebv0o6L0SfWYZ0WmC9lPsi9QXijFT/RVKmdVi19oOeBsb5Xd3Jf+zSlTbwgwXZTDMpIifdbP2GVHaUR3GU4kOcK5a1/fTGfLzMVewCRBxmb X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: f05136ca-1d2b-46c1-2763-08d8bbf4611a X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:35.0692 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: YWojah+DMRDdvfL+r5fsRk6kGezEqydO6K33/nlCb+36yFMpz1g3fdAxmWOC/WvcGlasAXBvAgy0cMOH6/hGGA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We can't allocate and submit IBs post device unplug. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index ad91c0c..5096351 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -31,6 +31,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_trace.h" #include "amdgpu_amdkfd.h" @@ -1604,7 +1605,10 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, struct amdgpu_vm_update_params params; enum amdgpu_sync_mode sync_mode; uint64_t pfn; - int r; + int r, idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return -ENOENT; memset(¶ms, 0, sizeof(params)); params.adev = adev; @@ -1647,6 +1651,8 @@ static int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev, if (r) goto error_unlock; + + drm_dev_exit(idx); do { uint64_t tmp, num_entries, addr; From patchwork Mon Jan 18 21:01:18 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 9985CC433E9 for ; Mon, 18 Jan 2021 21:02:48 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 5FE5A207C4 for ; Mon, 18 Jan 2021 21:02:48 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 5FE5A207C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1CF786E5C5; Mon, 18 Jan 2021 21:02:46 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2079.outbound.protection.outlook.com [40.107.220.79]) by gabe.freedesktop.org (Postfix) with ESMTPS id 523D76E5B6; Mon, 18 Jan 2021 21:02:39 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=j3KDIdkY1Nz26ijLTpw1jtUtEa78jIGPd7kYEudTpX/D964UPTT5yrplJw20TEANTMB0zZda/t9oOuohB57KZQM5PdpjWOC7xhg9BzhS4seSu4BlYFhKhHLkorn2ehCvVNvTVjt4TqsSR2jmvWY8ZK6VKNZ/r61pzQ18gzpiyl0zj4P/L6PqsMeGAe5Ov8JDZfaaaGvinxI+MhHvHVJfZ5sY9mJQoCVfGQAZCRoaT1XhWkuvUlOPdsuEmmMvXW1e1eb26bkUWybBXWCp4DPaCmhudRPK4pwMunDCTSuKjQOUt/yLJGHaC40XnIneOsuc2xWi7YFWaz1aHol40M66zw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cXXixyby5649rHgUJ4L5NHbYCr76Z9/oOHEt5mIK4Xw=; b=EE4TZHfPquuilesk/qR9cnBDyJpdeh0cOFvXEH2McAl81NueH8qQPHTDR6L2QIeSDQ2il5h5OjnZYZ6rO8MzIiM0vPIWVBALJiXmxx1yXS7zya1OAxwD7Ju03UqpTGBOMjw5LjpzhcxXjRI1U9P4FWW1W0DpVSVq3dDPRkeopIePTn5/OiRHm5NNqmD+T+Bi4CfHKH5ZGyIYoYdvJJp4iydZIN3u2BX1bQRUmXN8fywRwSsOlN/dwFKzvPzPQTIX8jZTLWD8yE44U2CNZ7ropvywSw2iByRT6PJpWdAhSk5+c9op/jpt8YGOwo2ISWgASsQSW/2yErwRZsCFTZzLJg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=cXXixyby5649rHgUJ4L5NHbYCr76Z9/oOHEt5mIK4Xw=; b=E1wPj1788oGUhxVnNdM5p8HGrYJ8irNfFyFyI4f4X5CHX4dMPO0YF+XJrAJom0omOy/NLC5e5+1/nSZnuCtPSuO6e1iMrSoPK7YnTMF4XShpU8yIHcqluIdEnsCZ9Tt/JapgubcUN2UbvpxkC9YBUUafDrvtVS9EA7Qu+gTpgZM= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:37 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:37 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 09/14] drm/amdgpu: Remap all page faults to per process dummy page. Date: Mon, 18 Jan 2021 16:01:18 -0500 Message-Id: <1611003683-3534-10-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:35 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 92a2c8c4-c457-41f7-3de9-08d8bbf4624a X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2000; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: VQytVmqpg+Cnj/y8ts2EZYXVXU12nYFYqb9Y2AWooaiiabM9pQ2L3Mnzq5K11JvAnpQwg5XLVuy3k6MvcoHEX8K/CWzSrnH52HGzkVyBSS5BuBHX+EHVmy/U6azweeLw/w5WF6euU4lR8nF2/sXyr8BXSMVBOlerLwnl+WQ9+fomGKhlGYs6pBioVAPh5oBtTjGsGfLrbApZdDp9SCHQjZs7Gs/XH8nOkEfWR6KSkJ6Xi3qp/ZzuehCLBA9c5oJIIsTuoYb8oOO5/TaRfBcTLb/aGMMTa9INipH6QUcWMliqN6SYU7AYN1Bzhsc32aNwTvb2OwEwV0gsc7wlk0vA5kJBIEq+Kw0MU3wFBgGphJT9rYLmlcBGXD9d8EBw52Vmr1Cad4zNGty+X7sYJ5zmIA== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 4Kl2gWI5dsHq7C0BMZQmQvdzq4U0WPIXFsB8IAW2dgv0wYiCVmEJQmo/IX96F5Vpkkjqmzsqcvs0h80tecyHkxC2ubLgol9bHoKdkvDlYQBYAkRZ+6sqbOrXxgqUOgan/DnJSp9JR1BWOgc2Zcg2uKKfGu91WnyBx5T/rJ1kSkEjb2+OrVnuE6N8EGP/bbjx19eir+r/oeBdFvVQiReMntzyBtatPToZXoNaSV0ZY2997i6vSi1bBvHAO8N8+NT32XWp4FbU8YANHUNreQdel6rUN4gMVgpCqNIGpDGfQCgBLryOKeFKz0BfP7kxIVOShXJNUPI5YgfhuE7voudB5WX4pMflWA2aMpD+mqIZFoHtt6TWmdkCNm+CeHOgFhnzVBBuRr4a6j3rRdYeMEh5KIXmOdvuQ+lctvsuOczX6DsoQkDxeZ91hiCqNGfmKemXZJqWwS4C1qIcXRglxI4eFKhBg8AWVrYeQCTIggeByWgnCXHZSMFFiMLRtmezmmRfL9uTyVi5S/5xIPMd+KVagR4vKp376h4avjlA6DS4II2gRV66vc9kt+Qg0ZlxmcMQ9caumtcWK+VqwcbJnSMMo0Bc9sNPjHrNaxyVZkyRuT0BxIGhZfAsTqYf0DSSjRCs7DQ0FbCK6RRGIqc+36xqq0chvQa7R+oynYufScaaLscD5C0vzjkzzwPzldbpjRkctTTBxyQvwIJl0BZv9I7BS7yTsGx7eJjO9lm9v2CQEDMYLg7h3eBElFp8H4FWKuqqLTAM/B1r4oegI8rI7i3o0YYoqsZwfmc2AzfRbhmjQctWu18aBsYElJ/4q11b9OfLLDE08bi09+cqGfJkMsOXn26E7PIjcIHOm3w2rejnswTZvgUu90rEgrWQ/qG8S2ENvq1RCw9MQozxiw47WbVUEw81kZLjcXxWVJTZyVNUGktBNG1edOWkLbCVkoOtitKUTuPINROSdb3Tq20TMuRHrNZCz/G4Ns++Ch9qQcD+E7wAceFV1A8cfR4LOxdFmrLeVq7SLoX2CYw6xRcdPKIbxkU0Yju1U2FzrTUl8X4vXgoyvzZMMZ8GYv8gQwdJJlA3pbdyT+CGBu/0XXC0X3AiVtdnsMCXI3TLkqtufO1SJIC/TfFkE3MetjRWC+ZciVbM X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 92a2c8c4-c457-41f7-3de9-08d8bbf4624a X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:37.7107 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: QwQHwvlU2l6oDf/n0s84643RDL0SS6wlIylj2+OqlPK2qKUdyJXWQk98Hn6KC4pOKmyRSrKwLvLbdG/T5BB79w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On device removal reroute all CPU mappings to dummy page per drm_file instance or imported GEM object. v4: Update for modified ttm_bo_vm_dummy_page Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 9fd2157..550dc5e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -49,6 +49,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_object.h" @@ -1982,18 +1983,28 @@ void amdgpu_ttm_set_buffer_funcs_status(struct amdgpu_device *adev, bool enable) static vm_fault_t amdgpu_ttm_fault(struct vm_fault *vmf) { struct ttm_buffer_object *bo = vmf->vma->vm_private_data; + struct drm_device *ddev = bo->base.dev; vm_fault_t ret; + int idx; ret = ttm_bo_vm_reserve(bo, vmf); if (ret) return ret; - ret = amdgpu_bo_fault_reserve_notify(bo); - if (ret) - goto unlock; + if (drm_dev_enter(ddev, &idx)) { + ret = amdgpu_bo_fault_reserve_notify(bo); + if (ret) { + drm_dev_exit(idx); + goto unlock; + } - ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, - TTM_BO_VM_NUM_PREFAULT, 1); + ret = ttm_bo_vm_fault_reserved(vmf, vmf->vma->vm_page_prot, + TTM_BO_VM_NUM_PREFAULT, 1); + + drm_dev_exit(idx); + } else { + ret = ttm_bo_vm_dummy_page(vmf, vmf->vma->vm_page_prot); + } if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) return ret; From patchwork Mon Jan 18 21:01:19 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 48D58C433E6 for ; Mon, 18 Jan 2021 21:02:46 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 05D3F207C4 for ; Mon, 18 Jan 2021 21:02:45 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 05D3F207C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 052EC6E5C1; Mon, 18 Jan 2021 21:02:44 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2059.outbound.protection.outlook.com [40.107.220.59]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1FEAA6E5C0; Mon, 18 Jan 2021 21:02:41 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=kVVDXWjP5WjwUXDqXqmh7p/UxQseTc7PRgGJNyd0DjjzNQRGMCdtAmwnun7OxK/KatRMpPf2Pl2oq5WrKQmX3+chil9t9X36V/L+nUPA71kGM5ELMHKRS6IbVEX1Uj8xDXqh0LkXSQXERR2zOG3TLfij8nchVimJEfoh60+ceRh9ln/qlUZnKmsrrlCXjlt6VwABleDalLIjLPZfQFR4NJJ5nj47cGTOxWnAcrwnIy8YRY6jZKlJIuKqVu2PHyE8xyRs7eH/5R6BzS71uyrUX2WzRM7hli46ke7tXTjj+QxNk63DsPhyuLwRB3fqwlTzFhJoloGxIQ4zCVKxtk6PAQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZZJkk8cFpYaKfEzH4uzSnGT5LR4Ffy0zVO+eOOmvhKU=; b=jEx2w8c5sTMtwVOtl/yfht60yK8Fq5LfiCDPJk7KZlVmbqQC6v2M3MoO7nM7CsTumTFwOJ+QVBG/kCT4/PBxdiMoY97s8ONwGhtJ24J/eJ8k72zh1Mh/qU7dBZI73rr1EQW/syjylY1c6w2LCsld4TjpYBaZQWKxIVgoW4DkRwGHKvFW952T7IYplr7fzr7Ebt1ESCrzP81MkwVBfqxARknTS+arBq3X3gnuvqijDAcMU/9XQ/xg5EY15rT6vqqmFqFN9Ffk6mbLNDJ/T8loKCWgaNoPhT6T2C2HDf+vXK8j2D8FLC5iWyL/yoYxM9+FMH/3CZItyhIKGfGhz4E6Wg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZZJkk8cFpYaKfEzH4uzSnGT5LR4Ffy0zVO+eOOmvhKU=; b=mfYc1INl73q//jWcREZmAsZ8TIn6GZLO4p738Ufz4cw30UH0e7NCXUvZI6VbIS2Gr2F+5CtrvtRlFi5zLfdJT1e7po55cwE0e6XN4BDCMf14rZscW0P1LgP0+EC4U17A/WtXpxpOk7n7LOnffmSZkaQQDi+wP/jGvJPj9D7oqfM= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:39 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:39 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 10/14] dmr/amdgpu: Move some sysfs attrs creation to default_attr Date: Mon, 18 Jan 2021 16:01:19 -0500 Message-Id: <1611003683-3534-11-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:38 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 0a26aff8-3467-468b-72e0-08d8bbf463f8 X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: mHy78ggpsj1/WUVFhYq0ryOFSOHnLGQOEdAjB8yGskYz6MjzqF9tAQHmtDEyChNvOyupWlBfNIBaxoXM344cPNMYWgAfFfGYpHKsfzez4QsbVjX6xSL8Rg7T1b5j+s/wm/taRSOljXYUEqooW+xCwl9xzGzVLMmZp39IG6VP3FhFTN6Sdut4bdXqpxEnVnmd2nFCI/fZOVkM1SKEJw/PBa/qXxSh7d834kiDpxeceyF6Bj2lP2dbmpFpsvbK4xaOOwbKOV5lvVAZ1ags0YKJ1/thra/6uSEI/zV0O+lM6sErMptjWJvSW1bTtprUWiDcqxS5tGzyC4cIoZ6dCwZJMR5EZVNk2DasA1GJIECBIXJZ0X34+OvD3/gVnJL6wxhOdeTisx7v01clYeOfzCMmzg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: wDa78jxgb5WF1s/4lHr5GglVsNG7tCcnv0p8QTuocwtSfmOjw7+GfKVQyb5eKgjDdS3u2ZBXkamAJOwTn0GRTf4ZeRuOLbKCV2CrQSY3L4ELnMD4Lvm0ARANEZToLy8xD2qD2lmlXvBINzm/xR6dFWJzFo+PEdCXCcrg/qDW4Jiql6iY/KX8ZnX5jxl2H6i7kFedg/KWeMR8gRiRhjj82iWot6JfH9JPjQvJy6g7OgbR8Fh3Hd3FIpjg4+qyevS8MpJzXtkxjXc7YzzyhmWd2oD+7Tr34eXQ0tW+yQKLIvz3lrqwqUeAW5kIdhAck0FcAQbdTed5O6pVvid89szMdpNeH7y1XWJsVZosWPirxJsTlYOVS/oY+GuxlgR4Qft08sGaucXFOoFfT4ppWxFLN6TJuSUfa1UV6VkToKeQAYPvpmfJaOnfCdy3lrh1qQZw1/cWgeqPzchTAdCGwQFCBN/i3mLZsB6N40uzvotSvgEzBJ4fm9/OdbuMCYJ27ng1/VA0sdtpDyeiUzauuf59Ir1USFTIv3ekuxdxLLrySflo1sqD+V3sciz1CmLoOH2taXjp9VF+HL2vNFFWB4yDVzXb3JJjBeKDbDFNcZJ5cYphTa+oA7qBmQds1twp04pg/B5fi8QSfngq0IcNmmMd7+ELyucOvD6qAgXNfeaTIrwgib1WCbFqN7I32tpG8OhSWj4ZLeIIAhnae1g3IgOMHlFnDRxCfxLPkBpNBQAYwg40yh0/zDo5TmVtshkwDI8QwwTaNXgBPEHVmG2WMI9Ob3TEDx7n6f8LdROjHBL92GjTc2n8UL566lQKZdC8qTV0koUthfXfSRA1u81E+rINLjfdCEf1AKgHv/ZPber1h/5erndhUorhqEs+6nKYyojYXjqg/eLxD77v5Ju/X7oZCChIkeU/LlVarE5eeqb4JS4JTSToltFSeO//0LlULDsGizfSt1TTxzyKPpuHTi/t5iUgi6LM2FF2tGaJKy9D1ufl5Qa+R7Z2xac0hE7P3SoNz2czvemKBROWm06y+RuwH1iBwTr1/DLdxqgnoYJIwgC4ggr0CVQ9+e58Kq8mTj/aRJyziHdIgzUTSKICvVDqkarxsYBTxoXBWhgpPuwwVNayqg9AK4eHvhKWxLWXH2Vo X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 0a26aff8-3467-468b-72e0-08d8bbf463f8 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:39.7895 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: w/W/3D8bkDM5kvgRS41XLA5xFKfwGp/BCB+ZgHcf7RI8VyGh6I7Tnj/VZ/bYFBiiXNmIvCq5pwDni111rU20Fw== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This allows to remove explicit creation and destruction of those attrs and by this avoids warnings on device finilizing post physical device extraction. Signed-off-by: Andrey Grodzovsky Acked-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 17 +++++++++-------- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 +++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 25 ++++++++++--------------- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 14 +++++--------- 4 files changed, 37 insertions(+), 32 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c index 86add0f..0346e12 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c @@ -1953,6 +1953,15 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev, static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version, NULL); +static struct attribute *amdgpu_vbios_version_attrs[] = { + &dev_attr_vbios_version.attr, + NULL +}; + +const struct attribute_group amdgpu_vbios_version_attr_group = { + .attrs = amdgpu_vbios_version_attrs +}; + /** * amdgpu_atombios_fini - free the driver info and callbacks for atombios * @@ -1972,7 +1981,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev) adev->mode_info.atom_context = NULL; kfree(adev->mode_info.atom_card_info); adev->mode_info.atom_card_info = NULL; - device_remove_file(adev->dev, &dev_attr_vbios_version); } /** @@ -1989,7 +1997,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) { struct card_info *atom_card_info = kzalloc(sizeof(struct card_info), GFP_KERNEL); - int ret; if (!atom_card_info) return -ENOMEM; @@ -2027,12 +2034,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) amdgpu_atombios_allocate_fb_scratch(adev); } - ret = device_create_file(adev->dev, &dev_attr_vbios_version); - if (ret) { - DRM_ERROR("Failed to create device file for VBIOS version\n"); - return ret; - } - return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 9c0cd00..8fddd74 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1587,6 +1587,18 @@ static struct pci_error_handlers amdgpu_pci_err_handler = { .resume = amdgpu_pci_resume, }; +extern const struct attribute_group amdgpu_vram_mgr_attr_group; +extern const struct attribute_group amdgpu_gtt_mgr_attr_group; +extern const struct attribute_group amdgpu_vbios_version_attr_group; + +static const struct attribute_group *amdgpu_sysfs_groups[] = { + &amdgpu_vram_mgr_attr_group, + &amdgpu_gtt_mgr_attr_group, + &amdgpu_vbios_version_attr_group, + NULL, +}; + + static struct pci_driver amdgpu_kms_pci_driver = { .name = DRIVER_NAME, .id_table = pciidlist, @@ -1595,6 +1607,7 @@ static struct pci_driver amdgpu_kms_pci_driver = { .shutdown = amdgpu_pci_shutdown, .driver.pm = &amdgpu_pm_ops, .err_handler = &amdgpu_pci_err_handler, + .driver.dev_groups = amdgpu_sysfs_groups, }; static int __init amdgpu_init(void) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 8980329..3b7150e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -77,6 +77,16 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO, amdgpu_mem_info_gtt_used_show, NULL); +static struct attribute *amdgpu_gtt_mgr_attributes[] = { + &dev_attr_mem_info_gtt_total.attr, + &dev_attr_mem_info_gtt_used.attr, + NULL +}; + +const struct attribute_group amdgpu_gtt_mgr_attr_group = { + .attrs = amdgpu_gtt_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_gtt_mgr_func; /** * amdgpu_gtt_mgr_init - init GTT manager and DRM MM @@ -91,7 +101,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) struct amdgpu_gtt_mgr *mgr = &adev->mman.gtt_mgr; struct ttm_resource_manager *man = &mgr->manager; uint64_t start, size; - int ret; man->use_tt = true; man->func = &amdgpu_gtt_mgr_func; @@ -104,17 +113,6 @@ int amdgpu_gtt_mgr_init(struct amdgpu_device *adev, uint64_t gtt_size) spin_lock_init(&mgr->lock); atomic64_set(&mgr->available, gtt_size >> PAGE_SHIFT); - ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_total); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_total\n"); - return ret; - } - ret = device_create_file(adev->dev, &dev_attr_mem_info_gtt_used); - if (ret) { - DRM_ERROR("Failed to create device file mem_info_gtt_used\n"); - return ret; - } - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -144,9 +142,6 @@ void amdgpu_gtt_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_TT, NULL); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index d2de2a7..9158d11 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -154,7 +154,7 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO, static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO, amdgpu_mem_info_vram_vendor, NULL); -static const struct attribute *amdgpu_vram_mgr_attributes[] = { +static struct attribute *amdgpu_vram_mgr_attributes[] = { &dev_attr_mem_info_vram_total.attr, &dev_attr_mem_info_vis_vram_total.attr, &dev_attr_mem_info_vram_used.attr, @@ -163,6 +163,10 @@ static const struct attribute *amdgpu_vram_mgr_attributes[] = { NULL }; +const struct attribute_group amdgpu_vram_mgr_attr_group = { + .attrs = amdgpu_vram_mgr_attributes +}; + static const struct ttm_resource_manager_func amdgpu_vram_mgr_func; /** @@ -176,7 +180,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) { struct amdgpu_vram_mgr *mgr = &adev->mman.vram_mgr; struct ttm_resource_manager *man = &mgr->manager; - int ret; ttm_resource_manager_init(man, adev->gmc.real_vram_size >> PAGE_SHIFT); @@ -187,11 +190,6 @@ int amdgpu_vram_mgr_init(struct amdgpu_device *adev) INIT_LIST_HEAD(&mgr->reservations_pending); INIT_LIST_HEAD(&mgr->reserved_pages); - /* Add the two VRAM-related sysfs files */ - ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - if (ret) - DRM_ERROR("Failed to register sysfs\n"); - ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, &mgr->manager); ttm_resource_manager_set_used(man, true); return 0; @@ -229,8 +227,6 @@ void amdgpu_vram_mgr_fini(struct amdgpu_device *adev) drm_mm_takedown(&mgr->mm); spin_unlock(&mgr->lock); - sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); - ttm_resource_manager_cleanup(man); ttm_set_driver_manager(&adev->mman.bdev, TTM_PL_VRAM, NULL); } From patchwork Mon Jan 18 21:01:20 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028263 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 2F42DC433E6 for ; Mon, 18 Jan 2021 21:02:56 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EFA15207C4 for ; Mon, 18 Jan 2021 21:02:55 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EFA15207C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id DAB8B6E7D0; Mon, 18 Jan 2021 21:02:54 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2070.outbound.protection.outlook.com [40.107.220.70]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4C2D56E5BE; Mon, 18 Jan 2021 21:02:43 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VGbkITEHtVq5AEFyS40h1ITecLd+76QvnISeg00sYmq5qEoDsPOH1scYLjdr/rf3OsuP0c/nCy3U5XXmblORphvlQfEaY5UvnpgnAvQEsU9YXKu4dns1m3f9u5efadtSs1VqTYdKJ1VkzLpiM0GaOqBsi4K9WpbKjRXCoWDh95UD8RYdvxYGNqC4xhe6uTKuxtKUnN1K9W7J0M/zDVFl/ngngkl6G5FCjJ2kmd3/XO4sBac4LD/yCwBciXexx6kZIvSALNtTsf+Zc7vU8L9tKj30J47NTp4MqB2PKJiAtC7Lf15sX/XuoIiYxGzErl4iBUi2sdE0a1myTbV4okg7Nw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=keU5rd9tfDfbyAQ+nfaxp0zWyerq6zZzsCruxbb7vKc=; b=XENJppeRFkCQqNk1Nx4AvhTpEMwznCPuCZJKA897yKgqJWVqVDSpZJmXL240Ka3FeVatpjbahmoGj71iWhaTxzsXFoMA8vI7hvQhnM9VTBFfHbETsNcN2+zputRF9IxwOBvL9LKx/meNE+1ziag9k4SEm7MBAdpWpUCl4mU4Apr7TDIyT8RIZu0WN+0OUzfmybk8DBx1eiNsIg2z1I4r2ljj58EOe9CP1V/lFgRM7tq/XA8/Ckmqyk9rI5oMn+VhBV9QjJaUsQvrAcruNOH3gdBXlEnC9BeAHPNLuLjxsjoxJPtDVzCKvl98ABv9CbMO3RfWQXF6o6CgvhSFlV3/9Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=keU5rd9tfDfbyAQ+nfaxp0zWyerq6zZzsCruxbb7vKc=; b=rrASj9+oNQE1YNl5O2DbkcprH20h0QdKSZDqrA7IojyIMt0TD5fbeXkuxqhHhaygxHnXLVrr9YdI7x5SOFMZNMjKsCsJbaN1tZurW0Hf6OQtYlTS4xS3rJOiO18XkKs+3laY5T+6pKYgg59AZQBwCl1SG5BeNZG2dGQg7kquOfk= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:41 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:41 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 11/14] drm/amdgpu: Guard against write accesses after device removal Date: Mon, 18 Jan 2021 16:01:20 -0500 Message-Id: <1611003683-3534-12-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:40 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 39e7b75f-7e56-44ef-79f5-08d8bbf4650c X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: SJNhAeQ7pEm50pyfKGDJRDJboHb+BFdQ4QjRgVMYKRxwbuZV5lSBbUaHiqZWbJfEuTqPzz6vqgadZXANXAuJ5cRIWhnXwJtcpzoVvgdJOY4rsN2t2WXC+F0CojinOdFajdo5CWRJ0DmGzpIxzKVgsCToV4NXIXXmfoSkRxhBfQFQ/p/+rDaQ9rhJrGAWQ5jSYT2CKbxmRuubCIUKh1wATRJTcwy8YkiFJ00m71piUZCGJfeFP2rNtDkwU0GBSF81g+QxG8FBEdNMNMk0y+o2+2/MvfwP1sce9eqjvx07JQoiCEWSlCGUk014OVDpEGexgqQJuFpFe43Jh2O/UmEJXl9t+huNCofXbnDhYzxkZHnlOX0+P0Z2M97IeCZ62dbLf/4xZBCCOs2EBE7CRssDYg== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(30864003)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: jMXMIsXjH+j2TZ7knWI8SRPKde7lDsbmuP9rnmzilBI1pUmchNnOGMTUM8eeqCRAi2I7lr7uHhLHFx0+jJ5cyBmKcE9w/8D/nXXplmwbQUZ0PD11bYQ15tV2R3mG0e8vYzaTxBm0LVKWnNnIxM94yRmB5LhX7xM3OjtLjsu5THe4ulIWJAsyiENY9LQG2FfBGhq2mfTUhkuFAQ9pL7Dqk0G7JkNND8CDjexlrQxwsIw7i53kYTvW/Yln69O4UDhdJ/NoIvO2N9fPdbBmHD09H0EMoqr+/UQ2gV9dwJtJmdxXTZZMSGx70IwHWu4MEelMP0JlCKBGVvuh/jog3Tgophr2Y+5GYFFO5pK35YceHf0nZZJGqBF8VodIsgyEeRcrq6UXHhMJRQbt2B488F2cELAnTJpapXipy9JZwn48bzzxc026b1PvSYdUDfwd2mqPRtKzjNr6JLiWVj5VwzxdLya0VYAlzGAK8DlxbyCgl+jzsa+gBFqt67wJRJaLDaIbhJ5qwpoo1EXILrEQ4A7ojhlJVE9MvNBGr7Cys6AT7QLgmR2eelhI0yi/CfbwXjRu9QdDZHWR5kDvhNGwDnpFCZEAG5rqNavOFVcEnsXhRsAZl0pRscut+H7Ofx73RyLW1AEqiQRhlWuymH918FwlqGBRtb87pPEVYcqP+EprY4PAlcoLcBQBekKP0gZQUzafruB7GiXKPABthw3voQwoWK6tS0jT8nkqr4iSbUADQ7S4coLKeWOUjsZezsHTGv7EEENP19AkRi8HW/JwZy3Y6pZEyDjpAxGA2H699jbptgKKdRt46G4rhHwv9cZpkMqj9auRin4uuN1kQf71ZfG9CJ5pxGOg2c2u8AedHSZFlxCkj96Vq4lW14h8l4bN2n/fjDy1eb2Wmb3ep4WEL94I7IowV3WiNPJ+xg0M+zZHpwkQkl14KFdePFuVt9uMz4LymBn4SXOuuVQHNl3QID+tA5Bn0AHg5Y1rA56XYK8yll7eMRxFEEaCIpWQuuck068/IXNWK0SpxiuZlKRRpPHbjObTehwAR9h30olFlpJKA3WQn3HEMcfP4EebeMYEFG3LwJUWaYTP0gbMtbLdQGtXXOL8cyRep2iDqa25VtkE1NubGYLMmfc/rEAa90HsDdPu X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 39e7b75f-7e56-44ef-79f5-08d8bbf4650c X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:41.6564 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: rvPy2wFGC5DyFAfO12+0in2NBLKkbeHONH9HFJtvqFPjf27Bf1j2yZJ8KqyI1GfvmEVRdFqDIR8QV7Q9+OlYYA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This should prevent writing to memory or IO ranges possibly already allocated for other uses after our device is removed. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 57 ++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 9 ++++ drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 53 +++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h | 3 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 70 ++++++++++++++++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 49 ++------------------- drivers/gpu/drm/amd/amdgpu/psp_v11_0.c | 16 ++----- drivers/gpu/drm/amd/amdgpu/psp_v12_0.c | 8 +--- drivers/gpu/drm/amd/amdgpu/psp_v3_1.c | 8 +--- 9 files changed, 184 insertions(+), 89 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index e99f4f1..0a9d73c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -72,6 +72,8 @@ #include +#include + MODULE_FIRMWARE("amdgpu/vega10_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/vega12_gpu_info.bin"); MODULE_FIRMWARE("amdgpu/raven_gpu_info.bin"); @@ -404,13 +406,21 @@ uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset) */ void amdgpu_mm_wreg8(struct amdgpu_device *adev, uint32_t offset, uint8_t value) { + int idx; + if (adev->in_pci_err_recovery) return; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return; + if (offset < adev->rmmio_size) writeb(value, adev->rmmio + offset); else BUG(); + + drm_dev_exit(idx); } /** @@ -427,9 +437,14 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v, uint32_t acc_flags) { + int idx; + if (adev->in_pci_err_recovery) return; + if (!drm_dev_enter(&adev->ddev, &idx)) + return; + if ((reg * 4) < adev->rmmio_size) { if (!(acc_flags & AMDGPU_REGS_NO_KIQ) && amdgpu_sriov_runtime(adev) && @@ -444,6 +459,8 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, } trace_amdgpu_device_wreg(adev->pdev->device, reg, v); + + drm_dev_exit(idx); } /* @@ -454,9 +471,14 @@ void amdgpu_device_wreg(struct amdgpu_device *adev, void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, uint32_t reg, uint32_t v) { + int idx; + if (adev->in_pci_err_recovery) return; + if (!drm_dev_enter(&adev->ddev, &idx)) + return; + if (amdgpu_sriov_fullaccess(adev) && adev->gfx.rlc.funcs && adev->gfx.rlc.funcs->is_rlcg_access_range) { @@ -465,6 +487,8 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev, } else { writel(v, ((void __iomem *)adev->rmmio) + (reg * 4)); } + + drm_dev_exit(idx); } /** @@ -499,15 +523,22 @@ u32 amdgpu_io_rreg(struct amdgpu_device *adev, u32 reg) */ void amdgpu_io_wreg(struct amdgpu_device *adev, u32 reg, u32 v) { + int idx; + if (adev->in_pci_err_recovery) return; + if (!drm_dev_enter(&adev->ddev, &idx)) + return; + if ((reg * 4) < adev->rio_mem_size) iowrite32(v, adev->rio_mem + (reg * 4)); else { iowrite32((reg * 4), adev->rio_mem + (mmMM_INDEX * 4)); iowrite32(v, adev->rio_mem + (mmMM_DATA * 4)); } + + drm_dev_exit(idx); } /** @@ -544,14 +575,21 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index) */ void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v) { + int idx; + if (adev->in_pci_err_recovery) return; + if (!drm_dev_enter(&adev->ddev, &idx)) + return; + if (index < adev->doorbell.num_doorbells) { writel(v, adev->doorbell.ptr + index); } else { DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index); } + + drm_dev_exit(idx); } /** @@ -588,14 +626,21 @@ u64 amdgpu_mm_rdoorbell64(struct amdgpu_device *adev, u32 index) */ void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v) { + int idx; + if (adev->in_pci_err_recovery) return; + if (!drm_dev_enter(&adev->ddev, &idx)) + return; + if (index < adev->doorbell.num_doorbells) { atomic64_set((atomic64_t *)(adev->doorbell.ptr + index), v); } else { DRM_ERROR("writing beyond doorbell aperture: 0x%08x!\n", index); } + + drm_dev_exit(idx); } /** @@ -682,6 +727,10 @@ void amdgpu_device_indirect_wreg(struct amdgpu_device *adev, unsigned long flags; void __iomem *pcie_index_offset; void __iomem *pcie_data_offset; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return; spin_lock_irqsave(&adev->pcie_idx_lock, flags); pcie_index_offset = (void __iomem *)adev->rmmio + pcie_index * 4; @@ -692,6 +741,8 @@ void amdgpu_device_indirect_wreg(struct amdgpu_device *adev, writel(reg_data, pcie_data_offset); readl(pcie_data_offset); spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); + + drm_dev_exit(idx); } /** @@ -711,6 +762,10 @@ void amdgpu_device_indirect_wreg64(struct amdgpu_device *adev, unsigned long flags; void __iomem *pcie_index_offset; void __iomem *pcie_data_offset; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return; spin_lock_irqsave(&adev->pcie_idx_lock, flags); pcie_index_offset = (void __iomem *)adev->rmmio + pcie_index * 4; @@ -727,6 +782,8 @@ void amdgpu_device_indirect_wreg64(struct amdgpu_device *adev, writel((u32)(reg_data >> 32), pcie_data_offset); readl(pcie_data_offset); spin_unlock_irqrestore(&adev->pcie_idx_lock, flags); + + drm_dev_exit(idx); } /** diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index fe1a39f..1beb4e6 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -31,6 +31,8 @@ #include "amdgpu_ras.h" #include "amdgpu_xgmi.h" +#include + /** * amdgpu_gmc_get_pde_for_bo - get the PDE for a BO * @@ -98,6 +100,10 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, { void __iomem *ptr = (void *)cpu_pt_addr; uint64_t value; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) + return 0; /* * The following is for PTE only. GART does not have PDEs. @@ -105,6 +111,9 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr, value = addr & 0x0000FFFFFFFFF000ULL; value |= flags; writeq(value, ptr + (gpu_page_idx * 8)); + + drm_dev_exit(idx); + return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c index 523d22d..89e2bfe 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c @@ -37,6 +37,8 @@ #include "amdgpu_ras.h" +#include + static int psp_sysfs_init(struct amdgpu_device *adev); static void psp_sysfs_fini(struct amdgpu_device *adev); @@ -248,7 +250,7 @@ psp_cmd_submit_buf(struct psp_context *psp, struct psp_gfx_cmd_resp *cmd, uint64_t fence_mc_addr) { int ret; - int index; + int index, idx; int timeout = 2000; bool ras_intr = false; bool skip_unsupport = false; @@ -256,6 +258,9 @@ psp_cmd_submit_buf(struct psp_context *psp, if (psp->adev->in_pci_err_recovery) return 0; + if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return 0; + mutex_lock(&psp->mutex); memset(psp->cmd_buf_mem, 0, PSP_CMD_BUFFER_SIZE); @@ -266,8 +271,7 @@ psp_cmd_submit_buf(struct psp_context *psp, ret = psp_ring_cmd_submit(psp, psp->cmd_buf_mc_addr, fence_mc_addr, index); if (ret) { atomic_dec(&psp->fence_value); - mutex_unlock(&psp->mutex); - return ret; + goto exit; } amdgpu_asic_invalidate_hdp(psp->adev, NULL); @@ -307,8 +311,8 @@ psp_cmd_submit_buf(struct psp_context *psp, psp->cmd_buf_mem->cmd_id, psp->cmd_buf_mem->resp.status); if (!timeout) { - mutex_unlock(&psp->mutex); - return -EINVAL; + ret = -EINVAL; + goto exit; } } @@ -316,8 +320,10 @@ psp_cmd_submit_buf(struct psp_context *psp, ucode->tmr_mc_addr_lo = psp->cmd_buf_mem->resp.fw_addr_lo; ucode->tmr_mc_addr_hi = psp->cmd_buf_mem->resp.fw_addr_hi; } - mutex_unlock(&psp->mutex); +exit: + mutex_unlock(&psp->mutex); + drm_dev_exit(idx); return ret; } @@ -354,8 +360,7 @@ static int psp_load_toc(struct psp_context *psp, if (!cmd) return -ENOMEM; /* Copy toc to psp firmware private buffer */ - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->toc_start_addr, psp->toc_bin_size); + psp_copy_fw(psp, psp->toc_start_addr, psp->toc_bin_size); psp_prep_load_toc_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->toc_bin_size); @@ -570,8 +575,7 @@ static int psp_asd_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->asd_start_addr, psp->asd_ucode_size); + psp_copy_fw(psp, psp->asd_start_addr, psp->asd_ucode_size); psp_prep_asd_load_cmd_buf(cmd, psp->fw_pri_mc_addr, psp->asd_ucode_size); @@ -726,8 +730,7 @@ static int psp_xgmi_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size); + psp_copy_fw(psp, psp->ta_xgmi_start_addr, psp->ta_xgmi_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -982,8 +985,7 @@ static int psp_ras_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_ras_start_addr, psp->ta_ras_ucode_size); + psp_copy_fw(psp, psp->ta_ras_start_addr, psp->ta_ras_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1219,8 +1221,7 @@ static int psp_hdcp_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_hdcp_start_addr, + psp_copy_fw(psp, psp->ta_hdcp_start_addr, psp->ta_hdcp_ucode_size); psp_prep_ta_load_cmd_buf(cmd, @@ -1366,8 +1367,7 @@ static int psp_dtm_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); + psp_copy_fw(psp, psp->ta_dtm_start_addr, psp->ta_dtm_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -1507,8 +1507,7 @@ static int psp_rap_load(struct psp_context *psp) if (!cmd) return -ENOMEM; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - memcpy(psp->fw_pri_buf, psp->ta_rap_start_addr, psp->ta_rap_ucode_size); + psp_copy_fw(psp, psp->ta_rap_start_addr, psp->ta_rap_ucode_size); psp_prep_ta_load_cmd_buf(cmd, psp->fw_pri_mc_addr, @@ -2778,6 +2777,20 @@ static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev, return count; } +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size) +{ + int idx; + + if (!drm_dev_enter(&psp->adev->ddev, &idx)) + return; + + memset(psp->fw_pri_buf, 0, PSP_1_MEG); + memcpy(psp->fw_pri_buf, start_addr, bin_size); + + drm_dev_exit(idx); +} + + static DEVICE_ATTR(usbc_pd_fw, S_IRUGO | S_IWUSR, psp_usbc_pd_fw_sysfs_read, psp_usbc_pd_fw_sysfs_write); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h index da250bc..ac69314 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h @@ -400,4 +400,7 @@ int psp_init_ta_microcode(struct psp_context *psp, const char *chip_name); int psp_get_fw_attestation_records_addr(struct psp_context *psp, uint64_t *output_ptr); + +void psp_copy_fw(struct psp_context *psp, uint8_t *start_addr, uint32_t bin_size); + #endif diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c index 1a612f5..d656494 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c @@ -35,6 +35,8 @@ #include "amdgpu.h" #include "atom.h" +#include + /* * Rings * Most engines on the GPU are fed via ring buffers. Ring @@ -463,3 +465,71 @@ int amdgpu_ring_test_helper(struct amdgpu_ring *ring) ring->sched.ready = !r; return r; } + +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) +{ + int idx; + int i = 0; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + while (i <= ring->buf_mask) + ring->ring[i++] = ring->funcs->nop; + + drm_dev_exit(idx); + +} + +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) +{ + int idx; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + if (ring->count_dw <= 0) + DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); + ring->ring[ring->wptr++ & ring->buf_mask] = v; + ring->wptr &= ring->ptr_mask; + ring->count_dw--; + + drm_dev_exit(idx); +} + +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, + void *src, int count_dw) +{ + unsigned occupied, chunk1, chunk2; + void *dst; + int idx; + + if (!drm_dev_enter(&ring->adev->ddev, &idx)) + return; + + if (unlikely(ring->count_dw < count_dw)) + DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); + + occupied = ring->wptr & ring->buf_mask; + dst = (void *)&ring->ring[occupied]; + chunk1 = ring->buf_mask + 1 - occupied; + chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1; + chunk2 = count_dw - chunk1; + chunk1 <<= 2; + chunk2 <<= 2; + + if (chunk1) + memcpy(dst, src, chunk1); + + if (chunk2) { + src += chunk1; + dst = (void *)ring->ring; + memcpy(dst, src, chunk2); + } + + ring->wptr += count_dw; + ring->wptr &= ring->ptr_mask; + ring->count_dw -= count_dw; + + drm_dev_exit(idx); +} diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h index accb243..f90b81f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h @@ -300,53 +300,12 @@ static inline void amdgpu_ring_set_preempt_cond_exec(struct amdgpu_ring *ring, *ring->cond_exe_cpu_addr = cond_exec; } -static inline void amdgpu_ring_clear_ring(struct amdgpu_ring *ring) -{ - int i = 0; - while (i <= ring->buf_mask) - ring->ring[i++] = ring->funcs->nop; - -} - -static inline void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v) -{ - if (ring->count_dw <= 0) - DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); - ring->ring[ring->wptr++ & ring->buf_mask] = v; - ring->wptr &= ring->ptr_mask; - ring->count_dw--; -} +void amdgpu_ring_clear_ring(struct amdgpu_ring *ring); -static inline void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, - void *src, int count_dw) -{ - unsigned occupied, chunk1, chunk2; - void *dst; - - if (unlikely(ring->count_dw < count_dw)) - DRM_ERROR("amdgpu: writing more dwords to the ring than expected!\n"); - - occupied = ring->wptr & ring->buf_mask; - dst = (void *)&ring->ring[occupied]; - chunk1 = ring->buf_mask + 1 - occupied; - chunk1 = (chunk1 >= count_dw) ? count_dw: chunk1; - chunk2 = count_dw - chunk1; - chunk1 <<= 2; - chunk2 <<= 2; +void amdgpu_ring_write(struct amdgpu_ring *ring, uint32_t v); - if (chunk1) - memcpy(dst, src, chunk1); - - if (chunk2) { - src += chunk1; - dst = (void *)ring->ring; - memcpy(dst, src, chunk2); - } - - ring->wptr += count_dw; - ring->wptr &= ring->ptr_mask; - ring->count_dw -= count_dw; -} +void amdgpu_ring_write_multiple(struct amdgpu_ring *ring, + void *src, int count_dw); int amdgpu_ring_test_helper(struct amdgpu_ring *ring); diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c index bd4248c..b3ce5be 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c @@ -269,10 +269,8 @@ static int psp_v11_0_bootloader_load_kdb(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP KDB binary to memory */ - memcpy(psp->fw_pri_buf, psp->kdb_start_addr, psp->kdb_bin_size); + psp_copy_fw(psp, psp->kdb_start_addr, psp->kdb_bin_size); /* Provide the PSP KDB to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -302,10 +300,8 @@ static int psp_v11_0_bootloader_load_spl(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP SPL binary to memory */ - memcpy(psp->fw_pri_buf, psp->spl_start_addr, psp->spl_bin_size); + psp_copy_fw(psp, psp->spl_start_addr, psp->spl_bin_size); /* Provide the PSP SPL to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -335,10 +331,8 @@ static int psp_v11_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -371,10 +365,8 @@ static int psp_v11_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c index c4828bd..618e5b6 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v12_0.c @@ -138,10 +138,8 @@ static int psp_v12_0_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -179,10 +177,8 @@ static int psp_v12_0_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c index f2e725f..d0a6cccd 100644 --- a/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c +++ b/drivers/gpu/drm/amd/amdgpu/psp_v3_1.c @@ -102,10 +102,8 @@ static int psp_v3_1_bootloader_load_sysdrv(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy PSP System Driver binary to memory */ - memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size); + psp_copy_fw(psp, psp->sys_start_addr, psp->sys_bin_size); /* Provide the sys driver to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, @@ -143,10 +141,8 @@ static int psp_v3_1_bootloader_load_sos(struct psp_context *psp) if (ret) return ret; - memset(psp->fw_pri_buf, 0, PSP_1_MEG); - /* Copy Secure OS binary to PSP memory */ - memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size); + psp_copy_fw(psp, psp->sos_start_addr, psp->sos_bin_size); /* Provide the PSP secure OS to bootloader */ WREG32_SOC15(MP0, 0, mmMP0_SMN_C2PMSG_36, From patchwork Mon Jan 18 21:01:21 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028257 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 398E6C43381 for ; Mon, 18 Jan 2021 21:02:50 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id F36AD22DD3 for ; Mon, 18 Jan 2021 21:02:49 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org F36AD22DD3 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1376C6E5CC; Mon, 18 Jan 2021 21:02:48 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2079.outbound.protection.outlook.com [40.107.220.79]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3E05F6E5C3; Mon, 18 Jan 2021 21:02:45 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=BgBAh6l1zjPuFS91l4PNKTXhW19TuVfAYs0588zwX/62iK5VOQclazoFmurTlY4y1VJUleElupb7GOixPGmXgCBF/JPs8348MDW9zRNRXo62o6fmLYym/s2WgOqqy2P8GuYR59SJDj+4rzIBTYlSModlQLH0e8ntu1G1P7XuKBrJ7J0gwFF2NKzKQDIhRx712lXjVYeXqgrjpXDioNbJeNfy4zVGve0bVYS5OP23O+QnE7gnuk30Vyz0JIRVSZCea/l2OVWJlsA5lTzK7m+WceKn5EO4GPhUQLbbNQlto0Cr+6lqzJnF0yMxEu9iNJbIhRt4o5VQKTMZv4dlwh4iag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YGLl3snTccK41uXYtQBDr+JZ8i16Ma3viKtMFJCf13A=; b=XoKCzel5PyDyEOcnBk4JT9AuPYM5vHwq2Eov9Ebay7t+qtbGHNedeZStxd1fy2r3cnDEC7etYkz1Amy0YYQ9RWq/ebeqyByZWNeyb+cPy1xyZiz6I6RLfFcThGSaPR/13AoybhanJ36wH0p27d9fdqJ8xu61sNIDMAwDj8ejTQl6IeIJtJS/9llOURUVQSvEyEK8PO+BaPKxcr46Xx/4YlYqKN47weSRzXVOkX/VuoLz6O3GJN/SlFN08gQir+ps2Oir7aEsM1LdAPnIBUOOVWKeKhhKXUcQeX2m7Iq0DuTCdMchj2NsdD/n8CFDKjCyb18FbfHxPtcxnfiL+0YaOw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=YGLl3snTccK41uXYtQBDr+JZ8i16Ma3viKtMFJCf13A=; b=UZeKQMJc0dk4XilsSuUmxo+Eu6Wbhg9lDz6e0WKmZJTLHs418J2+sLkHQKyH1J2bd/m8QIujxx0cDVQHqyO1dcDTfzqs9pOMNnUz/vOK3OAPwAYH8GF5ydl/36KTn0GTvntUlKSgBqRNragsFt1v7NyQyl9r/NzL/E1wSGk0B80= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:44 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:44 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 12/14] drm/scheduler: Job timeout handler returns status Date: Mon, 18 Jan 2021 16:01:21 -0500 Message-Id: <1611003683-3534-13-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:42 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 73d4d327-d672-4c4b-884e-08d8bbf46674 X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1284; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 3mwGnHm1Jce5dXoPgtdQhAP2EIYkYtucAYBXSixIkUZdhjM/OVkk8O2jZdr5JDiHF+YqWj4yLeRxq+AH35hE4vGiw39kyGpKlZX647kAcbfsVQe8Qrss/312l4L+1n6XW7tvr2zKvuNa+/CrFdTbePkVw8GgWErGje+UzoMj1uN/CiohDDrprNDUt2QDOAGtnhi4ikSduXhVNj3FvARIIgCyYJIjkWeGJxWtwOvU7yd7x+j9oJV6zTjK19Trz5BZf77s7DluOt4Ib2S9+r6Hb4nLmlMc8RMth7U3To3a4PXbqh7DgLfRxQa+nS0APEEDVYuEZjPRs0nLBSPoDO3l7X9PplsuGL8aO1cxHVbFIGE/2Fo2XcZyIWMXLVpvaGyLYdEdPjyLvxsS2MMmYEvQMw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(66574015)(30864003)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(54906003)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: =?utf-8?q?8i1Xqvu1zvH/wc+qXiiG0P/3dpKoC7?= =?utf-8?q?DQsBBXRbSKsgbsGAWHl19/kIff9Qd9nKtpVoPmzxc4zuwgHCdopF1CeuM3f40yaxd?= =?utf-8?q?YMzPUbOcnkecQT0fVd8rnryqB1p/mDoxVw0ZXsNOitaUtyAx4WAbeAOjv0x/B6eoP?= =?utf-8?q?jXlopXUM6TaQvxD52YIXW+uv3x+OXITUGn7KDuxP+VzyxM+SWGhBeznS97Sg62BjH?= =?utf-8?q?o9jLECUq6UIzzmopoAQQVEoB7+ICf21dQ8GrjsWZVXx1SP1eyYuQqYVpY7SnTgHno?= =?utf-8?q?PxCjVZFhdWXjtcqiS7hXbP8XlPHxiVIRtZHl39S5gZmrhmVwPqVAh+Na2TYw7tOzl?= =?utf-8?q?0uYivHzW6wL49WHvx9FlLNlwj8waxA9WP9SD5QPDOpn4oaTTV3WVdiWtVo7DNF39t?= =?utf-8?q?HawBBz9EBS5fyUXWKvrn0GZ6KUNve9oaBMW2rZhTUJ8MmkJA78AR2flfpLBPmitjI?= =?utf-8?q?/L8eiNF507lhx9Yei+FjQEp4YdnqSYHO4uY7BviyC60Gu4FtpIfxfSGOK4mv/EIMA?= =?utf-8?q?hV/a4eJsU13wOHOSx/B1lKVO/HWCmE76CgjKR3cnt7GG7a0KPuNfIOFGf2dHzzZTb?= =?utf-8?q?QR9ImZvAz2sfsea2huc9Z0yXe0Lw4XQkLGfmvKSWibOvgHX3TH+cYFvYKKNri7RCl?= =?utf-8?q?VoIXHwRALQfxck7egwlIZaAvYuhC6I+COFTPwF+ibwrTd5dKtB2MisGfKT7YR2J2O?= =?utf-8?q?mmUPqcVbfk0Uw3NwDa+Ufl57/PB4abDeDxWV5mDVPlcQuc1P+DW7rUCEQpxG61X2w?= =?utf-8?q?lb9WNxvrapSG94uhQx8ScNqIUFLG05c6WvvZBv69AJ9iLQgMc1cVFq2Z3wfUfliHH?= =?utf-8?q?ZBA+G2iE+Djvbs2LaGEtviVcsm++EO5xP2aVSPiCVNoUeaMQf80b36QuPk+AH9ot0?= =?utf-8?q?E4oMYqij5Lu+LuYlZ+K/MNgjvyCGV3AcZJVUJs7UJL0boUbP7Z9/gybvH8cCgi5rl?= =?utf-8?q?OPQo8nL1BxU80CY1tUOhkkvW+v5/4WGDU0Vx1k1FKRGBmQmnX+WcVOSLyjjX8wJCa?= =?utf-8?q?WmjeV4TMuljcorhBsvXakfv4nSujk+tIPsOu2sNivQyxHJehUYk+D14qv3nIIx4cO?= =?utf-8?q?MUq1H/0BH4gO9jakY?= X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 73d4d327-d672-4c4b-884e-08d8bbf46674 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:43.9871 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: 9y2P1qJLp3ie69WeasdJCWUYJxuGZgsOcvPlEXXqt6wdB5+m0kdZQF9kzu1cF2xYA+gWWTkaFAUrAA5RfP/J+w== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Tomeu Vizoso , gregkh@linuxfoundation.org, Steven Price , Luben Tuikov , Alyssa Rosenzweig , Russell King , Alexander.Deucher@amd.com, =?utf-8?q?Christian_K=C3=B6nig?= Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" From: Luben Tuikov This patch does not change current behaviour. The driver's job timeout handler now returns status indicating back to the DRM layer whether the task (job) was successfully aborted or whether more time should be given to the task to complete. Default behaviour as of this patch, is preserved, except in obvious-by-comment case in the Panfrost driver, as documented below. All drivers which make use of the drm_sched_backend_ops' .timedout_job() callback have been accordingly renamed and return the would've-been default value of DRM_TASK_STATUS_ALIVE to restart the task's timeout timer--this is the old behaviour, and is preserved by this patch. In the case of the Panfrost driver, its timedout callback correctly first checks if the job had completed in due time and if so, it now returns DRM_TASK_STATUS_COMPLETE to notify the DRM layer that the task can be moved to the done list, to be freed later. In the other two subsequent checks, the value of DRM_TASK_STATUS_ALIVE is returned, as per the default behaviour. A more involved driver's solutions can be had in subequent patches. v2: Use enum as the status of a driver's job timeout callback method. v4: (By Andrey Grodzovsky) Replace DRM_TASK_STATUS_COMPLETE with DRM_TASK_STATUS_ENODEV to enable a hint to the schduler for when NOT to rearm the timeout timer. Cc: Alexander Deucher Cc: Andrey Grodzovsky Cc: Christian König Cc: Daniel Vetter Cc: Lucas Stach Cc: Russell King Cc: Christian Gmeiner Cc: Qiang Yu Cc: Rob Herring Cc: Tomeu Vizoso Cc: Steven Price Cc: Alyssa Rosenzweig Cc: Eric Anholt Reported-by: kernel test robot Signed-off-by: Luben Tuikov Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 6 ++++-- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 10 +++++++++- drivers/gpu/drm/lima/lima_sched.c | 4 +++- drivers/gpu/drm/panfrost/panfrost_job.c | 9 ++++++--- drivers/gpu/drm/scheduler/sched_main.c | 4 +--- drivers/gpu/drm/v3d/v3d_sched.c | 32 +++++++++++++++++--------------- include/drm/gpu_scheduler.h | 17 ++++++++++++++--- 7 files changed, 54 insertions(+), 28 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index ff48101..a111326 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -28,7 +28,7 @@ #include "amdgpu.h" #include "amdgpu_trace.h" -static void amdgpu_job_timedout(struct drm_sched_job *s_job) +static enum drm_task_status amdgpu_job_timedout(struct drm_sched_job *s_job) { struct amdgpu_ring *ring = to_amdgpu_ring(s_job->sched); struct amdgpu_job *job = to_amdgpu_job(s_job); @@ -41,7 +41,7 @@ static void amdgpu_job_timedout(struct drm_sched_job *s_job) amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) { DRM_ERROR("ring %s timeout, but soft recovered\n", s_job->sched->name); - return; + return DRM_TASK_STATUS_ALIVE; } amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti); @@ -53,10 +53,12 @@ static void amdgpu_job_timedout(struct drm_sched_job *s_job) if (amdgpu_device_should_recover_gpu(ring->adev)) { amdgpu_device_gpu_recover(ring->adev, job); + return DRM_TASK_STATUS_ALIVE; } else { drm_sched_suspend_timeout(&ring->sched); if (amdgpu_sriov_vf(adev)) adev->virt.tdr_debug = true; + return DRM_TASK_STATUS_ALIVE; } } diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c index cd46c88..c495169 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c @@ -82,7 +82,8 @@ static struct dma_fence *etnaviv_sched_run_job(struct drm_sched_job *sched_job) return fence; } -static void etnaviv_sched_timedout_job(struct drm_sched_job *sched_job) +static enum drm_task_status etnaviv_sched_timedout_job(struct drm_sched_job + *sched_job) { struct etnaviv_gem_submit *submit = to_etnaviv_submit(sched_job); struct etnaviv_gpu *gpu = submit->gpu; @@ -120,9 +121,16 @@ static void etnaviv_sched_timedout_job(struct drm_sched_job *sched_job) drm_sched_resubmit_jobs(&gpu->sched); + /* Tell the DRM scheduler that this task needs + * more time. + */ + drm_sched_start(&gpu->sched, true); + return DRM_TASK_STATUS_ALIVE; + out_no_timeout: /* restart scheduler after GPU is usable again */ drm_sched_start(&gpu->sched, true); + return DRM_TASK_STATUS_ALIVE; } static void etnaviv_sched_free_job(struct drm_sched_job *sched_job) diff --git a/drivers/gpu/drm/lima/lima_sched.c b/drivers/gpu/drm/lima/lima_sched.c index 63b4c56..66d9236 100644 --- a/drivers/gpu/drm/lima/lima_sched.c +++ b/drivers/gpu/drm/lima/lima_sched.c @@ -415,7 +415,7 @@ static void lima_sched_build_error_task_list(struct lima_sched_task *task) mutex_unlock(&dev->error_task_list_lock); } -static void lima_sched_timedout_job(struct drm_sched_job *job) +static enum drm_task_status lima_sched_timedout_job(struct drm_sched_job *job) { struct lima_sched_pipe *pipe = to_lima_pipe(job->sched); struct lima_sched_task *task = to_lima_task(job); @@ -449,6 +449,8 @@ static void lima_sched_timedout_job(struct drm_sched_job *job) drm_sched_resubmit_jobs(&pipe->base); drm_sched_start(&pipe->base, true); + + return DRM_TASK_STATUS_ALIVE; } static void lima_sched_free_job(struct drm_sched_job *job) diff --git a/drivers/gpu/drm/panfrost/panfrost_job.c b/drivers/gpu/drm/panfrost/panfrost_job.c index 04e6f6f..10d41ac 100644 --- a/drivers/gpu/drm/panfrost/panfrost_job.c +++ b/drivers/gpu/drm/panfrost/panfrost_job.c @@ -432,7 +432,8 @@ static void panfrost_scheduler_start(struct panfrost_queue_state *queue) mutex_unlock(&queue->lock); } -static void panfrost_job_timedout(struct drm_sched_job *sched_job) +static enum drm_task_status panfrost_job_timedout(struct drm_sched_job + *sched_job) { struct panfrost_job *job = to_panfrost_job(sched_job); struct panfrost_device *pfdev = job->pfdev; @@ -443,7 +444,7 @@ static void panfrost_job_timedout(struct drm_sched_job *sched_job) * spurious. Bail out. */ if (dma_fence_is_signaled(job->done_fence)) - return; + return DRM_TASK_STATUS_ALIVE; dev_err(pfdev->dev, "gpu sched timeout, js=%d, config=0x%x, status=0x%x, head=0x%x, tail=0x%x, sched_job=%p", js, @@ -455,11 +456,13 @@ static void panfrost_job_timedout(struct drm_sched_job *sched_job) /* Scheduler is already stopped, nothing to do. */ if (!panfrost_scheduler_stop(&pfdev->js->queue[js], sched_job)) - return; + return DRM_TASK_STATUS_ALIVE; /* Schedule a reset if there's no reset in progress. */ if (!atomic_xchg(&pfdev->reset.pending, 1)) schedule_work(&pfdev->reset.work); + + return DRM_TASK_STATUS_ALIVE; } static const struct drm_sched_backend_ops panfrost_sched_ops = { diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 92637b7..73fccc5 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -527,7 +527,7 @@ void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery) EXPORT_SYMBOL(drm_sched_start); /** - * drm_sched_resubmit_jobs - helper to relunch job from pending ring list + * drm_sched_resubmit_jobs - helper to relaunch jobs from the pending list * * @sched: scheduler instance * @@ -561,8 +561,6 @@ void drm_sched_resubmit_jobs(struct drm_gpu_scheduler *sched) } else { s_job->s_fence->parent = fence; } - - } } EXPORT_SYMBOL(drm_sched_resubmit_jobs); diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c index 452682e..3740665e 100644 --- a/drivers/gpu/drm/v3d/v3d_sched.c +++ b/drivers/gpu/drm/v3d/v3d_sched.c @@ -259,7 +259,7 @@ v3d_cache_clean_job_run(struct drm_sched_job *sched_job) return NULL; } -static void +static enum drm_task_status v3d_gpu_reset_for_timeout(struct v3d_dev *v3d, struct drm_sched_job *sched_job) { enum v3d_queue q; @@ -285,6 +285,8 @@ v3d_gpu_reset_for_timeout(struct v3d_dev *v3d, struct drm_sched_job *sched_job) } mutex_unlock(&v3d->reset_lock); + + return DRM_TASK_STATUS_ALIVE; } /* If the current address or return address have changed, then the GPU @@ -292,7 +294,7 @@ v3d_gpu_reset_for_timeout(struct v3d_dev *v3d, struct drm_sched_job *sched_job) * could fail if the GPU got in an infinite loop in the CL, but that * is pretty unlikely outside of an i-g-t testcase. */ -static void +static enum drm_task_status v3d_cl_job_timedout(struct drm_sched_job *sched_job, enum v3d_queue q, u32 *timedout_ctca, u32 *timedout_ctra) { @@ -304,39 +306,39 @@ v3d_cl_job_timedout(struct drm_sched_job *sched_job, enum v3d_queue q, if (*timedout_ctca != ctca || *timedout_ctra != ctra) { *timedout_ctca = ctca; *timedout_ctra = ctra; - return; + return DRM_TASK_STATUS_ALIVE; } - v3d_gpu_reset_for_timeout(v3d, sched_job); + return v3d_gpu_reset_for_timeout(v3d, sched_job); } -static void +static enum drm_task_status v3d_bin_job_timedout(struct drm_sched_job *sched_job) { struct v3d_bin_job *job = to_bin_job(sched_job); - v3d_cl_job_timedout(sched_job, V3D_BIN, - &job->timedout_ctca, &job->timedout_ctra); + return v3d_cl_job_timedout(sched_job, V3D_BIN, + &job->timedout_ctca, &job->timedout_ctra); } -static void +static enum drm_task_status v3d_render_job_timedout(struct drm_sched_job *sched_job) { struct v3d_render_job *job = to_render_job(sched_job); - v3d_cl_job_timedout(sched_job, V3D_RENDER, - &job->timedout_ctca, &job->timedout_ctra); + return v3d_cl_job_timedout(sched_job, V3D_RENDER, + &job->timedout_ctca, &job->timedout_ctra); } -static void +static enum drm_task_status v3d_generic_job_timedout(struct drm_sched_job *sched_job) { struct v3d_job *job = to_v3d_job(sched_job); - v3d_gpu_reset_for_timeout(job->v3d, sched_job); + return v3d_gpu_reset_for_timeout(job->v3d, sched_job); } -static void +static enum drm_task_status v3d_csd_job_timedout(struct drm_sched_job *sched_job) { struct v3d_csd_job *job = to_csd_job(sched_job); @@ -348,10 +350,10 @@ v3d_csd_job_timedout(struct drm_sched_job *sched_job) */ if (job->timedout_batches != batches) { job->timedout_batches = batches; - return; + return DRM_TASK_STATUS_ALIVE; } - v3d_gpu_reset_for_timeout(v3d, sched_job); + return v3d_gpu_reset_for_timeout(v3d, sched_job); } static const struct drm_sched_backend_ops v3d_bin_sched_ops = { diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index 975e8a6..3ba36bc 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -206,6 +206,11 @@ static inline bool drm_sched_invalidate_job(struct drm_sched_job *s_job, return s_job && atomic_inc_return(&s_job->karma) > threshold; } +enum drm_task_status { + DRM_TASK_STATUS_ENODEV, + DRM_TASK_STATUS_ALIVE +}; + /** * struct drm_sched_backend_ops * @@ -230,10 +235,16 @@ struct drm_sched_backend_ops { struct dma_fence *(*run_job)(struct drm_sched_job *sched_job); /** - * @timedout_job: Called when a job has taken too long to execute, - * to trigger GPU recovery. + * @timedout_job: Called when a job has taken too long to execute, + * to trigger GPU recovery. + * + * Return DRM_TASK_STATUS_ALIVE, if the task (job) is healthy + * and executing in the hardware, i.e. it needs more time. + * + * Return DRM_TASK_STATUS_ENODEV, if the task (job) has + * been aborted. */ - void (*timedout_job)(struct drm_sched_job *sched_job); + enum drm_task_status (*timedout_job)(struct drm_sched_job *sched_job); /** * @free_job: Called once the job's finished fence has been signaled From patchwork Mon Jan 18 21:01:22 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028259 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 38A40C433E0 for ; Mon, 18 Jan 2021 21:02:52 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 01D49207C4 for ; Mon, 18 Jan 2021 21:02:51 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 01D49207C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 5E1796E5C3; Mon, 18 Jan 2021 21:02:50 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2066.outbound.protection.outlook.com [40.107.220.66]) by gabe.freedesktop.org (Postfix) with ESMTPS id 9E3376E5C6; Mon, 18 Jan 2021 21:02:47 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fztKaoj1xf6N+0OJx95luO7saIKkzT9MhLPYuuXszRhXpFsK48AG9xefm7ueGxCt66FR5yIFLzwlsptqb+d5pdz7zCl5pHQesGrTZLNz1w3LhxQ9u4voh++Hl6dDBO8KS+jcLaWQYBtUT7UofhyDAZvx/7LEM6Vnlhgp39+i1DxUtOVf6hVsOsyzMhBhkZd7jud9pNoQZLkR9IeF+JTg0cUZMdjVpzQsG2mchQd6v5FO6AuD8TOx8y2RlABebKhaJ1mp/xji6MTjDGFpPlHMirQI926WEOJX+y+zhrpeNd9ZnlhGb70q5UITVs2FIQ6g47LzDfe2bBQoGmXI6g30fw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=acABx/MxeJUqIxU2HQF5Pn9a6gec1/oPc60taOYLvqQ=; b=Ovj/vaXtux+yZolfTZ1AtydDgNbCFLGH1h5Z5zNrLG9RmbCK1CWMVfSYMhckmwYZQJ/v+WvsuXGSxmVZw8eZDS5N28J4FUb4FLQw5kv12qeqPRMV+h6xTmFiYPiZNFtY/qZztgH4b4UijoDEXvCZ+cnIe1eJj3fpuSXEHUaFEcv34sPpPWvQp5faFAlSKzVlDidYKMXTnC02g4cdfpo/KyorgsHDvBw7ubi9AG3/u9L4WDoWglZggDlRj/bbIdfh/iYE86UmQ+DlNlDFVG2RIKQ3inirkLdyio5oN87vUlT2AfQ0UxYSn5W8WECHIzcLcgk76rCelLhGbYjimhw+PA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=acABx/MxeJUqIxU2HQF5Pn9a6gec1/oPc60taOYLvqQ=; b=omTDcYXT5u2SP8mA192TsiScmuf07VhyC5RbtlzZB3SJ8JbIzfsm3q0k9N+mEnYkUlCnmW1IXiYydAMcbrGodcMAa+K6Ek62LTBjZAJ4/j+XqDAUCWGRAgUZQ2WG62uJ9WEph2K+LEIOFT4Lxu+25BWluV4rik4SGc2JvrwWNGU= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:46 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:46 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 13/14] drm/sched: Make timeout timer rearm conditional. Date: Mon, 18 Jan 2021 16:01:22 -0500 Message-Id: <1611003683-3534-14-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:44 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 1cca2f9e-ff3a-4bc7-c2a5-08d8bbf46794 X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2733; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: zVjKUfE6Z2AbstNbzK+zouTYBtt0biNvjbl3bRXmBuQA+dA/g44sNbIq2HO5wq9RIEtfMtv8crrAq2X9apB1LK3kH6+x6IuT1LUX/I1IO3xLms05yfhK6X6Gc2ZKJ8LiDLHV/cZRlTU5R0zXczelQ6i93inLYwokkYCtSLnWSqWVMR87q3cnvfEnBDVrwinR8e6PffEykFdmUH5akSX7Ekjdgg1VsZOXZB121YTQBm702rpLtObyZtRtNxofB+EFehaSJ10KMutTWjkgNmPGTLBJHJXNE8fUuJyHplGmn45kKa/+C/8Bk8+Lq8ygZNy9FhzV9SrD3SnK2+zXS2Ytd0ojAsuifsJP5FHhHQE6bBBGADP7taijQwgt6SFMj2QiOEJpeLLSmoZwTZZ6WHq+Mw== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: wQeGaxTiUE66IvIS6HcpZKqqKXSgoJWjvkCXuQy85zVXLyN/qC+UTCLkX1vVJ20oQk1UeuAxxITp9xs5W7qpMt2PQVvVI6PIsCQOciR1LNniAAvYvsTbUYpVf+eA/nX2SM3bbW35HX8YVyEBQ27PP4RdaCMJmE4xcmOtL4FpJfdJ2fRYBB65Loncu+V4K8w4b7hVsiC4kWUih+rj29PFNpfL558h+kd0c+FbJN3PTeFklV1X65U3jtWcdA3YdyswTHlc+KLKnKwEEK241JiHgD1kVc4IEHin+1EhKRUUBS7aV4p6ED9VzJ8DW2Q12DwEfUheZbBZdV1pwhT9suyocdwSrf48c0gAR7Fov9uThM6zjyUm96nQHKel8VzqWOgutQ+tQaC+Fqmh7lckCAveRHgxOr/VMDTnHVELE0BQx+/eQ6mzCz52N0FiQ0XNGYs7VeDMPCbytH8bt2wbRa15t92VK913fyJ3fPfY0w42ivvRsuPctmA8wjMWOtEVbDrh8uyEA10Psw47iZJdkUvVXvmASImtnGeeGQtw6eO0TsczbfAAdYWezXar7DqH4b+SaYmxkhaHSsFGVbjJaa6xOWm5ZOcKaG4xHSDOH7zewhFFEJv1bF7fG4UdvEYFHBe2RJP8QH4hJGusoLYhnyIyCiLxKeWAvp6NXL7FTc6znSvxA5N4k9kiMh1cogl2zJWKA8s6owqovni9TABQ2sZoQsvQgSIzFhp/50uF4FCTW//4gBUZxc1aQb28ccPbEoiOpj+ZY1jhOpTa8zWSLZVuLrwR1RUPk81BvJFZTIwESSZK5PMQau6mylw1eGlEAB2kTF/sOIkdZ5wBcuUpXJoBaxG7ToCYk05QYibm1M4dYPnMgSNF0EQlUSGwh0l3txaSxTnLBB4uSnyH22lfz9xibELo+XMN6lE7g5bbqhzdqFTIR2k8Mt668pCCN8EuFxfFFo4h+l3RjS9nvib5nsmJAhQ1UA//ATWVYkNas9VROdHsDLSAZSLXObyteOUl/WFn23FkbjW+CtyZkF+5lDcfote175dsKqJXmVmVewnsbHp7GcXiE+VCpHkvoe2imWsX2XXXkp5VMUSnIph7aPq4RfEQow744xKP+xtKEMVpXOVJJkHvKQE4osQCAdgVOupd X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 1cca2f9e-ff3a-4bc7-c2a5-08d8bbf46794 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:45.8820 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: mNhdL81Lv7AjvpbJ8euzeeTNZ6YifsRMJ3OYtF7BJQPxhAXViX2kPmvelfFgyNmt1Uqj5ZtjwVfFyV78nV3RKA== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We don't want to rearm the timer if driver hook reports that the device is gone. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/scheduler/sched_main.c | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index 73fccc5..9552334 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -314,6 +314,7 @@ static void drm_sched_job_timedout(struct work_struct *work) { struct drm_gpu_scheduler *sched; struct drm_sched_job *job; + enum drm_task_status status = DRM_TASK_STATUS_ALIVE; sched = container_of(work, struct drm_gpu_scheduler, work_tdr.work); @@ -331,7 +332,7 @@ static void drm_sched_job_timedout(struct work_struct *work) list_del_init(&job->list); spin_unlock(&sched->job_list_lock); - job->sched->ops->timedout_job(job); + status = job->sched->ops->timedout_job(job); /* * Guilty job did complete and hence needs to be manually removed @@ -345,9 +346,11 @@ static void drm_sched_job_timedout(struct work_struct *work) spin_unlock(&sched->job_list_lock); } - spin_lock(&sched->job_list_lock); - drm_sched_start_timeout(sched); - spin_unlock(&sched->job_list_lock); + if (status != DRM_TASK_STATUS_ENODEV) { + spin_lock(&sched->job_list_lock); + drm_sched_start_timeout(sched); + spin_unlock(&sched->job_list_lock); + } } /** From patchwork Mon Jan 18 21:01:23 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 12028261 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-16.6 required=3.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,MSGID_FROM_MTA_HEADER,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED, USER_AGENT_GIT autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7855BC433DB for ; Mon, 18 Jan 2021 21:02:53 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 42A6C207C4 for ; Mon, 18 Jan 2021 21:02:53 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 42A6C207C4 Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CDA3E6E5C6; Mon, 18 Jan 2021 21:02:50 +0000 (UTC) Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2044.outbound.protection.outlook.com [40.107.220.44]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3BA306E5BD; Mon, 18 Jan 2021 21:02:49 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=n/OvsYpaUlcYVeiZ2517ofmzVpPmRAMDfh+qOkdpM1xJhXZVOEEtm9BVsxKqVGoikUoXo7VU/nM6nypr7KaYGOwXswKG16RC3fzTwo51f2X4JsPD0AILPIDUj5u7oocmKetL//YOLVXlBWh6rLZJqurxQw+eh5fJeOWp7Ix5MsnPqvl5xZyZnqUJkFoLTtGqM80idrK3P0ODlnL/oujDI7Ssa7RLT5cNNjcXiKIG+uhdDNmwVWyLPDTMsCzOc5vGl2POCLCeFaVjn/Qwk6IeDuZv5A5MlJ06TxIaNxZ/h/TPXoS5ThIgJePUvO+k1IkYkOWCBOv7/LMdwSmCGrLRcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XDkDjw+QgfK8Gi5BC9Ui5oA7T9i8V54hxCZRAFVplHs=; b=UmrFu18V+mxRuVLeKkumBpVG6KeP/ZJ6g4nB/Mu9Sw6qf4wOomJzCOoFCsfbqf9D85f5h5E/UHKDfV6eKUHTCiaOCE99219BKge83zx2jwyDb6mzhh5hePUV2hlDwTueq+kKmjszR43jRsxjfufRvImqwke5kvEH/I0mNdvkkF2vJSuJNP9Kj6tUib0zjKbdJdzfd21JRZZw/2iKcBRhmhyWR+hTcgYNgquzzoQeT+2cJe1DzpD1ghq356xAP+KpbPnSIh2ARbPLz3+9zf7VVJwSrG5GRnExQgy96KOKgKOfQASl4aeKQwZAFIrLBqUt87woJJhZ//1rU2JsXB2meQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=amd.com; dmarc=pass action=none header.from=amd.com; dkim=pass header.d=amd.com; arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amd.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XDkDjw+QgfK8Gi5BC9Ui5oA7T9i8V54hxCZRAFVplHs=; b=Pix7s//9LWisbUL1fUFDNrIekOPX5uZHCZeHZy0eXKbCod0gd3Cxt8APMSin0WZ4qSG3LmmBwhk5wPdPb7kAoX2uAyi0A28G9Ely3dGHe9evFkUjZVSg0ZTxMwj2XG5FC0yD0jC18qrEagmqjS0qh6ZviCMAl6uBDnot5wXJGac= Authentication-Results: lists.freedesktop.org; dkim=none (message not signed) header.d=none; lists.freedesktop.org; dmarc=none action=none header.from=amd.com; Received: from SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) by SN6PR12MB4767.namprd12.prod.outlook.com (2603:10b6:805:e5::32) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3763.10; Mon, 18 Jan 2021 21:02:48 +0000 Received: from SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921]) by SN6PR12MB4623.namprd12.prod.outlook.com ([fe80::5d30:b29d:5f5b:6921%5]) with mapi id 15.20.3763.014; Mon, 18 Jan 2021 21:02:48 +0000 From: Andrey Grodzovsky To: amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, ckoenig.leichtzumerken@gmail.com, daniel.vetter@ffwll.ch, robh@kernel.org, l.stach@pengutronix.de, yuq825@gmail.com, eric@anholt.net Subject: [PATCH v4 14/14] drm/amdgpu: Prevent any job recoveries after device is unplugged. Date: Mon, 18 Jan 2021 16:01:23 -0500 Message-Id: <1611003683-3534-15-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> References: <1611003683-3534-1-git-send-email-andrey.grodzovsky@amd.com> X-Originating-IP: [2607:fea8:3edf:49b0:84d3:21cc:478c:efa7] X-ClientProxiedBy: YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) To SN6PR12MB4623.namprd12.prod.outlook.com (2603:10b6:805:e9::17) MIME-Version: 1.0 X-MS-Exchange-MessageSentRepresentingType: 1 Received: from ubuntu-1604-test.hitronhub.home (2607:fea8:3edf:49b0:84d3:21cc:478c:efa7) by YTXPR0101CA0020.CANPRD01.PROD.OUTLOOK.COM (2603:10b6:b00::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256) id 15.20.3763.9 via Frontend Transport; Mon, 18 Jan 2021 21:02:46 +0000 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-HT: Tenant X-MS-Office365-Filtering-Correlation-Id: 85738ac4-67b3-4735-3037-08d8bbf468c1 X-MS-TrafficTypeDiagnostic: SN6PR12MB4767: X-MS-Exchange-Transport-Forked: True X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2399; X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: RgjCHMqdZd4fH4Uw9Akk+V+Delipg5XRLu6FAPL6TUpH645QMD5dBoB6WHeOt/3gwYdM0urZWvAbSAY7ojyN31eUFTsqMujxkvaWg8HOT/4/WfonlOGQCslc/ECM0fSv2/HGJP3VQc6td0/qLc1pGgiPtNPTNoXkfex7peNN2wb/4IvaQoiyY+H0CEpM8K8+HHjEYlxy+l1A9yLh90ZCDaDRv0MEG1LZDaeY2/CbUwHfy+1QoQUAV/B4jiqaHUdTflCiK6zmZvdTUyL67rEBZsmgpz6Z2GeeNq5a7guXnmrlrvB7UFJw7bqJkfjJfiH9B/YnZrjpKJ9gzBqfB5yvaBBdwP9Oxje/H6jYLaoEjauaqsHzyElfNlX1empJtO7QJzEGf3oChrGELL9nksQ1ng== X-Forefront-Antispam-Report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SN6PR12MB4623.namprd12.prod.outlook.com; PTR:; CAT:NONE; SFS:(4636009)(396003)(376002)(346002)(39860400002)(366004)(136003)(86362001)(66946007)(6666004)(83380400001)(44832011)(8936002)(66556008)(316002)(6512007)(6486002)(36756003)(2616005)(66476007)(6506007)(4326008)(478600001)(5660300002)(7416002)(2906002)(52116002)(16526019)(8676002)(186003); DIR:OUT; SFP:1101; X-MS-Exchange-AntiSpam-MessageData: 34QxFK8BiRYbs2wMUeSN2r16M541wquDfSrKPfM4dHJBHhVxWm1Rzoh/sZWN9MTqNh+8Hn/Cyc4D1XM2qlU/TZaOmgT6/9VEwdrHzZsPFu47ls6WPUR0JufivRskIP9Us84WLjJ7T2UGPwIY5fuJNmWOVfAoEceFYyj8mllzGMYJjwUQqslq+Y31dX1Yy3Q0sXCApMGIMttYoAV9cW8Fn+RKJpiNbIZYVGAdth3+gQBzGe3r+1WDY5K7LZajxfLpV3u96jhDj/V0fKbrYNK4PyqQDiZRiFyPseoW7nYXaWOdce5hnyIV4gYXa92JmbVQTFt9SDxyNnrN6+SBrRSpfAEYK37SX7KgYZkvm78U8zzRDfFp9c7gXQu23yZOaWANltNXRglSDjKFD2K/x5c07V2SV20sXCZGt0q3bs2ODJimbFDNIKEsXjPiGWt6qEWZypSV9ol3G2gxikaKEz0wNADtNJpGm/xBpGquhPowX7TgkySsCb/MGSyeoZLW+wd0cbScZIBnXZ+JyvBWRcCsY50LWT5toHJhZNqEypceiy2hjZr7g4yCDcDhDCR6Ji+lVvLPCM2Nm9WmmsrYyrFolBP/FprSifJWf+1VRXkah0jCM9YyIGacxp2qqp02nDqOBnPNn3ZcsRp1ckmml2tbTK0e2+XzFROr0gd3xTo21lcB6knZh9m5rYcMQQIwQ59UlPjvHtSniRzcDJQRhDDW19Y0BgxYc7wfCHlvcXP2aDaYSVCYfW1Gw4ZmWrq7ygh4OeK86B8O2iI+TrBJDQPuwp30jzp5350gCWrJMe/Wacgt4GYexeWZgNpcZwQkM2O5v2tM2PHkJe3q0j23ji/bfVbig40zx1dAbUFEEsXQKybnuqh5X8QCZB5dQMp9NUGmnhnQ1bT7o/JfvKbXjtZdpEt2R3P3XTW+Lx6NG0q6Ri0UiugatwMWGZ3iDL1CtMWartLduJCcQGKf/zml1KRtqTnwlZp8DGevWapeyiaenS+94kPuBFUws4SWOlMT05cUXNskijI3jpOGttEk0l9/9KH80DG5JFkxAaGuKLRiAWg8hqjiVXsktAnRFJsAhFVML3+LFoJ7MRppUbdLBXAe5yi86JbW3xj0knhLCg/ISJSCOma0YeayC9zU3crZeWuG X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-Network-Message-Id: 85738ac4-67b3-4735-3037-08d8bbf468c1 X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4623.namprd12.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Internal X-MS-Exchange-CrossTenant-OriginalArrivalTime: 18 Jan 2021 21:02:48.0098 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-MailboxType: HOSTED X-MS-Exchange-CrossTenant-UserPrincipalName: dKpnSvqrU7rmyM9q6PZw42MqsBTaRoH0KytNH0buR1EwVcIi7MRCOXwNHbO/VtPuTeL/PbtXX0PbMdHLctHi9g== X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN6PR12MB4767 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Alexander.Deucher@amd.com, gregkh@linuxfoundation.org Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Return DRM_TASK_STATUS_ENODEV back to the scheduler when device is not present so they timeout timer will not be rearmed. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 19 ++++++++++++++++--- 1 file changed, 16 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index a111326..e4aa5fe 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -25,6 +25,8 @@ #include #include +#include + #include "amdgpu.h" #include "amdgpu_trace.h" @@ -34,6 +36,15 @@ static enum drm_task_status amdgpu_job_timedout(struct drm_sched_job *s_job) struct amdgpu_job *job = to_amdgpu_job(s_job); struct amdgpu_task_info ti; struct amdgpu_device *adev = ring->adev; + int idx; + + if (!drm_dev_enter(&adev->ddev, &idx)) { + DRM_INFO("%s - device unplugged skipping recovery on scheduler:%s", + __func__, s_job->sched->name); + + /* Effectively the job is aborted as the device is gone */ + return DRM_TASK_STATUS_ENODEV; + } memset(&ti, 0, sizeof(struct amdgpu_task_info)); @@ -41,7 +52,7 @@ static enum drm_task_status amdgpu_job_timedout(struct drm_sched_job *s_job) amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) { DRM_ERROR("ring %s timeout, but soft recovered\n", s_job->sched->name); - return DRM_TASK_STATUS_ALIVE; + goto exit; } amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti); @@ -53,13 +64,15 @@ static enum drm_task_status amdgpu_job_timedout(struct drm_sched_job *s_job) if (amdgpu_device_should_recover_gpu(ring->adev)) { amdgpu_device_gpu_recover(ring->adev, job); - return DRM_TASK_STATUS_ALIVE; } else { drm_sched_suspend_timeout(&ring->sched); if (amdgpu_sriov_vf(adev)) adev->virt.tdr_debug = true; - return DRM_TASK_STATUS_ALIVE; } + +exit: + drm_dev_exit(idx); + return DRM_TASK_STATUS_ALIVE; } int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,