From patchwork Sun Jun 21 06:03:01 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 11616441 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id BB55792A for ; Sun, 21 Jun 2020 06:05:47 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 8F13A2076A for ; Sun, 21 Jun 2020 06:05:47 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="0xSRYkJB" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 8F13A2076A Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C27286E428; Sun, 21 Jun 2020 06:05:43 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2055.outbound.protection.outlook.com [40.107.236.55]) by gabe.freedesktop.org (Postfix) with ESMTPS id C9E556E417; Sun, 21 Jun 2020 06:05:42 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=e0HkDzDH4SQe5k/DVfJCPJQK117eI66g87/gL7Cf6gbUw8ovGqU7DKOSVGH0IAzbcQTTr7wlJU8GJ7T3hfMSuvYiIeF1XbvMP0GMYClXENckZilgcXlinq2xkrPFuLqFh9taPjh62vAF4IK4WEjr0ebRyr9McwHINEBvXUcEfUvcF0JBRa5KBh4vxsz36O+ehjdOtUkyxq1nwM6uY231Uf6vslX9zS30L9BBCgS05c1uBi+iaHIEUUMCRajtt2NVlSQ83DlwFGgs1hrRJqcTDUi2ZN4jt3XiDc5JfarGEcpxbSkzKA/mhI/x9yfH3NXLdLSJ9/lnBr5f3MjX6CFFMg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2YLC4RzlLpAxXt+DPVp5IAwn3rTH/8N03QpL/JBrekk=; b=VT/LWtuXkIVw8zx0/9v7Ge41pYL3mj99Psm4fhFnYFtcy2ow+l6n36Bi7GZmBHJNEZ44sgVKCzamlHtymvos0qTW8tZm41P3rVZV3iY1RIsFbEqQqp9BZKNej6P4IpeZFsVQymJ1G1t+bh8WFaqvjd7doR1A3h2smLaw6RrKGJvUBZ8USMlN/3VohBx6bcZLNTg8rX8FacS3QawmfxPe+3nVMCU+JKjSEQSljEFNalaQqDqTjD6VsHhlge8kjp+fM0Q5TskM2v2bS9HsS1Sr0qpCpSEUjSVusaHoZN+113Zvd7KSahDvE3TzkAputLLI78dMvAGnCBbU/GZtZyL8TQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=permerror action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2YLC4RzlLpAxXt+DPVp5IAwn3rTH/8N03QpL/JBrekk=; b=0xSRYkJB/NeOrb2qQIjiNA+AmUXUaG0OQqrLuT7LFj5ndsrMuIKCNTCTWwsT3ptNIhlSbS4cR4r3pOCmdpt+Ls8kqNnJsX9fFeZb8c0jzfCv2qplUOdZJbjthzMS3vrN2rBFxRPh4YU3Zh+pCJpKXVzyjmKmmtwBYQe/EI5UZRA= Received: from BN6PR17CA0035.namprd17.prod.outlook.com (2603:10b6:405:75::24) by BN6PR1201MB0116.namprd12.prod.outlook.com (2603:10b6:405:56::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Sun, 21 Jun 2020 06:05:40 +0000 Received: from BN8NAM11FT038.eop-nam11.prod.protection.outlook.com (2603:10b6:405:75:cafe::b4) by BN6PR17CA0035.outlook.office365.com (2603:10b6:405:75::24) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:40 +0000 X-MS-Exchange-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXMB02.amd.com (165.204.84.17) by BN8NAM11FT038.mail.protection.outlook.com (10.13.176.246) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:38 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB02.amd.com (10.181.40.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:38 -0500 Received: from SATLEXMB01.amd.com (10.181.40.142) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:37 -0500 Received: from agrodzovsky-All-Series.amd.com (10.180.168.240) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Sun, 21 Jun 2020 01:05:37 -0500 From: Andrey Grodzovsky To: , Subject: [PATCH v2 1/8] drm: Add dummy page per device or GEM object Date: Sun, 21 Jun 2020 02:03:01 -0400 Message-ID: <1592719388-13819-2-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> References: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SATLEXMB02.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFTY:; SFS:(4636009)(136003)(346002)(376002)(396003)(39850400004)(46966005)(8676002)(186003)(26005)(86362001)(426003)(336012)(2616005)(44832011)(2906002)(36756003)(70586007)(5660300002)(83380400001)(70206006)(6666004)(81166007)(82310400002)(478600001)(8936002)(4326008)(7696005)(47076004)(110136005)(316002)(82740400003)(356005)(54906003); DIR:OUT; SFP:1101; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 0d57a965-9a68-4983-8d0a-08d815a91f37 X-MS-TrafficTypeDiagnostic: BN6PR1201MB0116: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:316; X-Forefront-PRVS: 04410E544A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: JscTEEYSZloLTR4/PZHHOrv4uRPlOM0qUa08O7+E/Necf9gLURuP9tzZsY7emAusj/l6/1lYsCaCNlcqL6JD+PDJoste42P45vfWCnvamQwXyEKR3ZU+gFv62ev+jgYZuIyKSU+nd1MdlGIRja3zub+iNYBH+8cqBl9cgRv1QfIQXupIvLdRehgY/B6ixIpypvC0Jh3MLA/knIsjD1DXudlnI1lGtOln8x6DjFFwQSz6Qaa+2A92o+fxdk5gskaB6SorTbG8DgtLdwUZyW3ysPgtxW3+1rIpS82kVag/tYCR7e8vy/633KStC6PWh+SB9okK10n5fCySm2VKvk5atHWEwcZkWDOFpnx4Y/fAvqUUaB2Cr/2UdSMCap0DTqaqD5RkfdiUaLY2E5uTsRDYgw== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2020 06:05:38.9886 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 0d57a965-9a68-4983-8d0a-08d815a91f37 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB02.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR1201MB0116 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: daniel.vetter@ffwll.ch, michel@daenzer.net, ckoenig.leichtzumerken@gmail.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Will be used to reroute CPU mapped BO's page faults once device is removed. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/drm_file.c | 8 ++++++++ drivers/gpu/drm/drm_prime.c | 10 ++++++++++ include/drm/drm_file.h | 2 ++ include/drm/drm_gem.h | 2 ++ 4 files changed, 22 insertions(+) diff --git a/drivers/gpu/drm/drm_file.c b/drivers/gpu/drm/drm_file.c index c4c704e..67c0770 100644 --- a/drivers/gpu/drm/drm_file.c +++ b/drivers/gpu/drm/drm_file.c @@ -188,6 +188,12 @@ struct drm_file *drm_file_alloc(struct drm_minor *minor) goto out_prime_destroy; } + file->dummy_page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!file->dummy_page) { + ret = -ENOMEM; + goto out_prime_destroy; + } + return file; out_prime_destroy: @@ -284,6 +290,8 @@ void drm_file_free(struct drm_file *file) if (dev->driver->postclose) dev->driver->postclose(dev, file); + __free_page(file->dummy_page); + drm_prime_destroy_file_private(&file->prime); WARN_ON(!list_empty(&file->event_list)); diff --git a/drivers/gpu/drm/drm_prime.c b/drivers/gpu/drm/drm_prime.c index 1de2cde..c482e9c 100644 --- a/drivers/gpu/drm/drm_prime.c +++ b/drivers/gpu/drm/drm_prime.c @@ -335,6 +335,13 @@ int drm_gem_prime_fd_to_handle(struct drm_device *dev, ret = drm_prime_add_buf_handle(&file_priv->prime, dma_buf, *handle); + + if (!ret) { + obj->dummy_page = alloc_page(GFP_KERNEL | __GFP_ZERO); + if (!obj->dummy_page) + ret = -ENOMEM; + } + mutex_unlock(&file_priv->prime.lock); if (ret) goto fail; @@ -1006,6 +1013,9 @@ void drm_prime_gem_destroy(struct drm_gem_object *obj, struct sg_table *sg) dma_buf_unmap_attachment(attach, sg, DMA_BIDIRECTIONAL); dma_buf = attach->dmabuf; dma_buf_detach(attach->dmabuf, attach); + + __free_page(obj->dummy_page); + /* remove the reference */ dma_buf_put(dma_buf); } diff --git a/include/drm/drm_file.h b/include/drm/drm_file.h index 19df802..349a658 100644 --- a/include/drm/drm_file.h +++ b/include/drm/drm_file.h @@ -335,6 +335,8 @@ struct drm_file { */ struct drm_prime_file_private prime; + struct page *dummy_page; + /* private: */ #if IS_ENABLED(CONFIG_DRM_LEGACY) unsigned long lock_count; /* DRI1 legacy lock count */ diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h index 0b37506..47460d1 100644 --- a/include/drm/drm_gem.h +++ b/include/drm/drm_gem.h @@ -310,6 +310,8 @@ struct drm_gem_object { * */ const struct drm_gem_object_funcs *funcs; + + struct page *dummy_page; }; /** From patchwork Sun Jun 21 06:03:02 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 11616443 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id E9F8492A for ; Sun, 21 Jun 2020 06:05:52 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id C857B20B80 for ; Sun, 21 Jun 2020 06:05:52 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="tchMP52q" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org C857B20B80 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id EBD2D6E432; Sun, 21 Jun 2020 06:05:46 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2053.outbound.protection.outlook.com [40.107.94.53]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0E1696E429; Sun, 21 Jun 2020 06:05:45 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=PfCkh8mtE8PzF3fcSBisOue9ty4JiyoX/UWl7rpY6Pz85Z9VLmT+95Od3oe1z7TvE9lOfUjj6ROSxJEUBnlhM7H6Y5aYLOr2xDFyalSRJkpbkoQ5DMQVhRx4kbC8LrhtegFeVcb5HZV2S1Z+bmTbMCCXR3qO0KPEBfY0F68pHfw/etIIu00tH++H/zp1qPAOAZkxVJ5FuDYDD9vrezJddcOAZudjb0xtwmmMWRKSedbxkgvZENkRKQnETA3M51wsdP6x0jJ0BM7F+4zVPrITxkRx/W/jSbHUuiRxukIhqPb4sox8iQMl5mCTUz/OdSVzTzmD5Gs+H19MWD63v+PfoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L1rpjBDgVFtT3sOaRH3UdkS29znTqSN7f10TADoAmIM=; b=k2/PfeyBKvwf9FJri8BbAuH2e8r+mpswsImd9/qEI9lIgljz3UXcQ5xDQDP9M5fPs7j7yYOglUgOLm18DPFa2rWGJx+/Y/S6Ub6zyw7Whi5nnXrv4H6ImRe18fz2MODoPsE5elSEo0VFGKMyrdaoRViHoF6YdpCITOq6VJam7qcPdTK0I2B5r7PSOHEZPZFX/U6A5rfNQ46rwg2paJp9IXIiAIjfpFv0v5Y+XP4cimFRHktiQlrOVpMJ/qkf+SYqGc+jo86PGVQnaia+YhFU+UB0x28ZxKRpTKgn/zTvMxiuemQJaqCB4aUNAAn+USUbjdW5ZmPGAr2KBTMtYUChTA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=permerror action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=L1rpjBDgVFtT3sOaRH3UdkS29znTqSN7f10TADoAmIM=; b=tchMP52qMq4UUu98bevekBCUtX8g7PflIl8yV4gbZ9Kb5S9NDwi/NY5Q1MIEi+gNaVKlMiAV9Msa2RMqWiy4BhJtJ829Un7S6nFWwpNsuUdajpjIKwn5X3GBPz4MDQA7OojAbMaPTISLkVSz1RzuJiRNt5mZeDOETHSoau2GkGc= Received: from BN6PR17CA0036.namprd17.prod.outlook.com (2603:10b6:405:75::25) by MN2PR12MB4520.namprd12.prod.outlook.com (2603:10b6:208:26f::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Sun, 21 Jun 2020 06:05:42 +0000 Received: from BN8NAM11FT038.eop-nam11.prod.protection.outlook.com (2603:10b6:405:75:cafe::f6) by BN6PR17CA0036.outlook.office365.com (2603:10b6:405:75::25) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21 via Frontend Transport; Sun, 21 Jun 2020 06:05:42 +0000 X-MS-Exchange-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXMB02.amd.com (165.204.84.17) by BN8NAM11FT038.mail.protection.outlook.com (10.13.176.246) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:40 +0000 Received: from SATLEXMB01.amd.com (10.181.40.142) by SATLEXMB02.amd.com (10.181.40.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:39 -0500 Received: from agrodzovsky-All-Series.amd.com (10.180.168.240) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Sun, 21 Jun 2020 01:05:39 -0500 From: Andrey Grodzovsky To: , Subject: [PATCH v2 2/8] drm/ttm: Remap all page faults to per process dummy page. Date: Sun, 21 Jun 2020 02:03:02 -0400 Message-ID: <1592719388-13819-3-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> References: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SATLEXMB02.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFTY:; SFS:(4636009)(396003)(39860400002)(376002)(346002)(136003)(46966005)(47076004)(82740400003)(186003)(6666004)(8936002)(8676002)(36756003)(70206006)(70586007)(110136005)(54906003)(316002)(478600001)(5660300002)(86362001)(7696005)(83380400001)(426003)(2906002)(356005)(336012)(44832011)(82310400002)(81166007)(26005)(2616005)(4326008); DIR:OUT; SFP:1101; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 9b81da50-a404-46e1-088c-08d815a92031 X-MS-TrafficTypeDiagnostic: MN2PR12MB4520: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2733; X-Forefront-PRVS: 04410E544A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 5QzQh19oe2gkEXow8N25mXhf5ASuH8rNT3Vx7slZBvXI889qeRi0vhx0bKvAuhoxidzjL7qNpT/vKI5tjMdAgzEEhby5EfTvIay42i5PhQdS6eDf7BRn0aqABv1jwZpDOOKfPCI2UJmgOXZ+4UHav5IUvtDCEuTQ2vhmreLRDHj/+AJ5KsYy4DRwPhNMorMlc/SuywUsBm3kqdkZ8hQUhOEMJ7sL3jR5AFSylo8DRrHBfBEtjZnxBWxesKd1OHL5npVt2rC1YumPDXbRPv5mxRaqZFtFc9h0qTibbJgVGNecwcvUVY4lNres/C8OCUJh/qdwvNysZKTXsNqdjbflH2DlZE50yHDYijHtAVWVRsA3nL+RGZELqaYnPgFzTmCGJwNagJxqcOGtPU2O9Pr2yQ== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2020 06:05:40.6226 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 9b81da50-a404-46e1-088c-08d815a92031 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB02.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4520 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: daniel.vetter@ffwll.ch, michel@daenzer.net, ckoenig.leichtzumerken@gmail.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" On device removal reroute all CPU mappings to dummy page per drm_file instance or imported GEM object. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/ttm/ttm_bo_vm.c | 65 ++++++++++++++++++++++++++++++++++++----- 1 file changed, 57 insertions(+), 8 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 389128b..2f8bf5e 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -35,6 +35,8 @@ #include #include #include +#include +#include #include #include #include @@ -328,19 +330,66 @@ vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf) pgprot_t prot; struct ttm_buffer_object *bo = vma->vm_private_data; vm_fault_t ret; + int idx; + struct drm_device *ddev = bo->base.dev; - ret = ttm_bo_vm_reserve(bo, vmf); - if (ret) - return ret; + if (drm_dev_enter(ddev, &idx)) { + ret = ttm_bo_vm_reserve(bo, vmf); + if (ret) + goto exit; + + prot = vma->vm_page_prot; - prot = vma->vm_page_prot; - ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT); - if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) + ret = ttm_bo_vm_fault_reserved(vmf, prot, TTM_BO_VM_NUM_PREFAULT); + if (ret == VM_FAULT_RETRY && !(vmf->flags & FAULT_FLAG_RETRY_NOWAIT)) + goto exit; + + dma_resv_unlock(bo->base.resv); + +exit: + drm_dev_exit(idx); return ret; + } else { - dma_resv_unlock(bo->base.resv); + struct drm_file *file = NULL; + struct page *dummy_page = NULL; + int handle; - return ret; + /* We are faulting on imported BO from dma_buf */ + if (bo->base.dma_buf && bo->base.import_attach) { + dummy_page = bo->base.dummy_page; + /* We are faulting on non imported BO, find drm_file owning the BO*/ + } else { + struct drm_gem_object *gobj; + + mutex_lock(&ddev->filelist_mutex); + list_for_each_entry(file, &ddev->filelist, lhead) { + spin_lock(&file->table_lock); + idr_for_each_entry(&file->object_idr, gobj, handle) { + if (gobj == &bo->base) { + dummy_page = file->dummy_page; + break; + } + } + spin_unlock(&file->table_lock); + } + mutex_unlock(&ddev->filelist_mutex); + } + + if (dummy_page) { + /* + * Let do_fault complete the PTE install e.t.c using vmf->page + * + * TODO - should i call free_page somewhere ? + */ + get_page(dummy_page); + vmf->page = dummy_page; + return 0; + } else { + return VM_FAULT_SIGSEGV; + } + } } EXPORT_SYMBOL(ttm_bo_vm_fault); -- 2.7.4 From patchwork Sun Jun 21 06:03:03 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 11616447 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 35FAC161F for ; Sun, 21 Jun 2020 06:05:59 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 137DC20B80 for ; Sun, 21 Jun 2020 06:05:59 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="o9x6I1eV" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 137DC20B80 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 536F76E440; Sun, 21 Jun 2020 06:05:49 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2088.outbound.protection.outlook.com [40.107.236.88]) by gabe.freedesktop.org (Postfix) with ESMTPS id 97EEC6E431; Sun, 21 Jun 2020 06:05:45 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=GcHLRq1aHU21EgUTCEZnxJrJpHXzeb0IUYA+MBwR00EyzGo9PKybRTEXeCR1bds4vJnPCVubDV7DVwrTZD7hZM/P7Qx6BKgfvw+xOOVVU7XgbK7n+/SaWODpgu1Fj/LrLBaDhteu+D+UN3jZjVni+u78XHuYjzqkGAMmtXQNHoQZtg5x9wUz5s72l8uNMuXPgyeHtK9CLIWwWLOiVSQ2Fp5wvDgIDKkGBvLWENI9kpTIt7rsQ5Ssi1UpGJbZvXaR11PxBIiqfPSQTzKaHXx4ojQy+gYoR1HBfwYArGdaUerdu+hfeedDzvimCbPn5Zz66QJJhTaTmCCKaWQpzUcdkA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0LK31aSdV7jTR9uF8W+DmCM+W+0H/07TTemjLEKoRhM=; b=VEZwSyHjVvJOitSCidmqzJH+QxqVSOSS01eAxKaSu9gkdckjAiRf79MCXTPa8Bj/H9dschiKQJi1WROheXyrngKehc6cKWE2C4tkUcd2sGQk8MKVe2SD5gwWSR9rCS/BJ1brUlBkUvL2A9XedZ8ncszrpbXoL+WyaG4bkbsJlon42UGUdR2dlAZLNkowQhqr6bf3X6u9dhqFAmAN3snjQ5us94UXfYKWjiVmuOfOil9AWCsL25yUkU52LRCmbwETwp0nt/6ir1RNND432RIFlconWeuNjDAOmXuxkPxydRytyIGEaU153iL/9UqTP+QtLIgqNoXJ7jv2/wVaGM3JAw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=permerror action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=0LK31aSdV7jTR9uF8W+DmCM+W+0H/07TTemjLEKoRhM=; b=o9x6I1eVhL6kCjaaKYYtotBSRbd2M31cCs25hBSEmuOd2uAZm3wiAQfiW/M6Ia4zvkA+SGscf5ZsQwWTXS4WJms71fi8OJz2CzxmfMU+WQM4g9ochkdjQVUd1a4pcPz39078dfQxDmdMmLaJ8C1zb5O53jGUm625jW8wtERjbvE= Received: from DM6PR17CA0031.namprd17.prod.outlook.com (2603:10b6:5:1b3::44) by DM6PR12MB4297.namprd12.prod.outlook.com (2603:10b6:5:211::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Sun, 21 Jun 2020 06:05:42 +0000 Received: from DM6NAM11FT061.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1b3:cafe::7f) by DM6PR17CA0031.outlook.office365.com (2603:10b6:5:1b3::44) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:42 +0000 X-MS-Exchange-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXMB01.amd.com (165.204.84.17) by DM6NAM11FT061.mail.protection.outlook.com (10.13.173.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:42 +0000 Received: from SATLEXMB03.amd.com (10.181.40.144) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:41 -0500 Received: from SATLEXMB01.amd.com (10.181.40.142) by SATLEXMB03.amd.com (10.181.40.144) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:40 -0500 Received: from agrodzovsky-All-Series.amd.com (10.180.168.240) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Sun, 21 Jun 2020 01:05:40 -0500 From: Andrey Grodzovsky To: , Subject: [PATCH v2 3/8] drm/ttm: Add unampping of the entire device address space Date: Sun, 21 Jun 2020 02:03:03 -0400 Message-ID: <1592719388-13819-4-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> References: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SATLEXMB01.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFTY:; SFS:(4636009)(376002)(39860400002)(346002)(396003)(136003)(46966005)(81166007)(36756003)(4326008)(478600001)(110136005)(316002)(54906003)(8676002)(336012)(186003)(70206006)(26005)(426003)(5660300002)(44832011)(82310400002)(2616005)(356005)(8936002)(7696005)(70586007)(47076004)(83380400001)(86362001)(82740400003)(6666004)(2906002); DIR:OUT; SFP:1101; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e9e56809-e162-4669-c5ca-08d815a92142 X-MS-TrafficTypeDiagnostic: DM6PR12MB4297: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:5516; X-Forefront-PRVS: 04410E544A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: YPEFZ3OAkLZZpmlxnyUG3RQG1QQoFDfKcF9Ln9qceIyQP8QwqxAz8d+I2x85w6R80MFklSP2CV5aWuuzBxGpj2i6Pm4OKcepcoiq6EtR/5rje1MvqVayiwq0o+BEkOEgIjp5gwcQDsvXv1UC4hUhQUWT2cHHyA97+AuZX7zNv83g81ILB43dorIOxB4UYkQG/OsgsM1XgViK6C9k+SCXXigmsKRMcfBvqJyTdurT61C4CCqxSas0s9cHo56TMWVWB2rfe1HV0Gkwnacwu4Lac7SlOoDGnb0/JpJqVb+t3Tt2k1VSWq/7Rum9N0YIsC1hU4yz9CRvQGgK7GnxqULcYQqXbH2y9X2PskIdPcJepoZ35U5tBfGbAusRpxidAX2wQ901V+XGz+PCQO4yUtNlWg== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2020 06:05:42.3766 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e9e56809-e162-4669-c5ca-08d815a92142 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4297 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: daniel.vetter@ffwll.ch, michel@daenzer.net, ckoenig.leichtzumerken@gmail.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Helper function to be used to invalidate all BOs CPU mappings once device is removed. Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/ttm/ttm_bo.c | 8 ++++++-- include/drm/ttm/ttm_bo_driver.h | 7 +++++++ 2 files changed, 13 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index c5b516f..926a365 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -1750,10 +1750,14 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) ttm_bo_unmap_virtual_locked(bo); ttm_mem_io_unlock(man); } - - EXPORT_SYMBOL(ttm_bo_unmap_virtual); +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev) +{ + unmap_mapping_range(bdev->dev_mapping, 0, 0, 1); +} +EXPORT_SYMBOL(ttm_bo_unmap_virtual_address_space); + int ttm_bo_wait(struct ttm_buffer_object *bo, bool interruptible, bool no_wait) { diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h index c9e0fd0..39ea44f 100644 --- a/include/drm/ttm/ttm_bo_driver.h +++ b/include/drm/ttm/ttm_bo_driver.h @@ -601,6 +601,13 @@ int ttm_bo_device_init(struct ttm_bo_device *bdev, void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); /** + * ttm_bo_unmap_virtual_address_space + * + * @bdev: tear down all the virtual mappings for this device + */ +void ttm_bo_unmap_virtual_address_space(struct ttm_bo_device *bdev); + +/** * ttm_bo_unmap_virtual * * @bo: tear down the virtual mappings for this BO From patchwork Sun Jun 21 06:03:04 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 11616449 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id A192592A for ; Sun, 21 Jun 2020 06:06:01 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7F65420B80 for ; Sun, 21 Jun 2020 06:06:01 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="GKTN4/0f" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 7F65420B80 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0CE9B6E44E; Sun, 21 Jun 2020 06:05:50 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2089.outbound.protection.outlook.com [40.107.243.89]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0660A6E434; Sun, 21 Jun 2020 06:05:47 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=YaHCOixe+7dMbtkcQ+oMAJHmyNkhxQ/a6MSG8R0yZ0MmIsj1Xq1nIvdO1VijA+iI3lkvkw6y1Ydv7MTixtA22hc381swdDcOSVuFbLJD7z9H3BCl6UmKNcqFECPLPJKjp6EheWchAhohGYLZYGvtqn2gZ8DA/DPL665SODHzBJ7289W3iDKdC5e7w2sjmOEda9+AzOT/hnxP9S6RUDvmMVzkP/kMXWwGp+7o0SEERC4c1CQxEvKYNwtcqAE2XX0RlHdJ9eBZoj0A8v7TGgb8XxL31rTmoMzlnP35oETxRPjnC+6Wjb+eGSRlzehRU5q55wIMWX700Oz0hnhPexKoUw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=niIOCFMpu2oNBW0pdWe+HP447O4g0QE+eF2dIelQ4Nw=; b=ICNEU3K5ampEJy4mnNW3iUHSRxhOE5SpiYHmkz7oKunGFBL1PyivTxRvreVJiLTBmNRnq/h54NHvj1BvbpvENA/n0FZEje4mmX5Hn+EktDHt8hsr+lIYY9UhR0yeW7X3ZwcW+niwOUR76MhcyybZ3jImr8+eaapqO6JfNm0gbPQoiDdrrP/fM7zHMXLUdv1kH3g4RJ49jyIVraxoKTkjUDhIJW5TMFC/DeCXODDI71GC3yrL9rxybpqsoIdm0hQtq9dxdXDfw83U8/yh2ki4iHu3yuIbQOYHFy0pt1CDrIrPOFN0XrDH2O7K8xW+fCAkv26svD60l2oVlZERVVLBbA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=permerror action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=niIOCFMpu2oNBW0pdWe+HP447O4g0QE+eF2dIelQ4Nw=; b=GKTN4/0fErQOdiFSqkARFZANW8zI/dtjUhjLgL8wv+IiZ6EODBuQ1MEY64svM84eiM0bmghfswjKUQKDJJ6DgvQUxUnF+e4I+Gsaoc+mFcoxGTulzQZO7MnbH6zToYNKYdNUwTz2ZL6yKmC5vRLe2ElACSbwrZZ8A7VL7gxZ8Zo= Received: from DM6PR17CA0025.namprd17.prod.outlook.com (2603:10b6:5:1b3::38) by BYAPR12MB3160.namprd12.prod.outlook.com (2603:10b6:a03:132::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Sun, 21 Jun 2020 06:05:43 +0000 Received: from DM6NAM11FT061.eop-nam11.prod.protection.outlook.com (2603:10b6:5:1b3:cafe::96) by DM6PR17CA0025.outlook.office365.com (2603:10b6:5:1b3::38) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:43 +0000 X-MS-Exchange-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXMB01.amd.com (165.204.84.17) by DM6NAM11FT061.mail.protection.outlook.com (10.13.173.138) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:42 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:42 -0500 Received: from SATLEXMB01.amd.com (10.181.40.142) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:42 -0500 Received: from agrodzovsky-All-Series.amd.com (10.180.168.240) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Sun, 21 Jun 2020 01:05:42 -0500 From: Andrey Grodzovsky To: , Subject: [PATCH v2 4/8] drm/amdgpu: Split amdgpu_device_fini into early and late Date: Sun, 21 Jun 2020 02:03:04 -0400 Message-ID: <1592719388-13819-5-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> References: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SATLEXMB01.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39860400002)(136003)(376002)(396003)(46966005)(70586007)(336012)(478600001)(7696005)(86362001)(26005)(186003)(8936002)(36756003)(5660300002)(8676002)(6666004)(2616005)(4326008)(426003)(70206006)(44832011)(356005)(316002)(54906003)(2906002)(82740400003)(82310400002)(47076004)(83380400001)(110136005)(81166007); DIR:OUT; SFP:1101; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 35af6b8a-e9c2-4037-9979-08d815a9219c X-MS-TrafficTypeDiagnostic: BYAPR12MB3160: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2958; X-Forefront-PRVS: 04410E544A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: TiyI5WrApFYPTU+u9BTKmZ+dG9qbjvK4UrGcBMg3z85DKsPYO1t07hdX62VZIiOAyGzr1CCS5ijf13DN7htTDnviz5fvTNctXjeaKpD5pACV7RI9iOtbnLbJ5hD20ZZdsIYKAhlVaGuR6wXaLhUHOognC1LrhH0AgdSueCSi1bpHQs1iRnxT+SQNcUT1jUHod+XN5w3RAB4AmVJckW3OLyDsLOXT9z+mDaDQmOVpEvg8uiQvQ42yFvucKPgqusfCUTH/t1zigrsQBtdGnxOzljUzFWM6yq4NaaveJqylHtmde8DQzq/PJS2GNxgkwkcV5SRmGb4MaA/vAJIhHdF5Kj0pc+s+KjzA0aJwvNpSAv6gjcSSDCm8Heav7jDPq72mdoH6BWyWd1kPwisI2Dsp2Q== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2020 06:05:42.9652 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 35af6b8a-e9c2-4037-9979-08d815a9219c X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR12MB3160 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: daniel.vetter@ffwll.ch, michel@daenzer.net, ckoenig.leichtzumerken@gmail.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Some of the stuff in amdgpu_device_fini such as HW interrupts disable and pending fences finilization must be done right away on pci_remove while most of the stuff which relates to finilizing and releasing driver data structures can be kept until drm_driver.release hook is called, i.e. when the last device reference is dropped. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 +++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 15 +++++++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 6 ++---- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c | 24 +++++++++++++++--------- drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 23 +++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c | 3 +++ 7 files changed, 54 insertions(+), 24 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 2a806cb..604a681 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -1003,7 +1003,9 @@ int amdgpu_device_init(struct amdgpu_device *adev, struct drm_device *ddev, struct pci_dev *pdev, uint32_t flags); -void amdgpu_device_fini(struct amdgpu_device *adev); +void amdgpu_device_fini_early(struct amdgpu_device *adev); +void amdgpu_device_fini_late(struct amdgpu_device *adev); + int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev); void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos, @@ -1188,6 +1190,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev); int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv); void amdgpu_driver_postclose_kms(struct drm_device *dev, struct drm_file *file_priv); +void amdgpu_driver_release_kms(struct drm_device *dev); + int amdgpu_device_ip_suspend(struct amdgpu_device *adev); int amdgpu_device_suspend(struct drm_device *dev, bool fbcon); int amdgpu_device_resume(struct drm_device *dev, bool fbcon); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index cc41e8f..e7b9065 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2309,6 +2309,8 @@ static int amdgpu_device_ip_fini(struct amdgpu_device *adev) { int i, r; + //DRM_ERROR("adev 0x%llx", (long long unsigned int)adev); + amdgpu_ras_pre_fini(adev); if (adev->gmc.xgmi.num_physical_nodes > 1) @@ -3304,10 +3306,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, * Tear down the driver info (all asics). * Called at driver shutdown. */ -void amdgpu_device_fini(struct amdgpu_device *adev) +void amdgpu_device_fini_early(struct amdgpu_device *adev) { - int r; - DRM_INFO("amdgpu: finishing device.\n"); flush_delayed_work(&adev->delayed_init_work); adev->shutdown = true; @@ -3330,7 +3330,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev) if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); amdgpu_fbdev_fini(adev); - r = amdgpu_device_ip_fini(adev); + + amdgpu_irq_fini_early(adev); +} + +void amdgpu_device_fini_late(struct amdgpu_device *adev) +{ + amdgpu_device_ip_fini(adev); if (adev->firmware.gpu_info_fw) { release_firmware(adev->firmware.gpu_info_fw); adev->firmware.gpu_info_fw = NULL; @@ -3368,6 +3374,7 @@ void amdgpu_device_fini(struct amdgpu_device *adev) amdgpu_pmu_fini(adev); if (amdgpu_discovery && adev->asic_type >= CHIP_NAVI10) amdgpu_discovery_fini(adev); + } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 9e5afa5..43592dc 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1134,12 +1134,9 @@ amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev); -#ifdef MODULE - if (THIS_MODULE->state != MODULE_STATE_GOING) -#endif - DRM_ERROR("Hotplug removal is not supported\n"); drm_dev_unplug(dev); amdgpu_driver_unload_kms(dev); + pci_disable_device(pdev); pci_set_drvdata(pdev, NULL); drm_dev_put(dev); @@ -1445,6 +1442,7 @@ static struct drm_driver kms_driver = { .dumb_create = amdgpu_mode_dumb_create, .dumb_map_offset = amdgpu_mode_dumb_mmap, .fops = &amdgpu_driver_kms_fops, + .release = &amdgpu_driver_release_kms, .prime_handle_to_fd = drm_gem_prime_handle_to_fd, .prime_fd_to_handle = drm_gem_prime_fd_to_handle, diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c index 0cc4c67..1697655 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c @@ -49,6 +49,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_ih.h" #include "atom.h" @@ -297,6 +298,20 @@ int amdgpu_irq_init(struct amdgpu_device *adev) return 0; } + +void amdgpu_irq_fini_early(struct amdgpu_device *adev) +{ + if (adev->irq.installed) { + drm_irq_uninstall(adev->ddev); + adev->irq.installed = false; + if (adev->irq.msi_enabled) + pci_free_irq_vectors(adev->pdev); + + if (!amdgpu_device_has_dc_support(adev)) + flush_work(&adev->hotplug_work); + } +} + /** * amdgpu_irq_fini - shut down interrupt handling * @@ -310,15 +325,6 @@ void amdgpu_irq_fini(struct amdgpu_device *adev) { unsigned i, j; - if (adev->irq.installed) { - drm_irq_uninstall(adev->ddev); - adev->irq.installed = false; - if (adev->irq.msi_enabled) - pci_free_irq_vectors(adev->pdev); - if (!amdgpu_device_has_dc_support(adev)) - flush_work(&adev->hotplug_work); - } - for (i = 0; i < AMDGPU_IRQ_CLIENTID_MAX; ++i) { if (!adev->irq.client[i].sources) continue; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h index c718e94..718c70f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h @@ -104,6 +104,7 @@ irqreturn_t amdgpu_irq_handler(int irq, void *arg); int amdgpu_irq_init(struct amdgpu_device *adev); void amdgpu_irq_fini(struct amdgpu_device *adev); +void amdgpu_irq_fini_early(struct amdgpu_device *adev); int amdgpu_irq_add_id(struct amdgpu_device *adev, unsigned client_id, unsigned src_id, struct amdgpu_irq_src *source); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c index c0b1904..9d0af22 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c @@ -29,6 +29,7 @@ #include "amdgpu.h" #include #include +#include #include "amdgpu_sched.h" #include "amdgpu_uvd.h" #include "amdgpu_vce.h" @@ -86,7 +87,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) amdgpu_unregister_gpu_instance(adev); if (adev->rmmio == NULL) - goto done_free; + return; if (adev->runpm) { pm_runtime_get_sync(dev->dev); @@ -95,11 +96,7 @@ void amdgpu_driver_unload_kms(struct drm_device *dev) amdgpu_acpi_fini(adev); - amdgpu_device_fini(adev); - -done_free: - kfree(adev); - dev->dev_private = NULL; + amdgpu_device_fini_early(adev); } void amdgpu_register_gpu_instance(struct amdgpu_device *adev) @@ -1108,6 +1105,20 @@ void amdgpu_driver_postclose_kms(struct drm_device *dev, pm_runtime_put_autosuspend(dev->dev); } + +void amdgpu_driver_release_kms (struct drm_device *dev) +{ + struct amdgpu_device *adev = dev->dev_private; + + amdgpu_device_fini_late(adev); + + kfree(adev); + dev->dev_private = NULL; + + drm_dev_fini(dev); + kfree(dev); +} + /* * VBlank related functions. */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c index 7348619..169c2239 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c @@ -2056,9 +2056,12 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev) { struct amdgpu_ras *con = amdgpu_ras_get_context(adev); + //DRM_ERROR("adev 0x%llx", (long long unsigned int)adev); + if (!con) return 0; + /* Need disable ras on all IPs here before ip [hw/sw]fini */ amdgpu_ras_disable_all_features(adev, 0); amdgpu_ras_recovery_fini(adev); From patchwork Sun Jun 21 06:03:05 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 11616455 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 13F1692A for ; Sun, 21 Jun 2020 06:06:09 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id E5F8F22363 for ; Sun, 21 Jun 2020 06:06:08 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="wmqvPRT3" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org E5F8F22363 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BF7556E44D; Sun, 21 Jun 2020 06:05:52 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM02-CY1-obe.outbound.protection.outlook.com (mail-eopbgr760044.outbound.protection.outlook.com [40.107.76.44]) by gabe.freedesktop.org (Postfix) with ESMTPS id 0F15C6E451; Sun, 21 Jun 2020 06:05:50 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=MUSTUCg8QA28QOsdK7ET1kzVlbReLYE0CLjJ2tGkvLL8G25BhKzr/BxiC8Gn3IIT3Q395vHO1EW1qbMj2YXlPrQSFHw+8W9P69hueLe8emSBTwUKZ08cmk9EBrnoN2hK5n1kVel/5B6P/8AreUAR9IgDm53m2KNpuesChLM6DdrLQAsvc/SW5+ceF55RRji2Uvl8SgbEHNaRWWkACq/toKxDHHIE/m65BfdSXT3C3r+qe4lDCYqfbUdvtcZJINyiFtUgXlpCU+aYqMUMK52J0Acgo6vfQOyi2sFVEgNV1BR2TkmibgvXj80Ykm416AS99veyOsn/KLpVGHrANUNlZA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yIrWzPMowm5TjuEA66Z8rI3MCFz1SjUvQwR6p2ItHPY=; b=GinsgcsGfjBZHghT5nPN1v2YTy3BXFimFoyrWwULYGlFFSMiteVj704tKgytZcNxfLNMpLWdCh5bNOP7xMkWru6FaSksNdBfH/6nsL/AoanWyUF0STOLpa2XEsTjknlRQZycVzKWRPiDPtkgpNKiyaLujhinWHNRRUAxLUdprb08DmT3PDbT4V/79fTDC3OBUmmkbWCO17hrPHPokMtv6ECU8BSgES1B7uuW8NUnMOTofCZpTNCjIXceBngc4irJ9+3L6No7092Y7Qz6yqI1iZrEmyveA485naNpz/tpkyAxVbEME2f5GNbOaS2bGKPgvwjBheG0yJqXCcJwiQIeew== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=permerror action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=yIrWzPMowm5TjuEA66Z8rI3MCFz1SjUvQwR6p2ItHPY=; b=wmqvPRT33I0deUOB1oexOgKakmY73LK4ByNNwEMho4P8YjoJYoMqIPIPbILsy6RV1PD1F2UhoPpiUbUumET/eN0KdbHQmFDIcjZlR4XXHkHKMvYAZbX4O2gG4CAKdLqCNIgBCBm/aaIy9KGdNuxOXtdO8OSmy+WKvrOlnJrTQgI= Received: from BN6PR22CA0069.namprd22.prod.outlook.com (2603:10b6:404:ca::31) by BN6PR12MB1185.namprd12.prod.outlook.com (2603:10b6:404:1e::10) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Sun, 21 Jun 2020 06:05:44 +0000 Received: from BN8NAM11FT043.eop-nam11.prod.protection.outlook.com (2603:10b6:404:ca:cafe::ea) by BN6PR22CA0069.outlook.office365.com (2603:10b6:404:ca::31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21 via Frontend Transport; Sun, 21 Jun 2020 06:05:44 +0000 X-MS-Exchange-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXMB02.amd.com (165.204.84.17) by BN8NAM11FT043.mail.protection.outlook.com (10.13.177.218) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:44 +0000 Received: from SATLEXMB06.amd.com (10.181.40.147) by SATLEXMB02.amd.com (10.181.40.143) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:44 -0500 Received: from SATLEXMB01.amd.com (10.181.40.142) by SATLEXMB06.amd.com (10.181.40.147) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:43 -0500 Received: from agrodzovsky-All-Series.amd.com (10.180.168.240) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Sun, 21 Jun 2020 01:05:43 -0500 From: Andrey Grodzovsky To: , Subject: [PATCH v2 5/8] drm/amdgpu: Refactor sysfs removal Date: Sun, 21 Jun 2020 02:03:05 -0400 Message-ID: <1592719388-13819-6-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> References: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SATLEXMB02.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFTY:; SFS:(4636009)(346002)(136003)(396003)(39850400004)(376002)(46966005)(336012)(70206006)(478600001)(7696005)(86362001)(26005)(186003)(8676002)(36756003)(30864003)(5660300002)(8936002)(70586007)(6666004)(2616005)(4326008)(426003)(44832011)(356005)(2906002)(82310400002)(47076004)(82740400003)(83380400001)(110136005)(81166007)(316002)(54906003); DIR:OUT; SFP:1101; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 84289a31-c179-4f89-2dc7-08d815a9228b X-MS-TrafficTypeDiagnostic: BN6PR12MB1185: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:2887; X-Forefront-PRVS: 04410E544A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: fR39fJ7bwDbEg9hgKXS4gpwjq9ac9cOWxtnXd0rVyb4rXqNRhxGMmxZ3fM2rII/bJykWL+xzMJIOrdm5IRfeIvwe7gQYX4YYhd0zdofdVVedWWcK0C0IJLrpEDb17SjurUH6pHA+qOqpU4s8HOZ2nWgF8iDMtaJQhqU+o1fOC7n5vNQ9dxSS+KYgjjhgnISWwLdjMnAxafa+rYi6uWAgxWgMc4d2oxS7+GXov4sgTdnUOC2ryld6qe3PNbbizqiNm2ZWStu0YpX9FZek8CgzWudi3rZLEsPt+f+01/rPxUB7dAiOXlkLg3YMItX2vfdEUcLNopV4JB9qiZ62tFIsoD9Vie8Up1GzpakFxj6X3a5LS3ohV11OJkMUJ8es2fufns4VZmuNMZBPvDiIkMixEQ== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2020 06:05:44.5698 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 84289a31-c179-4f89-2dc7-08d815a9228b X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB02.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR12MB1185 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: daniel.vetter@ffwll.ch, michel@daenzer.net, ckoenig.leichtzumerken@gmail.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Track sysfs files in a list so they all can be removed during pci remove since otherwise their removal after that causes crash because parent folder was already removed during pci remove. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 13 +++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c | 7 +++++- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 35 ++++++++++++++++++++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c | 12 ++++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c | 8 ++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c | 17 ++++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c | 13 ++++++++++- drivers/gpu/drm/amd/amdgpu/df_v3_6.c | 10 +++++--- 8 files changed, 99 insertions(+), 16 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 604a681..ba3775f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -726,6 +726,15 @@ struct amd_powerplay { #define AMDGPU_RESET_MAGIC_NUM 64 #define AMDGPU_MAX_DF_PERFMONS 4 + +struct amdgpu_sysfs_list_node { + struct list_head head; + struct device_attribute *attr; +}; + +#define AMDGPU_DEVICE_ATTR_LIST_NODE(_attr) \ + struct amdgpu_sysfs_list_node dev_attr_handle_##_attr = {.attr = &dev_attr_##_attr} + struct amdgpu_device { struct device *dev; struct drm_device *ddev; @@ -992,6 +1001,10 @@ struct amdgpu_device { char product_number[16]; char product_name[32]; char serial[16]; + + struct list_head sysfs_files_list; + struct mutex sysfs_files_list_lock; + }; static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_bo_device *bdev) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c index fdd52d8..c1549ee 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c @@ -1950,8 +1950,10 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev, return snprintf(buf, PAGE_SIZE, "%s\n", ctx->vbios_version); } + static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version, NULL); +static AMDGPU_DEVICE_ATTR_LIST_NODE(vbios_version); /** * amdgpu_atombios_fini - free the driver info and callbacks for atombios @@ -1972,7 +1974,6 @@ void amdgpu_atombios_fini(struct amdgpu_device *adev) adev->mode_info.atom_context = NULL; kfree(adev->mode_info.atom_card_info); adev->mode_info.atom_card_info = NULL; - device_remove_file(adev->dev, &dev_attr_vbios_version); } /** @@ -2038,6 +2039,10 @@ int amdgpu_atombios_init(struct amdgpu_device *adev) return ret; } + mutex_lock(&adev->sysfs_files_list_lock); + list_add_tail(&dev_attr_handle_vbios_version.head, &adev->sysfs_files_list); + mutex_unlock(&adev->sysfs_files_list_lock); + return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index e7b9065..3173046 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -2928,6 +2928,12 @@ static const struct attribute *amdgpu_dev_attributes[] = { NULL }; +static AMDGPU_DEVICE_ATTR_LIST_NODE(product_name); +static AMDGPU_DEVICE_ATTR_LIST_NODE(product_number); +static AMDGPU_DEVICE_ATTR_LIST_NODE(serial_number); +static AMDGPU_DEVICE_ATTR_LIST_NODE(pcie_replay_count); + + /** * amdgpu_device_init - initialize the driver * @@ -3029,6 +3035,9 @@ int amdgpu_device_init(struct amdgpu_device *adev, INIT_LIST_HEAD(&adev->shadow_list); mutex_init(&adev->shadow_list_lock); + INIT_LIST_HEAD(&adev->sysfs_files_list); + mutex_init(&adev->sysfs_files_list_lock); + INIT_DELAYED_WORK(&adev->delayed_init_work, amdgpu_device_delayed_init_work_handler); INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work, @@ -3281,6 +3290,13 @@ int amdgpu_device_init(struct amdgpu_device *adev, if (r) { dev_err(adev->dev, "Could not create amdgpu device attr\n"); return r; + } else { + mutex_lock(&adev->sysfs_files_list_lock); + list_add_tail(&dev_attr_handle_product_name.head, &adev->sysfs_files_list); + list_add_tail(&dev_attr_handle_product_number.head, &adev->sysfs_files_list); + list_add_tail(&dev_attr_handle_serial_number.head, &adev->sysfs_files_list); + list_add_tail(&dev_attr_handle_pcie_replay_count.head, &adev->sysfs_files_list); + mutex_unlock(&adev->sysfs_files_list_lock); } if (IS_ENABLED(CONFIG_PERF_EVENTS)) @@ -3298,6 +3314,16 @@ int amdgpu_device_init(struct amdgpu_device *adev, return r; } +static void amdgpu_sysfs_remove_files(struct amdgpu_device *adev) +{ + struct amdgpu_sysfs_list_node *node; + + mutex_lock(&adev->sysfs_files_list_lock); + list_for_each_entry(node, &adev->sysfs_files_list, head) + device_remove_file(adev->dev, node->attr); + mutex_unlock(&adev->sysfs_files_list_lock); +} + /** * amdgpu_device_fini - tear down the driver * @@ -3332,6 +3358,11 @@ void amdgpu_device_fini_early(struct amdgpu_device *adev) amdgpu_fbdev_fini(adev); amdgpu_irq_fini_early(adev); + + amdgpu_sysfs_remove_files(adev); + + if (adev->ucode_sysfs_en) + amdgpu_ucode_sysfs_fini(adev); } void amdgpu_device_fini_late(struct amdgpu_device *adev) @@ -3366,10 +3397,6 @@ void amdgpu_device_fini_late(struct amdgpu_device *adev) adev->rmmio = NULL; amdgpu_device_doorbell_fini(adev); - if (adev->ucode_sysfs_en) - amdgpu_ucode_sysfs_fini(adev); - - sysfs_remove_files(&adev->dev->kobj, amdgpu_dev_attributes); if (IS_ENABLED(CONFIG_PERF_EVENTS)) amdgpu_pmu_fini(adev); if (amdgpu_discovery && adev->asic_type >= CHIP_NAVI10) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c index 6271044..e7b6c4a 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c @@ -76,6 +76,9 @@ static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO, static DEVICE_ATTR(mem_info_gtt_used, S_IRUGO, amdgpu_mem_info_gtt_used_show, NULL); +static AMDGPU_DEVICE_ATTR_LIST_NODE(mem_info_gtt_total); +static AMDGPU_DEVICE_ATTR_LIST_NODE(mem_info_gtt_used); + /** * amdgpu_gtt_mgr_init - init GTT manager and DRM MM * @@ -114,6 +117,11 @@ static int amdgpu_gtt_mgr_init(struct ttm_mem_type_manager *man, return ret; } + mutex_lock(&adev->sysfs_files_list_lock); + list_add_tail(&dev_attr_handle_mem_info_gtt_total.head, &adev->sysfs_files_list); + list_add_tail(&dev_attr_handle_mem_info_gtt_used.head, &adev->sysfs_files_list); + mutex_unlock(&adev->sysfs_files_list_lock); + return 0; } @@ -127,7 +135,6 @@ static int amdgpu_gtt_mgr_init(struct ttm_mem_type_manager *man, */ static int amdgpu_gtt_mgr_fini(struct ttm_mem_type_manager *man) { - struct amdgpu_device *adev = amdgpu_ttm_adev(man->bdev); struct amdgpu_gtt_mgr *mgr = man->priv; spin_lock(&mgr->lock); drm_mm_takedown(&mgr->mm); @@ -135,9 +142,6 @@ static int amdgpu_gtt_mgr_fini(struct ttm_mem_type_manager *man) kfree(mgr); man->priv = NULL; - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_total); - device_remove_file(adev->dev, &dev_attr_mem_info_gtt_used); - return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c index ddb4af0c..554fec0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c @@ -2216,6 +2216,8 @@ static DEVICE_ATTR(usbc_pd_fw, S_IRUGO | S_IWUSR, psp_usbc_pd_fw_sysfs_read, psp_usbc_pd_fw_sysfs_write); +static AMDGPU_DEVICE_ATTR_LIST_NODE(usbc_pd_fw); + const struct amd_ip_funcs psp_ip_funcs = { @@ -2242,13 +2244,17 @@ static int psp_sysfs_init(struct amdgpu_device *adev) if (ret) DRM_ERROR("Failed to create USBC PD FW control file!"); + else { + mutex_lock(&adev->sysfs_files_list_lock); + list_add_tail(&dev_attr_handle_usbc_pd_fw.head, &adev->sysfs_files_list); + mutex_unlock(&adev->sysfs_files_list_lock); + } return ret; } static void psp_sysfs_fini(struct amdgpu_device *adev) { - device_remove_file(adev->dev, &dev_attr_usbc_pd_fw); } const struct amdgpu_ip_block_version psp_v3_1_ip_block = diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c index 7723937..39c400c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c @@ -148,6 +148,12 @@ static DEVICE_ATTR(mem_info_vis_vram_used, S_IRUGO, static DEVICE_ATTR(mem_info_vram_vendor, S_IRUGO, amdgpu_mem_info_vram_vendor, NULL); +static AMDGPU_DEVICE_ATTR_LIST_NODE(mem_info_vram_total); +static AMDGPU_DEVICE_ATTR_LIST_NODE(mem_info_vis_vram_total); +static AMDGPU_DEVICE_ATTR_LIST_NODE(mem_info_vram_used); +static AMDGPU_DEVICE_ATTR_LIST_NODE(mem_info_vis_vram_used); +static AMDGPU_DEVICE_ATTR_LIST_NODE(mem_info_vram_vendor); + static const struct attribute *amdgpu_vram_mgr_attributes[] = { &dev_attr_mem_info_vram_total.attr, &dev_attr_mem_info_vis_vram_total.attr, @@ -184,6 +190,15 @@ static int amdgpu_vram_mgr_init(struct ttm_mem_type_manager *man, ret = sysfs_create_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); if (ret) DRM_ERROR("Failed to register sysfs\n"); + else { + mutex_lock(&adev->sysfs_files_list_lock); + list_add_tail(&dev_attr_handle_mem_info_vram_total.head, &adev->sysfs_files_list); + list_add_tail(&dev_attr_handle_mem_info_vis_vram_total.head, &adev->sysfs_files_list); + list_add_tail(&dev_attr_handle_mem_info_vram_used.head, &adev->sysfs_files_list); + list_add_tail(&dev_attr_handle_mem_info_vis_vram_used.head, &adev->sysfs_files_list); + list_add_tail(&dev_attr_handle_mem_info_vram_vendor.head, &adev->sysfs_files_list); + mutex_unlock(&adev->sysfs_files_list_lock); + } return 0; } @@ -198,7 +213,6 @@ static int amdgpu_vram_mgr_init(struct ttm_mem_type_manager *man, */ static int amdgpu_vram_mgr_fini(struct ttm_mem_type_manager *man) { - struct amdgpu_device *adev = amdgpu_ttm_adev(man->bdev); struct amdgpu_vram_mgr *mgr = man->priv; spin_lock(&mgr->lock); @@ -206,7 +220,6 @@ static int amdgpu_vram_mgr_fini(struct ttm_mem_type_manager *man) spin_unlock(&mgr->lock); kfree(mgr); man->priv = NULL; - sysfs_remove_files(&adev->dev->kobj, amdgpu_vram_mgr_attributes); return 0; } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c index 90610b4..455eaa4 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c @@ -272,6 +272,9 @@ static ssize_t amdgpu_xgmi_show_error(struct device *dev, static DEVICE_ATTR(xgmi_device_id, S_IRUGO, amdgpu_xgmi_show_device_id, NULL); static DEVICE_ATTR(xgmi_error, S_IRUGO, amdgpu_xgmi_show_error, NULL); +static AMDGPU_DEVICE_ATTR_LIST_NODE(xgmi_device_id); +static AMDGPU_DEVICE_ATTR_LIST_NODE(xgmi_error); + static int amdgpu_xgmi_sysfs_add_dev_info(struct amdgpu_device *adev, struct amdgpu_hive_info *hive) { @@ -285,10 +288,19 @@ static int amdgpu_xgmi_sysfs_add_dev_info(struct amdgpu_device *adev, return ret; } + mutex_lock(&adev->sysfs_files_list_lock); + list_add_tail(&dev_attr_handle_xgmi_device_id.head, &adev->sysfs_files_list); + mutex_unlock(&adev->sysfs_files_list_lock); + /* Create xgmi error file */ ret = device_create_file(adev->dev, &dev_attr_xgmi_error); if (ret) pr_err("failed to create xgmi_error\n"); + else { + mutex_lock(&adev->sysfs_files_list_lock); + list_add_tail(&dev_attr_handle_xgmi_error.head, &adev->sysfs_files_list); + mutex_unlock(&adev->sysfs_files_list_lock); + } /* Create sysfs link to hive info folder on the first device */ @@ -325,7 +337,6 @@ static int amdgpu_xgmi_sysfs_add_dev_info(struct amdgpu_device *adev, static void amdgpu_xgmi_sysfs_rem_dev_info(struct amdgpu_device *adev, struct amdgpu_hive_info *hive) { - device_remove_file(adev->dev, &dev_attr_xgmi_device_id); sysfs_remove_link(&adev->dev->kobj, adev->ddev->unique); sysfs_remove_link(hive->kobj, adev->ddev->unique); } diff --git a/drivers/gpu/drm/amd/amdgpu/df_v3_6.c b/drivers/gpu/drm/amd/amdgpu/df_v3_6.c index a7b8292..f95b0b2 100644 --- a/drivers/gpu/drm/amd/amdgpu/df_v3_6.c +++ b/drivers/gpu/drm/amd/amdgpu/df_v3_6.c @@ -265,6 +265,8 @@ static ssize_t df_v3_6_get_df_cntr_avail(struct device *dev, /* device attr for available perfmon counters */ static DEVICE_ATTR(df_cntr_avail, S_IRUGO, df_v3_6_get_df_cntr_avail, NULL); +static AMDGPU_DEVICE_ATTR_LIST_NODE(df_cntr_avail); + static void df_v3_6_query_hashes(struct amdgpu_device *adev) { u32 tmp; @@ -299,6 +301,11 @@ static void df_v3_6_sw_init(struct amdgpu_device *adev) ret = device_create_file(adev->dev, &dev_attr_df_cntr_avail); if (ret) DRM_ERROR("failed to create file for available df counters\n"); + else { + mutex_lock(&adev->sysfs_files_list_lock); + list_add_tail(&dev_attr_handle_df_cntr_avail.head, &adev->sysfs_files_list); + mutex_unlock(&adev->sysfs_files_list_lock); + } for (i = 0; i < AMDGPU_MAX_DF_PERFMONS; i++) adev->df_perfmon_config_assign_mask[i] = 0; @@ -308,9 +315,6 @@ static void df_v3_6_sw_init(struct amdgpu_device *adev) static void df_v3_6_sw_fini(struct amdgpu_device *adev) { - - device_remove_file(adev->dev, &dev_attr_df_cntr_avail); - } static void df_v3_6_enable_broadcast_mode(struct amdgpu_device *adev, From patchwork Sun Jun 21 06:03:06 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 11616451 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id D00C91731 for ; Sun, 21 Jun 2020 06:06:03 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id AD69520B80 for ; Sun, 21 Jun 2020 06:06:03 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="19PqHaBO" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org AD69520B80 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 820F66E44B; Sun, 21 Jun 2020 06:05:51 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM02-SN1-obe.outbound.protection.outlook.com (mail-eopbgr770053.outbound.protection.outlook.com [40.107.77.53]) by gabe.freedesktop.org (Postfix) with ESMTPS id D8B5D6E44D; Sun, 21 Jun 2020 06:05:49 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jVIPbV57ZGrfNQjcsQ4vg+0LHUdZPP78+wvoaNf9eCufHwEcp80JwE57I3Ke+559VpSdFF+7OIORIwOAbCDbXNtbs7QE6fFD3Y0O9Uv2e5WIPW7Hvz4f2G+mHtIiXCqliQwAo3cpnqvt2lAOkCor174zmPLCzPcBK4OWhW1IgYf1U3USLHi+/JYw4zBHytAvO3mf85cWQJteDAC1FtB5IhJb/FqMaZFUyEAiAch+zgYdDiV5t/6q+pFs/g6Ysi1ohnC7TcvtDt6TrOdBZN7of0owO7WHFQI3wgCN5nb2tWII7sgnitcYK50PBN205Zzddg8giEL14jD2gtu25WtoBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K9TQbTbwKbdmkpe8dusD9dkfjTByEb0MvsSwznmmm0I=; b=mrkilx+/HkU2wPbEcRPXhKB06DvNoDyTIbdlOK7JAh3EVyxSFGbMcVxJQgk0Lv7b6YKsL8hFHNuhyk1VkWKpMZCiQhQWF+A8hq7F48fBxPaYEkDO34hX0MSIS/nT3PfUlm7mbyUf1/4k5EDj9QvtnQzWfh5bXZZU1VFeFcXbGrNKSHAgH+zsCJWa1i9Dj84qbe6wwxPHlS2Nt1UunE8ZNjoD97MBu1Eb47bGxg0mwWfUk+EeJrjMpgEhOCNMOnWgUegZ1e+xar6x3WnA9UapQikEglzL2+ZtBi1HJLK9rvaZ+IzGMj0Go4V8a4gvD5g7CzbRLE9LA6ytq8ZDoUZcdg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=permerror action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=K9TQbTbwKbdmkpe8dusD9dkfjTByEb0MvsSwznmmm0I=; b=19PqHaBOxUWtSVgrVjqJ/GL4ViKEkj622e6WXJlK70yUXiOkZJItJ7qY7iP8pjltcuqeUgplUjW3+XJ7RrfpmvxDn+ib4LECU98nqUkRJGD4lwI+sLfTWCsqhMxyiXzqdYQCWU4nuLJiNyLnL/b1Pmeu8rTog0+128dlrPo/nqQ= Received: from DM5PR12CA0011.namprd12.prod.outlook.com (2603:10b6:4:1::21) by DM5PR12MB1242.namprd12.prod.outlook.com (2603:10b6:3:6d::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Sun, 21 Jun 2020 06:05:47 +0000 Received: from DM6NAM11FT052.eop-nam11.prod.protection.outlook.com (2603:10b6:4:1:cafe::57) by DM5PR12CA0011.outlook.office365.com (2603:10b6:4:1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:47 +0000 X-MS-Exchange-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXMB01.amd.com (165.204.84.17) by DM6NAM11FT052.mail.protection.outlook.com (10.13.172.111) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:45 +0000 Received: from SATLEXMB01.amd.com (10.181.40.142) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:45 -0500 Received: from agrodzovsky-All-Series.amd.com (10.180.168.240) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Sun, 21 Jun 2020 01:05:44 -0500 From: Andrey Grodzovsky To: , Subject: [PATCH v2 6/8] drm/amdgpu: Unmap entire device address space on device remove. Date: Sun, 21 Jun 2020 02:03:06 -0400 Message-ID: <1592719388-13819-7-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> References: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SATLEXMB01.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFTY:; SFS:(4636009)(39860400002)(376002)(396003)(136003)(346002)(46966005)(2906002)(26005)(186003)(82740400003)(70586007)(4326008)(54906003)(70206006)(47076004)(6666004)(4744005)(316002)(7696005)(110136005)(5660300002)(36756003)(426003)(336012)(44832011)(478600001)(2616005)(81166007)(356005)(8676002)(8936002)(82310400002)(86362001)(83380400001); DIR:OUT; SFP:1101; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: e0f20f85-2f7c-49cc-3172-08d815a9232a X-MS-TrafficTypeDiagnostic: DM5PR12MB1242: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:849; X-Forefront-PRVS: 04410E544A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: uKdNuHt7slXmV5/lS8J4aaujY/b6cY0lhvIelb5Ozvb2Xd8Tw8VVCldGY5w9dMwdMsLbd7F5FIp77CgE1fH72Zdwu6RzRds9VqR/mXJJ343cVlippaGEt1L/mFM0at2z4qJVya5/8qy7sKsVePn11AoynnRQtDSR4328M1DHlT7jmEB8cEhbnj/9oNzAymMeCSdCPxYdVpDpickPSccWxceW/p8dP9YyeM4X8rI+tAknfFaLG0fNff5HPIAfgGa6tGnu6KyaEPFesghGb5sDMU1k89O5u8dz2HGl5muRgpx0obMsKDr65pIPEtfe8wCVu8sHL59+/KGQoJbhY59arDMoXUDRFNU3y3wPhlNe69Tv34D1QvQFcTV5xalpUcWZzbJTRGoqJDYKp7C6iafhZA== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2020 06:05:45.5709 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: e0f20f85-2f7c-49cc-3172-08d815a9232a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1242 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: daniel.vetter@ffwll.ch, michel@daenzer.net, ckoenig.leichtzumerken@gmail.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Use the new TTM interface to invalidate all exsisting BO CPU mappings form all user proccesses. Signed-off-by: Andrey Grodzovsky Reviewed-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 43592dc..6932d75 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1135,6 +1135,7 @@ amdgpu_pci_remove(struct pci_dev *pdev) struct drm_device *dev = pci_get_drvdata(pdev); drm_dev_unplug(dev); + ttm_bo_unmap_virtual_address_space(&adev->mman.bdev); amdgpu_driver_unload_kms(dev); pci_disable_device(pdev); From patchwork Sun Jun 21 06:03:07 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 11616457 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 1C49A92A for ; Sun, 21 Jun 2020 06:06:11 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id EDF73223E0 for ; Sun, 21 Jun 2020 06:06:10 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="oL1zLciT" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org EDF73223E0 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 605746E435; Sun, 21 Jun 2020 06:06:00 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2058.outbound.protection.outlook.com [40.107.244.58]) by gabe.freedesktop.org (Postfix) with ESMTPS id 8D6B46E46E; Sun, 21 Jun 2020 06:05:53 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=OZsUz83fI0CVhDsJSk4vQkWD1Ylli6UmX163x7arJ16a2x8U0ocfsMhJkM5mryjZIGM63dzHlN9QMWyVdWhvb1hvWShIyeQKl+k/nMzs5fBDesrhaI79ohvWnCQV8T8W7ws6bbcbSgy24w05dIKFBhZuyf/oQ4UH2iLZ35ZUtLbi+3Td5GEOwP7biyn2VA4AmZe/UbL5qg8/iRgnPeylVBpy0lCZsVI1+l4obEL53hEFoG1wrc4BVkEotPJRO/FpF7707FSwpVXDI1AjQBKdqpyprssQjZXi5F5REZvZP0Ijkbn6SRj6urpNPJK966pDfIUcmt6PnDYSl2yYeogXgA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mWbJvVp7IuCrsCYEhlhEzkyhqVj4OaLruQXzNsKht7k=; b=aEagy46gHbt4F4+2nSqE36S5XkoXVl8TdCbhpIaSDEorBTqa1WekVPx/4eOb2pjAyAhJ66dTaeJhbUD7VrC2/qPg8atDSi21WzuovpKd+xO20MXzMQ4n3CSKk/P6BuAVGZtJmb2JOvqsFTtsdFHhiZ7a3Bt3QUG7h6oB/AkylIf/7lTmDA+aPWjWN6a08LcMpY8FOfubeGxdlEFQJpDAt2lbmguTGR7+ZOMvL9gqLQPReVcrVKbwqEfjtRDRlAwdQj051hnhnblCiBAJVn2ndGaXZSCnCm2yPVUGVhbN7P9pDBhBCOK5m0D/cpBby7qM/rE8iwIj48W/sOad224FrQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=permerror action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=mWbJvVp7IuCrsCYEhlhEzkyhqVj4OaLruQXzNsKht7k=; b=oL1zLciTVEJ3GJ8m4X1OsZPn0W/oRgOfSUmrqZZDWshBiMQS697Ax0gtKph0lo6B2oyxfCC6WPmXpx18UM0yUb4Nyh4Xw3dsu9dMHPkFGiAzJJP8Uehmr8NZMH+IwFjUlGMHIHXLNSVdUMJaj83A0phL14vRW9Z2St1K8M4zVm8= Received: from DM5PR12CA0011.namprd12.prod.outlook.com (2603:10b6:4:1::21) by BY5PR12MB3777.namprd12.prod.outlook.com (2603:10b6:a03:1a9::14) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.21; Sun, 21 Jun 2020 06:05:48 +0000 Received: from DM6NAM11FT052.eop-nam11.prod.protection.outlook.com (2603:10b6:4:1:cafe::57) by DM5PR12CA0011.outlook.office365.com (2603:10b6:4:1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:47 +0000 X-MS-Exchange-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXMB01.amd.com (165.204.84.17) by DM6NAM11FT052.mail.protection.outlook.com (10.13.172.111) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:47 +0000 Received: from SATLEXMB01.amd.com (10.181.40.142) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:46 -0500 Received: from agrodzovsky-All-Series.amd.com (10.180.168.240) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Sun, 21 Jun 2020 01:05:45 -0500 From: Andrey Grodzovsky To: , Subject: [PATCH v2 7/8] drm/amdgpu: Fix sdma code crash post device unplug Date: Sun, 21 Jun 2020 02:03:07 -0400 Message-ID: <1592719388-13819-8-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> References: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SATLEXMB01.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFTY:; SFS:(4636009)(346002)(39860400002)(376002)(136003)(396003)(46966005)(70586007)(70206006)(478600001)(7696005)(86362001)(186003)(26005)(8936002)(5660300002)(36756003)(8676002)(6666004)(2616005)(4326008)(426003)(336012)(44832011)(54906003)(2906002)(82310400002)(82740400003)(83380400001)(47076004)(110136005)(81166007)(316002)(356005); DIR:OUT; SFP:1101; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d8c810c6-0c97-4b4d-9ae7-08d815a9247a X-MS-TrafficTypeDiagnostic: BY5PR12MB3777: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:165; X-Forefront-PRVS: 04410E544A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: djjX+oz3HObDeEOdsO24Mm2eiaTIIMbkFjlRslYbAz9B/aew5EC78CcIhx6woQUCFYxVjVih1ylJLKADetQdxarqV3UDac3nn0GfTrRuRrKWSXUvJuVjRTcC1LUORUNPp8O0Ua0kJ5zu1afak5i9jemXE+pnFJ4EYerofIOdhp98hMdQvMZlVx6YrUFkHWOa+9AZDQtM5e2ICFz7M9ykdsXBLb7M/J6hS+fZJ41C92Ugt86J1yYtXDwMIpQ2VVTcW3++uEy/X/fzDWIUG1JHINUWQ+HIXGwewRRJha0K0rrQDuXY7aqK33+ycckpeVTgVERbrDejiV5sIoHtldXqrpZkf1U/89X+aE990Wq2A00kuevINzcFCnOyhtF4HqwyaDHwdgYK12RyToyxQvmqtA== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2020 06:05:47.7756 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d8c810c6-0c97-4b4d-9ae7-08d815a9247a X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY5PR12MB3777 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: daniel.vetter@ffwll.ch, michel@daenzer.net, ckoenig.leichtzumerken@gmail.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" entity->rq becomes null aftre device unplugged so just return early in that case. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c index 8d9c6fe..d252427 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c @@ -24,6 +24,7 @@ #include "amdgpu_job.h" #include "amdgpu_object.h" #include "amdgpu_trace.h" +#include #define AMDGPU_VM_SDMA_MIN_NUM_DW 256u #define AMDGPU_VM_SDMA_MAX_NUM_DW (16u * 1024u) @@ -94,7 +95,12 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p, struct drm_sched_entity *entity; struct amdgpu_ring *ring; struct dma_fence *f; - int r; + int r, idx; + + if (!drm_dev_enter(p->adev->ddev, &idx)) { + r = -ENODEV; + goto nodev; + } entity = p->immediate ? &p->vm->immediate : &p->vm->delayed; ring = container_of(entity->rq->sched, struct amdgpu_ring, sched); @@ -104,7 +110,7 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p, WARN_ON(ib->length_dw > p->num_dw_left); r = amdgpu_job_submit(p->job, entity, AMDGPU_FENCE_OWNER_VM, &f); if (r) - goto error; + goto job_fail; if (p->unlocked) { struct dma_fence *tmp = dma_fence_get(f); @@ -118,10 +124,15 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p, if (fence && !p->immediate) swap(*fence, f); dma_fence_put(f); - return 0; -error: - amdgpu_job_free(p->job); + r = 0; + +job_fail: + drm_dev_exit(idx); +nodev: + if (r) + amdgpu_job_free(p->job); + return r; } From patchwork Sun Jun 21 06:03:08 2020 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 11616453 Return-Path: Received: from mail.kernel.org (pdx-korg-mail-1.web.codeaurora.org [172.30.200.123]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id 6840A161F for ; Sun, 21 Jun 2020 06:06:07 +0000 (UTC) Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 4336D22363 for ; Sun, 21 Jun 2020 06:06:07 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=fail reason="signature verification failed" (1024-bit key) header.d=amdcloud.onmicrosoft.com header.i=@amdcloud.onmicrosoft.com header.b="Ct1SPv6x" DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 4336D22363 Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=amd.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=dri-devel-bounces@lists.freedesktop.org Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D58FD6E451; Sun, 21 Jun 2020 06:05:52 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2066.outbound.protection.outlook.com [40.107.244.66]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3A73D6E454; Sun, 21 Jun 2020 06:05:50 +0000 (UTC) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=XFGqpcmZNiX2t1bqDxUgIQIUPgNy/A+chp8ZPSD0/TofcvrByQhDbEmnwVLstgWp+RD/dedUH+zMXiLGhjI4UQUXcl44VlMSbzVbF34jKsoRV3j5CK7gjtLpiSxKVfnpLli7e9BOY3Xo/9puUa6Lf66EUTUjAD2mTSg6FGV2XHgo+9ExkWJ6Is04mnDiOKRx8tAQy+2/l9Ng6Isr9JRFJgIjMYgsEfEBRqBrTfzzBvuOm//kOiIhGDSGglQkIeRoIkShzUeEqmyFj1j8yUbjbNP2cxHURCN28WCfDOblyHfxOy+DS8/N782xSzFqrIph6lyjNUk+sb218FNR0O3q3g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=92+us9FlJ1/JudX1QA0Ie+34DqI2ZMNc/GGMAzsjoGU=; b=Su6xaNGddKFfrPJ0nVTSKSf7Bk2Ods5Y0azR2e0cs0d51dMl2FjoVDx55f98UsxBDk5Mlrb+zDiGvuxexGawKyolS9MZ5KcuQmPCg87jRWDSNCFSb9C1V+Lg5QKPh94/VGYMnHxWb/+fkcKOd+ExxDfDFXBXGYiIcz02zXr7gSQN9d/sySly2Mt6cro5W7Up9QK/gjzHhSBhYBAH3AhuICjnpfm4c6oM9FzH1v8flrtET3IQXGMeGsC2uFxd1EYDdXpbrvc/m6bqM+9x51379VKeiPNQm4Wp2oPAf9gAsFTWBgnTRdVTVXSV+gA1tHRTAnzeF7dkhKKb2VTh5A9ZFg== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=none (sender ip is 165.204.84.17) smtp.rcpttodomain=lists.freedesktop.org smtp.mailfrom=amd.com; dmarc=permerror action=none header.from=amd.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amdcloud.onmicrosoft.com; s=selector2-amdcloud-onmicrosoft-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=92+us9FlJ1/JudX1QA0Ie+34DqI2ZMNc/GGMAzsjoGU=; b=Ct1SPv6xTA2Vk+7FVxOkVKF+otujiwFID3Uj6xGQz9as+YTMCoM3GoDhjdyTPweENldqGipJWOYXiXtVwyqNoIniBJ4j8Ke8pOl4U9+9jw5u/pTT5aqpu3w6rQ+6tRyLC8DFMomVK0OoFXhsQiV4Ab8rb4rQdGjZyGo6NDeAg6o= Received: from DM5PR12CA0011.namprd12.prod.outlook.com (2603:10b6:4:1::21) by DM5PR12MB1643.namprd12.prod.outlook.com (2603:10b6:4:11::8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22; Sun, 21 Jun 2020 06:05:49 +0000 Received: from DM6NAM11FT052.eop-nam11.prod.protection.outlook.com (2603:10b6:4:1:cafe::a6) by DM5PR12CA0011.outlook.office365.com (2603:10b6:4:1::21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:49 +0000 X-MS-Exchange-Authentication-Results: spf=none (sender IP is 165.204.84.17) smtp.mailfrom=amd.com; lists.freedesktop.org; dkim=none (message not signed) header.d=none;lists.freedesktop.org; dmarc=permerror action=none header.from=amd.com; Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXMB01.amd.com (165.204.84.17) by DM6NAM11FT052.mail.protection.outlook.com (10.13.172.111) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.20.3109.22 via Frontend Transport; Sun, 21 Jun 2020 06:05:48 +0000 Received: from SATLEXMB01.amd.com (10.181.40.142) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.1713.5; Sun, 21 Jun 2020 01:05:47 -0500 Received: from agrodzovsky-All-Series.amd.com (10.180.168.240) by SATLEXMB01.amd.com (10.181.40.142) with Microsoft SMTP Server id 15.1.1713.5 via Frontend Transport; Sun, 21 Jun 2020 01:05:47 -0500 From: Andrey Grodzovsky To: , Subject: [PATCH v2 8/8] drm/amdgpu: Prevent any job recoveries after device is unplugged. Date: Sun, 21 Jun 2020 02:03:08 -0400 Message-ID: <1592719388-13819-9-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> References: <1592719388-13819-1-git-send-email-andrey.grodzovsky@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; CTRY:US; LANG:en; SCL:1; SRV:; IPV:NLI; SFV:NSPM; H:SATLEXMB01.amd.com; PTR:InfoDomainNonexistent; CAT:NONE; SFTY:; SFS:(4636009)(39850400004)(136003)(396003)(376002)(346002)(46966005)(8676002)(4326008)(2906002)(186003)(82310400002)(26005)(44832011)(54906003)(8936002)(82740400003)(2616005)(36756003)(7696005)(81166007)(5660300002)(6666004)(110136005)(47076004)(478600001)(426003)(70206006)(86362001)(70586007)(316002)(356005)(336012); DIR:OUT; SFP:1101; X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 827b7709-5374-468b-0c69-08d815a92522 X-MS-TrafficTypeDiagnostic: DM5PR12MB1643: X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:849; X-Forefront-PRVS: 04410E544A X-MS-Exchange-SenderADCheck: 1 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: 0yyy/m/DNc7Wk102HbS3TV0GywUXYskp04+2XfTWvF9KPBuo00enr3F1Q27r/X+tqYZ33TcWBiRpRGZECrM8LLSsnTCWE2C8cfLIKSNeLhc1328hsb2A6sLuwCnwKqJGqpeQz52j8X6juqK9klU6+8PEwcZ84715ntZIuE3vrGS2sLEMYjjEurgBZg+0FftGRSmohaDhGkEHUJ+x06r6SoQfxHDXKMWUDtpmCHwB1aDwqT7n5+xwujUpXJzMnvkGFHYLvBrrxr9t/iVck5rC/L70Ga7IYQNOoGKrTcZoBXUtsxM2VzhkIlugjZergUKpBH7IdQTN6YpbLvjbMT8TtWQ1tDhW4mY9eAnNhKQ1zlucZbEIe1WL0DI8w/QqKMxxSrDjAOe/dz1Z4fA4nGycUA== X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Jun 2020 06:05:48.8740 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 827b7709-5374-468b-0c69-08d815a92522 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXMB01.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB1643 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: daniel.vetter@ffwll.ch, michel@daenzer.net, ckoenig.leichtzumerken@gmail.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" No point to try recovery if device is gone, just messes up things. Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 16 ++++++++++++++++ drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 8 ++++++++ 2 files changed, 24 insertions(+) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 6932d75..5d6d3d9 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -1129,12 +1129,28 @@ static int amdgpu_pci_probe(struct pci_dev *pdev, return ret; } +static void amdgpu_cancel_all_tdr(struct amdgpu_device *adev) +{ + int i; + + for (i = 0; i < AMDGPU_MAX_RINGS; ++i) { + struct amdgpu_ring *ring = adev->rings[i]; + + if (!ring || !ring->sched.thread) + continue; + + cancel_delayed_work_sync(&ring->sched.work_tdr); + } +} + static void amdgpu_pci_remove(struct pci_dev *pdev) { struct drm_device *dev = pci_get_drvdata(pdev); + struct amdgpu_device *adev = dev->dev_private; drm_dev_unplug(dev); + amdgpu_cancel_all_tdr(adev); ttm_bo_unmap_virtual_address_space(&adev->mman.bdev); amdgpu_driver_unload_kms(dev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index 4720718..87ff0c0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c @@ -28,6 +28,8 @@ #include "amdgpu.h" #include "amdgpu_trace.h" +#include + static void amdgpu_job_timedout(struct drm_sched_job *s_job) { struct amdgpu_ring *ring = to_amdgpu_ring(s_job->sched); @@ -37,6 +39,12 @@ static void amdgpu_job_timedout(struct drm_sched_job *s_job) memset(&ti, 0, sizeof(struct amdgpu_task_info)); + if (drm_dev_is_unplugged(adev->ddev)) { + DRM_INFO("ring %s timeout, but device unplugged, skipping.\n", + s_job->sched->name); + return; + } + if (amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) { DRM_ERROR("ring %s timeout, but soft recovered\n", s_job->sched->name);