From patchwork Thu Dec 20 19:23:34 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 10739441 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id CDE5D14E5 for ; Thu, 20 Dec 2018 19:23:54 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id B80B2271CB for ; Thu, 20 Dec 2018 19:23:54 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id A621A27CEE; Thu, 20 Dec 2018 19:23:54 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id B50AC271CB for ; Thu, 20 Dec 2018 19:23:53 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id CEC746F4D4; Thu, 20 Dec 2018 19:23:50 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM04-CO1-obe.outbound.protection.outlook.com (mail-eopbgr690086.outbound.protection.outlook.com [40.107.69.86]) by gabe.freedesktop.org (Postfix) with ESMTPS id D45696F4D2; Thu, 20 Dec 2018 19:23:48 +0000 (UTC) Received: from MWHPR1201CA0010.namprd12.prod.outlook.com (2603:10b6:301:4a::20) by DM5PR12MB2440.namprd12.prod.outlook.com (2603:10b6:4:b4::33) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1446.21; Thu, 20 Dec 2018 19:23:46 +0000 Received: from DM3NAM03FT050.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e49::203) by MWHPR1201CA0010.outlook.office365.com (2603:10b6:301:4a::20) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1446.17 via Frontend Transport; Thu, 20 Dec 2018 19:23:46 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV02.amd.com (165.204.84.17) by DM3NAM03FT050.mail.protection.outlook.com (10.152.82.252) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1446.11 via Frontend Transport; Thu, 20 Dec 2018 19:23:45 +0000 Received: from agrodzovsky-All-Series.amd.com (10.34.1.3) by SATLEXCHOV02.amd.com (10.181.40.72) with Microsoft SMTP Server id 14.3.389.1; Thu, 20 Dec 2018 13:23:43 -0600 From: Andrey Grodzovsky To: , , , , Subject: [PATCH v5 1/2] drm/sched: Refactor ring mirror list handling. Date: Thu, 20 Dec 2018 14:23:34 -0500 Message-ID: <1545333815-29870-1-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(346002)(376002)(39860400002)(136003)(396003)(2980300002)(428003)(189003)(199004)(186003)(77096007)(26005)(356004)(4744004)(110136005)(36756003)(486006)(54906003)(316002)(476003)(6666004)(126002)(16586007)(426003)(305945005)(7696005)(2616005)(478600001)(50466002)(50226002)(8936002)(2201001)(51416003)(86362001)(68736007)(14444005)(48376002)(44832011)(8676002)(106466001)(336012)(81166006)(81156014)(105586002)(4326008)(39060400002)(47776003)(53416004)(72206003)(104016004)(97736004)(53936002)(5660300001)(2906002)(2101003); DIR:OUT; SFP:1101; SCL:1; SRVR:DM5PR12MB2440; H:SATLEXCHOV02.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; MX:1; A:1; X-Microsoft-Exchange-Diagnostics: 1; DM3NAM03FT050; 1:2jTYYRDkbRpN7l98B4kx+Eu6wJhWP2lRc1dJPv3tHXiP2C58sN3z1oYlNxAp5H1Ifmwf9I0DwwLyHbuvoPRJxP2FFvsEOFERKIf0WKdxaFG9rqHuGASWCpa0ffhWSit4 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 359ef3c2-fedb-4531-ef10-08d666b0a925 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(5600074)(711020)(2017052603328)(7153060); SRVR:DM5PR12MB2440; X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB2440; 3:7Ak/cN9EXCvi6a6h/tHzoq/W9tFgscFjWjfgLhdX454rn2AAwDJEOAfNupolPzqsIh1HirkAa0od0gOa59cjBOkX3uqc3rcTsueYBLotkgu4ku+jwSha1PNKHTxEsELfZx1rsN6MSsIL5jDBmAHcMRq4y84+tyeY0JdRFECKgqSaFz3O0qHr32dbi6F7TPrzZGeqwAo9Z40IOjMjPqNVhRGD32HjgqOLPDMfYZyx3/xdKXglcL80Y2fIZIBQ20BaBxrFsXdpbjoC5BhjTXeQ4G24aGOiOj0ry2pnGGD2zHNOwrMECDMgzNfvUbVGhYtLH3TqZNaJUnb5jCQue42/wPx7NegE+9Gi96K9oJfIRBo=; 25:07fzCuRsGRj5Z/u0hcOzcXQjSYpULKsMYs7MeUrogZ9C32mwF5aqHg5AnuZtpMkiduRd2GJw6bSQ3fslw3+a3bwgRqpfuK8UoPY/dutTqdvq9hBRwVPOoUHn7YP64fll+0iNCVu1WDNdvqH4G/N6OGQxOxiklEn0N8PWQuwz0HFRGQyVHtXucOz+Ws933uFUDpDYRVLL8hMT6EKyyvI1wNuDJ+nnQnxaCTUljgZGPa9sZizIGDhNxhtJ6nYPKPS4Z3mSqTUzrcAcFaP5TdjYbWemNNwgEaAHojjh5Of29k02dcT5I5so86FPp6IfgVZN/Qemv9WEI1NCJNg1vwt+tA== X-MS-TrafficTypeDiagnostic: DM5PR12MB2440: X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB2440; 31:2ePZ6YB0+XVr3PKrmreGxMNFwN53VkU1UlkcqeUEq1+CpE7lZYQ54IQz9WdMPzBiAbu/xIjRVyxRkTakDh4uoLnczRjTQwjh9l6wYrmgIZ162eW74M74mIgtUPt09k1Litx9DlSML8/qWSXOIAGmfDT6gjMlCGPmVXs9nTXgP2JFl4C43gqSmZSY3J38Js7xdu6JKY5dLGAIpwmOC67YtKO70FeoRmiE9Bbb3Q5rPzE=; 20:RxgenMNHb7h/zEJjYnwJo3PA5K+DwCB+AOkFe5FoYu5Ndp72k/Ic62v45RqvUHLmeXdIhXAYMrD8A/GH7b2942GDyBxYcI6WGNiC+8FvZr441d9uI+9UgDJZlGriPY93CSVBmgjInUErPa3YiKsPjwqR9Tvuj88w+l1wqieLJrwIOm/E9q1jM/K3SPb+pNDEloADcKGPLEVSbE+eS+HvcBuXe4W4mueD703g1oMOb3DfJu+je6vVM83HOiRiS7/DOOxWdeTOU7P7DIMwgBtArWLyRgJpczzHbUkijZqs8JSR3kfyfLQw6h7zgfT7sg8Kuckzz6V+9f+TeC1uHb0NA70eQkc0l1MIo5seYPGiPD3wJZ0DPUI2QZMUXNyWRQeQgrwtqArx0+oZH4Zk+gSZP1/A1Ml6F6HtNucSZD+dHQ3O6vnNS/T1mHuWSPJqJ+FTa7onQNQbXl5JRtGzGNiXsVlPuDKZS1sMm6V4WcbmmEsk6cftoiQQJLuaCd/lfLG2 X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(999002)(5005026)(6040522)(2401047)(8121501046)(10201501046)(3002001)(3231475)(944501520)(52105112)(93006095)(93003095)(6055026)(149066)(150057)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123564045)(20161123558120)(20161123562045)(20161123560045)(201708071742011)(7699051)(76991095); SRVR:DM5PR12MB2440; BCL:0; PCL:0; RULEID:; SRVR:DM5PR12MB2440; X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB2440; 4:OfEqj4+vGwAHMZiFAwKvmqZCn79G5WYfEWY6dTLKffhnQV1CdE+WBS/rF+NPJqyMohnJFVtp2lueFdLIbwbju0QPKp2+QUnUI4dbAVnUgmyydhxkjMfdgrgHaRkAGzk0aHPXCDId7lPFaG2/SBewxYxt0GJZ5YKdUJX5/SyB+okYG9cJQe+ORW8OnztqOLrUlvRBJBjHTlalK7CQTUzr0pXVl333I3Q7k3QrZC1KRAtyDBD0YpfDKmYlXZT+0r9SPzWUsZ6ztSN3MDNSkX7d5A== X-Forefront-PRVS: 0892FA9A88 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; DM5PR12MB2440; 23:jl6pPk9Kl8NW/wFEkLmkqhuVd9dJU0WONF8Qdqd+w?= JBm9PElvylv/LJwFRzjquumKx46j32vndqF/EbdhWmyVHu4/QsrCFHpUtyOXTXJarMArKkW//mnIp5XcjN1ca3u77V+FcKDo32ZKQEXqIQUv9DN+TDGgVcFw5XETN2GLmbSalUljS+RgBte5zVyNLKG8MUH0vQ21fxEX1FoF9hFrkmqHsvyQwPSUAhre+uFaO/+C+BmCRRjeBR+6jGfr42+oAuwVg3Q5PaLvbnhFEwRoIgnbI7idsRNp98tHiebnFVk90LtSFmbSAiW9/m7+Ywh7WcjLelFZoH5EijRmZkHdG4Dd4XaigrQVjAMGBzQwZLzMqU/cG1/ZHmi+OF8v0bkmEgzGnvN6GwQdtiWbUwDVcNTi7owouxBNX15pB9d51U2hkWMzA1enDcga2aOxokDulRJRqg8wWjLizE5DHSs6PQqBzThcP60tbFOX881PvotNNy11/hh8eQdct+SvP0vuOzhB0Fds8bH4/JN2NTtvVnqDSaXIu+AfOz6ctvN6Xho43cWWXsSuJqQtXNHFsvlajZ1LcKAEVhP+Sfiy2DbZoGxUyUZ74JPsVittkosLBcBe4pBdk4VzA8Ruu78mJkvQUvAkZY+1duAuc5Wt73LoxPa+6q0SZO6GoeqJBe8sVdM51XhO2nsC+ztcSNMQaxBDk0xwENxMQNipD84itiPrnz+N+BZSMUb5EY2U0kokS7YzjGL6KKrElizW/biYtT6Km9dpGpbPmqowmK/cZKsuYEwpaC/5BiglmYkOqgPNQAvaPEJcpGcBVO182Qe+B2PndKOweCZEgNa95GpTeTUOzlmrmgqdTU5Lit8CL4eLnBqBUxZxvFcyJYdB+P+34sDZ1kCFA/OFRAocSfXibaG7tYOPnSzE+b5dpXghxG8gbQ7k1EVxr0AK6RCDODDZGSSj36t7wVxyRs4veBDoxA0olf7Vv1MByxZj4mya5IPT/nZvgiJ4SI5e1VkSTx1wufWOotTEmiivSxA1x/phAbq12oAD04pOAJxBVDmKZyejIocrW5EVwpiWKpNuAa4Xv591vkjDC7l447N/X8iKrfFSDKqc3jRq3iHv3VeNqMmBk+ibyAU7cGsH/ewp+H2kwBqPgvjLSABxUEE6wtliH2d/Rgh84J3hJjSmECnuiEHu64/PbL5vt8eoZRR0g5voBUOoi43PNRcDiuD8d/X3klB2w== X-Microsoft-Antispam-Message-Info: umDYa1xs/1groybO7c4M2iFz2U7mKx+qz45U/lyntQ4od7oxpCUa+5NjycSMRuw5+ZHpGfBGhmG8SODSEjOQ86oPOjTYPUGyHmHVYuwkGkYq/g55dNqO87i4kNUbXw7qPqdPXZ8R3ipfnSTT7vJPezdFRt2n7tSzFjpZMc86bSda7rifbsgrizATd17zi9s+odXOP5Lq/4phvlSP7eAXL/1ZULWrOeVxU3pmppSiadOqeF8XdKpH1pOwH8GRuHs8qHEK4x5ttEC6JsCliJxgsDRndx4DKpVBHp+pdc1quZ/W+Dz/IhzrlGugbpbdIKmQ X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB2440; 6:SbU3b6LQa90I1EgINeyj5Jsab6d7cwS1Afvcog2xMswFXgL1TjcVz3jIKxrNNotG5tCFNvv8RBnnWXPfSe36TJR3xqCCdSHXoXvmeVd3VRIJa4eKO+xL4kDvztRxxb35QdEHHLlgBIKBopMOUnpUfM/qvEUjzj6wyBYLyFCMoDcvHLR2hVqjXw6Bea6H2nbaCUYeAJPe/5vMCcd5vDf2iU4pO5p90T1FnGtmR/RiMZ37moQ1meZ+OZtkK8S1OyRGot/MUMK9hN320vW01g+xAXrtf/lz+cD/p89pEBvQXYEsTpE0c/vP9fI05Arrg0o8hYmY9UsIOGUPbyamRrsgGAcJcGvgj0rVOxDnFVrPITjNOX6kEGYCsWNzXx1UXDsR6WAH5c/J+wweD++rE0n3UnXWzgi7bzwlIRAcXrG1RPCXvOGWR3ba305u1Lyy6bclcycxqhXzyJtM5z9z4TEqvQ==; 5:EiGom1+ljnxFxMu3iRyFsF0jZdQZMPYZJ8DXG9x9zDC+u9UjKsi3d6R7eI8k/1lQLAeVRMUuZl2hO2Q5bDMV8uWsrH0+vMcD2pxpnmDHkak5iua7Ah9EOWmUf2nZhbC+cuHa0j2FarB1kSuHdO9TreL4JNhFQMCochUGyUY1p+I=; 7:8k35ynChdtb/fE956CSTxLs4jbmhxXycqH7GM+ooA+2nwGBl2q2m5XJWPWDvkb5dtIAFZa42pJtWkLuqcJMmO4VSRiIT0j8XzEtiGloZVD9kp8Srqne7vhzOxZXWUbOSSTLnhPshqxsLFml6+rfymw== SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; DM5PR12MB2440; 20:b8lwTAZLiWLx7BzxX7Ft+KVbjOv41VQQVJXjb+q0CzmHfTvSrPIfjfWfRnbKpx3C/PIDYThTFiSEQOWVrq2cFUumIAZ0NvHrg5sw0klmSEiLPfTTHMqowcSFma7OHJbVNpW5Ee/YESlLz60KRQAtr5imiUjBOcrj/+/lng8k08KD85hm9Ca+l12epMjUXDBW6fG5TfDeUCkosndPAUdyBZ73a/FPbmk+lmeaqQaSmqo6jL5aSNm5O2G5Mt9GP/N9 X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Dec 2018 19:23:45.3866 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 359ef3c2-fedb-4531-ef10-08d666b0a925 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV02.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB2440 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Monk.Liu@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Decauple sched threads stop and start and ring mirror list handling from the policy of what to do about the guilty jobs. When stoppping the sched thread and detaching sched fences from non signaled HW fenes wait for all signaled HW fences to complete before rerunning the jobs. v2: Fix resubmission of guilty job into HW after refactoring. v4: Full restart for all the jobs, not only from guilty ring. Extract karma increase into standalone function. v5: Rework waiting for signaled jobs without relying on the job struct itself as those might already be freed for non 'guilty' job's schedulers. Expose karma increase to drivers. Suggested-by: Christian Koenig Signed-off-by: Andrey Grodzovsky Signed-off-by: Andrey Grodzovsky Signed-off-by: Andrey Grodzovsky Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 18 +-- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 11 +- drivers/gpu/drm/scheduler/sched_main.c | 188 +++++++++++++++++++---------- drivers/gpu/drm/v3d/v3d_sched.c | 12 +- include/drm/gpu_scheduler.h | 10 +- 5 files changed, 151 insertions(+), 88 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 8a078f4..a4bd2d3 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3298,12 +3298,10 @@ static int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev, if (!ring || !ring->sched.thread) continue; - kthread_park(ring->sched.thread); + drm_sched_stop(&ring->sched, job ? &job->base : NULL); - if (job && job->base.sched != &ring->sched) - continue; - - drm_sched_hw_job_reset(&ring->sched, job ? &job->base : NULL); + if(job) + drm_sched_increase_karma(&job->base); /* after all hw jobs are reset, hw fence is meaningless, so force_completion */ amdgpu_fence_driver_force_completion(ring); @@ -3454,14 +3452,10 @@ static void amdgpu_device_post_asic_reset(struct amdgpu_device *adev, if (!ring || !ring->sched.thread) continue; - /* only need recovery sched of the given job's ring - * or all rings (in the case @job is NULL) - * after above amdgpu_reset accomplished - */ - if ((!job || job->base.sched == &ring->sched) && !adev->asic_reset_res) - drm_sched_job_recovery(&ring->sched); + if (!adev->asic_reset_res) + drm_sched_resubmit_jobs(&ring->sched); - kthread_unpark(ring->sched.thread); + drm_sched_start(&ring->sched, !adev->asic_reset_res); } if (!amdgpu_device_has_dc_support(adev)) { diff --git a/drivers/gpu/drm/etnaviv/etnaviv_sched.c b/drivers/gpu/drm/etnaviv/etnaviv_sched.c index 49a6763..6f1268f 100644 --- a/drivers/gpu/drm/etnaviv/etnaviv_sched.c +++ b/drivers/gpu/drm/etnaviv/etnaviv_sched.c @@ -109,16 +109,19 @@ static void etnaviv_sched_timedout_job(struct drm_sched_job *sched_job) } /* block scheduler */ - kthread_park(gpu->sched.thread); - drm_sched_hw_job_reset(&gpu->sched, sched_job); + drm_sched_stop(&gpu->sched, sched_job); + + if(sched_job) + drm_sched_increase_karma(sched_job); /* get the GPU back into the init state */ etnaviv_core_dump(gpu); etnaviv_gpu_recover_hang(gpu); + drm_sched_resubmit_jobs(&gpu->sched); + /* restart scheduler after GPU is usable again */ - drm_sched_job_recovery(&gpu->sched); - kthread_unpark(gpu->sched.thread); + drm_sched_start(&gpu->sched, true); } static void etnaviv_sched_free_job(struct drm_sched_job *sched_job) diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index dbb6906..b5c5bee 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -60,8 +60,6 @@ static void drm_sched_process_job(struct dma_fence *f, struct dma_fence_cb *cb); -static void drm_sched_expel_job_unlocked(struct drm_sched_job *s_job); - /** * drm_sched_rq_init - initialize a given run queue struct * @@ -335,6 +333,42 @@ static void drm_sched_job_timedout(struct work_struct *work) spin_unlock_irqrestore(&sched->job_list_lock, flags); } +void drm_sched_increase_karma(struct drm_sched_job *bad) +{ + int i; + struct drm_sched_entity *tmp; + struct drm_sched_entity *entity; + struct drm_gpu_scheduler *sched = bad->sched; + + /* don't increase @bad's karma if it's from KERNEL RQ, + * because sometimes GPU hang would cause kernel jobs (like VM updating jobs) + * corrupt but keep in mind that kernel jobs always considered good. + */ + if (bad->s_priority != DRM_SCHED_PRIORITY_KERNEL) { + atomic_inc(&bad->karma); + for (i = DRM_SCHED_PRIORITY_MIN; i < DRM_SCHED_PRIORITY_KERNEL; + i++) { + struct drm_sched_rq *rq = &sched->sched_rq[i]; + + spin_lock(&rq->lock); + list_for_each_entry_safe(entity, tmp, &rq->entities, list) { + if (bad->s_fence->scheduled.context == + entity->fence_context) { + if (atomic_read(&bad->karma) > + bad->sched->hang_limit) + if (entity->guilty) + atomic_set(entity->guilty, 1); + break; + } + } + spin_unlock(&rq->lock); + if (&entity->list != &rq->entities) + break; + } + } +} +EXPORT_SYMBOL(drm_sched_increase_karma); + /** * drm_sched_hw_job_reset - stop the scheduler if it contains the bad job * @@ -342,13 +376,22 @@ static void drm_sched_job_timedout(struct work_struct *work) * @bad: bad scheduler job * */ -void drm_sched_hw_job_reset(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad) +void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad) { - struct drm_sched_job *s_job; - struct drm_sched_entity *entity, *tmp; + struct drm_sched_job *s_job, *last_job; unsigned long flags; - int i; + struct dma_fence *wait_fence = NULL; + int r; + + kthread_park(sched->thread); + /* + * Verify all the signaled jobs in mirror list are removed from the ring + * by waiting for their respective scheduler fences to signal. + * Continually repeat traversing the ring mirror list until no more signaled + * fences are found + */ +retry_wait: spin_lock_irqsave(&sched->job_list_lock, flags); list_for_each_entry_reverse(s_job, &sched->ring_mirror_list, node) { if (s_job->s_fence->parent && @@ -357,35 +400,45 @@ void drm_sched_hw_job_reset(struct drm_gpu_scheduler *sched, struct drm_sched_jo dma_fence_put(s_job->s_fence->parent); s_job->s_fence->parent = NULL; atomic_dec(&sched->hw_rq_count); + } else { + wait_fence = dma_fence_get(&s_job->s_fence->finished); + last_job = s_job; + break; } } - spin_unlock_irqrestore(&sched->job_list_lock, flags); - if (bad && bad->s_priority != DRM_SCHED_PRIORITY_KERNEL) { - atomic_inc(&bad->karma); - /* don't increase @bad's karma if it's from KERNEL RQ, - * becuase sometimes GPU hang would cause kernel jobs (like VM updating jobs) - * corrupt but keep in mind that kernel jobs always considered good. - */ - for (i = DRM_SCHED_PRIORITY_MIN; i < DRM_SCHED_PRIORITY_KERNEL; i++ ) { - struct drm_sched_rq *rq = &sched->sched_rq[i]; + /* No signaled jobs in the ring, its safe to proceed to ASIC reset */ + if (!wait_fence) { + spin_unlock_irqrestore(&sched->job_list_lock, flags); + goto done; + } - spin_lock(&rq->lock); - list_for_each_entry_safe(entity, tmp, &rq->entities, list) { - if (bad->s_fence->scheduled.context == entity->fence_context) { - if (atomic_read(&bad->karma) > bad->sched->hang_limit) - if (entity->guilty) - atomic_set(entity->guilty, 1); - break; - } - } - spin_unlock(&rq->lock); - if (&entity->list != &rq->entities) - break; + /* Restore removed cb since removing again already removed cb is undefined */ + list_for_each_entry_reverse(s_job, &sched->ring_mirror_list, node) { + if(s_job == last_job) + break; + + if (s_job->s_fence->parent) { + r = dma_fence_add_callback(s_job->s_fence->parent, + &s_job->s_fence->cb, + drm_sched_process_job); + if (r) + DRM_ERROR("fence restore callback failed (%d)\n", + r); } } + spin_unlock_irqrestore(&sched->job_list_lock, flags); + + dma_fence_wait(wait_fence, false); + dma_fence_put(wait_fence); + wait_fence = NULL; + + goto retry_wait; + +done: + return; } -EXPORT_SYMBOL(drm_sched_hw_job_reset); +EXPORT_SYMBOL(drm_sched_stop); /** * drm_sched_job_recovery - recover jobs after a reset @@ -393,33 +446,21 @@ EXPORT_SYMBOL(drm_sched_hw_job_reset); * @sched: scheduler instance * */ -void drm_sched_job_recovery(struct drm_gpu_scheduler *sched) +void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery) { struct drm_sched_job *s_job, *tmp; - bool found_guilty = false; unsigned long flags; int r; + if (!full_recovery) + goto unpark; + spin_lock_irqsave(&sched->job_list_lock, flags); list_for_each_entry_safe(s_job, tmp, &sched->ring_mirror_list, node) { struct drm_sched_fence *s_fence = s_job->s_fence; - struct dma_fence *fence; - uint64_t guilty_context; - - if (!found_guilty && atomic_read(&s_job->karma) > sched->hang_limit) { - found_guilty = true; - guilty_context = s_job->s_fence->scheduled.context; - } - - if (found_guilty && s_job->s_fence->scheduled.context == guilty_context) - dma_fence_set_error(&s_fence->finished, -ECANCELED); - - spin_unlock_irqrestore(&sched->job_list_lock, flags); - fence = sched->ops->run_job(s_job); - atomic_inc(&sched->hw_rq_count); + struct dma_fence *fence = s_job->s_fence->parent; if (fence) { - s_fence->parent = dma_fence_get(fence); r = dma_fence_add_callback(fence, &s_fence->cb, drm_sched_process_job); if (r == -ENOENT) @@ -427,18 +468,47 @@ void drm_sched_job_recovery(struct drm_gpu_scheduler *sched) else if (r) DRM_ERROR("fence add callback failed (%d)\n", r); - dma_fence_put(fence); - } else { - if (s_fence->finished.error < 0) - drm_sched_expel_job_unlocked(s_job); + } else drm_sched_process_job(NULL, &s_fence->cb); - } - spin_lock_irqsave(&sched->job_list_lock, flags); } + drm_sched_start_timeout(sched); spin_unlock_irqrestore(&sched->job_list_lock, flags); + +unpark: + kthread_unpark(sched->thread); +} +EXPORT_SYMBOL(drm_sched_start); + +/** + * drm_sched_resubmit_jobs - helper to relunch job from mirror ring list + * + * @sched: scheduler instance + * + */ +void drm_sched_resubmit_jobs(struct drm_gpu_scheduler *sched) +{ + struct drm_sched_job *s_job, *tmp; + uint64_t guilty_context; + bool found_guilty = false; + + /*TODO DO we need spinlock here ? */ + list_for_each_entry_safe(s_job, tmp, &sched->ring_mirror_list, node) { + struct drm_sched_fence *s_fence = s_job->s_fence; + + if (!found_guilty && atomic_read(&s_job->karma) > sched->hang_limit) { + found_guilty = true; + guilty_context = s_job->s_fence->scheduled.context; + } + + if (found_guilty && s_job->s_fence->scheduled.context == guilty_context) + dma_fence_set_error(&s_fence->finished, -ECANCELED); + + s_job->s_fence->parent = sched->ops->run_job(s_job); + atomic_inc(&sched->hw_rq_count); + } } -EXPORT_SYMBOL(drm_sched_job_recovery); +EXPORT_SYMBOL(drm_sched_resubmit_jobs); /** * drm_sched_job_init - init a scheduler job @@ -634,26 +704,14 @@ static int drm_sched_main(void *param) DRM_ERROR("fence add callback failed (%d)\n", r); dma_fence_put(fence); - } else { - if (s_fence->finished.error < 0) - drm_sched_expel_job_unlocked(sched_job); + } else drm_sched_process_job(NULL, &s_fence->cb); - } wake_up(&sched->job_scheduled); } return 0; } -static void drm_sched_expel_job_unlocked(struct drm_sched_job *s_job) -{ - struct drm_gpu_scheduler *sched = s_job->sched; - - spin_lock(&sched->job_list_lock); - list_del_init(&s_job->node); - spin_unlock(&sched->job_list_lock); -} - /** * drm_sched_init - Init a gpu scheduler instance * diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c index 445b2ef..f76d9ed 100644 --- a/drivers/gpu/drm/v3d/v3d_sched.c +++ b/drivers/gpu/drm/v3d/v3d_sched.c @@ -178,18 +178,22 @@ v3d_job_timedout(struct drm_sched_job *sched_job) for (q = 0; q < V3D_MAX_QUEUES; q++) { struct drm_gpu_scheduler *sched = &v3d->queue[q].sched; - kthread_park(sched->thread); - drm_sched_hw_job_reset(sched, (sched_job->sched == sched ? + drm_sched_stop(sched, (sched_job->sched == sched ? sched_job : NULL)); + + if(sched_job) + drm_sched_increase_karma(sched_job); } /* get the GPU back into the init state */ v3d_reset(v3d); + for (q = 0; q < V3D_MAX_QUEUES; q++) + drm_sched_resubmit_jobs(sched_job->sched); + /* Unblock schedulers and restart their jobs. */ for (q = 0; q < V3D_MAX_QUEUES; q++) { - drm_sched_job_recovery(&v3d->queue[q].sched); - kthread_unpark(v3d->queue[q].sched.thread); + drm_sched_start(&v3d->queue[q].sched, true); } mutex_unlock(&v3d->reset_lock); diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index 47e1979..5ab2d97 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -175,6 +175,7 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f); * finished to remove the job from the * @drm_gpu_scheduler.ring_mirror_list. * @node: used to append this struct to the @drm_gpu_scheduler.ring_mirror_list. + * @finish_node: used in a list to wait on before resetting the scheduler * @id: a unique id assigned to each job scheduled on the scheduler. * @karma: increment on every hang caused by this job. If this exceeds the hang * limit of the scheduler then the job is marked guilty and will not @@ -193,6 +194,7 @@ struct drm_sched_job { struct dma_fence_cb finish_cb; struct work_struct finish_work; struct list_head node; + struct list_head finish_node; uint64_t id; atomic_t karma; enum drm_sched_priority s_priority; @@ -298,9 +300,11 @@ int drm_sched_job_init(struct drm_sched_job *job, void *owner); void drm_sched_job_cleanup(struct drm_sched_job *job); void drm_sched_wakeup(struct drm_gpu_scheduler *sched); -void drm_sched_hw_job_reset(struct drm_gpu_scheduler *sched, - struct drm_sched_job *job); -void drm_sched_job_recovery(struct drm_gpu_scheduler *sched); +void drm_sched_stop(struct drm_gpu_scheduler *sched, + struct drm_sched_job *job); +void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery); +void drm_sched_resubmit_jobs(struct drm_gpu_scheduler *sched); +void drm_sched_increase_karma(struct drm_sched_job *bad); bool drm_sched_dependency_optimized(struct dma_fence* fence, struct drm_sched_entity *entity); void drm_sched_fault(struct drm_gpu_scheduler *sched); From patchwork Thu Dec 20 19:23:35 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andrey Grodzovsky X-Patchwork-Id: 10739443 Return-Path: Received: from mail.wl.linuxfoundation.org (pdx-wl-mail.web.codeaurora.org [172.30.200.125]) by pdx-korg-patchwork-2.web.codeaurora.org (Postfix) with ESMTP id B02CD17E1 for ; Thu, 20 Dec 2018 19:23:59 +0000 (UTC) Received: from mail.wl.linuxfoundation.org (localhost [127.0.0.1]) by mail.wl.linuxfoundation.org (Postfix) with ESMTP id 9FB2A271CB for ; Thu, 20 Dec 2018 19:23:59 +0000 (UTC) Received: by mail.wl.linuxfoundation.org (Postfix, from userid 486) id 910A127CEE; Thu, 20 Dec 2018 19:23:59 +0000 (UTC) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on pdx-wl-mail.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-5.2 required=2.0 tests=BAD_ENC_HEADER,BAYES_00, MAILING_LIST_MULTI,RCVD_IN_DNSWL_MED autolearn=ham version=3.3.1 Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher DHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.wl.linuxfoundation.org (Postfix) with ESMTPS id 4179E271CB for ; Thu, 20 Dec 2018 19:23:59 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 3DB9A6F4D8; Thu, 20 Dec 2018 19:23:57 +0000 (UTC) X-Original-To: dri-devel@lists.freedesktop.org Delivered-To: dri-devel@lists.freedesktop.org Received: from NAM01-BN3-obe.outbound.protection.outlook.com (mail-eopbgr740083.outbound.protection.outlook.com [40.107.74.83]) by gabe.freedesktop.org (Postfix) with ESMTPS id A17C26F4D9; Thu, 20 Dec 2018 19:23:55 +0000 (UTC) Received: from DM5PR12CA0001.namprd12.prod.outlook.com (2603:10b6:4:1::11) by MWHPR12MB1326.namprd12.prod.outlook.com (2603:10b6:300:11::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.1446.19; Thu, 20 Dec 2018 19:23:53 +0000 Received: from DM3NAM03FT059.eop-NAM03.prod.protection.outlook.com (2a01:111:f400:7e49::203) by DM5PR12CA0001.outlook.office365.com (2603:10b6:4:1::11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1446.17 via Frontend Transport; Thu, 20 Dec 2018 19:23:52 +0000 Received-SPF: None (protection.outlook.com: amd.com does not designate permitted sender hosts) Received: from SATLEXCHOV02.amd.com (165.204.84.17) by DM3NAM03FT059.mail.protection.outlook.com (10.152.82.207) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.1446.11 via Frontend Transport; Thu, 20 Dec 2018 19:23:52 +0000 Received: from agrodzovsky-All-Series.amd.com (10.34.1.3) by SATLEXCHOV02.amd.com (10.181.40.72) with Microsoft SMTP Server id 14.3.389.1; Thu, 20 Dec 2018 13:23:50 -0600 From: Andrey Grodzovsky To: , , , , Subject: [PATCH v5 2/2] drm/sched: Rework HW fence processing. Date: Thu, 20 Dec 2018 14:23:35 -0500 Message-ID: <1545333815-29870-2-git-send-email-andrey.grodzovsky@amd.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1545333815-29870-1-git-send-email-andrey.grodzovsky@amd.com> References: <1545333815-29870-1-git-send-email-andrey.grodzovsky@amd.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-Office365-Filtering-HT: Tenant X-Forefront-Antispam-Report: CIP:165.204.84.17; IPV:NLI; CTRY:US; EFV:NLI; SFV:NSPM; SFS:(10009020)(346002)(396003)(376002)(136003)(39860400002)(2980300002)(428003)(199004)(189003)(26005)(305945005)(50226002)(8936002)(14444005)(5024004)(77096007)(36756003)(8676002)(6346003)(81156014)(81166006)(48376002)(2906002)(50466002)(2616005)(76176011)(186003)(51416003)(7696005)(72206003)(47776003)(68736007)(5660300001)(446003)(11346002)(86362001)(336012)(6666004)(53416004)(356004)(478600001)(104016004)(476003)(126002)(2201001)(44832011)(54906003)(106466001)(39060400002)(53936002)(486006)(105586002)(426003)(110136005)(4326008)(316002)(97736004)(16586007)(2101003); DIR:OUT; SFP:1101; SCL:1; SRVR:MWHPR12MB1326; H:SATLEXCHOV02.amd.com; FPR:; SPF:None; LANG:en; PTR:InfoDomainNonexistent; A:1; MX:1; X-Microsoft-Exchange-Diagnostics: 1; DM3NAM03FT059; 1:XTaEKpnREoHMFdKxm2wanb2h5zzA3QdT4ZcQHNkSWZkXabrg4b5y7zyqStVYhfAZSB2FN65Ko2tHCBquEYIYoaO/Z2BvW8yxhl+1lkSyjvqkNRVx4AyHnYZc+laPZweP X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: d29c6a43-4d2f-439c-7e1f-08d666b0ad29 X-Microsoft-Antispam: BCL:0; PCL:0; RULEID:(2390118)(7020095)(4652040)(8989299)(5600074)(711020)(4534185)(4627221)(201703031133081)(201702281549075)(8990200)(2017052603328)(7153060); SRVR:MWHPR12MB1326; X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1326; 3:48oUpcChJ/8VXhgSLqGINOEVVtz5WYImY0Tuwxn/GATkvL6tMFFbYdpZPD6E30M8FYTqxa3/t9MzC2YL/nNExW0JUc3D5rcMXNiwW/Rj7UfJetv9v2y37Hl+g+/ZKO249qapyixhrT8fwcofa6aanWnnnCs5m+aXmpUw44HIebB7EWN1mmQTgCO8nuIeEWb3tpHw4/3jrnMC7Uky6I4cENs06GMEBc6HEAtT90ugwhdUO9W2IBRHJgbEDT6WgY7+d61qOoafuNDymX1uEOklstHDsVFtjv4OHA6Qa0ANNFAfSRpXg/dRwZkdQHq2P4VTERdJS35RI4ECx5Uywv+HiL1n3DxHa6kCnukcWhXc4+w=; 25:wmOMjWeH2KU/Qb5Zai5uy1aMV1iKYPE8V6ysCHU9XB/cfP1eTuHVTmdgwfsOPNVWPUwZOWjbz5+tcSHmvbGQlXFjje1lWGYZryICvWrzZyaDp+ldnzgyotMH7UTlWMB4pkVnZuVo5lM+jDKCNPengdJoSJFK6nooAzqtgZ1pMfklYK4iOGMoO4o7n7pIH0jM6LFizmRLUO2XqumWYH3M5ub5lD6RQOR8Jnfi4cyw37L5FB6YqIQNtyuEw7CzstagLB7H5nqZ0Xv/QmLK/mlPqrhMe7sWgBLne/qaumA/2ogj1ysvY15xuyPQSZHL97oKouphBpKBdMN6SzT4I8NKEw== X-MS-TrafficTypeDiagnostic: MWHPR12MB1326: X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1326; 31:u5H9MsIlS6NU1w6LOZoM0VrbEiioRXW1RuNExHHy3fCGfnBOSvjAqYcJvQy18x5qG0q4Xu6AIadOJad1tZm5gClB8q7UywgVGMt12qefikC+PD5kG8EQXYOSW09u6xwVSrP83U2pwGhzJ6Jk+8yc/eTpXPZASqdFsZCy/EP7mQN5XYSpS4KHzeQaAXX44+U2waTFRW2ptrjIPq5tY2vJf/Rb5DzEL1VYghih9O8e4o8=; 20:9LjRzxLdeUtaVOFDr+pULRx41+bfzuAzTJrI/Ecy0Ok1VAgWVw7s8E2+QPBLfbgvEEZRaWObBHmbZrP0lMeNfu1o6nO9xatjVEwG6N+CjoWo/FF6ENCzIWD4POHJ2Bhgh9SmS2rClxmrEj9/Ji/qtRBcgyEE23L+sGJpdE5H3U8EtWxiqVk1Udd+14+fliwD2Sa5Qz91luJIKE8++8U1y7xZObu6nU96BzHlk5vh2rWBFIpnIiNXTToQQGYwOo8vA6H6gzt8INU0l1XTpV46OOhSqzNkbAgQK2tpPafiEj9/Lw4nftUEfSTbAn/XTsSsQyIVgotJkKElVmKHl6c/Asq4g2g6e2D+hH9IlIG89bkcFvTERsDz/67nbYdl8GM29XFMvWOCtLLPFkB/suEnjtvWNdCA0Fl7ZBT7zdyCtIqS2eeC7WKwbrRPyaG1pNj0P0aAqv4w2CrYTtn62d2ZbWFZ4t6HGLXOnrCz3pRxmUELJo73d2/bl0/0+kaMELHI X-Microsoft-Antispam-PRVS: X-MS-Exchange-SenderADCheck: 1 X-Exchange-Antispam-Report-CFA-Test: BCL:0; PCL:0; RULEID:(8211001083)(3230021)(999002)(5005026)(6040522)(2401047)(8121501046)(3231475)(944501520)(52105112)(93006095)(93003095)(10201501046)(3002001)(6055026)(149066)(150057)(6041310)(20161123558120)(20161123560045)(20161123562045)(20161123564045)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(201708071742011)(7699051)(76991095); SRVR:MWHPR12MB1326; BCL:0; PCL:0; RULEID:; SRVR:MWHPR12MB1326; X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1326; 4:6Ns0kKzcsnL1l7+JHy3n6CeRYGZBeqtwOI0ixvHx+Kl0YOkr9GCpqfXrj+0mCKVsxRuXUhHhLLJRenw7EK5Ylaxz6J8MSIvXUCeHmKMWSUNdK2NxTR4MfHxNalvmwD5yf2J716mramsKSPhAQp78+bqSvuZrDOTIcatKtTwn/cPaUC/17h3wPAw3FpCvvMzdnFqLB2YSz9FVyW80jNDukqxHUPPeIfhxz59uF2oLLqA6AUB9RqaBmSNoGBcfoRKZ/qxpkQrZZ+ypdFzM+K1HAw== X-Forefront-PRVS: 0892FA9A88 X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1; MWHPR12MB1326; 23:g39UX/omkbDfc/WsfnN2lHTDGaXOSVQ2xkIl1ksGR?= 1tzZSJXvq2fWynjhxyX5z7Yq0oo4b5iXqesr4V6UQk54tTDSAeogk1z7+Q6Bo9X5krxMMhWo9pI27nxxDygOA0p2ehr7sG5J4Sov5fPMT8tmErpI7iV2KdpiN3TyM45/iKMuYWSmIFLQB3KiHDVo/N0SRMUH4ZaEsBDYzHlBFYqsmuI5QXJidRQPK5Qg90twbyYQXHw0EKlQVUJU2mHWD0ZR0WosKiHVRj4OAiZqan8AqJmDSrPIvPSqsbtEVRKmExe86VTtAZXo4GvTCc8QYpe/nhK1OlvzHwY76gdHAH7kEHwtYNEvl6RuoWjom192x6NU2xr42ghAI7JD7scLPKOlb/IZFtdApC0lIqDHiZ5/fQL0VMYgt9qKW+0BPQ/rpougVlzBKP2KXKi5mrSAbZgD6hR7Mb9nH5CfyeHrrmcVgDNDCCFtCzNC4IIoJa9pqD8ZCLfHJeJOoMnoWKX7vy0TBNUec9oQP9hzifOpItVpkvdVBQ+YdG738gSOnzWegywNgr3MH94mdS34sitO/GITi3+GSQsNJwvPkuizO80BMfXUeLb7vca1wRl+UzGv/HQ6KMvELVC9xkFCpSlgNVq1BG0G+/aIOfkhZZTKuHCNFSg6io13lqfs4kY0SW3xZUmlozAlTsDRxUh3huKP36m7TigqOC6WEaFBQk8DY1L9nAFGqRk5Ue6DAoHzDOCzeRf9uj14BEpQfFox42CjcJr7c+WIxFdWUrfyNdRziI3+aAUMtTNJjokSf/Sprt2WQ6KhuTie4EdcrGcgOKq8x6r6YDgvGIAcJqREGA+08P/xZ5Jz4+fONVvF9cjVubGM00vsNTgSKT/CTdsBVnRa6fo5Mxyo9nUtoSXhj4Q1z0vAFiDNLdn1bGs2P6weSxvDs82Ep2qtuMIUBLde58WaHC2Q5vKtFHkE8HyQ1BivR5lVKnStTBMeFNXxrcuKGlARi93pv71yh7NBOy5FsjDAGMTNBaMo4OUp+xOcd1a3xGiR5UoCkCjHcjh9gYUAhcrwf1pvuffWeYz9To2oLF5N/SnoQdZW+3DvGxBvHys0kDlAnfGTezvjLBmRHUvaqGLHTIWDUSR7VoXxRzExpHssQkcejIKmau1lEeGwE8CkM8y7To+Zr0u7Gb0GlTO+NyrTIVFdUw2n/2G+jCF4FAkjW+lVHVo8DLLlD0osrUSkpuM1MpWUSqvzugGTWOd0Aqw+zlYExCCjJW0w6rx1QERCuIY++DLd1mCvGQKB4rQ0blN+A== X-Microsoft-Antispam-Message-Info: 5bfnPNo8rhf24mW16sH2fAHCRiXao04Mtr5XUvWXFeUpBts6qDqASMxjN2gw0TfBjErUhSgPyu5YarswikdbdMboRKCK5C0Go5BMYESQBFunqZoeJ0+uq3kSh5CemAwKoQ+n5oYT2w14rkGm8QiBc9f4OadxSOxsk1TOWTQa961mWTK882ftPU4n/BF/39q4GWyBEFi+Nj/ERgltNEuhEkCSR3O3sBFSARMmLfnvWALKoQciParCf4DvwKANxkz6TNM89F/wdS0UtD0oqOcV9ntmpH2yVWDbsm0wWu6GB1QkweRO1xRUThfgs5TIywWJ X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1326; 6:ZAdj+Q/nMIuGgLPDkfeQRF7YmBFHZdEbORu2t9ZzWGgfv5nYSgyl861WGAhXcPNFRK6aGWtbnjlNMZ4cSaSY1e1jFPzZUiVt++YZYtXOd7nee64id5Eo83RwLAtue1F+mR3QqJKQm3TagGY8qgsXEWxTt5R028Fnjj6OzqwfWh8x/PBssZXi9sKdfqYTDc00ErvDSnNUrXC6ToDTEj+ZMHEOM6sdmnXBAZ1VOUqejUmpC8fu8PpFn0THSJYPoHpvbv07kznf4xsW1rp4eJmXmY3lxPs63kXXJdaNaujW0Lfa4OpUy3y12P00Xv32VHs+xmuu8KuONF6qlN9tdHsXXSZalIedPMlHHZXa+YWGICz+Qbk5EbH6/+wdyWTJwMFtYCsmGwh43Hu7J5Vj1xqXbe+dIbkJzReY1l5PLX48Iw4VqzVAJztrHl8F0JF9HFyLBQa9i6WuUnLLkNXWk0xtQA==; 5:95gOCMo7Su7hXHiufKg/MS5Sy2xWHNHxKdaK4P4oGK5ky3THZ5fWlT0CV+WXEGcQUgWyS5yM907uu9NsyATH1nPwebegn3AR8P0RdOzWfHqoysboBdfk0xU1FhasDUCKuiXRn9JAoaAj542t8V1CYa5hXm+pYhEzhMlyQyieUdI=; 7:vmsXz+vZWVc4bjpDwXoL4SEXdECpt4fvBAN0Qrexng/rVSqHqCamtYsVzB0bdT5jO4djXNBH9dwq/j/EN80pxtKpbZ7hBLYsh4HYK55e11wilte5TOdMEE9t/NGpk+bDzo+W28DzqQdzkGefKapQQQ== SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1; MWHPR12MB1326; 20:y8Rnm16F/g0WrgsEg+G4WzYQ2G+0coHAGkhPDY9zmQ/0prwN/ri63yFu47gcoRlKwrC7q6KXwemTwG3en6q7cOGXx2UrrbTuLT3cyMesul2til4Kbv3Br6LqSP4SAyf+yRAkWSEizlPzOP9eDhHWwqVrSJuaXUgNaTwkwJWyxeSGRb32gshYxx0G884hFeK2q+b/ZeRFQ6LuiYmi/b6l4wx75mLS62P5Jx4FYbge6PsrhJVTnuZynd0VQwZ7x5pL X-OriginatorOrg: amd.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Dec 2018 19:23:52.1677 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: d29c6a43-4d2f-439c-7e1f-08d666b0ad29 X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d; Ip=[165.204.84.17]; Helo=[SATLEXCHOV02.amd.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR12MB1326 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Monk.Liu@amd.com Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" X-Virus-Scanned: ClamAV using ClamSMTP Expedite job deletion from ring mirror list to the HW fence signal callback instead from finish_work, together with waiting for all such fences to signal in drm_sched_stop we garantee that already signaled job will not be processed twice. Remove the sched finish fence callback and just submit finish_work directly from the HW fence callback. v2: Fix comments. v3: Attach hw fence cb to sched_job v5: Rebase Suggested-by: Christian Koenig Signed-off-by: Andrey Grodzovsky --- drivers/gpu/drm/scheduler/sched_main.c | 57 +++++++++++++++++----------------- include/drm/gpu_scheduler.h | 6 ++-- 2 files changed, 30 insertions(+), 33 deletions(-) diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c index b5c5bee..5f5b187 100644 --- a/drivers/gpu/drm/scheduler/sched_main.c +++ b/drivers/gpu/drm/scheduler/sched_main.c @@ -284,8 +284,6 @@ static void drm_sched_job_finish(struct work_struct *work) cancel_delayed_work_sync(&sched->work_tdr); spin_lock_irqsave(&sched->job_list_lock, flags); - /* remove job from ring_mirror_list */ - list_del_init(&s_job->node); /* queue TDR for next job */ drm_sched_start_timeout(sched); spin_unlock_irqrestore(&sched->job_list_lock, flags); @@ -293,22 +291,11 @@ static void drm_sched_job_finish(struct work_struct *work) sched->ops->free_job(s_job); } -static void drm_sched_job_finish_cb(struct dma_fence *f, - struct dma_fence_cb *cb) -{ - struct drm_sched_job *job = container_of(cb, struct drm_sched_job, - finish_cb); - schedule_work(&job->finish_work); -} - static void drm_sched_job_begin(struct drm_sched_job *s_job) { struct drm_gpu_scheduler *sched = s_job->sched; unsigned long flags; - dma_fence_add_callback(&s_job->s_fence->finished, &s_job->finish_cb, - drm_sched_job_finish_cb); - spin_lock_irqsave(&sched->job_list_lock, flags); list_add_tail(&s_job->node, &sched->ring_mirror_list); drm_sched_start_timeout(sched); @@ -396,7 +383,7 @@ void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad) list_for_each_entry_reverse(s_job, &sched->ring_mirror_list, node) { if (s_job->s_fence->parent && dma_fence_remove_callback(s_job->s_fence->parent, - &s_job->s_fence->cb)) { + &s_job->cb)) { dma_fence_put(s_job->s_fence->parent); s_job->s_fence->parent = NULL; atomic_dec(&sched->hw_rq_count); @@ -420,7 +407,7 @@ void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad) if (s_job->s_fence->parent) { r = dma_fence_add_callback(s_job->s_fence->parent, - &s_job->s_fence->cb, + &s_job->cb, drm_sched_process_job); if (r) DRM_ERROR("fence restore callback failed (%d)\n", @@ -449,31 +436,34 @@ EXPORT_SYMBOL(drm_sched_stop); void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery) { struct drm_sched_job *s_job, *tmp; - unsigned long flags; int r; if (!full_recovery) goto unpark; - spin_lock_irqsave(&sched->job_list_lock, flags); + /* + * Locking the list is not required here as the sched thread is parked + * so no new jobs are being pushed in to HW and in drm_sched_stop we + * flushed all the jobs who were still in mirror list but who already + * signaled and removed them self from the list. Also concurrent + * GPU recovers can't run in parallel. + */ list_for_each_entry_safe(s_job, tmp, &sched->ring_mirror_list, node) { - struct drm_sched_fence *s_fence = s_job->s_fence; struct dma_fence *fence = s_job->s_fence->parent; if (fence) { - r = dma_fence_add_callback(fence, &s_fence->cb, + r = dma_fence_add_callback(fence, &s_job->cb, drm_sched_process_job); if (r == -ENOENT) - drm_sched_process_job(fence, &s_fence->cb); + drm_sched_process_job(fence, &s_job->cb); else if (r) DRM_ERROR("fence add callback failed (%d)\n", r); } else - drm_sched_process_job(NULL, &s_fence->cb); + drm_sched_process_job(NULL, &s_job->cb); } drm_sched_start_timeout(sched); - spin_unlock_irqrestore(&sched->job_list_lock, flags); unpark: kthread_unpark(sched->thread); @@ -622,18 +612,27 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched) */ static void drm_sched_process_job(struct dma_fence *f, struct dma_fence_cb *cb) { - struct drm_sched_fence *s_fence = - container_of(cb, struct drm_sched_fence, cb); + struct drm_sched_job *s_job = container_of(cb, struct drm_sched_job, cb); + struct drm_sched_fence *s_fence = s_job->s_fence; struct drm_gpu_scheduler *sched = s_fence->sched; + unsigned long flags; + + cancel_delayed_work(&sched->work_tdr); - dma_fence_get(&s_fence->finished); atomic_dec(&sched->hw_rq_count); atomic_dec(&sched->num_jobs); + + spin_lock_irqsave(&sched->job_list_lock, flags); + /* remove job from ring_mirror_list */ + list_del_init(&s_job->node); + spin_unlock_irqrestore(&sched->job_list_lock, flags); + drm_sched_fence_finished(s_fence); trace_drm_sched_process_job(s_fence); - dma_fence_put(&s_fence->finished); wake_up_interruptible(&sched->wake_up_worker); + + schedule_work(&s_job->finish_work); } /** @@ -696,16 +695,16 @@ static int drm_sched_main(void *param) if (fence) { s_fence->parent = dma_fence_get(fence); - r = dma_fence_add_callback(fence, &s_fence->cb, + r = dma_fence_add_callback(fence, &sched_job->cb, drm_sched_process_job); if (r == -ENOENT) - drm_sched_process_job(fence, &s_fence->cb); + drm_sched_process_job(fence, &sched_job->cb); else if (r) DRM_ERROR("fence add callback failed (%d)\n", r); dma_fence_put(fence); } else - drm_sched_process_job(NULL, &s_fence->cb); + drm_sched_process_job(NULL, &sched_job->cb); wake_up(&sched->job_scheduled); } diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h index 5ab2d97..6621f74 100644 --- a/include/drm/gpu_scheduler.h +++ b/include/drm/gpu_scheduler.h @@ -138,10 +138,6 @@ struct drm_sched_fence { struct dma_fence finished; /** - * @cb: the callback for the parent fence below. - */ - struct dma_fence_cb cb; - /** * @parent: the fence returned by &drm_sched_backend_ops.run_job * when scheduling the job on hardware. We signal the * &drm_sched_fence.finished fence once parent is signalled. @@ -182,6 +178,7 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f); * be scheduled further. * @s_priority: the priority of the job. * @entity: the entity to which this job belongs. + * @cb: the callback for the parent fence in s_fence. * * A job is created by the driver using drm_sched_job_init(), and * should call drm_sched_entity_push_job() once it wants the scheduler @@ -199,6 +196,7 @@ struct drm_sched_job { atomic_t karma; enum drm_sched_priority s_priority; struct drm_sched_entity *entity; + struct dma_fence_cb cb; }; static inline bool drm_sched_invalidate_job(struct drm_sched_job *s_job,