From patchwork Fri Nov 25 10:21:29 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 13055720 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A3422C4332F for ; Fri, 25 Nov 2022 10:22:04 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id BF9AE10E693; Fri, 25 Nov 2022 10:21:49 +0000 (UTC) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by gabe.freedesktop.org (Postfix) with ESMTPS id E68A510E607; Fri, 25 Nov 2022 10:21:40 +0000 (UTC) Received: by mail-wr1-x429.google.com with SMTP id n7so6034815wrr.13; Fri, 25 Nov 2022 02:21:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=Ew8M0PkxgyobeEwJ5uJ+RaPpgEkLlL9vZ8bbzJQpK5g=; b=X6xsPsWINWOSAfY9hPpBorv7Z0WCK8r5TrALjl4x9wyjMeN1+VVmmtPo3vU0f5I4wK jdVorSyh/AWuallSiiEOwxxwhcyI4ojYSFbIpcjg261sH3oasNhmUjCwfzUCwj6ebEbQ vISeT83wmwsyrFyMrWXqslevhIUsI+CUGJyTOOLfhLgV00dzUlHJh1Gpc75LFcp71cql aXBVJi2/PJFhe5Ms92B2CFi1eh1N5/VnrYQEPlxRdZlaxKaPSRdEVLBa5cC3OJqX80II IrAfwMvv7hKYFM8hK99AOrRZPtEjAmSguRINf/lTAVTNDn7kx559UzGTVaMldLTRJuOp +M+g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Ew8M0PkxgyobeEwJ5uJ+RaPpgEkLlL9vZ8bbzJQpK5g=; b=WH+0mZVSdRq14Lje4VJLzxSGjtR/z03eyJvCxGvrDRHoC4Lyo7e5DEfm6rffHlua+c Ve1/jO24s5JsCdq56QAeiSjDqOa3Qbuc3inDkKOCSvgqMxahQnB6xbbojt4roensMfEQ j487n+MgDrWa4y8IfQ2QxNgdyYQLPtaSFpVN/m9wgN+7RJ0sD7rX6s1X+c728bXynkdz r9HG1WhfKERyY7rXkKAeh0tqC2MqxJt27GtkVAQ9EPKCNfqvS4ZbyGmxy9HXJvEi5s5K +MVG6/wcJRC135TBQhQp5Z729kFmLRHRZF8j4uf05dAaOI8mTvQL3j0ro84PYSLaUnrK 0nNQ== X-Gm-Message-State: ANoB5pmOazNpXC6Uurd5ADC3UZ8X7aX84J7N3xM7zHIxF8fj7xYBiWkl ROQ3CYQsKb+u8h7RF3hkZQgioEZjj6Q= X-Google-Smtp-Source: AA0mqf6g1+4ACRi0mg/NkSnOj4jRe/IFu2qMW4OULjNf0AjDL4JFFzD0XfinhDL8/d7XYd20hJ0Tcw== X-Received: by 2002:a5d:620e:0:b0:241:e32f:8371 with SMTP id y14-20020a5d620e000000b00241e32f8371mr9827636wru.156.1669371699275; Fri, 25 Nov 2022 02:21:39 -0800 (PST) Received: from able.fritz.box (p5b0ea229.dip0.t-ipconnect.de. [91.14.162.41]) by smtp.gmail.com with ESMTPSA id u2-20020a056000038200b00241cfe6e286sm3351193wrf.98.2022.11.25.02.21.38 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Nov 2022 02:21:38 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: Intel Graphics Development , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 1/9] drm/amdgpu: generally allow over-commit during BO allocation Date: Fri, 25 Nov 2022 11:21:29 +0100 Message-Id: <20221125102137.1801-1-christian.koenig@amd.com> X-Mailer: git-send-email 2.34.1 MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" We already fallback to a dummy BO with no backing store when we allocate GDS,GWS and OA resources and to GTT when we allocate VRAM. Drop all those workarounds and generalize this for GTT as well. This fixes ENOMEM issues with runaway applications which try to allocate/free GTT in a loop and are otherwise only limited by the CPU speed. The CS will wait for the cleanup of freed up BOs to satisfy the various domain specific limits and so effectively throttle those buggy applications down to a sane allocation behavior again. Signed-off-by: Christian König Reviewed-by: Alex Deucher Reviewed-by: Arunpravin Paneer Selvam --- drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 16 +++------------- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 6 +----- 2 files changed, 4 insertions(+), 18 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index a0780a4e3e61..62e98f1ad770 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -113,7 +113,7 @@ int amdgpu_gem_object_create(struct amdgpu_device *adev, unsigned long size, bp.resv = resv; bp.preferred_domain = initial_domain; bp.flags = flags; - bp.domain = initial_domain; + bp.domain = initial_domain | AMDGPU_GEM_DOMAIN_CPU; bp.bo_ptr_size = sizeof(struct amdgpu_bo); r = amdgpu_bo_create_user(adev, &bp, &ubo); @@ -332,20 +332,10 @@ int amdgpu_gem_create_ioctl(struct drm_device *dev, void *data, } initial_domain = (u32)(0xffffffff & args->in.domains); -retry: r = amdgpu_gem_object_create(adev, size, args->in.alignment, - initial_domain, - flags, ttm_bo_type_device, resv, &gobj); + initial_domain, flags, ttm_bo_type_device, + resv, &gobj); if (r && r != -ERESTARTSYS) { - if (flags & AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED) { - flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED; - goto retry; - } - - if (initial_domain == AMDGPU_GEM_DOMAIN_VRAM) { - initial_domain |= AMDGPU_GEM_DOMAIN_GTT; - goto retry; - } DRM_DEBUG("Failed to allocate GEM object (%llu, %d, %llu, %d)\n", size, initial_domain, args->in.alignment, r); } diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c index 974e85d8b6cc..919bbea2e3ac 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c @@ -581,11 +581,7 @@ int amdgpu_bo_create(struct amdgpu_device *adev, bo->flags |= AMDGPU_GEM_CREATE_VRAM_WIPE_ON_RELEASE; bo->tbo.bdev = &adev->mman.bdev; - if (bp->domain & (AMDGPU_GEM_DOMAIN_GWS | AMDGPU_GEM_DOMAIN_OA | - AMDGPU_GEM_DOMAIN_GDS)) - amdgpu_bo_placement_from_domain(bo, AMDGPU_GEM_DOMAIN_CPU); - else - amdgpu_bo_placement_from_domain(bo, bp->domain); + amdgpu_bo_placement_from_domain(bo, bp->domain); if (bp->type == ttm_bo_type_kernel) bo->tbo.priority = 1; From patchwork Fri Nov 25 10:21:30 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 13055719 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 404BEC4332F for ; Fri, 25 Nov 2022 10:21:53 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1FBAB10E61F; Fri, 25 Nov 2022 10:21:48 +0000 (UTC) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by gabe.freedesktop.org (Postfix) with ESMTPS id B173810E60A; Fri, 25 Nov 2022 10:21:41 +0000 (UTC) Received: by mail-wr1-x42b.google.com with SMTP id d1so6041538wrs.12; Fri, 25 Nov 2022 02:21:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=Y9Lgo8n5sdKm8tWhlkpLlkTZnjMu0xGNkJqCOeSsc58=; b=ISXOjzd3FKQ2PWEb2IJBc/2zp1K923cZNhHoR/1SD/z/3+PCX1w7uTRohN0BNBXsHP w7ABuAHY4C1LTmGuLQ5W8CBMeU8506Xw/OU76nyn3OG5LJPqEyEbWmk1Lr8njrQyMsCK C/0AaOAFAW523mFR/d2F/hQPsV8ZxxjiTYZ+zzJsJtJa/jj0YIgL+Qyje3c5A408fC7S E+v9FjOK4F48yk6P9fcSO+WaSbMbW+NgevjEBqP30thO0FMrX0KyHCPBDdyJ2ei0wNaw p/BGJSlVs8NU5GkLxlNJFdJFBuDtMdXaHP0YFP+EDrWFm+VFMeNdq7ygrzGJk/4efT1E WP5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Y9Lgo8n5sdKm8tWhlkpLlkTZnjMu0xGNkJqCOeSsc58=; b=ieQecuXUZMBaRfI3E0STtT56fhLfvTLatze6UB6xS2TzmZ0FfrfDEIJF3kEb7T1Lp3 FYodu1Sap7xEwUHgxAVaPyVUNzsfo/lTQu2/L7jD2Q6yy66A8WwgFUB0w+hXhVSWSREa lb8PZ7S1Q1Fc6QOCL9tjcYfaG9+9wi+VF3ypZ4hcrk8LT8r4+9hLMWpQkAR++IN5qH1G okzDpmCmnKbF8apDASPIFsE+Y07M0FO0wIUd7XBRX7yYH+JlP8yE2WBW/fCkfKi6dB5u CdZOWApRhZ2BtAAk+ti62WAzp8fuHJRCgI8ozBUrvKnj6zCFUg1UqLpsEvnJFxrZ2stG 3EMQ== X-Gm-Message-State: ANoB5pmyANOQxAXMP6RHuBnKQ5IAOJKaUfYRdzM3AaH0zkgPO6V2QjxP yB1WHEUy6wS2wbsVCRX/WsqYiJjWrvk= X-Google-Smtp-Source: AA0mqf4e4M8SzO4WD5xmTrfJ5Pu3IX9cRC6GcbAwnhWuya7MV+uZp2L6Y8rUwHJKin/0GOh71WYuHQ== X-Received: by 2002:adf:fb12:0:b0:236:60e8:3cca with SMTP id c18-20020adffb12000000b0023660e83ccamr16436896wrr.471.1669371700201; Fri, 25 Nov 2022 02:21:40 -0800 (PST) Received: from able.fritz.box (p5b0ea229.dip0.t-ipconnect.de. [91.14.162.41]) by smtp.gmail.com with ESMTPSA id u2-20020a056000038200b00241cfe6e286sm3351193wrf.98.2022.11.25.02.21.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Nov 2022 02:21:39 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: Intel Graphics Development , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 2/9] drm/ttm: remove ttm_bo_(un)lock_delayed_workqueue Date: Fri, 25 Nov 2022 11:21:30 +0100 Message-Id: <20221125102137.1801-2-christian.koenig@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221125102137.1801-1-christian.koenig@amd.com> References: <20221125102137.1801-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Those functions never worked correctly since it is still perfectly possible that a buffer object is released and the background worker restarted even after calling them. Signed-off-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c | 6 +----- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 4 +--- drivers/gpu/drm/radeon/radeon_device.c | 5 ----- drivers/gpu/drm/radeon/radeon_pm.c | 4 +--- drivers/gpu/drm/ttm/ttm_bo.c | 14 -------------- include/drm/ttm/ttm_bo_api.h | 16 ---------------- 6 files changed, 3 insertions(+), 46 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c index 0f16d3c09309..f60753f97ac5 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c @@ -1717,7 +1717,7 @@ static void amdgpu_ib_preempt_mark_partial_job(struct amdgpu_ring *ring) static int amdgpu_debugfs_ib_preempt(void *data, u64 val) { - int r, resched, length; + int r, length; struct amdgpu_ring *ring; struct dma_fence **fences = NULL; struct amdgpu_device *adev = (struct amdgpu_device *)data; @@ -1747,8 +1747,6 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) /* stop the scheduler */ kthread_park(ring->sched.thread); - resched = ttm_bo_lock_delayed_workqueue(&adev->mman.bdev); - /* preempt the IB */ r = amdgpu_ring_preempt_ib(ring); if (r) { @@ -1785,8 +1783,6 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) up_read(&adev->reset_domain->sem); - ttm_bo_unlock_delayed_workqueue(&adev->mman.bdev, resched); - pro_end: kfree(fences); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index b2b1c66bfe39..2b1db37e25c1 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3983,10 +3983,8 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) } amdgpu_fence_driver_hw_fini(adev); - if (adev->mman.initialized) { + if (adev->mman.initialized) flush_delayed_work(&adev->mman.bdev.wq); - ttm_bo_lock_delayed_workqueue(&adev->mman.bdev); - } if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c index 6344454a7721..9a556f505685 100644 --- a/drivers/gpu/drm/radeon/radeon_device.c +++ b/drivers/gpu/drm/radeon/radeon_device.c @@ -1772,7 +1772,6 @@ int radeon_gpu_reset(struct radeon_device *rdev) bool saved = false; int i, r; - int resched; down_write(&rdev->exclusive_lock); @@ -1784,8 +1783,6 @@ int radeon_gpu_reset(struct radeon_device *rdev) atomic_inc(&rdev->gpu_reset_counter); radeon_save_bios_scratch_regs(rdev); - /* block TTM */ - resched = ttm_bo_lock_delayed_workqueue(&rdev->mman.bdev); radeon_suspend(rdev); radeon_hpd_fini(rdev); @@ -1844,8 +1841,6 @@ int radeon_gpu_reset(struct radeon_device *rdev) /* reset hpd state */ radeon_hpd_init(rdev); - ttm_bo_unlock_delayed_workqueue(&rdev->mman.bdev, resched); - rdev->in_reset = true; rdev->needs_reset = false; diff --git a/drivers/gpu/drm/radeon/radeon_pm.c b/drivers/gpu/drm/radeon/radeon_pm.c index 04c693ca419a..cbc554928bcc 100644 --- a/drivers/gpu/drm/radeon/radeon_pm.c +++ b/drivers/gpu/drm/radeon/radeon_pm.c @@ -1853,11 +1853,10 @@ static bool radeon_pm_debug_check_in_vbl(struct radeon_device *rdev, bool finish static void radeon_dynpm_idle_work_handler(struct work_struct *work) { struct radeon_device *rdev; - int resched; + rdev = container_of(work, struct radeon_device, pm.dynpm_idle_work.work); - resched = ttm_bo_lock_delayed_workqueue(&rdev->mman.bdev); mutex_lock(&rdev->pm.mutex); if (rdev->pm.dynpm_state == DYNPM_STATE_ACTIVE) { int not_processed = 0; @@ -1908,7 +1907,6 @@ static void radeon_dynpm_idle_work_handler(struct work_struct *work) msecs_to_jiffies(RADEON_IDLE_LOOP_MS)); } mutex_unlock(&rdev->pm.mutex); - ttm_bo_unlock_delayed_workqueue(&rdev->mman.bdev, resched); } /* diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index c3f4b33136e5..b77262a623e0 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -418,20 +418,6 @@ void ttm_bo_put(struct ttm_buffer_object *bo) } EXPORT_SYMBOL(ttm_bo_put); -int ttm_bo_lock_delayed_workqueue(struct ttm_device *bdev) -{ - return cancel_delayed_work_sync(&bdev->wq); -} -EXPORT_SYMBOL(ttm_bo_lock_delayed_workqueue); - -void ttm_bo_unlock_delayed_workqueue(struct ttm_device *bdev, int resched) -{ - if (resched) - schedule_delayed_work(&bdev->wq, - ((HZ / 100) < 1) ? 1 : HZ / 100); -} -EXPORT_SYMBOL(ttm_bo_unlock_delayed_workqueue); - static int ttm_bo_bounce_temp_buffer(struct ttm_buffer_object *bo, struct ttm_resource **mem, struct ttm_operation_ctx *ctx, diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 44a538ee5e2a..7758347c461c 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -290,22 +290,6 @@ void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo); void ttm_bo_set_bulk_move(struct ttm_buffer_object *bo, struct ttm_lru_bulk_move *bulk); -/** - * ttm_bo_lock_delayed_workqueue - * - * Prevent the delayed workqueue from running. - * Returns - * True if the workqueue was queued at the time - */ -int ttm_bo_lock_delayed_workqueue(struct ttm_device *bdev); - -/** - * ttm_bo_unlock_delayed_workqueue - * - * Allows the delayed workqueue to run. - */ -void ttm_bo_unlock_delayed_workqueue(struct ttm_device *bdev, int resched); - /** * ttm_bo_eviction_valuable * From patchwork Fri Nov 25 10:21:31 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 13055721 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 5EA11C4321E for ; Fri, 25 Nov 2022 10:22:17 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6CCA510E610; Fri, 25 Nov 2022 10:21:47 +0000 (UTC) Received: from mail-wm1-x331.google.com (mail-wm1-x331.google.com [IPv6:2a00:1450:4864:20::331]) by gabe.freedesktop.org (Postfix) with ESMTPS id B8EE710E607; Fri, 25 Nov 2022 10:21:42 +0000 (UTC) Received: by mail-wm1-x331.google.com with SMTP id j5-20020a05600c410500b003cfa9c0ea76so3055733wmi.3; Fri, 25 Nov 2022 02:21:42 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=fIM1ll25GubEqhBH5bZZxcPqlVg9s6cvqfKXlkBrrPY=; b=qfayAvZ9W57gec8azJ14/0eQN0OTgL1CiUUfl4vLA9lE9b5f3GzriSiWkBRPdwkTr8 apXcb5tDNEGlvRffxwNcUjU0deIFb1cfFne3FlE8uk2b+6pN2irY18VA+S+/grJ5BLe4 f+Pj6InHXaFMSSrrrMuMZhaZCll2/5GkGyw4e6RumYnsdl8liG4CQWqKGJv8B3dkWWLj oajbsy/lDU6m+kULJCPqJwFHntsxmDWCMQ/OCRjsQ7NIxC8wnD8APYpcegQ8vWO7LZv3 PE7/yQEyhnQ8oZ0NiI+RaundOfjmEjCiucvYSiR8NMlYaVi5pQFF5KBKKC2IsCQXzxGR PnJQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=fIM1ll25GubEqhBH5bZZxcPqlVg9s6cvqfKXlkBrrPY=; b=JrFmKhM06v/bZuD0IRvXzdPJs4FmlllE54m5CzXMHMk4N9U+J8ncSMz40laArNhcZW xwsdDmGFMn674QZ+qpSnc8nyo1rOoA4P2ecGVEPV7GsNOPohuzsMfNelETY8MCzrVvNw qpgPPUGQsrSsPGHjELz9WLt3SrEAuVQba6h7SNbvim6rmJoukut2lGm2sy2l/II8bbc+ AJJjfe7c3NiCOUg/gPrdFaxBSD6p06JE7YsRdOY+rYkUV5ne8U7A2dClTLwY0bOQ3qt0 +4FMcviSmb7uDEK39BHHftraIFKTe77MSSQ1TvhXCOfLmE92yQH5gqDJCh9IxfBjrohN o1Ig== X-Gm-Message-State: ANoB5pmqNrcoWkiIAn/KKB7DOxPpDvePVfO4/zSjYkb9ub529B+AXdpk eyw+A50ehcAE8jnxb2Ooun7+H8aROgw= X-Google-Smtp-Source: AA0mqf6M7G9W4/uGaBih2Kuee+PbdRxx2mRR7SDUx/Tt/ZxcVGWcGBWCvpDAxpYPTTLe4ZKUf4daMg== X-Received: by 2002:a05:600c:26c4:b0:3cf:7004:7b7 with SMTP id 4-20020a05600c26c400b003cf700407b7mr16365701wmv.156.1669371701147; Fri, 25 Nov 2022 02:21:41 -0800 (PST) Received: from able.fritz.box (p5b0ea229.dip0.t-ipconnect.de. [91.14.162.41]) by smtp.gmail.com with ESMTPSA id u2-20020a056000038200b00241cfe6e286sm3351193wrf.98.2022.11.25.02.21.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Nov 2022 02:21:40 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: Intel Graphics Development , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 3/9] drm/ttm: use per BO cleanup workers Date: Fri, 25 Nov 2022 11:21:31 +0100 Message-Id: <20221125102137.1801-3-christian.koenig@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221125102137.1801-1-christian.koenig@amd.com> References: <20221125102137.1801-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Instead of a single worker going over the list of delete BOs in regular intervals use a per BO worker which blocks for the resv object and locking of the BO. This not only simplifies the handling massively, but also results in much better response time when cleaning up buffers. Signed-off-by: Christian König Reviewed-by: Felix Kuehling --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 +- drivers/gpu/drm/i915/i915_gem.c | 2 +- drivers/gpu/drm/i915/intel_region_ttm.c | 2 +- drivers/gpu/drm/ttm/ttm_bo.c | 112 ++++++++------------- drivers/gpu/drm/ttm/ttm_bo_util.c | 1 - drivers/gpu/drm/ttm/ttm_device.c | 24 ++--- include/drm/ttm/ttm_bo_api.h | 18 +--- include/drm/ttm/ttm_device.h | 7 +- 8 files changed, 57 insertions(+), 111 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index 2b1db37e25c1..74ccbd566777 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3984,7 +3984,7 @@ void amdgpu_device_fini_hw(struct amdgpu_device *adev) amdgpu_fence_driver_hw_fini(adev); if (adev->mman.initialized) - flush_delayed_work(&adev->mman.bdev.wq); + drain_workqueue(adev->mman.bdev.wq); if (adev->pm_sysfs_en) amdgpu_pm_sysfs_fini(adev); diff --git a/drivers/gpu/drm/i915/i915_gem.c b/drivers/gpu/drm/i915/i915_gem.c index 8468ca9885fd..c38306f156d6 100644 --- a/drivers/gpu/drm/i915/i915_gem.c +++ b/drivers/gpu/drm/i915/i915_gem.c @@ -1099,7 +1099,7 @@ void i915_gem_drain_freed_objects(struct drm_i915_private *i915) { while (atomic_read(&i915->mm.free_count)) { flush_work(&i915->mm.free_work); - flush_delayed_work(&i915->bdev.wq); + drain_workqueue(i915->bdev.wq); rcu_barrier(); } } diff --git a/drivers/gpu/drm/i915/intel_region_ttm.c b/drivers/gpu/drm/i915/intel_region_ttm.c index cf89d0c2a2d9..657bbc16a48a 100644 --- a/drivers/gpu/drm/i915/intel_region_ttm.c +++ b/drivers/gpu/drm/i915/intel_region_ttm.c @@ -132,7 +132,7 @@ int intel_region_ttm_fini(struct intel_memory_region *mem) break; msleep(20); - flush_delayed_work(&mem->i915->bdev.wq); + drain_workqueue(mem->i915->bdev.wq); } /* If we leaked objects, Don't free the region causing use after free */ diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index b77262a623e0..4749b65bedc4 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -280,14 +280,13 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo, ret = 0; } - if (ret || unlikely(list_empty(&bo->ddestroy))) { + if (ret) { if (unlock_resv) dma_resv_unlock(bo->base.resv); spin_unlock(&bo->bdev->lru_lock); return ret; } - list_del_init(&bo->ddestroy); spin_unlock(&bo->bdev->lru_lock); ttm_bo_cleanup_memtype_use(bo); @@ -300,47 +299,21 @@ static int ttm_bo_cleanup_refs(struct ttm_buffer_object *bo, } /* - * Traverse the delayed list, and call ttm_bo_cleanup_refs on all - * encountered buffers. + * Block for the dma_resv object to become idle, lock the buffer and clean up + * the resource and tt object. */ -bool ttm_bo_delayed_delete(struct ttm_device *bdev, bool remove_all) +static void ttm_bo_delayed_delete(struct work_struct *work) { - struct list_head removed; - bool empty; - - INIT_LIST_HEAD(&removed); - - spin_lock(&bdev->lru_lock); - while (!list_empty(&bdev->ddestroy)) { - struct ttm_buffer_object *bo; - - bo = list_first_entry(&bdev->ddestroy, struct ttm_buffer_object, - ddestroy); - list_move_tail(&bo->ddestroy, &removed); - if (!ttm_bo_get_unless_zero(bo)) - continue; - - if (remove_all || bo->base.resv != &bo->base._resv) { - spin_unlock(&bdev->lru_lock); - dma_resv_lock(bo->base.resv, NULL); - - spin_lock(&bdev->lru_lock); - ttm_bo_cleanup_refs(bo, false, !remove_all, true); - - } else if (dma_resv_trylock(bo->base.resv)) { - ttm_bo_cleanup_refs(bo, false, !remove_all, true); - } else { - spin_unlock(&bdev->lru_lock); - } + struct ttm_buffer_object *bo; - ttm_bo_put(bo); - spin_lock(&bdev->lru_lock); - } - list_splice_tail(&removed, &bdev->ddestroy); - empty = list_empty(&bdev->ddestroy); - spin_unlock(&bdev->lru_lock); + bo = container_of(work, typeof(*bo), delayed_delete); - return empty; + dma_resv_wait_timeout(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP, false, + MAX_SCHEDULE_TIMEOUT); + dma_resv_lock(bo->base.resv, NULL); + ttm_bo_cleanup_memtype_use(bo); + dma_resv_unlock(bo->base.resv); + ttm_bo_put(bo); } static void ttm_bo_release(struct kref *kref) @@ -369,44 +342,40 @@ static void ttm_bo_release(struct kref *kref) drm_vma_offset_remove(bdev->vma_manager, &bo->base.vma_node); ttm_mem_io_free(bdev, bo->resource); - } - - if (!dma_resv_test_signaled(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP) || - !dma_resv_trylock(bo->base.resv)) { - /* The BO is not idle, resurrect it for delayed destroy */ - ttm_bo_flush_all_fences(bo); - bo->deleted = true; - spin_lock(&bo->bdev->lru_lock); + if (!dma_resv_test_signaled(bo->base.resv, + DMA_RESV_USAGE_BOOKKEEP) || + !dma_resv_trylock(bo->base.resv)) { + /* The BO is not idle, resurrect it for delayed destroy */ + ttm_bo_flush_all_fences(bo); + bo->deleted = true; - /* - * Make pinned bos immediately available to - * shrinkers, now that they are queued for - * destruction. - * - * FIXME: QXL is triggering this. Can be removed when the - * driver is fixed. - */ - if (bo->pin_count) { - bo->pin_count = 0; - ttm_resource_move_to_lru_tail(bo->resource); - } + spin_lock(&bo->bdev->lru_lock); - kref_init(&bo->kref); - list_add_tail(&bo->ddestroy, &bdev->ddestroy); - spin_unlock(&bo->bdev->lru_lock); + /* + * Make pinned bos immediately available to + * shrinkers, now that they are queued for + * destruction. + * + * FIXME: QXL is triggering this. Can be removed when the + * driver is fixed. + */ + if (bo->pin_count) { + bo->pin_count = 0; + ttm_resource_move_to_lru_tail(bo->resource); + } - schedule_delayed_work(&bdev->wq, - ((HZ / 100) < 1) ? 1 : HZ / 100); - return; - } + kref_init(&bo->kref); + spin_unlock(&bo->bdev->lru_lock); - spin_lock(&bo->bdev->lru_lock); - list_del(&bo->ddestroy); - spin_unlock(&bo->bdev->lru_lock); + INIT_WORK(&bo->delayed_delete, ttm_bo_delayed_delete); + queue_work(bdev->wq, &bo->delayed_delete); + return; + } - ttm_bo_cleanup_memtype_use(bo); - dma_resv_unlock(bo->base.resv); + ttm_bo_cleanup_memtype_use(bo); + dma_resv_unlock(bo->base.resv); + } atomic_dec(&ttm_glob.bo_count); bo->destroy(bo); @@ -946,7 +915,6 @@ int ttm_bo_init_reserved(struct ttm_device *bdev, struct ttm_buffer_object *bo, int ret; kref_init(&bo->kref); - INIT_LIST_HEAD(&bo->ddestroy); bo->bdev = bdev; bo->type = type; bo->page_alignment = alignment; diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c index ba3aa0a0fc43..ae4b7922ee1a 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_util.c +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c @@ -230,7 +230,6 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo, */ atomic_inc(&ttm_glob.bo_count); - INIT_LIST_HEAD(&fbo->base.ddestroy); drm_vma_node_reset(&fbo->base.base.vma_node); kref_init(&fbo->base.kref); diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c index e7147e304637..e9bedca4dfdc 100644 --- a/drivers/gpu/drm/ttm/ttm_device.c +++ b/drivers/gpu/drm/ttm/ttm_device.c @@ -175,16 +175,6 @@ int ttm_device_swapout(struct ttm_device *bdev, struct ttm_operation_ctx *ctx, } EXPORT_SYMBOL(ttm_device_swapout); -static void ttm_device_delayed_workqueue(struct work_struct *work) -{ - struct ttm_device *bdev = - container_of(work, struct ttm_device, wq.work); - - if (!ttm_bo_delayed_delete(bdev, false)) - schedule_delayed_work(&bdev->wq, - ((HZ / 100) < 1) ? 1 : HZ / 100); -} - /** * ttm_device_init * @@ -215,15 +205,19 @@ int ttm_device_init(struct ttm_device *bdev, struct ttm_device_funcs *funcs, if (ret) return ret; + bdev->wq = alloc_workqueue("ttm", WQ_MEM_RECLAIM | WQ_HIGHPRI, 16); + if (!bdev->wq) { + ttm_global_release(); + return -ENOMEM; + } + bdev->funcs = funcs; ttm_sys_man_init(bdev); ttm_pool_init(&bdev->pool, dev, use_dma_alloc, use_dma32); bdev->vma_manager = vma_manager; - INIT_DELAYED_WORK(&bdev->wq, ttm_device_delayed_workqueue); spin_lock_init(&bdev->lru_lock); - INIT_LIST_HEAD(&bdev->ddestroy); INIT_LIST_HEAD(&bdev->pinned); bdev->dev_mapping = mapping; mutex_lock(&ttm_global_mutex); @@ -247,10 +241,8 @@ void ttm_device_fini(struct ttm_device *bdev) list_del(&bdev->device_list); mutex_unlock(&ttm_global_mutex); - cancel_delayed_work_sync(&bdev->wq); - - if (ttm_bo_delayed_delete(bdev, true)) - pr_debug("Delayed destroy list was clean\n"); + drain_workqueue(bdev->wq); + destroy_workqueue(bdev->wq); spin_lock(&bdev->lru_lock); for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo_api.h index 7758347c461c..69e62bbb01e3 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo_api.h @@ -92,7 +92,6 @@ struct ttm_tt; * @ttm: TTM structure holding system pages. * @evicted: Whether the object was evicted without user-space knowing. * @deleted: True if the object is only a zombie and already deleted. - * @ddestroy: List head for the delayed destroy list. * @swap: List head for swap LRU list. * @offset: The current GPU offset, which can have different meanings * depending on the memory type. For SYSTEM type memory, it should be 0. @@ -135,19 +134,14 @@ struct ttm_buffer_object { struct ttm_tt *ttm; bool deleted; struct ttm_lru_bulk_move *bulk_move; + unsigned priority; + unsigned pin_count; /** - * Members protected by the bdev::lru_lock. - */ - - struct list_head ddestroy; - - /** - * Members protected by a bo reservation. + * @delayed_delete: Work item used when we can't delete the BO + * immediately */ - - unsigned priority; - unsigned pin_count; + struct work_struct delayed_delete; /** * Special members that are protected by the reserve lock @@ -448,8 +442,6 @@ void ttm_bo_vm_close(struct vm_area_struct *vma); int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write); -bool ttm_bo_delayed_delete(struct ttm_device *bdev, bool remove_all); - vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot); #endif diff --git a/include/drm/ttm/ttm_device.h b/include/drm/ttm/ttm_device.h index 95b3c04b1ab9..4f3e81eac6f3 100644 --- a/include/drm/ttm/ttm_device.h +++ b/include/drm/ttm/ttm_device.h @@ -251,11 +251,6 @@ struct ttm_device { */ spinlock_t lru_lock; - /** - * @ddestroy: Destroyed but not yet cleaned up buffer objects. - */ - struct list_head ddestroy; - /** * @pinned: Buffer objects which are pinned and so not on any LRU list. */ @@ -270,7 +265,7 @@ struct ttm_device { /** * @wq: Work queue structure for the delayed delete workqueue. */ - struct delayed_work wq; + struct workqueue_struct *wq; }; int ttm_global_swapout(struct ttm_operation_ctx *ctx, gfp_t gfp_flags); From patchwork Fri Nov 25 10:21:32 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 13055733 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7BA0DC4167B for ; Fri, 25 Nov 2022 10:22:27 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 60CDE10E627; Fri, 25 Nov 2022 10:21:54 +0000 (UTC) Received: from mail-wr1-x429.google.com (mail-wr1-x429.google.com [IPv6:2a00:1450:4864:20::429]) by gabe.freedesktop.org (Postfix) with ESMTPS id 337FF10E607; Fri, 25 Nov 2022 10:21:44 +0000 (UTC) Received: by mail-wr1-x429.google.com with SMTP id n7so6035114wrr.13; Fri, 25 Nov 2022 02:21:44 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=vNvKKwZ8yOBHyFdlYI8xsYuxbxPPoIJ+mjPWASz4JZU=; b=pU0BXePLNKqlnb4f04CKN8cHcWGbqWKYGJJYgHOjwZNTbo2o0k97JR3BQwHDRLaxOh sSwauDKnLPUiHVwHr/2iU0faT5n/BqSR4w4jAIyu1b5aMpeFv0IY0ttZWa1onW+r3azy +PJefaohNwyHfPlZF0zymjIN1uBHhCTAjYrE/ITBYSgK3zoVh+YBtygs3wnEuhfrpkBP OOBVp0IrVG7gbdqX4iT5AaVC/QqahLWFRPAIWFZaRodxuyZezJQ6KIAjuI5fMMnlIm32 VnfM+tIoN4Gi693marFqCSYxbNp7XPYSO9/TJJ0qRCdX6GNVmZF9VjzDUzVokTcMjvEv 2lPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=vNvKKwZ8yOBHyFdlYI8xsYuxbxPPoIJ+mjPWASz4JZU=; b=JU2OXXMApWa4J+M8p+qVYxIFLdJ+RibiS4fwO+wx+tat/hGJZ7rEbJGQN/jK631Rxm FH0kgMkXDS5lQlGrgXwJDbnZwKTpIH4Q+wEycKFT9eSUA0qELlQlzI0xkcao0KgAFeXB 6IYfotwdigIe260fthJHq3mLG59hiCUZ9CDcTKbY7Dp8N+FZ6W6xngs+KvEw9a5TYtgn 2BIcarfA50t7V1pg98DJBh+xakPgPOthW4QnTu7PBwqP5MS31S79lcMwtUddQNxDCPNy XpY4T17PBbuI3hXr8w68Awp7Zvj8b+cBVDFgNuLrOjX1HdN62UBrKyFWJKJvtesW0dXQ eyTg== X-Gm-Message-State: ANoB5pkGvwHESXiMQUQRfGMPoxD/xhO5OapDzayPQB1cbTyuenaY8gdd NtjcdsBVklBB4alqo1zFC+bDUNzzpHU= X-Google-Smtp-Source: AA0mqf69bUpf0S4EIGaCEByOQbsqHWHagqnuTFJ0NMyHidzjxSltRqz7I8n3io/OivBPseXBSE+QTQ== X-Received: by 2002:adf:dec2:0:b0:236:d485:527e with SMTP id i2-20020adfdec2000000b00236d485527emr22325187wrn.169.1669371702703; Fri, 25 Nov 2022 02:21:42 -0800 (PST) Received: from able.fritz.box (p5b0ea229.dip0.t-ipconnect.de. [91.14.162.41]) by smtp.gmail.com with ESMTPSA id u2-20020a056000038200b00241cfe6e286sm3351193wrf.98.2022.11.25.02.21.41 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Nov 2022 02:21:42 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: Intel Graphics Development , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 4/9] drm/ttm: merge ttm_bo_api.h and ttm_bo_driver.h Date: Fri, 25 Nov 2022 11:21:32 +0100 Message-Id: <20221125102137.1801-4-christian.koenig@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221125102137.1801-1-christian.koenig@amd.com> References: <20221125102137.1801-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Merge and cleanup the two headers into a single description of the object API. Also move all the documentation to the implementation and drop unecessary includes from the header. No functional change. Signed-off-by: Christian König --- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 3 +- .../gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_cs.h | 2 + drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.h | 1 - drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 4 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h | 2 +- drivers/gpu/drm/amd/amdkfd/kfd_svm.c | 1 + drivers/gpu/drm/drm_gem_ttm_helper.c | 2 + drivers/gpu/drm/drm_gem_vram_helper.c | 1 + .../gpu/drm/i915/gem/i915_gem_object_types.h | 2 +- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 2 +- drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c | 2 +- drivers/gpu/drm/i915/i915_deps.c | 2 +- drivers/gpu/drm/i915/i915_ttm_buddy_manager.c | 2 +- drivers/gpu/drm/i915/intel_region_ttm.c | 1 - drivers/gpu/drm/nouveau/nouveau_bo.c | 1 + drivers/gpu/drm/nouveau/nouveau_bo.h | 3 +- drivers/gpu/drm/nouveau/nouveau_drv.h | 3 +- drivers/gpu/drm/nouveau/nouveau_mem.c | 3 +- drivers/gpu/drm/nouveau/nouveau_mem.h | 2 +- drivers/gpu/drm/nouveau/nouveau_prime.c | 1 + drivers/gpu/drm/nouveau/nouveau_sgdma.c | 1 + drivers/gpu/drm/qxl/qxl_drv.h | 3 +- drivers/gpu/drm/qxl/qxl_ttm.c | 4 +- drivers/gpu/drm/radeon/radeon.h | 3 +- drivers/gpu/drm/radeon/radeon_prime.c | 2 + drivers/gpu/drm/radeon/radeon_ttm.c | 4 +- drivers/gpu/drm/ttm/ttm_bo.c | 81 +++- drivers/gpu/drm/ttm/ttm_bo_util.c | 110 +++++- drivers/gpu/drm/ttm/ttm_bo_vm.c | 19 +- drivers/gpu/drm/ttm/ttm_device.c | 2 +- drivers/gpu/drm/ttm/ttm_execbuf_util.c | 6 +- drivers/gpu/drm/ttm/ttm_pool.c | 3 +- drivers/gpu/drm/ttm/ttm_range_manager.c | 2 +- drivers/gpu/drm/ttm/ttm_resource.c | 3 +- drivers/gpu/drm/ttm/ttm_tt.c | 3 +- drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c | 2 +- drivers/gpu/drm/vmwgfx/vmwgfx_drv.c | 1 - drivers/gpu/drm/vmwgfx/vmwgfx_drv.h | 4 +- drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c | 2 +- drivers/gpu/drm/vmwgfx/vmwgfx_gmr.c | 1 - drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c | 1 - .../gpu/drm/vmwgfx/vmwgfx_system_manager.c | 1 - drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c | 1 - include/drm/drm_gem_ttm_helper.h | 3 +- include/drm/drm_gem_vram_helper.h | 4 +- include/drm/ttm/{ttm_bo_api.h => ttm_bo.h} | 345 +++++++++--------- include/drm/ttm/ttm_bo_driver.h | 303 --------------- include/drm/ttm/ttm_execbuf_util.h | 4 +- 55 files changed, 410 insertions(+), 557 deletions(-) rename include/drm/ttm/{ttm_bo_api.h => ttm_bo.h} (67%) delete mode 100644 include/drm/ttm/ttm_bo_driver.h diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h index 6b74df446694..2644cd991210 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h @@ -52,8 +52,7 @@ #include #include -#include -#include +#include #include #include diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c index 3a763916a5a1..ab450f12c445 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c @@ -25,6 +25,7 @@ #include #include #include +#include #include "amdgpu_object.h" #include "amdgpu_gem.h" diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h index e4d78491bcc7..ededdc01ca28 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_bo_list.h @@ -28,6 +28,8 @@ struct hmm_range; +struct drm_file; + struct amdgpu_device; struct amdgpu_bo; struct amdgpu_bo_va; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 8516c814bc9b..8b7a09b392ac 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -32,6 +32,8 @@ #include #include +#include + #include "amdgpu_cs.h" #include "amdgpu.h" #include "amdgpu_trace.h" diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.h index 113f39510a72..fb3e3d56d427 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.h @@ -23,6 +23,8 @@ #ifndef __AMDGPU_CS_H__ #define __AMDGPU_CS_H__ +#include + #include "amdgpu_job.h" #include "amdgpu_bo_list.h" #include "amdgpu_ring.h" diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c index 271e30e34d93..0c001bb8fc2b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c @@ -37,6 +37,7 @@ #include "amdgpu_dma_buf.h" #include "amdgpu_xgmi.h" #include +#include #include #include #include diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.h index 41a4c7056729..e86834bfea1d 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fdinfo.h @@ -30,7 +30,6 @@ #include #include #include -#include #include #include "amdgpu_sync.h" diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c index 62e98f1ad770..b9d9c523c1fd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c @@ -34,6 +34,7 @@ #include #include #include +#include #include "amdgpu.h" #include "amdgpu_display.h" diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c index 4365ede42855..3169a942dbbb 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c @@ -35,6 +35,7 @@ #include "amdgpu_xgmi.h" #include +#include /** * amdgpu_gmc_pdb0_alloc - allocate vram for pdb0 diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 7b5074e776f4..068c2d8495fd 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -44,10 +44,10 @@ #include #include -#include -#include +#include #include #include +#include #include #include diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c index 003aa9e47085..fea25519227f 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c @@ -33,6 +33,7 @@ #include #include +#include #include "amdgpu.h" #include "amdgpu_trace.h" #include "amdgpu_amdkfd.h" diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h index 6546e786bf00..44157c6c0804 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h @@ -29,7 +29,7 @@ #include #include #include -#include +#include #include #include "amdgpu_sync.h" diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c index 814f99888ab1..37219198d518 100644 --- a/drivers/gpu/drm/amd/amdkfd/kfd_svm.c +++ b/drivers/gpu/drm/amd/amdkfd/kfd_svm.c @@ -23,6 +23,7 @@ #include #include +#include #include "amdgpu_sync.h" #include "amdgpu_object.h" #include "amdgpu_vm.h" diff --git a/drivers/gpu/drm/drm_gem_ttm_helper.c b/drivers/gpu/drm/drm_gem_ttm_helper.c index d5962a34c01d..3734aa2d1c5b 100644 --- a/drivers/gpu/drm/drm_gem_ttm_helper.c +++ b/drivers/gpu/drm/drm_gem_ttm_helper.c @@ -3,6 +3,8 @@ #include #include +#include +#include /** * DOC: overview diff --git a/drivers/gpu/drm/drm_gem_vram_helper.c b/drivers/gpu/drm/drm_gem_vram_helper.c index b6c7e3803bb3..f59adffd938a 100644 --- a/drivers/gpu/drm/drm_gem_vram_helper.c +++ b/drivers/gpu/drm/drm_gem_vram_helper.c @@ -19,6 +19,7 @@ #include #include +#include static const struct drm_gem_object_funcs drm_gem_vram_object_funcs; diff --git a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h index d0d6772e6f36..a7b70701617a 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_object_types.h +++ b/drivers/gpu/drm/i915/gem/i915_gem_object_types.h @@ -10,7 +10,7 @@ #include #include -#include +#include #include #include "i915_active.h" diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 1e50fb0d6bfc..5247d88b3c13 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -5,8 +5,8 @@ #include -#include #include +#include #include #include "i915_drv.h" diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c index f59f812dc6d2..2ebaaf4d663c 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm_move.c @@ -3,7 +3,7 @@ * Copyright © 2021 Intel Corporation */ -#include +#include #include "i915_deps.h" #include "i915_drv.h" diff --git a/drivers/gpu/drm/i915/i915_deps.c b/drivers/gpu/drm/i915/i915_deps.c index 297b8e4e42ee..91c61864285a 100644 --- a/drivers/gpu/drm/i915/i915_deps.c +++ b/drivers/gpu/drm/i915/i915_deps.c @@ -6,7 +6,7 @@ #include #include -#include +#include #include "i915_deps.h" diff --git a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c index 7e611476c7a4..a72698a2dbc8 100644 --- a/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c +++ b/drivers/gpu/drm/i915/i915_ttm_buddy_manager.c @@ -5,8 +5,8 @@ #include -#include #include +#include #include diff --git a/drivers/gpu/drm/i915/intel_region_ttm.c b/drivers/gpu/drm/i915/intel_region_ttm.c index 657bbc16a48a..4dc0702081b8 100644 --- a/drivers/gpu/drm/i915/intel_region_ttm.c +++ b/drivers/gpu/drm/i915/intel_region_ttm.c @@ -2,7 +2,6 @@ /* * Copyright © 2021 Intel Corporation */ -#include #include #include diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c index a11871e3119c..335fa91ca4ad 100644 --- a/drivers/gpu/drm/nouveau/nouveau_bo.c +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c @@ -28,6 +28,7 @@ */ #include +#include #include "nouveau_drv.h" #include "nouveau_chan.h" diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.h b/drivers/gpu/drm/nouveau/nouveau_bo.h index c2d3f9c48eba..774dd93ca76b 100644 --- a/drivers/gpu/drm/nouveau/nouveau_bo.h +++ b/drivers/gpu/drm/nouveau/nouveau_bo.h @@ -1,8 +1,9 @@ /* SPDX-License-Identifier: MIT */ #ifndef __NOUVEAU_BO_H__ #define __NOUVEAU_BO_H__ -#include #include +#include +#include struct nouveau_channel; struct nouveau_cli; diff --git a/drivers/gpu/drm/nouveau/nouveau_drv.h b/drivers/gpu/drm/nouveau/nouveau_drv.h index d6dd07bfa64a..b5de312a523f 100644 --- a/drivers/gpu/drm/nouveau/nouveau_drv.h +++ b/drivers/gpu/drm/nouveau/nouveau_drv.h @@ -51,8 +51,7 @@ #include #include -#include -#include +#include #include #include diff --git a/drivers/gpu/drm/nouveau/nouveau_mem.c b/drivers/gpu/drm/nouveau/nouveau_mem.c index 1fde3a5d7c32..25f31d5169e5 100644 --- a/drivers/gpu/drm/nouveau/nouveau_mem.c +++ b/drivers/gpu/drm/nouveau/nouveau_mem.c @@ -19,11 +19,12 @@ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR * OTHER DEALINGS IN THE SOFTWARE. */ +#include + #include "nouveau_mem.h" #include "nouveau_drv.h" #include "nouveau_bo.h" -#include #include #include diff --git a/drivers/gpu/drm/nouveau/nouveau_mem.h b/drivers/gpu/drm/nouveau/nouveau_mem.h index 1ee6cdb9ad9b..76c86d8bb01e 100644 --- a/drivers/gpu/drm/nouveau/nouveau_mem.h +++ b/drivers/gpu/drm/nouveau/nouveau_mem.h @@ -1,6 +1,6 @@ #ifndef __NOUVEAU_MEM_H__ #define __NOUVEAU_MEM_H__ -#include +#include struct ttm_tt; #include diff --git a/drivers/gpu/drm/nouveau/nouveau_prime.c b/drivers/gpu/drm/nouveau/nouveau_prime.c index 9608121e49b7..f42c2b1b0363 100644 --- a/drivers/gpu/drm/nouveau/nouveau_prime.c +++ b/drivers/gpu/drm/nouveau/nouveau_prime.c @@ -23,6 +23,7 @@ */ #include +#include #include "nouveau_drv.h" #include "nouveau_gem.h" diff --git a/drivers/gpu/drm/nouveau/nouveau_sgdma.c b/drivers/gpu/drm/nouveau/nouveau_sgdma.c index 85c03c83259b..b14895f75b3c 100644 --- a/drivers/gpu/drm/nouveau/nouveau_sgdma.c +++ b/drivers/gpu/drm/nouveau/nouveau_sgdma.c @@ -1,6 +1,7 @@ // SPDX-License-Identifier: MIT #include #include +#include #include "nouveau_drv.h" #include "nouveau_mem.h" diff --git a/drivers/gpu/drm/qxl/qxl_drv.h b/drivers/gpu/drm/qxl/qxl_drv.h index 76f060810f63..ea993d7162e8 100644 --- a/drivers/gpu/drm/qxl/qxl_drv.h +++ b/drivers/gpu/drm/qxl/qxl_drv.h @@ -42,8 +42,7 @@ #include #include #include -#include -#include +#include #include #include diff --git a/drivers/gpu/drm/qxl/qxl_ttm.c b/drivers/gpu/drm/qxl/qxl_ttm.c index ee95001e6b5e..a92a5b0d4c25 100644 --- a/drivers/gpu/drm/qxl/qxl_ttm.c +++ b/drivers/gpu/drm/qxl/qxl_ttm.c @@ -29,10 +29,10 @@ #include #include #include -#include -#include +#include #include #include +#include #include "qxl_drv.h" #include "qxl_object.h" diff --git a/drivers/gpu/drm/radeon/radeon.h b/drivers/gpu/drm/radeon/radeon.h index 2e7161acd443..57e20780a458 100644 --- a/drivers/gpu/drm/radeon/radeon.h +++ b/drivers/gpu/drm/radeon/radeon.h @@ -73,8 +73,7 @@ #include #endif -#include -#include +#include #include #include diff --git a/drivers/gpu/drm/radeon/radeon_prime.c b/drivers/gpu/drm/radeon/radeon_prime.c index 42a87948e28c..b3cfc99f4d7e 100644 --- a/drivers/gpu/drm/radeon/radeon_prime.c +++ b/drivers/gpu/drm/radeon/radeon_prime.c @@ -29,6 +29,8 @@ #include #include +#include + #include "radeon.h" #include "radeon_prime.h" diff --git a/drivers/gpu/drm/radeon/radeon_ttm.c b/drivers/gpu/drm/radeon/radeon_ttm.c index 30402b5ce4c5..1e8e287e113c 100644 --- a/drivers/gpu/drm/radeon/radeon_ttm.c +++ b/drivers/gpu/drm/radeon/radeon_ttm.c @@ -42,10 +42,10 @@ #include #include #include -#include -#include +#include #include #include +#include #include "radeon_reg.h" #include "radeon.h" diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 4749b65bedc4..f9d9fd2d865d 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -31,8 +31,10 @@ #define pr_fmt(fmt) "[TTM] " fmt -#include +#include #include +#include + #include #include #include @@ -381,6 +383,13 @@ static void ttm_bo_release(struct kref *kref) bo->destroy(bo); } +/** + * ttm_bo_put + * + * @bo: The buffer object. + * + * Unreference a buffer object. + */ void ttm_bo_put(struct ttm_buffer_object *bo) { kref_put(&bo->kref, ttm_bo_release); @@ -467,6 +476,14 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo, return ret; } +/** + * ttm_bo_eviction_valuable + * + * @bo: The buffer object to evict + * @place: the placement we need to make room for + * + * Check if it is valuable to evict the BO to make room for the given placement. + */ bool ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, const struct ttm_place *place) { @@ -726,13 +743,23 @@ static int ttm_bo_mem_force_space(struct ttm_buffer_object *bo, return ttm_bo_add_move_fence(bo, man, *mem, ctx->no_wait_gpu); } -/* - * Creates space for memory region @mem according to its type. +/** + * ttm_bo_mem_space * - * This function first searches for free space in compatible memory types in - * the priority order defined by the driver. If free space isn't found, then - * ttm_bo_mem_force_space is attempted in priority order to evict and find - * space. + * @bo: Pointer to a struct ttm_buffer_object. the data of which + * we want to allocate space for. + * @proposed_placement: Proposed new placement for the buffer object. + * @mem: A struct ttm_resource. + * @ctx: if and how to sleep, lock buffers and alloc memory + * + * Allocate memory space for the buffer object pointed to by @bo, using + * the placement flags in @placement, potentially evicting other idle buffer objects. + * This function may sleep while waiting for space to become available. + * Returns: + * -EBUSY: No space available (only if no_wait == 1). + * -ENOMEM: Could not allocate memory for the buffer object, either due to + * fragmentation or concurrent allocators. + * -ERESTARTSYS: An interruptible sleep was interrupted by a signal. */ int ttm_bo_mem_space(struct ttm_buffer_object *bo, struct ttm_placement *placement, @@ -838,6 +865,21 @@ static int ttm_bo_move_buffer(struct ttm_buffer_object *bo, return ret; } +/** + * ttm_bo_validate + * + * @bo: The buffer object. + * @placement: Proposed placement for the buffer object. + * @ctx: validation parameters. + * + * Changes placement and caching policy of the buffer object + * according proposed placement. + * Returns + * -EINVAL on invalid proposed placement. + * -ENOMEM on out-of-memory condition. + * -EBUSY if no_wait is true and buffer busy. + * -ERESTARTSYS if interrupted by a signal. + */ int ttm_bo_validate(struct ttm_buffer_object *bo, struct ttm_placement *placement, struct ttm_operation_ctx *ctx) @@ -1030,6 +1072,11 @@ EXPORT_SYMBOL(ttm_bo_init_validate); * buffer object vm functions. */ +/** + * ttm_bo_unmap_virtual + * + * @bo: tear down the virtual mappings for this BO + */ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) { struct ttm_device *bdev = bo->bdev; @@ -1039,6 +1086,20 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) } EXPORT_SYMBOL(ttm_bo_unmap_virtual); +/** + * ttm_bo_wait - wait for buffer idle. + * + * @bo: The buffer object. + * @interruptible: Use interruptible wait. + * @no_wait: Return immediately if buffer is busy. + * + * This function must be called with the bo::mutex held, and makes + * sure any previous rendering to the buffer is completed. + * Note: It might be necessary to block validations before the + * wait by reserving the buffer. + * Returns -EBUSY if no_wait is true and the buffer is busy. + * Returns -ERESTARTSYS if interrupted by a signal. + */ int ttm_bo_wait(struct ttm_buffer_object *bo, bool interruptible, bool no_wait) { @@ -1063,6 +1124,12 @@ int ttm_bo_wait(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_wait); +int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx) +{ + return ttm_bo_wait(bo, ctx->interruptible, ctx->no_wait_gpu); +} +EXPORT_SYMBOL(ttm_bo_wait_ctx); + int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, gfp_t gfp_flags) { diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c index ae4b7922ee1a..fee7c20775c0 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_util.c +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c @@ -29,18 +29,11 @@ * Authors: Thomas Hellstrom */ -#include +#include #include +#include + #include -#include -#include -#include -#include -#include -#include -#include -#include -#include struct ttm_transfer_obj { struct ttm_buffer_object base; @@ -128,6 +121,23 @@ void ttm_move_memcpy(bool clear, } EXPORT_SYMBOL(ttm_move_memcpy); +/** + * ttm_bo_move_memcpy + * + * @bo: A pointer to a struct ttm_buffer_object. + * @interruptible: Sleep interruptible if waiting. + * @no_wait_gpu: Return immediately if the GPU is busy. + * @new_mem: struct ttm_resource indicating where to move. + * + * Fallback move function for a mappable buffer object in mappable memory. + * The function will, if successful, + * free any old aperture space, and set (@new_mem)->mm_node to NULL, + * and update the (@bo)->mem placement flags. If unsuccessful, the old + * data remains untouched, and it's up to the caller to free the + * memory space indicated by @new_mem. + * Returns: + * !0: Failure. + */ int ttm_bo_move_memcpy(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, struct ttm_resource *dst_mem) @@ -266,6 +276,16 @@ static int ttm_buffer_object_transfer(struct ttm_buffer_object *bo, return 0; } +/** + * ttm_io_prot + * + * bo: ttm buffer object + * res: ttm resource object + * @tmp: Page protection flag for a normal, cached mapping. + * + * Utility function that returns the pgprot_t that should be used for + * setting up a PTE with the caching model indicated by @c_state. + */ pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res, pgprot_t tmp) { @@ -347,6 +367,22 @@ static int ttm_bo_kmap_ttm(struct ttm_buffer_object *bo, return (!map->virtual) ? -ENOMEM : 0; } +/** + * ttm_bo_kmap + * + * @bo: The buffer object. + * @start_page: The first page to map. + * @num_pages: Number of pages to map. + * @map: pointer to a struct ttm_bo_kmap_obj representing the map. + * + * Sets up a kernel virtual mapping, using ioremap, vmap or kmap to the + * data in the buffer object. The ttm_kmap_obj_virtual function can then be + * used to obtain a virtual address to the data. + * + * Returns + * -ENOMEM: Out of memory. + * -EINVAL: Invalid range. + */ int ttm_bo_kmap(struct ttm_buffer_object *bo, unsigned long start_page, unsigned long num_pages, struct ttm_bo_kmap_obj *map) @@ -374,6 +410,13 @@ int ttm_bo_kmap(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_kmap); +/** + * ttm_bo_kunmap + * + * @map: Object describing the map to unmap. + * + * Unmaps a kernel map set up by ttm_bo_kmap. + */ void ttm_bo_kunmap(struct ttm_bo_kmap_obj *map) { if (!map->virtual) @@ -399,6 +442,20 @@ void ttm_bo_kunmap(struct ttm_bo_kmap_obj *map) } EXPORT_SYMBOL(ttm_bo_kunmap); +/** + * ttm_bo_vmap + * + * @bo: The buffer object. + * @map: pointer to a struct iosys_map representing the map. + * + * Sets up a kernel virtual mapping, using ioremap or vmap to the + * data in the buffer object. The parameter @map returns the virtual + * address as struct iosys_map. Unmap the buffer with ttm_bo_vunmap(). + * + * Returns + * -ENOMEM: Out of memory. + * -EINVAL: Invalid range. + */ int ttm_bo_vmap(struct ttm_buffer_object *bo, struct iosys_map *map) { struct ttm_resource *mem = bo->resource; @@ -460,6 +517,14 @@ int ttm_bo_vmap(struct ttm_buffer_object *bo, struct iosys_map *map) } EXPORT_SYMBOL(ttm_bo_vmap); +/** + * ttm_bo_vunmap + * + * @bo: The buffer object. + * @map: Object describing the map to unmap. + * + * Unmaps a kernel map set up by ttm_bo_vmap(). + */ void ttm_bo_vunmap(struct ttm_buffer_object *bo, struct iosys_map *map) { struct ttm_resource *mem = bo->resource; @@ -553,6 +618,22 @@ static void ttm_bo_move_pipeline_evict(struct ttm_buffer_object *bo, ttm_resource_free(bo, &bo->resource); } +/** + * ttm_bo_move_accel_cleanup. + * + * @bo: A pointer to a struct ttm_buffer_object. + * @fence: A fence object that signals when moving is complete. + * @evict: This is an evict move. Don't return until the buffer is idle. + * @pipeline: evictions are to be pipelined. + * @new_mem: struct ttm_resource indicating where to move. + * + * Accelerated move function to be called when an accelerated move + * has been scheduled. The function will create a new temporary buffer object + * representing the old placement, and put the sync object on both buffer + * objects. After that the newly created buffer object is unref'd to be + * destroyed when the move is complete. This will help pipeline + * buffer moves. + */ int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo, struct dma_fence *fence, bool evict, @@ -581,6 +662,15 @@ int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo, } EXPORT_SYMBOL(ttm_bo_move_accel_cleanup); +/** + * ttm_bo_move_sync_cleanup. + * + * @bo: A pointer to a struct ttm_buffer_object. + * @new_mem: struct ttm_resource indicating where to move. + * + * Special case of ttm_bo_move_accel_cleanup where the bo is guaranteed + * by the caller to be idle. Typically used after memcpy buffer moves. + */ void ttm_bo_move_sync_cleanup(struct ttm_buffer_object *bo, struct ttm_resource *new_mem) { diff --git a/drivers/gpu/drm/ttm/ttm_bo_vm.c b/drivers/gpu/drm/ttm/ttm_bo_vm.c index 5a3e4b891377..3ecda6db24b8 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_vm.c +++ b/drivers/gpu/drm/ttm/ttm_bo_vm.c @@ -31,17 +31,12 @@ #define pr_fmt(fmt) "[TTM] " fmt -#include +#include #include -#include +#include + #include #include -#include -#include -#include -#include -#include -#include static vm_fault_t ttm_bo_vm_fault_idle(struct ttm_buffer_object *bo, struct vm_fault *vmf) @@ -446,6 +441,14 @@ static const struct vm_operations_struct ttm_bo_vm_ops = { .access = ttm_bo_vm_access, }; +/** + * ttm_bo_mmap_obj - mmap memory backed by a ttm buffer object. + * + * @vma: vma as input from the fbdev mmap method. + * @bo: The bo backing the address space. + * + * Maps a buffer object. + */ int ttm_bo_mmap_obj(struct vm_area_struct *vma, struct ttm_buffer_object *bo) { /* Enforce no COW since would have really strange behavior with it. */ diff --git a/drivers/gpu/drm/ttm/ttm_device.c b/drivers/gpu/drm/ttm/ttm_device.c index e9bedca4dfdc..c7a1862f322a 100644 --- a/drivers/gpu/drm/ttm/ttm_device.c +++ b/drivers/gpu/drm/ttm/ttm_device.c @@ -29,10 +29,10 @@ #include +#include #include #include #include -#include #include "ttm_module.h" diff --git a/drivers/gpu/drm/ttm/ttm_execbuf_util.c b/drivers/gpu/drm/ttm/ttm_execbuf_util.c index dbee34a058df..f1c60fa80c2d 100644 --- a/drivers/gpu/drm/ttm/ttm_execbuf_util.c +++ b/drivers/gpu/drm/ttm/ttm_execbuf_util.c @@ -27,11 +27,7 @@ **************************************************************************/ #include -#include -#include -#include -#include -#include +#include static void ttm_eu_backoff_reservation_reverse(struct list_head *list, struct ttm_validate_buffer *entry) diff --git a/drivers/gpu/drm/ttm/ttm_pool.c b/drivers/gpu/drm/ttm/ttm_pool.c index 9f6764bf3b15..aa116a7bbae3 100644 --- a/drivers/gpu/drm/ttm/ttm_pool.c +++ b/drivers/gpu/drm/ttm/ttm_pool.c @@ -33,6 +33,7 @@ #include #include +#include #include #include @@ -41,8 +42,8 @@ #endif #include -#include #include +#include #include "ttm_module.h" diff --git a/drivers/gpu/drm/ttm/ttm_range_manager.c b/drivers/gpu/drm/ttm/ttm_range_manager.c index 0a8bc0b7f380..ae11d07eb63a 100644 --- a/drivers/gpu/drm/ttm/ttm_range_manager.c +++ b/drivers/gpu/drm/ttm/ttm_range_manager.c @@ -32,7 +32,7 @@ #include #include #include -#include +#include #include #include #include diff --git a/drivers/gpu/drm/ttm/ttm_resource.c b/drivers/gpu/drm/ttm/ttm_resource.c index 328391bb1d87..b8a826a24fb2 100644 --- a/drivers/gpu/drm/ttm/ttm_resource.c +++ b/drivers/gpu/drm/ttm/ttm_resource.c @@ -26,8 +26,9 @@ #include #include +#include +#include #include -#include /** * ttm_lru_bulk_move_init - initialize a bulk move structure diff --git a/drivers/gpu/drm/ttm/ttm_tt.c b/drivers/gpu/drm/ttm/ttm_tt.c index d505603930a7..ab725d9d14a6 100644 --- a/drivers/gpu/drm/ttm/ttm_tt.c +++ b/drivers/gpu/drm/ttm/ttm_tt.c @@ -36,7 +36,8 @@ #include #include #include -#include +#include +#include #include "ttm_module.h" diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c index 3c06df2a5474..2b843ff4b437 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c @@ -28,7 +28,7 @@ #include #include -#include +#include #include "vmwgfx_drv.h" diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c index bd02cb0e6837..9ad28346aff7 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.c @@ -40,7 +40,6 @@ #include #include #include -#include #include #include #include diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h index b062b020b378..4b612fc9758c 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_drv.h @@ -37,8 +37,10 @@ #include #include -#include #include +#include +#include +#include #include "ttm_object.h" diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c index a5379f6fb5ab..43cec8e37e4d 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c @@ -29,7 +29,7 @@ #include "vmwgfx_drv.h" #include "vmwgfx_reg.h" -#include +#include #include #include "vmwgfx_so.h" #include "vmwgfx_binding.h" diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gmr.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gmr.c index c482e5298e11..20158a92acc7 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_gmr.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gmr.c @@ -25,7 +25,6 @@ * **************************************************************************/ -#include #include "vmwgfx_drv.h" diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c index abd5e3323ebf..ceb4d3d3b965 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_gmrid_manager.c @@ -29,7 +29,6 @@ */ #include "vmwgfx_drv.h" -#include #include #include #include diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c b/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c index d3007bf1b8f5..ee7964cbdaca 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_system_manager.c @@ -26,7 +26,6 @@ #include "vmwgfx_drv.h" -#include #include #include #include diff --git a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c index 4e3938e62c08..856a352a72a6 100644 --- a/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c +++ b/drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c @@ -26,7 +26,6 @@ **************************************************************************/ #include "vmwgfx_drv.h" -#include #include static const struct ttm_place vram_placement_flags = { diff --git a/include/drm/drm_gem_ttm_helper.h b/include/drm/drm_gem_ttm_helper.h index 4c003b4f173e..7b53d673ae7e 100644 --- a/include/drm/drm_gem_ttm_helper.h +++ b/include/drm/drm_gem_ttm_helper.h @@ -7,8 +7,7 @@ #include #include -#include -#include +#include struct iosys_map; diff --git a/include/drm/drm_gem_vram_helper.h b/include/drm/drm_gem_vram_helper.h index c083a1d71cf4..d3e8920c0b64 100644 --- a/include/drm/drm_gem_vram_helper.h +++ b/include/drm/drm_gem_vram_helper.h @@ -8,8 +8,8 @@ #include #include #include -#include -#include +#include +#include #include #include diff --git a/include/drm/ttm/ttm_bo_api.h b/include/drm/ttm/ttm_bo.h similarity index 67% rename from include/drm/ttm/ttm_bo_api.h rename to include/drm/ttm/ttm_bo.h index 69e62bbb01e3..cfaf88e2d678 100644 --- a/include/drm/ttm/ttm_bo_api.h +++ b/include/drm/ttm/ttm_bo.h @@ -32,28 +32,24 @@ #define _TTM_BO_API_H_ #include -#include + #include #include -#include -#include -#include -#include -#include - -#include "ttm_resource.h" -struct ttm_global; +#include "ttm_device.h" -struct ttm_device; +/* Default number of pre-faulted pages in the TTM fault handler */ +#define TTM_BO_VM_NUM_PREFAULT 16 struct iosys_map; -struct drm_mm_node; - +struct ttm_global; +struct ttm_device; struct ttm_placement; - struct ttm_place; +struct ttm_resource; +struct ttm_resource_manager; +struct ttm_tt; /** * enum ttm_bo_type @@ -68,15 +64,12 @@ struct ttm_place; * @ttm_bo_type_sg: Buffer made from dmabuf sg table shared with another * driver. */ - enum ttm_bo_type { ttm_bo_type_device, ttm_bo_type_kernel, ttm_bo_type_sg }; -struct ttm_tt; - /** * struct ttm_buffer_object * @@ -85,17 +78,11 @@ struct ttm_tt; * @type: The bo type. * @page_alignment: Page alignment. * @destroy: Destruction function. If NULL, kfree is used. - * @num_pages: Actual number of pages. * @kref: Reference count of this buffer object. When this refcount reaches * zero, the object is destroyed or put on the delayed delete list. - * @mem: structure describing current placement. + * @resource: structure describing current placement. * @ttm: TTM structure holding system pages. - * @evicted: Whether the object was evicted without user-space knowing. * @deleted: True if the object is only a zombie and already deleted. - * @swap: List head for swap LRU list. - * @offset: The current GPU offset, which can have different meanings - * depending on the memory type. For SYSTEM type memory, it should be 0. - * @cur_placement: Hint of current placement. * * Base class for TTM buffer object, that deals with data placement and CPU * mappings. GPU mappings are really up to the driver, but for simpler GPUs @@ -108,28 +95,25 @@ struct ttm_tt; * The destroy member, the API visibility of this object makes it possible * to derive driver specific types. */ - struct ttm_buffer_object { struct drm_gem_object base; - /** + /* * Members constant at init. */ - struct ttm_device *bdev; enum ttm_bo_type type; uint32_t page_alignment; void (*destroy) (struct ttm_buffer_object *); - /** + /* * Members not needing protection. */ struct kref kref; - /** + /* * Members protected by the bo::resv::reserved lock. */ - struct ttm_resource *resource; struct ttm_tt *ttm; bool deleted; @@ -148,8 +132,7 @@ struct ttm_buffer_object { * and the bo::lock when written to. Can be read with * either of these locks held. */ - - struct sg_table *sg; + struct sg_table *sg; }; /** @@ -164,7 +147,6 @@ struct ttm_buffer_object { * mapping can either be an ioremap, a vmap, a kmap or part of a * premapped region. */ - #define TTM_BO_MAP_IOMEM_MASK 0x80 struct ttm_bo_kmap_obj { void *virtual; @@ -232,79 +214,119 @@ ttm_bo_get_unless_zero(struct ttm_buffer_object *bo) } /** - * ttm_bo_wait - wait for buffer idle. + * ttm_bo_reserve: + * + * @bo: A pointer to a struct ttm_buffer_object. + * @interruptible: Sleep interruptible if waiting. + * @no_wait: Don't sleep while trying to reserve, rather return -EBUSY. + * @ticket: ticket used to acquire the ww_mutex. * - * @bo: The buffer object. - * @interruptible: Use interruptible wait. - * @no_wait: Return immediately if buffer is busy. + * Locks a buffer object for validation. (Or prevents other processes from + * locking it for validation), while taking a number of measures to prevent + * deadlocks. * - * This function must be called with the bo::mutex held, and makes - * sure any previous rendering to the buffer is completed. - * Note: It might be necessary to block validations before the - * wait by reserving the buffer. - * Returns -EBUSY if no_wait is true and the buffer is busy. - * Returns -ERESTARTSYS if interrupted by a signal. + * Returns: + * -EDEADLK: The reservation may cause a deadlock. + * Release all buffer reservations, wait for @bo to become unreserved and + * try again. + * -ERESTARTSYS: A wait for the buffer to become unreserved was interrupted by + * a signal. Release all buffer reservations and return to user-space. + * -EBUSY: The function needed to sleep, but @no_wait was true + * -EALREADY: Bo already reserved using @ticket. This error code will only + * be returned if @use_ticket is set to true. */ -int ttm_bo_wait(struct ttm_buffer_object *bo, bool interruptible, bool no_wait); - -static inline int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx) +static inline int ttm_bo_reserve(struct ttm_buffer_object *bo, + bool interruptible, bool no_wait, + struct ww_acquire_ctx *ticket) { - return ttm_bo_wait(bo, ctx->interruptible, ctx->no_wait_gpu); + int ret = 0; + + if (no_wait) { + bool success; + if (WARN_ON(ticket)) + return -EBUSY; + + success = dma_resv_trylock(bo->base.resv); + return success ? 0 : -EBUSY; + } + + if (interruptible) + ret = dma_resv_lock_interruptible(bo->base.resv, ticket); + else + ret = dma_resv_lock(bo->base.resv, ticket); + if (ret == -EINTR) + return -ERESTARTSYS; + return ret; } /** - * ttm_bo_validate - * - * @bo: The buffer object. - * @placement: Proposed placement for the buffer object. - * @ctx: validation parameters. - * - * Changes placement and caching policy of the buffer object - * according proposed placement. - * Returns - * -EINVAL on invalid proposed placement. - * -ENOMEM on out-of-memory condition. - * -EBUSY if no_wait is true and buffer busy. - * -ERESTARTSYS if interrupted by a signal. + * ttm_bo_reserve_slowpath: + * @bo: A pointer to a struct ttm_buffer_object. + * @interruptible: Sleep interruptible if waiting. + * @sequence: Set (@bo)->sequence to this value after lock + * + * This is called after ttm_bo_reserve returns -EAGAIN and we backed off + * from all our other reservations. Because there are no other reservations + * held by us, this function cannot deadlock any more. */ -int ttm_bo_validate(struct ttm_buffer_object *bo, - struct ttm_placement *placement, - struct ttm_operation_ctx *ctx); +static inline int ttm_bo_reserve_slowpath(struct ttm_buffer_object *bo, + bool interruptible, + struct ww_acquire_ctx *ticket) +{ + if (interruptible) { + int ret = dma_resv_lock_slow_interruptible(bo->base.resv, + ticket); + if (ret == -EINTR) + ret = -ERESTARTSYS; + return ret; + } + dma_resv_lock_slow(bo->base.resv, ticket); + return 0; +} + +void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo); + +static inline void +ttm_bo_move_to_lru_tail_unlocked(struct ttm_buffer_object *bo) +{ + spin_lock(&bo->bdev->lru_lock); + ttm_bo_move_to_lru_tail(bo); + spin_unlock(&bo->bdev->lru_lock); +} + +static inline void ttm_bo_assign_mem(struct ttm_buffer_object *bo, + struct ttm_resource *new_mem) +{ + WARN_ON(bo->resource); + bo->resource = new_mem; +} /** - * ttm_bo_put - * - * @bo: The buffer object. + * ttm_bo_move_null = assign memory for a buffer object. + * @bo: The bo to assign the memory to + * @new_mem: The memory to be assigned. * - * Unreference a buffer object. + * Assign the memory from new_mem to the memory of the buffer object bo. */ -void ttm_bo_put(struct ttm_buffer_object *bo); - -void ttm_bo_move_to_lru_tail(struct ttm_buffer_object *bo); -void ttm_bo_set_bulk_move(struct ttm_buffer_object *bo, - struct ttm_lru_bulk_move *bulk); +static inline void ttm_bo_move_null(struct ttm_buffer_object *bo, + struct ttm_resource *new_mem) +{ + ttm_resource_free(bo, &bo->resource); + ttm_bo_assign_mem(bo, new_mem); +} /** - * ttm_bo_eviction_valuable + * ttm_bo_unreserve * - * @bo: The buffer object to evict - * @place: the placement we need to make room for + * @bo: A pointer to a struct ttm_buffer_object. * - * Check if it is valuable to evict the BO to make room for the given placement. + * Unreserve a previous reservation of @bo. */ -bool ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, - const struct ttm_place *place); - -int ttm_bo_init_reserved(struct ttm_device *bdev, struct ttm_buffer_object *bo, - enum ttm_bo_type type, struct ttm_placement *placement, - uint32_t alignment, struct ttm_operation_ctx *ctx, - struct sg_table *sg, struct dma_resv *resv, - void (*destroy) (struct ttm_buffer_object *)); -int ttm_bo_init_validate(struct ttm_device *bdev, struct ttm_buffer_object *bo, - enum ttm_bo_type type, struct ttm_placement *placement, - uint32_t alignment, bool interruptible, - struct sg_table *sg, struct dma_resv *resv, - void (*destroy) (struct ttm_buffer_object *)); +static inline void ttm_bo_unreserve(struct ttm_buffer_object *bo) +{ + ttm_bo_move_to_lru_tail_unlocked(bo); + dma_resv_unlock(bo->base.resv); +} /** * ttm_kmap_obj_virtual @@ -324,124 +346,85 @@ static inline void *ttm_kmap_obj_virtual(struct ttm_bo_kmap_obj *map, return map->virtual; } -/** - * ttm_bo_kmap - * - * @bo: The buffer object. - * @start_page: The first page to map. - * @num_pages: Number of pages to map. - * @map: pointer to a struct ttm_bo_kmap_obj representing the map. - * - * Sets up a kernel virtual mapping, using ioremap, vmap or kmap to the - * data in the buffer object. The ttm_kmap_obj_virtual function can then be - * used to obtain a virtual address to the data. - * - * Returns - * -ENOMEM: Out of memory. - * -EINVAL: Invalid range. - */ +int ttm_bo_wait(struct ttm_buffer_object *bo, bool interruptible, bool no_wait); +int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, + struct ttm_operation_ctx *ctx); +int ttm_bo_validate(struct ttm_buffer_object *bo, + struct ttm_placement *placement, + struct ttm_operation_ctx *ctx); +void ttm_bo_put(struct ttm_buffer_object *bo); +void ttm_bo_set_bulk_move(struct ttm_buffer_object *bo, + struct ttm_lru_bulk_move *bulk); +int ttm_bo_lock_delayed_workqueue(struct ttm_device *bdev); +void ttm_bo_unlock_delayed_workqueue(struct ttm_device *bdev, int resched); +bool ttm_bo_eviction_valuable(struct ttm_buffer_object *bo, + const struct ttm_place *place); +int ttm_bo_init_reserved(struct ttm_device *bdev, struct ttm_buffer_object *bo, + enum ttm_bo_type type, struct ttm_placement *placement, + uint32_t alignment, struct ttm_operation_ctx *ctx, + struct sg_table *sg, struct dma_resv *resv, + void (*destroy) (struct ttm_buffer_object *)); +int ttm_bo_init_validate(struct ttm_device *bdev, struct ttm_buffer_object *bo, + enum ttm_bo_type type, struct ttm_placement *placement, + uint32_t alignment, bool interruptible, + struct sg_table *sg, struct dma_resv *resv, + void (*destroy) (struct ttm_buffer_object *)); int ttm_bo_kmap(struct ttm_buffer_object *bo, unsigned long start_page, unsigned long num_pages, struct ttm_bo_kmap_obj *map); - -/** - * ttm_bo_kunmap - * - * @map: Object describing the map to unmap. - * - * Unmaps a kernel map set up by ttm_bo_kmap. - */ void ttm_bo_kunmap(struct ttm_bo_kmap_obj *map); - -/** - * ttm_bo_vmap - * - * @bo: The buffer object. - * @map: pointer to a struct iosys_map representing the map. - * - * Sets up a kernel virtual mapping, using ioremap or vmap to the - * data in the buffer object. The parameter @map returns the virtual - * address as struct iosys_map. Unmap the buffer with ttm_bo_vunmap(). - * - * Returns - * -ENOMEM: Out of memory. - * -EINVAL: Invalid range. - */ int ttm_bo_vmap(struct ttm_buffer_object *bo, struct iosys_map *map); - -/** - * ttm_bo_vunmap - * - * @bo: The buffer object. - * @map: Object describing the map to unmap. - * - * Unmaps a kernel map set up by ttm_bo_vmap(). - */ void ttm_bo_vunmap(struct ttm_buffer_object *bo, struct iosys_map *map); - -/** - * ttm_bo_mmap_obj - mmap memory backed by a ttm buffer object. - * - * @vma: vma as input from the fbdev mmap method. - * @bo: The bo backing the address space. - * - * Maps a buffer object. - */ int ttm_bo_mmap_obj(struct vm_area_struct *vma, struct ttm_buffer_object *bo); - -/** - * ttm_bo_io - * - * @bdev: Pointer to the struct ttm_device. - * @filp: Pointer to the struct file attempting to read / write. - * @wbuf: User-space pointer to address of buffer to write. NULL on read. - * @rbuf: User-space pointer to address of buffer to read into. - * Null on write. - * @count: Number of bytes to read / write. - * @f_pos: Pointer to current file position. - * @write: 1 for read, 0 for write. - * - * This function implements read / write into ttm buffer objects, and is - * intended to - * be called from the fops::read and fops::write method. - * Returns: - * See man (2) write, man(2) read. In particular, - * the function may return -ERESTARTSYS if - * interrupted by a signal. - */ -ssize_t ttm_bo_io(struct ttm_device *bdev, struct file *filp, - const char __user *wbuf, char __user *rbuf, - size_t count, loff_t *f_pos, bool write); - int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, gfp_t gfp_flags); - void ttm_bo_pin(struct ttm_buffer_object *bo); void ttm_bo_unpin(struct ttm_buffer_object *bo); - int ttm_mem_evict_first(struct ttm_device *bdev, struct ttm_resource_manager *man, const struct ttm_place *place, struct ttm_operation_ctx *ctx, struct ww_acquire_ctx *ticket); - -/* Default number of pre-faulted pages in the TTM fault handler */ -#define TTM_BO_VM_NUM_PREFAULT 16 - vm_fault_t ttm_bo_vm_reserve(struct ttm_buffer_object *bo, struct vm_fault *vmf); - vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, pgprot_t prot, pgoff_t num_prefault); - vm_fault_t ttm_bo_vm_fault(struct vm_fault *vmf); - void ttm_bo_vm_open(struct vm_area_struct *vma); - void ttm_bo_vm_close(struct vm_area_struct *vma); - int ttm_bo_vm_access(struct vm_area_struct *vma, unsigned long addr, void *buf, int len, int write); vm_fault_t ttm_bo_vm_dummy_page(struct vm_fault *vmf, pgprot_t prot); +int ttm_bo_mem_space(struct ttm_buffer_object *bo, + struct ttm_placement *placement, + struct ttm_resource **mem, + struct ttm_operation_ctx *ctx); + +void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); +/* + * ttm_bo_util.c + */ +int ttm_mem_io_reserve(struct ttm_device *bdev, + struct ttm_resource *mem); +void ttm_mem_io_free(struct ttm_device *bdev, + struct ttm_resource *mem); +void ttm_move_memcpy(bool clear, + u32 num_pages, + struct ttm_kmap_iter *dst_iter, + struct ttm_kmap_iter *src_iter); +int ttm_bo_move_memcpy(struct ttm_buffer_object *bo, + struct ttm_operation_ctx *ctx, + struct ttm_resource *new_mem); +int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo, + struct dma_fence *fence, bool evict, + bool pipeline, + struct ttm_resource *new_mem); +void ttm_bo_move_sync_cleanup(struct ttm_buffer_object *bo, + struct ttm_resource *new_mem); +int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo); +pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res, + pgprot_t tmp); +void ttm_bo_tt_destroy(struct ttm_buffer_object *bo); + #endif diff --git a/include/drm/ttm/ttm_bo_driver.h b/include/drm/ttm/ttm_bo_driver.h deleted file mode 100644 index 1afa891f488a..000000000000 --- a/include/drm/ttm/ttm_bo_driver.h +++ /dev/null @@ -1,303 +0,0 @@ -/************************************************************************** - * - * Copyright (c) 2006-2009 Vmware, Inc., Palo Alto, CA., USA - * All Rights Reserved. - * - * Permission is hereby granted, free of charge, to any person obtaining a - * copy of this software and associated documentation files (the - * "Software"), to deal in the Software without restriction, including - * without limitation the rights to use, copy, modify, merge, publish, - * distribute, sub license, and/or sell copies of the Software, and to - * permit persons to whom the Software is furnished to do so, subject to - * the following conditions: - * - * The above copyright notice and this permission notice (including the - * next paragraph) shall be included in all copies or substantial portions - * of the Software. - * - * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR - * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, - * FITNESS FOR A PARTICULAR PURPOSE AND NON-INFRINGEMENT. IN NO EVENT SHALL - * THE COPYRIGHT HOLDERS, AUTHORS AND/OR ITS SUPPLIERS BE LIABLE FOR ANY CLAIM, - * DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR - * OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE - * USE OR OTHER DEALINGS IN THE SOFTWARE. - * - **************************************************************************/ -/* - * Authors: Thomas Hellstrom - */ -#ifndef _TTM_BO_DRIVER_H_ -#define _TTM_BO_DRIVER_H_ - -#include -#include -#include -#include -#include -#include - -#include - -#include "ttm_bo_api.h" -#include "ttm_kmap_iter.h" -#include "ttm_placement.h" -#include "ttm_tt.h" -#include "ttm_pool.h" - -/* - * ttm_bo.c - */ - -/** - * ttm_bo_mem_space - * - * @bo: Pointer to a struct ttm_buffer_object. the data of which - * we want to allocate space for. - * @proposed_placement: Proposed new placement for the buffer object. - * @mem: A struct ttm_resource. - * @interruptible: Sleep interruptible when sliping. - * @no_wait_gpu: Return immediately if the GPU is busy. - * - * Allocate memory space for the buffer object pointed to by @bo, using - * the placement flags in @mem, potentially evicting other idle buffer objects. - * This function may sleep while waiting for space to become available. - * Returns: - * -EBUSY: No space available (only if no_wait == 1). - * -ENOMEM: Could not allocate memory for the buffer object, either due to - * fragmentation or concurrent allocators. - * -ERESTARTSYS: An interruptible sleep was interrupted by a signal. - */ -int ttm_bo_mem_space(struct ttm_buffer_object *bo, - struct ttm_placement *placement, - struct ttm_resource **mem, - struct ttm_operation_ctx *ctx); - -/** - * ttm_bo_unmap_virtual - * - * @bo: tear down the virtual mappings for this BO - */ -void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo); - -/** - * ttm_bo_reserve: - * - * @bo: A pointer to a struct ttm_buffer_object. - * @interruptible: Sleep interruptible if waiting. - * @no_wait: Don't sleep while trying to reserve, rather return -EBUSY. - * @ticket: ticket used to acquire the ww_mutex. - * - * Locks a buffer object for validation. (Or prevents other processes from - * locking it for validation), while taking a number of measures to prevent - * deadlocks. - * - * Returns: - * -EDEADLK: The reservation may cause a deadlock. - * Release all buffer reservations, wait for @bo to become unreserved and - * try again. - * -ERESTARTSYS: A wait for the buffer to become unreserved was interrupted by - * a signal. Release all buffer reservations and return to user-space. - * -EBUSY: The function needed to sleep, but @no_wait was true - * -EALREADY: Bo already reserved using @ticket. This error code will only - * be returned if @use_ticket is set to true. - */ -static inline int ttm_bo_reserve(struct ttm_buffer_object *bo, - bool interruptible, bool no_wait, - struct ww_acquire_ctx *ticket) -{ - int ret; - - if (no_wait) { - bool success; - if (WARN_ON(ticket)) - return -EBUSY; - - success = dma_resv_trylock(bo->base.resv); - return success ? 0 : -EBUSY; - } - - if (interruptible) - ret = dma_resv_lock_interruptible(bo->base.resv, ticket); - else - ret = dma_resv_lock(bo->base.resv, ticket); - if (ret == -EINTR) - return -ERESTARTSYS; - return ret; -} - -/** - * ttm_bo_reserve_slowpath: - * @bo: A pointer to a struct ttm_buffer_object. - * @interruptible: Sleep interruptible if waiting. - * @sequence: Set (@bo)->sequence to this value after lock - * - * This is called after ttm_bo_reserve returns -EAGAIN and we backed off - * from all our other reservations. Because there are no other reservations - * held by us, this function cannot deadlock any more. - */ -static inline int ttm_bo_reserve_slowpath(struct ttm_buffer_object *bo, - bool interruptible, - struct ww_acquire_ctx *ticket) -{ - if (interruptible) { - int ret = dma_resv_lock_slow_interruptible(bo->base.resv, - ticket); - if (ret == -EINTR) - ret = -ERESTARTSYS; - return ret; - } - dma_resv_lock_slow(bo->base.resv, ticket); - return 0; -} - -static inline void -ttm_bo_move_to_lru_tail_unlocked(struct ttm_buffer_object *bo) -{ - spin_lock(&bo->bdev->lru_lock); - ttm_bo_move_to_lru_tail(bo); - spin_unlock(&bo->bdev->lru_lock); -} - -static inline void ttm_bo_assign_mem(struct ttm_buffer_object *bo, - struct ttm_resource *new_mem) -{ - WARN_ON(bo->resource); - bo->resource = new_mem; -} - -/** - * ttm_bo_move_null = assign memory for a buffer object. - * @bo: The bo to assign the memory to - * @new_mem: The memory to be assigned. - * - * Assign the memory from new_mem to the memory of the buffer object bo. - */ -static inline void ttm_bo_move_null(struct ttm_buffer_object *bo, - struct ttm_resource *new_mem) -{ - ttm_resource_free(bo, &bo->resource); - ttm_bo_assign_mem(bo, new_mem); -} - -/** - * ttm_bo_unreserve - * - * @bo: A pointer to a struct ttm_buffer_object. - * - * Unreserve a previous reservation of @bo. - */ -static inline void ttm_bo_unreserve(struct ttm_buffer_object *bo) -{ - ttm_bo_move_to_lru_tail_unlocked(bo); - dma_resv_unlock(bo->base.resv); -} - -/* - * ttm_bo_util.c - */ -int ttm_mem_io_reserve(struct ttm_device *bdev, - struct ttm_resource *mem); -void ttm_mem_io_free(struct ttm_device *bdev, - struct ttm_resource *mem); - -/** - * ttm_bo_move_memcpy - * - * @bo: A pointer to a struct ttm_buffer_object. - * @interruptible: Sleep interruptible if waiting. - * @no_wait_gpu: Return immediately if the GPU is busy. - * @new_mem: struct ttm_resource indicating where to move. - * - * Fallback move function for a mappable buffer object in mappable memory. - * The function will, if successful, - * free any old aperture space, and set (@new_mem)->mm_node to NULL, - * and update the (@bo)->mem placement flags. If unsuccessful, the old - * data remains untouched, and it's up to the caller to free the - * memory space indicated by @new_mem. - * Returns: - * !0: Failure. - */ - -int ttm_bo_move_memcpy(struct ttm_buffer_object *bo, - struct ttm_operation_ctx *ctx, - struct ttm_resource *new_mem); - -/** - * ttm_bo_move_accel_cleanup. - * - * @bo: A pointer to a struct ttm_buffer_object. - * @fence: A fence object that signals when moving is complete. - * @evict: This is an evict move. Don't return until the buffer is idle. - * @pipeline: evictions are to be pipelined. - * @new_mem: struct ttm_resource indicating where to move. - * - * Accelerated move function to be called when an accelerated move - * has been scheduled. The function will create a new temporary buffer object - * representing the old placement, and put the sync object on both buffer - * objects. After that the newly created buffer object is unref'd to be - * destroyed when the move is complete. This will help pipeline - * buffer moves. - */ -int ttm_bo_move_accel_cleanup(struct ttm_buffer_object *bo, - struct dma_fence *fence, bool evict, - bool pipeline, - struct ttm_resource *new_mem); - -/** - * ttm_bo_move_sync_cleanup. - * - * @bo: A pointer to a struct ttm_buffer_object. - * @new_mem: struct ttm_resource indicating where to move. - * - * Special case of ttm_bo_move_accel_cleanup where the bo is guaranteed - * by the caller to be idle. Typically used after memcpy buffer moves. - */ -void ttm_bo_move_sync_cleanup(struct ttm_buffer_object *bo, - struct ttm_resource *new_mem); - -/** - * ttm_bo_pipeline_gutting. - * - * @bo: A pointer to a struct ttm_buffer_object. - * - * Pipelined gutting a BO of its backing store. - */ -int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo); - -/** - * ttm_io_prot - * - * bo: ttm buffer object - * res: ttm resource object - * @tmp: Page protection flag for a normal, cached mapping. - * - * Utility function that returns the pgprot_t that should be used for - * setting up a PTE with the caching model indicated by @c_state. - */ -pgprot_t ttm_io_prot(struct ttm_buffer_object *bo, struct ttm_resource *res, - pgprot_t tmp); - -/** - * ttm_bo_tt_bind - * - * Bind the object tt to a memory resource. - */ -int ttm_bo_tt_bind(struct ttm_buffer_object *bo, struct ttm_resource *mem); - -/** - * ttm_bo_tt_destroy. - */ -void ttm_bo_tt_destroy(struct ttm_buffer_object *bo); - -void ttm_move_memcpy(bool clear, - u32 num_pages, - struct ttm_kmap_iter *dst_iter, - struct ttm_kmap_iter *src_iter); - -struct ttm_kmap_iter * -ttm_kmap_iter_iomap_init(struct ttm_kmap_iter_iomap *iter_io, - struct io_mapping *iomap, - struct sg_table *st, - resource_size_t start); -#endif diff --git a/include/drm/ttm/ttm_execbuf_util.h b/include/drm/ttm/ttm_execbuf_util.h index a99d7fdf2964..03aca29d3ce4 100644 --- a/include/drm/ttm/ttm_execbuf_util.h +++ b/include/drm/ttm/ttm_execbuf_util.h @@ -33,7 +33,9 @@ #include -#include "ttm_bo_api.h" +struct ww_acquire_ctx; +struct dma_fence; +struct ttm_buffer_object; /** * struct ttm_validate_buffer From patchwork Fri Nov 25 10:21:33 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 13055734 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0B26FC4332F for ; Fri, 25 Nov 2022 10:22:35 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7597410E6FB; Fri, 25 Nov 2022 10:21:59 +0000 (UTC) Received: from mail-wm1-x335.google.com (mail-wm1-x335.google.com [IPv6:2a00:1450:4864:20::335]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5656410E60A; Fri, 25 Nov 2022 10:21:45 +0000 (UTC) Received: by mail-wm1-x335.google.com with SMTP id c65-20020a1c3544000000b003cfffd00fc0so5856514wma.1; Fri, 25 Nov 2022 02:21:45 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=qFb6DuLYLT1qnxM9CyElgz3eLWM7FoqC04HZ9bOT06I=; b=QIEzVG9CadqhWUP0CV9vQgijhyWkzsuhlieVS2CTpJj/Q2iycy5hGkBpQ3+rnlCAzo Mth6fLY3FZNqgbmh53dWxNnEY4RGguWWyS8eOkfW2LZuUdPHF+nVfhPwysfgIhhGj+ll njzJ2cVrgEcUabFhaHGyfgR9asLm+jQM8kLGi9BLttQC/R/A+8jpuH9Tu36PFy4Nr4WL qvMyUs6zN7c8vOi3dZBihlSYWkUkzLkXw8u8v0EjStP7NF+bFrvGYgiJYzLPdM6uUPto lPUEoMMO54TAbJa8RTGLBaAD14gFwcUj7h/bK7Ndr5gRHafqJcse3OcVqsEjH8o+KJn5 hbEQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qFb6DuLYLT1qnxM9CyElgz3eLWM7FoqC04HZ9bOT06I=; b=kRhU5P9iJ6PKlnOcH6462hM+rhJ0SwePU34FRteTVYQl0Wu0x2T+yJO8hJ67qQ9HwB /bC907Rn9iZAwoq5aaCu/ffyhAMdeiSYMWDQ0NAK1r7+E/DVK3ROapTTlhhhR8i5BdXA huB+mbdwRl2t2wpgwjySu7NFJKHQQi0VFXWTHKYMOXxXgyIaq6Zkn71OrIaFr9bpCdAx 7o34fHTxNIAq7eThafOz7hxWcZ3svZ8ACS+w6Ci/ZoK7ezXPz8vSF7VtRvHtsnlwS7sk a012KKEFahwB0nSunv80vZqc6VYUOP2EECJUG1Pa539lUXDi8J3LrNjcbQIuYwevaPU8 R32w== X-Gm-Message-State: ANoB5pk02zzHA1m8kFjwqojnyVB66Odo/U3n5e+RW5r3rFEuniKQ8vNh 3oaPjAIcJsAdOBadIvzzn/mqf+mtod0= X-Google-Smtp-Source: AA0mqf7kQV0uXZbs1jfxko1sTrCWCb+Dkdl4SbxHdMTH/0snr/heDcVUx+5GegPW/yWzcBQiX1BSpQ== X-Received: by 2002:a1c:7504:0:b0:3cf:8ebf:69b with SMTP id o4-20020a1c7504000000b003cf8ebf069bmr17811854wmc.86.1669371703655; Fri, 25 Nov 2022 02:21:43 -0800 (PST) Received: from able.fritz.box (p5b0ea229.dip0.t-ipconnect.de. [91.14.162.41]) by smtp.gmail.com with ESMTPSA id u2-20020a056000038200b00241cfe6e286sm3351193wrf.98.2022.11.25.02.21.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Nov 2022 02:21:43 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: Intel Graphics Development , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 5/9] drm/nouveau: stop using ttm_bo_wait Date: Fri, 25 Nov 2022 11:21:33 +0100 Message-Id: <20221125102137.1801-5-christian.koenig@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221125102137.1801-1-christian.koenig@amd.com> References: <20221125102137.1801-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" TTM is just wrapping core DMA functionality here, remove the mid-layer. No functional change. Signed-off-by: Christian König --- drivers/gpu/drm/nouveau/nouveau_bo.c | 6 +++++- drivers/gpu/drm/nouveau/nouveau_gem.c | 11 ++++++++--- 2 files changed, 13 insertions(+), 4 deletions(-) diff --git a/drivers/gpu/drm/nouveau/nouveau_bo.c b/drivers/gpu/drm/nouveau/nouveau_bo.c index 335fa91ca4ad..288eebc70a67 100644 --- a/drivers/gpu/drm/nouveau/nouveau_bo.c +++ b/drivers/gpu/drm/nouveau/nouveau_bo.c @@ -922,6 +922,7 @@ static void nouveau_bo_move_ntfy(struct ttm_buffer_object *bo, struct nouveau_mem *mem = new_reg ? nouveau_mem(new_reg) : NULL; struct nouveau_bo *nvbo = nouveau_bo(bo); struct nouveau_vma *vma; + long ret; /* ttm can now (stupidly) pass the driver bos it didn't create... */ if (bo->destroy != nouveau_bo_del_ttm) @@ -936,7 +937,10 @@ static void nouveau_bo_move_ntfy(struct ttm_buffer_object *bo, } } else { list_for_each_entry(vma, &nvbo->vma_list, head) { - WARN_ON(ttm_bo_wait(bo, false, false)); + ret = dma_resv_wait_timeout(bo->base.resv, + DMA_RESV_USAGE_BOOKKEEP, + false, 15 * HZ); + WARN_ON(ret <= 0); nouveau_vma_unmap(vma); } } diff --git a/drivers/gpu/drm/nouveau/nouveau_gem.c b/drivers/gpu/drm/nouveau/nouveau_gem.c index ac5793c96957..f77e44958037 100644 --- a/drivers/gpu/drm/nouveau/nouveau_gem.c +++ b/drivers/gpu/drm/nouveau/nouveau_gem.c @@ -645,7 +645,7 @@ nouveau_gem_pushbuf_reloc_apply(struct nouveau_cli *cli, struct drm_nouveau_gem_pushbuf_reloc *reloc, struct drm_nouveau_gem_pushbuf_bo *bo) { - int ret = 0; + long ret = 0; unsigned i; for (i = 0; i < req->nr_relocs; i++) { @@ -703,9 +703,14 @@ nouveau_gem_pushbuf_reloc_apply(struct nouveau_cli *cli, data |= r->vor; } - ret = ttm_bo_wait(&nvbo->bo, false, false); + ret = dma_resv_wait_timeout(nvbo->bo.base.resv, + DMA_RESV_USAGE_BOOKKEEP, + false, 15 * HZ); + if (ret == 0) + ret = -EBUSY; if (ret) { - NV_PRINTK(err, cli, "reloc wait_idle failed: %d\n", ret); + NV_PRINTK(err, cli, "reloc wait_idle failed: %ld\n", + ret); break; } From patchwork Fri Nov 25 10:21:34 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 13055736 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F0476C4167B for ; Fri, 25 Nov 2022 10:22:42 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id D614110E712; Fri, 25 Nov 2022 10:22:00 +0000 (UTC) Received: from mail-wr1-x42b.google.com (mail-wr1-x42b.google.com [IPv6:2a00:1450:4864:20::42b]) by gabe.freedesktop.org (Postfix) with ESMTPS id 5913210E607; Fri, 25 Nov 2022 10:21:46 +0000 (UTC) Received: by mail-wr1-x42b.google.com with SMTP id n3so6074239wrp.5; Fri, 25 Nov 2022 02:21:46 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=33DOQ6vIILkZQ3xEa6fyI46ngystOIlWDLLPn82kuV0=; b=kJX7IkC4crz3dmOgYPbRKZRKOuNuneH1sLMxyeWft8WqHHa4+WXc4YmD8PlV3ChHfF Oz9HnywlMZ5YTlEfkQN5gIX7Fhm2l4+nzFh+5JwTqStPgrGkv9Rw+XAdshtkAlOVaUPI QdE5NguTdAPu7d5Xl8yFVSWtrDpFndF6F6zeY6ZtLm2FBolDa5DuiPc0PPKuzwShawRr jV/Kc1G5Siihh9RTGwLuwgtIXV9YbAd8nAWfXRDtBkMY7uT6JjHdTAjlf2vKg5vMhyz1 Tcb6kKct98aQNPPEH6q364vPQv2mtTwnGa4BTSkFUrD/zC/3bWWcmbLo3rWv7aczAya6 vIAQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=33DOQ6vIILkZQ3xEa6fyI46ngystOIlWDLLPn82kuV0=; b=Kbg4zPXiDf/yPptakMkZNSEoUuYMzKY1X9R0RwfCaiRNg0iyYasW364FuoVVDkz91N 5EPvA2HwJdXU7Nz59bfpuytU/P6ybxIinGdOdQf+DRjz84mWKcRpQCLpmv4QOydQ9XQa O6wZfLUc/yQgpRJ7RV0/1X97lq/ce4V5zISXwMwkyFJz0Oe9PnaRpDORmsBeVJhaH+pn HSZuGI4dbxyp5YzaYgpI4TV2xrJFQAkKZgSUsBQ7Crj/yLGJ9/d/k++4eIpjsrrCUlSN t+UR5TkmgQGNUuKYwirCIdGjxx0QF1JjItDXtFC8j2joYaX3K4z4aHMyDHEd9KEiI6Wv z48g== X-Gm-Message-State: ANoB5pnZYZA69RCh3LMAhEI+GCRXnijaLImYMZrOXkLxrnYiKf7qYu2P BWDGAqTosFLn1tgsapcBvcqbR5cmzdY= X-Google-Smtp-Source: AA0mqf7E4aDTtlz9t7bv0fqT7LUToARjYgjvdOa3757tYVgKsoerJq6QppVNY6973wjwqcA2B0zWKg== X-Received: by 2002:a5d:4143:0:b0:241:ea66:88dc with SMTP id c3-20020a5d4143000000b00241ea6688dcmr7966462wrq.86.1669371704530; Fri, 25 Nov 2022 02:21:44 -0800 (PST) Received: from able.fritz.box (p5b0ea229.dip0.t-ipconnect.de. [91.14.162.41]) by smtp.gmail.com with ESMTPSA id u2-20020a056000038200b00241cfe6e286sm3351193wrf.98.2022.11.25.02.21.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Nov 2022 02:21:44 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: Intel Graphics Development , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 6/9] drm/qxl: stop using ttm_bo_wait Date: Fri, 25 Nov 2022 11:21:34 +0100 Message-Id: <20221125102137.1801-6-christian.koenig@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221125102137.1801-1-christian.koenig@amd.com> References: <20221125102137.1801-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" TTM is just wrapping core DMA functionality here, remove the mid-layer. No functional change. Signed-off-by: Christian König Acked-by: Dave Airlie --- drivers/gpu/drm/qxl/qxl_cmd.c | 16 ++++++++++++++-- 1 file changed, 14 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/qxl/qxl_cmd.c b/drivers/gpu/drm/qxl/qxl_cmd.c index 63aa96a69752..281edab518cd 100644 --- a/drivers/gpu/drm/qxl/qxl_cmd.c +++ b/drivers/gpu/drm/qxl/qxl_cmd.c @@ -579,7 +579,7 @@ void qxl_surface_evict(struct qxl_device *qdev, struct qxl_bo *surf, bool do_upd static int qxl_reap_surf(struct qxl_device *qdev, struct qxl_bo *surf, bool stall) { - int ret; + long ret; ret = qxl_bo_reserve(surf); if (ret) @@ -588,7 +588,19 @@ static int qxl_reap_surf(struct qxl_device *qdev, struct qxl_bo *surf, bool stal if (stall) mutex_unlock(&qdev->surf_evict_mutex); - ret = ttm_bo_wait(&surf->tbo, true, !stall); + if (stall) { + ret = dma_resv_wait_timeout(surf->tbo.base.resv, + DMA_RESV_USAGE_BOOKKEEP, true, + 15 * HZ); + if (ret > 0) + ret = 0; + else if (ret == 0) + ret = -EBUSY; + } else { + ret = dma_resv_test_signaled(surf->tbo.base.resv, + DMA_RESV_USAGE_BOOKKEEP); + ret = ret ? -EBUSY : 0; + } if (stall) mutex_lock(&qdev->surf_evict_mutex); From patchwork Fri Nov 25 10:21:35 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 13055722 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 977D9C4332F for ; Fri, 25 Nov 2022 10:22:22 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id F0BC410E70F; Fri, 25 Nov 2022 10:21:53 +0000 (UTC) Received: from mail-wm1-x32c.google.com (mail-wm1-x32c.google.com [IPv6:2a00:1450:4864:20::32c]) by gabe.freedesktop.org (Postfix) with ESMTPS id 6F1A510E614; Fri, 25 Nov 2022 10:21:47 +0000 (UTC) Received: by mail-wm1-x32c.google.com with SMTP id p13-20020a05600c468d00b003cf8859ed1bso3067981wmo.1; Fri, 25 Nov 2022 02:21:47 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=WANO23gI2Zg38u80cknASIYETE8/F7wtFNyAR3nQEls=; b=dFbHJ2qyTCJCLmjuuI+/TZKwaHPRMNcqZl6D6o+fFN19OaBDUhdbHT6uQSGt+izI/f 2H3AY+gYu+E5eErzFlgeV2mhm44F7R86Fj2xrpC1wPJRgiuiZwxCkojnyT89PDc0bxQn ZcMM+hrLLV7BBQexFX0i0qDYb8MKvhRQKJvIEI19FZGtuB2NfgCwjg/iDBxFM8H+rRmE iaVWBPRulMF9FGy9y+CSlMx+/NWTE+4w9rT5CJV/XO7tnMpoVyebAmdS9WeXrqcQM3i8 vd0u7D2clfl97a6aCp8Hgf7CXLd0w/nTDv4KhoDBv5AH25I/xY+cXuhuFm3JmFoxhgYk AgSg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=WANO23gI2Zg38u80cknASIYETE8/F7wtFNyAR3nQEls=; b=ahEbI64LjNkAkISd5i2ZdL7pXjvweNaG9+v5qDolIp9Z8m43DAlA3b6pLWjt6srqfT nGeWfUK5Tp5/TMooL2yrIABmH2M3sXaVk1aQyIxiEDh1X9kTPJ/V++6KAQcSK1jMspWj q5tl0/nCjEAdT5/ebdOELk1kOYtgzd9CRwMIufgiKjo8kMYPOCfeCt7TkoCLvyS1oI72 suGc2BXvzYuTljBNvk1+NIVNF+J9ZXOf5EjrWjJA4sJfis1cLbWjUgSk0TZGqTvWDe+h qi1rPaGLe/Wj6jtg2iYqaI34WNluQYS4lXCNgcK1jJ+pwaTakucPJ8AcKXmUTesll50i 1Gqg== X-Gm-Message-State: ANoB5plorJbdisWz2idEaFoELi/DKaRd7/jl5jRHuOrYSn0Cx+BnVcf4 1OxB5H0QbVHvIo5goqqovkd3UY7cWP0= X-Google-Smtp-Source: AA0mqf5XHgbIhWUnVE90w1tR3HG6xBZyGA/AlQWOACiC4mOeibO8aASN2xyR9IyGhKg4V1HBkEsI6A== X-Received: by 2002:a05:600c:1e0f:b0:3cf:e9a4:3261 with SMTP id ay15-20020a05600c1e0f00b003cfe9a43261mr31053035wmb.41.1669371705443; Fri, 25 Nov 2022 02:21:45 -0800 (PST) Received: from able.fritz.box (p5b0ea229.dip0.t-ipconnect.de. [91.14.162.41]) by smtp.gmail.com with ESMTPSA id u2-20020a056000038200b00241cfe6e286sm3351193wrf.98.2022.11.25.02.21.44 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Nov 2022 02:21:45 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: Intel Graphics Development , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 7/9] drm/i915: stop using ttm_bo_wait Date: Fri, 25 Nov 2022 11:21:35 +0100 Message-Id: <20221125102137.1801-7-christian.koenig@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221125102137.1801-1-christian.koenig@amd.com> References: <20221125102137.1801-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" TTM is just wrapping core DMA functionality here, remove the mid-layer. No functional change. Signed-off-by: Christian König Reviewed-by: Matthew Auld --- drivers/gpu/drm/i915/gem/i915_gem_ttm.c | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c index 5247d88b3c13..d409a77449a3 100644 --- a/drivers/gpu/drm/i915/gem/i915_gem_ttm.c +++ b/drivers/gpu/drm/i915/gem/i915_gem_ttm.c @@ -599,13 +599,16 @@ i915_ttm_resource_get_st(struct drm_i915_gem_object *obj, static int i915_ttm_truncate(struct drm_i915_gem_object *obj) { struct ttm_buffer_object *bo = i915_gem_to_ttm(obj); - int err; + long err; WARN_ON_ONCE(obj->mm.madv == I915_MADV_WILLNEED); - err = ttm_bo_wait(bo, true, false); - if (err) + err = dma_resv_wait_timeout(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP, + true, 15 * HZ); + if (err < 0) return err; + if (err == 0) + return -EBUSY; err = i915_ttm_move_notify(bo); if (err) From patchwork Fri Nov 25 10:21:36 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 13055735 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7D61DC4332F for ; Fri, 25 Nov 2022 10:22:38 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 6DAFD10E711; Fri, 25 Nov 2022 10:22:00 +0000 (UTC) Received: from mail-wm1-x32b.google.com (mail-wm1-x32b.google.com [IPv6:2a00:1450:4864:20::32b]) by gabe.freedesktop.org (Postfix) with ESMTPS id 4B7A010E627; Fri, 25 Nov 2022 10:21:48 +0000 (UTC) Received: by mail-wm1-x32b.google.com with SMTP id o30so3102036wms.2; Fri, 25 Nov 2022 02:21:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=ThoC9BlA4uS4+BJNMy5V7KH5AR+BD77ENoFY2rH+hUc=; b=H2qMSFsxldnwxvc2sxGcBsHzaEz+N6vNI6ZhHVgifIbPqBfPAN4E/8E/OX12Nx55Hk VuQuNDCWwQtPUvPsMNFXmuyAk1AlVz2ZGKl8TG8PP0OjiLQtu4/1nUVz2JmgkdUBWaYK v3ac8clJ9hhw3EXtlGk5fS4YIYC9rh9IozCScHGST/h1pJg2T1U5nDFcLT3Wmor+DNGS KQBLO8JHqBxIwUJSSiDFbnVrG2rULQQq7NPlnh22bKpPkV12yk5xL6PXOTlGLTazrPgL y27GPD0n/ZOAIVn5gApa6clJqCBjtkoX+CwpU3FNsBIk3Q4BjkjM4Wfcm7bekH4kic00 kEAw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ThoC9BlA4uS4+BJNMy5V7KH5AR+BD77ENoFY2rH+hUc=; b=M07hDhIJbGAlRQ8F4smwBaYb3e4/zC1FmPSsPCJZB3KFabneMRW/Q9cum/22+uiSSf b3OPWLQDfLobpkO8Q7A4DZrwPfpQtxfwoN1t3zf4+hC6Mo6zNrp0nVi7Ppv2xk3o2Dt/ kfUzfHBtNY7aH67k8tB/gmjCIA6KRqLJtUCXhAkn2Lx1lX5xVL4VrAfYHJeZ2pEQXB2I RtWIRAQ7wU/e0HVG3Ay1igfv+XNL4OKgTA1cKvMkAtmcPGsC+Hx1/RTZ8srmz2GCGD+j wdS/eZxByCcymV7bviLY3gSPfZlUwz6xwSuZP31bA400pLLbCpJ/jcd5Z4/NpmwpmcFG qetQ== X-Gm-Message-State: ANoB5pnWxjUjnWbPdoyolcuSjL67YBhfKF9odkUnQEzZ7uAREESDCh/m w+c1mZGJKAXM7bR2kuJKdKZ0FONoQrY= X-Google-Smtp-Source: AA0mqf4cGoMfIGODx1aFJKkRXq9EvQGvcTRP7x/5Zkg4ldqc8sxA0I31KExdlqkTQ1d9Q2WPaqTmuQ== X-Received: by 2002:a05:600c:19cc:b0:3cf:7bdd:e014 with SMTP id u12-20020a05600c19cc00b003cf7bdde014mr21042693wmq.1.1669371706306; Fri, 25 Nov 2022 02:21:46 -0800 (PST) Received: from able.fritz.box (p5b0ea229.dip0.t-ipconnect.de. [91.14.162.41]) by smtp.gmail.com with ESMTPSA id u2-20020a056000038200b00241cfe6e286sm3351193wrf.98.2022.11.25.02.21.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Nov 2022 02:21:45 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: Intel Graphics Development , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 8/9] drm/ttm: use ttm_bo_wait_ctx instead of ttm_bo_wait Date: Fri, 25 Nov 2022 11:21:36 +0100 Message-Id: <20221125102137.1801-8-christian.koenig@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221125102137.1801-1-christian.koenig@amd.com> References: <20221125102137.1801-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Make sure that we use the correct settings from the context. Signed-off-by: Christian König --- drivers/gpu/drm/ttm/ttm_bo.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index f9d9fd2d865d..cd266a067773 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -439,7 +439,7 @@ static int ttm_bo_evict(struct ttm_buffer_object *bo, bdev->funcs->evict_flags(bo, &placement); if (!placement.num_placement && !placement.num_busy_placement) { - ret = ttm_bo_wait(bo, true, false); + ret = ttm_bo_wait_ctx(bo, ctx); if (ret) return ret; @@ -1190,7 +1190,7 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, /* * Make sure BO is idle. */ - ret = ttm_bo_wait(bo, false, false); + ret = ttm_bo_wait_ctx(bo, ctx); if (unlikely(ret != 0)) goto out; From patchwork Fri Nov 25 10:21:37 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christian_K=C3=B6nig?= X-Patchwork-Id: 13055737 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B27E2C4332F for ; Fri, 25 Nov 2022 10:22:45 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 1FD9E10E715; Fri, 25 Nov 2022 10:22:02 +0000 (UTC) Received: from mail-wr1-x431.google.com (mail-wr1-x431.google.com [IPv6:2a00:1450:4864:20::431]) by gabe.freedesktop.org (Postfix) with ESMTPS id 2EF0610E66E; Fri, 25 Nov 2022 10:21:49 +0000 (UTC) Received: by mail-wr1-x431.google.com with SMTP id g12so6042043wrs.10; Fri, 25 Nov 2022 02:21:49 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:from:to:cc:subject:date:message-id :reply-to; bh=o0YPpxQ/Ip2/zYWzBSiDluBmslbFNM8kf+Iwh6rl5dU=; b=qvPwPI3TQq+pXriukE3ongoP8ru+5u2PtE8ILSCPLM9KaxdiMIsqGryaAx21TuSZmi YI/Q0mA63SIIGWF1VTQQwzs39cFpohVwJbQ6pf/kyNRM+WHK4MVAzbUAAWRbNUYgioKP fxVtXF1o/0E36ArfWP+muTkCL07Vo4EK888ckxt+aPprjSo0zlGwCR7URjQKuFRVZNk9 9YD76q4z5Xkv2VZlB9q+7j1e7F57IxcdTxtLzUleiqA8QwDARM0dpdm98cDCeM1KfN8w r19dGN9kWL+RoqSj/+DTzDobtiGoy2Tz1/20uGKQVGVjPoGpMr6eo4iw5zp8CcoOue0t 2Dbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=o0YPpxQ/Ip2/zYWzBSiDluBmslbFNM8kf+Iwh6rl5dU=; b=eHxD7Ji77RvLXznMPN5YKQUxcMaLay5BOFxIBfm5lAcqJf0h7NK05pTaxAOADrllAg mmM3ZQhMNCkItDFtBI9Lvwf5aQ4ZGTKjWVqhUjgFwk9yoI+EDGkTKYwCXafiFsxEp1VC tdVnv/7HHO2VR7UcqUy9vPGP0608qDwVCHZuw+3K2ZSbbUbJymX8HAIwOvjXUBjCGJZB EvzI8gjF/m2DQxGj8rvFAT/0T7ibChfFnOpzGG5x6vCN17r44qR7FHU/nU9u++khcwrq zSMRCOGM/iRcjXWFyFK8bkGcV/094I2RaPu1jfwGbVxXVk9h4W68r3QHJNNDbbCt1ajY 6lKg== X-Gm-Message-State: ANoB5pm5o+OZNdcaZAAqlAvEHPwaYMblRL38FxmdFuv5HSKFM/WGMbLs Du0/s4YZEiAARzxKtrMyzIkaCvTErZk= X-Google-Smtp-Source: AA0mqf7FdnazNRzg6PH5Y4TfoO5Yag7tByPKY+W5qQcW7+MSFmegEOJpBQx/VaayGvfB9r9Yie1RUA== X-Received: by 2002:a5d:604b:0:b0:236:6442:3a48 with SMTP id j11-20020a5d604b000000b0023664423a48mr12982207wrt.41.1669371707310; Fri, 25 Nov 2022 02:21:47 -0800 (PST) Received: from able.fritz.box (p5b0ea229.dip0.t-ipconnect.de. [91.14.162.41]) by smtp.gmail.com with ESMTPSA id u2-20020a056000038200b00241cfe6e286sm3351193wrf.98.2022.11.25.02.21.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 25 Nov 2022 02:21:46 -0800 (PST) From: " =?utf-8?q?Christian_K=C3=B6nig?= " X-Google-Original-From: =?utf-8?q?Christian_K=C3=B6nig?= To: Intel Graphics Development , dri-devel@lists.freedesktop.org, amd-gfx@lists.freedesktop.org Subject: [PATCH 9/9] drm/ttm: move ttm_bo_wait into VMWGFX Date: Fri, 25 Nov 2022 11:21:37 +0100 Message-Id: <20221125102137.1801-9-christian.koenig@amd.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20221125102137.1801-1-christian.koenig@amd.com> References: <20221125102137.1801-1-christian.koenig@amd.com> MIME-Version: 1.0 X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Not used anymore by other drivers or TTM itself. Signed-off-by: Christian König --- drivers/gpu/drm/ttm/ttm_bo.c | 44 +++++++++++------------------ drivers/gpu/drm/ttm/ttm_bo_util.c | 19 ++++++++----- drivers/gpu/drm/vmwgfx/ttm_object.h | 11 ++++++++ include/drm/ttm/ttm_bo.h | 1 - 4 files changed, 39 insertions(+), 36 deletions(-) diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index cd266a067773..326a3d13a829 100644 --- a/drivers/gpu/drm/ttm/ttm_bo.c +++ b/drivers/gpu/drm/ttm/ttm_bo.c @@ -1087,47 +1087,35 @@ void ttm_bo_unmap_virtual(struct ttm_buffer_object *bo) EXPORT_SYMBOL(ttm_bo_unmap_virtual); /** - * ttm_bo_wait - wait for buffer idle. + * ttm_bo_wait_ctx - wait for buffer idle. * * @bo: The buffer object. - * @interruptible: Use interruptible wait. - * @no_wait: Return immediately if buffer is busy. + * @ctx: defines how to wait * - * This function must be called with the bo::mutex held, and makes - * sure any previous rendering to the buffer is completed. - * Note: It might be necessary to block validations before the - * wait by reserving the buffer. - * Returns -EBUSY if no_wait is true and the buffer is busy. - * Returns -ERESTARTSYS if interrupted by a signal. + * Waits for the buffer to be idle. Used timeout depends on the context. + * Returns -EBUSY if wait timed outt, -ERESTARTSYS if interrupted by a signal or + * zero on success. */ -int ttm_bo_wait(struct ttm_buffer_object *bo, - bool interruptible, bool no_wait) +int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx) { - long timeout = 15 * HZ; + long ret; - if (no_wait) { - if (dma_resv_test_signaled(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP)) + if (ctx->no_wait_gpu) { + if (dma_resv_test_signaled(bo->base.resv, + DMA_RESV_USAGE_BOOKKEEP)) return 0; else return -EBUSY; } - timeout = dma_resv_wait_timeout(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP, - interruptible, timeout); - if (timeout < 0) - return timeout; - - if (timeout == 0) + ret = dma_resv_wait_timeout(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP, + ctx->interruptible, 15 * HZ); + if (unlikely(ret < 0)) + return ret; + if (unlikely(ret == 0)) return -EBUSY; - return 0; } -EXPORT_SYMBOL(ttm_bo_wait); - -int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx) -{ - return ttm_bo_wait(bo, ctx->interruptible, ctx->no_wait_gpu); -} EXPORT_SYMBOL(ttm_bo_wait_ctx); int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, @@ -1135,7 +1123,7 @@ int ttm_bo_swapout(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx, { struct ttm_place place; bool locked; - int ret; + long ret; /* * While the bo may already reside in SYSTEM placement, set diff --git a/drivers/gpu/drm/ttm/ttm_bo_util.c b/drivers/gpu/drm/ttm/ttm_bo_util.c index fee7c20775c0..ed2b28734541 100644 --- a/drivers/gpu/drm/ttm/ttm_bo_util.c +++ b/drivers/gpu/drm/ttm/ttm_bo_util.c @@ -547,9 +547,13 @@ EXPORT_SYMBOL(ttm_bo_vunmap); static int ttm_bo_wait_free_node(struct ttm_buffer_object *bo, bool dst_use_tt) { - int ret; - ret = ttm_bo_wait(bo, false, false); - if (ret) + long ret; + + ret = dma_resv_wait_timeout(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP, + false, 15 * HZ); + if (ret == 0) + return -EBUSY; + if (ret < 0) return ret; if (!dst_use_tt) @@ -710,8 +714,7 @@ int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo) return ret; /* If already idle, no need for ghost object dance. */ - ret = ttm_bo_wait(bo, false, true); - if (ret != -EBUSY) { + if (dma_resv_test_signaled(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP)) { if (!bo->ttm) { /* See comment below about clearing. */ ret = ttm_tt_create(bo, true); @@ -748,8 +751,10 @@ int ttm_bo_pipeline_gutting(struct ttm_buffer_object *bo) ret = dma_resv_copy_fences(&ghost->base._resv, bo->base.resv); /* Last resort, wait for the BO to be idle when we are OOM */ - if (ret) - ttm_bo_wait(bo, false, false); + if (ret) { + dma_resv_wait_timeout(bo->base.resv, DMA_RESV_USAGE_BOOKKEEP, + false, MAX_SCHEDULE_TIMEOUT); + } dma_resv_unlock(&ghost->base._resv); ttm_bo_put(ghost); diff --git a/drivers/gpu/drm/vmwgfx/ttm_object.h b/drivers/gpu/drm/vmwgfx/ttm_object.h index f0ebbe340ad6..95a9679f9d39 100644 --- a/drivers/gpu/drm/vmwgfx/ttm_object.h +++ b/drivers/gpu/drm/vmwgfx/ttm_object.h @@ -42,6 +42,8 @@ #include #include +#include + /** * enum ttm_object_type * @@ -321,4 +323,13 @@ static inline void ttm_base_object_noref_release(void) __acquire(RCU); rcu_read_unlock(); } + +static inline int ttm_bo_wait(struct ttm_buffer_object *bo, bool intr, + bool no_wait) +{ + struct ttm_operation_ctx ctx = { intr, no_wait }; + + return ttm_bo_wait_ctx(bo, &ctx); +} + #endif diff --git a/include/drm/ttm/ttm_bo.h b/include/drm/ttm/ttm_bo.h index cfaf88e2d678..6dd8f63a5715 100644 --- a/include/drm/ttm/ttm_bo.h +++ b/include/drm/ttm/ttm_bo.h @@ -346,7 +346,6 @@ static inline void *ttm_kmap_obj_virtual(struct ttm_bo_kmap_obj *map, return map->virtual; } -int ttm_bo_wait(struct ttm_buffer_object *bo, bool interruptible, bool no_wait); int ttm_bo_wait_ctx(struct ttm_buffer_object *bo, struct ttm_operation_ctx *ctx); int ttm_bo_validate(struct ttm_buffer_object *bo,