From patchwork Mon Sep 30 13:52:36 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 13816502 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 88A6FCE8347 for ; Mon, 30 Sep 2024 13:52:46 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 0E56B10E501; Mon, 30 Sep 2024 13:52:46 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="iiFrfIbe"; dkim-atps=neutral Received: from mail-ed1-f42.google.com (mail-ed1-f42.google.com [209.85.208.42]) by gabe.freedesktop.org (Postfix) with ESMTPS id 17E6810E500; Mon, 30 Sep 2024 13:52:44 +0000 (UTC) Received: by mail-ed1-f42.google.com with SMTP id 4fb4d7f45d1cf-5c883459b19so3727424a12.2; Mon, 30 Sep 2024 06:52:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727704362; x=1728309162; darn=lists.freedesktop.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=MLrAGGRIrPLNX0fIU3MHlAkZUYfcrMnizkIk1EY/dAY=; b=iiFrfIbedI8YgB6n1IBL2WG4vr9ADLdNVbfcUC87T3eWcXOzSIR6OEpyVOwR8Rkeun bNUEBOi0605mIw1FnbOe8rNfoX9m2MxyQp10ozQ4LsWrfDUZDlEg5vqQ92sqiO50fM9T w7YedBJFuE/7gMioKpcpOlXKCT2eVSMQjGF2iidiYJvAw1ikj2N/FqiHGELvUuMlkOET ivwK+pURhL0ErKZmVl1AVkMXlFhxZeHgE421v655XIkGeWtwlmycKzh2zMRexfcFLYM3 dxVVqYTmljNolEGoWWdcjcRP97I+rbL1LshvEajXXe9kJccgbDH7hwPQPininS7lbcy2 ZNtg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727704362; x=1728309162; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=MLrAGGRIrPLNX0fIU3MHlAkZUYfcrMnizkIk1EY/dAY=; b=Ubsk7FfoYLHr/n1oC2TLvnRpgLvwKBpKsxuyygpIl5Clw4eQx37HcM+YTCv5IqKWvT c3kHBNc5YRpvYbbl6FiGA4atU+f3SPU5R+9HYCvjzYgRgIigsM9qRBv8i4FC990hklIR lHrxihBdGEFztGRnqOh6l4ILF6Mj8soi1WdaDW8IQhHEFIeYm+5MT1yzq9IEDSiYIiU5 HaDFaG+XV4X/ljoo1vTMBJDwREdgUHUlkax3dEJ2MZ0g4uRcFL5+uABNRvjW+iGwb+Zo hVIebqFhmA3Hpd3lFQ3+IPwPAaCV6lIWjcmDa1cm4NvPKTp1SidNWcGqU9y4ra7KWmhk 8DzA== X-Forwarded-Encrypted: i=1; AJvYcCUT6J1W0Uu8K3xaGOwW+UOLt8fTq2bb5rZWKCeu2k9K3dJJ7eN6Iibrw94Ln5+miczujB1kCm38pik=@lists.freedesktop.org, AJvYcCXbX1N5XJ8j64ACKUNEhbmnOaHSIMdexKtCX554Sy3D8mo4FEis0qn8rqRt1iJADB/ey3CPpFEz9UIR@lists.freedesktop.org X-Gm-Message-State: AOJu0YxEHDGzYf3igFgs1cqlDBCEyBh07+U96bux9R5T3jx4JS9bOUnt OgPJ+s8zq6zVTFWOkPJDs+YigEKOdfDj8w4muq7a2C9DFU349kpP X-Google-Smtp-Source: AGHT+IEFwDfI/yyd85ivmRYj59a+IDIZDYGFuKouqmE64yOQiR05oWwF2sdSrNbu7DDgzfcifNS3kw== X-Received: by 2002:a05:6402:1ece:b0:5c8:9f44:a24d with SMTP id 4fb4d7f45d1cf-5c89f44a2femr1198181a12.9.1727704361914; Mon, 30 Sep 2024 06:52:41 -0700 (PDT) Received: from [192.168.1.17] (host-95-250-55-22.retail.telecomitalia.it. [95.250.55.22]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88245ea57sm4507540a12.55.2024.09.30.06.52.39 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 06:52:40 -0700 (PDT) From: Antonino Maniscalco Date: Mon, 30 Sep 2024 15:52:36 +0200 Subject: [PATCH v7 01/12] drm/msm: Fix bv_fence being used as bv_rptr MIME-Version: 1.0 Message-Id: <20240930-preemption-a750-t-v7-1-47803c7a5a64@gmail.com> References: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> In-Reply-To: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Akhil P Oommen , Neil Armstrong X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727704357; l=1862; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=a7i9oji+Kb0tTlTXnErd/GxgUbRIJJuFO80e+whAz5c=; b=8RLHfSwmKTKKYIKNjHZtEWAuGYdw62UTNn2WYfVTjtQN2MI6YKOgdgL/t97WnkqA9PvydBoyX x5k4icV35nrDXQKKh9xUJbaUtjVv5cUD2YpFFwtzb6XwEPKxQusJI5q X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" The bv_fence field of rbmemptrs was being used incorrectly as the BV rptr shadow pointer in some places. Add a bv_rptr field and change the code to use that instead. Reviewed-by: Akhil P Oommen Tested-by: Rob Clark Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 2 +- drivers/gpu/drm/msm/msm_ringbuffer.h | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index bcaec86ac67a5c90544922372cd46fbdd8cf359e..32a4faa93d7f072ea6b8d949f4dc9d2a58cec6b9 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -1132,7 +1132,7 @@ static int hw_init(struct msm_gpu *gpu) /* ..which means "always" on A7xx, also for BV shadow */ if (adreno_is_a7xx(adreno_gpu)) { gpu_write64(gpu, REG_A7XX_CP_BV_RB_RPTR_ADDR, - rbmemptr(gpu->rb[0], bv_fence)); + rbmemptr(gpu->rb[0], bv_rptr)); } /* Always come up on rb 0 */ diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.h b/drivers/gpu/drm/msm/msm_ringbuffer.h index 0d6beb8cd39a7b297e73741d2018915246a710d4..40791b2ade46ef0e16e2a4088291a575d3be9e82 100644 --- a/drivers/gpu/drm/msm/msm_ringbuffer.h +++ b/drivers/gpu/drm/msm/msm_ringbuffer.h @@ -31,6 +31,7 @@ struct msm_rbmemptrs { volatile uint32_t rptr; volatile uint32_t fence; /* Introduced on A7xx */ + volatile uint32_t bv_rptr; volatile uint32_t bv_fence; volatile struct msm_gpu_submit_stats stats[MSM_GPU_SUBMIT_STATS_COUNT]; From patchwork Mon Sep 30 13:52:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 13816503 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6528ECE8347 for ; Mon, 30 Sep 2024 13:52:50 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C198910E503; Mon, 30 Sep 2024 13:52:49 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="hJeVBmt3"; dkim-atps=neutral Received: from mail-ed1-f48.google.com (mail-ed1-f48.google.com [209.85.208.48]) by gabe.freedesktop.org (Postfix) with ESMTPS id 62AE710E501; Mon, 30 Sep 2024 13:52:45 +0000 (UTC) Received: by mail-ed1-f48.google.com with SMTP id 4fb4d7f45d1cf-5c88d7c8dbaso1461276a12.2; Mon, 30 Sep 2024 06:52:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727704364; x=1728309164; darn=lists.freedesktop.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=qYDoFr9Rcb4U8qB9JjDnoMu8I4N6v3+THQGXkHPg2F0=; b=hJeVBmt3f/AJo7UvdcO4QJnonK1+ANnk1AbI15FOU7j8qp9C5/i5iHB5APeaQu34CR q90FD+BjIi4x/Ty3jUHMSBzMMAYNBPFRSw63O1Zq7X7xQgUtYuytD7l4DDKVTAV+Yk3v qooAesLG0U2AFo9uCzVHNmSxaAS55VO/RuDP7BV9/9We2M9cFyTgp6EPx0YK57iGJ24N bk7Cjx2rSKR/0nLPBWGrHqVSKgBY2u0tJ33vGvKSSsyzAuMWTZ94ohmXSGXrPrgnsHN2 j+qeQh5hbXNB0GNrC7OJuwFdLB7Mmt0Je56Df38aPbd1IgOAQpnAyNVb4O6iGez8N3Qo cD8g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727704364; x=1728309164; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qYDoFr9Rcb4U8qB9JjDnoMu8I4N6v3+THQGXkHPg2F0=; b=AIBdDmcNvRMuHt+b3J/LnFuriR2NJUUrrMDx6/C+lEXnWc8kv6TYhuKaAKNqcqjWB1 oKctgoBZ3YCjSgqqLySPEp8Do+LMHLvOcrzTQjhQMp0Ep6TxbALNng0MJlNd1IPMAhir XgtLam6EVTaqvnaSAe4NBMVHGaswnyqFINLx/miygkkw1SgcbxJHEJCKD1ScyFt/NVhN A42P0grshrj/ye1Qnm7PsnycAXtctL4m3xizz06FblREd4/zN3h4kDc1s3r2eLPrjkxq EJKANdisYNt6AjXGstVHlAz1rTA1Qe6/wiO3nZk4vA5UpNKKZzj9vnYE+RgfaKICObSZ ZCxQ== X-Forwarded-Encrypted: i=1; AJvYcCWm8UO7RwN2Es2P93Wa1zONVvxxRPo7z74TpqlhV2k0/Prak97jN8M76mQFACh5f+y4aCfNJxOyWhc=@lists.freedesktop.org, AJvYcCXpXbyzSFVzdpRGdjdUq6fvTrvpdmuwHpZV8QX7R/9TOmBYwKkVrc5pVeK7PPRgcmp+hCcNfz8gec8g@lists.freedesktop.org X-Gm-Message-State: AOJu0Yy94R/DVStBpCFk0vAYjxuLXPJUB8WbeZiow38bHrDP3TCzFrhX NW/GO6GfHBRv0OModBaKmfiFtIWlGY3+YJS8hMXDQC8IKE0xgAde X-Google-Smtp-Source: AGHT+IH4JrcfMQYAjy2fyNe948G2C0pX+g5oMJjPtAa4bSbtIoTyYO7R5h1Q1yKIKjs/BBIg7J8Z/g== X-Received: by 2002:a05:6402:13cb:b0:5c8:9696:bae2 with SMTP id 4fb4d7f45d1cf-5c89696be95mr8412100a12.32.1727704363438; Mon, 30 Sep 2024 06:52:43 -0700 (PDT) Received: from [192.168.1.17] (host-95-250-55-22.retail.telecomitalia.it. [95.250.55.22]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88245ea57sm4507540a12.55.2024.09.30.06.52.42 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 06:52:43 -0700 (PDT) From: Antonino Maniscalco Date: Mon, 30 Sep 2024 15:52:37 +0200 Subject: [PATCH v7 02/12] drm/msm/a6xx: Track current_ctx_seqno per ring MIME-Version: 1.0 Message-Id: <20240930-preemption-a750-t-v7-2-47803c7a5a64@gmail.com> References: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> In-Reply-To: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Neil Armstrong X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727704357; l=8467; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=yIHp38RtvtLDCz+dKB6Vf596mZ26ig0Y5efzj1HIfzs=; b=wgxA7k1QZvSJdjmM2/VOw+MPuEvBn6t36T2WxuOAf1yO8IS8NkziEbf6guMwPf8rNXNyW19rh 2guE+u3xY4MAIlQPpx6CkYcsGT8+yme6PCmyZ2mjWxiTt+fRjgL8Tp9 X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" With preemption it is not enough to track the current_ctx_seqno globally as execution might switch between rings. This is especially problematic when current_ctx_seqno is used to determine whether a page table switch is necessary as it might lead to security bugs. Track current context per ring. Tested-by: Rob Clark Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +++--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 10 ++++++---- drivers/gpu/drm/msm/msm_gpu.c | 2 +- drivers/gpu/drm/msm/msm_gpu.h | 11 ----------- drivers/gpu/drm/msm/msm_ringbuffer.h | 10 ++++++++++ 8 files changed, 23 insertions(+), 22 deletions(-) diff --git a/drivers/gpu/drm/msm/adreno/a2xx_gpu.c b/drivers/gpu/drm/msm/adreno/a2xx_gpu.c index 0dc255ddf5ceba87090f64d5cb9f078b61104063..379a3d346c300f3ccc9e9bd08ef2a32aa3e24ceb 100644 --- a/drivers/gpu/drm/msm/adreno/a2xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a2xx_gpu.c @@ -22,7 +22,7 @@ static void a2xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: /* ignore if there has not been a ctx switch: */ - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: diff --git a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c index 5273dc8498381ce09e878894f4eb56263900be39..945fe64f835cc6094f1880ea20f20584de74a464 100644 --- a/drivers/gpu/drm/msm/adreno/a3xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a3xx_gpu.c @@ -40,7 +40,7 @@ static void a3xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: /* ignore if there has not been a ctx switch: */ - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: diff --git a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c index 8b4cdf95f4453bb76e7efb93d86080ef678c9f68..50c490b492f08a1a7ebfe33b2f206cafd91a84ba 100644 --- a/drivers/gpu/drm/msm/adreno/a4xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a4xx_gpu.c @@ -34,7 +34,7 @@ static void a4xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: /* ignore if there has not been a ctx switch: */ - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c index c0b5373e90d7139caa023aec6f272545456acb0a..80b441fe8e3a823c5bd1f74cd8c7bb418d0674fb 100644 --- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c @@ -75,7 +75,7 @@ static void a5xx_submit_in_rb(struct msm_gpu *gpu, struct msm_gem_submit *submit case MSM_SUBMIT_CMD_IB_TARGET_BUF: break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: @@ -129,7 +129,7 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) unsigned int i, ibs = 0; if (IS_ENABLED(CONFIG_DRM_MSM_GPU_SUDO) && submit->in_rb) { - gpu->cur_ctx_seqno = 0; + ring->cur_ctx_seqno = 0; a5xx_submit_in_rb(gpu, submit); return; } @@ -164,7 +164,7 @@ static void a5xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) case MSM_SUBMIT_CMD_IB_TARGET_BUF: break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 32a4faa93d7f072ea6b8d949f4dc9d2a58cec6b9..6e065500b64d6d95599d89c33e6703c92f210047 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -109,7 +109,7 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu, u32 asid; u64 memptr = rbmemptr(ring, ttbr0); - if (ctx->seqno == a6xx_gpu->base.base.cur_ctx_seqno) + if (ctx->seqno == ring->cur_ctx_seqno) return; if (msm_iommu_pagetable_params(ctx->aspace->mmu, &ttbr, &asid)) @@ -219,7 +219,7 @@ static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) case MSM_SUBMIT_CMD_IB_TARGET_BUF: break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: @@ -305,7 +305,7 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) case MSM_SUBMIT_CMD_IB_TARGET_BUF: break; case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: - if (gpu->cur_ctx_seqno == submit->queue->ctx->seqno) + if (ring->cur_ctx_seqno == submit->queue->ctx->seqno) break; fallthrough; case MSM_SUBMIT_CMD_BUF: @@ -843,6 +843,7 @@ static int hw_init(struct msm_gpu *gpu) struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); struct a6xx_gmu *gmu = &a6xx_gpu->gmu; u64 gmem_range_min; + unsigned int i; int ret; if (!adreno_has_gmu_wrapper(adreno_gpu)) { @@ -1138,7 +1139,8 @@ static int hw_init(struct msm_gpu *gpu) /* Always come up on rb 0 */ a6xx_gpu->cur_ring = gpu->rb[0]; - gpu->cur_ctx_seqno = 0; + for (i = 0; i < gpu->nr_rings; i++) + gpu->rb[i]->cur_ctx_seqno = 0; /* Enable the SQE_to start the CP engine */ gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1); diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index 3666b42b4ecd7f91b24302d7f229eeefdc3c39b7..c063b3896dc1c193e41b8fc380a91a9076376811 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -783,7 +783,7 @@ void msm_gpu_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) mutex_unlock(&gpu->active_lock); gpu->funcs->submit(gpu, submit); - gpu->cur_ctx_seqno = submit->queue->ctx->seqno; + submit->ring->cur_ctx_seqno = submit->queue->ctx->seqno; pm_runtime_put(&gpu->pdev->dev); hangcheck_timer_reset(gpu); diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index 1f02bb9956be2720a2760646ccdf92f8bead7dd0..7cabc8480d7c5461ab8d8726fcc21690cbaf7366 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -193,17 +193,6 @@ struct msm_gpu { */ refcount_t sysprof_active; - /** - * cur_ctx_seqno: - * - * The ctx->seqno value of the last context to submit rendering, - * and the one with current pgtables installed (for generations - * that support per-context pgtables). Tracked by seqno rather - * than pointer value to avoid dangling pointers, and cases where - * a ctx can be freed and a new one created with the same address. - */ - int cur_ctx_seqno; - /** * lock: * diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.h b/drivers/gpu/drm/msm/msm_ringbuffer.h index 40791b2ade46ef0e16e2a4088291a575d3be9e82..174f83137a49940ec80b1fbf548e214fa3c32784 100644 --- a/drivers/gpu/drm/msm/msm_ringbuffer.h +++ b/drivers/gpu/drm/msm/msm_ringbuffer.h @@ -100,6 +100,16 @@ struct msm_ringbuffer { * preemption. Can be aquired from irq context. */ spinlock_t preempt_lock; + + /** + * cur_ctx_seqno: + * + * The ctx->seqno value of the last context to submit to this ring + * Tracked by seqno rather than pointer value to avoid dangling + * pointers, and cases where a ctx can be freed and a new one created + * with the same address. + */ + int cur_ctx_seqno; }; struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id, From patchwork Mon Sep 30 13:52:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 13816504 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1FB9DCE8348 for ; Mon, 30 Sep 2024 13:52:51 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C697C10E505; Mon, 30 Sep 2024 13:52:49 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="O+o2hFjb"; dkim-atps=neutral Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by gabe.freedesktop.org (Postfix) with ESMTPS id E4A6C10E504; Mon, 30 Sep 2024 13:52:46 +0000 (UTC) Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-5c721803a89so5750586a12.1; Mon, 30 Sep 2024 06:52:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727704365; x=1728309165; darn=lists.freedesktop.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=wyb3geUIdOaiHlo8mwWpCyJKar3stgK5wmtiUF5Pomg=; b=O+o2hFjbXTxSMT9hufFnjlInph08S2DiA2JgoqlEm+g8S3hJo308WnQ0hdeeGSwGP4 M1YXHVAmmQRy3WZFWS/UkYtodNY/WXlABUzx3xnRjeBBjADLwwCaKPbdPezxvxWdGYei 3xqyd8zkkDIhynIEqBc7UNOPZURdG/wV0mCm3xmpS6WpETBf/Y5/UckDyZ2b9gknd4UQ ym+6tfOarmztXoDPYqLW29YED/6MVGmyFMSIm2PFxPneyVaNwkm06GUwVqzYe8ic2Nki 4Uhrc5WZ/qO4maSew7uCv5wQEuaFnP0cpoHv4wPEtViLP7cKTIgNFylSj7A7MkQyafy4 M82g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727704365; x=1728309165; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=wyb3geUIdOaiHlo8mwWpCyJKar3stgK5wmtiUF5Pomg=; b=mjjfy8fM3L8FgHDGzvOq9vKW7hRw1maRHf+mjkKFXLfBcSsbLXAX3jni2gAqdD0D0o mYgTck6lekfAif8VPHNRKCjHbbHAK+Cmldp4rznsuW2IaXL4CMXkLLuRVLNIB1o53MB7 V/KLzSWgF0XORCyeUP+7AoBPmlt+eRe98A1tc22/iRdbo9fFDKVwJ1GTtSCy6PDoGpxM fHpwE3WbnXqQULG6NzM6QkjucK/QWIqyh9accRAScuQOK6f7RquZ74PIQWo1qEVs08Gg Tz7hePq50UtsoABaKsrxVpNk0T6cVtA53nWuO5KXpcuEslMM7OV2P5HazHs+vW8LMMM8 10vQ== X-Forwarded-Encrypted: i=1; AJvYcCUJlWmLyw/JLnWh6aKe845zTe8KKzyyL3oXzobRSFsQ/GHXhPyDd6TjrzOIs+M1hKhYnKHxDXJuyaM=@lists.freedesktop.org, AJvYcCWsES4D6gSp+R+Go3O1gq7mZ597gK/Q66doe7YbIIJ8PcxWyBoxAVCtTmwVK3zcyJYzARbtGCAqL020@lists.freedesktop.org X-Gm-Message-State: AOJu0Yw1phrnx4k13UZ0CRgfPX+253Jq4u7EIeca9NewrJnnztO4nUBM urmx1tptPA6w6nGjyBcYo/8urEkxdi+F3TpkvOWjqeHayMHpHit+ X-Google-Smtp-Source: AGHT+IGTeHDATKcDuVJkAlYhRG53MODRQ/p+tRk29qkIyg/dqn9ovQDdtmiJ5QOZiPHQDuGxMOcuvA== X-Received: by 2002:a05:6402:2425:b0:5c7:227f:3a5b with SMTP id 4fb4d7f45d1cf-5c8826035e5mr12035835a12.26.1727704365016; Mon, 30 Sep 2024 06:52:45 -0700 (PDT) Received: from [192.168.1.17] (host-95-250-55-22.retail.telecomitalia.it. [95.250.55.22]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88245ea57sm4507540a12.55.2024.09.30.06.52.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 06:52:44 -0700 (PDT) From: Antonino Maniscalco Date: Mon, 30 Sep 2024 15:52:38 +0200 Subject: [PATCH v7 03/12] drm/msm: Add a `preempt_record_size` field MIME-Version: 1.0 Message-Id: <20240930-preemption-a750-t-v7-3-47803c7a5a64@gmail.com> References: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> In-Reply-To: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Akhil P Oommen , Neil Armstrong X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727704357; l=2480; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=mw98QEzBIwYx44vG8hfTrLa/RurlCEDImHR+oHkKWF4=; b=og/agVTW9E09HAuOJIvQI1XqSludyePluz9FVZDA7lFv7IrrkDYKFjq9owaaLYx2jnHfnFAc8 SjZSH6hyj3YC15gvuFH09l6O9vT7KbmtLfnTyRh+zUqIa4LVk8IWu5d X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Adds a field to `adreno_info` to store the GPU specific preempt record size. Reviewed-by: Akhil P Oommen Tested-by: Rob Clark Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 4 ++++ drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 + 2 files changed, 5 insertions(+) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c index 68ba9aed5506ea2f367ff0282a73fdd1122f2526..316f23ca91671d973797f2a5b69344f376707325 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c @@ -1190,6 +1190,7 @@ static const struct adreno_info a7xx_gpus[] = { .protect = &a730_protect, }, .address_space_size = SZ_16G, + .preempt_record_size = 2860 * SZ_1K, }, { .chip_ids = ADRENO_CHIP_IDS(0x43050a01), /* "C510v2" */ .family = ADRENO_7XX_GEN2, @@ -1209,6 +1210,7 @@ static const struct adreno_info a7xx_gpus[] = { .gmu_chipid = 0x7020100, }, .address_space_size = SZ_16G, + .preempt_record_size = 4192 * SZ_1K, }, { .chip_ids = ADRENO_CHIP_IDS(0x43050c01), /* "C512v2" */ .family = ADRENO_7XX_GEN2, @@ -1227,6 +1229,7 @@ static const struct adreno_info a7xx_gpus[] = { .gmu_chipid = 0x7050001, }, .address_space_size = SZ_256G, + .preempt_record_size = 4192 * SZ_1K, }, { .chip_ids = ADRENO_CHIP_IDS(0x43051401), /* "C520v2" */ .family = ADRENO_7XX_GEN3, @@ -1245,6 +1248,7 @@ static const struct adreno_info a7xx_gpus[] = { .gmu_chipid = 0x7090100, }, .address_space_size = SZ_16G, + .preempt_record_size = 3572 * SZ_1K, } }; DECLARE_ADRENO_GPULIST(a7xx); diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h index 1ab523a163a00b333a85b099e9eb9209e6a2e646..6b1888280a83e6288c4b071733d5d6097afe3a99 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h @@ -111,6 +111,7 @@ struct adreno_info { * {SHRT_MAX, 0} sentinal. */ struct adreno_speedbin *speedbins; + u64 preempt_record_size; }; #define ADRENO_CHIP_IDS(tbl...) (uint32_t[]) { tbl, 0 } From patchwork Mon Sep 30 13:52:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 13816505 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 89132CE8349 for ; Mon, 30 Sep 2024 13:52:52 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 39D3410E50B; Mon, 30 Sep 2024 13:52:50 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="LraVYW5C"; dkim-atps=neutral Received: from mail-lj1-f171.google.com (mail-lj1-f171.google.com [209.85.208.171]) by gabe.freedesktop.org (Postfix) with ESMTPS id E1D4010E503; Mon, 30 Sep 2024 13:52:48 +0000 (UTC) Received: by mail-lj1-f171.google.com with SMTP id 38308e7fff4ca-2fabfc06c26so18234121fa.2; Mon, 30 Sep 2024 06:52:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727704367; x=1728309167; darn=lists.freedesktop.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=AkSz1D5Hr/cIydOboRgyQ/kGqlozWFahzIe7ud0a0O8=; b=LraVYW5CE/X8Le15oI4fWBS+ncNb2+2/tzVDmqM16akyj8Qc2opfp46f8k/YkQrBBX Oo/ThzrZwT1H0224wdZbhJ+8O2heMHN8loD58E2arUQbOMdLgP75zESL8aS6fIoTKYRg CMQnR/OrOgivZDu0s99R9MWXH+TwGkBxzI0ERl6TzRyGNwCmVjiqSxvgkeVHi1c70FQy YM57KgadBJ/1iUhluV4scHFAafW5Tko7wEBQyNBwfA8YK8N1e/27Z5udAMqqtoJoZ2q3 b/ySe+u0i3LaKzmqM0n/fSiSyistOlrjXpIuYCP3HQ1dCske1tgkeJ0VRBBZNWIuRth6 NPUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727704367; x=1728309167; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=AkSz1D5Hr/cIydOboRgyQ/kGqlozWFahzIe7ud0a0O8=; b=GrMt+ssWVm8GvCXFZTlimbsYabTOR6e0nGkJTGBo7qv3ranTvTUirSjM28UTRf3DWF uJ3Z0Civ1/pwjssrKnz4Gm2CpIXNcjEoLjBPwBNtdkTJpoQhJDQLtisRnwIPKVRikNZ7 Z/G1cnvocwVpAgo1NEnN4/XiP68v7YwVJiU4chOQ45C9wRPWlHMDvttUimWAHjv5haxM GUbJyiFqxmSBxyw4FVTKpPVwFmMWl8NxNqKGxaJX+hUXn0ZdXd0N7gblpT9YZxbwx64U 4as8XeBnyjIbifCkVS3S0R7v3cjXGypwzdKQfktRWKXGfIH3lTWP8CT3wglNcHmxl2fg RXmQ== X-Forwarded-Encrypted: i=1; AJvYcCU8kDfVHwWk+EyXyaN12F6DwwVvKGCY6apVg4o4SQen2/IvBZ0Fow46YbxjV0oHS+iHg+/4K6QR4rp7@lists.freedesktop.org, AJvYcCWZn4wPeurgnI0kllMO7kLM16OyL93H0QXg2M9QJZ4EU1rJkLCWXz1RhNi5JfVfbbOuuDxh5yJcnKE=@lists.freedesktop.org X-Gm-Message-State: AOJu0YzGHMSZUAWXhc4dGBMs/klXQ43LA99DfV3rYAbpvdB6Sf8aOHu8 74F7JKF2iYvXuV9Mnd4GKQHAAOV4yW8UoFXBfLR4rebpSVgD5llX X-Google-Smtp-Source: AGHT+IHFWGcngO0Ta4UMRaZut6aiuFnh2a+knY7824+0F//PEHWOi3H8hOBcbL8yQAmt+yHj298wYA== X-Received: by 2002:a2e:802:0:b0:2f7:53b8:ca57 with SMTP id 38308e7fff4ca-2f9d3e55c41mr46525921fa.19.1727704366634; Mon, 30 Sep 2024 06:52:46 -0700 (PDT) Received: from [192.168.1.17] (host-95-250-55-22.retail.telecomitalia.it. [95.250.55.22]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88245ea57sm4507540a12.55.2024.09.30.06.52.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 06:52:46 -0700 (PDT) From: Antonino Maniscalco Date: Mon, 30 Sep 2024 15:52:39 +0200 Subject: [PATCH v7 04/12] drm/msm: Add CONTEXT_SWITCH_CNTL bitfields MIME-Version: 1.0 Message-Id: <20240930-preemption-a750-t-v7-4-47803c7a5a64@gmail.com> References: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> In-Reply-To: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Neil Armstrong X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727704357; l=1523; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=95uWYhYXrJwsS/Si/bR6gRgvDdDmC6nz9iFGaH/gMIk=; b=x48nLkZ298lfcSnmRx8/BH6OnDPu2DerFLczl32o6cWttIfpIJb8/vtPTH9eRQXD2sgAhA6Ik qWu8XBcqR6ODjpn2Y2egn9FNzdTNkI9N14kPii6AULXWcqGESZ4Ggbw X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add missing bitfields to CONTEXT_SWITCH_CNTL in a6xx.xml. Tested-by: Rob Clark Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/registers/adreno/a6xx.xml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/msm/registers/adreno/a6xx.xml b/drivers/gpu/drm/msm/registers/adreno/a6xx.xml index 2dfe6913ab4f52449b76c2f75b2d101c08115d16..fd31d1d7a11eef7f38dcc2975dc1034f6b7a2e41 100644 --- a/drivers/gpu/drm/msm/registers/adreno/a6xx.xml +++ b/drivers/gpu/drm/msm/registers/adreno/a6xx.xml @@ -1337,7 +1337,12 @@ to upconvert to 32b float internally? - + + + + + + From patchwork Mon Sep 30 13:52:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 13816510 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4A192CE834A for ; Mon, 30 Sep 2024 13:53:06 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 587C710E510; Mon, 30 Sep 2024 13:53:05 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="a9hxcfW6"; dkim-atps=neutral Received: from mail-ed1-f51.google.com (mail-ed1-f51.google.com [209.85.208.51]) by gabe.freedesktop.org (Postfix) with ESMTPS id 3851F10E50A; Mon, 30 Sep 2024 13:52:50 +0000 (UTC) Received: by mail-ed1-f51.google.com with SMTP id 4fb4d7f45d1cf-5c883459b19so3727555a12.2; Mon, 30 Sep 2024 06:52:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727704368; x=1728309168; darn=lists.freedesktop.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=/V/6a2TW5ipcM35DEEW7Cwm4LM4Qc3jfg/uzlisxQGI=; b=a9hxcfW6t303WCMMPdckDOozgQ2qA4EuwfBfVX/oXEr78UPWEmrqWoPTQjoph8QNnZ XIY28ctQcA/iCtaihIjsp8OzydxdJ0MtHk9sqglGaDLwku6boa7L6+Bk5FsHww/yR0yf /LMOSQLaMDR0wubdCPgnels9ufQIt/AiEIf7FBZ1ZtpJ9CG6VUb9FptV2GQ0PU7OE6kG T6CtcWwDVoVQz/edwLjqTMv6bt3j2eIa1xCqqWhTW4oNBRPIPzYchMeYqU6CSvP09wOQ ysLbfkyHRugN5WEvJboQ+rQHtN6kJCllo4/JIE+fW6//EEYtr339gxUaLB8/HtBF3U6j sUdg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727704368; x=1728309168; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=/V/6a2TW5ipcM35DEEW7Cwm4LM4Qc3jfg/uzlisxQGI=; b=lxrfeE/POg3T127ykCsm1/6BVGG54ioT12hp0FQpWA+1uhsmn7jKm6TTSY66OX+UUl sCSgeflzj0a2ukit2aIjeHUD+qATrMwLoYaso1d/iSClygbzOMkA5bQOCp+jZko/jb54 3XkybKg7AUeQkMuI03aqWLiRr6Xn975qFiizCxEFLcNPgmpA6PwLKHbGLvbwZq2cos3V OpqvkYoctHmDbMlSXgyfG/1xWtQ22xK0X83RCbo6NAsykP0mmMMQY6qDC4Y7zFzFRXHD 82NhsDMiAWpCLRcOLG+RSNLLPt6SFhcz7PbxNzMdZ5UsxQKsEeNNRDdvenQnF3CCUaxP GhBQ== X-Forwarded-Encrypted: i=1; AJvYcCU8u3Zz1ab3KWOvSNlIapBRjxwhhJYNmquFThr610tXWBCRclq816z5+kM7o317ICvJj31SHZl7lF5Y@lists.freedesktop.org, AJvYcCW8jUoTbDaDdZvqjcypNREeFNy6aQJSlMVyudU0zrdAygM/h8bUjgf1GPFLlHsjCtu/ZrQ18AvGgig=@lists.freedesktop.org X-Gm-Message-State: AOJu0YzBmFGXuQeBvOImr9tc4CkIzJ9XbYrLB98k0OhZobihCsF5An3R 0lWWJ0tFcl9c2cWlCqSczt7HkR0hnNmXJJwbowQeq3xsA0hljP4E X-Google-Smtp-Source: AGHT+IH2aChTJSUVF3XESy19JSmQKMiEKzpArEt+3XVCFugQyJajyfAyptI74iVTiYJ0M9Lxu6TJkQ== X-Received: by 2002:a05:6402:34d6:b0:5c8:8668:e564 with SMTP id 4fb4d7f45d1cf-5c88668e6ccmr7847854a12.30.1727704368209; Mon, 30 Sep 2024 06:52:48 -0700 (PDT) Received: from [192.168.1.17] (host-95-250-55-22.retail.telecomitalia.it. [95.250.55.22]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88245ea57sm4507540a12.55.2024.09.30.06.52.46 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 06:52:47 -0700 (PDT) From: Antonino Maniscalco Date: Mon, 30 Sep 2024 15:52:40 +0200 Subject: [PATCH v7 05/12] drm/msm/a6xx: Add a pwrup_list field to a6xx_info MIME-Version: 1.0 Message-Id: <20240930-preemption-a750-t-v7-5-47803c7a5a64@gmail.com> References: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> In-Reply-To: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727704357; l=4169; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=7OY9AiaQ19Ia+PqqCV6YZvq+XDpZw+wy+7bMWhQ2Bvs=; b=Z9vJl6mXetFLxPDYS9eNlCYXOWz7jjXIDUi6wzwP7IWRkBvq8tuFHo0j6a61bkHl0fKUQ8rcx sJ+3DQpAAd3B+WaLXO8oqY7Q2XRbkH4E3wsnKpPCg0RzHqjHK8krxjK X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" Add a field to contain the pwup_reglist needed for preemption. Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/adreno/a6xx_catalog.c | 26 ++++++++++++++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 2 ++ drivers/gpu/drm/msm/adreno/adreno_gpu.h | 13 +++++++++++++ 3 files changed, 41 insertions(+) diff --git a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c index 316f23ca91671d973797f2a5b69344f376707325..e4d271fa89cc66f188be04206e267fabd83cca83 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_catalog.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_catalog.c @@ -1150,6 +1150,28 @@ static const u32 a730_protect_regs[] = { }; DECLARE_ADRENO_PROTECT(a730_protect, 48); +static const uint32_t a7xx_pwrup_reglist_regs[] = { + REG_A6XX_UCHE_TRAP_BASE, + REG_A6XX_UCHE_TRAP_BASE + 1, + REG_A6XX_UCHE_WRITE_THRU_BASE, + REG_A6XX_UCHE_WRITE_THRU_BASE + 1, + REG_A6XX_UCHE_GMEM_RANGE_MIN, + REG_A6XX_UCHE_GMEM_RANGE_MIN + 1, + REG_A6XX_UCHE_GMEM_RANGE_MAX, + REG_A6XX_UCHE_GMEM_RANGE_MAX + 1, + REG_A6XX_UCHE_CACHE_WAYS, + REG_A6XX_UCHE_MODE_CNTL, + REG_A6XX_RB_NC_MODE_CNTL, + REG_A6XX_RB_CMP_DBG_ECO_CNTL, + REG_A7XX_GRAS_NC_MODE_CNTL, + REG_A6XX_RB_CONTEXT_SWITCH_GMEM_SAVE_RESTORE, + REG_A6XX_UCHE_GBIF_GX_CONFIG, + REG_A6XX_UCHE_CLIENT_PF, + REG_A6XX_TPL1_DBG_ECO_CNTL1, +}; + +DECLARE_ADRENO_REGLIST_LIST(a7xx_pwrup_reglist); + static const struct adreno_info a7xx_gpus[] = { { .chip_ids = ADRENO_CHIP_IDS(0x07000200), @@ -1188,6 +1210,7 @@ static const struct adreno_info a7xx_gpus[] = { .a6xx = &(const struct a6xx_info) { .hwcg = a730_hwcg, .protect = &a730_protect, + .pwrup_reglist = &a7xx_pwrup_reglist, }, .address_space_size = SZ_16G, .preempt_record_size = 2860 * SZ_1K, @@ -1207,6 +1230,7 @@ static const struct adreno_info a7xx_gpus[] = { .a6xx = &(const struct a6xx_info) { .hwcg = a740_hwcg, .protect = &a730_protect, + .pwrup_reglist = &a7xx_pwrup_reglist, .gmu_chipid = 0x7020100, }, .address_space_size = SZ_16G, @@ -1226,6 +1250,7 @@ static const struct adreno_info a7xx_gpus[] = { .a6xx = &(const struct a6xx_info) { .hwcg = a740_hwcg, .protect = &a730_protect, + .pwrup_reglist = &a7xx_pwrup_reglist, .gmu_chipid = 0x7050001, }, .address_space_size = SZ_256G, @@ -1245,6 +1270,7 @@ static const struct adreno_info a7xx_gpus[] = { .zapfw = "gen70900_zap.mbn", .a6xx = &(const struct a6xx_info) { .protect = &a730_protect, + .pwrup_reglist = &a7xx_pwrup_reglist, .gmu_chipid = 0x7090100, }, .address_space_size = SZ_16G, diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h index e3e5c53ae8af2cc59a21160f05c59fd125cb94b1..2f5f307ac3b6845197b06ca3754abc88d698007c 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h @@ -17,10 +17,12 @@ extern bool hang_debug; * * @hwcg: hw clock gating register sequence * @protect: CP_PROTECT settings + * @pwrup_reglist pwrup reglist for preemption */ struct a6xx_info { const struct adreno_reglist *hwcg; const struct adreno_protect *protect; + const struct adreno_reglist_list *pwrup_reglist; u32 gmu_chipid; }; diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h index 6b1888280a83e6288c4b071733d5d6097afe3a99..58c63d0fbbff9818393fc62ee3cf2703365bec23 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h @@ -157,6 +157,19 @@ static const struct adreno_protect name = { \ .count_max = __count_max, \ }; +struct adreno_reglist_list { + /** @reg: List of register **/ + const u32 *regs; + /** @count: Number of registers in the list **/ + u32 count; +}; + +#define DECLARE_ADRENO_REGLIST_LIST(name) \ +static const struct adreno_reglist_list name = { \ + .regs = name ## _regs, \ + .count = ARRAY_SIZE(name ## _regs), \ +}; + struct adreno_gpu { struct msm_gpu base; const struct adreno_info *info; From patchwork Mon Sep 30 13:52:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 13816507 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 40A22CE8347 for ; Mon, 30 Sep 2024 13:52:58 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id C06F510E502; Mon, 30 Sep 2024 13:52:57 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="SE8hYuEr"; dkim-atps=neutral Received: from mail-lj1-f174.google.com (mail-lj1-f174.google.com [209.85.208.174]) by gabe.freedesktop.org (Postfix) with ESMTPS id 1AC0910E4FE; Mon, 30 Sep 2024 13:52:53 +0000 (UTC) Received: by mail-lj1-f174.google.com with SMTP id 38308e7fff4ca-2fabfc06de3so20564251fa.1; Mon, 30 Sep 2024 06:52:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727704371; x=1728309171; darn=lists.freedesktop.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=OGm16tBc/mYoL2p/kWnBQEKJKKAd6HTwsmMI5nBn7Bk=; b=SE8hYuErN9Q1KZFbTU2kyT1ioMJ7KO+DT7STk4A7wN3wnbb5yRHPU+apyJVq0+Ws7+ iZpZFoOOXWBAYl80gdFxY+mybJCT1+HmIkXO5xL/o7Vks/gLIxye6hRS/1TgQi1XcjcH fsVY4vKcl+4TebwnTFjdevC/GiZ14bN1tUnXVXxbagTfJ3XS6NwSElZPksYEtjMfLnHv 95ueedHUSwVWm3Sac1zLP8hj4ktIDMysCc0vUIy6adgP7hjDn3m3nOvCEPBHNRcP1EP4 Nc/Z98QWiTVpJ3vE61h77nUJnD2y+zWOGMxkDeZBIMFK21Ny68WcOyhmZReSKwi3l7EW XR1g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727704371; x=1728309171; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OGm16tBc/mYoL2p/kWnBQEKJKKAd6HTwsmMI5nBn7Bk=; b=BvPRuYBhynfSNBgsMyX597D1KzdLhGniIzcHkcQCcUC6MccL/J6BuVeA2h6cEXAX0t kGKlUEK8Cl0pW4NcY6gQppEyO3JACpkpcwa98Ru9wGRiLNq4SsqS0KdhAMhD2KvxsaAy 5bLW86MTRkxol63fM+Z8VqjNE/i3v7w2at8L4ns61xdWOqWoUtH0XtN4t0+rCrrIWik9 O2CKFrmT3Gyu9q5Vc/z7OT7g+C9ILAsqsoIOhFRm6nz0ondeLnqFOWKwHZ/o3gC2IiIU qiDRxptop8TpafJynFLgbLE1r4KVp0OVeTnhvqkHtmIFwWHZGknhqr2CHX1CLl/u8BhR 7dhQ== X-Forwarded-Encrypted: i=1; AJvYcCU/ViGRa+Y9tFlnQRS42m1sFrYUxXRdORIgzujnCAEcJxcQ2tQ3fK+mOEiTwonPutoG0m4zqnQuw9SW@lists.freedesktop.org, AJvYcCVyucyt5rAv4LgnituF4vYfjUSlE4LE2Os22aV6ZqbP1o74Uv/dXIxZhW0EXMaeWUyGIMKOM8T6huo=@lists.freedesktop.org X-Gm-Message-State: AOJu0Yxwp+xFLuURU4vbLwjPLl8SkjlnKwOs9tqEHjdsYVitY1+VivJt t6JlVEglOT+W6f3vWu4c67glWHtbdKacWELF45FSg94oLbMmRIGD X-Google-Smtp-Source: AGHT+IG7bV2DAxnqwnV5zDHiZf4f+FtwvPOqpa4eN9l/50E7XeohzkrTOwPtMFjG5QlhjsfoMn8/yQ== X-Received: by 2002:a2e:e1a:0:b0:2f6:484d:cd61 with SMTP id 38308e7fff4ca-2f9d41db37cmr50589861fa.43.1727704370735; Mon, 30 Sep 2024 06:52:50 -0700 (PDT) Received: from [192.168.1.17] (host-95-250-55-22.retail.telecomitalia.it. [95.250.55.22]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88245ea57sm4507540a12.55.2024.09.30.06.52.48 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 06:52:49 -0700 (PDT) From: Antonino Maniscalco Date: Mon, 30 Sep 2024 15:52:41 +0200 Subject: [PATCH v7 06/12] drm/msm/a6xx: Implement preemption for a7xx targets MIME-Version: 1.0 Message-Id: <20240930-preemption-a750-t-v7-6-47803c7a5a64@gmail.com> References: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> In-Reply-To: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Akhil P Oommen , Neil Armstrong , Sharat Masetty X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727704357; l=32233; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=la2yYs2rvztdmbAN8YutY92BLRLI4ynYhLnG47NiSsY=; b=3scw0JROkq0O1kpr6p7ZjZ+G5hhCqgiGKCwcA659Ao1jZHMpD6vQf0SUZhBHM88769MQGYRH/ aJrY0nLrVdUAdC8KvXG5Pgob+T5UulNX/q4KPOLEmKlL/riuFRP5ETj X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" This patch implements preemption feature for A6xx targets, this allows the GPU to switch to a higher priority ringbuffer if one is ready. A6XX hardware as such supports multiple levels of preemption granularities, ranging from coarse grained(ringbuffer level) to a more fine grained such as draw-call level or a bin boundary level preemption. This patch enables the basic preemption level, with more fine grained preemption support to follow. Reviewed-by: Akhil P Oommen Tested-by: Rob Clark Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Sharat Masetty Signed-off-by: Antonino Maniscalco --- drivers/gpu/drm/msm/Makefile | 1 + drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 193 ++++++++++++++- drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 162 ++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 393 ++++++++++++++++++++++++++++++ drivers/gpu/drm/msm/msm_ringbuffer.h | 7 + 5 files changed, 745 insertions(+), 11 deletions(-) diff --git a/drivers/gpu/drm/msm/Makefile b/drivers/gpu/drm/msm/Makefile index f5e2838c6a76505b353f83c9fe9c997f1c282701..32e915109a59dda96ed76ddd2b4f57bb225f4572 100644 --- a/drivers/gpu/drm/msm/Makefile +++ b/drivers/gpu/drm/msm/Makefile @@ -23,6 +23,7 @@ adreno-y := \ adreno/a6xx_gpu.o \ adreno/a6xx_gmu.o \ adreno/a6xx_hfi.o \ + adreno/a6xx_preempt.o \ adreno-$(CONFIG_DEBUG_FS) += adreno/a5xx_debugfs.o \ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c index 6e065500b64d6d95599d89c33e6703c92f210047..b4cd562f1dd8660c77430e1aeac7e4b7af32bfc7 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -68,6 +68,8 @@ static void update_shadow_rptr(struct msm_gpu *gpu, struct msm_ringbuffer *ring) static void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring) { + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); uint32_t wptr; unsigned long flags; @@ -81,12 +83,17 @@ static void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring) /* Make sure to wrap wptr if we need to */ wptr = get_wptr(ring); - spin_unlock_irqrestore(&ring->preempt_lock, flags); - - /* Make sure everything is posted before making a decision */ - mb(); + /* Update HW if this is the current ring and we are not in preempt*/ + if (!a6xx_in_preempt(a6xx_gpu)) { + if (a6xx_gpu->cur_ring == ring) + gpu_write(gpu, REG_A6XX_CP_RB_WPTR, wptr); + else + ring->restore_wptr = true; + } else { + ring->restore_wptr = true; + } - gpu_write(gpu, REG_A6XX_CP_RB_WPTR, wptr); + spin_unlock_irqrestore(&ring->preempt_lock, flags); } static void get_stats_counter(struct msm_ringbuffer *ring, u32 counter, @@ -138,12 +145,14 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu, /* * Write the new TTBR0 to the memstore. This is good for debugging. + * Needed for preemption */ - OUT_PKT7(ring, CP_MEM_WRITE, 4); + OUT_PKT7(ring, CP_MEM_WRITE, 5); OUT_RING(ring, CP_MEM_WRITE_0_ADDR_LO(lower_32_bits(memptr))); OUT_RING(ring, CP_MEM_WRITE_1_ADDR_HI(upper_32_bits(memptr))); OUT_RING(ring, lower_32_bits(ttbr)); - OUT_RING(ring, (asid << 16) | upper_32_bits(ttbr)); + OUT_RING(ring, upper_32_bits(ttbr)); + OUT_RING(ring, ctx->seqno); /* * Sync both threads after switching pagetables and enable BR only @@ -268,6 +277,34 @@ static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) a6xx_flush(gpu, ring); } +static void a6xx_emit_set_pseudo_reg(struct msm_ringbuffer *ring, + struct a6xx_gpu *a6xx_gpu, struct msm_gpu_submitqueue *queue) +{ + OUT_PKT7(ring, CP_SET_PSEUDO_REG, 12); + + OUT_RING(ring, SMMU_INFO); + /* don't save SMMU, we write the record from the kernel instead */ + OUT_RING(ring, 0); + OUT_RING(ring, 0); + + /* privileged and non secure buffer save */ + OUT_RING(ring, NON_SECURE_SAVE_ADDR); + OUT_RING(ring, lower_32_bits( + a6xx_gpu->preempt_iova[ring->id])); + OUT_RING(ring, upper_32_bits( + a6xx_gpu->preempt_iova[ring->id])); + + /* user context buffer save, seems to be unnused by fw */ + OUT_RING(ring, NON_PRIV_SAVE_ADDR); + OUT_RING(ring, 0); + OUT_RING(ring, 0); + + OUT_RING(ring, COUNTER); + /* seems OK to set to 0 to disable it */ + OUT_RING(ring, 0); + OUT_RING(ring, 0); +} + static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) { unsigned int index = submit->seqno % MSM_GPU_SUBMIT_STATS_COUNT; @@ -285,6 +322,13 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) a6xx_set_pagetable(a6xx_gpu, ring, submit->queue->ctx); + /* + * If preemption is enabled, then set the pseudo register for the save + * sequence + */ + if (gpu->nr_rings > 1) + a6xx_emit_set_pseudo_reg(ring, a6xx_gpu, submit->queue); + get_stats_counter(ring, REG_A7XX_RBBM_PERFCTR_CP(0), rbmemptr_stats(ring, index, cpcycles_start)); get_stats_counter(ring, REG_A6XX_CP_ALWAYS_ON_COUNTER, @@ -376,6 +420,8 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) OUT_RING(ring, upper_32_bits(rbmemptr(ring, bv_fence))); OUT_RING(ring, submit->seqno); + a6xx_gpu->last_seqno[ring->id] = submit->seqno; + /* write the ringbuffer timestamp */ OUT_PKT7(ring, CP_EVENT_WRITE, 4); OUT_RING(ring, CACHE_CLEAN | CP_EVENT_WRITE_0_IRQ | BIT(27)); @@ -389,10 +435,32 @@ static void a7xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit) OUT_PKT7(ring, CP_SET_MARKER, 1); OUT_RING(ring, 0x100); /* IFPC enable */ + /* If preemption is enabled */ + if (gpu->nr_rings > 1) { + /* Yield the floor on command completion */ + OUT_PKT7(ring, CP_CONTEXT_SWITCH_YIELD, 4); + + /* + * If dword[2:1] are non zero, they specify an address for + * the CP to write the value of dword[3] to on preemption + * complete. Write 0 to skip the write + */ + OUT_RING(ring, 0x00); + OUT_RING(ring, 0x00); + /* Data value - not used if the address above is 0 */ + OUT_RING(ring, 0x01); + /* generate interrupt on preemption completion */ + OUT_RING(ring, 0x00); + } + + trace_msm_gpu_submit_flush(submit, gpu_read64(gpu, REG_A6XX_CP_ALWAYS_ON_COUNTER)); a6xx_flush(gpu, ring); + + /* Check to see if we need to start preemption */ + a6xx_preempt_trigger(gpu); } static void a6xx_set_hwcg(struct msm_gpu *gpu, bool state) @@ -588,6 +656,77 @@ static void a6xx_set_ubwc_config(struct msm_gpu *gpu) adreno_gpu->ubwc_config.min_acc_len << 23 | hbb_lo << 21); } +static void a7xx_patch_pwrup_reglist(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + const struct adreno_reglist_list *reglist; + void *ptr = a6xx_gpu->pwrup_reglist_ptr; + struct cpu_gpu_lock *lock = ptr; + u32 *dest = (u32 *)&lock->regs[0]; + int i; + + reglist = adreno_gpu->info->a6xx->pwrup_reglist; + + lock->gpu_req = lock->cpu_req = lock->turn = 0; + lock->ifpc_list_len = 0; + lock->preemption_list_len = reglist->count; + + /* + * For each entry in each of the lists, write the offset and the current + * register value into the GPU buffer + */ + for (i = 0; i < reglist->count; i++) { + *dest++ = reglist->regs[i]; + *dest++ = gpu_read(gpu, reglist->regs[i]); + } + + /* + * The overall register list is composed of + * 1. Static IFPC-only registers + * 2. Static IFPC + preemption registers + * 3. Dynamic IFPC + preemption registers (ex: perfcounter selects) + * + * The first two lists are static. Size of these lists are stored as + * number of pairs in ifpc_list_len and preemption_list_len + * respectively. With concurrent binning, Some of the perfcounter + * registers being virtualized, CP needs to know the pipe id to program + * the aperture inorder to restore the same. Thus, third list is a + * dynamic list with triplets as + * (
), and the length is + * stored as number for triplets in dynamic_list_len. + */ + lock->dynamic_list_len = 0; +} + +static int a7xx_preempt_start(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct msm_ringbuffer *ring = gpu->rb[0]; + + if (gpu->nr_rings <= 1) + return 0; + + /* Turn CP protection off */ + OUT_PKT7(ring, CP_SET_PROTECTED_MODE, 1); + OUT_RING(ring, 0); + + a6xx_emit_set_pseudo_reg(ring, a6xx_gpu, NULL); + + /* Yield the floor on command completion */ + OUT_PKT7(ring, CP_CONTEXT_SWITCH_YIELD, 4); + OUT_RING(ring, 0x00); + OUT_RING(ring, 0x00); + OUT_RING(ring, 0x00); + /* Generate interrupt on preemption completion */ + OUT_RING(ring, 0x00); + + a6xx_flush(gpu, ring); + + return a6xx_idle(gpu, ring) ? 0 : -EINVAL; +} + static int a6xx_cp_init(struct msm_gpu *gpu) { struct msm_ringbuffer *ring = gpu->rb[0]; @@ -619,6 +758,8 @@ static int a6xx_cp_init(struct msm_gpu *gpu) static int a7xx_cp_init(struct msm_gpu *gpu) { + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); struct msm_ringbuffer *ring = gpu->rb[0]; u32 mask; @@ -656,11 +797,11 @@ static int a7xx_cp_init(struct msm_gpu *gpu) /* *Don't* send a power up reg list for concurrent binning (TODO) */ /* Lo address */ - OUT_RING(ring, 0x00000000); + OUT_RING(ring, lower_32_bits(a6xx_gpu->pwrup_reglist_iova)); /* Hi address */ - OUT_RING(ring, 0x00000000); + OUT_RING(ring, upper_32_bits(a6xx_gpu->pwrup_reglist_iova)); /* BIT(31) set => read the regs from the list */ - OUT_RING(ring, 0x00000000); + OUT_RING(ring, BIT(31)); a6xx_flush(gpu, ring); return a6xx_idle(gpu, ring) ? 0 : -EINVAL; @@ -784,6 +925,16 @@ static int a6xx_ucode_load(struct msm_gpu *gpu) msm_gem_object_set_name(a6xx_gpu->shadow_bo, "shadow"); } + a6xx_gpu->pwrup_reglist_ptr = msm_gem_kernel_new(gpu->dev, PAGE_SIZE, + MSM_BO_WC | MSM_BO_MAP_PRIV, + gpu->aspace, &a6xx_gpu->pwrup_reglist_bo, + &a6xx_gpu->pwrup_reglist_iova); + + if (IS_ERR(a6xx_gpu->pwrup_reglist_ptr)) + return PTR_ERR(a6xx_gpu->pwrup_reglist_ptr); + + msm_gem_object_set_name(a6xx_gpu->pwrup_reglist_bo, "pwrup_reglist"); + return 0; } @@ -1128,6 +1279,8 @@ static int hw_init(struct msm_gpu *gpu) if (a6xx_gpu->shadow_bo) { gpu_write64(gpu, REG_A6XX_CP_RB_RPTR_ADDR, shadowptr(a6xx_gpu, gpu->rb[0])); + for (unsigned int i = 0; i < gpu->nr_rings; i++) + a6xx_gpu->shadow[i] = 0; } /* ..which means "always" on A7xx, also for BV shadow */ @@ -1136,6 +1289,8 @@ static int hw_init(struct msm_gpu *gpu) rbmemptr(gpu->rb[0], bv_rptr)); } + a6xx_preempt_hw_init(gpu); + /* Always come up on rb 0 */ a6xx_gpu->cur_ring = gpu->rb[0]; @@ -1145,6 +1300,11 @@ static int hw_init(struct msm_gpu *gpu) /* Enable the SQE_to start the CP engine */ gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1); + if (adreno_is_a7xx(adreno_gpu) && !a6xx_gpu->pwrup_reglist_emitted) { + a7xx_patch_pwrup_reglist(gpu); + a6xx_gpu->pwrup_reglist_emitted = true; + } + ret = adreno_is_a7xx(adreno_gpu) ? a7xx_cp_init(gpu) : a6xx_cp_init(gpu); if (ret) goto out; @@ -1182,6 +1342,10 @@ static int hw_init(struct msm_gpu *gpu) out: if (adreno_has_gmu_wrapper(adreno_gpu)) return ret; + + /* Last step - yield the ringbuffer */ + a7xx_preempt_start(gpu); + /* * Tell the GMU that we are done touching the GPU and it can start power * management @@ -1559,8 +1723,13 @@ static irqreturn_t a6xx_irq(struct msm_gpu *gpu) if (status & A6XX_RBBM_INT_0_MASK_SWFUSEVIOLATION) a7xx_sw_fuse_violation_irq(gpu); - if (status & A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS) + if (status & A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS) { msm_gpu_retire(gpu); + a6xx_preempt_trigger(gpu); + } + + if (status & A6XX_RBBM_INT_0_MASK_CP_SW) + a6xx_preempt_irq(gpu); return IRQ_HANDLED; } @@ -2333,6 +2502,8 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) a6xx_fault_handler); a6xx_calc_ubwc_config(adreno_gpu); + /* Set up the preemption specific bits and pieces for each ringbuffer */ + a6xx_preempt_init(gpu); return gpu; } diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h index 2f5f307ac3b6845197b06ca3754abc88d698007c..070bce058d3ff92aa7452b697763a16ab7e2faee 100644 --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h @@ -12,6 +12,24 @@ extern bool hang_debug; +struct cpu_gpu_lock { + uint32_t gpu_req; + uint32_t cpu_req; + uint32_t turn; + union { + struct { + uint16_t list_length; + uint16_t list_offset; + }; + struct { + uint8_t ifpc_list_len; + uint8_t preemption_list_len; + uint16_t dynamic_list_len; + }; + }; + uint64_t regs[62]; +}; + /** * struct a6xx_info - a6xx specific information from device table * @@ -33,6 +51,23 @@ struct a6xx_gpu { uint64_t sqe_iova; struct msm_ringbuffer *cur_ring; + struct msm_ringbuffer *next_ring; + + struct drm_gem_object *preempt_bo[MSM_GPU_MAX_RINGS]; + void *preempt[MSM_GPU_MAX_RINGS]; + uint64_t preempt_iova[MSM_GPU_MAX_RINGS]; + struct drm_gem_object *preempt_smmu_bo[MSM_GPU_MAX_RINGS]; + void *preempt_smmu[MSM_GPU_MAX_RINGS]; + uint64_t preempt_smmu_iova[MSM_GPU_MAX_RINGS]; + uint32_t last_seqno[MSM_GPU_MAX_RINGS]; + + atomic_t preempt_state; + spinlock_t eval_lock; + struct timer_list preempt_timer; + + unsigned int preempt_level; + bool uses_gmem; + bool skip_save_restore; struct a6xx_gmu gmu; @@ -40,6 +75,11 @@ struct a6xx_gpu { uint64_t shadow_iova; uint32_t *shadow; + struct drm_gem_object *pwrup_reglist_bo; + void *pwrup_reglist_ptr; + uint64_t pwrup_reglist_iova; + bool pwrup_reglist_emitted; + bool has_whereami; void __iomem *llc_mmio; @@ -51,6 +91,100 @@ struct a6xx_gpu { #define to_a6xx_gpu(x) container_of(x, struct a6xx_gpu, base) +/* + * In order to do lockless preemption we use a simple state machine to progress + * through the process. + * + * PREEMPT_NONE - no preemption in progress. Next state START. + * PREEMPT_START - The trigger is evaluating if preemption is possible. Next + * states: TRIGGERED, NONE + * PREEMPT_FINISH - An intermediate state before moving back to NONE. Next + * state: NONE. + * PREEMPT_TRIGGERED: A preemption has been executed on the hardware. Next + * states: FAULTED, PENDING + * PREEMPT_FAULTED: A preemption timed out (never completed). This will trigger + * recovery. Next state: N/A + * PREEMPT_PENDING: Preemption complete interrupt fired - the callback is + * checking the success of the operation. Next state: FAULTED, NONE. + */ + +enum a6xx_preempt_state { + PREEMPT_NONE = 0, + PREEMPT_START, + PREEMPT_FINISH, + PREEMPT_TRIGGERED, + PREEMPT_FAULTED, + PREEMPT_PENDING, +}; + +/* + * struct a6xx_preempt_record is a shared buffer between the microcode and the + * CPU to store the state for preemption. The record itself is much larger + * (2112k) but most of that is used by the CP for storage. + * + * There is a preemption record assigned per ringbuffer. When the CPU triggers a + * preemption, it fills out the record with the useful information (wptr, ring + * base, etc) and the microcode uses that information to set up the CP following + * the preemption. When a ring is switched out, the CP will save the ringbuffer + * state back to the record. In this way, once the records are properly set up + * the CPU can quickly switch back and forth between ringbuffers by only + * updating a few registers (often only the wptr). + * + * These are the CPU aware registers in the record: + * @magic: Must always be 0xAE399D6EUL + * @info: Type of the record - written 0 by the CPU, updated by the CP + * @errno: preemption error record + * @data: Data field in YIELD and SET_MARKER packets, Written and used by CP + * @cntl: Value of RB_CNTL written by CPU, save/restored by CP + * @rptr: Value of RB_RPTR written by CPU, save/restored by CP + * @wptr: Value of RB_WPTR written by CPU, save/restored by CP + * @_pad: Reserved/padding + * @rptr_addr: Value of RB_RPTR_ADDR_LO|HI written by CPU, save/restored by CP + * @rbase: Value of RB_BASE written by CPU, save/restored by CP + * @counter: GPU address of the storage area for the preemption counters + * @bv_rptr_addr: Value of BV_RB_RPTR_ADDR_LO|HI written by CPU, save/restored by CP + */ +struct a6xx_preempt_record { + u32 magic; + u32 info; + u32 errno; + u32 data; + u32 cntl; + u32 rptr; + u32 wptr; + u32 _pad; + u64 rptr_addr; + u64 rbase; + u64 counter; + u64 bv_rptr_addr; +}; + +#define A6XX_PREEMPT_RECORD_MAGIC 0xAE399D6EUL + +#define PREEMPT_SMMU_INFO_SIZE 4096 + +#define PREEMPT_RECORD_SIZE(adreno_gpu) \ + ((adreno_gpu->info->preempt_record_size) == 0 ? \ + 4192 * SZ_1K : (adreno_gpu->info->preempt_record_size)) + +/* + * The preemption counter block is a storage area for the value of the + * preemption counters that are saved immediately before context switch. We + * append it on to the end of the allocation for the preemption record. + */ +#define A6XX_PREEMPT_COUNTER_SIZE (16 * 4) + +struct a7xx_cp_smmu_info { + u32 magic; + u32 _pad4; + u64 ttbr0; + u32 asid; + u32 context_idr; + u32 context_bank; +}; + +#define GEN7_CP_SMMU_INFO_MAGIC 0x241350d5UL + /* * Given a register and a count, return a value to program into * REG_CP_PROTECT_REG(n) - this will block both reads and writes for @@ -108,6 +242,34 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node); int a6xx_gmu_wrapper_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node); void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu); +void a6xx_preempt_init(struct msm_gpu *gpu); +void a6xx_preempt_hw_init(struct msm_gpu *gpu); +void a6xx_preempt_trigger(struct msm_gpu *gpu); +void a6xx_preempt_irq(struct msm_gpu *gpu); +void a6xx_preempt_fini(struct msm_gpu *gpu); +int a6xx_preempt_submitqueue_setup(struct msm_gpu *gpu, + struct msm_gpu_submitqueue *queue); +void a6xx_preempt_submitqueue_close(struct msm_gpu *gpu, + struct msm_gpu_submitqueue *queue); + +/* Return true if we are in a preempt state */ +static inline bool a6xx_in_preempt(struct a6xx_gpu *a6xx_gpu) +{ + /* + * Make sure the read to preempt_state is ordered with respect to reads + * of other variables before ... + */ + smp_rmb(); + + int preempt_state = atomic_read(&a6xx_gpu->preempt_state); + + /* ... and after. */ + smp_rmb(); + + return !(preempt_state == PREEMPT_NONE || + preempt_state == PREEMPT_FINISH); +} + void a6xx_gmu_set_freq(struct msm_gpu *gpu, struct dev_pm_opp *opp, bool suspended); unsigned long a6xx_gmu_get_freq(struct msm_gpu *gpu); diff --git a/drivers/gpu/drm/msm/adreno/a6xx_preempt.c b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c new file mode 100644 index 0000000000000000000000000000000000000000..fd2a90360740591d45758f7311ec5cdda1855b20 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_preempt.c @@ -0,0 +1,393 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2018, The Linux Foundation. All rights reserved. */ +/* Copyright (c) 2023 Collabora, Ltd. */ +/* Copyright (c) 2024 Valve Corporation */ + +#include "msm_gem.h" +#include "a6xx_gpu.h" +#include "a6xx_gmu.xml.h" +#include "msm_mmu.h" + +/* + * Try to transition the preemption state from old to new. Return + * true on success or false if the original state wasn't 'old' + */ +static inline bool try_preempt_state(struct a6xx_gpu *a6xx_gpu, + enum a6xx_preempt_state old, enum a6xx_preempt_state new) +{ + enum a6xx_preempt_state cur = atomic_cmpxchg(&a6xx_gpu->preempt_state, + old, new); + + return (cur == old); +} + +/* + * Force the preemption state to the specified state. This is used in cases + * where the current state is known and won't change + */ +static inline void set_preempt_state(struct a6xx_gpu *gpu, + enum a6xx_preempt_state new) +{ + /* + * preempt_state may be read by other cores trying to trigger a + * preemption or in the interrupt handler so barriers are needed + * before... + */ + smp_mb__before_atomic(); + atomic_set(&gpu->preempt_state, new); + /* ... and after*/ + smp_mb__after_atomic(); +} + +/* Write the most recent wptr for the given ring into the hardware */ +static inline void update_wptr(struct msm_gpu *gpu, struct msm_ringbuffer *ring) +{ + unsigned long flags; + uint32_t wptr; + + spin_lock_irqsave(&ring->preempt_lock, flags); + + if (ring->restore_wptr) { + wptr = get_wptr(ring); + + gpu_write(gpu, REG_A6XX_CP_RB_WPTR, wptr); + + ring->restore_wptr = false; + } + + spin_unlock_irqrestore(&ring->preempt_lock, flags); +} + +/* Return the highest priority ringbuffer with something in it */ +static struct msm_ringbuffer *get_next_ring(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + unsigned long flags; + int i; + + for (i = 0; i < gpu->nr_rings; i++) { + bool empty; + struct msm_ringbuffer *ring = gpu->rb[i]; + + spin_lock_irqsave(&ring->preempt_lock, flags); + empty = (get_wptr(ring) == gpu->funcs->get_rptr(gpu, ring)); + if (!empty && ring == a6xx_gpu->cur_ring) + empty = ring->memptrs->fence == a6xx_gpu->last_seqno[i]; + spin_unlock_irqrestore(&ring->preempt_lock, flags); + + if (!empty) + return ring; + } + + return NULL; +} + +static void a6xx_preempt_timer(struct timer_list *t) +{ + struct a6xx_gpu *a6xx_gpu = from_timer(a6xx_gpu, t, preempt_timer); + struct msm_gpu *gpu = &a6xx_gpu->base.base; + struct drm_device *dev = gpu->dev; + + if (!try_preempt_state(a6xx_gpu, PREEMPT_TRIGGERED, PREEMPT_FAULTED)) + return; + + dev_err(dev->dev, "%s: preemption timed out\n", gpu->name); + kthread_queue_work(gpu->worker, &gpu->recover_work); +} + +void a6xx_preempt_irq(struct msm_gpu *gpu) +{ + uint32_t status; + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct drm_device *dev = gpu->dev; + + if (!try_preempt_state(a6xx_gpu, PREEMPT_TRIGGERED, PREEMPT_PENDING)) + return; + + /* Delete the preemption watchdog timer */ + del_timer(&a6xx_gpu->preempt_timer); + + /* + * The hardware should be setting the stop bit of CP_CONTEXT_SWITCH_CNTL + * to zero before firing the interrupt, but there is a non zero chance + * of a hardware condition or a software race that could set it again + * before we have a chance to finish. If that happens, log and go for + * recovery + */ + status = gpu_read(gpu, REG_A6XX_CP_CONTEXT_SWITCH_CNTL); + if (unlikely(status & A6XX_CP_CONTEXT_SWITCH_CNTL_STOP)) { + DRM_DEV_ERROR(&gpu->pdev->dev, + "!!!!!!!!!!!!!!!! preemption faulted !!!!!!!!!!!!!! irq\n"); + set_preempt_state(a6xx_gpu, PREEMPT_FAULTED); + dev_err(dev->dev, "%s: Preemption failed to complete\n", + gpu->name); + kthread_queue_work(gpu->worker, &gpu->recover_work); + return; + } + + a6xx_gpu->cur_ring = a6xx_gpu->next_ring; + a6xx_gpu->next_ring = NULL; + + set_preempt_state(a6xx_gpu, PREEMPT_FINISH); + + update_wptr(gpu, a6xx_gpu->cur_ring); + + set_preempt_state(a6xx_gpu, PREEMPT_NONE); + + /* + * Retrigger preemption to avoid a deadlock that might occur when preemption + * is skipped due to it being already in flight when requested. + */ + a6xx_preempt_trigger(gpu); +} + +void a6xx_preempt_hw_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int i; + + /* No preemption if we only have one ring */ + if (gpu->nr_rings == 1) + return; + + for (i = 0; i < gpu->nr_rings; i++) { + struct a6xx_preempt_record *record_ptr = a6xx_gpu->preempt[i]; + + record_ptr->wptr = 0; + record_ptr->rptr = 0; + record_ptr->rptr_addr = shadowptr(a6xx_gpu, gpu->rb[i]); + record_ptr->info = 0; + record_ptr->data = 0; + record_ptr->rbase = gpu->rb[i]->iova; + } + + /* Write a 0 to signal that we aren't switching pagetables */ + gpu_write64(gpu, REG_A6XX_CP_CONTEXT_SWITCH_SMMU_INFO, 0); + + /* Enable the GMEM save/restore feature for preemption */ + gpu_write(gpu, REG_A6XX_RB_CONTEXT_SWITCH_GMEM_SAVE_RESTORE, 0x1); + + /* Reset the preemption state */ + set_preempt_state(a6xx_gpu, PREEMPT_NONE); + + spin_lock_init(&a6xx_gpu->eval_lock); + + /* Always come up on rb 0 */ + a6xx_gpu->cur_ring = gpu->rb[0]; +} + +void a6xx_preempt_trigger(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + unsigned long flags; + struct msm_ringbuffer *ring; + unsigned int cntl; + + if (gpu->nr_rings == 1) + return; + + /* + * Lock to make sure another thread attempting preemption doesn't skip it + * while we are still evaluating the next ring. This makes sure the other + * thread does start preemption if we abort it and avoids a soft lock. + */ + spin_lock_irqsave(&a6xx_gpu->eval_lock, flags); + + /* + * Try to start preemption by moving from NONE to START. If + * unsuccessful, a preemption is already in flight + */ + if (!try_preempt_state(a6xx_gpu, PREEMPT_NONE, PREEMPT_START)) { + spin_unlock_irqrestore(&a6xx_gpu->eval_lock, flags); + return; + } + + cntl = A6XX_CP_CONTEXT_SWITCH_CNTL_LEVEL(a6xx_gpu->preempt_level); + + if (a6xx_gpu->skip_save_restore) + cntl |= A6XX_CP_CONTEXT_SWITCH_CNTL_SKIP_SAVE_RESTORE; + + if (a6xx_gpu->uses_gmem) + cntl |= A6XX_CP_CONTEXT_SWITCH_CNTL_USES_GMEM; + + cntl |= A6XX_CP_CONTEXT_SWITCH_CNTL_STOP; + + /* Get the next ring to preempt to */ + ring = get_next_ring(gpu); + + /* + * If no ring is populated or the highest priority ring is the current + * one do nothing except to update the wptr to the latest and greatest + */ + if (!ring || (a6xx_gpu->cur_ring == ring)) { + set_preempt_state(a6xx_gpu, PREEMPT_FINISH); + update_wptr(gpu, a6xx_gpu->cur_ring); + set_preempt_state(a6xx_gpu, PREEMPT_NONE); + spin_unlock_irqrestore(&a6xx_gpu->eval_lock, flags); + return; + } + + spin_unlock_irqrestore(&a6xx_gpu->eval_lock, flags); + + spin_lock_irqsave(&ring->preempt_lock, flags); + + struct a7xx_cp_smmu_info *smmu_info_ptr = + a6xx_gpu->preempt_smmu[ring->id]; + struct a6xx_preempt_record *record_ptr = a6xx_gpu->preempt[ring->id]; + u64 ttbr0 = ring->memptrs->ttbr0; + u32 context_idr = ring->memptrs->context_idr; + + smmu_info_ptr->ttbr0 = ttbr0; + smmu_info_ptr->context_idr = context_idr; + record_ptr->wptr = get_wptr(ring); + + /* + * The GPU will write the wptr we set above when we preempt. Reset + * restore_wptr to make sure that we don't write WPTR to the same + * thing twice. It's still possible subsequent submissions will update + * wptr again, in which case they will set the flag to true. This has + * to be protected by the lock for setting the flag and updating wptr + * to be atomic. + */ + ring->restore_wptr = false; + + spin_unlock_irqrestore(&ring->preempt_lock, flags); + + gpu_write64(gpu, + REG_A6XX_CP_CONTEXT_SWITCH_SMMU_INFO, + a6xx_gpu->preempt_smmu_iova[ring->id]); + + gpu_write64(gpu, + REG_A6XX_CP_CONTEXT_SWITCH_PRIV_NON_SECURE_RESTORE_ADDR, + a6xx_gpu->preempt_iova[ring->id]); + + a6xx_gpu->next_ring = ring; + + /* Start a timer to catch a stuck preemption */ + mod_timer(&a6xx_gpu->preempt_timer, jiffies + msecs_to_jiffies(10000)); + + /* Set the preemption state to triggered */ + set_preempt_state(a6xx_gpu, PREEMPT_TRIGGERED); + + /* Trigger the preemption */ + gpu_write(gpu, REG_A6XX_CP_CONTEXT_SWITCH_CNTL, cntl); +} + +static int preempt_init_ring(struct a6xx_gpu *a6xx_gpu, + struct msm_ringbuffer *ring) +{ + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + struct msm_gpu *gpu = &adreno_gpu->base; + struct drm_gem_object *bo = NULL; + phys_addr_t ttbr; + u64 iova = 0; + void *ptr; + int asid; + + ptr = msm_gem_kernel_new(gpu->dev, + PREEMPT_RECORD_SIZE(adreno_gpu), + MSM_BO_WC | MSM_BO_MAP_PRIV, gpu->aspace, &bo, &iova); + + if (IS_ERR(ptr)) + return PTR_ERR(ptr); + + memset(ptr, 0, PREEMPT_RECORD_SIZE(adreno_gpu)); + + msm_gem_object_set_name(bo, "preempt_record ring%d", ring->id); + + a6xx_gpu->preempt_bo[ring->id] = bo; + a6xx_gpu->preempt_iova[ring->id] = iova; + a6xx_gpu->preempt[ring->id] = ptr; + + struct a6xx_preempt_record *record_ptr = ptr; + + ptr = msm_gem_kernel_new(gpu->dev, + PREEMPT_SMMU_INFO_SIZE, + MSM_BO_WC | MSM_BO_MAP_PRIV | MSM_BO_GPU_READONLY, + gpu->aspace, &bo, &iova); + + if (IS_ERR(ptr)) + return PTR_ERR(ptr); + + memset(ptr, 0, PREEMPT_SMMU_INFO_SIZE); + + msm_gem_object_set_name(bo, "preempt_smmu_info ring%d", ring->id); + + a6xx_gpu->preempt_smmu_bo[ring->id] = bo; + a6xx_gpu->preempt_smmu_iova[ring->id] = iova; + a6xx_gpu->preempt_smmu[ring->id] = ptr; + + struct a7xx_cp_smmu_info *smmu_info_ptr = ptr; + + msm_iommu_pagetable_params(gpu->aspace->mmu, &ttbr, &asid); + + smmu_info_ptr->magic = GEN7_CP_SMMU_INFO_MAGIC; + smmu_info_ptr->ttbr0 = ttbr; + smmu_info_ptr->asid = 0xdecafbad; + smmu_info_ptr->context_idr = 0; + + /* Set up the defaults on the preemption record */ + record_ptr->magic = A6XX_PREEMPT_RECORD_MAGIC; + record_ptr->info = 0; + record_ptr->data = 0; + record_ptr->rptr = 0; + record_ptr->wptr = 0; + record_ptr->cntl = MSM_GPU_RB_CNTL_DEFAULT; + record_ptr->rbase = ring->iova; + record_ptr->counter = 0; + record_ptr->bv_rptr_addr = rbmemptr(ring, bv_rptr); + + return 0; +} + +void a6xx_preempt_fini(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int i; + + for (i = 0; i < gpu->nr_rings; i++) + msm_gem_kernel_put(a6xx_gpu->preempt_bo[i], gpu->aspace); +} + +void a6xx_preempt_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int i; + + /* No preemption if we only have one ring */ + if (gpu->nr_rings <= 1) + return; + + for (i = 0; i < gpu->nr_rings; i++) { + if (preempt_init_ring(a6xx_gpu, gpu->rb[i])) + goto fail; + } + + /* TODO: make this configurable? */ + a6xx_gpu->preempt_level = 1; + a6xx_gpu->uses_gmem = 1; + a6xx_gpu->skip_save_restore = 1; + + timer_setup(&a6xx_gpu->preempt_timer, a6xx_preempt_timer, 0); + + return; +fail: + /* + * On any failure our adventure is over. Clean up and + * set nr_rings to 1 to force preemption off + */ + a6xx_preempt_fini(gpu); + gpu->nr_rings = 1; + + DRM_DEV_ERROR(&gpu->pdev->dev, + "preemption init failed, disabling preemption\n"); + + return; +} diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.h b/drivers/gpu/drm/msm/msm_ringbuffer.h index 174f83137a49940ec80b1fbf548e214fa3c32784..d1e49f701c8176e50d2b9a5cca35acee67f75209 100644 --- a/drivers/gpu/drm/msm/msm_ringbuffer.h +++ b/drivers/gpu/drm/msm/msm_ringbuffer.h @@ -36,6 +36,7 @@ struct msm_rbmemptrs { volatile struct msm_gpu_submit_stats stats[MSM_GPU_SUBMIT_STATS_COUNT]; volatile u64 ttbr0; + volatile u32 context_idr; }; struct msm_cp_state { @@ -101,6 +102,12 @@ struct msm_ringbuffer { */ spinlock_t preempt_lock; + /* + * Whether we skipped writing wptr and it needs to be updated in the + * future when the ring becomes current. + */ + bool restore_wptr; + /** * cur_ctx_seqno: * From patchwork Mon Sep 30 13:52:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Antonino Maniscalco X-Patchwork-Id: 13816511 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from gabe.freedesktop.org (gabe.freedesktop.org [131.252.210.177]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 63F0CCE8348 for ; Mon, 30 Sep 2024 13:53:05 +0000 (UTC) Received: from gabe.freedesktop.org (localhost [127.0.0.1]) by gabe.freedesktop.org (Postfix) with ESMTP id 7594210E512; Mon, 30 Sep 2024 13:53:04 +0000 (UTC) Authentication-Results: gabe.freedesktop.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="VDakuKNY"; dkim-atps=neutral Received: from mail-ed1-f45.google.com (mail-ed1-f45.google.com [209.85.208.45]) by gabe.freedesktop.org (Postfix) with ESMTPS id 33A2E10E500; Mon, 30 Sep 2024 13:52:54 +0000 (UTC) Received: by mail-ed1-f45.google.com with SMTP id 4fb4d7f45d1cf-5c89f3f28b6so610886a12.2; Mon, 30 Sep 2024 06:52:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1727704372; x=1728309172; darn=lists.freedesktop.org; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:from:to:cc:subject:date:message-id :reply-to; bh=U8vL189vTh8iXNA4lYnscfArOlPBvDZ3K4+HECdKLUM=; b=VDakuKNYAEsSLIr+BTCw723upD+0FhNLJm6+/wK3nKZzDh4Tv+GYdiIkZZEwNv5zHP Je/bExUsZd1zkXOwnFD5Qo6wKGLpPiY9TRDpHu0pG1//delH3emqMw8lPVIWHxaTrBOC zbGz4kzuHsrbxuscQEL/b3jTREt0UPmj0yvxqFVrS6t50D5f0B9nRCfPW0Fj07vgOpc/ eqen9YQT3mRPyHbzjA53XZqsB8fw5Yusr8IpBazZEhsqIvIKyWzgSNX1Ji9Msd9ebnmI VKgihdxWqW/oVk1cycH2/dRQdXhGHqyt8y22Wi7UW8z4gYTrhZWKPKp+9J0cByyrSeOo 9UGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1727704372; x=1728309172; h=cc:to:in-reply-to:references:message-id:content-transfer-encoding :mime-version:subject:date:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=U8vL189vTh8iXNA4lYnscfArOlPBvDZ3K4+HECdKLUM=; b=echoLFwkS1afb1Sbj7NxBFuIWAxZSV4dzYEgEzDQCvcScZ1GIFPc9XUR/PcslIitqA F+vGwx82RXI8szQYHAbxkayjztFPj2+5t3FqB5wXT3fxqtxtopD3C2obT9rKW2N0ds+y BRc9aDRNg1mH9ezAU2smBiPvBGiIZEiwLR3AtbOM3ztdsSJSouPE9mTb1tMTkFt25e6+ Tqr0e5XsUTwurfg2XjvASflftDsOEAD/crrCv9YlEtvQVVZxtAAstwqDrovWzadBbIOp ftB0q+ZP02YlDadR6BnNzRsZqk9g4h23dSW3gW+fPP+xrnJNMAmSZyE66JiweoadkUgq g7jw== X-Forwarded-Encrypted: i=1; AJvYcCU98+DCkb/MZk1wNS331y00pe6Ivzc/rz6umZdYrSKuBYGqAx4NZZC84lRiWZ0Mp01wYp31JS5tcD0=@lists.freedesktop.org, AJvYcCWNcwjYhiHM4ox1BYdFrlmkUuIOEYCs4XfUW/S4EU7weZmg7/tVWjKEfxB0+349PxOnuNU0em5Kg5FU@lists.freedesktop.org X-Gm-Message-State: AOJu0Yy8S2XWSijV1VTvzaiqugSjHIkuhR/bIxBlGfHqKukI3jPtC2ht Mejwnh90ibJylGVVKXfrHIL5gZqYdE4pZIkB2d8ppLCFMwISEd8K X-Google-Smtp-Source: AGHT+IEtEHvA++ACAq3+/Xn8PG90rxIADGbdFhcm0asOS75rHEmMB3MrcHbNxTrRQNa66Fn864KPBg== X-Received: by 2002:a05:6402:34c4:b0:5c8:8381:c1bf with SMTP id 4fb4d7f45d1cf-5c88381c226mr8680369a12.3.1727704372364; Mon, 30 Sep 2024 06:52:52 -0700 (PDT) Received: from [192.168.1.17] (host-95-250-55-22.retail.telecomitalia.it. [95.250.55.22]) by smtp.gmail.com with ESMTPSA id 4fb4d7f45d1cf-5c88245ea57sm4507540a12.55.2024.09.30.06.52.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 30 Sep 2024 06:52:52 -0700 (PDT) From: Antonino Maniscalco Date: Mon, 30 Sep 2024 15:52:42 +0200 Subject: [PATCH v7 07/12] drm/msm/a6xx: Sync relevant adreno_pm4.xml changes MIME-Version: 1.0 Message-Id: <20240930-preemption-a750-t-v7-7-47803c7a5a64@gmail.com> References: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> In-Reply-To: <20240930-preemption-a750-t-v7-0-47803c7a5a64@gmail.com> To: Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Dmitry Baryshkov , Marijn Suijten , David Airlie , Daniel Vetter , Maarten Lankhorst , Maxime Ripard , Thomas Zimmermann , Jonathan Corbet Cc: linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Antonino Maniscalco , Neil Armstrong X-Mailer: b4 0.14.2 X-Developer-Signature: v=1; a=ed25519-sha256; t=1727704357; l=4077; i=antomani103@gmail.com; s=20240815; h=from:subject:message-id; bh=/wWoq9phTQOtf3DbnY9akGSVrFARm8V71ASGyLN3bu4=; b=47iWK2j17SvgYMevDnKcX9MYYXBX8tLf6jqCulkRTmMTXG4gLjA4OKUlyksvgteyBIqChCyvv TepYYJSTT2TCq5ZNoJBtZx1Pm0264bU1fTYjgGm5b0/lLy7oPBXaaWw X-Developer-Key: i=antomani103@gmail.com; a=ed25519; pk=0zicFb38tVla+iHRo4kWpOMsmtUrpGBEa7LkFF81lyY= X-BeenThere: dri-devel@lists.freedesktop.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Direct Rendering Infrastructure - Development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dri-devel-bounces@lists.freedesktop.org Sender: "dri-devel" In mesa CP_SET_CTXSWITCH_IB is renamed to CP_SET_AMBLE and some other names are changed to match KGSL. Import those changes. The changes have not been merged yet in mesa but are necessary for this series. Tested-by: Rob Clark Tested-by: Neil Armstrong # on SM8650-QRD Tested-by: Neil Armstrong # on SM8550-QRD Tested-by: Neil Armstrong # on SM8450-HDK Signed-off-by: Antonino Maniscalco --- .../gpu/drm/msm/registers/adreno/adreno_pm4.xml | 39 ++++++++++------------ 1 file changed, 17 insertions(+), 22 deletions(-) diff --git a/drivers/gpu/drm/msm/registers/adreno/adreno_pm4.xml b/drivers/gpu/drm/msm/registers/adreno/adreno_pm4.xml index cab01af55d22268ccf88f1a5032b6081d8e4e475..55a35182858ccac3292849faaf12727257e053c7 100644 --- a/drivers/gpu/drm/msm/registers/adreno/adreno_pm4.xml +++ b/drivers/gpu/drm/msm/registers/adreno/adreno_pm4.xml @@ -581,8 +581,7 @@ xsi:schemaLocation="https://gitlab.freedesktop.org/freedreno/ rules-fd.xsd"> and forcibly switch to the indicated context. - - +