From patchwork Tue May 9 10:30:18 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13235601 X-Patchwork-Delegate: palmer@dabbelt.com Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2AF39C7EE24 for ; Tue, 9 May 2023 10:32:24 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=4YvH7LDuw9eKzo0P1YdgUaffCrIwFjoW3VdP6q6mwjU=; b=kosh1tNwBQcIky fjsO2MGXYmjx6td7FtIuED6u9qk/VNzRPQNWGxqhM56mZkD8565Gs+RTN2+2qiuuukM8r0KZurd/+ eVQS7ciHccoJWFKAas3dtSO8SDIFlRruzbF9pDZzP9HSWSv29hdeeZQztpOi1hH/+GDeqB+KOkPBm /SS6F7FbAP6UMZfjqXMtjgxCNSaHZSZbZEV/3QwbIDctQBp4cBV1+fUC2VcTbHrgVygWKmbjZOYo1 D0cbjIWnhAfefJtuLkXSoSCKWk0CC85sEllKQeQZByrH1al3VGPyGzRb4bgSiX52ksNdBnyn03wFy 0P5/7MibWsEOYasWIsUw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pwKdk-002tRO-1u; Tue, 09 May 2023 10:32:16 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pwKdi-002tOe-2e for linux-riscv@bombadil.infradead.org; Tue, 09 May 2023 10:32:14 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:Content-ID:Content-Description; bh=ahp5kQnTghgkZmiSTxR6+eMYSF2yeYW/B9Sl3mLm86I=; b=MmEJ9TigkUFzSjKCnUbalkNsGy /uyh10/nNOivDb6Exj7IJNypuwpPPKKUbsarEPFo9DrGQgbCmBKozvn6bW8wK8mu/jf00LvgQigsm 032PoBmxl8aGFcTCj9UX/Q0IrvQDVjLyHJztcdL//vRGVIL1DuJEXX6Q0ghYOCz6YSxWAaj1cjPGz 5FpIkNQ50TJaL65J4kVpvc55hRtn2OxIWApbjf75EwTcyAOoSqx4dj6RmejMxvPEqU9S/U1IxOijb eScADJpIxzLbI59bwSOJqI3SndGgQv4srRAxX46NTmXNNG1C5w4NnF+Q+S9izcMBXq8F1jLeSsRJP Hq+3bzUg==; Received: from mail-pl1-x62c.google.com ([2607:f8b0:4864:20::62c]) by desiato.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pwKdc-00666h-0o for linux-riscv@lists.infradead.org; Tue, 09 May 2023 10:32:13 +0000 Received: by mail-pl1-x62c.google.com with SMTP id d9443c01a7336-1aad55244b7so43143045ad.2 for ; Tue, 09 May 2023 03:32:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1683628326; x=1686220326; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ahp5kQnTghgkZmiSTxR6+eMYSF2yeYW/B9Sl3mLm86I=; b=KfhD6w8KZHsd6v9aUu137matM7Ud86Ork+O59oxBKe2TV9bPYAcytyN3pit3i26NkU jvDo7ysnSvVOCb+SGiVkyo6K06HsFVbmGtHgwxwb+w7SBMCWCVavWo+Cbh/7swt8C7R2 5aZaYguuEa1PBX6GsklEyZ9vPZoy7N3Emm+rAxildNuVMy6bxIKzPq/NjqgnWjnL9rsF CXU8M2vdMyGRJAKQ8wh6/M+55fIX8AEQJovM8gkN+LlyB6Y6QZJDgRElrWivDSODu3QG CsH7Hq/bzPXYgyKt12QSjYXkhdJj6XwJsHhfggpK5ISp3zh9sZYTGjRx+IRndd1jd0kW 2dmQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1683628326; x=1686220326; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ahp5kQnTghgkZmiSTxR6+eMYSF2yeYW/B9Sl3mLm86I=; b=aI5d/9m3oeRboF0oxOFsHMImDT70/4o9wDzeyZLyHYESDA73SsiY+csC1GwBJyIE5Z FwERZQyixlbaA0bil9dYnDxt/uR1TNGXLUhMPmFCxGFl9WbyZntopTFUA6F6vyHItTDA KReHg644NV3PXc5OloV6kOnBjZW6l+qqTGe7jhseVA2e24L2wrYq/8HVKaC736MDV9Wn TMAgTPa+PLfukjVb3Ttn6zWS8XCVZbtLz3oaoYVH8o0QrANnoCWMwvqnR+u+HslPlt/y mHhQQP4l5pcCTr+jkrxf1C8Hb1K7lSzf1s9UzCtPmu292+Pj0tyElxfjeSx9iZX3ExSq 2FMA== X-Gm-Message-State: AC+VfDyNesEmbkAjvbes0SjCwCqisBhKSdVOslOhARBkVGLZX90MI2mX Tj0Gggvj/pomSUwaszlwTh3kMyXqCyD54oBzl/OXi9BvqGPk3FXu0qYxj40qOfEU0o4DAQYRqUs ek/irGTKSw2D56Yj0ydPdblN2HvnbVCeU0PnFEfQ7z5989MlMPU2IY+/0qBsfwd2EtJggiLt9aY zbewuhcOnU3HfX X-Google-Smtp-Source: ACHHUZ6KFsz81viA3s3MvRxm/AxDuAtt4z22AwlOnW2jTQrrGqzIl05BEki990/hiZjbPe1AoMhs/g== X-Received: by 2002:a17:902:ba86:b0:1a6:bd5c:649d with SMTP id k6-20020a170902ba8600b001a6bd5c649dmr13435174pls.56.1683628325622; Tue, 09 May 2023 03:32:05 -0700 (PDT) Received: from hsinchu25.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id o11-20020a170902d4cb00b001a076025715sm1195191plg.117.2023.05.09.03.32.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 May 2023 03:32:05 -0700 (PDT) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com, anup@brainfault.org, atishp@atishpatra.org, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org Cc: vineetg@rivosinc.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, Vincent Chen , Andy Chiu , Paul Walmsley , Albert Ou , Heiko Stuebner , Guo Ren , Conor Dooley Subject: [PATCH -next v19 09/24] riscv: Introduce struct/helpers to save/restore per-task Vector state Date: Tue, 9 May 2023 10:30:18 +0000 Message-Id: <20230509103033.11285-10-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230509103033.11285-1-andy.chiu@sifive.com> References: <20230509103033.11285-1-andy.chiu@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230509_113210_204421_306676AC X-CRM114-Status: GOOD ( 12.98 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Greentime Hu Add vector state context struct to be added later in thread_struct. And prepare low-level helper functions to save/restore vector contexts. This include Vector Regfile and CSRs holding dynamic configuration state (vstart, vl, vtype, vcsr). The Vec Register width could be implementation defined, but same for all processes, so that is saved separately. This is not yet wired into final thread_struct - will be done when __switch_to actually starts doing this in later patches. Given the variable (and potentially large) size of regfile, they are saved in dynamically allocated memory, pointed to by datap pointer in __riscv_v_ext_state. Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu Signed-off-by: Vineet Gupta Signed-off-by: Andy Chiu Acked-by: Conor Dooley Reviewed-by: Guo Ren Reviewed-by: Björn Töpel Reviewed-by: Heiko Stuebner Tested-by: Heiko Stuebner --- arch/riscv/include/asm/vector.h | 95 ++++++++++++++++++++++++++++ arch/riscv/include/uapi/asm/ptrace.h | 17 +++++ 2 files changed, 112 insertions(+) diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index 68c9fe831a41..7a56bb0769aa 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -11,8 +11,10 @@ #ifdef CONFIG_RISCV_ISA_V +#include #include #include +#include extern unsigned long riscv_v_vsize; int riscv_v_setup_vsize(void); @@ -22,6 +24,26 @@ static __always_inline bool has_vector(void) return riscv_has_extension_likely(RISCV_ISA_EXT_v); } +static inline void __riscv_v_vstate_clean(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_CLEAN; +} + +static inline void riscv_v_vstate_off(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_OFF; +} + +static inline void riscv_v_vstate_on(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_INITIAL; +} + +static inline bool riscv_v_vstate_query(struct pt_regs *regs) +{ + return (regs->status & SR_VS) != 0; +} + static __always_inline void riscv_v_enable(void) { csr_set(CSR_SSTATUS, SR_VS); @@ -32,13 +54,86 @@ static __always_inline void riscv_v_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) +{ + asm volatile ( + "csrr %0, " __stringify(CSR_VSTART) "\n\t" + "csrr %1, " __stringify(CSR_VTYPE) "\n\t" + "csrr %2, " __stringify(CSR_VL) "\n\t" + "csrr %3, " __stringify(CSR_VCSR) "\n\t" + : "=r" (dest->vstart), "=r" (dest->vtype), "=r" (dest->vl), + "=r" (dest->vcsr) : :); +} + +static __always_inline void __vstate_csr_restore(struct __riscv_v_ext_state *src) +{ + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvl x0, %2, %1\n\t" + ".option pop\n\t" + "csrw " __stringify(CSR_VSTART) ", %0\n\t" + "csrw " __stringify(CSR_VCSR) ", %3\n\t" + : : "r" (src->vstart), "r" (src->vtype), "r" (src->vl), + "r" (src->vcsr) :); +} + +static inline void __riscv_v_vstate_save(struct __riscv_v_ext_state *save_to, + void *datap) +{ + unsigned long vl; + + riscv_v_enable(); + __vstate_csr_save(save_to); + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %0, x0, e8, m8, ta, ma\n\t" + "vse8.v v0, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v8, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v16, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v24, (%1)\n\t" + ".option pop\n\t" + : "=&r" (vl) : "r" (datap) : "memory"); + riscv_v_disable(); +} + +static inline void __riscv_v_vstate_restore(struct __riscv_v_ext_state *restore_from, + void *datap) +{ + unsigned long vl; + + riscv_v_enable(); + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %0, x0, e8, m8, ta, ma\n\t" + "vle8.v v0, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v8, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v16, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v24, (%1)\n\t" + ".option pop\n\t" + : "=&r" (vl) : "r" (datap) : "memory"); + __vstate_csr_restore(restore_from); + riscv_v_disable(); +} + #else /* ! CONFIG_RISCV_ISA_V */ struct pt_regs; static inline int riscv_v_setup_vsize(void) { return -EOPNOTSUPP; } static __always_inline bool has_vector(void) { return false; } +static inline bool riscv_v_vstate_query(struct pt_regs *regs) { return false; } #define riscv_v_vsize (0) +#define riscv_v_vstate_off(regs) do {} while (0) +#define riscv_v_vstate_on(regs) do {} while (0) #endif /* CONFIG_RISCV_ISA_V */ diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index 882547f6bd5c..586786d023c4 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -77,6 +77,23 @@ union __riscv_fp_state { struct __riscv_q_ext_state q; }; +struct __riscv_v_ext_state { + unsigned long vstart; + unsigned long vl; + unsigned long vtype; + unsigned long vcsr; + void *datap; + /* + * In signal handler, datap will be set a correct user stack offset + * and vector registers will be copied to the address of datap + * pointer. + * + * In ptrace syscall, datap will be set to zero and the vector + * registers will be copied to the address right after this + * structure. + */ +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */