From patchwork Thu May 18 16:19:32 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13247095 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 04E6EC77B7A for ; Thu, 18 May 2023 16:40:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=i/rDMSs5+iY2woHYJMhPSh0mUnR5d595KEHgWmVao7I=; b=skI+iaeTKL91lp t5CYQTA+Zow6P9wBaGdHuGvLgWTFi0k5nMaAKmwW24uzuZiVId7m+jNE8/CcrkgTYcUjfkxv5qXpt qzQc7LLeVGZnwzWHEIm3shBUxkD/eKK14CCfaErKdNnwBfdYYQqK0xMhL0JzvPPTa66x/rQLrlTYn 5yxK0Thrv+opWnUCTt+0/tdWn7doLrgPQ5tBRgQ8JEmXuyy09e+H5+jIQ9W4J5SvzkGLfJLd3/LYN BhAvrZcoyqd9izkXVhG1dIPdPhEyLyUtDJlTKyvOF3tNqUWygpqYCCcaYti3goI0d1b7h1pyIFI2a /Jw5xcleiChpxCl/ZAqg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1pzgfv-00DUuZ-1q; Thu, 18 May 2023 16:40:23 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pzgft-00DUtP-1Z for linux-riscv@bombadil.infradead.org; Thu, 18 May 2023 16:40:21 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :MIME-Version:References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From: Sender:Reply-To:Content-ID:Content-Description; bh=Zgwm1t43SJfpy9DrAiV+ciMDfg3wElSoTQtIFJq3cgU=; b=GEjjdWNzu8p1/+Zwyo1Als6MPf BzxJrR1Zog0bxkQeUwd7cy6cZti8ZcdF+RKzLB6jltGDGE7WnBq6Gzjzy0mapdWK9ApbrK8dKd21N 0rJW9Lm8mIGjFPHSmV/SeWoN4HUHaa6tO4xaVnZ1gT+dEFpu15HXLyVjvSpHirjeE86eNqWCF7IGz kMgS8aqHn6xhhJz/DQrCbfnLsS47/DbjEHsSFtP1n+D6vulCkjLrpqCe/xV3p0u66yjmBlMk+qTjZ VjUSSnDmBBvXN8YjOxHQyc6s7lLSsUyoZvsS6+f53j6FGEdB8JpJaCVcttljjzZEM2dHfhzkECPrH yYsQynJw==; Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]) by desiato.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1pzgNH-00ErLT-2S for linux-riscv@lists.infradead.org; Thu, 18 May 2023 16:21:09 +0000 Received: by mail-pf1-x432.google.com with SMTP id d2e1a72fcca58-64d2981e3abso287098b3a.1 for ; Thu, 18 May 2023 09:21:07 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1684426865; x=1687018865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Zgwm1t43SJfpy9DrAiV+ciMDfg3wElSoTQtIFJq3cgU=; b=LyM9bhV16q6PUaafI2xVfP/P6GftAZwrqsnAY/9ocZAILuEt68EoHhLyrLnBNgiwY8 mdEbS8Cv745w6pvBD+bVERM37H23B6k94R+f4dn+tcsczxbm2LcdlWghGtMYi+IwOM/4 uPXSpwg/fKsKa3rqdcoXou4ru1OC+JNKCh7LfVkO5Jo9E1Gep+cJ1ar4UpiGy6iehbAV 3U4Hjx1mKN6ePhw8rlKZHMm4rdpmAivPOgwWllE49OevgFtaewRVis3H62WSBmi0bJMb CvpoRr/XstvU04g8s6rLfdOCkjVpJhwwOu2KOJdwtro9N2s8tMdz+2ifmgRCDb3NfZ3h gHPg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1684426865; x=1687018865; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Zgwm1t43SJfpy9DrAiV+ciMDfg3wElSoTQtIFJq3cgU=; b=GsteRav4gTRIJupBJ7BXgsh6P3CuHGX7kmq0D2DwhKGHg7k5fbaFUoVy0zv5wEkIr3 KWsq0a/rK+rpLeCGIbnL//GD9cdjJLJO8Gn8LCSvWiD3cimWJXcnjFEPwNi+oDGEuC4k h3AalcsR8TwKqtE1pK5nVMm9p5stw4fPrlNkj2ZgUwONA89lcz1k+OczxJi61FMFK+1j dhLVRYBmZk5csThTYTlEqvB0z2slCVDZylffmWcBFMVPnkrWKQROBLTrW2CzYhVUnInY xdGoGzNF/pqa+VzDjCsbzxPcJoLSNDHLzHHSYjqsXEMF1heFy3jag19Z+5Mp4TTRnasG /1ag== X-Gm-Message-State: AC+VfDzy64oYWrLc6xz+Efctv5Fcg/EpW0Apzr9uvLO27+BCqzqUnjm4 W74WeAKT6MrU1LSI56Unjt37O5gvZAk3bP69hDqcLw14KJnO3P3UWCgGunt874xwAKxs7Xu8zBH jhQZUITtzvtoKHaK2qUxlR+S8tjq+ziT/u0vlkVXMK/Gg+/AuFvP8CzelfyIiwSaIWbZCSlO7IW GtamTHCMJayiPNbWU= X-Google-Smtp-Source: ACHHUZ5/uqfj4c5FsQaeIh/+by7OIWeOL9yYb4WWUqPBhdAtvNIA/9TgeTb0VHs89J84EarVxCDTbg== X-Received: by 2002:a05:6a00:24d1:b0:643:2559:80f3 with SMTP id d17-20020a056a0024d100b00643255980f3mr6553015pfv.2.1684426865160; Thu, 18 May 2023 09:21:05 -0700 (PDT) Received: from hsinchu25.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id x23-20020a62fb17000000b006414b2c9efasm1515862pfm.123.2023.05.18.09.21.02 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 18 May 2023 09:21:04 -0700 (PDT) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com, anup@brainfault.org, atishp@atishpatra.org, kvm-riscv@lists.infradead.org, kvm@vger.kernel.org Cc: vineetg@rivosinc.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, Vincent Chen , Andy Chiu , Paul Walmsley , Albert Ou , Heiko Stuebner , Guo Ren , Conor Dooley Subject: [PATCH -next v20 09/26] riscv: Introduce struct/helpers to save/restore per-task Vector state Date: Thu, 18 May 2023 16:19:32 +0000 Message-Id: <20230518161949.11203-10-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20230518161949.11203-1-andy.chiu@sifive.com> References: <20230518161949.11203-1-andy.chiu@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230518_172107_906694_5F1AB653 X-CRM114-Status: GOOD ( 12.36 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Greentime Hu Add vector state context struct to be added later in thread_struct. And prepare low-level helper functions to save/restore vector contexts. This include Vector Regfile and CSRs holding dynamic configuration state (vstart, vl, vtype, vcsr). The Vec Register width could be implementation defined, but same for all processes, so that is saved separately. This is not yet wired into final thread_struct - will be done when __switch_to actually starts doing this in later patches. Given the variable (and potentially large) size of regfile, they are saved in dynamically allocated memory, pointed to by datap pointer in __riscv_v_ext_state. Co-developed-by: Vincent Chen Signed-off-by: Vincent Chen Signed-off-by: Greentime Hu Signed-off-by: Vineet Gupta Signed-off-by: Andy Chiu Acked-by: Conor Dooley Reviewed-by: Guo Ren Reviewed-by: Björn Töpel Reviewed-by: Heiko Stuebner Tested-by: Heiko Stuebner Reviewed-by: Palmer Dabbelt --- arch/riscv/include/asm/vector.h | 95 ++++++++++++++++++++++++++++ arch/riscv/include/uapi/asm/ptrace.h | 17 +++++ 2 files changed, 112 insertions(+) diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index df3b5caecc87..3c29f4eb552a 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -11,8 +11,10 @@ #ifdef CONFIG_RISCV_ISA_V +#include #include #include +#include extern unsigned long riscv_v_vsize; int riscv_v_setup_vsize(void); @@ -22,6 +24,26 @@ static __always_inline bool has_vector(void) return riscv_has_extension_unlikely(RISCV_ISA_EXT_v); } +static inline void __riscv_v_vstate_clean(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_CLEAN; +} + +static inline void riscv_v_vstate_off(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_OFF; +} + +static inline void riscv_v_vstate_on(struct pt_regs *regs) +{ + regs->status = (regs->status & ~SR_VS) | SR_VS_INITIAL; +} + +static inline bool riscv_v_vstate_query(struct pt_regs *regs) +{ + return (regs->status & SR_VS) != 0; +} + static __always_inline void riscv_v_enable(void) { csr_set(CSR_SSTATUS, SR_VS); @@ -32,13 +54,86 @@ static __always_inline void riscv_v_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) +{ + asm volatile ( + "csrr %0, " __stringify(CSR_VSTART) "\n\t" + "csrr %1, " __stringify(CSR_VTYPE) "\n\t" + "csrr %2, " __stringify(CSR_VL) "\n\t" + "csrr %3, " __stringify(CSR_VCSR) "\n\t" + : "=r" (dest->vstart), "=r" (dest->vtype), "=r" (dest->vl), + "=r" (dest->vcsr) : :); +} + +static __always_inline void __vstate_csr_restore(struct __riscv_v_ext_state *src) +{ + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvl x0, %2, %1\n\t" + ".option pop\n\t" + "csrw " __stringify(CSR_VSTART) ", %0\n\t" + "csrw " __stringify(CSR_VCSR) ", %3\n\t" + : : "r" (src->vstart), "r" (src->vtype), "r" (src->vl), + "r" (src->vcsr) :); +} + +static inline void __riscv_v_vstate_save(struct __riscv_v_ext_state *save_to, + void *datap) +{ + unsigned long vl; + + riscv_v_enable(); + __vstate_csr_save(save_to); + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %0, x0, e8, m8, ta, ma\n\t" + "vse8.v v0, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v8, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v16, (%1)\n\t" + "add %1, %1, %0\n\t" + "vse8.v v24, (%1)\n\t" + ".option pop\n\t" + : "=&r" (vl) : "r" (datap) : "memory"); + riscv_v_disable(); +} + +static inline void __riscv_v_vstate_restore(struct __riscv_v_ext_state *restore_from, + void *datap) +{ + unsigned long vl; + + riscv_v_enable(); + asm volatile ( + ".option push\n\t" + ".option arch, +v\n\t" + "vsetvli %0, x0, e8, m8, ta, ma\n\t" + "vle8.v v0, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v8, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v16, (%1)\n\t" + "add %1, %1, %0\n\t" + "vle8.v v24, (%1)\n\t" + ".option pop\n\t" + : "=&r" (vl) : "r" (datap) : "memory"); + __vstate_csr_restore(restore_from); + riscv_v_disable(); +} + #else /* ! CONFIG_RISCV_ISA_V */ struct pt_regs; static inline int riscv_v_setup_vsize(void) { return -EOPNOTSUPP; } static __always_inline bool has_vector(void) { return false; } +static inline bool riscv_v_vstate_query(struct pt_regs *regs) { return false; } #define riscv_v_vsize (0) +#define riscv_v_vstate_off(regs) do {} while (0) +#define riscv_v_vstate_on(regs) do {} while (0) #endif /* CONFIG_RISCV_ISA_V */ diff --git a/arch/riscv/include/uapi/asm/ptrace.h b/arch/riscv/include/uapi/asm/ptrace.h index 882547f6bd5c..586786d023c4 100644 --- a/arch/riscv/include/uapi/asm/ptrace.h +++ b/arch/riscv/include/uapi/asm/ptrace.h @@ -77,6 +77,23 @@ union __riscv_fp_state { struct __riscv_q_ext_state q; }; +struct __riscv_v_ext_state { + unsigned long vstart; + unsigned long vl; + unsigned long vtype; + unsigned long vcsr; + void *datap; + /* + * In signal handler, datap will be set a correct user stack offset + * and vector registers will be copied to the address of datap + * pointer. + * + * In ptrace syscall, datap will be set to zero and the vector + * registers will be copied to the address right after this + * structure. + */ +}; + #endif /* __ASSEMBLY__ */ #endif /* _UAPI_ASM_RISCV_PTRACE_H */