[v10,00/14] riscv: Add support for xtheadvector

Message

Charlie Jenkins Sept. 12, 2024, 5:55 a.m. UTC

xtheadvector is a custom extension that is based upon riscv vector
version 0.7.1 [1]. All of the vector routines have been modified to
support this alternative vector version based upon whether xtheadvector
was determined to be supported at boot.

vlenb is not supported on the existing xtheadvector hardware, so a
devicetree property thead,vlenb is added to provide the vlenb to Linux.

There is a new hwprobe key RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 that is
used to request which thead vendor extensions are supported on the
current platform. This allows future vendors to allocate hwprobe keys
for their vendor.

Support for xtheadvector is also added to the vector kselftests.

Signed-off-by: Charlie Jenkins <[e-mail]>

[1] https://github.com/T-head-Semi/thead-extension-spec/blob/95358cb2cca9489361c61d335e03d3134b14133f/xtheadvector.adoc

---
This series is a continuation of a different series that was fragmented
into two other series in an attempt to get part of it merged in the 6.10
merge window. The split-off series did not get merged due to a NAK on
the series that added the generic riscv,vlenb devicetree entry. This
series has converted riscv,vlenb to thead,vlenb to remedy this issue.

The original series is titled "riscv: Support vendor extensions and
xtheadvector" [3].

The series titled "riscv: Extend cpufeature.c to detect vendor
extensions" is still under development and this series is based on that
series! [4]

I have tested this with an Allwinner Nezha board. I used SkiffOS [1] to
manage building the image, but upgraded the U-Boot version to Samuel
Holland's more up-to-date version [2] and changed out the device tree
used by U-Boot with the device trees that are present in upstream linux
and this series. Thank you Samuel for all of the work you did to make
this task possible.

[1] https://github.com/skiffos/SkiffOS/tree/master/configs/allwinner/nezha
[2] https://github.com/smaeul/u-boot/commit/2e89b706f5c956a70c989cd31665f1429e9a0b48
[3] https://lore.kernel.org/all/[e-mail]/
[4] https://lore.kernel.org/lkml/[e-mail]/T/

---
Changes in v10:
- In DT probing disable vector with new function to clear vendor
  extension bits for xtheadvector
- Add ghostwrite mitigations for c9xx CPUs. This disables xtheadvector
  unless mitigations=off is set as a kernel boot arg
- Link to v9: https://lore.kernel.org/r/[e-mail]

Changes in v9:
- Rebase onto palmer's for-next
- Fix sparse error in arch/riscv/kernel/vendor_extensions/thead.c
- Fix maybe-uninitialized warning in arch/riscv/include/asm/vendor_extensions/vendor_hwprobe.h
- Wrap some long lines
- Link to v8: https://lore.kernel.org/r/[e-mail]

Changes in v8:
- Rebase onto palmer's for-next
- Link to v7: https://lore.kernel.org/r/[e-mail]

Changes in v7:
- Add defs for has_xtheadvector_no_alternatives() and has_xtheadvector()
  when vector disabled. (Palmer)
- Link to v6: https://lore.kernel.org/r/[e-mail]

Changes in v6:
- Fix return type of is_vector_supported()/is_xthead_supported() to be bool
- Link to v5: https://lore.kernel.org/r/[e-mail]

Changes in v5:
- Rebase on for-next
- Link to v4: https://lore.kernel.org/r/[e-mail]

Changes in v4:
- Replace inline asm with C (Samuel)
- Rename VCSRs to CSRs (Samuel)
- Replace .insn directives with .4byte directives
- Link to v3: https://lore.kernel.org/r/[e-mail]

Changes in v3:
- Add back Heiko's signed-off-by (Conor)
- Mark RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 as a bitmask
- Link to v2: https://lore.kernel.org/r/[e-mail]

Changes in v2:
- Removed extraneous references to "riscv,vlenb" (Jess)
- Moved declaration of "thead,vlenb" into cpus.yaml and added
  restriction that it's only applicable to thead cores (Conor)
- Check CONFIG_RISCV_ISA_XTHEADVECTOR instead of CONFIG_RISCV_ISA_V for
  thead,vlenb (Jess)
- Fix naming of hwprobe variables (Evan)
- Link to v1: https://lore.kernel.org/r/[e-mail]

---
Charlie Jenkins (13):
      dt-bindings: riscv: Add xtheadvector ISA extension description
      dt-bindings: cpus: add a thead vlen register length property
      riscv: dts: allwinner: Add xtheadvector to the D1/D1s devicetree
      riscv: Add thead and xtheadvector as a vendor extension
      riscv: vector: Use vlenb from DT for thead
      riscv: csr: Add CSR encodings for CSR_VXRM/CSR_VXSAT
      riscv: Add xtheadvector instruction definitions
      riscv: vector: Support xtheadvector save/restore
      riscv: hwprobe: Add thead vendor extension probing
      riscv: hwprobe: Document thead vendor extensions and xtheadvector extension
      selftests: riscv: Fix vector tests
      selftests: riscv: Support xtheadvector in vector tests
      riscv: Add ghostwrite vulnerability

Heiko Stuebner (1):
      RISC-V: define the elements of the VCSR vector CSR

 Documentation/arch/riscv/hwprobe.rst               |  10 +
 Documentation/devicetree/bindings/riscv/cpus.yaml  |  19 ++
 .../devicetree/bindings/riscv/extensions.yaml      |  10 +
 arch/riscv/Kconfig.errata                          |  11 +
 arch/riscv/Kconfig.vendor                          |  26 ++
 arch/riscv/boot/dts/allwinner/sun20i-d1s.dtsi      |   3 +-
 arch/riscv/errata/thead/errata.c                   |  28 ++
 arch/riscv/include/asm/bugs.h                      |  22 ++
 arch/riscv/include/asm/cpufeature.h                |   2 +
 arch/riscv/include/asm/csr.h                       |  15 +
 arch/riscv/include/asm/errata_list.h               |   3 +-
 arch/riscv/include/asm/hwprobe.h                   |   3 +-
 arch/riscv/include/asm/switch_to.h                 |   2 +-
 arch/riscv/include/asm/vector.h                    | 225 +++++++++++----
 arch/riscv/include/asm/vendor_extensions/thead.h   |  48 ++++
 .../include/asm/vendor_extensions/thead_hwprobe.h  |  19 ++
 .../include/asm/vendor_extensions/vendor_hwprobe.h |  37 +++
 arch/riscv/include/uapi/asm/hwprobe.h              |   3 +-
 arch/riscv/include/uapi/asm/vendor/thead.h         |   3 +
 arch/riscv/kernel/Makefile                         |   2 +
 arch/riscv/kernel/bugs.c                           |  55 ++++
 arch/riscv/kernel/cpufeature.c                     |  58 +++-
 arch/riscv/kernel/kernel_mode_vector.c             |   8 +-
 arch/riscv/kernel/process.c                        |   4 +-
 arch/riscv/kernel/signal.c                         |   6 +-
 arch/riscv/kernel/sys_hwprobe.c                    |   5 +
 arch/riscv/kernel/vector.c                         |  24 +-
 arch/riscv/kernel/vendor_extensions.c              |  10 +
 arch/riscv/kernel/vendor_extensions/Makefile       |   2 +
 arch/riscv/kernel/vendor_extensions/thead.c        |  29 ++
 .../riscv/kernel/vendor_extensions/thead_hwprobe.c |  19 ++
 drivers/base/cpu.c                                 |   3 +
 include/linux/cpu.h                                |   1 +
 tools/testing/selftests/riscv/vector/.gitignore    |   3 +-
 tools/testing/selftests/riscv/vector/Makefile      |  17 +-
 .../selftests/riscv/vector/v_exec_initval_nolibc.c |  94 +++++++
 tools/testing/selftests/riscv/vector/v_helpers.c   |  68 +++++
 tools/testing/selftests/riscv/vector/v_helpers.h   |   8 +
 tools/testing/selftests/riscv/vector/v_initval.c   |  22 ++
 .../selftests/riscv/vector/v_initval_nolibc.c      |  68 -----
 .../selftests/riscv/vector/vstate_exec_nolibc.c    |  20 +-
 .../testing/selftests/riscv/vector/vstate_prctl.c  | 305 +++++++++++++--------
 42 files changed, 1048 insertions(+), 272 deletions(-)
---
base-commit: 0e3f3649d44bf1b388a7613ade14c29cbdedf075
change-id: 20240530-xtheadvector-833d3d17b423

Comments

Aoba K Sept. 29, 2024, 12:44 p.m. UTC | #1

Hello Charlie,

I've been working on bringing up the Sipeed Lichee RV Dock
(which also uses the D1 SoC) with the kernel patches you provided.
The patches applied cleanly to Palmer's for-next branch,
but I've encountered a couple of issues:

1. Skiffos Compilation Error during the compilation process of `cgo`:
`unknown relocation type 17; compiled without -fpic?`
Unfortunately, I closed the terminal before saving the full log,
so I don't have the complete details, but the result should be reproducible.
While this should be a SkiffOS issue, mention it in case SkiffOS is the method
that you mentioned for bringing up the device.

2. Image Building with sehraf/riscv-arch-image-builder:
After building the image, the device failed to start at an early stage.
I suspect this may be related to incorrect RAM size detection,
as the board only has 512MB of RAM.
Interestingly, the vendor image reports 1GB, and the Sipeed website also states
that the Dock has 1GB, despite there being no extra memory bank present.

You can find the boot log here: https://fars.ee/bdYk.log

Any help would be appreciated, and big thanks to your work
to make the efficient part of this board to work (again)!

Cheers,
Aoba

Aoba K Sept. 29, 2024, 4:07 p.m. UTC | #2

On 2024/9/12 13:55, Charlie Jenkins wrote:
> xtheadvector is a custom extension that is based upon riscv vector
> version 0.7.1 [1]. All of the vector routines have been modified to
> support this alternative vector version based upon whether xtheadvector
> was determined to be supported at boot.
>
> vlenb is not supported on the existing xtheadvector hardware, so a
> devicetree property thead,vlenb is added to provide the vlenb to Linux.
>
> There is a new hwprobe key RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 that is
> used to request which thead vendor extensions are supported on the
> current platform. This allows future vendors to allocate hwprobe keys
> for their vendor.
>
> Support for xtheadvector is also added to the vector kselftests.
>
> Signed-off-by: Charlie Jenkins <[e-mail]>
>
> [1] https://github.com/T-head-Semi/thead-extension-spec/blob/95358cb2cca9489361c61d335e03d3134b14133f/xtheadvector.adoc
>
> ---
> This series is a continuation of a different series that was fragmented
> into two other series in an attempt to get part of it merged in the 6.10
> merge window. The split-off series did not get merged due to a NAK on
> the series that added the generic riscv,vlenb devicetree entry. This
> series has converted riscv,vlenb to thead,vlenb to remedy this issue.
>
> The original series is titled "riscv: Support vendor extensions and
> xtheadvector" [3].
>
> The series titled "riscv: Extend cpufeature.c to detect vendor
> extensions" is still under development and this series is based on that
> series! [4]
>
> I have tested this with an Allwinner Nezha board. I used SkiffOS [1] to
> manage building the image, but upgraded the U-Boot version to Samuel
> Holland's more up-to-date version [2] and changed out the device tree
> used by U-Boot with the device trees that are present in upstream linux
> and this series. Thank you Samuel for all of the work you did to make
> this task possible.
>
> [1] https://github.com/skiffos/SkiffOS/tree/master/configs/allwinner/nezha
> [2] https://github.com/smaeul/u-boot/commit/2e89b706f5c956a70c989cd31665f1429e9a0b48
> [3] https://lore.kernel.org/all/[e-mail]/
> [4] https://lore.kernel.org/lkml/[e-mail]/T/
>
> ---
> Changes in v10:
> - In DT probing disable vector with new function to clear vendor
>    extension bits for xtheadvector
> - Add ghostwrite mitigations for c9xx CPUs. This disables xtheadvector
>    unless mitigations=off is set as a kernel boot arg
> - Link to v9: https://lore.kernel.org/r/[e-mail]
>
> Changes in v9:
> - Rebase onto palmer's for-next
> - Fix sparse error in arch/riscv/kernel/vendor_extensions/thead.c
> - Fix maybe-uninitialized warning in arch/riscv/include/asm/vendor_extensions/vendor_hwprobe.h
> - Wrap some long lines
> - Link to v8: https://lore.kernel.org/r/[e-mail]
>
> Changes in v8:
> - Rebase onto palmer's for-next
> - Link to v7: https://lore.kernel.org/r/[e-mail]
>
> Changes in v7:
> - Add defs for has_xtheadvector_no_alternatives() and has_xtheadvector()
>    when vector disabled. (Palmer)
> - Link to v6: https://lore.kernel.org/r/[e-mail]
>
> Changes in v6:
> - Fix return type of is_vector_supported()/is_xthead_supported() to be bool
> - Link to v5: https://lore.kernel.org/r/[e-mail]
>
> Changes in v5:
> - Rebase on for-next
> - Link to v4: https://lore.kernel.org/r/[e-mail]
>
> Changes in v4:
> - Replace inline asm with C (Samuel)
> - Rename VCSRs to CSRs (Samuel)
> - Replace .insn directives with .4byte directives
> - Link to v3: https://lore.kernel.org/r/[e-mail]
>
> Changes in v3:
> - Add back Heiko's signed-off-by (Conor)
> - Mark RISCV_HWPROBE_KEY_VENDOR_EXT_THEAD_0 as a bitmask
> - Link to v2: https://lore.kernel.org/r/[e-mail]
>
> Changes in v2:
> - Removed extraneous references to "riscv,vlenb" (Jess)
> - Moved declaration of "thead,vlenb" into cpus.yaml and added
>    restriction that it's only applicable to thead cores (Conor)
> - Check CONFIG_RISCV_ISA_XTHEADVECTOR instead of CONFIG_RISCV_ISA_V for
>    thead,vlenb (Jess)
> - Fix naming of hwprobe variables (Evan)
> - Link to v1: https://lore.kernel.org/r/[e-mail]
>
> ---
> Charlie Jenkins (13):
>        dt-bindings: riscv: Add xtheadvector ISA extension description
>        dt-bindings: cpus: add a thead vlen register length property
>        riscv: dts: allwinner: Add xtheadvector to the D1/D1s devicetree
>        riscv: Add thead and xtheadvector as a vendor extension
>        riscv: vector: Use vlenb from DT for thead
>        riscv: csr: Add CSR encodings for CSR_VXRM/CSR_VXSAT
>        riscv: Add xtheadvector instruction definitions
>        riscv: vector: Support xtheadvector save/restore
>        riscv: hwprobe: Add thead vendor extension probing
>        riscv: hwprobe: Document thead vendor extensions and xtheadvector extension
>        selftests: riscv: Fix vector tests
>        selftests: riscv: Support xtheadvector in vector tests
>        riscv: Add ghostwrite vulnerability
>
> Heiko Stuebner (1):
>        RISC-V: define the elements of the VCSR vector CSR
>
>   Documentation/arch/riscv/hwprobe.rst               |  10 +
>   Documentation/devicetree/bindings/riscv/cpus.yaml  |  19 ++
>   .../devicetree/bindings/riscv/extensions.yaml      |  10 +
>   arch/riscv/Kconfig.errata                          |  11 +
>   arch/riscv/Kconfig.vendor                          |  26 ++
>   arch/riscv/boot/dts/allwinner/sun20i-d1s.dtsi      |   3 +-
>   arch/riscv/errata/thead/errata.c                   |  28 ++
>   arch/riscv/include/asm/bugs.h                      |  22 ++
>   arch/riscv/include/asm/cpufeature.h                |   2 +
>   arch/riscv/include/asm/csr.h                       |  15 +
>   arch/riscv/include/asm/errata_list.h               |   3 +-
>   arch/riscv/include/asm/hwprobe.h                   |   3 +-
>   arch/riscv/include/asm/switch_to.h                 |   2 +-
>   arch/riscv/include/asm/vector.h                    | 225 +++++++++++----
>   arch/riscv/include/asm/vendor_extensions/thead.h   |  48 ++++
>   .../include/asm/vendor_extensions/thead_hwprobe.h  |  19 ++
>   .../include/asm/vendor_extensions/vendor_hwprobe.h |  37 +++
>   arch/riscv/include/uapi/asm/hwprobe.h              |   3 +-
>   arch/riscv/include/uapi/asm/vendor/thead.h         |   3 +
>   arch/riscv/kernel/Makefile                         |   2 +
>   arch/riscv/kernel/bugs.c                           |  55 ++++
>   arch/riscv/kernel/cpufeature.c                     |  58 +++-
>   arch/riscv/kernel/kernel_mode_vector.c             |   8 +-
>   arch/riscv/kernel/process.c                        |   4 +-
>   arch/riscv/kernel/signal.c                         |   6 +-
>   arch/riscv/kernel/sys_hwprobe.c                    |   5 +
>   arch/riscv/kernel/vector.c                         |  24 +-
>   arch/riscv/kernel/vendor_extensions.c              |  10 +
>   arch/riscv/kernel/vendor_extensions/Makefile       |   2 +
>   arch/riscv/kernel/vendor_extensions/thead.c        |  29 ++
>   .../riscv/kernel/vendor_extensions/thead_hwprobe.c |  19 ++
>   drivers/base/cpu.c                                 |   3 +
>   include/linux/cpu.h                                |   1 +
>   tools/testing/selftests/riscv/vector/.gitignore    |   3 +-
>   tools/testing/selftests/riscv/vector/Makefile      |  17 +-
>   .../selftests/riscv/vector/v_exec_initval_nolibc.c |  94 +++++++
>   tools/testing/selftests/riscv/vector/v_helpers.c   |  68 +++++
>   tools/testing/selftests/riscv/vector/v_helpers.h   |   8 +
>   tools/testing/selftests/riscv/vector/v_initval.c   |  22 ++
>   .../selftests/riscv/vector/v_initval_nolibc.c      |  68 -----
>   .../selftests/riscv/vector/vstate_exec_nolibc.c    |  20 +-
>   .../testing/selftests/riscv/vector/vstate_prctl.c  | 305 +++++++++++++--------
>   42 files changed, 1048 insertions(+), 272 deletions(-)
> ---
> base-commit: 0e3f3649d44bf1b388a7613ade14c29cbdedf075
> change-id: 20240530-xtheadvector-833d3d17b423


Hello Charlie,


Apologize for the last email, when dealing with pastebin it deleted the 
last version...

tl,dr for the last email: Patches not boot on Sipeed Lichee RV Dock 
(with same D1 SoC).

Logs here:

https://fars.ee/XFzR

(the board resets without kernel panic)


Cheers,

Aoba K

Conor Dooley Sept. 30, 2024, 2:53 p.m. UTC | #3

On Sun, Sep 29, 2024 at 12:44:07PM +0000, Aoba K wrote:
> Hello Charlie,
> 
> I've been working on bringing up the Sipeed Lichee RV Dock
> (which also uses the D1 SoC) with the kernel patches you provided.
> The patches applied cleanly to Palmer's for-next branch,
> but I've encountered a couple of issues:
> 
> 1. Skiffos Compilation Error during the compilation process of `cgo`:
> `unknown relocation type 17; compiled without -fpic?`
> Unfortunately, I closed the terminal before saving the full log,
> so I don't have the complete details, but the result should be reproducible.
> While this should be a SkiffOS issue, mention it in case SkiffOS is the method
> that you mentioned for bringing up the device.
> 
> 2. Image Building with sehraf/riscv-arch-image-builder:
> After building the image, the device failed to start at an early stage.
> I suspect this may be related to incorrect RAM size detection,
> as the board only has 512MB of RAM.
> Interestingly, the vendor image reports 1GB, and the Sipeed website also states
> that the Dock has 1GB, despite there being no extra memory bank present.
> 
> You can find the boot log here: https://fars.ee/bdYk.log
> 
> Any help would be appreciated, and big thanks to your work
> to make the efficient part of this board to work (again)!

The log you posted at https://fars.ee/XFzR appears to be using the
devicetree of a Nezha not the Lichee RV Dock. Why are you doing that,
when the Lichee RV Dock is supported in the kernel already?

Cheers,
Conor.

Aoba K Sept. 30, 2024, 3:50 p.m. UTC | #4

The devicetree name shown in OpenSBI is
the one packed with U-Boot SPL.
As the image builder didn't put the kernel
devicetree with the image under boot partition,
The whole boot process only uses the U-Boot device tree.

The reason why the log shows "Nezha"
but not "Lichee RV" is that this log is produced
which U-Boot revision is d1-2022-10-31 from samuel's repo.
Build from d1-wip can show "Lichee RV" correctly
(manually edit config to specify LicheeRV dtb should have same effect though),
while the issue persists when kernel load to
the procedure same as the dmesg mentioned above.


Cheers,
Aoba K

2024年9月30日 下午10:53:40 Conor Dooley <[e-mail]>:

> On Sun, Sep 29, 2024 at 12:44:07PM +0000, Aoba K wrote:
>> Hello Charlie,
>> 
>> I've been working on bringing up the Sipeed Lichee RV Dock
>> (which also uses the D1 SoC) with the kernel patches you provided.
>> The patches applied cleanly to Palmer's for-next branch,
>> but I've encountered a couple of issues:
>> 
>> 1. Skiffos Compilation Error during the compilation process of `cgo`:
>> `unknown relocation type 17; compiled without -fpic?`
>> Unfortunately, I closed the terminal before saving the full log,
>> so I don't have the complete details, but the result should be reproducible.
>> While this should be a SkiffOS issue, mention it in case SkiffOS is the method
>> that you mentioned for bringing up the device.
>> 
>> 2. Image Building with sehraf/riscv-arch-image-builder:
>> After building the image, the device failed to start at an early stage.
>> I suspect this may be related to incorrect RAM size detection,
>> as the board only has 512MB of RAM.
>> Interestingly, the vendor image reports 1GB, and the Sipeed website also states
>> that the Dock has 1GB, despite there being no extra memory bank present.
>> 
>> You can find the boot log here: https://fars.ee/bdYk.log
>> 
>> Any help would be appreciated, and big thanks to your work
>> to make the efficient part of this board to work (again)!
> 
> The log you posted at https://fars.ee/XFzR appears to be using the
> devicetree of a Nezha not the Lichee RV Dock. Why are you doing that,
> when the Lichee RV Dock is supported in the kernel already?
> 
> Cheers,
> Conor.