| Message ID | 20230313-riscv-zicsr-zifencei-fiasco-v1-1-dd1b7840a551@kernel.org (mailing list archive) |
|---|---|
| State | Accepted |
| Commit | e89c2e815e76471cb507bd95728bf26da7976430 |
| Headers | show |
| Series | riscv: Handle zicsr/zifencei issues between clang and binutils | expand |
| Context | Check | Description |
|---|---|---|
| conchuod/alphanumeric_selects | warning | Out of order selects before the patch: 728 and now 750 |
| conchuod/cover_letter | success | Single patches do not need cover letters |
| conchuod/tree_selection | success | Guessed tree name to be for-next |
| conchuod/fixes_present | success | Fixes tag not required for -next series |
| conchuod/maintainers_pattern | success | MAINTAINERS pattern errors before the patch: 1 and now 1 |
| conchuod/verify_signedoff | success | Signed-off-by tag matches author and committer |
| conchuod/kdoc | success | Errors and warnings before: 0 this patch: 0 |
| conchuod/build_rv64_clang_allmodconfig | success | Errors and warnings before: 18 this patch: 18 |
| conchuod/module_param | success | Was 0 now: 0 |
| conchuod/build_rv64_gcc_allmodconfig | success | Errors and warnings before: 18 this patch: 18 |
| conchuod/build_rv32_defconfig | success | Build OK |
| conchuod/dtb_warn_rv64 | success | Errors and warnings before: 3 this patch: 3 |
| conchuod/header_inline | success | No static functions without inline keyword in header files |
| conchuod/checkpatch | success | total: 0 errors, 0 warnings, 0 checks, 44 lines checked |
| conchuod/source_inline | success | Was 0 now: 0 |
| conchuod/build_rv64_nommu_k210_defconfig | success | Build OK |
| conchuod/verify_fixes | success | No Fixes tag |
| conchuod/build_rv64_nommu_virt_defconfig | success | Build OK |
Hey Nathan, On Mon, Mar 13, 2023 at 04:00:23PM -0700, Nathan Chancellor wrote: > There are two related issues that appear in certain combinations with > clang and GNU binutils. > > The first occurs when a version of clang that supports zicsr or zifencei > via '-march=' [1] (i.e, >= 17.x) is used in combination with a version > of GNU binutils that do not recognize zicsr and zifencei in the > '-march=' value (i.e., < 2.36): > > riscv64-linux-gnu-ld: -march=rv64i2p0_m2p0_a2p0_c2p0_zicsr2p0_zifencei2p0: Invalid or unknown z ISA extension: 'zifencei' > riscv64-linux-gnu-ld: failed to merge target specific data of file fs/efivarfs/file.o > riscv64-linux-gnu-ld: -march=rv64i2p0_m2p0_a2p0_c2p0_zicsr2p0_zifencei2p0: Invalid or unknown z ISA extension: 'zifencei' > riscv64-linux-gnu-ld: failed to merge target specific data of file fs/efivarfs/super.o > > The second occurs when a version of clang that does not support zicsr or > zifencei via '-march=' (i.e., <= 16.x) is used in combination with a > version of GNU as that defaults to a newer ISA base spec, which requires > specifying zicsr and zifencei in the '-march=' value explicitly (i.e, >= > 2.38): > > ../arch/riscv/kernel/kexec_relocate.S: Assembler messages: > ../arch/riscv/kernel/kexec_relocate.S:147: Error: unrecognized opcode `fence.i', extension `zifencei' required > clang-12: error: assembler command failed with exit code 1 (use -v to see invocation) > > This is the same issue addressed by commit 6df2a016c0c8 ("riscv: fix > build with binutils 2.38") (see [2] for additional information) but > older versions of clang miss out on it because the cc-option check > fails: > > clang-12: error: invalid arch name 'rv64imac_zicsr_zifencei', unsupported standard user-level extension 'zicsr' > clang-12: error: invalid arch name 'rv64imac_zicsr_zifencei', unsupported standard user-level extension 'zicsr' > > To resolve the first issue, only attempt to add zicsr and zifencei to > the march string when using the GNU assembler 2.38 or newer, which is > when the default ISA spec was updated, requiring these extensions to be > specified explicitly. LLVM implements an older version of the base > specification for all currently released versions, so these instructions > are available as part of the 'i' extension. If LLVM's implementation is > updated in the future, a CONFIG_AS_IS_LLVM condition can be added to > CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI. > > To resolve the second issue, use version 2.2 of the base ISA spec when > using an older version of clang that does not support zicsr or zifencei > via '-march=', as that is the spec version most compatible with the one > clang/LLVM implements and avoids the need to specify zicsr and zifencei > explicitly due to still being a part of 'i'. > > [1]: https://github.com/llvm/llvm-project/commit/22e199e6afb1263c943c0c0d4498694e15bf8a16 > [2]: https://lore.kernel.org/ZAxT7T9Xy1Fo3d5W@aurel32.net/ > > Cc: stable@vger.kernel.org > Link: https://github.com/ClangBuiltLinux/linux/issues/1808 > Co-developed-by: Conor Dooley <conor.dooley@microchip.com> > Signed-off-by: Conor Dooley <conor.dooley@microchip.com> TBH, barely identifiable as having much to do with my V1 anymore, so could easily have dropped those. Either way, thanks for sorting this out while I've been sick :) Acked-by: Conor Dooley <conor.dooley@microchip.com> > Signed-off-by: Nathan Chancellor <nathan@kernel.org> > --- > This is essentially a v3 of Conor's v1 and v2 but since I am sending the > patch after finding a separate but related issue, I left it at v1: > > - v1: https://lore.kernel.org/20230223220546.52879-1-conor@kernel.org/ > - v2: https://lore.kernel.org/20230308220842.1231003-1-conor@kernel.org/ > > I have built allmodconfig with the following toolchain combinations to > confirm this problem is resolved: > > - clang 12/17 + GNU as and ld 2.35/2.39 > - clang 12/17 with the integrated assembler + GNU ld 2.35/2.39 > - clang 12/17 with the integrated assembler + ld.lld > > There are a couple of other incompatibilities between clang-17 and GNU > binutils that I had to patch to get allmodconfig to build successfully > but those are less likely to be hit in practice because the full LLVM > stack can be used with LLVM versions 13.x and newer. > I will follow up > with separate issues and patches. Cool, I'll "look forward" to those... Thanks, Conor.
On Mon, 13 Mar 2023 16:00:23 -0700, Nathan Chancellor wrote: > There are two related issues that appear in certain combinations with > clang and GNU binutils. > > The first occurs when a version of clang that supports zicsr or zifencei > via '-march=' [1] (i.e, >= 17.x) is used in combination with a version > of GNU binutils that do not recognize zicsr and zifencei in the > '-march=' value (i.e., < 2.36): > > [...] Applied, thanks! [1/1] riscv: Handle zicsr/zifencei issues between clang and binutils https://git.kernel.org/palmer/c/e89c2e815e76 Best regards,
On Mon, 13 Mar 2023 16:00:23 PDT (-0700), nathan@kernel.org wrote: > There are two related issues that appear in certain combinations with > clang and GNU binutils. > > The first occurs when a version of clang that supports zicsr or zifencei > via '-march=' [1] (i.e, >= 17.x) is used in combination with a version > of GNU binutils that do not recognize zicsr and zifencei in the > '-march=' value (i.e., < 2.36): > > riscv64-linux-gnu-ld: -march=rv64i2p0_m2p0_a2p0_c2p0_zicsr2p0_zifencei2p0: Invalid or unknown z ISA extension: 'zifencei' > riscv64-linux-gnu-ld: failed to merge target specific data of file fs/efivarfs/file.o > riscv64-linux-gnu-ld: -march=rv64i2p0_m2p0_a2p0_c2p0_zicsr2p0_zifencei2p0: Invalid or unknown z ISA extension: 'zifencei' > riscv64-linux-gnu-ld: failed to merge target specific data of file fs/efivarfs/super.o > > The second occurs when a version of clang that does not support zicsr or > zifencei via '-march=' (i.e., <= 16.x) is used in combination with a > version of GNU as that defaults to a newer ISA base spec, which requires > specifying zicsr and zifencei in the '-march=' value explicitly (i.e, >= > 2.38): > > ../arch/riscv/kernel/kexec_relocate.S: Assembler messages: > ../arch/riscv/kernel/kexec_relocate.S:147: Error: unrecognized opcode `fence.i', extension `zifencei' required > clang-12: error: assembler command failed with exit code 1 (use -v to see invocation) > > This is the same issue addressed by commit 6df2a016c0c8 ("riscv: fix > build with binutils 2.38") (see [2] for additional information) but > older versions of clang miss out on it because the cc-option check > fails: > > clang-12: error: invalid arch name 'rv64imac_zicsr_zifencei', unsupported standard user-level extension 'zicsr' > clang-12: error: invalid arch name 'rv64imac_zicsr_zifencei', unsupported standard user-level extension 'zicsr' > > To resolve the first issue, only attempt to add zicsr and zifencei to > the march string when using the GNU assembler 2.38 or newer, which is > when the default ISA spec was updated, requiring these extensions to be > specified explicitly. LLVM implements an older version of the base > specification for all currently released versions, so these instructions > are available as part of the 'i' extension. If LLVM's implementation is > updated in the future, a CONFIG_AS_IS_LLVM condition can be added to > CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI. > > To resolve the second issue, use version 2.2 of the base ISA spec when > using an older version of clang that does not support zicsr or zifencei > via '-march=', as that is the spec version most compatible with the one > clang/LLVM implements and avoids the need to specify zicsr and zifencei > explicitly due to still being a part of 'i'. > > [1]: https://github.com/llvm/llvm-project/commit/22e199e6afb1263c943c0c0d4498694e15bf8a16 > [2]: https://lore.kernel.org/ZAxT7T9Xy1Fo3d5W@aurel32.net/ > > Cc: stable@vger.kernel.org > Link: https://github.com/ClangBuiltLinux/linux/issues/1808 > Co-developed-by: Conor Dooley <conor.dooley@microchip.com> > Signed-off-by: Conor Dooley <conor.dooley@microchip.com> > Signed-off-by: Nathan Chancellor <nathan@kernel.org> > --- > This is essentially a v3 of Conor's v1 and v2 but since I am sending the > patch after finding a separate but related issue, I left it at v1: > > - v1: https://lore.kernel.org/20230223220546.52879-1-conor@kernel.org/ > - v2: https://lore.kernel.org/20230308220842.1231003-1-conor@kernel.org/ > > I have built allmodconfig with the following toolchain combinations to > confirm this problem is resolved: > > - clang 12/17 + GNU as and ld 2.35/2.39 > - clang 12/17 with the integrated assembler + GNU ld 2.35/2.39 > - clang 12/17 with the integrated assembler + ld.lld > > There are a couple of other incompatibilities between clang-17 and GNU > binutils that I had to patch to get allmodconfig to build successfully > but those are less likely to be hit in practice because the full LLVM > stack can be used with LLVM versions 13.x and newer. I will follow up > with separate issues and patches. > --- > arch/riscv/Kconfig | 22 ++++++++++++++++++++++ > arch/riscv/Makefile | 10 ++++++---- > 2 files changed, 28 insertions(+), 4 deletions(-) > > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig > index c5e42cc37604..5b182d1c196c 100644 > --- a/arch/riscv/Kconfig > +++ b/arch/riscv/Kconfig > @@ -464,6 +464,28 @@ config TOOLCHAIN_HAS_ZIHINTPAUSE > depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zihintpause) > depends on LLD_VERSION >= 150000 || LD_VERSION >= 23600 > > +config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI > + def_bool y > + # https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc > + depends on AS_IS_GNU && AS_VERSION >= 23800 > + help > + Newer binutils versions default to ISA spec version 20191213 which > + moves some instructions from the I extension to the Zicsr and Zifencei > + extensions. > + > +config TOOLCHAIN_NEEDS_OLD_ISA_SPEC > + def_bool y > + depends on TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI > + # https://github.com/llvm/llvm-project/commit/22e199e6afb1263c943c0c0d4498694e15bf8a16 > + depends on CC_IS_CLANG && CLANG_VERSION < 170000 > + help > + Certain versions of clang do not support zicsr and zifencei via -march > + but newer versions of binutils require it for the reasons noted in the > + help text of CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI. This > + option causes an older ISA spec compatible with these older versions > + of clang to be passed to GAS, which has the same result as passing zicsr > + and zifencei to -march. > + > config FPU > bool "FPU support" > default y > diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile > index 4de83b9b1772..b05e833a022d 100644 > --- a/arch/riscv/Makefile > +++ b/arch/riscv/Makefile > @@ -57,10 +57,12 @@ riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima > riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd > riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c > > -# Newer binutils versions default to ISA spec version 20191213 which moves some > -# instructions from the I extension to the Zicsr and Zifencei extensions. > -toolchain-need-zicsr-zifencei := $(call cc-option-yn, -march=$(riscv-march-y)_zicsr_zifencei) > -riscv-march-$(toolchain-need-zicsr-zifencei) := $(riscv-march-y)_zicsr_zifencei > +ifdef CONFIG_TOOLCHAIN_NEEDS_OLD_ISA_SPEC > +KBUILD_CFLAGS += -Wa,-misa-spec=2.2 > +KBUILD_AFLAGS += -Wa,-misa-spec=2.2 > +else > +riscv-march-$(CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI) := $(riscv-march-y)_zicsr_zifencei > +endif > > # Check if the toolchain supports Zihintpause extension > riscv-march-$(CONFIG_TOOLCHAIN_HAS_ZIHINTPAUSE) := $(riscv-march-y)_zihintpause > > --- > base-commit: eeac8ede17557680855031c6f305ece2378af326 > change-id: 20230313-riscv-zicsr-zifencei-fiasco-2941caebe7dc Is that a b4 thing? Having change IDs with names is nice, it's way easier to remember what's what when sorting through backports. Also, this is on fixes. Thanks! > Best regards,
On Thu, Mar 23, 2023 at 01:49:57PM -0700, Palmer Dabbelt wrote: > On Mon, 13 Mar 2023 16:00:23 PDT (-0700), nathan@kernel.org wrote: > > base-commit: eeac8ede17557680855031c6f305ece2378af326 > > change-id: 20230313-riscv-zicsr-zifencei-fiasco-2941caebe7dc > > Is that a b4 thing? Having change IDs with names is nice, it's way easier > to remember what's what when sorting through backports. It's `b4 send`, for anyone that's lurking on the list and hasn't seen it before: https://b4.docs.kernel.org/en/latest/contributor/send.html b4 is now a tool for contributing, not just maintaining, although I'm not sure that the distro copies of it are recent enough to make use of those features.
Hello: This patch was applied to riscv/linux.git (fixes) by Palmer Dabbelt <palmer@rivosinc.com>: On Mon, 13 Mar 2023 16:00:23 -0700 you wrote: > There are two related issues that appear in certain combinations with > clang and GNU binutils. > > The first occurs when a version of clang that supports zicsr or zifencei > via '-march=' [1] (i.e, >= 17.x) is used in combination with a version > of GNU binutils that do not recognize zicsr and zifencei in the > '-march=' value (i.e., < 2.36): > > [...] Here is the summary with links: - riscv: Handle zicsr/zifencei issues between clang and binutils https://git.kernel.org/riscv/c/e89c2e815e76 You are awesome, thank you!
diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index c5e42cc37604..5b182d1c196c 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -464,6 +464,28 @@ config TOOLCHAIN_HAS_ZIHINTPAUSE depends on !32BIT || $(cc-option,-mabi=ilp32 -march=rv32ima_zihintpause) depends on LLD_VERSION >= 150000 || LD_VERSION >= 23600 +config TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI + def_bool y + # https://sourceware.org/git/?p=binutils-gdb.git;a=commit;h=aed44286efa8ae8717a77d94b51ac3614e2ca6dc + depends on AS_IS_GNU && AS_VERSION >= 23800 + help + Newer binutils versions default to ISA spec version 20191213 which + moves some instructions from the I extension to the Zicsr and Zifencei + extensions. + +config TOOLCHAIN_NEEDS_OLD_ISA_SPEC + def_bool y + depends on TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI + # https://github.com/llvm/llvm-project/commit/22e199e6afb1263c943c0c0d4498694e15bf8a16 + depends on CC_IS_CLANG && CLANG_VERSION < 170000 + help + Certain versions of clang do not support zicsr and zifencei via -march + but newer versions of binutils require it for the reasons noted in the + help text of CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI. This + option causes an older ISA spec compatible with these older versions + of clang to be passed to GAS, which has the same result as passing zicsr + and zifencei to -march. + config FPU bool "FPU support" default y diff --git a/arch/riscv/Makefile b/arch/riscv/Makefile index 4de83b9b1772..b05e833a022d 100644 --- a/arch/riscv/Makefile +++ b/arch/riscv/Makefile @@ -57,10 +57,12 @@ riscv-march-$(CONFIG_ARCH_RV64I) := rv64ima riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c -# Newer binutils versions default to ISA spec version 20191213 which moves some -# instructions from the I extension to the Zicsr and Zifencei extensions. -toolchain-need-zicsr-zifencei := $(call cc-option-yn, -march=$(riscv-march-y)_zicsr_zifencei) -riscv-march-$(toolchain-need-zicsr-zifencei) := $(riscv-march-y)_zicsr_zifencei +ifdef CONFIG_TOOLCHAIN_NEEDS_OLD_ISA_SPEC +KBUILD_CFLAGS += -Wa,-misa-spec=2.2 +KBUILD_AFLAGS += -Wa,-misa-spec=2.2 +else +riscv-march-$(CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI) := $(riscv-march-y)_zicsr_zifencei +endif # Check if the toolchain supports Zihintpause extension riscv-march-$(CONFIG_TOOLCHAIN_HAS_ZIHINTPAUSE) := $(riscv-march-y)_zihintpause
There are two related issues that appear in certain combinations with clang and GNU binutils. The first occurs when a version of clang that supports zicsr or zifencei via '-march=' [1] (i.e, >= 17.x) is used in combination with a version of GNU binutils that do not recognize zicsr and zifencei in the '-march=' value (i.e., < 2.36): riscv64-linux-gnu-ld: -march=rv64i2p0_m2p0_a2p0_c2p0_zicsr2p0_zifencei2p0: Invalid or unknown z ISA extension: 'zifencei' riscv64-linux-gnu-ld: failed to merge target specific data of file fs/efivarfs/file.o riscv64-linux-gnu-ld: -march=rv64i2p0_m2p0_a2p0_c2p0_zicsr2p0_zifencei2p0: Invalid or unknown z ISA extension: 'zifencei' riscv64-linux-gnu-ld: failed to merge target specific data of file fs/efivarfs/super.o The second occurs when a version of clang that does not support zicsr or zifencei via '-march=' (i.e., <= 16.x) is used in combination with a version of GNU as that defaults to a newer ISA base spec, which requires specifying zicsr and zifencei in the '-march=' value explicitly (i.e, >= 2.38): ../arch/riscv/kernel/kexec_relocate.S: Assembler messages: ../arch/riscv/kernel/kexec_relocate.S:147: Error: unrecognized opcode `fence.i', extension `zifencei' required clang-12: error: assembler command failed with exit code 1 (use -v to see invocation) This is the same issue addressed by commit 6df2a016c0c8 ("riscv: fix build with binutils 2.38") (see [2] for additional information) but older versions of clang miss out on it because the cc-option check fails: clang-12: error: invalid arch name 'rv64imac_zicsr_zifencei', unsupported standard user-level extension 'zicsr' clang-12: error: invalid arch name 'rv64imac_zicsr_zifencei', unsupported standard user-level extension 'zicsr' To resolve the first issue, only attempt to add zicsr and zifencei to the march string when using the GNU assembler 2.38 or newer, which is when the default ISA spec was updated, requiring these extensions to be specified explicitly. LLVM implements an older version of the base specification for all currently released versions, so these instructions are available as part of the 'i' extension. If LLVM's implementation is updated in the future, a CONFIG_AS_IS_LLVM condition can be added to CONFIG_TOOLCHAIN_NEEDS_EXPLICIT_ZICSR_ZIFENCEI. To resolve the second issue, use version 2.2 of the base ISA spec when using an older version of clang that does not support zicsr or zifencei via '-march=', as that is the spec version most compatible with the one clang/LLVM implements and avoids the need to specify zicsr and zifencei explicitly due to still being a part of 'i'. [1]: https://github.com/llvm/llvm-project/commit/22e199e6afb1263c943c0c0d4498694e15bf8a16 [2]: https://lore.kernel.org/ZAxT7T9Xy1Fo3d5W@aurel32.net/ Cc: stable@vger.kernel.org Link: https://github.com/ClangBuiltLinux/linux/issues/1808 Co-developed-by: Conor Dooley <conor.dooley@microchip.com> Signed-off-by: Conor Dooley <conor.dooley@microchip.com> Signed-off-by: Nathan Chancellor <nathan@kernel.org> --- This is essentially a v3 of Conor's v1 and v2 but since I am sending the patch after finding a separate but related issue, I left it at v1: - v1: https://lore.kernel.org/20230223220546.52879-1-conor@kernel.org/ - v2: https://lore.kernel.org/20230308220842.1231003-1-conor@kernel.org/ I have built allmodconfig with the following toolchain combinations to confirm this problem is resolved: - clang 12/17 + GNU as and ld 2.35/2.39 - clang 12/17 with the integrated assembler + GNU ld 2.35/2.39 - clang 12/17 with the integrated assembler + ld.lld There are a couple of other incompatibilities between clang-17 and GNU binutils that I had to patch to get allmodconfig to build successfully but those are less likely to be hit in practice because the full LLVM stack can be used with LLVM versions 13.x and newer. I will follow up with separate issues and patches. --- arch/riscv/Kconfig | 22 ++++++++++++++++++++++ arch/riscv/Makefile | 10 ++++++---- 2 files changed, 28 insertions(+), 4 deletions(-) --- base-commit: eeac8ede17557680855031c6f305ece2378af326 change-id: 20230313-riscv-zicsr-zifencei-fiasco-2941caebe7dc Best regards,