| Message ID | 20240111131558.31211-1-andy.chiu@sifive.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | riscv: support kernel-mode Vector | expand |
Hello: This series was applied to riscv/linux.git (for-next) by Palmer Dabbelt <palmer@rivosinc.com>: On Thu, 11 Jan 2024 13:15:48 +0000 you wrote: > This series provides support running Vector in kernel mode. > Additionally, kernel-mode Vector can be configured to run without > turnning off preemption on a CONFIG_PREEMPT kernel. Along with the > suport, we add Vector optimized copy_{to,from}_user. And provide a > simple threshold to decide when to run the vectorized functions. > > We decided to drop vectorized memcpy/memset/memmove for the moment due > to the concern of memory side-effect in kernel_vector_begin(). The > detailed description can be found at v9[1] > > [...] Here is the summary with links: - [v10,01/10] riscv: Add support for kernel mode vector https://git.kernel.org/riscv/c/c0ae350f714f - [v10,02/10] riscv: vector: make Vector always available for softirq context https://git.kernel.org/riscv/c/ebf52ac30e4f - [v10,03/10] riscv: Add vector extension XOR implementation https://git.kernel.org/riscv/c/9ff97211a623 - [v10,04/10] riscv: sched: defer restoring Vector context for user https://git.kernel.org/riscv/c/f4471252f3b9 - [v10,05/10] riscv: lib: vectorize copy_to_user/copy_from_user https://git.kernel.org/riscv/c/145ca6eddd70 - [v10,06/10] riscv: fpu: drop SR_SD bit checking https://git.kernel.org/riscv/c/25a830944773 - [v10,07/10] riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() https://git.kernel.org/riscv/c/9dece8bf0343 - [v10,08/10] riscv: vector: use a mask to write vstate_ctrl https://git.kernel.org/riscv/c/c05992747c96 - [v10,09/10] riscv: vector: use kmem_cache to manage vector context https://git.kernel.org/riscv/c/660217429614 - [v10,10/10] riscv: vector: allow kernel-mode Vector with preemption https://git.kernel.org/riscv/c/aa23c6172d33 You are awesome, thank you!
Andy, > Hello: > > This series was applied to riscv/linux.git (for-next) > by Palmer Dabbelt <palmer@rivosinc.com>: > I'm getting some boot issues with this series applied to riscv/for-next. The full runs (with logs) is here: https://github.com/linux-riscv/linux-riscv/actions/runs/7498706326 Typically it fails in two ways: Ubuntu rootfs: --8<-- [ 4.346414] (sd-gens)[68]: Failed to extract file name from '': Invalid argument [ 4.390832] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. [ 4.440164] systemd[1]: Freezing execution. --8<-- or: --8<-- [ 14.909912] (sd-gens)[71]: Assertion '!strv_isempty(dirs)' failed at src/shared/exec-util.c:211, function execute_directories(). Aborting. [ 15.008480] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. [ 15.111989] systemd[1]: Freezing execution. --8<-- and Alpine with: --8<-- [ 0.036703] Kernel panic - not syncing: kmem_cache_create_usercopy: Failed to create slab 'riscv_vector_ctx'. Error -22 [ 0.039195] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.7.0-rc1-defconfig_plain-gdf944704182e #1 [ 0.040744] Hardware name: riscv-virtio,qemu (DT) [ 0.041975] Call Trace: [ 0.042813] [<ffffffff800067a4>] dump_backtrace+0x1c/0x24 [ 0.044832] [<ffffffff80945980>] show_stack+0x2c/0x38 [ 0.045724] [<ffffffff80952214>] dump_stack_lvl+0x3c/0x54 [ 0.046841] [<ffffffff80952240>] dump_stack+0x14/0x1c [ 0.047428] [<ffffffff80945e7c>] panic+0x106/0x29e [ 0.047998] [<ffffffff8015f14c>] kmem_cache_create_usercopy+0x20e/0x258 [ 0.048786] [<ffffffff80a044dc>] riscv_v_setup_ctx_cache+0x2c/0x3c [ 0.049521] [<ffffffff80a03a48>] arch_task_cache_init+0x10/0x18 [ 0.057832] [<ffffffff80a0706c>] fork_init+0x42/0x168 [ 0.058737] [<ffffffff80a00d70>] start_kernel+0x6ba/0x73a --8<-- The Alpine boot can be fixed with something like: --8<-- diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index f9769703fd39..0ac79a9cdba5 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -53,6 +53,9 @@ int riscv_v_setup_vsize(void) void __init riscv_v_setup_ctx_cache(void) { + if (!riscv_v_vsize) + return; + riscv_v_user_cachep = kmem_cache_create_usercopy("riscv_vector_ctx", riscv_v_vsize, 16, SLAB_PANIC, 0, riscv_v_vsize, NULL); --8<-- but with this "fix" in place I still get Ubuntu boot failures. To reproduce the CI locally: | git fetch https://github.com/linux-riscv/linux-riscv e2aad75b340d65b0be4d1a689db3e10c6ed3f18e | git checkout FETCH_HEAD | docker pull ghcr.io/linux-riscv/pw-builder-multi:latest | docker run -it --volume $PWD:/build/my-linux ghcr.io/linux-riscv/pw-builder-multi:latest bash | # In container | bash -l | mkdir -p /build/kernels/logs | .github/scripts/series/prepare_tests.sh | cd /build/my-linux | .github/scripts/series/kernel_builder.sh rv64 defconfig plain gcc | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc ubuntu | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc alpine Logs in /build/tests/run_test* I'll continue to debug in the meantime. Björn
On Fri, Jan 12, 2024 at 11:29 PM Björn Töpel <bjorn@kernel.org> wrote: > > Andy, > > > Hello: > > > > This series was applied to riscv/linux.git (for-next) > > by Palmer Dabbelt <palmer@rivosinc.com>: > > > > I'm getting some boot issues with this series applied to riscv/for-next. > > The full runs (with logs) is here: > https://github.com/linux-riscv/linux-riscv/actions/runs/7498706326 > > Typically it fails in two ways: > Ubuntu rootfs: > --8<-- > [ 4.346414] (sd-gens)[68]: Failed to extract file name from '': Invalid argument > [ 4.390832] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error > [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. > [ 4.440164] systemd[1]: Freezing execution. > --8<-- > > or: > --8<-- > [ 14.909912] (sd-gens)[71]: Assertion '!strv_isempty(dirs)' failed at src/shared/exec-util.c:211, function execute_directories(). Aborting. > [ 15.008480] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error > [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. > [ 15.111989] systemd[1]: Freezing execution. > --8<-- > > and Alpine with: > --8<-- > [ 0.036703] Kernel panic - not syncing: kmem_cache_create_usercopy: Failed to create slab 'riscv_vector_ctx'. Error -22 > [ 0.039195] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.7.0-rc1-defconfig_plain-gdf944704182e #1 > [ 0.040744] Hardware name: riscv-virtio,qemu (DT) > [ 0.041975] Call Trace: > [ 0.042813] [<ffffffff800067a4>] dump_backtrace+0x1c/0x24 > [ 0.044832] [<ffffffff80945980>] show_stack+0x2c/0x38 > [ 0.045724] [<ffffffff80952214>] dump_stack_lvl+0x3c/0x54 > [ 0.046841] [<ffffffff80952240>] dump_stack+0x14/0x1c > [ 0.047428] [<ffffffff80945e7c>] panic+0x106/0x29e > [ 0.047998] [<ffffffff8015f14c>] kmem_cache_create_usercopy+0x20e/0x258 > [ 0.048786] [<ffffffff80a044dc>] riscv_v_setup_ctx_cache+0x2c/0x3c > [ 0.049521] [<ffffffff80a03a48>] arch_task_cache_init+0x10/0x18 > [ 0.057832] [<ffffffff80a0706c>] fork_init+0x42/0x168 > [ 0.058737] [<ffffffff80a00d70>] start_kernel+0x6ba/0x73a > --8<-- > > The Alpine boot can be fixed with something like: > --8<-- > diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c > index f9769703fd39..0ac79a9cdba5 100644 > --- a/arch/riscv/kernel/vector.c > +++ b/arch/riscv/kernel/vector.c > @@ -53,6 +53,9 @@ int riscv_v_setup_vsize(void) > > void __init riscv_v_setup_ctx_cache(void) > { > + if (!riscv_v_vsize) > + return; > + > riscv_v_user_cachep = kmem_cache_create_usercopy("riscv_vector_ctx", > riscv_v_vsize, 16, SLAB_PANIC, > 0, riscv_v_vsize, NULL); > --8<-- Sorry for that! I forgot to do a has_vector() check before creating the cache. I am going to send a patch to fix it. > > but with this "fix" in place I still get Ubuntu boot failures. To > reproduce the CI locally: > > | git fetch https://github.com/linux-riscv/linux-riscv e2aad75b340d65b0be4d1a689db3e10c6ed3f18e > | git checkout FETCH_HEAD > | docker pull ghcr.io/linux-riscv/pw-builder-multi:latest > | docker run -it --volume $PWD:/build/my-linux ghcr.io/linux-riscv/pw-builder-multi:latest bash > | # In container > | bash -l > | mkdir -p /build/kernels/logs > | .github/scripts/series/prepare_tests.sh > | cd /build/my-linux > | .github/scripts/series/kernel_builder.sh rv64 defconfig plain gcc > | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc ubuntu > | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc alpine It's weird that these errors do not show up in my test environment. I will try to reproduce it with the script above. > > Logs in /build/tests/run_test* > > I'll continue to debug in the meantime. > > > Björn Thanks, Andy
On Fri, 12 Jan 2024 08:03:24 PST (-0800), andy.chiu@sifive.com wrote: > On Fri, Jan 12, 2024 at 11:29 PM Björn Töpel <bjorn@kernel.org> wrote: >> >> Andy, >> >> > Hello: >> > >> > This series was applied to riscv/linux.git (for-next) >> > by Palmer Dabbelt <palmer@rivosinc.com>: >> > >> >> I'm getting some boot issues with this series applied to riscv/for-next. >> >> The full runs (with logs) is here: >> https://github.com/linux-riscv/linux-riscv/actions/runs/7498706326 >> >> Typically it fails in two ways: >> Ubuntu rootfs: >> --8<-- >> [ 4.346414] (sd-gens)[68]: Failed to extract file name from '': Invalid argument >> [ 4.390832] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error >> [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. >> [ 4.440164] systemd[1]: Freezing execution. >> --8<-- >> >> or: >> --8<-- >> [ 14.909912] (sd-gens)[71]: Assertion '!strv_isempty(dirs)' failed at src/shared/exec-util.c:211, function execute_directories(). Aborting. >> [ 15.008480] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error >> [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. >> [ 15.111989] systemd[1]: Freezing execution. >> --8<-- >> >> and Alpine with: >> --8<-- >> [ 0.036703] Kernel panic - not syncing: kmem_cache_create_usercopy: Failed to create slab 'riscv_vector_ctx'. Error -22 >> [ 0.039195] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.7.0-rc1-defconfig_plain-gdf944704182e #1 >> [ 0.040744] Hardware name: riscv-virtio,qemu (DT) >> [ 0.041975] Call Trace: >> [ 0.042813] [<ffffffff800067a4>] dump_backtrace+0x1c/0x24 >> [ 0.044832] [<ffffffff80945980>] show_stack+0x2c/0x38 >> [ 0.045724] [<ffffffff80952214>] dump_stack_lvl+0x3c/0x54 >> [ 0.046841] [<ffffffff80952240>] dump_stack+0x14/0x1c >> [ 0.047428] [<ffffffff80945e7c>] panic+0x106/0x29e >> [ 0.047998] [<ffffffff8015f14c>] kmem_cache_create_usercopy+0x20e/0x258 >> [ 0.048786] [<ffffffff80a044dc>] riscv_v_setup_ctx_cache+0x2c/0x3c >> [ 0.049521] [<ffffffff80a03a48>] arch_task_cache_init+0x10/0x18 >> [ 0.057832] [<ffffffff80a0706c>] fork_init+0x42/0x168 >> [ 0.058737] [<ffffffff80a00d70>] start_kernel+0x6ba/0x73a >> --8<-- >> >> The Alpine boot can be fixed with something like: >> --8<-- >> diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c >> index f9769703fd39..0ac79a9cdba5 100644 >> --- a/arch/riscv/kernel/vector.c >> +++ b/arch/riscv/kernel/vector.c >> @@ -53,6 +53,9 @@ int riscv_v_setup_vsize(void) >> >> void __init riscv_v_setup_ctx_cache(void) >> { >> + if (!riscv_v_vsize) >> + return; >> + >> riscv_v_user_cachep = kmem_cache_create_usercopy("riscv_vector_ctx", >> riscv_v_vsize, 16, SLAB_PANIC, >> 0, riscv_v_vsize, NULL); >> --8<-- > > Sorry for that! I forgot to do a has_vector() check before creating > the cache. I am going to send a patch to fix it. They don't fail for me either, which is how they ended up on for-next. So sorry I missed it. Do you have a fix? Otherwise I can just drop these from for-next and we can do a v11 -- it's the tip of for-next, so still pretty easy ta back out. > >> >> but with this "fix" in place I still get Ubuntu boot failures. To >> reproduce the CI locally: >> >> | git fetch https://github.com/linux-riscv/linux-riscv e2aad75b340d65b0be4d1a689db3e10c6ed3f18e >> | git checkout FETCH_HEAD >> | docker pull ghcr.io/linux-riscv/pw-builder-multi:latest >> | docker run -it --volume $PWD:/build/my-linux ghcr.io/linux-riscv/pw-builder-multi:latest bash >> | # In container >> | bash -l >> | mkdir -p /build/kernels/logs >> | .github/scripts/series/prepare_tests.sh >> | cd /build/my-linux >> | .github/scripts/series/kernel_builder.sh rv64 defconfig plain gcc >> | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc ubuntu >> | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc alpine > > It's weird that these errors do not show up in my test environment. I > will try to reproduce it with the script above. > >> >> Logs in /build/tests/run_test* >> >> I'll continue to debug in the meantime. >> >> >> Björn > > Thanks, > Andy > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv
Hi Björn, On Sat, Jan 13, 2024 at 12:03 AM Andy Chiu <andy.chiu@sifive.com> wrote: > > On Fri, Jan 12, 2024 at 11:29 PM Björn Töpel <bjorn@kernel.org> wrote: > > > > Andy, > > > > > Hello: > > > > > > This series was applied to riscv/linux.git (for-next) > > > by Palmer Dabbelt <palmer@rivosinc.com>: > > > > > > > I'm getting some boot issues with this series applied to riscv/for-next. > > > > The full runs (with logs) is here: > > https://github.com/linux-riscv/linux-riscv/actions/runs/7498706326 > > > > Typically it fails in two ways: > > Ubuntu rootfs: > > --8<-- > > [ 4.346414] (sd-gens)[68]: Failed to extract file name from '': Invalid argument > > [ 4.390832] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error > > [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. > > [ 4.440164] systemd[1]: Freezing execution. > > --8<-- > > > > or: > > --8<-- > > [ 14.909912] (sd-gens)[71]: Assertion '!strv_isempty(dirs)' failed at src/shared/exec-util.c:211, function execute_directories(). Aborting. > > [ 15.008480] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error > > [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. > > [ 15.111989] systemd[1]: Freezing execution. > > --8<-- > > > > and Alpine with: > > --8<-- > > [ 0.036703] Kernel panic - not syncing: kmem_cache_create_usercopy: Failed to create slab 'riscv_vector_ctx'. Error -22 > > [ 0.039195] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.7.0-rc1-defconfig_plain-gdf944704182e #1 > > [ 0.040744] Hardware name: riscv-virtio,qemu (DT) > > [ 0.041975] Call Trace: > > [ 0.042813] [<ffffffff800067a4>] dump_backtrace+0x1c/0x24 > > [ 0.044832] [<ffffffff80945980>] show_stack+0x2c/0x38 > > [ 0.045724] [<ffffffff80952214>] dump_stack_lvl+0x3c/0x54 > > [ 0.046841] [<ffffffff80952240>] dump_stack+0x14/0x1c > > [ 0.047428] [<ffffffff80945e7c>] panic+0x106/0x29e > > [ 0.047998] [<ffffffff8015f14c>] kmem_cache_create_usercopy+0x20e/0x258 > > [ 0.048786] [<ffffffff80a044dc>] riscv_v_setup_ctx_cache+0x2c/0x3c > > [ 0.049521] [<ffffffff80a03a48>] arch_task_cache_init+0x10/0x18 > > [ 0.057832] [<ffffffff80a0706c>] fork_init+0x42/0x168 > > [ 0.058737] [<ffffffff80a00d70>] start_kernel+0x6ba/0x73a > > --8<-- > > > > The Alpine boot can be fixed with something like: > > --8<-- > > diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c > > index f9769703fd39..0ac79a9cdba5 100644 > > --- a/arch/riscv/kernel/vector.c > > +++ b/arch/riscv/kernel/vector.c > > @@ -53,6 +53,9 @@ int riscv_v_setup_vsize(void) > > > > void __init riscv_v_setup_ctx_cache(void) > > { > > + if (!riscv_v_vsize) > > + return; > > + > > riscv_v_user_cachep = kmem_cache_create_usercopy("riscv_vector_ctx", > > riscv_v_vsize, 16, SLAB_PANIC, > > 0, riscv_v_vsize, NULL); > > --8<-- > > Sorry for that! I forgot to do a has_vector() check before creating > the cache. I am going to send a patch to fix it. > > > > > but with this "fix" in place I still get Ubuntu boot failures. To > > reproduce the CI locally: > > > > | git fetch https://github.com/linux-riscv/linux-riscv e2aad75b340d65b0be4d1a689db3e10c6ed3f18e > > | git checkout FETCH_HEAD > > | docker pull ghcr.io/linux-riscv/pw-builder-multi:latest > > | docker run -it --volume $PWD:/build/my-linux ghcr.io/linux-riscv/pw-builder-multi:latest bash > > | # In container > > | bash -l > > | mkdir -p /build/kernels/logs > > | .github/scripts/series/prepare_tests.sh > > | cd /build/my-linux > > | .github/scripts/series/kernel_builder.sh rv64 defconfig plain gcc > > | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc ubuntu > > | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc alpine > > It's weird that these errors do not show up in my test environment. I > will try to reproduce it with the script above. I just located the boot fail with some experiments. It is related to the fallback logic in enter_vector_usercopy(). It seems like booting is successful if we restarted scalar fallback with its original copy size. It is not affecting preempt_v because preempt_v will never goes into this branch. It's late for me. I will figure out the reason and hopefully fix it at the root cause in the weekend. --- a/arch/riscv/lib/riscv_v_helpers.c +++ b/arch/riscv/lib/riscv_v_helpers.c @@ -30,9 +30,6 @@ asmlinkage int enter_vector_usercopy(void *dst, void *src, size_t n) kernel_vector_end(); if (remain) { - copied = n - remain; - dst += copied; - src += copied; goto fallback; } > > > > > Logs in /build/tests/run_test* > > > > I'll continue to debug in the meantime. > > > > > > Björn > > Thanks, > Andy Thanks, Andy
Hi Palmer, On Sat, Jan 13, 2024 at 2:35 AM Palmer Dabbelt <palmer@dabbelt.com> wrote: > > On Fri, 12 Jan 2024 08:03:24 PST (-0800), andy.chiu@sifive.com wrote: > > On Fri, Jan 12, 2024 at 11:29 PM Björn Töpel <bjorn@kernel.org> wrote: > >> > >> Andy, > >> > >> > Hello: > >> > > >> > This series was applied to riscv/linux.git (for-next) > >> > by Palmer Dabbelt <palmer@rivosinc.com>: > >> > > >> > >> I'm getting some boot issues with this series applied to riscv/for-next. > >> > >> The full runs (with logs) is here: > >> https://github.com/linux-riscv/linux-riscv/actions/runs/7498706326 > >> > >> Typically it fails in two ways: > >> Ubuntu rootfs: > >> --8<-- > >> [ 4.346414] (sd-gens)[68]: Failed to extract file name from '': Invalid argument > >> [ 4.390832] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error > >> [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. > >> [ 4.440164] systemd[1]: Freezing execution. > >> --8<-- > >> > >> or: > >> --8<-- > >> [ 14.909912] (sd-gens)[71]: Assertion '!strv_isempty(dirs)' failed at src/shared/exec-util.c:211, function execute_directories(). Aborting. > >> [ 15.008480] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error > >> [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. > >> [ 15.111989] systemd[1]: Freezing execution. > >> --8<-- > >> > >> and Alpine with: > >> --8<-- > >> [ 0.036703] Kernel panic - not syncing: kmem_cache_create_usercopy: Failed to create slab 'riscv_vector_ctx'. Error -22 > >> [ 0.039195] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.7.0-rc1-defconfig_plain-gdf944704182e #1 > >> [ 0.040744] Hardware name: riscv-virtio,qemu (DT) > >> [ 0.041975] Call Trace: > >> [ 0.042813] [<ffffffff800067a4>] dump_backtrace+0x1c/0x24 > >> [ 0.044832] [<ffffffff80945980>] show_stack+0x2c/0x38 > >> [ 0.045724] [<ffffffff80952214>] dump_stack_lvl+0x3c/0x54 > >> [ 0.046841] [<ffffffff80952240>] dump_stack+0x14/0x1c > >> [ 0.047428] [<ffffffff80945e7c>] panic+0x106/0x29e > >> [ 0.047998] [<ffffffff8015f14c>] kmem_cache_create_usercopy+0x20e/0x258 > >> [ 0.048786] [<ffffffff80a044dc>] riscv_v_setup_ctx_cache+0x2c/0x3c > >> [ 0.049521] [<ffffffff80a03a48>] arch_task_cache_init+0x10/0x18 > >> [ 0.057832] [<ffffffff80a0706c>] fork_init+0x42/0x168 > >> [ 0.058737] [<ffffffff80a00d70>] start_kernel+0x6ba/0x73a > >> --8<-- > >> > >> The Alpine boot can be fixed with something like: > >> --8<-- > >> diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c > >> index f9769703fd39..0ac79a9cdba5 100644 > >> --- a/arch/riscv/kernel/vector.c > >> +++ b/arch/riscv/kernel/vector.c > >> @@ -53,6 +53,9 @@ int riscv_v_setup_vsize(void) > >> > >> void __init riscv_v_setup_ctx_cache(void) > >> { > >> + if (!riscv_v_vsize) > >> + return; > >> + > >> riscv_v_user_cachep = kmem_cache_create_usercopy("riscv_vector_ctx", > >> riscv_v_vsize, 16, SLAB_PANIC, > >> 0, riscv_v_vsize, NULL); > >> --8<-- > > > > Sorry for that! I forgot to do a has_vector() check before creating > > the cache. I am going to send a patch to fix it. > > They don't fail for me either, which is how they ended up on for-next. > So sorry I missed it. > > Do you have a fix? Otherwise I can just drop these from for-next and we > can do a v11 -- it's the tip of for-next, so still pretty easy ta back > out. I don't have the full fix for now, so I think we should drop to prevent blocking others' workflow. > > > > >> > >> but with this "fix" in place I still get Ubuntu boot failures. To > >> reproduce the CI locally: > >> > >> | git fetch https://github.com/linux-riscv/linux-riscv e2aad75b340d65b0be4d1a689db3e10c6ed3f18e > >> | git checkout FETCH_HEAD > >> | docker pull ghcr.io/linux-riscv/pw-builder-multi:latest > >> | docker run -it --volume $PWD:/build/my-linux ghcr.io/linux-riscv/pw-builder-multi:latest bash > >> | # In container > >> | bash -l > >> | mkdir -p /build/kernels/logs > >> | .github/scripts/series/prepare_tests.sh > >> | cd /build/my-linux > >> | .github/scripts/series/kernel_builder.sh rv64 defconfig plain gcc > >> | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc ubuntu > >> | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc alpine > > > > It's weird that these errors do not show up in my test environment. I > > will try to reproduce it with the script above. > > > >> > >> Logs in /build/tests/run_test* > >> > >> I'll continue to debug in the meantime. > >> > >> > >> Björn > > > > Thanks, > > Andy > > > > _______________________________________________ > > linux-riscv mailing list > > linux-riscv@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/linux-riscv Sorry for that! Thanks, Andy
On Fri, 12 Jan 2024 10:47:19 PST (-0800), andy.chiu@sifive.com wrote: > Hi Palmer, > > On Sat, Jan 13, 2024 at 2:35 AM Palmer Dabbelt <palmer@dabbelt.com> wrote: >> >> On Fri, 12 Jan 2024 08:03:24 PST (-0800), andy.chiu@sifive.com wrote: >> > On Fri, Jan 12, 2024 at 11:29 PM Björn Töpel <bjorn@kernel.org> wrote: >> >> >> >> Andy, >> >> >> >> > Hello: >> >> > >> >> > This series was applied to riscv/linux.git (for-next) >> >> > by Palmer Dabbelt <palmer@rivosinc.com>: >> >> > >> >> >> >> I'm getting some boot issues with this series applied to riscv/for-next. >> >> >> >> The full runs (with logs) is here: >> >> https://github.com/linux-riscv/linux-riscv/actions/runs/7498706326 >> >> >> >> Typically it fails in two ways: >> >> Ubuntu rootfs: >> >> --8<-- >> >> [ 4.346414] (sd-gens)[68]: Failed to extract file name from '': Invalid argument >> >> [ 4.390832] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error >> >> [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. >> >> [ 4.440164] systemd[1]: Freezing execution. >> >> --8<-- >> >> >> >> or: >> >> --8<-- >> >> [ 14.909912] (sd-gens)[71]: Assertion '!strv_isempty(dirs)' failed at src/shared/exec-util.c:211, function execute_directories(). Aborting. >> >> [ 15.008480] systemd[1]: Failed to fork off sandboxing environment for executing generators: Protocol error >> >> [ESC[0;1;31m!!!!!!ESC[0m] Failed to start up manager. >> >> [ 15.111989] systemd[1]: Freezing execution. >> >> --8<-- >> >> >> >> and Alpine with: >> >> --8<-- >> >> [ 0.036703] Kernel panic - not syncing: kmem_cache_create_usercopy: Failed to create slab 'riscv_vector_ctx'. Error -22 >> >> [ 0.039195] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.7.0-rc1-defconfig_plain-gdf944704182e #1 >> >> [ 0.040744] Hardware name: riscv-virtio,qemu (DT) >> >> [ 0.041975] Call Trace: >> >> [ 0.042813] [<ffffffff800067a4>] dump_backtrace+0x1c/0x24 >> >> [ 0.044832] [<ffffffff80945980>] show_stack+0x2c/0x38 >> >> [ 0.045724] [<ffffffff80952214>] dump_stack_lvl+0x3c/0x54 >> >> [ 0.046841] [<ffffffff80952240>] dump_stack+0x14/0x1c >> >> [ 0.047428] [<ffffffff80945e7c>] panic+0x106/0x29e >> >> [ 0.047998] [<ffffffff8015f14c>] kmem_cache_create_usercopy+0x20e/0x258 >> >> [ 0.048786] [<ffffffff80a044dc>] riscv_v_setup_ctx_cache+0x2c/0x3c >> >> [ 0.049521] [<ffffffff80a03a48>] arch_task_cache_init+0x10/0x18 >> >> [ 0.057832] [<ffffffff80a0706c>] fork_init+0x42/0x168 >> >> [ 0.058737] [<ffffffff80a00d70>] start_kernel+0x6ba/0x73a >> >> --8<-- >> >> >> >> The Alpine boot can be fixed with something like: >> >> --8<-- >> >> diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c >> >> index f9769703fd39..0ac79a9cdba5 100644 >> >> --- a/arch/riscv/kernel/vector.c >> >> +++ b/arch/riscv/kernel/vector.c >> >> @@ -53,6 +53,9 @@ int riscv_v_setup_vsize(void) >> >> >> >> void __init riscv_v_setup_ctx_cache(void) >> >> { >> >> + if (!riscv_v_vsize) >> >> + return; >> >> + >> >> riscv_v_user_cachep = kmem_cache_create_usercopy("riscv_vector_ctx", >> >> riscv_v_vsize, 16, SLAB_PANIC, >> >> 0, riscv_v_vsize, NULL); >> >> --8<-- >> > >> > Sorry for that! I forgot to do a has_vector() check before creating >> > the cache. I am going to send a patch to fix it. >> >> They don't fail for me either, which is how they ended up on for-next. >> So sorry I missed it. >> >> Do you have a fix? Otherwise I can just drop these from for-next and we >> can do a v11 -- it's the tip of for-next, so still pretty easy ta back >> out. > > I don't have the full fix for now, so I think we should drop to > prevent blocking others' workflow. OK, I dropped it. > >> >> > >> >> >> >> but with this "fix" in place I still get Ubuntu boot failures. To >> >> reproduce the CI locally: >> >> >> >> | git fetch https://github.com/linux-riscv/linux-riscv e2aad75b340d65b0be4d1a689db3e10c6ed3f18e >> >> | git checkout FETCH_HEAD >> >> | docker pull ghcr.io/linux-riscv/pw-builder-multi:latest >> >> | docker run -it --volume $PWD:/build/my-linux ghcr.io/linux-riscv/pw-builder-multi:latest bash >> >> | # In container >> >> | bash -l >> >> | mkdir -p /build/kernels/logs >> >> | .github/scripts/series/prepare_tests.sh >> >> | cd /build/my-linux >> >> | .github/scripts/series/kernel_builder.sh rv64 defconfig plain gcc >> >> | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc ubuntu >> >> | .github/scripts/series/test_runner.sh rv64 defconfig plain gcc alpine >> > >> > It's weird that these errors do not show up in my test environment. I >> > will try to reproduce it with the script above. >> > >> >> >> >> Logs in /build/tests/run_test* >> >> >> >> I'll continue to debug in the meantime. >> >> >> >> >> >> Björn >> > >> > Thanks, >> > Andy >> > >> > _______________________________________________ >> > linux-riscv mailing list >> > linux-riscv@lists.infradead.org >> > http://lists.infradead.org/mailman/listinfo/linux-riscv > > Sorry for that! > > Thanks, > Andy
On Fri, Jan 12, 2024 at 6:32 AM <patchwork-bot+linux-riscv@kernel.org> wrote: > > Hello: > > This series was applied to riscv/linux.git (for-next) > by Palmer Dabbelt <palmer@rivosinc.com>: > > On Thu, 11 Jan 2024 13:15:48 +0000 you wrote: > > This series provides support running Vector in kernel mode. > > Additionally, kernel-mode Vector can be configured to run without > > turnning off preemption on a CONFIG_PREEMPT kernel. Along with the > > suport, we add Vector optimized copy_{to,from}_user. And provide a > > simple threshold to decide when to run the vectorized functions. > > > > We decided to drop vectorized memcpy/memset/memmove for the moment due > > to the concern of memory side-effect in kernel_vector_begin(). The > > detailed description can be found at v9[1] > > > > [...] > > Here is the summary with links: > - [v10,01/10] riscv: Add support for kernel mode vector > https://git.kernel.org/riscv/c/c0ae350f714f > - [v10,02/10] riscv: vector: make Vector always available for softirq context > https://git.kernel.org/riscv/c/ebf52ac30e4f > - [v10,03/10] riscv: Add vector extension XOR implementation > https://git.kernel.org/riscv/c/9ff97211a623 > - [v10,04/10] riscv: sched: defer restoring Vector context for user > https://git.kernel.org/riscv/c/f4471252f3b9 > - [v10,05/10] riscv: lib: vectorize copy_to_user/copy_from_user > https://git.kernel.org/riscv/c/145ca6eddd70 > - [v10,06/10] riscv: fpu: drop SR_SD bit checking > https://git.kernel.org/riscv/c/25a830944773 > - [v10,07/10] riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() > https://git.kernel.org/riscv/c/9dece8bf0343 > - [v10,08/10] riscv: vector: use a mask to write vstate_ctrl > https://git.kernel.org/riscv/c/c05992747c96 > - [v10,09/10] riscv: vector: use kmem_cache to manage vector context > https://git.kernel.org/riscv/c/660217429614 > - [v10,10/10] riscv: vector: allow kernel-mode Vector with preemption > https://git.kernel.org/riscv/c/aa23c6172d33 > With this series merged in RZ/Five stops booting [0], I dont get any panic as such but it's a kernel freeze. Reverting this series all boots up OK. [0] https://paste.debian.net/hidden/a8293240/ Cheers, Prabhakar > You are awesome, thank you! > -- > Deet-doot-dot, I am a bot. > https://korg.docs.kernel.org/patchwork/pwbot.html > > > > _______________________________________________ > linux-riscv mailing list > linux-riscv@lists.infradead.org > http://lists.infradead.org/mailman/listinfo/linux-riscv
Hi Prabhakar, On Sat, Jan 13, 2024 at 4:05 AM Lad, Prabhakar <prabhakar.csengg@gmail.com> wrote: > > On Fri, Jan 12, 2024 at 6:32 AM <patchwork-bot+linux-riscv@kernel.org> wrote: > > > > Hello: > > > > This series was applied to riscv/linux.git (for-next) > > by Palmer Dabbelt <palmer@rivosinc.com>: > > > > On Thu, 11 Jan 2024 13:15:48 +0000 you wrote: > > > This series provides support running Vector in kernel mode. > > > Additionally, kernel-mode Vector can be configured to run without > > > turnning off preemption on a CONFIG_PREEMPT kernel. Along with the > > > suport, we add Vector optimized copy_{to,from}_user. And provide a > > > simple threshold to decide when to run the vectorized functions. > > > > > > We decided to drop vectorized memcpy/memset/memmove for the moment due > > > to the concern of memory side-effect in kernel_vector_begin(). The > > > detailed description can be found at v9[1] > > > > > > [...] > > > > Here is the summary with links: > > - [v10,01/10] riscv: Add support for kernel mode vector > > https://git.kernel.org/riscv/c/c0ae350f714f > > - [v10,02/10] riscv: vector: make Vector always available for softirq context > > https://git.kernel.org/riscv/c/ebf52ac30e4f > > - [v10,03/10] riscv: Add vector extension XOR implementation > > https://git.kernel.org/riscv/c/9ff97211a623 > > - [v10,04/10] riscv: sched: defer restoring Vector context for user > > https://git.kernel.org/riscv/c/f4471252f3b9 > > - [v10,05/10] riscv: lib: vectorize copy_to_user/copy_from_user > > https://git.kernel.org/riscv/c/145ca6eddd70 > > - [v10,06/10] riscv: fpu: drop SR_SD bit checking > > https://git.kernel.org/riscv/c/25a830944773 > > - [v10,07/10] riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() > > https://git.kernel.org/riscv/c/9dece8bf0343 > > - [v10,08/10] riscv: vector: use a mask to write vstate_ctrl > > https://git.kernel.org/riscv/c/c05992747c96 > > - [v10,09/10] riscv: vector: use kmem_cache to manage vector context > > https://git.kernel.org/riscv/c/660217429614 > > - [v10,10/10] riscv: vector: allow kernel-mode Vector with preemption > > https://git.kernel.org/riscv/c/aa23c6172d33 > > > With this series merged in RZ/Five stops booting [0], I dont get any > panic as such but it's a kernel freeze. Reverting this series all > boots up OK. > > [0] https://paste.debian.net/hidden/a8293240/ Thanks for the note. Unfortunately I didn't find a direct clue of failure from your log. But I believe this is the same case as the alpine one above, because they would fail very early. I have updated a branch here[0]. It should be fixed now. Also, it should fix the boot failure that appears on Ubuntu. It would be nice if you could have a try. I will send it out as v11 if no further issue is detected before the week ends. [0] https://github.com/sifive/riscv-linux/tree/2b25df2ad3a3651bc1f77fec65c95a1ae656d675 > > Cheers, > Prabhakar > > > You are awesome, thank you! > > -- > > Deet-doot-dot, I am a bot. > > https://korg.docs.kernel.org/patchwork/pwbot.html > > > > > > > > _______________________________________________ > > linux-riscv mailing list > > linux-riscv@lists.infradead.org > > http://lists.infradead.org/mailman/listinfo/linux-riscv Thanks, Andy
Hi Andy, On Sat, Jan 13, 2024 at 1:26 PM Andy Chiu <andy.chiu@sifive.com> wrote: > > Hi Prabhakar, > > On Sat, Jan 13, 2024 at 4:05 AM Lad, Prabhakar > <prabhakar.csengg@gmail.com> wrote: > > > > On Fri, Jan 12, 2024 at 6:32 AM <patchwork-bot+linux-riscv@kernel.org> wrote: > > > > > > Hello: > > > > > > This series was applied to riscv/linux.git (for-next) > > > by Palmer Dabbelt <palmer@rivosinc.com>: > > > > > > On Thu, 11 Jan 2024 13:15:48 +0000 you wrote: > > > > This series provides support running Vector in kernel mode. > > > > Additionally, kernel-mode Vector can be configured to run without > > > > turnning off preemption on a CONFIG_PREEMPT kernel. Along with the > > > > suport, we add Vector optimized copy_{to,from}_user. And provide a > > > > simple threshold to decide when to run the vectorized functions. > > > > > > > > We decided to drop vectorized memcpy/memset/memmove for the moment due > > > > to the concern of memory side-effect in kernel_vector_begin(). The > > > > detailed description can be found at v9[1] > > > > > > > > [...] > > > > > > Here is the summary with links: > > > - [v10,01/10] riscv: Add support for kernel mode vector > > > https://git.kernel.org/riscv/c/c0ae350f714f > > > - [v10,02/10] riscv: vector: make Vector always available for softirq context > > > https://git.kernel.org/riscv/c/ebf52ac30e4f > > > - [v10,03/10] riscv: Add vector extension XOR implementation > > > https://git.kernel.org/riscv/c/9ff97211a623 > > > - [v10,04/10] riscv: sched: defer restoring Vector context for user > > > https://git.kernel.org/riscv/c/f4471252f3b9 > > > - [v10,05/10] riscv: lib: vectorize copy_to_user/copy_from_user > > > https://git.kernel.org/riscv/c/145ca6eddd70 > > > - [v10,06/10] riscv: fpu: drop SR_SD bit checking > > > https://git.kernel.org/riscv/c/25a830944773 > > > - [v10,07/10] riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() > > > https://git.kernel.org/riscv/c/9dece8bf0343 > > > - [v10,08/10] riscv: vector: use a mask to write vstate_ctrl > > > https://git.kernel.org/riscv/c/c05992747c96 > > > - [v10,09/10] riscv: vector: use kmem_cache to manage vector context > > > https://git.kernel.org/riscv/c/660217429614 > > > - [v10,10/10] riscv: vector: allow kernel-mode Vector with preemption > > > https://git.kernel.org/riscv/c/aa23c6172d33 > > > > > With this series merged in RZ/Five stops booting [0], I dont get any > > panic as such but it's a kernel freeze. Reverting this series all > > boots up OK. > > > > [0] https://paste.debian.net/hidden/a8293240/ > > Thanks for the note. > > Unfortunately I didn't find a direct clue of failure from your log. > But I believe this is the same case as the alpine one above, because > they would fail very early. I have updated a branch here[0]. It should > be fixed now. Also, it should fix the boot failure that appears on > Ubuntu. It would be nice if you could have a try. I will send it out > as v11 if no further issue is detected before the week ends. > > [0] https://github.com/sifive/riscv-linux/tree/2b25df2ad3a3651bc1f77fec65c95a1ae656d675 > I can confirm with the patches applied from above link, the RZ/Five SMARC platform can boot as normal. Cheers, Prabhakar > > > > Cheers, > > Prabhakar > > > > > You are awesome, thank you! > > > -- > > > Deet-doot-dot, I am a bot. > > > https://korg.docs.kernel.org/patchwork/pwbot.html > > > > > > > > > > > > _______________________________________________ > > > linux-riscv mailing list > > > linux-riscv@lists.infradead.org > > > http://lists.infradead.org/mailman/listinfo/linux-riscv > > Thanks, > Andy
This series provides support running Vector in kernel mode. Additionally, kernel-mode Vector can be configured to run without turnning off preemption on a CONFIG_PREEMPT kernel. Along with the suport, we add Vector optimized copy_{to,from}_user. And provide a simple threshold to decide when to run the vectorized functions. We decided to drop vectorized memcpy/memset/memmove for the moment due to the concern of memory side-effect in kernel_vector_begin(). The detailed description can be found at v9[1] This series is composed by 4 parts: patch 1-4: adds basic support for kernel-mode Vector patch 5: includes vectorized copy_{to,from}_user into the kernel patch 6: refactor context switch code in fpu [2] patch 7-10: provides some code refactors and support for preemptible kernel-mode Vector. This series can be merged if we feel any part of {1~4, 5, 6, 7~10} is mature enough. This patch is tested on a QEMU with V and verified that booting, normal userspace operations all work as usual with thresholds set to 0. Also, we test by launching multiple kernel threads which continuously executes and verifies Vector operations in the background. The module that tests these operation is expected to be upstream later. v9 of this series can be found at [1] [1]:https://lore.kernel.org/all/20231229143627.22898-1-andy.chiu@sifive.com/ Patch summary: - Updated patches: 1, 5, 10 - New patch: (none) - Unchanged patch: 2, 3, 4, 6, 7, 8, 9 - Deleted patch: (none) Changelog v10: - Refactor comments (1), Eric - Remove duplicate assembly code (5), Charlie - Optimize unnecessary compiler barriers in preempt_v (10) - Address bugs in context-saving for preempt_v (10) - Correct dirty marking/clearing code for preempt_v (10) Changelog v9: - Use one bit to record the on/off status of kernel-mode Vector - Temporarily drop vectorized mem* functions - Add a patch to refactor context switch in fpu - silence lockdep and use WARN_ON instead Changelog v8: - Address build fail on no-mmu config - Fix build fail with W=1 - Refactor patches (1, 2), Eric Changelog v7: - Fix build fail for allmodconfig and test building the series with allmodconfig/allyesconfig Changelog v6: - Provide a more robust check on the use of non-preemptible Vector. - Add Kconfigs to set threshold value at compile time. (Charlie) - Add a patch to utilize kmem_cache_* for V context allocations. - Re-write and add preemptible Vector. Changelog v5: - Rebase on top of riscv for-next (6.7-rc1) Changelog v4: - Use kernel_v_flags and helpers to track vector context. - Prevent softirq from nesting V context for non-preempt V - Add user copy and mem* routines Changelog v3: - Rebase on top of riscv for-next (6.6-rc1) - Fix a build issue (Conor) - Guard vstate_save, vstate_restore with {get,put}_cpu_vector_context. - Save V context after disabling preemption. (Guo) - Remove irqs_disabled() check from may_use_simd(). (Björn) - Comment about nesting V context. Changelog v2: - fix build issues - Follow arm's way of starting kernel-mode simd code: - add include/asm/simd.h and rename may_use_vector() -> may_use_simd() - return void in kernel_vector_begin(), and BUG_ON if may_use_simd() fails - Change naming scheme for functions/macros (Conor): - remove KMV - 's/rvv/vector/' - 's/RISCV_ISA_V_PREEMPTIVE_KMV/RISCV_ISA_V_PREEMPTIVE/' - 's/TIF_RISCV_V_KMV/TIF_RISCV_V_KERNEL_MODE/' Andy Chiu (8): riscv: vector: make Vector always available for softirq context riscv: sched: defer restoring Vector context for user riscv: lib: vectorize copy_to_user/copy_from_user riscv: fpu: drop SR_SD bit checking riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() riscv: vector: use a mask to write vstate_ctrl riscv: vector: use kmem_cache to manage vector context riscv: vector: allow kernel-mode Vector with preemption Greentime Hu (2): riscv: Add support for kernel mode vector riscv: Add vector extension XOR implementation arch/riscv/Kconfig | 22 +++ arch/riscv/include/asm/asm-prototypes.h | 27 +++ arch/riscv/include/asm/entry-common.h | 17 ++ arch/riscv/include/asm/processor.h | 41 +++- arch/riscv/include/asm/simd.h | 64 ++++++ arch/riscv/include/asm/switch_to.h | 3 +- arch/riscv/include/asm/thread_info.h | 2 + arch/riscv/include/asm/vector.h | 90 +++++++-- arch/riscv/include/asm/xor.h | 68 +++++++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/entry.S | 8 + arch/riscv/kernel/kernel_mode_vector.c | 247 ++++++++++++++++++++++++ arch/riscv/kernel/process.c | 13 +- arch/riscv/kernel/ptrace.c | 7 +- arch/riscv/kernel/signal.c | 7 +- arch/riscv/kernel/vector.c | 50 ++++- arch/riscv/lib/Makefile | 7 +- arch/riscv/lib/riscv_v_helpers.c | 44 +++++ arch/riscv/lib/uaccess.S | 10 + arch/riscv/lib/uaccess_vector.S | 44 +++++ arch/riscv/lib/xor.S | 81 ++++++++ 21 files changed, 825 insertions(+), 28 deletions(-) create mode 100644 arch/riscv/include/asm/simd.h create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/riscv_v_helpers.c create mode 100644 arch/riscv/lib/uaccess_vector.S create mode 100644 arch/riscv/lib/xor.S