From patchwork Thu Jan 11 13:15:48 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13517375 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8D8B4C47077 for ; Thu, 11 Jan 2024 13:16:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=G8FEFbFONDxHYpDB1h7/5/xT3S834LVm5o0dciAVUII=; b=29SNVSNfXXAUq9 P9aJAX28O5mcHFY0Wt+1BazjQfm8WcCyKhKESr/BMGIolBh08njs/7xlle0Q28s7PiKnFTff37lQD 1pxYEAqDNXILxQ6I2uQb7in9QKaByntCKeOfsi5hGkpGRXeOBvKkm37CkqDeAHUIRvCfF1DEWLgho llHntlhNlWyf6g5L8OgePtVDEfUcP67htqoWLSWcmeI85w5fBoBS+Sx8bfMsIzLSvy2UmWTVAIv+S eX8MkMsHym1qhDie/E/mHJsQx0lq6ANUiWp7HX4D1v+npab+uWAusZVrxwelVq3jpbrDk8FZfDuF/ 7aPdrfnpRC453up1/w9w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rNuuq-0007IE-13; Thu, 11 Jan 2024 13:16:12 +0000 Received: from mail-pg1-x530.google.com ([2607:f8b0:4864:20::530]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rNuun-0007He-0C for linux-riscv@lists.infradead.org; Thu, 11 Jan 2024 13:16:10 +0000 Received: by mail-pg1-x530.google.com with SMTP id 41be03b00d2f7-5bdbe2de25fso4021008a12.3 for ; Thu, 11 Jan 2024 05:16:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1704978967; x=1705583767; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=slaT0MsVFLydqpDXErRLzDs5yBDLHQhej7B6K9n82ak=; b=XbyWA38c9iq9HelgQCmF5pks71nXXV/ijLwLp4pZwWtxsuYMna/dWFsJnA+iMfgpq2 j43aHI+k5GrOKMw6AM70loNPnjIQkiYaaPCrn0u3rckoIe4egTeE5NzUKOvSWU5THOgw miSkb9ddGKSXNdimJqqDKJzuKSy89uCUblpkXjIupQMVHq20lNUWV6Jq2kSmFMya6B8o kXqsJ7eXTvrvc7fJZJb5ysI8xh4+oEvuJdRrbv5M3lkTObC8yMErKGf45sAEExcVWjsw y7wBUdQESZblZ227rX6JDkonTte+OAu9U3WZ9bS6b+FmW1+u6x/yMl9/Oo6EAqkwcGDS J3og== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1704978967; x=1705583767; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=slaT0MsVFLydqpDXErRLzDs5yBDLHQhej7B6K9n82ak=; b=n+s8a4JnUQfcPBnCJFKtTzTUvBdLcpKtvc1QCelRjAxicU+OF1tpNC8mUqNLCxn/Pi P+jJE1my20q+csRM+T+o93iC5u4Kad253pjI3ASZwxpb4vDcYldEFwQCf8p2p/hLQ6F5 Jcw6zOkeKaIbQOr/UlyVCXfLSpLjju4JjJM1wjPK1NbBQlABbX+RTdoxb0Ete60cMu8u vxMBVe4OT8HTI3ZhL7KNn7HftOH6AF2jjNCkE4y7jFmUPWXlGpL29Qjuv8/J9bXZAoff 3VFJcv+Mi5IFp+0Vp3z3qNqUneAH5EEk3b/treU3OhHqo1sHgXVePWih4OMEUHvUO2+v N+Wg== X-Gm-Message-State: AOJu0YzCkc1VV/ItWYPFLzZX8phZvsxqXBhWSUvQQv5qE6EcuByXCuGv Lkq9N4Ful07ONMx2hK5J9Xo3raUcQ51fFWSsf28Y5fSXYgTDEJIxNQl/Yj29IGxEhi4HgactXqW x7c9kysuutmD8xdjhsupurnrzAShT7LoTXm+PqdcoYgrgm7u6v8Q6+4J0pYVG1sM1SHcVjFDqKH q3CvJxJ3t2D91lYCQW/6sK X-Google-Smtp-Source: AGHT+IHahjSONPKxAnurJsJc+15k5alR5RF3SODZ4jyoW/piPAC9UEYf9hOGz/1Ug5hdGRWXWbYYpA== X-Received: by 2002:a05:6a20:8e01:b0:199:db4a:face with SMTP id y1-20020a056a208e0100b00199db4afacemr1073719pzj.85.1704978966938; Thu, 11 Jan 2024 05:16:06 -0800 (PST) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id ei30-20020a056a0080de00b006d9a6a9992dsm1103202pfb.123.2024.01.11.05.16.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 11 Jan 2024 05:16:06 -0800 (PST) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: paul.walmsley@sifive.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, bjorn@kernel.org, charlie@rivosinc.com, ardb@kernel.org, arnd@arndb.de, peterz@infradead.org, tglx@linutronix.de, ebiggers@kernel.org, Andy Chiu , Albert Ou Subject: [v10, 00/10] riscv: support kernel-mode Vector Date: Thu, 11 Jan 2024 13:15:48 +0000 Message-Id: <20240111131558.31211-1-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240111_051609_114575_FAD787D2 X-CRM114-Status: GOOD ( 20.66 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This series provides support running Vector in kernel mode. Additionally, kernel-mode Vector can be configured to run without turnning off preemption on a CONFIG_PREEMPT kernel. Along with the suport, we add Vector optimized copy_{to,from}_user. And provide a simple threshold to decide when to run the vectorized functions. We decided to drop vectorized memcpy/memset/memmove for the moment due to the concern of memory side-effect in kernel_vector_begin(). The detailed description can be found at v9[1] This series is composed by 4 parts: patch 1-4: adds basic support for kernel-mode Vector patch 5: includes vectorized copy_{to,from}_user into the kernel patch 6: refactor context switch code in fpu [2] patch 7-10: provides some code refactors and support for preemptible kernel-mode Vector. This series can be merged if we feel any part of {1~4, 5, 6, 7~10} is mature enough. This patch is tested on a QEMU with V and verified that booting, normal userspace operations all work as usual with thresholds set to 0. Also, we test by launching multiple kernel threads which continuously executes and verifies Vector operations in the background. The module that tests these operation is expected to be upstream later. v9 of this series can be found at [1] [1]:https://lore.kernel.org/all/20231229143627.22898-1-andy.chiu@sifive.com/ Patch summary: - Updated patches: 1, 5, 10 - New patch: (none) - Unchanged patch: 2, 3, 4, 6, 7, 8, 9 - Deleted patch: (none) Changelog v10: - Refactor comments (1), Eric - Remove duplicate assembly code (5), Charlie - Optimize unnecessary compiler barriers in preempt_v (10) - Address bugs in context-saving for preempt_v (10) - Correct dirty marking/clearing code for preempt_v (10) Changelog v9: - Use one bit to record the on/off status of kernel-mode Vector - Temporarily drop vectorized mem* functions - Add a patch to refactor context switch in fpu - silence lockdep and use WARN_ON instead Changelog v8: - Address build fail on no-mmu config - Fix build fail with W=1 - Refactor patches (1, 2), Eric Changelog v7: - Fix build fail for allmodconfig and test building the series with allmodconfig/allyesconfig Changelog v6: - Provide a more robust check on the use of non-preemptible Vector. - Add Kconfigs to set threshold value at compile time. (Charlie) - Add a patch to utilize kmem_cache_* for V context allocations. - Re-write and add preemptible Vector. Changelog v5: - Rebase on top of riscv for-next (6.7-rc1) Changelog v4: - Use kernel_v_flags and helpers to track vector context. - Prevent softirq from nesting V context for non-preempt V - Add user copy and mem* routines Changelog v3: - Rebase on top of riscv for-next (6.6-rc1) - Fix a build issue (Conor) - Guard vstate_save, vstate_restore with {get,put}_cpu_vector_context. - Save V context after disabling preemption. (Guo) - Remove irqs_disabled() check from may_use_simd(). (Björn) - Comment about nesting V context. Changelog v2: - fix build issues - Follow arm's way of starting kernel-mode simd code: - add include/asm/simd.h and rename may_use_vector() -> may_use_simd() - return void in kernel_vector_begin(), and BUG_ON if may_use_simd() fails - Change naming scheme for functions/macros (Conor): - remove KMV - 's/rvv/vector/' - 's/RISCV_ISA_V_PREEMPTIVE_KMV/RISCV_ISA_V_PREEMPTIVE/' - 's/TIF_RISCV_V_KMV/TIF_RISCV_V_KERNEL_MODE/' Andy Chiu (8): riscv: vector: make Vector always available for softirq context riscv: sched: defer restoring Vector context for user riscv: lib: vectorize copy_to_user/copy_from_user riscv: fpu: drop SR_SD bit checking riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() riscv: vector: use a mask to write vstate_ctrl riscv: vector: use kmem_cache to manage vector context riscv: vector: allow kernel-mode Vector with preemption Greentime Hu (2): riscv: Add support for kernel mode vector riscv: Add vector extension XOR implementation arch/riscv/Kconfig | 22 +++ arch/riscv/include/asm/asm-prototypes.h | 27 +++ arch/riscv/include/asm/entry-common.h | 17 ++ arch/riscv/include/asm/processor.h | 41 +++- arch/riscv/include/asm/simd.h | 64 ++++++ arch/riscv/include/asm/switch_to.h | 3 +- arch/riscv/include/asm/thread_info.h | 2 + arch/riscv/include/asm/vector.h | 90 +++++++-- arch/riscv/include/asm/xor.h | 68 +++++++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/entry.S | 8 + arch/riscv/kernel/kernel_mode_vector.c | 247 ++++++++++++++++++++++++ arch/riscv/kernel/process.c | 13 +- arch/riscv/kernel/ptrace.c | 7 +- arch/riscv/kernel/signal.c | 7 +- arch/riscv/kernel/vector.c | 50 ++++- arch/riscv/lib/Makefile | 7 +- arch/riscv/lib/riscv_v_helpers.c | 44 +++++ arch/riscv/lib/uaccess.S | 10 + arch/riscv/lib/uaccess_vector.S | 44 +++++ arch/riscv/lib/xor.S | 81 ++++++++ 21 files changed, 825 insertions(+), 28 deletions(-) create mode 100644 arch/riscv/include/asm/simd.h create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/riscv_v_helpers.c create mode 100644 arch/riscv/lib/uaccess_vector.S create mode 100644 arch/riscv/lib/xor.S