From patchwork Wed Dec 20 07:54:02 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13499630 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 0619CC3DA6E for ; Wed, 20 Dec 2023 07:57:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=Nx/FrjSOrVEIvIn7uYcB4wumk6le1SX7eSRXoEXhSwo=; b=0pGzhrg/4w0tEi IfNn02WK+5nNSGNX6rZjG72MsYJG0NrnC8iPKCiTlVCWqnOajxZS52NslT6nz8TgX3yctf8ucMgvO Vo9bAd0yHMW1+ZKj+eYHzF8jHPuHyMSJZFLkkd8WnbcG2Xc3WEa/oaELXCAjfK/mbCBBiZKBD+yI9 IkzxeI49PGoXcM91cj3Nw93B/9s9dXBWt3CG7JHPI8fjkC2LwOV1KtQYXrYSTBnlonIyEu+smPKv/ 11JsTj+GHxPKZ2HMkvv3FWTF6m8QDWlQpvnNYyA/KPawc5UwAI5nSP9/4CU7rocaJ7a2Rar3kSd6p bzirotj3bCvT7J6/qQaw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rFrRy-00GUHT-1F; Wed, 20 Dec 2023 07:57:06 +0000 Received: from mail-pl1-x630.google.com ([2607:f8b0:4864:20::630]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rFrRv-00GUG5-2B for linux-riscv@lists.infradead.org; Wed, 20 Dec 2023 07:57:05 +0000 Received: by mail-pl1-x630.google.com with SMTP id d9443c01a7336-1d3e416f303so7031875ad.0 for ; Tue, 19 Dec 2023 23:57:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1703059019; x=1703663819; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=BdzoMQiXq1+xyOC1DiyAXBbEpg2SAWo3Xz+Mk18/zJM=; b=ZC8iyA5ADUKIcYpfe6UzDuVKDcIiFUMuT4H30ZQ3QMkRCGhV8hd+vERWNMGzbhUgDD F63azHtdyXwAClKy4+2EgUWJnxy6NveGAOru7d37Z6tMnrVyDEEoPTZr66iOJXL2MBil 1gYsd8fJS2g+57z8r73gBREDZOZRbHcHD16G1hI6uJRr5bDVDbNbjT9JsVaG5QmF1Zyn cQIElYVZZS6DmSzFsGIr9ujFd/EHmihWMsOMnvw4hcjT1jnuGmprYujPpd7T92cxb+nh If/gMg5FLkYvDJtTo7IpKqleRQFpZ1+bh5W99VSMMsk3hS/JFnHqmMD6WgthDDnx20Dd FzQg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703059019; x=1703663819; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=BdzoMQiXq1+xyOC1DiyAXBbEpg2SAWo3Xz+Mk18/zJM=; b=Dn4o0jS1MgzVlcCSKPdxRm1OGC+0x3LeVppWWWFvm6cOD6y3oOlAbrVI259ZBYP2K8 rYEyN3oQF2iG0YrdxZXpWgDAomAVfCtewCI4JOXkpNWu/na3J9wlHC7Wixs3e16cahWb E0K71dX0jb12CLgebgDXQeqdB/fojvGswidCPN4LKqdyK2honWVrrqmgfl3dEXUtrs0E Bk2F1jwF/mED6vAj6dHOMtMc6pkFQNtoIxdP//iq1M7VkycdyQMc6XsyVpAdh2eCWstY oarqPi5J/PYYmMQyf4TcPTwCyCqQleotcpzCk01u2Lrw81AtBVyNAi/Pmj1nOKhRuurv iAvg== X-Gm-Message-State: AOJu0YyEVz31bhYd3N9dPWDMgUoorZbN7m6TwSsgEp13Pg7veou0+nDF B39EXpHYSYKgro8KA0exn3tHKYVqG/FOiG1QydiXQTFR7bYA2lEwiuq0q2sHEgEqgV7toNqmj2U 3hki1G+FoLdBW3/WTVmq5O3mkNfUmPMK5vasl0RklaJo0nBxbrvlqItaHK/VwT0oFyEq7oCuv7+ BWf6qq3Fqym/38 X-Google-Smtp-Source: AGHT+IGDrE88EGbuHuOBSj94jAbUYqODTrV868/WdmWEeoq0hBPuOH8V16KlDcpqsnuXuttpF770tQ== X-Received: by 2002:a17:902:684c:b0:1d3:f285:15a5 with SMTP id f12-20020a170902684c00b001d3f28515a5mr101960pln.35.1703059018790; Tue, 19 Dec 2023 23:56:58 -0800 (PST) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id q20-20020a170902789400b001cf8546335fsm3441453pll.5.2023.12.19.23.56.54 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 19 Dec 2023 23:56:56 -0800 (PST) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: paul.walmsley@sifive.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, bjorn@kernel.org, charlie@rivosinc.com, ardb@kernel.org, arnd@arndb.de, peterz@infradead.org, tglx@linutronix.de, Andy Chiu , Albert Ou Subject: [v6, 00/10] riscv: support kernel-mode Vector Date: Wed, 20 Dec 2023 07:54:02 +0000 Message-Id: <20231220075412.24084-1-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231219_235703_769102_45C3F3CF X-CRM114-Status: GOOD ( 16.88 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This series provides support for running Vector code in kernel mode. Along with the suport, we add some Vector optimized routines. And provide a simple threshold to decide when to run the vectorized functions. I am adding back preemptible Vector as they are ready for review now. Having them together makes it easier to be reviewed and tested. Meanwhile, as they are appended at the end, this series can be merged incrementally if we feel any part of {1~4, 5~6, 7~10} is mature enough. This patch is tested on a QEMU with V and verified that booting, normal userspace operations all work as usual with thresholds set to 0. Also, we test by launching multiple kernel threads which continuously executes and verifies Vector operations in the background. v5 of this series can be found at [1] Link: https://lore.kernel.org/all/20231214155721.1753-1-andy.chiu@sifive.com/ Patch summary (on current order, 7~10 are compared with v3): - Updated patches: 1, 5, 6, 10 - New patch: 8, 9 - Unchanged patch: 2, 3, 4, 7 - Deleted patch: none Changelog v6: - Provide a more robust check on the use of non-preemptible Vector. - Add Kconfigs to set threshold value at compile time. (Charlie) - Add a patch to utilize kmem_cache_* for V context allocations. - Re-write and add preemptible Vector. Changelog v5: - Rebase on top of riscv for-next (6.7-rc1) Changelog v4: - Use kernel_v_flags and helpers to track vector context. - Prevent softirq from nesting V context for non-preempt V - Add user copy and mem* routines Changelog v3: - Rebase on top of riscv for-next (6.6-rc1) - Fix a build issue (Conor) - Guard vstate_save, vstate_restore with {get,put}_cpu_vector_context. - Save V context after disabling preemption. (Guo) - Remove irqs_disabled() check from may_use_simd(). (Björn) - Comment about nesting V context. Changelog v2: - fix build issues - Follow arm's way of starting kernel-mode simd code: - add include/asm/simd.h and rename may_use_vector() -> may_use_simd() - return void in kernel_vector_begin(), and BUG_ON if may_use_simd() fails - Change naming scheme for functions/macros (Conor): - remove KMV - 's/rvv/vector/' - 's/RISCV_ISA_V_PREEMPTIVE_KMV/RISCV_ISA_V_PREEMPTIVE/' - 's/TIF_RISCV_V_KMV/TIF_RISCV_V_KERNEL_MODE/' Andy Chiu (8): riscv: vector: make Vector always available for softirq context riscv: sched: defer restoring Vector context for user riscv: lib: vectorize copy_to_user/copy_from_user riscv: lib: add vectorized mem* routines riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() riscv: vector: use a mask to write vstate_ctrl riscv: vector: use kmem_cache to manage vector context riscv: vector: allow kernel-mode Vector with preemption Greentime Hu (2): riscv: Add support for kernel mode vector riscv: Add vector extension XOR implementation arch/riscv/Kconfig | 46 ++++++ arch/riscv/include/asm/entry-common.h | 17 ++ arch/riscv/include/asm/processor.h | 39 ++++- arch/riscv/include/asm/simd.h | 62 ++++++++ arch/riscv/include/asm/thread_info.h | 2 + arch/riscv/include/asm/vector.h | 103 ++++++++++-- arch/riscv/include/asm/xor.h | 82 ++++++++++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/entry.S | 8 + arch/riscv/kernel/kernel_mode_vector.c | 212 +++++++++++++++++++++++++ arch/riscv/kernel/process.c | 14 +- arch/riscv/kernel/ptrace.c | 7 +- arch/riscv/kernel/signal.c | 7 +- arch/riscv/kernel/vector.c | 50 +++++- arch/riscv/lib/Makefile | 6 + arch/riscv/lib/memcpy_vector.S | 29 ++++ arch/riscv/lib/memmove_vector.S | 49 ++++++ arch/riscv/lib/memset_vector.S | 33 ++++ arch/riscv/lib/riscv_v_helpers.c | 60 +++++++ arch/riscv/lib/uaccess.S | 10 ++ arch/riscv/lib/uaccess_vector.S | 50 ++++++ arch/riscv/lib/xor.S | 81 ++++++++++ 22 files changed, 942 insertions(+), 26 deletions(-) create mode 100644 arch/riscv/include/asm/simd.h create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/memcpy_vector.S create mode 100644 arch/riscv/lib/memmove_vector.S create mode 100644 arch/riscv/lib/memset_vector.S create mode 100644 arch/riscv/lib/riscv_v_helpers.c create mode 100644 arch/riscv/lib/uaccess_vector.S create mode 100644 arch/riscv/lib/xor.S