From patchwork Thu Dec 21 13:43:07 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13502192 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 84432C35274 for ; Thu, 21 Dec 2023 13:43:38 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=IQP/S9QIo10s0aVI3VIUqfTCEOuwDSWG6S3wbr7HXkc=; b=16UH3WNFTkIcsu LSc5EB5xaXpkWntn85Vtf7yPXws7DCJtOorzFqAYqTgMBahZjkL7O3gP8HfjY/retpGKejNcOazxp v1ra8i4o1yS77Qwq0ukvWIlJYxkzjxV6hVqRjx/rb9OUc9gOvjHOn7qpa20EvVy5BgqXKpnYLtOvn p54tKhV4KX+k8UQGXn52RtzC0sPqDnrkCiEVBaxC/XA5TILShA6zi7qAeYdWp94R6ehcYIvlP8dOs TG/cy+Y23C/3nll3DGk/XjfJfD0GG2BLCs27foLEU3BB1XVul5PlpzfdpAhrpSKSPiBDZIONhkiQl 5MH8zdxSr5hfqsYpe2Fw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rGJKl-0031XP-0C; Thu, 21 Dec 2023 13:43:31 +0000 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rGJKi-0031W3-1L for linux-riscv@lists.infradead.org; Thu, 21 Dec 2023 13:43:30 +0000 Received: by mail-pg1-x535.google.com with SMTP id 41be03b00d2f7-5cd54e5fbb2so351757a12.2 for ; Thu, 21 Dec 2023 05:43:26 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1703166205; x=1703771005; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=I3KNa9AnSJE/Di5gNTgZNEwGJWU9DjXcRbTulcD+gAQ=; b=CXe7/cYYdBrpdDLKDan3cBo0/iIyqtI4H4iZWB9hujtZhhFk30azjLa5vL6kiMyuy7 vvq2JFWT3fIPcEhTspi/BWm3ZIEcoQpLRdHQJp2zpZCU6NWTByWCPES73oNRFwt15uMa bP/BXRW/MXUT+EpGWHYFb5ou43Jp134uexQDzT2QwcnU872B3QKiFOXo5vYoMcp1o4Hq UYIJWsBf0fz5GB3SM4On9r3tlUBt8Zvm2bryv6cOzk3PKxufrCjyZVVpgzyOb0Csugx2 QXXuhHfrFGhMrQxeungTpKclVyWw0C52hqavHLieFRLXlX86wYe4kGo4U679bE9dt5/r neFA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1703166205; x=1703771005; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=I3KNa9AnSJE/Di5gNTgZNEwGJWU9DjXcRbTulcD+gAQ=; b=OvjhHhayHoZ8zAbz4x8FeE60lB2/clTtYF9kIERSbUbCCpBRn7Yn8KPkOQILOvh+uc IcPEKqXA+FDhMLv7+FEGdZoZsU96KuQ/Fzp0XtE4QCRS0+ZmJ3zeAntfN6BNiBFBTp0F rOuYjLGIggCPIrhgC4JhBpDG0vrRe+ljohbr2m+j0dT2LtWiD4SVSsB3npkORaXdwBlE ParjYoLsam1o0aubzxDh4HwJRaoOYWxQcE+jBRmmT3naNSg/2l6rrOH5iP6jfRD9NZVt Yz0QB/vj1fkH2dh5AODthD70qv13C+FrF+/IZl/3utqyQQOxe8NGXPIILu4brNNI//tE E5JQ== X-Gm-Message-State: AOJu0YzxJyEXUxAN5388lDF3aK1lpDtluVH/cS/MchDIP6DjndcPd8NO F9zX0vA1GiNsehk21QUcrJ2ajKtN5H//yyuQLTFKv/73wwnapWkcu+k2bkO5S3WJo723nsedprb DoGdZTHqjD3NWiwaObzVzSNXMic54WK69Sg5qbIBmVIePWYF9MHNUY/zdlRAkMcHm3m6TfjETRA ZfLHiPHJguphM/ X-Google-Smtp-Source: AGHT+IGc+qTk5KIh0CzKbQ9K1MW9q6LvvFwre+6H/Qlqp4IA9wlxKKZYhtc8pZEoGfqnst1k/oBOsA== X-Received: by 2002:a17:90a:fc96:b0:28c:168:a476 with SMTP id ci22-20020a17090afc9600b0028c0168a476mr64770pjb.44.1703166204923; Thu, 21 Dec 2023 05:43:24 -0800 (PST) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id iw3-20020a170903044300b001c72d5e16acsm1646001plb.57.2023.12.21.05.43.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 21 Dec 2023 05:43:24 -0800 (PST) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: paul.walmsley@sifive.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, bjorn@kernel.org, charlie@rivosinc.com, ardb@kernel.org, arnd@arndb.de, peterz@infradead.org, tglx@linutronix.de, Andy Chiu , Albert Ou Subject: [v7, 00/10] riscv: support kernel-mode Vector Date: Thu, 21 Dec 2023 13:43:07 +0000 Message-Id: <20231221134318.28105-1-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231221_054328_466157_9F39CC34 X-CRM114-Status: GOOD ( 16.70 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This series provides support for running Vector code in kernel mode. Along with the suport, we add some Vector optimized routines. And provide a simple threshold to decide when to run the vectorized functions. I am adding back preemptible Vector as they are ready for review now. Having them together makes it easier to be reviewed and tested. Meanwhile, as they are appended at the end, this series can be merged incrementally if we feel any part of {1~4, 5~6, 7~10} is mature enough. This patch is tested on a QEMU with V and verified that booting, normal userspace operations all work as usual with thresholds set to 0. Also, we test by launching multiple kernel threads which continuously executes and verifies Vector operations in the background. The module that tests these operation is expected to be upstream later. v6 of this series can be found at [1] Link: https://lore.kernel.org/all/20231220075412.24084-1-andy.chiu@sifive.com/ Patch summary: - Updated patches: 1, 3, 6 - New patch: none - Unchanged patch: 2, 4, 5, 7, 8, 9, 10 - Deleted patch: none Changelog v7: - Fix build fail for allmodconfig and test building the series with allmodconfig/allyesconfig Changelog v6: - Provide a more robust check on the use of non-preemptible Vector. - Add Kconfigs to set threshold value at compile time. (Charlie) - Add a patch to utilize kmem_cache_* for V context allocations. - Re-write and add preemptible Vector. Changelog v5: - Rebase on top of riscv for-next (6.7-rc1) Changelog v4: - Use kernel_v_flags and helpers to track vector context. - Prevent softirq from nesting V context for non-preempt V - Add user copy and mem* routines Changelog v3: - Rebase on top of riscv for-next (6.6-rc1) - Fix a build issue (Conor) - Guard vstate_save, vstate_restore with {get,put}_cpu_vector_context. - Save V context after disabling preemption. (Guo) - Remove irqs_disabled() check from may_use_simd(). (Björn) - Comment about nesting V context. Changelog v2: - fix build issues - Follow arm's way of starting kernel-mode simd code: - add include/asm/simd.h and rename may_use_vector() -> may_use_simd() - return void in kernel_vector_begin(), and BUG_ON if may_use_simd() fails - Change naming scheme for functions/macros (Conor): - remove KMV - 's/rvv/vector/' - 's/RISCV_ISA_V_PREEMPTIVE_KMV/RISCV_ISA_V_PREEMPTIVE/' - 's/TIF_RISCV_V_KMV/TIF_RISCV_V_KERNEL_MODE/' Andy Chiu (8): riscv: vector: make Vector always available for softirq context riscv: sched: defer restoring Vector context for user riscv: lib: vectorize copy_to_user/copy_from_user riscv: lib: add vectorized mem* routines riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() riscv: vector: use a mask to write vstate_ctrl riscv: vector: use kmem_cache to manage vector context riscv: vector: allow kernel-mode Vector with preemption Greentime Hu (2): riscv: Add support for kernel mode vector riscv: Add vector extension XOR implementation arch/riscv/Kconfig | 46 +++++ arch/riscv/include/asm/asm-prototypes.h | 14 ++ arch/riscv/include/asm/entry-common.h | 17 ++ arch/riscv/include/asm/processor.h | 39 ++++- arch/riscv/include/asm/simd.h | 64 +++++++ arch/riscv/include/asm/thread_info.h | 2 + arch/riscv/include/asm/vector.h | 103 ++++++++++-- arch/riscv/include/asm/xor.h | 68 ++++++++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/entry.S | 8 + arch/riscv/kernel/kernel_mode_vector.c | 212 ++++++++++++++++++++++++ arch/riscv/kernel/process.c | 14 +- arch/riscv/kernel/ptrace.c | 7 +- arch/riscv/kernel/signal.c | 7 +- arch/riscv/kernel/vector.c | 50 +++++- arch/riscv/lib/Makefile | 6 + arch/riscv/lib/memcpy_vector.S | 29 ++++ arch/riscv/lib/memmove_vector.S | 49 ++++++ arch/riscv/lib/memset_vector.S | 33 ++++ arch/riscv/lib/riscv_v_helpers.c | 64 +++++++ arch/riscv/lib/uaccess.S | 10 ++ arch/riscv/lib/uaccess_vector.S | 50 ++++++ arch/riscv/lib/xor.S | 81 +++++++++ 23 files changed, 948 insertions(+), 26 deletions(-) create mode 100644 arch/riscv/include/asm/simd.h create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/memcpy_vector.S create mode 100644 arch/riscv/lib/memmove_vector.S create mode 100644 arch/riscv/lib/memset_vector.S create mode 100644 arch/riscv/lib/riscv_v_helpers.c create mode 100644 arch/riscv/lib/uaccess_vector.S create mode 100644 arch/riscv/lib/xor.S