From patchwork Mon Jan 15 05:59:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13519342 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 92AA8C3DA79 for ; Mon, 15 Jan 2024 07:00:35 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=HonQDit0elK5lbxl/uq/t1nmdoB7cTZ/9O5yH7dQHQk=; b=wl1KefULrVl4eN vSSIILJcOWb10ONWkxd2Kn7uwbKIPsEPRAh8oJyJQNhGjBSBw3lPcgCA39l89MpQkIfubN/6VhNMz K/cRQfSb3dXjaoBWI9lsMiPPyOdY2ReG3W/R3QmhrIHzB4kUAoJ3qZndCHodaXR6EOwednuuQ4lot Pa2YhocbWzEj6GAIlYiwa5Fywqpaoa3OVH5IXC18vZ72btozBYvWD6XtcILUFnV5sdmg6WZeTLRSG wJNVrFYypNmlUQjRzHiXvm2ChxOj5dzttyzkrt2SICBCWa1BzuQEHFFGnzuF7x5NDNlRfaQxyffY2 2kDYypcaAO6GOFat9JsA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rPGxG-0081N9-1W; Mon, 15 Jan 2024 07:00:18 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rPG0l-007qBe-32 for linux-riscv@bombadil.infradead.org; Mon, 15 Jan 2024 05:59:52 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:Content-Type :MIME-Version:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To:Content-ID: Content-Description:In-Reply-To:References; bh=jrq8wMwl32S4LobLTsnl3WO9iYRitKzpRDaUW/RsfOA=; b=jegPtifV61mduCvP9GyLMzOV+G tCyZC5yTriiRF1CbKnt3LT7ONAqREolSu8l0Ja0IiOg+QAiYq3Wy6DHydFf6yOE2Jyzypd2lcp0pZ efsS+7HyDez5dOlMb3DmBnQB/6eZqYa0eWR4OSevNQUqLRqiQdci3QeISqThHZhrrCVbcAD38UjhF RXY2q3jqJDbvdyPW4rhpLmwfFxF6Fd956AuT3YjW49PsluWN8PVsfeJX7A4Sz4PUhhHf6+ywSRe9X EJQV1Nx2NkDd2voHHt+qRcVSgwbz54ngIE5jNNMoM9ahvehCm9kR76Ph/fGhW2YbkdXZNeabduPV8 ZKhWc7Ig==; Received: from mail-oi1-x22a.google.com ([2607:f8b0:4864:20::22a]) by desiato.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rPG0h-00D1Y8-10 for linux-riscv@lists.infradead.org; Mon, 15 Jan 2024 05:59:50 +0000 Received: by mail-oi1-x22a.google.com with SMTP id 5614622812f47-3bd623813c3so482688b6e.2 for ; Sun, 14 Jan 2024 21:59:41 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1705298377; x=1705903177; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=jrq8wMwl32S4LobLTsnl3WO9iYRitKzpRDaUW/RsfOA=; b=g+rK3Jnlw3n+gru7Dp2iRSynwnSHkAfQlEyBvEZCaW0tEV+Z5BDyLc5TWTyOrVufdq YgT0Y6zLkiH03nmpiSqRxj/X7V0RnK7rH3qyFhFnhNSIp6bdElzSeCBHzMyVT9CXQ1ye MN6TfTX1mijTPhyBFEWmaZRcP4zKiLmpK9yD/DKFZGk6Dhtoln9teHoMuZQUYwRxyISy hytcsOdQ0pGDakrtdVMG3o09Lks9JxAKOjF5e8uiVrnd90m58Zo8hi/WKDRtBhIBOASM aR5RO2BLn+0k0UdTD8/tCgNwUnu2UoK4gf3djQlrrgMmovJIuw6e2pWDjZVWeCCHJKgF CXPQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1705298377; x=1705903177; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=jrq8wMwl32S4LobLTsnl3WO9iYRitKzpRDaUW/RsfOA=; b=RBTuhrRroxEd0sgd5yk8G+OJwzHMXhxn/lMS2MzZ53/04Q1QX1r5NTIg/L+9XUQdTu YpS4Njsm6vPVNHMBLqM1tzk2Y9Wfg/vQ6C9/GXkKsDBz5ClahxmsQIbR0VL453XnlIhy zZNsIcN+Q85adH4HA3gYO1kPGQNFCgYmAfMxnNxuuviVIoDE7/ykS081gXT3joorMT3c XMRHUys0ozlACCyYlcvy2WiidGqHMXd6rJvhi2uxeF9uTykhHbk0oTObJHFI/oNI06NN jpUz+Clu94CfdN2qgNbI1rhp8OytAH5QcFTeqFHEcDFhMbL28ZJN0/PNnmbay8QIlb2+ nZ7A== X-Gm-Message-State: AOJu0Yxih5owbeBSwPU2W2rO72eG5sG27ZH7EB/cJM1v2jFNZfRLEKOS aX9ze3kR7D4ItpaizvMPlSUtFxFu8Pw2n6lOnVM0flHGg2wbdyPM0DKQEwy1/QxExZNJNORHhnU Ql8IFiY6f5KYW3Ya4nBZ8Zzd3DJYEHPPBXBl+zplfGZfVGMfY6swDKaMR7mmnpkEdUNK+hV7NN3 Uf0aHx+k2UaP+n0wWVz7bl X-Google-Smtp-Source: AGHT+IHDLGFsMByM1+IrVS5oOv0GNxUWbzy18fk25d0p2QnSF4XkPnN10fH3Pc3OW4EEcrvUZXwyDA== X-Received: by 2002:a05:6808:114e:b0:3bc:397f:8417 with SMTP id u14-20020a056808114e00b003bc397f8417mr3655621oiu.71.1705298377421; Sun, 14 Jan 2024 21:59:37 -0800 (PST) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id o18-20020a637e52000000b005b9083b81f0sm7392988pgn.36.2024.01.14.21.59.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 14 Jan 2024 21:59:36 -0800 (PST) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: paul.walmsley@sifive.com, greentime.hu@sifive.com, guoren@linux.alibaba.com, bjorn@kernel.org, charlie@rivosinc.com, ardb@kernel.org, arnd@arndb.de, peterz@infradead.org, tglx@linutronix.de, ebiggers@kernel.org, Andy Chiu , Albert Ou Subject: [v11, 00/10] riscv: support kernel-mode Vector Date: Mon, 15 Jan 2024 05:59:19 +0000 Message-Id: <20240115055929.4736-1-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240115_055947_800846_8B99902D X-CRM114-Status: GOOD ( 21.78 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This series provides support running Vector in kernel mode. Additionally, kernel-mode Vector can be configured to run without turnning off preemption on a CONFIG_PREEMPT kernel. Along with the suport, we add Vector optimized copy_{to,from}_user. And provide a simple threshold to decide when to run the vectorized functions. We decided to drop vectorized memcpy/memset/memmove for the moment due to the concern of memory side-effect in kernel_vector_begin(). The detailed description can be found at v9[0] This series is composed by 4 parts: patch 1-4: adds basic support for kernel-mode Vector patch 5: includes vectorized copy_{to,from}_user into the kernel patch 6: refactor context switch code in fpu [1] patch 7-10: provides some code refactors and support for preemptible kernel-mode Vector. This series can be merged if we feel any part of {1~4, 5, 6, 7~10} is mature enough. This patch is tested on a QEMU with V and verified that booting, normal userspace operations all work as usual with thresholds set to 0. Also, we test by launching multiple kernel threads which continuously executes and verifies Vector operations in the background. The module that tests these operation is expected to be upstream later. v10 of this series can be found at [2] [0]https://lore.kernel.org/all/20231229143627.22898-1-andy.chiu@sifive.com/ [1]https://lore.kernel.org/all/CABgGipX7Jf7M8ZYgeRPcE9tkzc7XWpfWErsiacn2Pa9h=vG2cQ@mail.gmail.com/T/ [2]https://lore.kernel.org/all/20240111131558.31211-1-andy.chiu@sifive.com/ Patch summary: - Updated patches: 5, 9 - New patch: (none) - Unchanged patch: 1, 2, 3, 4, 6, 7, 8, 10 - Deleted patch: (none) Changelog v11: - This is a quick respin to address boot failing on ubuntu and alpine. - Pass the updated copy size when calling scalar fallback. - Skip some bytes for scalar fallback when fauting at vse8.v with a non-zero $vstart. (Guo) - Guard riscv_v_setup_ctx_cache() with has_vector() check. Changelog v10: - Refactor comments (1), Eric - Remove duplicate assembly code (5), Charlie - Optimize unnecessary compiler barriers in preempt_v (10) - Address bugs in context-saving for preempt_v (10) - Correct dirty marking/clearing code for preempt_v (10) Changelog v9: - Use one bit to record the on/off status of kernel-mode Vector - Temporarily drop vectorized mem* functions - Add a patch to refactor context switch in fpu - silence lockdep and use WARN_ON instead Changelog v8: - Address build fail on no-mmu config - Fix build fail with W=1 - Refactor patches (1, 2), Eric Changelog v7: - Fix build fail for allmodconfig and test building the series with allmodconfig/allyesconfig Changelog v6: - Provide a more robust check on the use of non-preemptible Vector. - Add Kconfigs to set threshold value at compile time. (Charlie) - Add a patch to utilize kmem_cache_* for V context allocations. - Re-write and add preemptible Vector. Changelog v5: - Rebase on top of riscv for-next (6.7-rc1) Changelog v4: - Use kernel_v_flags and helpers to track vector context. - Prevent softirq from nesting V context for non-preempt V - Add user copy and mem* routines Changelog v3: - Rebase on top of riscv for-next (6.6-rc1) - Fix a build issue (Conor) - Guard vstate_save, vstate_restore with {get,put}_cpu_vector_context. - Save V context after disabling preemption. (Guo) - Remove irqs_disabled() check from may_use_simd(). (Björn) - Comment about nesting V context. Changelog v2: - fix build issues - Follow arm's way of starting kernel-mode simd code: - add include/asm/simd.h and rename may_use_vector() -> may_use_simd() - return void in kernel_vector_begin(), and BUG_ON if may_use_simd() fails - Change naming scheme for functions/macros (Conor): - remove KMV - 's/rvv/vector/' - 's/RISCV_ISA_V_PREEMPTIVE_KMV/RISCV_ISA_V_PREEMPTIVE/' - 's/TIF_RISCV_V_KMV/TIF_RISCV_V_KERNEL_MODE/' Andy Chiu (8): riscv: vector: make Vector always available for softirq context riscv: sched: defer restoring Vector context for user riscv: lib: vectorize copy_to_user/copy_from_user riscv: fpu: drop SR_SD bit checking riscv: vector: do not pass task_struct into riscv_v_vstate_{save,restore}() riscv: vector: use a mask to write vstate_ctrl riscv: vector: use kmem_cache to manage vector context riscv: vector: allow kernel-mode Vector with preemption Greentime Hu (2): riscv: Add support for kernel mode vector riscv: Add vector extension XOR implementation arch/riscv/Kconfig | 22 +++ arch/riscv/include/asm/asm-prototypes.h | 27 +++ arch/riscv/include/asm/entry-common.h | 17 ++ arch/riscv/include/asm/processor.h | 41 +++- arch/riscv/include/asm/simd.h | 64 ++++++ arch/riscv/include/asm/switch_to.h | 3 +- arch/riscv/include/asm/thread_info.h | 2 + arch/riscv/include/asm/vector.h | 90 +++++++-- arch/riscv/include/asm/xor.h | 68 +++++++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/entry.S | 8 + arch/riscv/kernel/kernel_mode_vector.c | 247 ++++++++++++++++++++++++ arch/riscv/kernel/process.c | 13 +- arch/riscv/kernel/ptrace.c | 7 +- arch/riscv/kernel/signal.c | 7 +- arch/riscv/kernel/vector.c | 53 ++++- arch/riscv/lib/Makefile | 7 +- arch/riscv/lib/riscv_v_helpers.c | 45 +++++ arch/riscv/lib/uaccess.S | 10 + arch/riscv/lib/uaccess_vector.S | 53 +++++ arch/riscv/lib/xor.S | 81 ++++++++ 21 files changed, 838 insertions(+), 28 deletions(-) create mode 100644 arch/riscv/include/asm/simd.h create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/riscv_v_helpers.c create mode 100644 arch/riscv/lib/uaccess_vector.S create mode 100644 arch/riscv/lib/xor.S Tested-by: Björn Töpel Tested-by: Lad Prabhakar