From patchwork Wed Dec 13 13:13:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13491000 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id F4052C4332F for ; Wed, 13 Dec 2023 13:13:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=EvONdeuZOf07o2/qKv3KqW2Iog/Mzd7QGJe0TNv705M=; b=eKxO/siNu9+J5q pI/HIopUQthccRc4cNf/00EiJ6V8SkF9aeN/TGOn/tEwpbw3iFFt8nDkt+l3G/aXX1j8iR/xusdDX rxlzvOBpQJL/6THAEY6n0CzmZpjdT0Y1g+7G8t+ehgVlMGBl8AskGTj/hx/ubkx7aLOYJuPdNYu9B mQPqN68blSv3YS+slGt603q6GxwWjHD1w60Pq1c+73e7uRJ7i2dIFx09c9C/cvEFLXMGxPyVEa7zM zey2v0GNhnIYcqKmwW4SaSVIpmRSpZrNvq36EC5PdqernDlXTGGkTeCpTQth17vE9LoDFx/+nLZUR w9t2VBCKvQxmWDqhWn1A==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rDP3b-00Ekov-1I; Wed, 13 Dec 2023 13:13:47 +0000 Received: from mail-pf1-x434.google.com ([2607:f8b0:4864:20::434]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rDP3X-00Eknd-1S for linux-riscv@lists.infradead.org; Wed, 13 Dec 2023 13:13:45 +0000 Received: by mail-pf1-x434.google.com with SMTP id d2e1a72fcca58-6ce939ecfc2so6272946b3a.2 for ; Wed, 13 Dec 2023 05:13:40 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1702473219; x=1703078019; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=WFwtIbL//4RaPbpliYdco6dnfSaYDCPRp10C0v7FTQ8=; b=QF+9WL/COwXEqDTKkyDilpTb35CQOO6uoP0ocaEoP8k+QDSLvErKqHtHtEEnf8FCSy jGk3c/ekI0kW/RsyZv6cvpNkYXCu0hAMobW/oN9MENCKveq7O0tYglPmOTyxyiEWHVpc +lq9Jajmw9uJSwWUpzarCqQOuqasn6CbE+GaAAj9J8eb5V9HUCftOPyefgXqtD3FAuJY 1ovVgoFJTdJErX5BBr7LopULQjGtL96bRKD5mPNkAobiiOzYuTQuKE6EKFvpcn7qZKFS fvsPkfT9Y92wz4Eq9GoeuQgREtagPtoq3C/4k79zNqn30lW9esVr0xTmeVTYVbWx3Uxc Bghg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702473219; x=1703078019; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=WFwtIbL//4RaPbpliYdco6dnfSaYDCPRp10C0v7FTQ8=; b=MYqj7LXGTu69wAG9shYxDAxm9gbaweY6GnNYFRGLzanlkR+5lQZUGoajJBh+EQue8A FlcrcQ5NsunTyesxbkLbXXIgAVj5NstQ+LZr7MJ2xXhkEi2qcT/yIALZDiajiJ0RO7CJ 1kxL54SqL54s/u3RLjgOj6BzhkP0uePLNRvhkBIQ5DLZCmmIAwwNqLA1h9Phx01voJWk ExoyLTk/PdxHjUxCyGMnA/fmihcT0evgLtPnAWvgAcISlbAy1BgEpw+r1aNUR67YCQ1u ofk74csx1rZQZOSILsjQqmMBE0jQgismNPlJFeE2quYkgCxDSbD4SYHVHVVN8WnMXj1/ 2fOw== X-Gm-Message-State: AOJu0Yxsyl/dY2c/ul4POfgrgUSOzb5269ofGkpCDZvFiCAzIW30ZnkQ aUkmnRoOkqt8wyMoek06zJx2+E9s1SdgdACkFWZuKW0VMos0//FMdJX9GvvcY6cHgdW7WXll/sT c50oe4G9X2NRXVY6twCuZKx63KJGlEqGnO3Q5tUl0DJz/Uff+5oEShfRQ9YWvctJ3buEKBg/hE9 v7VdajV/AiqsvS X-Google-Smtp-Source: AGHT+IFlnotUmYrqQ4cJxOVP9Ctj3xHckr0CCDzE43+hY4W3MD9qa/emjKr2RJYDljZq1SYeq07fUw== X-Received: by 2002:a05:6a00:1798:b0:6cb:a93c:dfd1 with SMTP id s24-20020a056a00179800b006cba93cdfd1mr9542257pfg.14.1702473218927; Wed, 13 Dec 2023 05:13:38 -0800 (PST) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id fn7-20020a056a002fc700b006cecaff9e29sm9928601pfb.128.2023.12.13.05.13.36 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 13 Dec 2023 05:13:38 -0800 (PST) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: greentime.hu@sifive.com, guoren@linux.alibaba.com, bjorn@kernel.org, charlie@rivosinc.com, ardb@kernel.org, arnd@arndb.de, Andy Chiu , Paul Walmsley , Albert Ou Subject: [v4, 0/6] riscv: support kernel-mode Vector Date: Wed, 13 Dec 2023 13:13:15 +0000 Message-Id: <20231213131321.12862-1-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231213_051343_541831_31943606 X-CRM114-Status: GOOD ( 14.26 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This series provides support for running Vector code in kernel mode. Along with the suport, we add some Vector optimized routines. And provide a simple threshold to decide when to run the vectorized functions. We decided to split the patch series into 2 parts. The first part contains running kernel-mode Vector with preemption disabled. The second part will add support for running Vector with preemption. This patch is tested on a QEMU with V and verified that booting, normal userspace operations all work as usual with thresholds set to 0. v3 of this series can be found at [1] Link: https://lore.kernel.org/all/20231019154552.23351-1-andy.chiu@sifive.com/ Patch summary (on current order): - Updated patches: 1, 4 - New patch: 2, 5, 6 - Unchanged patch: 3 - Deleted patch: Patches related to preempt_v will be sent separately Changelog v4: - Rebase on top of riscv for-next (6.7-rc1) - Use kernel_v_flags and helpers to track vector context. - Prevent softirq from nesting V context for non-preempt V - Add user copy and mem* routines Changelog v3: - Rebase on top of riscv for-next (6.6-rc1) - Fix a build issue (Conor) - Guard vstate_save, vstate_restore with {get,put}_cpu_vector_context. - Save V context after disabling preemption. (Guo) - Remove irqs_disabled() check from may_use_simd(). (Björn) - Comment about nesting V context. Changelog v2: - fix build issues - Follow arm's way of starting kernel-mode simd code: - add include/asm/simd.h and rename may_use_vector() -> may_use_simd() - return void in kernel_vector_begin(), and BUG_ON if may_use_simd() fails - Change naming scheme for functions/macros (Conor): - remove KMV - 's/rvv/vector/' - 's/RISCV_ISA_V_PREEMPTIVE_KMV/RISCV_ISA_V_PREEMPTIVE/' - 's/TIF_RISCV_V_KMV/TIF_RISCV_V_KERNEL_MODE/' Andy Chiu (4): riscv: vector: make Vector always available for softirq context riscv: sched: defer restoring Vector context for user riscv: lib: vectorize copy_to_user/copy_from_user riscv: lib: add vectorized mem* routines Greentime Hu (2): riscv: Add support for kernel mode vector riscv: Add vector extension XOR implementation arch/riscv/include/asm/entry-common.h | 17 +++++ arch/riscv/include/asm/processor.h | 15 +++- arch/riscv/include/asm/simd.h | 46 +++++++++++ arch/riscv/include/asm/thread_info.h | 2 + arch/riscv/include/asm/vector.h | 32 +++++++- arch/riscv/include/asm/xor.h | 82 ++++++++++++++++++++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/kernel_mode_vector.c | 101 +++++++++++++++++++++++++ arch/riscv/kernel/process.c | 4 +- arch/riscv/kernel/ptrace.c | 5 +- arch/riscv/kernel/signal.c | 5 +- arch/riscv/kernel/vector.c | 2 +- arch/riscv/lib/Makefile | 6 ++ arch/riscv/lib/memcpy_vector.S | 29 +++++++ arch/riscv/lib/memmove_vector.S | 49 ++++++++++++ arch/riscv/lib/memset.S | 2 +- arch/riscv/lib/memset_vector.S | 33 ++++++++ arch/riscv/lib/riscv_v_helpers.c | 59 +++++++++++++++ arch/riscv/lib/uaccess.S | 11 +++ arch/riscv/lib/uaccess_vector.S | 55 ++++++++++++++ arch/riscv/lib/xor.S | 81 ++++++++++++++++++++ 21 files changed, 630 insertions(+), 7 deletions(-) create mode 100644 arch/riscv/include/asm/simd.h create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/memcpy_vector.S create mode 100644 arch/riscv/lib/memmove_vector.S create mode 100644 arch/riscv/lib/memset_vector.S create mode 100644 arch/riscv/lib/riscv_v_helpers.c create mode 100644 arch/riscv/lib/uaccess_vector.S create mode 100644 arch/riscv/lib/xor.S