From patchwork Thu Dec 14 15:57:15 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13493217 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 86849C4332F for ; Thu, 14 Dec 2023 15:58:19 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:Message-Id:Date:Subject:Cc :To:From:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=vda/KblSBCwXJNeh5sD0SYyL8TgXwSp5XTmNOwDNacU=; b=GbQ2osWnUR1fm8 J66kcjBw70nmxD5DXTM5CcJ1TPWGwWcND8obd+1yXzxj5LZh/xCQMeDYNXjJD9RyYiW438p6kfF+5 x7OyIZV3CE1TW8bWgWYj68k0TwlxgmYYV8SYdy/MOBRylA/OJVBK/pO9PlPPPsrTP8rirpi+o4O2Y YnLkD/LqyHz3Z7pCzYAkuBN6xcfgRN9ml/4XpgV9FPR3IrR41jjxTeFHifvqkeG1wGAmLR70PNAsa IDDwpU2e1PakuUSM3m9oK2CHoOSTseZ24aNRllP4eT+53F4w9S4PHDO7nj3bA00pjsbQK557Nbrg2 kZCME9E4U7xNe5dkZRvg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1rDo6B-000jGU-2K; Thu, 14 Dec 2023 15:58:07 +0000 Received: from mail-pl1-x632.google.com ([2607:f8b0:4864:20::632]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1rDo68-000jFp-0Z for linux-riscv@lists.infradead.org; Thu, 14 Dec 2023 15:58:06 +0000 Received: by mail-pl1-x632.google.com with SMTP id d9443c01a7336-1d307cf18fdso28787375ad.3 for ; Thu, 14 Dec 2023 07:58:00 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1702569479; x=1703174279; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:from:to:cc:subject:date:message-id:reply-to; bh=ofzx+PtD9NebQ35OrFIzznCWiX3/O4uzvKbDVCH/QBs=; b=QaD91XhhsiKJcKZjI7E0SHVrAGc5YLsNW0jkmMq4iJPa22/5VKYfEncwLS2ojjlehL zbVbX6l/9tar0ovda/ABjp4RDUIsd1hOb/FHD/zoRkSKznAS2CL+cXQm882C/ck4Wn4m 3zhVAk7fAxs9AvhszIkMV0T9YuQ9UD50O+ONRFGpSEOyxD/JAJuqJZStRK7JznmtYIRM aKb0ZxPH1lJnJ+VQYk0YV2xwq6v9QEcuvnvXz3Mhpxm8cufdFZpqWNpL5c1oVT5MsIGF R/LhDU4fZZ1bfnQK/hlGKwvtjEgu1+tQLA6jv/VMs876pUTUfO5sRrQRqJ/3v9NckqBs fRMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1702569479; x=1703174279; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=ofzx+PtD9NebQ35OrFIzznCWiX3/O4uzvKbDVCH/QBs=; b=DD4z+ZthqLyUfWg4TMBhvE+1Z5uFjI0X9ClQnDMJYx39SaWEbCMEp71CFVVbLCqo5V cM4pUsuPtz6EhZTdQcjYkqQZebmkygmfR7nZk2unwcGX3KkjjInSu/UmOaBuCOeZF2ph mhvvcWS5uQQoVeaA7x/st8HqaEoIkjKZpP1jm/2suGDJqn0xVmKpaXcqYRxETCf57o5e 4zHeOFwKhFeuRgZtoXC4RcS2RgsosPH7QKbEBIdhENxSeGkYcXSD0sxPG0Tzx0JqZl5A ZrhaS2TiwXlfWcIlvPgxXjebRjKA+yS9oRF/rRoISMQyl1xRubd3qek929IR50VKNeiL 6kaQ== X-Gm-Message-State: AOJu0YwiaQlI3x1kAQxg1aMhOiQcI1LX5VXyAAehY0nm7ZTUg6NU8ZMq gR9tQOxQ8yXG0YcSXluR+Ft69xcdDpfbcND6l89ffHs4JSeojZcDz9yenttodONb6vfeQA7KtLf PpW3d5i7pHfK1eZscn5b9tcW0E1t3eVU2Nzh0vnp0igtKjQATHSQStr+P7Jwwg09/+NK0LFGJ5s tpoaCh1sL8dRHY X-Google-Smtp-Source: AGHT+IE8hdgvVVSU8e7X0o4TnMpNNfqHCE+eG8XEWM2JNwTjlan4ICN60f7qYrnQkQLVncfEYyGP0g== X-Received: by 2002:a17:902:e852:b0:1d0:6ffd:ceaf with SMTP id t18-20020a170902e85200b001d06ffdceafmr6714006plg.104.1702569479041; Thu, 14 Dec 2023 07:57:59 -0800 (PST) Received: from hsinchu26.internal.sifive.com (59-124-168-89.hinet-ip.hinet.net. [59.124.168.89]) by smtp.gmail.com with ESMTPSA id f4-20020a170902e98400b001d35223d0besm3320799plb.251.2023.12.14.07.57.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 14 Dec 2023 07:57:58 -0800 (PST) From: Andy Chiu To: linux-riscv@lists.infradead.org, palmer@dabbelt.com Cc: greentime.hu@sifive.com, guoren@linux.alibaba.com, bjorn@kernel.org, charlie@rivosinc.com, ardb@kernel.org, arnd@arndb.de, Andy Chiu , Paul Walmsley , Albert Ou Subject: [v5, 0/6] riscv: support kernel-mode Vector Date: Thu, 14 Dec 2023 15:57:15 +0000 Message-Id: <20231214155721.1753-1-andy.chiu@sifive.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231214_075804_249992_7558179B X-CRM114-Status: GOOD ( 14.64 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org This series provides support for running Vector code in kernel mode. Along with the suport, we add some Vector optimized routines. And provide a simple threshold to decide when to run the vectorized functions. We decided to split the patch series into 2 parts. The first part contains running kernel-mode Vector with preemption disabled. The second part will add support for running Vector with preemption. This patch is tested on a QEMU with V and verified that booting, normal userspace operations all work as usual with thresholds set to 0. v3 of this series can be found at [1] Link: https://lore.kernel.org/all/20231019154552.23351-1-andy.chiu@sifive.com/ Patch summary (on current order): - Updated patches: 1, 4 - New patch: 2, 5, 6 - Unchanged patch: 3 - Deleted patch: Patches related to preempt_v will be sent separately Changelog v5: - Rebase on top of riscv for-next (6.7-rc1) Changelog v4: - Use kernel_v_flags and helpers to track vector context. - Prevent softirq from nesting V context for non-preempt V - Add user copy and mem* routines Changelog v3: - Rebase on top of riscv for-next (6.6-rc1) - Fix a build issue (Conor) - Guard vstate_save, vstate_restore with {get,put}_cpu_vector_context. - Save V context after disabling preemption. (Guo) - Remove irqs_disabled() check from may_use_simd(). (Björn) - Comment about nesting V context. Changelog v2: - fix build issues - Follow arm's way of starting kernel-mode simd code: - add include/asm/simd.h and rename may_use_vector() -> may_use_simd() - return void in kernel_vector_begin(), and BUG_ON if may_use_simd() fails - Change naming scheme for functions/macros (Conor): - remove KMV - 's/rvv/vector/' - 's/RISCV_ISA_V_PREEMPTIVE_KMV/RISCV_ISA_V_PREEMPTIVE/' - 's/TIF_RISCV_V_KMV/TIF_RISCV_V_KERNEL_MODE/' Andy Chiu (4): riscv: vector: make Vector always available for softirq context riscv: sched: defer restoring Vector context for user riscv: lib: vectorize copy_to_user/copy_from_user riscv: lib: add vectorized mem* routines Greentime Hu (2): riscv: Add support for kernel mode vector riscv: Add vector extension XOR implementation *** BLURB HERE *** Andy Chiu (4): riscv: vector: make Vector always available for softirq context riscv: sched: defer restoring Vector context for user riscv: lib: vectorize copy_to_user/copy_from_user riscv: lib: add vectorized mem* routines Greentime Hu (2): riscv: Add support for kernel mode vector riscv: Add vector extension XOR implementation arch/riscv/include/asm/entry-common.h | 17 +++++ arch/riscv/include/asm/processor.h | 15 +++- arch/riscv/include/asm/simd.h | 46 +++++++++++ arch/riscv/include/asm/thread_info.h | 2 + arch/riscv/include/asm/vector.h | 32 +++++++- arch/riscv/include/asm/xor.h | 82 ++++++++++++++++++++ arch/riscv/kernel/Makefile | 1 + arch/riscv/kernel/kernel_mode_vector.c | 101 +++++++++++++++++++++++++ arch/riscv/kernel/process.c | 4 +- arch/riscv/kernel/ptrace.c | 5 +- arch/riscv/kernel/signal.c | 5 +- arch/riscv/kernel/vector.c | 2 +- arch/riscv/lib/Makefile | 6 ++ arch/riscv/lib/memcpy_vector.S | 29 +++++++ arch/riscv/lib/memmove_vector.S | 49 ++++++++++++ arch/riscv/lib/memset_vector.S | 33 ++++++++ arch/riscv/lib/riscv_v_helpers.c | 59 +++++++++++++++ arch/riscv/lib/uaccess.S | 11 +++ arch/riscv/lib/uaccess_vector.S | 55 ++++++++++++++ arch/riscv/lib/xor.S | 81 ++++++++++++++++++++ 20 files changed, 629 insertions(+), 6 deletions(-) create mode 100644 arch/riscv/include/asm/simd.h create mode 100644 arch/riscv/include/asm/xor.h create mode 100644 arch/riscv/kernel/kernel_mode_vector.c create mode 100644 arch/riscv/lib/memcpy_vector.S create mode 100644 arch/riscv/lib/memmove_vector.S create mode 100644 arch/riscv/lib/memset_vector.S create mode 100644 arch/riscv/lib/riscv_v_helpers.c create mode 100644 arch/riscv/lib/uaccess_vector.S create mode 100644 arch/riscv/lib/xor.S