From patchwork Thu Jun 13 19:16:10 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13697396 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9805FC27C75 for ; Thu, 13 Jun 2024 19:16:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=WB9HboDUX+V5X14JNR95hCgLBsnhKtbP0C0d5GMuOx0=; b=ZB38fN6HNBnQmz wmbXqitRx1WCOA8GTW+Q4i54pdYpjjvZOpFS7uTMkjfW9Tm5Q/CtAzKIFwJQwnMtLaktuetNReVYE AkBgVdjONl49vm8IFREPGQtyzp47GaZAaVMzjAicdgEQhMP/8WcCB5KISSZahGINtaqZpL3M8Vq22 Ciq0nWKmqJKFlF01oHgvIXtTlr/hPuIF/WczyskOrr4bSw6jDv0D/sk+tgNBGawmT7d2/yLQNhkpt iYjl7iu+lRcNdVqpxQnfN8qcUHos4ek0HBuQ6gWSjHUD6U8gjL0wfTcWBKktuFglWsorv6BML29ow IV/TQBe6wxiEo/YciVdg==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpwM-00000000E9K-01Su; Thu, 13 Jun 2024 19:16:54 +0000 Received: from mail-pl1-x62a.google.com ([2607:f8b0:4864:20::62a]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpwJ-00000000E7d-1Vf2 for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 19:16:53 +0000 Received: by mail-pl1-x62a.google.com with SMTP id d9443c01a7336-1f32a3b9491so13562505ad.0 for ; Thu, 13 Jun 2024 12:16:50 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1718306210; x=1718911010; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=zG2srFmoIcbdvzr37LUHEYY5+MFhpHwRfzd3fbsl58g=; b=xzN2/vfd5fg9ysbf2zf6tZyfbRyvTHUPwleZQE3Xqxkv9Myho9VVKFTM+yTpkvWKIV wYuub2BcWpHl4HvJiFcaA8wimIPfbXQKmEV5K+CvwlOIeT0usYS8pnqAx/vxNDEnGl+R YPjea7YkX7HobebazIHSyN8mYYGJFew9xVpWDYtbV/bL4vh9pzSjDbFCRJsR2vcCPUYr 9Q+QeMfEyeafnMjWAgk9EckxW9NZDqEgtnqN2/+yOoSzsdi0gr2wZQAOcjhDtQuikqbk mopPXgB3hlSsNPr86hFe/QKkCzQ2n/A0yqDOPJCpCLC5Dki7hO4YgD7HvMfD6mucB9zY oICA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718306210; x=1718911010; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=zG2srFmoIcbdvzr37LUHEYY5+MFhpHwRfzd3fbsl58g=; b=FNxFDGIJodl1Jq5CHbWRIknvdoYIHdBvqtCkETlnwegVccKzVwlW4Doha6XH+dLso/ UMD3YwIMRxQSThp8Q9y8nukAHB2RsnXVWdof8SmqZnlqR4j316JfOkhd1kcb3xBaeudw NNSg1P3okV7nKO5Kt9cYFhZaH3Hc7UOuf1Eh9NlKEz0SSM6+YpRIYrWAf9sU55Lbn6Kw n5lBp2xHBszzoRC/yLWDqMeyBixAR2LrekLufvooCMoIl89RrH7Tho1mXcX4dSDOaVgH og32lwwkjzOvLuZnxNwUrnWrnb852JK0NezQTwu1NNwfCyuAdYAqR6TkUDj65Miqn6RK zRpg== X-Gm-Message-State: AOJu0Yw1MmDw/efxplLV2m7nMChzRpVetAfvTULOujSZ96hrIJ3B6//S C84bGWdFNBDFYWjKjNT+RyT8mne5m4PsK+MbvtoTItRJOZBVClQTYjEf9MrVMsTIPnDbf0O3nxJ Dzpo= X-Google-Smtp-Source: AGHT+IGuN/l/5zfVYi7tKtQXsElXN0btBKGS7+gTuf5SnTUGY1GzvbrK3u85QYPF+7gnQUxWOLcnuQ== X-Received: by 2002:a17:902:ea07:b0:1f3:266b:ea23 with SMTP id d9443c01a7336-1f8625c1641mr6730485ad.13.1718306209838; Thu, 13 Jun 2024 12:16:49 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e7ca78sm17471015ad.106.2024.06.13.12.16.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 12:16:49 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v2 1/6] RISC-V: Add Zicclsm to cpufeature and hwprobe Date: Thu, 13 Jun 2024 15:16:10 -0400 Message-ID: <20240613191616.2101821-2-jesse@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240613191616.2101821-1-jesse@rivosinc.com> References: <20240613191616.2101821-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_121651_487682_897B6F34 X-CRM114-Status: GOOD ( 17.92 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org > Zicclsm Misaligned loads and stores to main memory regions with both > the cacheability and coherence PMAs must be supported. > Note: > This introduces a new extension name for this feature. > This requires misaligned support for all regular load and store > instructions (including scalar and vector) but not AMOs or other > specialized forms of memory access. Even though mandated, misaligned > loads and stores might execute extremely slowly. Standard software > distributions should assume their existence only for correctness, > not for performance. Detecing zicclsm allows the kernel to report if the hardware supports misaligned accesses even if support wasn't probed. This is useful for usermode to know if vector misaligned accesses are supported. Signed-off-by: Jesse Taube Reviewed-by: Conor Dooley Reviewed-by: Andy Chiu --- V1 -> V2: - Add documentation for Zicclsm - Move Zicclsm to correct location --- Documentation/arch/riscv/hwprobe.rst | 3 +++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/uapi/asm/hwprobe.h | 1 + arch/riscv/kernel/cpufeature.c | 1 + arch/riscv/kernel/sys_hwprobe.c | 1 + 5 files changed, 7 insertions(+) diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst index df5045103e73..7085a694b801 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -207,6 +207,9 @@ The following keys are defined: * :c:macro:`RISCV_HWPROBE_EXT_ZVE64D`: The Vector sub-extension Zve64d is supported, as defined by version 1.0 of the RISC-V Vector extension manual. + * :c:macro:`RISCV_HWPROBE_EXT_ZICCLSM`: The Zicclsm extension is supported as + defined in the RISC-V RVA Profiles Specification. + * :c:macro:`RISCV_HWPROBE_KEY_CPUPERF_0`: Deprecated. Returns similar values to :c:macro:`RISCV_HWPROBE_KEY_MISALIGNED_PERF`, but the key was mistakenly classified as a bitmask rather than a value. diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index f64d4e98e67c..0b3bd8885a2b 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -86,6 +86,7 @@ #define RISCV_ISA_EXT_ZVE64X 77 #define RISCV_ISA_EXT_ZVE64F 78 #define RISCV_ISA_EXT_ZVE64D 79 +#define RISCV_ISA_EXT_ZICCLSM 80 #define RISCV_ISA_EXT_XLINUXENVCFG 127 diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h index 2fb8a8185e7a..023b7771d1b7 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -65,6 +65,7 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_EXT_ZVE64X (1ULL << 39) #define RISCV_HWPROBE_EXT_ZVE64F (1ULL << 40) #define RISCV_HWPROBE_EXT_ZVE64D (1ULL << 41) +#define RISCV_HWPROBE_EXT_ZICCLSM (1ULL << 42) #define RISCV_HWPROBE_KEY_CPUPERF_0 5 #define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0) #define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index 1d6e4fda00f8..83c5ae16ad5e 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -283,6 +283,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = { __RISCV_ISA_EXT_DATA(h, RISCV_ISA_EXT_h), __RISCV_ISA_EXT_SUPERSET(zicbom, RISCV_ISA_EXT_ZICBOM, riscv_xlinuxenvcfg_exts), __RISCV_ISA_EXT_SUPERSET(zicboz, RISCV_ISA_EXT_ZICBOZ, riscv_xlinuxenvcfg_exts), + __RISCV_ISA_EXT_DATA(zicclsm, RISCV_ISA_EXT_ZICCLSM), __RISCV_ISA_EXT_DATA(zicntr, RISCV_ISA_EXT_ZICNTR), __RISCV_ISA_EXT_DATA(zicond, RISCV_ISA_EXT_ZICOND), __RISCV_ISA_EXT_DATA(zicsr, RISCV_ISA_EXT_ZICSR), diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index e4ec9166339f..e910e2971984 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -96,6 +96,7 @@ static void hwprobe_isa_ext0(struct riscv_hwprobe *pair, EXT_KEY(ZBB); EXT_KEY(ZBS); EXT_KEY(ZICBOZ); + EXT_KEY(ZICCLSM); EXT_KEY(ZBC); EXT_KEY(ZBKB); From patchwork Thu Jun 13 19:16:11 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13697397 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 7EC5FC27C75 for ; Thu, 13 Jun 2024 19:17:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=p1YNacTnUiLnB2cnhRAP4c3yQJUlbpI7q+gd+rzG21E=; b=yPbzh7A5Nv/03/ Ee6QMeoNqd2GKTca1JC/8bwK6zbKpErRhed5JgCwBVvjpCEZzwCcPFibtht2vnIoUBgmlmKW32R4Y VlAKtO8K4AMXbr02qFJMFO5Kmi9A8ixIBjnVSrzJP9za/RTTEsDcWug9yd4FlkEAMf3n8HiV/UJR7 cQvppxraSwHodXmsQ07pbzUGq7ERKOqvvHrOIDhwYsvXSGLJB/WCrRrK7/LqocmahgKulh3FeArH0 V9PN1+BPTdkVszLYfzyDGj9Q2s+x/2t2oRAgszn2YvhESS4zqJiwIyIo8+wny6W9WxU/txdVV+y8D E/QTrjJ6d080nm8F3KAQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpwa-00000000EDz-1JO2; Thu, 13 Jun 2024 19:17:08 +0000 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpwX-00000000ECy-2RMM for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 19:17:06 +0000 Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1f717608231so11053145ad.2 for ; Thu, 13 Jun 2024 12:17:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1718306224; x=1718911024; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=qpE2Ww+0dze0r9EN+Ae4oSiwfKGmmGKW1FHSshUbIM8=; b=OY7lPKmijKla4x4kRduFKdBr/DWGGJncaAUa1G4FsF1kgWz0siRJtydX54gpfXJSV/ 7L17VV/8QKmOXUQ7Xp+OptNDm0Xplji08+uW3qK67vO2aIinrmiNyp5BKpDzVLxASMKF CNkTJJjTZYR6vfr2eqTM9LxYxwDfXOaV/Ux+rUniifkBaUCfinpvHnVzqtz8rz5JCWGJ e6l3iVQ0ZjHE+Kx6UXU5H5VahQ1pMK5CAOhjoHG12y1a31jbor3+5V+AzaPEZCcXapUe nEMcDxbSAED8vtLNvKTvTFbfz8EO2L18AEmsRs+CQXLRBReWmzmLWKJQiTzmxC1Krdr1 UN0g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718306224; x=1718911024; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=qpE2Ww+0dze0r9EN+Ae4oSiwfKGmmGKW1FHSshUbIM8=; b=lkhoxQcVrgmUChqm45aGmEjSTAZJTKDetJ6IYBxI1U1WR2A468J5/5uKTdP2+AzEY1 xFRH0hrgxWUY/AW4wsAzKA3xqT7C/bBvPRIyQJkk1kTzkAK6uwBN2CfW+QgYwad7u6Ot 8IOHMXQgj4yEPLKL/T5nHFNOh1FgAC79EAoeEtoCYUMtrzXHb54lZVxxpINRfDIRVFSm +n1wrSr1vw1IxhOJTM7EzpTSA/LqnHSCqrzjfEofB57mcMZIw/njnveSroVAmXeN+o50 UeD5HT9AYWCv7EFAko6ghDg2MYu7panU44mxR6j5Whjo5Tdtv8icNDrJRF8MeAWIiDZU so7A== X-Gm-Message-State: AOJu0YzlEshnqnH9ORJBET8zoaCS1P+KvQ4kS8AZgA0oaAqlm81U2L+p 73rc7/bgIHsaMW6+eXe5SdqEA5f0wKf2b9cIE4iFrLIYEsTnjdF9eOtGSV7tKmF7JTmHj3vvEn9 mg+k= X-Google-Smtp-Source: AGHT+IEBHUiOGS6vq7hxV351Pocq5uf+7spoaKHpPHfr2Yp/P2zIdVv4yVxvGq5/azSD93awMnHT+g== X-Received: by 2002:a17:903:228f:b0:1f7:317f:5454 with SMTP id d9443c01a7336-1f862804bb9mr8516685ad.51.1718306224484; Thu, 13 Jun 2024 12:17:04 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e7ca78sm17471015ad.106.2024.06.13.12.16.50 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 12:17:04 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v2 2/6] dt-bindings: riscv: Add Zicclsm ISA extension description. Date: Thu, 13 Jun 2024 15:16:11 -0400 Message-ID: <20240613191616.2101821-3-jesse@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240613191616.2101821-1-jesse@rivosinc.com> References: <20240613191616.2101821-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_121705_648193_8C5B47D4 X-CRM114-Status: UNSURE ( 9.80 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Add description for Zicclsm ISA extension. Signed-off-by: Jesse Taube Acked-by: Conor Dooley --- V1 -> V2: - New patch --- Documentation/devicetree/bindings/riscv/extensions.yaml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Documentation/devicetree/bindings/riscv/extensions.yaml b/Documentation/devicetree/bindings/riscv/extensions.yaml index cfed80ad5540..9f6aae1f5b65 100644 --- a/Documentation/devicetree/bindings/riscv/extensions.yaml +++ b/Documentation/devicetree/bindings/riscv/extensions.yaml @@ -317,6 +317,13 @@ properties: The standard Zicboz extension for cache-block zeroing as ratified in commit 3dd606f ("Create cmobase-v1.0.pdf") of riscv-CMOs. + - const: zicclsm + description: + The standard Zicclsm extension for misaligned support for all regular + load and store instructions (including scalar and vector) but not AMOs + or other specialized forms of memory access. Defined in the + RISC-V RVA Profiles Specification. + - const: zicntr description: The standard Zicntr extension for base counters and timers, as From patchwork Thu Jun 13 19:16:12 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13697398 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id A677FC27C4F for ; Thu, 13 Jun 2024 19:17:26 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=OHaYnfBE4HV1Vz2CFv7xEogzBAYmB3cYCkKL/d/3f+I=; b=zWHP1+YCWmszCG wsVbUgcrPyIf7lYtGNvU1niCUzT548g2zV5zysTqh84FVxtUFCAOFFMAlnpYYE2L24koTJxJTN2tv yeXFHTBzfeZJqOAwRFTV8LLszWWwIMahrVRp0rWUPauF+Jvl83cAcsAqO+hPTSF9qXC/7F4fRUKIo b2O/094pdhVh5frihpniRHCZTx2MPWwfKVKx/9e6VZ0I8tHzo1GsZ20UhuUIyVUO6DsMia+6Vh0d7 z4j0Aq8U4Zmdb9Dkpcc1am3v15wZc1rLT/9MSxP2qMgRv2L7GTXMAFpqFF+npZgKsc++d/B+GIA73 Cfad8rDBXh7U1kwuohew==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpwo-00000000EL7-3kra; Thu, 13 Jun 2024 19:17:22 +0000 Received: from mail-pl1-x62e.google.com ([2607:f8b0:4864:20::62e]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpwm-00000000EJA-12ak for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 19:17:22 +0000 Received: by mail-pl1-x62e.google.com with SMTP id d9443c01a7336-1f862f7c7edso1460035ad.3 for ; Thu, 13 Jun 2024 12:17:19 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1718306239; x=1718911039; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=d2si2x/qxMiiRgx/ptA73I3ih5KSBTyIVIweBGPWOt8=; b=o7o76VHMQ8USZnPVdWP2owyMfhuIReoypEZE/jRvj9vzIujZyKhzDxhh7cw3hCx6Yg 4hw8ZsPnBOcYX34KbJsWM/XqOpGUfqH01UKTzmFhAfPGvaPiKVaOFi799nbR/7fMlZZC FuqdvLyzzx9DYguiC1/qcsi7kyNrU6M6mKGKph62A05FbEAR7z3/TadRalm+LETtNgeh b7tcY4OI+Jy1gBJhh6z1kDaD+qBPXTNwbyCKxXnuOzsz7sYZyFUa04LCP1RgySNXzMwj wILOsTkDAHN+go5XzpVwd9DFBxLJcZUFngYkFE3jTha5eb+XeLW4frPTaRZmHDDgNvR3 BRGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718306239; x=1718911039; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=d2si2x/qxMiiRgx/ptA73I3ih5KSBTyIVIweBGPWOt8=; b=hzqrTMnHqy6rC0x+WeUvbkzOhX4686N8zEgEE1zeQSwVqoRrBvRz9YGAcS5AiQKkLP XnmNagvXBbcmJU/N5RxtLts7IWwl642CeSU1Hg32I4MgOtbLZf0Y+pB4qdrDNIpv5nZO WZPrzrIUPjYBmePjsaNCi9DXIu0W4hQ/f4NqssslMLumdDQCMGBo+taTOVVlIZyqKkhe ZTrw7WxaTgfrZ3xTQX5NwGfwiOFCUenTJM4TXuMp9ruEJQSi+ogl0PQYgf6N8x8jV33Z LjWSjempb8VUY95K+hZ8YGNx+6amwlLzNaGEee32eTIDzcLDCKWslnRDeeJk27/p9xLb dKNA== X-Gm-Message-State: AOJu0YyJEoj7VYe6mkA/S+doRydlNZzxSOFhfNMru0zbvUtLTMcCuO05 c34fT96RxChXGUMbXkuYwXMRcgIOsQFSI1itGgb7HNn3QuBdhhEnWRCEIrgOikM6ndDTNtAWya2 seNk= X-Google-Smtp-Source: AGHT+IEe/pgm4wbdzYk9w01vIQjdm01pqBBmuWzC0VnQ7Vhdd+iuX8Zf6okO8XfjDwZx3EhOCo7asg== X-Received: by 2002:a17:903:2341:b0:1f6:7ee8:8935 with SMTP id d9443c01a7336-1f8629fef3bmr7650085ad.59.1718306239110; Thu, 13 Jun 2024 12:17:19 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e7ca78sm17471015ad.106.2024.06.13.12.17.05 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 12:17:18 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v2 3/6] RISC-V: Check scalar unaligned access on all CPUs Date: Thu, 13 Jun 2024 15:16:12 -0400 Message-ID: <20240613191616.2101821-4-jesse@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240613191616.2101821-1-jesse@rivosinc.com> References: <20240613191616.2101821-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_121720_319662_D2DE7597 X-CRM114-Status: GOOD ( 16.77 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Originally, the check_unaligned_access_emulated_all_cpus function only checked the boot hart. This fixes the function to check all harts. Check for Zicclsm before checking for unaligned access. This will greatly reduce the boot up time as finding the access speed is no longer necessary. Signed-off-by: Jesse Taube --- V1 -> V2: - New patch --- arch/riscv/kernel/traps_misaligned.c | 23 ++++++---------------- arch/riscv/kernel/unaligned_access_speed.c | 23 +++++++++++++--------- 2 files changed, 20 insertions(+), 26 deletions(-) diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps_misaligned.c index b62d5a2f4541..8fadbe00dd62 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -526,31 +526,17 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; } -static bool check_unaligned_access_emulated(int cpu) +static void check_unaligned_access_emulated(struct work_struct *unused) { + int cpu = smp_processor_id(); long *mas_ptr = per_cpu_ptr(&misaligned_access_speed, cpu); unsigned long tmp_var, tmp_val; - bool misaligned_emu_detected; *mas_ptr = RISCV_HWPROBE_MISALIGNED_UNKNOWN; __asm__ __volatile__ ( " "REG_L" %[tmp], 1(%[ptr])\n" : [tmp] "=r" (tmp_val) : [ptr] "r" (&tmp_var) : "memory"); - - misaligned_emu_detected = (*mas_ptr == RISCV_HWPROBE_MISALIGNED_EMULATED); - /* - * If unaligned_ctl is already set, this means that we detected that all - * CPUS uses emulated misaligned access at boot time. If that changed - * when hotplugging the new cpu, this is something we don't handle. - */ - if (unlikely(unaligned_ctl && !misaligned_emu_detected)) { - pr_crit("CPU misaligned accesses non homogeneous (expected all emulated)\n"); - while (true) - cpu_relax(); - } - - return misaligned_emu_detected; } bool check_unaligned_access_emulated_all_cpus(void) @@ -562,8 +548,11 @@ bool check_unaligned_access_emulated_all_cpus(void) * accesses emulated since tasks requesting such control can run on any * CPU. */ + schedule_on_each_cpu(check_unaligned_access_emulated); + for_each_online_cpu(cpu) - if (!check_unaligned_access_emulated(cpu)) + if (per_cpu(misaligned_access_speed, cpu) + != RISCV_HWPROBE_MISALIGNED_EMULATED) return false; unaligned_ctl = true; diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel/unaligned_access_speed.c index a9a6bcb02acf..70c1588fc353 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -259,23 +259,28 @@ static int check_unaligned_access_speed_all_cpus(void) kfree(bufs); return 0; } +#endif /* CONFIG_RISCV_PROBE_UNALIGNED_ACCESS */ static int check_unaligned_access_all_cpus(void) { - bool all_cpus_emulated = check_unaligned_access_emulated_all_cpus(); + bool all_cpus_emulated; + int cpu; + if (riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICCLSM)) { + for_each_online_cpu(cpu) { + per_cpu(misaligned_access_speed, cpu) = RISCV_HWPROBE_MISALIGNED_FAST; + } + return 0; + } + + all_cpus_emulated = check_unaligned_access_emulated_all_cpus(); + +#ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS if (!all_cpus_emulated) return check_unaligned_access_speed_all_cpus(); +#endif return 0; } -#else /* CONFIG_RISCV_PROBE_UNALIGNED_ACCESS */ -static int check_unaligned_access_all_cpus(void) -{ - check_unaligned_access_emulated_all_cpus(); - - return 0; -} -#endif arch_initcall(check_unaligned_access_all_cpus); From patchwork Thu Jun 13 19:16:13 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13697399 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 4D1FCC27C75 for ; Thu, 13 Jun 2024 19:17:42 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=uHCh2Ovo6ztMl1YIQx5GWLGF0U1MjE/u8jamIIwDIAQ=; b=xDjnEGdL7IOmXn c67/qDLtbzF3nYNSpXXvY8HXoEdOWl7llzwB28J4bJXCDkF80RvBppw9ffjkPejDT3Jz/RC6dlZWo q4PKZfsAlaYlDRWeYFd+GtvJso8utTnwc5vjoyzKijOgvuJrvFUhjsQGgDZiMmhe7Ys9hJNoBAJiD G+nogcgJpFba2H/8pc5GxlmqrIff/2FWPqpxPWz/p+69/F34fsFM1DRtKu1S8czshOfrhjoX1VAbv VLDkQm6Y0iNp8TGkofQzeCLXFJ0Jf5sr2/U2P9lU3mlaIUZxMPHnN6Gea5M+iNJZBMH1bZvgu8TC2 3dBn0mxkJoh1D9sa2HoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpx4-00000000EUf-1YVB; Thu, 13 Jun 2024 19:17:38 +0000 Received: from mail-pg1-x529.google.com ([2607:f8b0:4864:20::529]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpx1-00000000ET1-01Ts for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 19:17:36 +0000 Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-6ce533b643dso1038303a12.3 for ; Thu, 13 Jun 2024 12:17:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1718306254; x=1718911054; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=muXFJgsYoKCYgzsWRlHOJmTR9Kg/zixWqbAUEZb+IAk=; b=at9A0lhWE9kU+mnAPz/Cn09O4+nxcW90AVYnBAFSxJdzdkDoKFOx3/t1FH6TLdDJW7 VUmCx7nuglW1vdq7UuuZ3yhWhh1OiQto7hjHimmGtuIU3pYIjMRF8zxZFLdYM4wQNLY2 IqSCxhYih2TJoGHsPy6vNxODFil1majhycuJah6mqssDrKVWfk6sRzCX8CiRWVDIgGQ9 kcqcjU+rFeTsAM5K6KFfWApTGxKor2A7QspNhwyoNjY8dftDFO3cWDHDo8RT7D/cPS+A UXVx6aDll4GFcSmbFKXGKH/T8rTc8RF3/QrWdyyYZL5i2ywygeUc5YFut5A4dIt6yp1g SKGQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718306254; x=1718911054; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=muXFJgsYoKCYgzsWRlHOJmTR9Kg/zixWqbAUEZb+IAk=; b=UQPthoHtlf8ByvE/6NbsYWqsWqcRSATYf2fzU2sgGovububo7hUfUjy6VT7N2UOmip jSCQ5JSjr1YIgfTRWaQcAI7mAsKH0FZ7pwsdLAEYpTGY5KyUwFa4GFfGtSDtbOgclv1M 5Utnm7LESPAWOZXNt28UVVlISyo18Dpufhl+tvUtS4F6irZtO5qslNdVkaKltpvOAvUs BH6do6+fYJ6fEUtrpJl2OFL4uv7PxbO5qr3L7aIUzjgdihdsrGrgxWzCYYZu2rl3K40p RN0LCxqh41k33+1f1EbXEOPo6fQtlv4zdQH30AXo+UjhZnVts8Q/AMGLbpCE8l9cah5X GYqA== X-Gm-Message-State: AOJu0Yywn1bV0Q/9btaPPWOlX7My9QRkgELPl3dFHvIy1OR36iHJxGsJ 7qajk+60yFz7j/WrYMnXm7J8cxyeyuM8NYErlsCu3tMmZ9mThirUGNLssaKnLsUlYn0jtreOQpX 5uCU= X-Google-Smtp-Source: AGHT+IE5SiAOXuRQi90VkoN3+jqAIhtZnqDv6P9KtRxeggeK3SAZ6fJEsAO5NNsLNSZ2/jrdFBDNcg== X-Received: by 2002:a17:902:d2d0:b0:1f4:ac10:3ede with SMTP id d9443c01a7336-1f8627d1668mr5735685ad.21.1718306253854; Thu, 13 Jun 2024 12:17:33 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e7ca78sm17471015ad.106.2024.06.13.12.17.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 12:17:33 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v2 4/6] RISC-V: Detect unaligned vector accesses supported. Date: Thu, 13 Jun 2024 15:16:13 -0400 Message-ID: <20240613191616.2101821-5-jesse@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240613191616.2101821-1-jesse@rivosinc.com> References: <20240613191616.2101821-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_121735_131756_7599B99D X-CRM114-Status: GOOD ( 25.18 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Run a unaligned vector access to test if the system supports vector unaligned access. Add the result to a new key in hwprobe. This is useful for usermode to know if vector misaligned accesses are supported and if they are faster or slower than equivalent byte accesses. Signed-off-by: Jesse Taube --- V1 -> V2: - Add Kconfig options - Add insn_is_vector - Add handle_vector_misaligned_load - Fix build - Seperate vector from scalar misaligned access - This patch was almost completely rewritten --- arch/riscv/Kconfig | 41 +++++++ arch/riscv/include/asm/cpufeature.h | 7 +- arch/riscv/include/asm/entry-common.h | 11 -- arch/riscv/include/asm/hwprobe.h | 2 +- arch/riscv/include/asm/vector.h | 1 + arch/riscv/include/uapi/asm/hwprobe.h | 5 + arch/riscv/kernel/Makefile | 4 +- arch/riscv/kernel/sys_hwprobe.c | 41 +++++++ arch/riscv/kernel/traps_misaligned.c | 119 ++++++++++++++++++++- arch/riscv/kernel/unaligned_access_speed.c | 9 +- arch/riscv/kernel/vector.c | 2 +- 11 files changed, 221 insertions(+), 21 deletions(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index b94176e25be1..f12df0ca6c18 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -723,6 +723,12 @@ config RISCV_MISALIGNED help Embed support for emulating misaligned loads and stores. +config RISCV_VECTOR_MISALIGNED + bool + depends on RISCV_ISA_V + help + Enable detecting support for vector misaligned loads and stores. + choice prompt "Unaligned Accesses Support" default RISCV_PROBE_UNALIGNED_ACCESS @@ -774,6 +780,41 @@ config RISCV_EFFICIENT_UNALIGNED_ACCESS endchoice +choice + prompt "Vector unaligned Accesses Support" + depends on RISCV_ISA_V + default RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + help + This determines the level of support for vector unaligned accesses. This + information is used by the kernel to perform optimizations. It is also + exposed to user space via the hwprobe syscall. The hardware will be + probed at boot by default. + +config RISCV_DETECT_VECTOR_UNALIGNED_ACCESS + bool "Detect support for vector unaligned accesses" + select RISCV_VECTOR_MISALIGNED + help + During boot, the kernel will detect if the system supports vector + unaligned accesses. + +config RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + bool "Probe speed of vector unaligned accesses" + select RISCV_VECTOR_MISALIGNED + help + During boot, the kernel will run a series of tests to determine the + speed of vector unaligned accesses if they are supported. This probing + will dynamically determine the speed of vector unaligned accesses on + the underlying system if they are supported. + +config CONFIG_RISCV_UNALIGNED_ACCESS_UNSUPPORTED + bool "Assume the system does not support vector unaligned memory accesses" + help + Assume that the system does not support vector unaligned memory accesses. + The kernel and userspace programs may run them successfully on systems + that do support vector unaligned memory accesses. + +endchoice + endmenu # "Platform type" menu "Kernel features" diff --git a/arch/riscv/include/asm/cpufeature.h b/arch/riscv/include/asm/cpufeature.h index 347805446151..d0ea5921ab20 100644 --- a/arch/riscv/include/asm/cpufeature.h +++ b/arch/riscv/include/asm/cpufeature.h @@ -33,8 +33,8 @@ extern struct riscv_isainfo hart_isa[NR_CPUS]; void riscv_user_isa_enable(void); -#if defined(CONFIG_RISCV_MISALIGNED) bool check_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_MISALIGNED) void unaligned_emulation_finish(void); bool unaligned_ctl_available(void); DECLARE_PER_CPU(long, misaligned_access_speed); @@ -45,6 +45,11 @@ static inline bool unaligned_ctl_available(void) } #endif +bool check_vector_unaligned_access_emulated_all_cpus(void); +#if defined(CONFIG_RISCV_VECTOR_MISALIGNED) +DECLARE_PER_CPU(long, vector_misaligned_access); +#endif + #if defined(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) DECLARE_STATIC_KEY_FALSE(fast_unaligned_access_speed_key); diff --git a/arch/riscv/include/asm/entry-common.h b/arch/riscv/include/asm/entry-common.h index 2293e535f865..7b32d2b08bb6 100644 --- a/arch/riscv/include/asm/entry-common.h +++ b/arch/riscv/include/asm/entry-common.h @@ -25,18 +25,7 @@ static inline void arch_exit_to_user_mode_prepare(struct pt_regs *regs, void handle_page_fault(struct pt_regs *regs); void handle_break(struct pt_regs *regs); -#ifdef CONFIG_RISCV_MISALIGNED int handle_misaligned_load(struct pt_regs *regs); int handle_misaligned_store(struct pt_regs *regs); -#else -static inline int handle_misaligned_load(struct pt_regs *regs) -{ - return -1; -} -static inline int handle_misaligned_store(struct pt_regs *regs) -{ - return -1; -} -#endif #endif /* _ASM_RISCV_ENTRY_COMMON_H */ diff --git a/arch/riscv/include/asm/hwprobe.h b/arch/riscv/include/asm/hwprobe.h index 150a9877b0af..ef01c182af2b 100644 --- a/arch/riscv/include/asm/hwprobe.h +++ b/arch/riscv/include/asm/hwprobe.h @@ -8,7 +8,7 @@ #include -#define RISCV_HWPROBE_MAX_KEY 7 +#define RISCV_HWPROBE_MAX_KEY 8 static inline bool riscv_hwprobe_key_is_valid(__s64 key) { diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index be7d309cca8a..99b0f91db9ee 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -21,6 +21,7 @@ extern unsigned long riscv_v_vsize; int riscv_v_setup_vsize(void); +bool insn_is_vector(u32 insn_buf); bool riscv_v_first_use_handler(struct pt_regs *regs); void kernel_vector_begin(void); void kernel_vector_end(void); diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h index 023b7771d1b7..2fee870e41bb 100644 --- a/arch/riscv/include/uapi/asm/hwprobe.h +++ b/arch/riscv/include/uapi/asm/hwprobe.h @@ -75,6 +75,11 @@ struct riscv_hwprobe { #define RISCV_HWPROBE_MISALIGNED_MASK (7 << 0) #define RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE 6 #define RISCV_HWPROBE_KEY_MISALIGNED_PERF 7 +#define RISCV_HWPROBE_KEY_VEC_MISALIGNED_PERF 8 +#define RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN 0 +#define RISCV_HWPROBE_VEC_MISALIGNED_SLOW 2 +#define RISCV_HWPROBE_VEC_MISALIGNED_FAST 3 +#define RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED 4 /* Increase RISCV_HWPROBE_MAX_KEY when adding items. */ /* Flags */ diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 5b243d46f4b1..62ac19c029f1 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -62,8 +62,8 @@ obj-y += probes/ obj-y += tests/ obj-$(CONFIG_MMU) += vdso.o vdso/ -obj-$(CONFIG_RISCV_MISALIGNED) += traps_misaligned.o -obj-$(CONFIG_RISCV_MISALIGNED) += unaligned_access_speed.o +obj-y += traps_misaligned.o +obj-y += unaligned_access_speed.o obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) += copy-unaligned.o obj-$(CONFIG_FPU) += fpu.o diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index e910e2971984..c40df314058b 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -194,6 +194,43 @@ static u64 hwprobe_misaligned(const struct cpumask *cpus) } #endif +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + int cpu; + u64 perf = -1ULL; + + if (IS_ENABLED(CONFIG_RISCV_UNALIGNED_ACCESS_UNSUPPORTED)) + return RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; + + /* Return if supported or not even if speed wasn't probed */ + for_each_cpu(cpu, cpus) { + int this_perf = per_cpu(vector_misaligned_access, cpu); + + if (perf == -1ULL) + perf = this_perf; + + if (perf != this_perf) { + perf = RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; + break; + } + } + + if (perf == -1ULL) + return RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; + + return perf; +} +#else +static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) +{ + if (IS_ENABLED(CONFIG_RISCV_UNALIGNED_ACCESS_UNSUPPORTED)) + return RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; + + return RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; +} +#endif + static void hwprobe_one_pair(struct riscv_hwprobe *pair, const struct cpumask *cpus) { @@ -222,6 +259,10 @@ static void hwprobe_one_pair(struct riscv_hwprobe *pair, pair->value = hwprobe_misaligned(cpus); break; + case RISCV_HWPROBE_KEY_VEC_MISALIGNED_PERF: + pair->value = hwprobe_vec_misaligned(cpus); + break; + case RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE: pair->value = 0; if (hwprobe_ext0_has(cpus, RISCV_HWPROBE_EXT_ZICBOZ)) diff --git a/arch/riscv/kernel/traps_misaligned.c b/arch/riscv/kernel/traps_misaligned.c index 8fadbe00dd62..6f0264a8c9de 100644 --- a/arch/riscv/kernel/traps_misaligned.c +++ b/arch/riscv/kernel/traps_misaligned.c @@ -16,6 +16,7 @@ #include #include #include +#include #define INSN_MATCH_LB 0x3 #define INSN_MASK_LB 0x707f @@ -322,12 +323,37 @@ union reg_data { u64 data_u64; }; -static bool unaligned_ctl __read_mostly; - /* sysctl hooks */ int unaligned_enabled __read_mostly = 1; /* Enabled by default */ -int handle_misaligned_load(struct pt_regs *regs) +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc = regs->epc; + unsigned long insn; + + if (get_insn(regs, epc, &insn)) + return -1; + + /* Only return 0 when in check_vector_unaligned_access_emulated */ + if (*this_cpu_ptr(&vector_misaligned_access) == RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN) { + *this_cpu_ptr(&vector_misaligned_access) = RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; + regs->epc = epc + INSN_LEN(insn); + return 0; + } + + /* If vector instruction we don't emulate it yet */ + regs->epc = epc; + return -1; +} +#else +static int handle_vector_misaligned_load(struct pt_regs *regs) +{ + return -1; +} +#endif + +static int handle_scalar_misaligned_load(struct pt_regs *regs) { union reg_data val; unsigned long epc = regs->epc; @@ -435,7 +461,7 @@ int handle_misaligned_load(struct pt_regs *regs) return 0; } -int handle_misaligned_store(struct pt_regs *regs) +static int handle_scalar_misaligned_store(struct pt_regs *regs) { union reg_data val; unsigned long epc = regs->epc; @@ -526,6 +552,85 @@ int handle_misaligned_store(struct pt_regs *regs) return 0; } +int handle_misaligned_load(struct pt_regs *regs) +{ + unsigned long epc = regs->epc; + unsigned long insn; + + if (IS_ENABLED(CONFIG_RISCV_VECTOR_MISALIGNED)) { + if (get_insn(regs, epc, &insn)) + return -1; + + if (insn_is_vector(insn)) + return handle_vector_misaligned_load(regs); + } + + if (IS_ENABLED(CONFIG_RISCV_MISALIGNED)) + return handle_scalar_misaligned_load(regs); + + return -1; +} + +int handle_misaligned_store(struct pt_regs *regs) +{ + if (IS_ENABLED(CONFIG_RISCV_MISALIGNED)) + return handle_scalar_misaligned_store(regs); + + return -1; +} + +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED +static void check_vector_unaligned_access_emulated(struct work_struct *unused) +{ + long *mas_ptr = this_cpu_ptr(&vector_misaligned_access); + unsigned long tmp_var; + + *mas_ptr = RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN; + + local_irq_enable(); + kernel_vector_begin(); + __asm__ __volatile__ ( + ".balign 4\n\t" + ".option push\n\t" + ".option arch, +zve32x\n\t" + " vsetivli zero, 1, e16, m1, ta, ma\n\t" // Vectors of 16b + " vle16.v v0, (%[ptr])\n\t" // Load bytes + ".option pop\n\t" + : : [ptr] "r" ((u8 *)&tmp_var + 1) : "v0"); + kernel_vector_end(); +} + +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + int cpu; + bool ret = true; + + if (!has_vector()) { + for_each_online_cpu(cpu) + per_cpu(vector_misaligned_access, cpu) = RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; + return false; + } + + schedule_on_each_cpu(check_vector_unaligned_access_emulated); + + for_each_online_cpu(cpu) + if (per_cpu(vector_misaligned_access, cpu) + != RISCV_HWPROBE_VEC_MISALIGNED_SLOW) + return false; + + return ret; +} +#else +bool check_vector_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif + +#ifdef CONFIG_RISCV_MISALIGNED + +static bool unaligned_ctl __read_mostly; + static void check_unaligned_access_emulated(struct work_struct *unused) { int cpu = smp_processor_id(); @@ -563,3 +668,9 @@ bool unaligned_ctl_available(void) { return unaligned_ctl; } +#else +bool check_unaligned_access_emulated_all_cpus(void) +{ + return false; +} +#endif diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel/unaligned_access_speed.c index 70c1588fc353..c6106bd4a25a 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -19,7 +19,8 @@ #define MISALIGNED_BUFFER_ORDER get_order(MISALIGNED_BUFFER_SIZE) #define MISALIGNED_COPY_SIZE ((MISALIGNED_BUFFER_SIZE / 2) - 0x80) -DEFINE_PER_CPU(long, misaligned_access_speed); +DEFINE_PER_CPU(long, misaligned_access_speed) = RISCV_HWPROBE_MISALIGNED_UNKNOWN; +DEFINE_PER_CPU(long, vector_misaligned_access) = RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; #ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS static cpumask_t fast_misaligned_access; @@ -268,12 +269,18 @@ static int check_unaligned_access_all_cpus(void) if (riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICCLSM)) { for_each_online_cpu(cpu) { +#ifdef CONFIG_RISCV_VECTOR_MISALIGNED + per_cpu(vector_misaligned_access, cpu) = RISCV_HWPROBE_VEC_MISALIGNED_FAST; +#endif +#ifdef CONFIG_RISCV_MISALIGNED per_cpu(misaligned_access_speed, cpu) = RISCV_HWPROBE_MISALIGNED_FAST; +#endif } return 0; } all_cpus_emulated = check_unaligned_access_emulated_all_cpus(); + check_vector_unaligned_access_emulated_all_cpus(); #ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS if (!all_cpus_emulated) diff --git a/arch/riscv/kernel/vector.c b/arch/riscv/kernel/vector.c index 682b3feee451..821818886fab 100644 --- a/arch/riscv/kernel/vector.c +++ b/arch/riscv/kernel/vector.c @@ -66,7 +66,7 @@ void __init riscv_v_setup_ctx_cache(void) #endif } -static bool insn_is_vector(u32 insn_buf) +bool insn_is_vector(u32 insn_buf) { u32 opcode = insn_buf & __INSN_OPCODE_MASK; u32 width, csr; From patchwork Thu Jun 13 19:16:14 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13697400 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8B07AC27C4F for ; Thu, 13 Jun 2024 19:17:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=kFBNS7s1fXIOv1mzluoiwwhrEmMaXoN2L5CjRFl++/k=; b=OC0keNmfD21riK 4V7l35H07KiQqwf5FvqiXSFcIlXOegx2X6Cuh0k6ciOg3lA5cUrvA/KLsw2uUP6XmirXfu7B/WLYt vhU8ztUx9H2h8CjH8Mu990NhGPQhGvtFWwfAVBQfVDjI3gD5f0RkNcY8+bhMWCC04jZNvC8q8l5B8 5eD+3TSwmAO8b2srjiTmixRhhldHGWIS+UPwSWM1/tvR6eMeGIXWdpcNuaYVqZ7qgo7XVG5+Yiu0t U4LLLS8H+ZWgBEH3phQP9qOxk9JZScMrcLP4cj0tYHYCh0cSNQ9dssvDv5CEn5k8fTRJtaVVS7qJ6 J/IawEtZOLlv3o5dH1Zw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpxI-00000000Ef0-0p6N; Thu, 13 Jun 2024 19:17:52 +0000 Received: from mail-pg1-x535.google.com ([2607:f8b0:4864:20::535]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpxF-00000000EdH-3avR for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 19:17:51 +0000 Received: by mail-pg1-x535.google.com with SMTP id 41be03b00d2f7-652fd0bb5e6so1110701a12.0 for ; Thu, 13 Jun 2024 12:17:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1718306269; x=1718911069; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=l5Of0EkcykOsZIYMENcTbIOpaZnOyRzNH4j4//yvpcs=; b=ckWIXnMu9n/+mNVkC0gsHS7/kzohR39TNL5hphVBkdvPUfWQ238kWUwWZnbA3RxNFw u8YL+iX+mZldqwoBBsVlMPCagpmgOeR487/QYw/W9xSL1d7j1XwNjdfBW+nxM3lVuAI+ j7qnZhCO5046NPrv48ICz4FVuCKt9cOrErWvXb8UKz6F9YJ+eEZQ0h3SxefQry1gU09x Z/ZvaVoEzLgBejolczYgfA6c/rfaAjLWSDDdK1kbT6wSG175A72WTcCdsZGGaUloXgJv jtgni165kUCxMMrItUBuL7x02qF91xAqESWL+KNhm5OQxRThCQY/MwbFUZ+CuH/KimUP rRBg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718306269; x=1718911069; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=l5Of0EkcykOsZIYMENcTbIOpaZnOyRzNH4j4//yvpcs=; b=BCqfgBTVifMDcXD5wCal+9wVgYfh/4W6AU3bL0fWhTSBA8CtRfdSpUDrzzkD1+jR3P 4YQLlwg468TPlXw7hDlZy/WfGwh9eXrUjK2BY9yD5SzXbFCjLM1mIWoDrB/YY6mgVu8b jWuN0MmTwiRe2LKn9zCra10c2cS/hC4st7qp0LeQgNCT4ziTdO2+zQRBe72JIu8FO+x1 icyM1ht+YJNnXLLR7hDMI1ccS8MqBIsM2RxEDl2TIRoSaA3z8ncYkknad+auYzN9bHv8 aSBCVACvu7JQx9am3BH82OP/OQvLoIL8KeP19yNXp5MzZfjBwW/idWX+ua89EgmJRb8H GZ/g== X-Gm-Message-State: AOJu0Yw4D5+Tu6THR5Q7mBNESRjR0ryRBVsiHNJTNIbZucEf/V0fYGJi /nIiWAUBRTPb0kRK4uxFrWCXYNdiGGrcuFVE1MNR8NdT3mEebq3WvnLocVUnHldsH7MYInLZg06 q1h4= X-Google-Smtp-Source: AGHT+IFeYYnuwrF9NiGYw5EanheGBxXcQDznRNds6KUvN7djCnDAEnowkfw/KxTzTFdopU/Ia85E1Q== X-Received: by 2002:a17:902:e88a:b0:1f6:d368:7dd7 with SMTP id d9443c01a7336-1f862b155e2mr6566675ad.45.1718306268578; Thu, 13 Jun 2024 12:17:48 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e7ca78sm17471015ad.106.2024.06.13.12.17.34 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 12:17:48 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v2 5/6] RISC-V: Report vector unaligned access speed hwprobe Date: Thu, 13 Jun 2024 15:16:14 -0400 Message-ID: <20240613191616.2101821-6-jesse@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240613191616.2101821-1-jesse@rivosinc.com> References: <20240613191616.2101821-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_121749_931041_89BD7E1C X-CRM114-Status: GOOD ( 28.29 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Detect if vector misaligned accesses are faster or slower than equivalent vector byte accesses. This is useful for usermode to know whether vector byte accesses or vector misaligned accesses have a better bandwidth for operations like memcpy. Signed-off-by: Jesse Taube --- V1 -> V2: - Add Kconfig options - Add WORD_EEW to vec-copy-unaligned.S --- arch/riscv/Kconfig | 18 +++ arch/riscv/kernel/Makefile | 3 +- arch/riscv/kernel/copy-unaligned.h | 5 + arch/riscv/kernel/sys_hwprobe.c | 6 + arch/riscv/kernel/unaligned_access_speed.c | 130 ++++++++++++++++++++- arch/riscv/kernel/vec-copy-unaligned.S | 58 +++++++++ 6 files changed, 217 insertions(+), 3 deletions(-) create mode 100644 arch/riscv/kernel/vec-copy-unaligned.S diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index f12df0ca6c18..b181b35f37ca 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -813,6 +813,24 @@ config CONFIG_RISCV_UNALIGNED_ACCESS_UNSUPPORTED The kernel and userspace programs may run them successfully on systems that do support vector unaligned memory accesses. +config RISCV_SLOW_VEC_UNALIGNED_ACCESS + bool "Assume the system supports slow vector unaligned memory accesses" + depends on NONPORTABLE + help + Assume that the system supports slow vector unaligned memory accesses. The + kernel and userspace programs may not be able to run at all on systems + that do not support unaligned memory accesses. + +config RISCV_EFFICIENT_VEC_UNALIGNED_ACCESS + bool "Assume the system supports fast vector unaligned memory accesses" + depends on NONPORTABLE + help + Assume that the system supports fast vector unaligned memory accesses. When + enabled, this option improves the performance of the kernel on such + systems. However, the kernel and userspace programs will run much more + slowly, or will not be able to run at all, on systems that do not + support efficient unaligned memory accesses. + endchoice endmenu # "Platform type" diff --git a/arch/riscv/kernel/Makefile b/arch/riscv/kernel/Makefile index 62ac19c029f1..c4d6f54705a4 100644 --- a/arch/riscv/kernel/Makefile +++ b/arch/riscv/kernel/Makefile @@ -64,7 +64,8 @@ obj-$(CONFIG_MMU) += vdso.o vdso/ obj-y += traps_misaligned.o obj-y += unaligned_access_speed.o -obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) += copy-unaligned.o +obj-$(CONFIG_RISCV_PROBE_UNALIGNED_ACCESS) += copy-unaligned.o +obj-$(CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS) += vec-copy-unaligned.o obj-$(CONFIG_FPU) += fpu.o obj-$(CONFIG_FPU) += kernel_mode_fpu.o diff --git a/arch/riscv/kernel/copy-unaligned.h b/arch/riscv/kernel/copy-unaligned.h index e3d70d35b708..85d4d11450cb 100644 --- a/arch/riscv/kernel/copy-unaligned.h +++ b/arch/riscv/kernel/copy-unaligned.h @@ -10,4 +10,9 @@ void __riscv_copy_words_unaligned(void *dst, const void *src, size_t size); void __riscv_copy_bytes_unaligned(void *dst, const void *src, size_t size); +#ifdef CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS +void __riscv_copy_vec_words_unaligned(void *dst, const void *src, size_t size); +void __riscv_copy_vec_bytes_unaligned(void *dst, const void *src, size_t size); +#endif + #endif /* __RISCV_KERNEL_COPY_UNALIGNED_H */ diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c index c40df314058b..1679db4fe360 100644 --- a/arch/riscv/kernel/sys_hwprobe.c +++ b/arch/riscv/kernel/sys_hwprobe.c @@ -203,6 +203,12 @@ static u64 hwprobe_vec_misaligned(const struct cpumask *cpus) if (IS_ENABLED(CONFIG_RISCV_UNALIGNED_ACCESS_UNSUPPORTED)) return RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED; + if (IS_ENABLED(CONFIG_RISCV_EFFICIENT_VEC_UNALIGNED_ACCESS)) + return RISCV_HWPROBE_VEC_MISALIGNED_FAST; + + if (IS_ENABLED(CONFIG_RISCV_SLOW_VEC_UNALIGNED_ACCESS)) + return RISCV_HWPROBE_VEC_MISALIGNED_SLOW; + /* Return if supported or not even if speed wasn't probed */ for_each_cpu(cpu, cpus) { int this_perf = per_cpu(vector_misaligned_access, cpu); diff --git a/arch/riscv/kernel/unaligned_access_speed.c b/arch/riscv/kernel/unaligned_access_speed.c index c6106bd4a25a..ddc49cce54e3 100644 --- a/arch/riscv/kernel/unaligned_access_speed.c +++ b/arch/riscv/kernel/unaligned_access_speed.c @@ -8,9 +8,11 @@ #include #include #include +#include #include #include #include +#include #include "copy-unaligned.h" @@ -262,9 +264,126 @@ static int check_unaligned_access_speed_all_cpus(void) } #endif /* CONFIG_RISCV_PROBE_UNALIGNED_ACCESS */ +#ifdef CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS +static void check_vector_unaligned_access(struct work_struct *unused) +{ + int cpu = smp_processor_id(); + u64 start_cycles, end_cycles; + u64 word_cycles; + u64 byte_cycles; + int ratio; + unsigned long start_jiffies, now; + struct page *page; + void *dst; + void *src; + long speed = RISCV_HWPROBE_VEC_MISALIGNED_SLOW; + + if (per_cpu(vector_misaligned_access, cpu) != RISCV_HWPROBE_VEC_MISALIGNED_SLOW) + return; + + page = alloc_pages(GFP_KERNEL, MISALIGNED_BUFFER_ORDER); + if (!page) { + pr_warn("Allocation failure, not measuring vector misaligned performance\n"); + return; + } + + /* Make an unaligned destination buffer. */ + dst = (void *)((unsigned long)page_address(page) | 0x1); + /* Unalign src as well, but differently (off by 1 + 2 = 3). */ + src = dst + (MISALIGNED_BUFFER_SIZE / 2); + src += 2; + word_cycles = -1ULL; + + /* Do a warmup. */ + local_irq_enable(); + kernel_vector_begin(); + __riscv_copy_vec_words_unaligned(dst, src, MISALIGNED_COPY_SIZE); + + start_jiffies = jiffies; + while ((now = jiffies) == start_jiffies) + cpu_relax(); + + /* + * For a fixed amount of time, repeatedly try the function, and take + * the best time in cycles as the measurement. + */ + while (time_before(jiffies, now + (1 << MISALIGNED_ACCESS_JIFFIES_LG2))) { + start_cycles = get_cycles64(); + /* Ensure the CSR read can't reorder WRT to the copy. */ + mb(); + __riscv_copy_vec_words_unaligned(dst, src, MISALIGNED_COPY_SIZE); + /* Ensure the copy ends before the end time is snapped. */ + mb(); + end_cycles = get_cycles64(); + if ((end_cycles - start_cycles) < word_cycles) + word_cycles = end_cycles - start_cycles; + } + + byte_cycles = -1ULL; + __riscv_copy_vec_bytes_unaligned(dst, src, MISALIGNED_COPY_SIZE); + start_jiffies = jiffies; + while ((now = jiffies) == start_jiffies) + cpu_relax(); + + while (time_before(jiffies, now + (1 << MISALIGNED_ACCESS_JIFFIES_LG2))) { + start_cycles = get_cycles64(); + mb(); + __riscv_copy_vec_bytes_unaligned(dst, src, MISALIGNED_COPY_SIZE); + mb(); + end_cycles = get_cycles64(); + if ((end_cycles - start_cycles) < byte_cycles) + byte_cycles = end_cycles - start_cycles; + } + + kernel_vector_end(); + + /* Don't divide by zero. */ + if (!word_cycles || !byte_cycles) { + pr_warn("cpu%d: rdtime lacks granularity needed to measure unaligned vector access speed\n", + cpu); + + return; + } + + if (word_cycles < byte_cycles) + speed = RISCV_HWPROBE_VEC_MISALIGNED_FAST; + + ratio = div_u64((byte_cycles * 100), word_cycles); + pr_info("cpu%d: Ratio of vector byte access time to vector unaligned word access is %d.%02d, unaligned accesses are %s\n", + cpu, + ratio / 100, + ratio % 100, + (speed == RISCV_HWPROBE_VEC_MISALIGNED_FAST) ? "fast" : "slow"); + + per_cpu(vector_misaligned_access, cpu) = speed; +} + +static int riscv_online_cpu_vec(unsigned int cpu) +{ + check_vector_unaligned_access(NULL); + return 0; +} + +/* Measure unaligned access speed on all CPUs present at boot in parallel. */ +static int vec_check_unaligned_access_speed_all_cpus(void *unused) +{ + /* Schecudles work que*/ + schedule_on_each_cpu(check_vector_unaligned_access); + + /* + * Setup hotplug callbacks for any new CPUs that come online or go + * offline. + */ + cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "riscv:online", + riscv_online_cpu_vec, NULL); + + return 0; +} +#endif /* CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS */ + static int check_unaligned_access_all_cpus(void) { - bool all_cpus_emulated; + bool all_cpus_emulated, all_cpus_vec_supported; int cpu; if (riscv_has_extension_unlikely(RISCV_ISA_EXT_ZICCLSM)) { @@ -280,7 +399,14 @@ static int check_unaligned_access_all_cpus(void) } all_cpus_emulated = check_unaligned_access_emulated_all_cpus(); - check_vector_unaligned_access_emulated_all_cpus(); + all_cpus_vec_supported = check_vector_unaligned_access_emulated_all_cpus(); + +#ifdef CONFIG_RISCV_PROBE_VECTOR_UNALIGNED_ACCESS + if (all_cpus_vec_supported) { + kthread_run(vec_check_unaligned_access_speed_all_cpus, + NULL, "vec_check_unaligned_access_speed_all_cpus"); + } +#endif #ifdef CONFIG_RISCV_PROBE_UNALIGNED_ACCESS if (!all_cpus_emulated) diff --git a/arch/riscv/kernel/vec-copy-unaligned.S b/arch/riscv/kernel/vec-copy-unaligned.S new file mode 100644 index 000000000000..e5bc94917e60 --- /dev/null +++ b/arch/riscv/kernel/vec-copy-unaligned.S @@ -0,0 +1,58 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (C) 2024 Rivos Inc. */ + +#include +#include +#include + + .text + +#define WORD_EEW 32 + +#define WORD_SEW CONCATENATE(e, WORD_EEW) +#define VEC_L CONCATENATE(vle, WORD_EEW).v +#define VEC_S CONCATENATE(vle, WORD_EEW).v + +/* void __riscv_copy_vec_words_unaligned(void *, const void *, size_t) */ +/* Performs a memcpy without aligning buffers, using word loads and stores. */ +/* Note: The size is truncated to a multiple of WORD_EEW */ +SYM_FUNC_START(__riscv_copy_vec_words_unaligned) + andi a4, a2, ~(WORD_EEW-1) + beqz a4, 2f + add a3, a1, a4 + .option push + .option arch, +zve32x +1: + vsetivli t0, 8, WORD_SEW, m8, ta, ma + VEC_L v0, (a1) + VEC_S v0, (a0) + addi a0, a0, WORD_EEW + addi a1, a1, WORD_EEW + bltu a1, a3, 1b + +2: + .option pop + ret +SYM_FUNC_END(__riscv_copy_vec_words_unaligned) + +/* void __riscv_copy_vec_bytes_unaligned(void *, const void *, size_t) */ +/* Performs a memcpy without aligning buffers, using only byte accesses. */ +/* Note: The size is truncated to a multiple of 8 */ +SYM_FUNC_START(__riscv_copy_vec_bytes_unaligned) + andi a4, a2, ~(8-1) + beqz a4, 2f + add a3, a1, a4 + .option push + .option arch, +zve32x +1: + vsetivli t0, 8, e8, m8, ta, ma + vle8.v v0, (a1) + vse8.v v0, (a0) + addi a0, a0, 8 + addi a1, a1, 8 + bltu a1, a3, 1b + +2: + .option pop + ret +SYM_FUNC_END(__riscv_copy_vec_bytes_unaligned) From patchwork Thu Jun 13 19:16:15 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Jesse Taube X-Patchwork-Id: 13697401 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 918F0C27C75 for ; Thu, 13 Jun 2024 19:18:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=1TujCnrRgQ5aZGBO3ZQGhdynC13uYstL6rkTABsedko=; b=gXWAqbjVur1Onm QzGpJYbjfi2Y8ue4g6tvgh0fy+P/B5mkrO9r0oELx1nqzGiQiDfkSZid0iB59yMwm4DIbIoeDVymb vl1xvj+DE/7WRxrgVFg+mPjpqEbSwh8pb41vmO41aYsEkCSkiFPDGFV5YPfdzLriv4k/kKw4aFdhB spk1kNJQDHhYxAsXidr5KlBQ7+Yee/LmO8VtXjPboNxpDOO+7xNozYu9JtIUztIoq91NFCTvg6gwH 2C0eH5A+/8BW/cSwgQDUSpwT451tlTOtq0zLuVKuj0BFOlstAAi4dUoHIJgzgaVCewlsMX/SHUo2G xLyQfTbvOAyo0QeQyuyQ==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpxX-00000000Emz-3L85; Thu, 13 Jun 2024 19:18:07 +0000 Received: from mail-pl1-x635.google.com ([2607:f8b0:4864:20::635]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1sHpxU-00000000El4-1OL2 for linux-riscv@lists.infradead.org; Thu, 13 Jun 2024 19:18:05 +0000 Received: by mail-pl1-x635.google.com with SMTP id d9443c01a7336-1f6fada63a6so12060555ad.3 for ; Thu, 13 Jun 2024 12:18:03 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rivosinc-com.20230601.gappssmtp.com; s=20230601; t=1718306283; x=1718911083; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=lznOWuT3zuZ+7YZNo94Ic++y7Z2ydYmmMFGw8gZ0Wmk=; b=K91+dDHeIZ2qDtXmf5F6n6t9oj/W/l8YawyRY9tM/LE3oVKTRWVTZxPfxqqyy6v0PC cgWaMDxNgK7IKsoqAvVQC9CObmQjR2Wop0XKPqSn3qHXCYyKwUnnoaZdp3qTiWET1/jM Y1+4AlDNEGqm9KJA6D41P7/5Hsy+F/AkCnkGIhg8/N4ikgdUIr58OIfHcDaQxop73ww9 oUR2WM3saBOOs93kFVwHmR9MDffGDTHSiMjH1O9Vl//A/h0Ljq5iic06bGauTZ9O5Ed6 HniyLCX1SCT3GYOqUb0pb41B0m0GdShmkxqtRBW/dsalOZq0FPffB8GofIMsQKB5fxYK c6gg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718306283; x=1718911083; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=lznOWuT3zuZ+7YZNo94Ic++y7Z2ydYmmMFGw8gZ0Wmk=; b=vMHxo0R/SAKC7d9pwlrJkIimFaVWfG7xcbwNDx9DW9D6sUKWAtJ3SZ9Uq8uB1vb9p+ lrQjxMrF8co5Zsweb5szkvzvn9zOeXf3jwb3KzH4sSsL9XMyHFrAcSQOFh/8IyF6QpLB 4WkkbW2tcViMFn3DIypWausioYT+1TOJFoelM3F69Niej2a311O/0YfACG0YdYztpw0K r8fh1iLGiSb1pA4jU6Ewv8T/hmuC3XAT0pO5eN1TRJRS5sYgAdEF2g4mVm8MbxGdLkT1 Vb3W9/OhHp0BZIElZnSww3bX2e99Pw8sWGZWBtztf8SK9IraBehwrx8z8ijBdyfiE+v9 S4Kg== X-Gm-Message-State: AOJu0YxFDu4Fk9ZP9ecjMpjEvDUiXo2JNG9xuILWw6U7npQK21RW1oXF h1Hflu84kySirIszIqqIOUo6XLghfLPZrJhWtZL5QmnyjXbkkNGW3N8IQa+mtzn81GMjEnc8HZ3 dEPU= X-Google-Smtp-Source: AGHT+IH16Z30xWnnVCPD+SWE8CuZmhFACgLpQTZU9/kCBeM9FuPyEaJYp2X2ogoYFLKkXmxabg3ZrQ== X-Received: by 2002:a17:902:f68b:b0:1f7:1706:25ba with SMTP id d9443c01a7336-1f8625cf2b0mr6158425ad.15.1718306283241; Thu, 13 Jun 2024 12:18:03 -0700 (PDT) Received: from jesse-desktop.. (pool-108-26-179-17.bstnma.fios.verizon.net. [108.26.179.17]) by smtp.gmail.com with ESMTPSA id d9443c01a7336-1f855e7ca78sm17471015ad.106.2024.06.13.12.17.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 12:18:02 -0700 (PDT) From: Jesse Taube To: linux-riscv@lists.infradead.org Cc: Jonathan Corbet , Paul Walmsley , Palmer Dabbelt , Albert Ou , Conor Dooley , Rob Herring , Krzysztof Kozlowski , =?utf-8?b?Q2zDqW1lbnQgTMOpZ2Vy?= , Evan Green , Andrew Jones , Jesse Taube , Charlie Jenkins , Xiao Wang , Andy Chiu , Eric Biggers , Greentime Hu , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Heiko Stuebner , Costa Shulyupin , Andrew Morton , Baoquan He , Anup Patel , Zong Li , Sami Tolvanen , Ben Dooks , Alexandre Ghiti , "Gustavo A. R. Silva" , Erick Archer , Joel Granados , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, devicetree@vger.kernel.org Subject: [PATCH v2 6/6] RISC-V: hwprobe: Document unaligned vector perf key Date: Thu, 13 Jun 2024 15:16:15 -0400 Message-ID: <20240613191616.2101821-7-jesse@rivosinc.com> X-Mailer: git-send-email 2.43.0 In-Reply-To: <20240613191616.2101821-1-jesse@rivosinc.com> References: <20240613191616.2101821-1-jesse@rivosinc.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240613_121804_495895_8D1D334F X-CRM114-Status: GOOD ( 11.55 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Document key for reporting the speed of unaligned vector accesses. The descriptions are the same as the scalar equivalent values. Signed-off-by: Jesse Taube --- V1 -> V2: - New patch --- Documentation/arch/riscv/hwprobe.rst | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst index 7085a694b801..344bea1e21bd 100644 --- a/Documentation/arch/riscv/hwprobe.rst +++ b/Documentation/arch/riscv/hwprobe.rst @@ -236,3 +236,19 @@ The following keys are defined: * :c:macro:`RISCV_HWPROBE_KEY_ZICBOZ_BLOCK_SIZE`: An unsigned int which represents the size of the Zicboz block in bytes. + +* :c:macro:`RISCV_HWPROBE_KEY_VEC_MISALIGNED_PERF`: An enum value describing the + performance of misaligned vector accesses on the selected set of processors. + + * :c:macro:`RISCV_HWPROBE_VEC_MISALIGNED_UNKNOWN`: The performance of misaligned + accesses is unknown. + + * :c:macro:`RISCV_HWPROBE_VEC_MISALIGNED_SLOW`: Misaligned accesses are slower + than equivalent byte accesses. Misaligned accesses may be supported + directly in hardware, or trapped and emulated by software. + + * :c:macro:`RISCV_HWPROBE_VEC_MISALIGNED_FAST`: Misaligned accesses are faster + than equivalent byte accesses. + + * :c:macro:`RISCV_HWPROBE_VEC_MISALIGNED_UNSUPPORTED`: Misaligned accesses are + not supported at all and will generate a misaligned address fault.