From patchwork Fri Nov 24 07:21:38 2023 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: =?utf-8?q?Christoph_M=C3=BCllner?= X-Patchwork-Id: 13467076 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 32818C61D97 for ; Fri, 24 Nov 2023 07:22:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=5vquF8qdB9lO2qsV5KLRas7YJd90dQDucSTkI/dyYKU=; b=e2e6wVGvrPkIf3 3pOuetzS/RIsxhObfFQqGB5EFURNpmMdwdEpdibZYOtRfZmpmzen+czBrgaCOKw0igE2Nhr+zQ/ij +RbMsl/5Ku3DvgbjEmO8D39NwMml3yAlxIPCDgYk6pDFjFeK4wYidF6Mtdp2bJYRMRXybbutpIfAE Th8GE2HT5/2UXu5UZ8lsr1E2N1BDF96pk82OelocesANSU3OqvCRqwWen3rwjSm4Bx6xIz5ZYgfEB yLFt1t9bAYA5hE+k0oBN2jxMLoE6VS/IqN1CNh/5SqgC1qf4w9Ovo9oXWjpYgokZS7fJnBvqe2qlC wdxtkT/5ZWCr4imPZkLA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.96 #2 (Red Hat Linux)) id 1r6QW0-006NNF-2o; Fri, 24 Nov 2023 07:22:16 +0000 Received: from mail-ej1-x62b.google.com ([2a00:1450:4864:20::62b]) by bombadil.infradead.org with esmtps (Exim 4.96 #2 (Red Hat Linux)) id 1r6QVw-006NJt-0r for linux-riscv@lists.infradead.org; Fri, 24 Nov 2023 07:22:14 +0000 Received: by mail-ej1-x62b.google.com with SMTP id a640c23a62f3a-a00ac0101d9so213749866b.0 for ; Thu, 23 Nov 2023 23:22:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=vrull.eu; s=google; t=1700810526; x=1701415326; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=LZ48+YrsFa7qeiwW9XToPWJsnyGNFxWvEVuNMJnE/C8=; b=idCA8cwSI7lEq422vNRVUiNbI8bxyt3W06PZmE6PAbTZr56tsJ8Se9ZA46J78xsH+g XYTBgCfyAdhXXoHt3A69m8RKVtdSVIHKx0hnWaiknMv+9FTWg2AQqN/wCG7Deu7fx0EK je28SWDa3t+1xMD22LITvZ8R3dGxPcIYfPhv4aCUJmNed0z4szA7SAPzqxSicAeIsxpW /bvoyHP8ScFF5rFnmZ0utW8gdX88lpu+TnO5V1pPgvD9IW/vm90yG2JYSlmqXgTTcwJk /9Kp5Hh9laKUfDkKUHhJRFI19gN9eusbAJ87Yv+kRti8iokvWnPVRU8QNa+qkpTB7G2M oq5Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1700810526; x=1701415326; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=LZ48+YrsFa7qeiwW9XToPWJsnyGNFxWvEVuNMJnE/C8=; b=YI+XavbShHvegyEsOvNubYyPZa/h9vVEMLkJy1iTnOayH3fQLEgLPLOyPrG276RgQ3 gy0onADO7AXJHnTWsdXwlZMEihkRjq1+8GCRTfLtfR5d/iKqYIDS+kxmziSdfpc7Y4Qw 2ehh2xRMWX0iSuQUkXAdIktLwC84YeoSH2vOWJxMUFwNiA/aeHII5IKCSvUYVTZ5pvgY oagntrJqV8x1xpVqzXAl0uQUtjyTHdP+V6L00WuWjsfVgbuTC3v5UeFb4iOG2GanOiQZ g3nXlX3Y89DfiwXle+1ECOa+lC+ONfOewgaHSPEhYVxhvJvMScpL9Dl7TWZOaPUvSmDN q98g== X-Gm-Message-State: AOJu0YxHRnmXP9Ev2ApqvhfuTMjZSl4Q68Veu8B/5y8+d7kT8JNFkvyw DodvanCuAGQ984wbfKdEn8hy9CiSsxDA/ZGLjiYuOw7m X-Google-Smtp-Source: AGHT+IGrxBHMPl0kbFz5aJ0kbrWM+mllmiLaiB9I2FJCwXD18VcETZ/Dnt6zPPorDkdAcgvZjQLlKw== X-Received: by 2002:a17:906:583:b0:a00:893f:58cf with SMTP id 3-20020a170906058300b00a00893f58cfmr1233875ejn.54.1700810526424; Thu, 23 Nov 2023 23:22:06 -0800 (PST) Received: from beast.fritz.box (62-178-148-172.cable.dynamic.surfer.at. [62.178.148.172]) by smtp.gmail.com with ESMTPSA id q18-20020a1709060e5200b00992f2befcbcsm1709930eji.180.2023.11.23.23.22.04 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 23 Nov 2023 23:22:05 -0800 (PST) From: Christoph Muellner To: linux-riscv@lists.infradead.org, linux-kselftest@vger.kernel.org, linux-kernel@vger.kernel.org, linux-doc@vger.kernel.org, Palmer Dabbelt , Paul Walmsley , Albert Ou , Andrew Morton , Shuah Khan , Jonathan Corbet , Anup Patel , Philipp Tomsich , Andrew Jones , Guo Ren , Daniel Henrique Barboza , Conor Dooley , =?utf-8?b?QmrDtnJuIFTDtnBlbA==?= , Alan Stern , Andrea Parri , Will Deacon , Daniel Lustig , Peter Zijlstra Cc: =?utf-8?q?Christoph_M=C3=BCllner?= Subject: [RFC PATCH 1/5] RISC-V: Add basic Ssdtso support Date: Fri, 24 Nov 2023 08:21:38 +0100 Message-ID: <20231124072142.2786653-2-christoph.muellner@vrull.eu> X-Mailer: git-send-email 2.41.0 In-Reply-To: <20231124072142.2786653-1-christoph.muellner@vrull.eu> References: <20231124072142.2786653-1-christoph.muellner@vrull.eu> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20231123_232212_301521_18C4D8A7 X-CRM114-Status: GOOD ( 25.01 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Christoph Müllner Ssdtso is a RISC-V ISA extension, which allows to switch the memory consistency model from RVWMO to TSO (and back) at runtime. The active model is controlled by a DTSO bit in the {m,h,s}envcfg CSRs (per-hart state). TSO is a stronger memory ordering than RVWMO, which means that executing software that was written for RVWMO can also run under TSO without causing memory consistency issues. Since RVWMO is the default model, switching to TSO is safe. The patch introduces Ssdtso basic support: * define the relevant bits * register the the extension in hwcap/cpufeatures * extend thread_struct to keep the state across context switches * add the relevant code to store/restore the DTSO state Following the pattern of existing code, this patch also introduces a Kconfig symbol ('RISCV_ISA_SSDTSO') to disable Ssdtso support. Signed-off-by: Christoph Müllner --- arch/riscv/Kconfig | 10 ++++ arch/riscv/include/asm/csr.h | 1 + arch/riscv/include/asm/dtso.h | 74 ++++++++++++++++++++++++++++++ arch/riscv/include/asm/hwcap.h | 1 + arch/riscv/include/asm/processor.h | 1 + arch/riscv/include/asm/switch_to.h | 3 ++ arch/riscv/kernel/cpufeature.c | 1 + arch/riscv/kernel/process.c | 4 ++ 8 files changed, 95 insertions(+) create mode 100644 arch/riscv/include/asm/dtso.h diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 95a2a06acc6a..c62718fa8e7f 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -457,6 +457,16 @@ config RISCV_ISA_C If you don't know what to do here, say Y. +config RISCV_ISA_SSDTSO + bool "Ssdtso extension support for dynamic TSO memory ordering" + default y + help + Adds support to dynamically detect the presence of the Ssdtso + ISA-extension and allows user-space processes to activate/deactivate + the TSO memory ordering model at run-time. + + If you don't know what to do here, say Y. + config RISCV_ISA_SVNAPOT bool "Svnapot extension support for supervisor mode NAPOT pages" depends on 64BIT && MMU diff --git a/arch/riscv/include/asm/csr.h b/arch/riscv/include/asm/csr.h index 306a19a5509c..2689ad6b2b60 100644 --- a/arch/riscv/include/asm/csr.h +++ b/arch/riscv/include/asm/csr.h @@ -194,6 +194,7 @@ /* xENVCFG flags */ #define ENVCFG_STCE (_AC(1, ULL) << 63) #define ENVCFG_PBMTE (_AC(1, ULL) << 62) +#define ENVCFG_DTSO (_AC(1, UL) << 8) #define ENVCFG_CBZE (_AC(1, UL) << 7) #define ENVCFG_CBCFE (_AC(1, UL) << 6) #define ENVCFG_CBIE_SHIFT 4 diff --git a/arch/riscv/include/asm/dtso.h b/arch/riscv/include/asm/dtso.h new file mode 100644 index 000000000000..f8a758c45e05 --- /dev/null +++ b/arch/riscv/include/asm/dtso.h @@ -0,0 +1,74 @@ +/* SPDX-License-Identifier: GPL-2.0-or-later */ +/* + * Copyright (c) 2023 Christoph Muellner + */ + +#ifndef __ASM_RISCV_DTSO_H +#define __ASM_RISCV_DTSO_H + +#ifdef CONFIG_RISCV_ISA_SSDTSO + +#include +#include +#include + +static __always_inline bool has_dtso(void) +{ + return riscv_has_extension_unlikely(RISCV_ISA_EXT_SSDTSO); +} + +static inline bool dtso_is_enabled(void) +{ + if (has_dtso()) + return csr_read(CSR_SENVCFG) & ENVCFG_DTSO; + return 0; +} + +static inline void dtso_disable(void) +{ + if (has_dtso()) + csr_clear(CSR_SENVCFG, ENVCFG_DTSO); +} + +static inline void dtso_enable(void) +{ + if (has_dtso()) + csr_set(CSR_SENVCFG, ENVCFG_DTSO); +} + +static inline void dtso_save(struct task_struct *task) +{ + task->thread.dtso_ena = dtso_is_enabled(); +} + +static inline void dtso_restore(struct task_struct *task) +{ + if (task->thread.dtso_ena) + dtso_enable(); + else + dtso_disable(); +} + +static inline void __switch_to_dtso(struct task_struct *prev, + struct task_struct *next) +{ + struct pt_regs *regs; + + regs = task_pt_regs(prev); + dtso_save(prev); + dtso_restore(next); +} + +#else /* ! CONFIG_RISCV_ISA_SSDTSO */ + +static __always_inline bool has_dtso(void) { return false; } +static __always_inline bool dtso_is_enabled(void) { return false; } +#define dtso_disable() do { } while (0) +#define dtso_enable() do { } while (0) +#define dtso_save(task) do { } while (0) +#define dtso_restore(task) do { } while (0) +#define __switch_to_dtso(prev, next) do { } while (0) + +#endif /* CONFIG_RISCV_ISA_SSDTSO */ + +#endif /* ! __ASM_RISCV_DTSO_H */ diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h index 06d30526ef3b..cbf924d6dfb7 100644 --- a/arch/riscv/include/asm/hwcap.h +++ b/arch/riscv/include/asm/hwcap.h @@ -57,6 +57,7 @@ #define RISCV_ISA_EXT_ZIHPM 42 #define RISCV_ISA_EXT_SMSTATEEN 43 #define RISCV_ISA_EXT_ZICOND 44 +#define RISCV_ISA_EXT_SSDTSO 45 #define RISCV_ISA_EXT_MAX 64 diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index f19f861cda54..79cc5e6377b8 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -84,6 +84,7 @@ struct thread_struct { unsigned long vstate_ctrl; struct __riscv_v_ext_state vstate; unsigned long align_ctl; + bool dtso_ena; /* Dynamic TSO enable */ }; /* Whitelist the fstate from the task_struct for hardened usercopy */ diff --git a/arch/riscv/include/asm/switch_to.h b/arch/riscv/include/asm/switch_to.h index f90d8e42f3c7..f07180a3b533 100644 --- a/arch/riscv/include/asm/switch_to.h +++ b/arch/riscv/include/asm/switch_to.h @@ -9,6 +9,7 @@ #include #include #include +#include #include #include #include @@ -81,6 +82,8 @@ do { \ __switch_to_fpu(__prev, __next); \ if (has_vector()) \ __switch_to_vector(__prev, __next); \ + if (has_dtso()) \ + __switch_to_dtso(__prev, __next); \ ((last) = __switch_to(__prev, __next)); \ } while (0) diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c index b3785ffc1570..381ba02689ca 100644 --- a/arch/riscv/kernel/cpufeature.c +++ b/arch/riscv/kernel/cpufeature.c @@ -181,6 +181,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = { __RISCV_ISA_EXT_DATA(smstateen, RISCV_ISA_EXT_SMSTATEEN), __RISCV_ISA_EXT_DATA(ssaia, RISCV_ISA_EXT_SSAIA), __RISCV_ISA_EXT_DATA(sscofpmf, RISCV_ISA_EXT_SSCOFPMF), + __RISCV_ISA_EXT_DATA(ssdtso, RISCV_ISA_EXT_SSDTSO), __RISCV_ISA_EXT_DATA(sstc, RISCV_ISA_EXT_SSTC), __RISCV_ISA_EXT_DATA(svinval, RISCV_ISA_EXT_SVINVAL), __RISCV_ISA_EXT_DATA(svnapot, RISCV_ISA_EXT_SVNAPOT), diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c index 4f21d970a129..65462b675740 100644 --- a/arch/riscv/kernel/process.c +++ b/arch/riscv/kernel/process.c @@ -172,6 +172,10 @@ void flush_thread(void) kfree(current->thread.vstate.datap); memset(¤t->thread.vstate, 0, sizeof(struct __riscv_v_ext_state)); #endif +#ifdef CONFIG_RISCV_ISA_SSDTSO + /* Reset DTSO state */ + current->thread.dtso_ena = false; +#endif } void arch_release_task_struct(struct task_struct *tsk)