From patchwork Wed Nov 27 17:29:02 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13887251 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 915A8D6ACF7 for ; Wed, 27 Nov 2024 17:30:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=+pBlPr9mSkHsFxaFCwJIQIRJCYJtvVHzW8bosl9CNOA=; b=K6q29V7mlgdREN SBW9tlTmZNOheNgowqSuWqUapdJk/HooNAsxceVPSItiUjABCcf6Y2UsqvjQ42n8FMOloWgPKNoTz TKEzMjWX5j16ZEM471QOGIymhYTRg1edwimM1QvLzojG1CMa1ME0Ppk1wUEtEnZ43Y3qHqjr9aASA UmyqDZnN97qI+7gg+wQCmITcAqYcg3MwZGlpyCtuh5VPaN5S7pI2lBxEd2/dz6AF99dWA4RtNaOtf YxlbQ6OLvzdancSMQ/0h++zhLFddBB/hTQ5SjF9KQrgiJ09g9ebBYZFeZupnFZraS+s/XcB0lwShp ttK7WClKAG1V4Hvcz64Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrR-0000000DkOy-4ASV; Wed, 27 Nov 2024 17:29:57 +0000 Received: from mail-pf1-x429.google.com ([2607:f8b0:4864:20::429]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrP-0000000DkNY-1wcT for linux-riscv@lists.infradead.org; Wed, 27 Nov 2024 17:29:56 +0000 Received: by mail-pf1-x429.google.com with SMTP id d2e1a72fcca58-724f74d6457so47989b3a.0 for ; Wed, 27 Nov 2024 09:29:54 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732728594; x=1733333394; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VhqF3E4/w+YtOL58NNfG2rcUNmjyen3grJ9NA1x6juQ=; b=QvdIM6p6p+C2AeQSxowNyDTDj7fZvSmaXYNciXcB21LCHpDT4Zb4e2IwD5EaoH1a4e zvRkqWSyJSdGCd7vWSQ5Hduj1W4RcM9AfD/KAmgpZFayjboHQSQjPWj6V0woWhNwFMYk 0Iyc/gJdayGjKU8dQGIwfmMXvDYHaGx7A3PoFyLFfR4QqOUf0ZeQlkkRILak4Jbab0tk BdikAUEjHyh2Htm6YPONaNKqm9IOhkuA77mfj8HcNtLjN8QWwKkC8rGGiSyLAEFUUh7Q MhFaoxajpbHXm3Ug+wtvDkM5A/L4iY3Cj9mFHDZoAV/42JvLEZ4HUlJnTdp6U+lCPCUd m5iQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732728594; x=1733333394; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VhqF3E4/w+YtOL58NNfG2rcUNmjyen3grJ9NA1x6juQ=; b=eIx3m4YSsvkLGJ166VBygo4s/10XqTMmnAfXreCOJkfuRUzTlHX8z4MDIcKm8Ajmez Mmy2A4su9xqOntfjG5/KV8F/W83nT2KRldDpqRve6KF4PK10AwVvvcBLCpxSTf/6dZ+/ VIOGBsOQwSAZI0iTpuXA9rDFyWdmq/uSIcVwBMM/bR8flseLgm4EJj+EGQBzzFJhA/ve lvwKPCWw8tjCzTQWn5B3JuePydSgYQDk99QgW1SdTXUOyDER1pRYlwjde1l3vD6/qCyg hCz3SY/remI6LvN4F/MURLA2lC3UpFl176vbqA3GVtyEYwQrHHye67z53VD8nxv209TP yajQ== X-Forwarded-Encrypted: i=1; AJvYcCVR4p5icj8Vd08WnFzaZDxHM2Ovp/5jFrvAGVxdQbzTdc0dxPvC8zdiO0/DMNuGj4tRlecuQQWSgieDMQ==@lists.infradead.org X-Gm-Message-State: AOJu0YzXCiE2XFsMBolwpdQ5+Hzqrz7pCrwvqehh21SJ7hRPvhRt8pLY jeJyNs2Vgcns8enj7FI2CmAxy0kRzK7lagjbgw5J936ax6kRtLQv X-Gm-Gg: ASbGncvRhfyawT4yJow6ynSi24cZbR9IRGIOYPo7EfT/ZzkK3fOOn6o7Fyrc8YJE8k9 J1OXw8GXtshCSyb7FhemnTV21iTKG7x2ZiZOqC99BAfIjI67xOSxB/397MiU+Zn+SWvnIpQ38kL 5Qe2UR9i9ga2pNC21+TG5/2zUzpd5ofZmbOFCyh6jUVeJe2WhV2ioxZmrKl9MlS/uhZ/QhTo+f+ z7zaQJ3rmlj1XxhlIJdrjhH6cN1PwsnhsmNn6x970JO5Ddtw9yb6U1pDabaxwpL7UzrmyE/HpOG U7S1074caUcpxwnEQ0hCXs7C+eAK X-Google-Smtp-Source: AGHT+IHe4vfUFn689BzH733Y3azaB4OQQfGNQ2U5d2DUk6y3aLJvqA63KG7l5W6jxcEPLUbmlo8oFg== X-Received: by 2002:a05:6a00:3c89:b0:724:eac3:576a with SMTP id d2e1a72fcca58-725301a5b92mr5270747b3a.25.1732728594123; Wed, 27 Nov 2024 09:29:54 -0800 (PST) Received: from localhost.localdomain (1-171-29-17.dynamic-ip.hinet.net. [1.171.29.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fbcc1e3fdbsm9359582a12.30.2024.11.27.09.29.50 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 27 Nov 2024 09:29:53 -0800 (PST) From: Andy Chiu To: Steven Rostedt , Masami Hiramatsu , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou , Nathan Chancellor , Nick Desaulniers , Bill Wendling , Justin Stitt Cc: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, llvm@lists.linux.dev, bjorn@rivosinc.com, puranjay12@gmail.com, alexghiti@rivosinc.com, yongxuan.wang@sifive.com, greentime.hu@sifive.com, nick.hu@sifive.com, nylon.chen@sifive.com, tommy.wu@sifive.com, eric.lin@sifive.com, viccent.chen@sifive.com, zong.li@sifive.com, samuel.holland@sifive.com Subject: [PATCH v3 1/7] riscv: ftrace: support fastcc in Clang for WITH_ARGS Date: Thu, 28 Nov 2024 01:29:02 +0800 Message-Id: <20241127172908.17149-2-andybnac@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20241127172908.17149-1-andybnac@gmail.com> References: <20241127172908.17149-1-andybnac@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241127_092955_505112_CC487D29 X-CRM114-Status: GOOD ( 10.82 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Andy Chiu Some caller-saved registers which are not defined as function arguments in the ABI can still be passed as arguments when the kernel is compiled with Clang. As a result, we must save and restore those registers to prevent ftrace from clobbering them. - [1]: https://reviews.llvm.org/D68559 Reported-by: Evgenii Shatokhin Closes: https://lore.kernel.org/linux-riscv/7e7c7914-445d-426d-89a0-59a9199c45b1@yadro.com/ Acked-by: Nathan Chancellor Signed-off-by: Andy Chiu --- arch/riscv/include/asm/ftrace.h | 7 +++++++ arch/riscv/kernel/asm-offsets.c | 7 +++++++ arch/riscv/kernel/mcount-dyn.S | 16 ++++++++++++++-- 3 files changed, 28 insertions(+), 2 deletions(-) diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h index 2cddd79ff21b..4ca7ce7f34d7 100644 --- a/arch/riscv/include/asm/ftrace.h +++ b/arch/riscv/include/asm/ftrace.h @@ -143,6 +143,13 @@ struct ftrace_regs { unsigned long a5; unsigned long a6; unsigned long a7; +#ifdef CONFIG_CC_IS_CLANG + unsigned long t2; + unsigned long t3; + unsigned long t4; + unsigned long t5; + unsigned long t6; +#endif }; }; }; diff --git a/arch/riscv/kernel/asm-offsets.c b/arch/riscv/kernel/asm-offsets.c index e94180ba432f..59789dfb2d5d 100644 --- a/arch/riscv/kernel/asm-offsets.c +++ b/arch/riscv/kernel/asm-offsets.c @@ -504,6 +504,13 @@ void asm_offsets(void) DEFINE(FREGS_SP, offsetof(struct ftrace_regs, sp)); DEFINE(FREGS_S0, offsetof(struct ftrace_regs, s0)); DEFINE(FREGS_T1, offsetof(struct ftrace_regs, t1)); +#ifdef CONFIG_CC_IS_CLANG + DEFINE(FREGS_T2, offsetof(struct ftrace_regs, t2)); + DEFINE(FREGS_T3, offsetof(struct ftrace_regs, t3)); + DEFINE(FREGS_T4, offsetof(struct ftrace_regs, t4)); + DEFINE(FREGS_T5, offsetof(struct ftrace_regs, t5)); + DEFINE(FREGS_T6, offsetof(struct ftrace_regs, t6)); +#endif DEFINE(FREGS_A0, offsetof(struct ftrace_regs, a0)); DEFINE(FREGS_A1, offsetof(struct ftrace_regs, a1)); DEFINE(FREGS_A2, offsetof(struct ftrace_regs, a2)); diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S index 745dd4c4a69c..e988bd26b28b 100644 --- a/arch/riscv/kernel/mcount-dyn.S +++ b/arch/riscv/kernel/mcount-dyn.S @@ -96,7 +96,13 @@ REG_S x8, FREGS_S0(sp) #endif REG_S x6, FREGS_T1(sp) - +#ifdef CONFIG_CC_IS_CLANG + REG_S x7, FREGS_T2(sp) + REG_S x28, FREGS_T3(sp) + REG_S x29, FREGS_T4(sp) + REG_S x30, FREGS_T5(sp) + REG_S x31, FREGS_T6(sp) +#endif // save the arguments REG_S x10, FREGS_A0(sp) REG_S x11, FREGS_A1(sp) @@ -115,7 +121,13 @@ REG_L x8, FREGS_S0(sp) #endif REG_L x6, FREGS_T1(sp) - +#ifdef CONFIG_CC_IS_CLANG + REG_L x7, FREGS_T2(sp) + REG_L x28, FREGS_T3(sp) + REG_L x29, FREGS_T4(sp) + REG_L x30, FREGS_T5(sp) + REG_L x31, FREGS_T6(sp) +#endif // restore the arguments REG_L x10, FREGS_A0(sp) REG_L x11, FREGS_A1(sp) From patchwork Wed Nov 27 17:29:03 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13887252 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BE6EBD6ACF9 for ; Wed, 27 Nov 2024 17:30:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=A9k91n/2uKtNjtcucJhzthGZEuknWJJoBpLRwAipMgo=; b=zrc7EW5rcWQ6M/ +1et48NtvBdy0AOFGrEgKBGD79fEwloPFgNcRyqSdHdWT1yf1jYUHLutaC1Cnw+BVu6AjPMmmbxRy /ojnZ47YD/hxNsz28DbsY2cucW9y/tiziKd4QhNciaffh8EEmmI4y6vIgSOtMDYopBz5ynhLzHhsv iB/GWoh33TdQD+wewtKGZoo5E6YH3ecmD7YixTWNZ6DJOLj7fGIxhpiOrX2mB6xkpTK9sGYGiR8LQ b7My8mw4PYyiKkSMEjt/0pyY1eBY+v+eG4NTrlww3Ck+7jIPLvooXcMorXGdSWx5MlxOlVIcyIjPM ye1GUKxdZcS2+9ooBc4Q==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrU-0000000DkQO-2O2a; Wed, 27 Nov 2024 17:30:00 +0000 Received: from mail-pg1-x532.google.com ([2607:f8b0:4864:20::532]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrS-0000000DkPK-3GKc for linux-riscv@lists.infradead.org; Wed, 27 Nov 2024 17:30:00 +0000 Received: by mail-pg1-x532.google.com with SMTP id 41be03b00d2f7-7ee7e87f6e4so5797203a12.2 for ; Wed, 27 Nov 2024 09:29:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732728598; x=1733333398; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=oUtTSvLb/awnb4pf5MloBg9/ARoAYzcuulYeAOWZ3lk=; b=gVo4XzzxU0i8UYKuCxxZH8XpaP+Ymf5SxLagcPzx70mly9Ou+jnZm8KuQM6XeLJGqM sZtbRDSYdnuZ0LroGfe7lT+Jgu/0PEbHtgwZmCZPBJkcFY7briI06MaEmH90gCaeU4F5 tud6v4nZWt4dadcgQc2HIJy0753WT5+p98Eyj6ZYfzlj5x8vsnc29GdH04psvgu7Gldv z7PjFYlp3vA6s2LfPcyojEWOPwJX5bZzA4M3Ay54NCvMYeakNhYlPhEEKPtIMchOyBvV 1UaHIsgkGVWsPUwqagCt6LXESs4xkBt2nZqDvt7EHrFWrn4b51a2smw7qnHa6Dmi1IvN etWg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732728598; x=1733333398; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=oUtTSvLb/awnb4pf5MloBg9/ARoAYzcuulYeAOWZ3lk=; b=WJocFrdh1o7g9o0mjpdfNxcI7OeZazf5mKSvTbru69Q8RkSBpK82i7hY91nuSk8wJy 8rJJaSMabA/WNheioweUA4vhARGc4QknxSpUCHB1o2c6stMwmFYeR6IxlMQUsIu7ksCD mmX+W4unGt37KyIhcrMyHLBjdF5XEA9Dfqn8Qk0wJwvi/D+s7M05/7ky4GUNVABmhAa0 b0FXEdxJ6wQ9zWddV3DgqvE8VZ9IE6BuVqSbsL00LeB2kzu9JOV8dRcvzdvsaj+WNFb9 oT6EP2K/daP6Smp5B0OVxQdJZGvFsNeJmsUWMLQLPEx8Za5yGM2TBxkDrSG+Hf2MNKu5 I7Aw== X-Gm-Message-State: AOJu0Yxa2PiKu1A6JFhtlPlfRFG1ck7CtNrn6Z0/vBh2SOuG+5UrosxS 7Ws+A0s3vHDMW/9+uMi43kjEIDpY2ntxrTTMus0mN4VVjQA9mXQX X-Gm-Gg: ASbGncu42PAQLO9pwzlXnbHg0js59Ng9+nk+2r6IKvCVXuqNqg4H16GbwKP7/NSbM46 xNNeH3LZpXQ4mfA9/AwPiE4iECk8BgLmuzPTI8BZuEGA4YEc7mJwTkmT3wmmEHr+6UhWBLOAoBE 5RKC+u2wrl/NDvNpQYf5tbr3xGlxcK7INEoIPgoOoM4i9/pEQKcuaMj7qgTnLx8Dk9n9ObfjZpr GG2659f9uFBCkaQNrU2AE+GAVz51W/PK+teBDVYaU5Kj5oN0JzwSSUNrGZVZdq3Gu/5y/GzRVuq IdcejUecri+WgqeLIKFOgkV0gjqJ X-Google-Smtp-Source: AGHT+IGZTrwY9TsRiJj+/2qS9smtrrSi2Kd6N3yOUCcWlvOO+wK88Dm28z0bcTIf0ysSO3Z6//fqeQ== X-Received: by 2002:a05:6a20:6a09:b0:1e0:cfa5:e32e with SMTP id adf61e73a8af0-1e0e0b8b8eemr5787909637.44.1732728598105; Wed, 27 Nov 2024 09:29:58 -0800 (PST) Received: from localhost.localdomain (1-171-29-17.dynamic-ip.hinet.net. [1.171.29.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fbcc1e3fdbsm9359582a12.30.2024.11.27.09.29.55 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 27 Nov 2024 09:29:57 -0800 (PST) From: Andy Chiu To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, bjorn@rivosinc.com, puranjay12@gmail.com, alexghiti@rivosinc.com, yongxuan.wang@sifive.com, greentime.hu@sifive.com, nick.hu@sifive.com, nylon.chen@sifive.com, tommy.wu@sifive.com, eric.lin@sifive.com, viccent.chen@sifive.com, zong.li@sifive.com, samuel.holland@sifive.com Subject: [PATCH v3 2/7] riscv: ftrace: align patchable functions to 4 Byte boundary Date: Thu, 28 Nov 2024 01:29:03 +0800 Message-Id: <20241127172908.17149-3-andybnac@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20241127172908.17149-1-andybnac@gmail.com> References: <20241127172908.17149-1-andybnac@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241127_092958_814011_FCBA2765 X-CRM114-Status: GOOD ( 11.84 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Andy Chiu We are changing ftrace code patching in order to remove dependency from stop_machine() and enable kernel preemption. This requires us to align functions entry at a 4-B align address. However, -falign-functions on older versions of GCC alone was not strong enoungh to align all functions. In fact, cold functions are not aligned after turning on optimizations. We consider this is a bug in GCC and turn off guess-branch-probility as a workaround to align all functions. GCC bug id: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=88345 The option -fmin-function-alignment is able to align all functions properly on newer versions of gcc. So, we add a cc-option to test if the toolchain supports it. Suggested-by: Evgenii Shatokhin Signed-off-by: Andy Chiu Reviewed-by: Björn Töpel --- arch/riscv/Kconfig | 2 ++ 1 file changed, 2 insertions(+) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 5bdda86ada37..75a5ebde4427 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -142,6 +142,7 @@ config RISCV select HAVE_DEBUG_KMEMLEAK select HAVE_DMA_CONTIGUOUS if MMU select HAVE_DYNAMIC_FTRACE if !XIP_KERNEL && MMU && (CLANG_SUPPORTS_DYNAMIC_FTRACE || GCC_SUPPORTS_DYNAMIC_FTRACE) + select FUNCTION_ALIGNMENT_4B if HAVE_DYNAMIC_FTRACE && RISCV_ISA_C select HAVE_DYNAMIC_FTRACE_WITH_DIRECT_CALLS select HAVE_DYNAMIC_FTRACE_WITH_ARGS if HAVE_DYNAMIC_FTRACE select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL @@ -226,6 +227,7 @@ config CLANG_SUPPORTS_DYNAMIC_FTRACE config GCC_SUPPORTS_DYNAMIC_FTRACE def_bool CC_IS_GCC depends on $(cc-option,-fpatchable-function-entry=8) + depends on CC_HAS_MIN_FUNCTION_ALIGNMENT || !RISCV_ISA_C config HAVE_SHADOW_CALL_STACK def_bool $(cc-option,-fsanitize=shadow-call-stack) From patchwork Wed Nov 27 17:29:04 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13887253 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 9725CD6ACF7 for ; Wed, 27 Nov 2024 17:30:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mNGI6LV1h7hIo1JYa+ANlFNEl44e46ywhR/mUIhMsoo=; b=IIvRzNb5n3wYBo PtSwcwshaPBv6DkfQOlTEPJ5e7fViNbdaJWlbwUlZ3/k8/SuqOWzOIsw78jf4IluxwCqzJAQd7YGy 0K7XEBKDg5K/WLuipQ8vaueDbLoBra9PgORvpkrW5TAEwWbM1g1t+K7u4Ki4xeHGwdhRGT7R0nb/t pw9pciPrBxaf/jeQXhYKzaMoISCpRW+0oMNxcEj/R8CwK5i8I26wloVtaxnkpwbK2Qj/iCh099tC7 LjztGE2m5BNdXKpOnOmkRJDK+pFRO0OP9QfWRjxwq0iKwICFSWc62nDTI383LXje9LKS+J8OU6ATc 6gwE5WnuF/Rr+U7/Sosw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrb-0000000DkTf-0s4p; Wed, 27 Nov 2024 17:30:07 +0000 Received: from mail-ot1-x32d.google.com ([2607:f8b0:4864:20::32d]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrY-0000000DkSZ-3WFa for linux-riscv@lists.infradead.org; Wed, 27 Nov 2024 17:30:05 +0000 Received: by mail-ot1-x32d.google.com with SMTP id 46e09a7af769-71d5eabaec6so1027171a34.1 for ; Wed, 27 Nov 2024 09:30:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732728603; x=1733333403; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=1IR7fJnOIGqt6b1nWAQZJYrqcHCadpu+itzDI+rPODg=; b=MN+xatMcjx0YbB08WOm3XFldnQGu9YlJKazCoVt3v5aK5DUHjZuea/Tm8MxkMvPru8 FYpI42qxygd+f7MdepK6JuRdom33qZl5ZtNnixUypmMh1YmcHkP6D8e0s9U9OyZkWFcw 4Z8wcLcPYNdkA/WPs4FWWLqS/zuYkaH4tqjNk7sJqM0OGuBOs/R9MzR7kXmJHA1MJM6H QmGMKmgSIJ5WSwnJtMbkWp7+FFe3uTMARDGvN79P6q713SAcGiefWVSR1lkPMx0zwkxz K4fThntWBnNAE8MxJP/N8U3GW8YPq+5ctZVHtGVpePii/5U674NT99XyAH85HMJpZPcO LCfQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732728603; x=1733333403; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=1IR7fJnOIGqt6b1nWAQZJYrqcHCadpu+itzDI+rPODg=; b=uEiAA03Pv04Cj2HBSZxWy0ijNgC/ce2UBHyY+hpmlwhqR+w9tOi7c/JzTtYfkwRPG6 rAqCsQ9hX0x2+xIJ07kYB5YmsTWbiVd8GT164rLQCcVZfvGLu1u+CTDhKYQqoYVPlmVu /yNgW4YQ5wNxAjVTqp5wb18eNr0Lb41p2ZSLK5ohgSCuNWX7leWFJ8pR7RV9LG7JHR1x EhGJ9c3g/1Q9VY/W+CpSIBh/pq4/pRBeu6l97oZXI/gmqD0fHWclSXiMeHf45Te9xluh J6tYD6hym3wiyWiz4aNMvdqy5v9NZaCkt1mNaXjuHP1yg2hUm0xsYfCd873cws5wTm/l a0gQ== X-Forwarded-Encrypted: i=1; AJvYcCUZd9CrUxI8ZYSx2iI5uMOBpioETLgyLIq9gCRrJmIEP6+MaQl7TnYZmNO/SftudqUf8kae4jOcyOa4MQ==@lists.infradead.org X-Gm-Message-State: AOJu0Yz0KV68+I0C1iBkhUqha601Oe//C1HrXj137SDksgkLVWmRRw97 3v6/+ml8EMpFvQ4f4Mn9PCeXlvCm0D7RQ1OdningV8gYPOutgcTp X-Gm-Gg: ASbGnct6FrxkqEu8iGWZriIM8Vr1gxnrhRUSRIB1oCFY5o7Rcv8SiuOFmVK7j+Yn65p V1hZvXpGd/roG5uWQc3fXWxVs45X0hKRB7oZ8aZedrHQP4J6QCVB33noxOonvu2kHDD/pLNiWiS kHzwO042aEDx5rGFXo+h3N2n3zrDMzUHgj+GxvBrt/MgryREYBkkPbcKY+F5rpzrtU3sq6WNM5s rva7Z+8Woa9nFneiDCvmAC+/wafghXts6wztyBjLOMTjZlAPluM//aen3iRVFJj4oQX6tPcLvBv xHIdp5TQwPjJXswX4Sqqt89syMAU X-Google-Smtp-Source: AGHT+IHruXTFUTfiaojyTMlhAizY2Drw8OyNcxeu1vFbTUarM1m1Pec3RyTRvOVVRoAzScO0flWjxQ== X-Received: by 2002:a05:6830:6e90:b0:71d:4bd9:8b98 with SMTP id 46e09a7af769-71d65d07e94mr4292715a34.25.1732728603299; Wed, 27 Nov 2024 09:30:03 -0800 (PST) Received: from localhost.localdomain (1-171-29-17.dynamic-ip.hinet.net. [1.171.29.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fbcc1e3fdbsm9359582a12.30.2024.11.27.09.29.59 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 27 Nov 2024 09:30:02 -0800 (PST) From: Andy Chiu To: Steven Rostedt , Masami Hiramatsu , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, bjorn@rivosinc.com, puranjay12@gmail.com, alexghiti@rivosinc.com, yongxuan.wang@sifive.com, greentime.hu@sifive.com, nick.hu@sifive.com, nylon.chen@sifive.com, tommy.wu@sifive.com, eric.lin@sifive.com, viccent.chen@sifive.com, zong.li@sifive.com, samuel.holland@sifive.com Subject: [PATCH v3 3/7] riscv: ftrace: prepare ftrace for atomic code patching Date: Thu, 28 Nov 2024 01:29:04 +0800 Message-Id: <20241127172908.17149-4-andybnac@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20241127172908.17149-1-andybnac@gmail.com> References: <20241127172908.17149-1-andybnac@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241127_093004_880823_7A02B329 X-CRM114-Status: GOOD ( 21.93 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Andy Chiu We use an AUIPC+JALR pair to jump into a ftrace trampoline. Since instruction fetch can break down to 4 byte at a time, it is impossible to update two instructions without a race. In order to mitigate it, we initialize the patchable entry to AUIPC + NOP4. Then, the run-time code patching can change NOP4 to JALR to eable/disable ftrcae from a function. This limits the reach of each ftrace entry to +-2KB displacing from ftrace_caller. Starting from the trampoline, we add a level of indirection for it to reach ftrace caller target. Now, it loads the target address from a memory location, then perform the jump. This enable the kernel to update the target atomically. The ordering of reading/updating the targert address should be guarded by generic ftrace code, where it sends smp_rmb ipi. Signed-off-by: Andy Chiu --- arch/riscv/include/asm/ftrace.h | 4 ++ arch/riscv/kernel/ftrace.c | 80 +++++++++++++++++++++------------ arch/riscv/kernel/mcount-dyn.S | 9 ++-- 3 files changed, 62 insertions(+), 31 deletions(-) diff --git a/arch/riscv/include/asm/ftrace.h b/arch/riscv/include/asm/ftrace.h index 4ca7ce7f34d7..36734d285aad 100644 --- a/arch/riscv/include/asm/ftrace.h +++ b/arch/riscv/include/asm/ftrace.h @@ -80,6 +80,7 @@ struct dyn_arch_ftrace { #define JALR_T0 (0x000282e7) #define AUIPC_T0 (0x00000297) #define NOP4 (0x00000013) +#define JALR_RANGE (JALR_SIGN_MASK - 1) #define to_jalr_t0(offset) \ (((offset & JALR_OFFSET_MASK) << JALR_SHIFT) | JALR_T0) @@ -117,6 +118,9 @@ do { \ * Let auipc+jalr be the basic *mcount unit*, so we make it 8 bytes here. */ #define MCOUNT_INSN_SIZE 8 +#define MCOUNT_AUIPC_SIZE 4 +#define MCOUNT_JALR_SIZE 4 +#define MCOUNT_NOP4_SIZE 4 #ifndef __ASSEMBLY__ struct dyn_ftrace; diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c index 4b95c574fd04..5ebe412280ef 100644 --- a/arch/riscv/kernel/ftrace.c +++ b/arch/riscv/kernel/ftrace.c @@ -64,42 +64,64 @@ static int ftrace_check_current_call(unsigned long hook_pos, return 0; } -static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, - bool enable, bool ra) +static int __ftrace_modify_call(unsigned long hook_pos, unsigned long target, bool validate) { unsigned int call[2]; - unsigned int nops[2] = {NOP4, NOP4}; + unsigned int replaced[2]; + + make_call_t0(hook_pos, target, call); - if (ra) - make_call_ra(hook_pos, target, call); - else - make_call_t0(hook_pos, target, call); + if (validate) { + /* + * Read the text we want to modify; + * return must be -EFAULT on read error + */ + if (copy_from_kernel_nofault(replaced, (void *)hook_pos, + MCOUNT_INSN_SIZE)) + return -EFAULT; + + if (replaced[0] != call[0]) { + pr_err("%p: expected (%08x) but got (%08x)\n", + (void *)hook_pos, call[0], replaced[0]); + return -EINVAL; + } + } - /* Replace the auipc-jalr pair at once. Return -EPERM on write error. */ - if (patch_insn_write((void *)hook_pos, enable ? call : nops, MCOUNT_INSN_SIZE)) + /* Replace the jalr at once. Return -EPERM on write error. */ + if (patch_insn_write((void *)(hook_pos + MCOUNT_AUIPC_SIZE), call + 1, MCOUNT_JALR_SIZE)) return -EPERM; return 0; } -int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) +static int __ftrace_modify_call_site(ftrace_func_t *hook_pos, ftrace_func_t target, bool enable) { - unsigned int call[2]; + ftrace_func_t call = target; + ftrace_func_t nops = &ftrace_stub; - make_call_t0(rec->ip, addr, call); - - if (patch_insn_write((void *)rec->ip, call, MCOUNT_INSN_SIZE)) - return -EPERM; + WRITE_ONCE(*hook_pos, enable ? call : nops); return 0; } +int ftrace_make_call(struct dyn_ftrace *rec, unsigned long addr) +{ + unsigned long distance, orig_addr; + + orig_addr = (unsigned long)&ftrace_caller; + distance = addr > orig_addr ? addr - orig_addr : orig_addr - addr; + if (distance > JALR_RANGE) + return -EINVAL; + + return __ftrace_modify_call(rec->ip, addr, false); +} + int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, unsigned long addr) { - unsigned int nops[2] = {NOP4, NOP4}; + unsigned int nops[1] = {NOP4}; - if (patch_insn_write((void *)rec->ip, nops, MCOUNT_INSN_SIZE)) + if (patch_insn_write((void *)(rec->ip + MCOUNT_AUIPC_SIZE), nops, MCOUNT_NOP4_SIZE)) return -EPERM; return 0; @@ -114,21 +136,23 @@ int ftrace_make_nop(struct module *mod, struct dyn_ftrace *rec, */ int ftrace_init_nop(struct module *mod, struct dyn_ftrace *rec) { + unsigned int nops[2]; int out; + make_call_t0(rec->ip, &ftrace_caller, nops); + nops[1] = NOP4; + mutex_lock(&text_mutex); - out = ftrace_make_nop(mod, rec, MCOUNT_ADDR); + out = patch_insn_write((void *)rec->ip, nops, MCOUNT_INSN_SIZE); mutex_unlock(&text_mutex); return out; } +ftrace_func_t ftrace_call_dest = ftrace_stub; int ftrace_update_ftrace_func(ftrace_func_t func) { - int ret = __ftrace_modify_call((unsigned long)&ftrace_call, - (unsigned long)func, true, true); - - return ret; + return __ftrace_modify_call_site(&ftrace_call_dest, func, true); } struct ftrace_modify_param { @@ -182,7 +206,7 @@ int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr, if (ret) return ret; - return __ftrace_modify_call(caller, addr, true, false); + return __ftrace_modify_call(caller, addr, true); } #endif @@ -217,17 +241,17 @@ void ftrace_graph_func(unsigned long ip, unsigned long parent_ip, prepare_ftrace_return(&fregs->ra, ip, fregs->s0); } #else /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ -extern void ftrace_graph_call(void); +ftrace_func_t ftrace_graph_call_dest = ftrace_stub; int ftrace_enable_ftrace_graph_caller(void) { - return __ftrace_modify_call((unsigned long)&ftrace_graph_call, - (unsigned long)&prepare_ftrace_return, true, true); + return __ftrace_modify_call_site(&ftrace_graph_call_dest, + &prepare_ftrace_return, true); } int ftrace_disable_ftrace_graph_caller(void) { - return __ftrace_modify_call((unsigned long)&ftrace_graph_call, - (unsigned long)&prepare_ftrace_return, false, true); + return __ftrace_modify_call_site(&ftrace_graph_call_dest, + &prepare_ftrace_return, false); } #endif /* CONFIG_DYNAMIC_FTRACE_WITH_ARGS */ #endif /* CONFIG_DYNAMIC_FTRACE */ diff --git a/arch/riscv/kernel/mcount-dyn.S b/arch/riscv/kernel/mcount-dyn.S index e988bd26b28b..bc06e8ab81cf 100644 --- a/arch/riscv/kernel/mcount-dyn.S +++ b/arch/riscv/kernel/mcount-dyn.S @@ -162,7 +162,8 @@ SYM_FUNC_START(ftrace_caller) mv a3, sp SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) - call ftrace_stub + REG_L ra, ftrace_call_dest + jalr 0(ra) #ifdef CONFIG_FUNCTION_GRAPH_TRACER addi a0, sp, ABI_RA @@ -172,7 +173,8 @@ SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) mv a2, s0 #endif SYM_INNER_LABEL(ftrace_graph_call, SYM_L_GLOBAL) - call ftrace_stub + REG_L ra, ftrace_graph_call_dest + jalr 0(ra) #endif RESTORE_ABI jr t0 @@ -185,7 +187,8 @@ SYM_FUNC_START(ftrace_caller) PREPARE_ARGS SYM_INNER_LABEL(ftrace_call, SYM_L_GLOBAL) - call ftrace_stub + REG_L ra, ftrace_call_dest + jalr 0(ra) RESTORE_ABI_REGS bnez t1, .Ldirect From patchwork Wed Nov 27 17:29:05 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13887254 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id BD670D6ACF7 for ; Wed, 27 Nov 2024 17:30:15 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=CJKDc+ng8onFCrpYuphGkwEaWRH5J63KF6MOcQNkmsI=; b=uM3o1phnZ1WflJ o+dKWIYcDzRQF6TXG88PqsqQk1wcdc0Z5AIsI0iymAclGJ5w+QtfTsCOTKiZltR8q1vVZc5SDnamP LkUHBcc1moVkCg4MW6kwy7sIzwt98C5GrJJ75uS06al0VhQR4Yd7YpYec5NEza4w6qC7ntx3c2mU7 kopMQD96UQ9zxScdHFKBu9BD/qOOp+zh45UxHjoxBkIt3sLn0Kpjr6F64qpovndrlh5mGN+HJ3hwj LiQuS3NUei6faEHHMjSj+TA0nkk8oMdbh4fg1i4mHtMkWsrbJB1OlvgRKcGedYIRmL+3GArJBYWr/ 5wzb5TpBNdPV4tT+h1Bw==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tGLre-0000000DkW0-3V7y; Wed, 27 Nov 2024 17:30:10 +0000 Received: from mail-pg1-x52b.google.com ([2607:f8b0:4864:20::52b]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrd-0000000DkUT-0V0N for linux-riscv@lists.infradead.org; Wed, 27 Nov 2024 17:30:10 +0000 Received: by mail-pg1-x52b.google.com with SMTP id 41be03b00d2f7-7fbce800ee5so5090805a12.2 for ; Wed, 27 Nov 2024 09:30:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732728608; x=1733333408; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=IJ4wyksNmGKwRpxqsruWC4A8GzGFI32FmHe+iaxPpao=; b=TvleQo0LI3ScMpczEyTprgFTFIyYlT24Zq7L2b11MpE98vnyJxbMsTBvTngu3rpGvj aYDx7neN4jbuN66dEaMU5S/P13LXZ74LZDPGe21YFQuBHMcICT8xmoZnntcptjUYZHRx P5B/dD3f7LxJHimMF4sNQ9UquBapl9gm7JDirYyZhnZTpV4UfmhMh1TzttEK/TaxlD7j XtUAP9XmHZwjc/tQ1G+tHb/tYBamG8vY0Fah8gTZb1ij+4p6XqXkNsSbN3jEEkhgp8Cf tfiSgPs7rN63aEsiZxa3e7GC+EBewu5Iyxep87U0DnBb75/WmEFpB7CgNK3HQ0Y/daxq P7qQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732728608; x=1733333408; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=IJ4wyksNmGKwRpxqsruWC4A8GzGFI32FmHe+iaxPpao=; b=nGSFC7e8sGqPPlovE+AEtThn5df4U7KoBjKtgCozCXj9Tc02jCJmcAor2wd+vo110i SuT0C09LSEVLo0c6mE+KOOIbRrQ8wwEwkOrnY3nLyVUw/MvwWnnMm7fJofpP4g64bEMo VpjGyRvyQI/CLztN4bli8RSS1loDEmXEw/gFZt/nON03GctVIiycIyumU2j95+4xi5P6 7g3tajmUH2j5p8bat6PgfJgOjLp533AYjIFAx+nUMugvncQ4L1ihOvBBddq8oNkbW2pp sOYjwoZMaGsBliD9N/TylZxSrAXlIgMHPQ4hJ7KL7VdSB5Lk47izQu9ucy3lR1SVuQAM t9sg== X-Forwarded-Encrypted: i=1; AJvYcCVr/xQjMfskFv1zJFFSPon3ZLpevQfX/Gnd3yZrHJYGq7wqIdZ/9DyyFuypkMvSCBhoGrtus0MxlztZmw==@lists.infradead.org X-Gm-Message-State: AOJu0YyalBbH5So5mHCsVetSah9oAHytKNjCBVcBYIOkIYT3KJ7ZO4TO xr/gVC6LHm/zcW1yFObZot1aLYelqhcqoS/jREoDRaJfP7GO98fv X-Gm-Gg: ASbGncs4vWCTxtO2cRZO3Hbi0381sZDhyxCDPI8q8SLLM7jtVrz8ncinIsvhtcpEfvm ZefRn1BjIxzrmpyr+JOp1+nnyFQfrwKgrgeRbpxYO7/hg+6Rre6H+Z61yQ4araCeaHfRdLiFWW7 OsOzqvc8O14B05sQZIcAON/qoO0kUH332eyt/W9oFMkEbuNXVJX9yJ1RM92L2GeVb+cx1Zpn+aZ mnMri0j9FYg94RvNcnf8P9k/PW5IMVxP+J87T9yMGL/7RIBv0MV5mvekKzcT9S+7pE/gKamCq5N slkeQ7Cvtgkl5uuMU68o1J+98Agg X-Google-Smtp-Source: AGHT+IF4fq/f76pFPUxSVNMwuLs8cfvtvFw2AwTwF+Xja2556ySDeUDGNdUbpJFYW6y4k7ZnLfhsXg== X-Received: by 2002:a05:6a20:914b:b0:1e0:d123:7154 with SMTP id adf61e73a8af0-1e0e0ade54cmr5763536637.17.1732728607749; Wed, 27 Nov 2024 09:30:07 -0800 (PST) Received: from localhost.localdomain (1-171-29-17.dynamic-ip.hinet.net. [1.171.29.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fbcc1e3fdbsm9359582a12.30.2024.11.27.09.30.04 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 27 Nov 2024 09:30:07 -0800 (PST) From: Andy Chiu To: Steven Rostedt , Masami Hiramatsu , Mark Rutland , Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org, linux-riscv@lists.infradead.org, bjorn@rivosinc.com, puranjay12@gmail.com, alexghiti@rivosinc.com, yongxuan.wang@sifive.com, greentime.hu@sifive.com, nick.hu@sifive.com, nylon.chen@sifive.com, tommy.wu@sifive.com, eric.lin@sifive.com, viccent.chen@sifive.com, zong.li@sifive.com, samuel.holland@sifive.com Subject: [PATCH v3 4/7] riscv: ftrace: do not use stop_machine to update code Date: Thu, 28 Nov 2024 01:29:05 +0800 Message-Id: <20241127172908.17149-5-andybnac@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20241127172908.17149-1-andybnac@gmail.com> References: <20241127172908.17149-1-andybnac@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241127_093009_157042_A66B853C X-CRM114-Status: GOOD ( 13.95 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Andy Chiu Now it is safe to remove dependency from stop_machine() for us to patch code in ftrace. Signed-off-by: Andy Chiu --- arch/riscv/kernel/ftrace.c | 53 +++----------------------------------- 1 file changed, 4 insertions(+), 49 deletions(-) diff --git a/arch/riscv/kernel/ftrace.c b/arch/riscv/kernel/ftrace.c index 5ebe412280ef..57a6558e212e 100644 --- a/arch/riscv/kernel/ftrace.c +++ b/arch/riscv/kernel/ftrace.c @@ -13,23 +13,13 @@ #include #ifdef CONFIG_DYNAMIC_FTRACE -void ftrace_arch_code_modify_prepare(void) __acquires(&text_mutex) +void arch_ftrace_update_code(int command) { mutex_lock(&text_mutex); - - /* - * The code sequences we use for ftrace can't be patched while the - * kernel is running, so we need to use stop_machine() to modify them - * for now. This doesn't play nice with text_mutex, we use this flag - * to elide the check. - */ - riscv_patch_in_stop_machine = true; -} - -void ftrace_arch_code_modify_post_process(void) __releases(&text_mutex) -{ - riscv_patch_in_stop_machine = false; + command |= FTRACE_MAY_SLEEP; + ftrace_modify_all_code(command); mutex_unlock(&text_mutex); + flush_icache_all(); } static int ftrace_check_current_call(unsigned long hook_pos, @@ -155,41 +145,6 @@ int ftrace_update_ftrace_func(ftrace_func_t func) return __ftrace_modify_call_site(&ftrace_call_dest, func, true); } -struct ftrace_modify_param { - int command; - atomic_t cpu_count; -}; - -static int __ftrace_modify_code(void *data) -{ - struct ftrace_modify_param *param = data; - - if (atomic_inc_return(¶m->cpu_count) == num_online_cpus()) { - ftrace_modify_all_code(param->command); - /* - * Make sure the patching store is effective *before* we - * increment the counter which releases all waiting CPUs - * by using the release variant of atomic increment. The - * release pairs with the call to local_flush_icache_all() - * on the waiting CPU. - */ - atomic_inc_return_release(¶m->cpu_count); - } else { - while (atomic_read(¶m->cpu_count) <= num_online_cpus()) - cpu_relax(); - - local_flush_icache_all(); - } - - return 0; -} - -void arch_ftrace_update_code(int command) -{ - struct ftrace_modify_param param = { command, ATOMIC_INIT(0) }; - - stop_machine(__ftrace_modify_code, ¶m, cpu_online_mask); -} #endif #ifdef CONFIG_DYNAMIC_FTRACE_WITH_DIRECT_CALLS From patchwork Wed Nov 27 17:29:06 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13887255 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 363AFD6ACF9 for ; Wed, 27 Nov 2024 17:30:22 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=NKAr+qfLEKd7pDLeyY5HKm4InZv655DnEZsuCwfFNIs=; b=Oln3Znz3dtFQk9 HjTF1DVFiVT5phLUlR99uQZTMYsfsG24Om6JYE3DEfHC+X4uoAM/SlFCB9zR2S6UYN7SapmLQstyy bYJv0cDC1hl/vMUWehwMsqm58qh0ATEc+xs+E9S9hd5K+cPP0DoHly+dyUb3MMtID0uRWCfl/4dK5 teygVR9zrZuqyrFnB3LODOHnDvXWn180rl29nxyR+o3cPQQePCMQlKcF6QeLvN3zWCMc7BAGgvOgf GNAmBQsuPWWX7o1S8+n0Yi6qjDaSoTuEfucJqtGXEMgQ8mVymFLffQ9IMGN9jfi/w0LgfN0lsuMrX iq0heKfM6RRehM8Ium9w==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrl-0000000DkaF-1nRW; Wed, 27 Nov 2024 17:30:17 +0000 Received: from mail-pg1-x534.google.com ([2607:f8b0:4864:20::534]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrh-0000000DkXF-0ccb for linux-riscv@lists.infradead.org; Wed, 27 Nov 2024 17:30:15 +0000 Received: by mail-pg1-x534.google.com with SMTP id 41be03b00d2f7-7fbbe0fb0b8so5465226a12.0 for ; Wed, 27 Nov 2024 09:30:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732728612; x=1733333412; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=Hlixj4dxi1hv7t1eaE352YtuPYrxmNR32TDpbYYyUtI=; b=MJDsib9WBTdRvwrCNKHy99DUyuFr27l2kM8/Rop0rdJGH4fLiTDTALS0GSrQ55peu8 JjXZe7iCjw50x1eicKd0N+wC2ZEL8qYmvVrjBwGERoj1nbtBgvtpUbHW5ALiABVsgCN0 QnLpF0LoMtLozy2ytWnsmV/+dfdCajvh8zzd264TUzMzUg4tMorq5zQJ8toHFl4Rsli9 nFEZDs5Gm0yYsigMps48VXJ2/2gXKbEq3ri+19ZI8GS1dKkkBGUJ8U86acH1Zxnr1XYd EzxwsiUz7sK8ogHl4LW542bJt+L3Nb9VEQS3ny0yZmmpT3nPl4A+4Nci6eVm+H8pGmZv g0jg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732728612; x=1733333412; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=Hlixj4dxi1hv7t1eaE352YtuPYrxmNR32TDpbYYyUtI=; b=KQop1Ns1YwUOly8TlPEeVb/8sKQjA+XvLrEI14kmjpeKZSnRKz8HqSZ4xUZjkIJk+h tQmGw8FR66+BQ/LymoyO3cJyVl5XXuw2xLItpspH9qW6SPSqYX2Im08h+3HKu6WZPeeb tlCjwXr3tRA1b7x2xxZP06nhW24wrxtf6lLOKKq09MrSuAxUZMDsCCj4iy2YYAYuE2Fh LbFulFcJRFQpvWozUszocvdAUS9jSo49xS0VZajPL4Lk5MJ1JE3qAg3upF9F97WgA+ET tPs0vAnLq+BaZ5QiwOoeJfjOlvE7ybAZArtZC69LcPxLmMu4tBWWId/6MFCC7JqvBUnU borA== X-Gm-Message-State: AOJu0Ywf5Yivzh7+OhY6aFCRaMw4gSxCayGbitIzmMYRswxyarBKAdIZ B8FT9z8Y2Xe0qiesEC/5hfg0vkKCIZolDNFjh0HgjpZdX2HhUliVq1OeZrFJ X-Gm-Gg: ASbGncuTmnaTDNdrY2X2DKyLQO07136gdIidsbfrFpfZXdOr34yF0GRhGgaEiuI6DDD yo34c99GQVpP8CysQ+5Tcd7Qip3nZRwcgrtmU2hZDGe2vmOnvOenOd5QJoejstsU4YuApmxDRKY 1+qhBYXy7oMGDxf5LISuJCuLPYIx+RWMCAPbCde4EET6NLIMYxMvXZmgbYnPGZx7clHSYWIoAbp 5oDiISJnJMsHrniBA5JnHHmtm7Zlth7voGwnm++g4Rx5tYlXlZWNV619zb6HubZ+PagxZeUV5M6 6K/HISV+MwizYHth+NX6D4bpx6vs X-Google-Smtp-Source: AGHT+IG4NurKvJEqxA2cpsneqQG887+tFx88JjKBx3q3M02mZmw0BTNFpg0m60820Xm5RX2iTEn9oA== X-Received: by 2002:a05:6a21:e8c:b0:1e0:c56f:7da8 with SMTP id adf61e73a8af0-1e0e0aa83camr5657700637.4.1732728611949; Wed, 27 Nov 2024 09:30:11 -0800 (PST) Received: from localhost.localdomain (1-171-29-17.dynamic-ip.hinet.net. [1.171.29.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fbcc1e3fdbsm9359582a12.30.2024.11.27.09.30.09 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 27 Nov 2024 09:30:11 -0800 (PST) From: Andy Chiu To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, bjorn@rivosinc.com, puranjay12@gmail.com, alexghiti@rivosinc.com, yongxuan.wang@sifive.com, greentime.hu@sifive.com, nick.hu@sifive.com, nylon.chen@sifive.com, tommy.wu@sifive.com, eric.lin@sifive.com, viccent.chen@sifive.com, zong.li@sifive.com, samuel.holland@sifive.com Subject: [PATCH v3 5/7] riscv: vector: Support calling schedule() for preemptible Vector Date: Thu, 28 Nov 2024 01:29:06 +0800 Message-Id: <20241127172908.17149-6-andybnac@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20241127172908.17149-1-andybnac@gmail.com> References: <20241127172908.17149-1-andybnac@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241127_093013_184760_6E3B02D5 X-CRM114-Status: GOOD ( 15.16 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Andy Chiu Each function entry implies a call to ftrace infrastructure. And it may call into schedule in some cases. So, it is possible for preemptible kernel-mode Vector to implicitly call into schedule. Since all V-regs are caller-saved, it is possible to drop all V context when a thread voluntarily call schedule(). Besides, we currently don't pass argument through vector register, so we don't have to save/restore V-regs in ftrace trampoline. Signed-off-by: Andy Chiu --- arch/riscv/include/asm/processor.h | 5 +++++ arch/riscv/include/asm/vector.h | 22 +++++++++++++++++++--- 2 files changed, 24 insertions(+), 3 deletions(-) diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h index 5f56eb9d114a..9c1cc716b891 100644 --- a/arch/riscv/include/asm/processor.h +++ b/arch/riscv/include/asm/processor.h @@ -79,6 +79,10 @@ struct pt_regs; * Thus, the task does not own preempt_v. Any use of Vector will have to * save preempt_v, if dirty, and fallback to non-preemptible kernel-mode * Vector. + * - bit 29: The thread voluntarily calls schedule() while holding an active + * preempt_v. All preempt_v context should be dropped in such case because + * V-regs are caller-saved. Only sstatus.VS=ON is persisted across a + * schedule() call. * - bit 30: The in-kernel preempt_v context is saved, and requries to be * restored when returning to the context that owns the preempt_v. * - bit 31: The in-kernel preempt_v context is dirty, as signaled by the @@ -93,6 +97,7 @@ struct pt_regs; #define RISCV_PREEMPT_V 0x00000100 #define RISCV_PREEMPT_V_DIRTY 0x80000000 #define RISCV_PREEMPT_V_NEED_RESTORE 0x40000000 +#define RISCV_PREEMPT_V_IN_SCHEDULE 0x20000000 /* CPU-specific state of a task */ struct thread_struct { diff --git a/arch/riscv/include/asm/vector.h b/arch/riscv/include/asm/vector.h index c7c023afbacd..c5b6070db99f 100644 --- a/arch/riscv/include/asm/vector.h +++ b/arch/riscv/include/asm/vector.h @@ -76,6 +76,11 @@ static __always_inline void riscv_v_disable(void) csr_clear(CSR_SSTATUS, SR_VS); } +static __always_inline bool riscv_v_is_on(void) +{ + return !!(csr_read(CSR_SSTATUS) & SR_VS); +} + static __always_inline void __vstate_csr_save(struct __riscv_v_ext_state *dest) { asm volatile ( @@ -244,6 +249,11 @@ static inline void __switch_to_vector(struct task_struct *prev, struct pt_regs *regs; if (riscv_preempt_v_started(prev)) { + if (riscv_v_is_on()) { + WARN_ON(prev->thread.riscv_v_flags & RISCV_V_CTX_DEPTH_MASK); + riscv_v_disable(); + prev->thread.riscv_v_flags |= RISCV_PREEMPT_V_IN_SCHEDULE; + } if (riscv_preempt_v_dirty(prev)) { __riscv_v_vstate_save(&prev->thread.kernel_vstate, prev->thread.kernel_vstate.datap); @@ -254,10 +264,16 @@ static inline void __switch_to_vector(struct task_struct *prev, riscv_v_vstate_save(&prev->thread.vstate, regs); } - if (riscv_preempt_v_started(next)) - riscv_preempt_v_set_restore(next); - else + if (riscv_preempt_v_started(next)) { + if (next->thread.riscv_v_flags & RISCV_PREEMPT_V_IN_SCHEDULE) { + next->thread.riscv_v_flags &= ~RISCV_PREEMPT_V_IN_SCHEDULE; + riscv_v_enable(); + } else { + riscv_preempt_v_set_restore(next); + } + } else { riscv_v_vstate_set_restore(next, task_pt_regs(next)); + } } void riscv_v_vstate_ctrl_init(struct task_struct *tsk); From patchwork Wed Nov 27 17:29:07 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13887291 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 35426D6ACFD for ; Wed, 27 Nov 2024 18:37:04 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=lGJV8v39eSl1B0IihMBCQQJrpXtbSGpSZATWzfBVDas=; b=Bti1KkpUduE8GV 4pBI5O37atBKtOq5acVhL58II3Ns5gXveawwtkMDV+tdQTQlwqdWLUzylw8MmwGh8DiudaAIatBNd YYm4c6N//9ZlSRzPBWykroV4z2olKCL3zm2jqmTL/HtCsUNMKM66mubeS2Gl3EtVtIVBuiKjSOhZr VPVwiyQ1W4kH5PGXtFUrDaZ7My2IIUkJsyJpzghkoV2IjAi+9BMw9RtLzCvpFmMrqWK5TFRFZ89Af uwE4DCFsqSyJc2IpYrinCWmgpFWcvd94sOW4I4swPP32l61DNyBWNhuF64D73RxLqlI866rLfWeS+ 7sqbCFVAnQHpWT/vUOUA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tGMuF-0000000DqNG-2L7c; Wed, 27 Nov 2024 18:36:55 +0000 Received: from mail-pg1-x536.google.com ([2607:f8b0:4864:20::536]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrl-0000000Dka3-1BKW for linux-riscv@lists.infradead.org; Wed, 27 Nov 2024 17:30:18 +0000 Received: by mail-pg1-x536.google.com with SMTP id 41be03b00d2f7-7e6cbf6cd1dso4788464a12.3 for ; Wed, 27 Nov 2024 09:30:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732728616; x=1733333416; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=VbG7sBq4g0QTtfrKwDeWenWGnth2OE1nZlbtmRTgTgY=; b=kEfI91GajQCLiJxkWk3ZE0TW0w4cd56lm7PxiKmyY6ddMiRZsGEYQzjGxPib44OICv Np8WXFCRjInRqNzM/bGODog731yubxk9wK5lXlsSMdT4tndDz/kd826SNk8v/YAmkMoq g/jj+vxComKQjCqaZ9O95kba2tqYwQ/McVRkz0fHkodi34h2gIHgd752FTUwM25bIUz8 Ipn8VirbN8jcsuFm5EJTTDeQCa8nFkmvYzcJvQyB9lL99cAX2stXKFYuuBanrrkVClwU wVpQlaIjoftBSKyM7gixGwtxkSuaqNqYBomD+9j5qWq62W5DCAryRCRCFrlJ/qDciFhs mq2g== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732728616; x=1733333416; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=VbG7sBq4g0QTtfrKwDeWenWGnth2OE1nZlbtmRTgTgY=; b=KilMsLo7MkJFa3E6+WZtX/vrFtFT2d2UrohoKaaKSChKevcgWfiXqtccBaDkH8RHLt DTXs3TjOnCpIJfeHoQ7hF3+I/+ESx97O/ybN/zLMohcZnsxKSlCp2GcsPvqJeATiz2Wt TyvPiALCq6n/yE4K+CAt0tLs9NHoB+JqGe6i1efEurz/U0kSXmze651gPwP3sPXgYWKq TCsR1dgftifpZA56IOJ7PGsMelIap2YNLoXKYX7/mjtLU12PKkYXwcesoctYTYDorWq6 AQkMhNsdJUTITYejFYXY/ow+RSHk2MA0LGAxNbFunpTE+0S7DnSaMeP5UFkd6GZHw75M xu+w== X-Forwarded-Encrypted: i=1; AJvYcCUAmaqfQZEu4m5xfldMroY+PnPDMROkybbEx7tX3zFdybgYY5xE3t+kcLWX6A5+2aIwVr5mFcuF4KqcQQ==@lists.infradead.org X-Gm-Message-State: AOJu0YzW2y1FMYumKwko19HbkJ8qA5B9hpR+1OebZKcUBPn6irrX+ykB ++5abik+ixwfcnSqS5fBXbL6/WKnty8Bap8tiRA5F0Q5kI+JM1NI X-Gm-Gg: ASbGncuRN0HoRqdq7UxWLZzZA+mUvmhxUXtSUotwr/RTMtuep5luzVuUp3WxOPmpk7B uMvyaDPvoryvF+fHJ0cAGafqVRYvI1vlyaU9rstubC4n1Bw/1zVma0usFxd0F5c0KjYq5UQ8Ml4 RvNW+TuKwJlPpzr3nvA+y+cYqo1sFFNROACInlTYYq56qxIM9muCHan+JqVuboWq4fqQWj8pvyR sC5I9rwBH/VspN6PCiBQzyWuhvRnccN7QEBEI6WexQmyyh90QAyY59Zntv6rQGcYxY/StR1MFo5 JtDQYwqRU2gNiYI16KL/Y8xgb8yr X-Google-Smtp-Source: AGHT+IGTKYhqvbNwHQrWIr6lxxlwHTKRDAW2bOpNEEsjIU38gWo2Z6YtRqdQ+jNZ7IFsVssJEcCwIg== X-Received: by 2002:a05:6a21:3997:b0:1e0:ce11:b0ce with SMTP id adf61e73a8af0-1e0e0b90f7bmr8440657637.35.1732728616138; Wed, 27 Nov 2024 09:30:16 -0800 (PST) Received: from localhost.localdomain (1-171-29-17.dynamic-ip.hinet.net. [1.171.29.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fbcc1e3fdbsm9359582a12.30.2024.11.27.09.30.13 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 27 Nov 2024 09:30:15 -0800 (PST) From: Andy Chiu To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: Andy Chiu , linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, bjorn@rivosinc.com, puranjay12@gmail.com, alexghiti@rivosinc.com, yongxuan.wang@sifive.com, greentime.hu@sifive.com, nick.hu@sifive.com, nylon.chen@sifive.com, tommy.wu@sifive.com, eric.lin@sifive.com, viccent.chen@sifive.com, zong.li@sifive.com, samuel.holland@sifive.com Subject: [PATCH v3 6/7] riscv: add a data fence for CMODX in the kernel mode Date: Thu, 28 Nov 2024 01:29:07 +0800 Message-Id: <20241127172908.17149-7-andybnac@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20241127172908.17149-1-andybnac@gmail.com> References: <20241127172908.17149-1-andybnac@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241127_093017_882488_85D4DF85 X-CRM114-Status: GOOD ( 13.17 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org RISC-V spec explicitly calls out that a local fence.i is not enough for the code modification to be visble from a remote hart. In fact, it states: To make a store to instruction memory visible to all RISC-V harts, the writing hart also has to execute a data FENCE before requesting that all remote RISC-V harts execute a FENCE.I. Thus, add a fence here to order data writes before making the IPI. Signed-off-by: Andy Chiu --- arch/riscv/mm/cacheflush.c | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/arch/riscv/mm/cacheflush.c b/arch/riscv/mm/cacheflush.c index b81672729887..b2e4b81763f8 100644 --- a/arch/riscv/mm/cacheflush.c +++ b/arch/riscv/mm/cacheflush.c @@ -24,7 +24,20 @@ void flush_icache_all(void) if (num_online_cpus() < 2) return; - else if (riscv_use_sbi_for_rfence()) + + /* + * Make sure all previous writes to the D$ are ordered before making + * the IPI. The RISC-V spec states that a hart must execute a data fence + * before triggering a remote fence.i in order to make the modification + * visable for remote harts. + * + * IPIs on RISC-V are triggered by MMIO writes to either CLINT or + * S-IMSIC, so the fence ensures previous data writes "happen before" + * the MMIO. + */ + RISCV_FENCE(w, o); + + if (riscv_use_sbi_for_rfence()) sbi_remote_fence_i(NULL); else on_each_cpu(ipi_remote_fence_i, NULL, 1); From patchwork Wed Nov 27 17:29:08 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Chiu X-Patchwork-Id: 13887292 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 2294BD6ACFC for ; Wed, 27 Nov 2024 18:37:13 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-Id:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=NoNPYGpxoCVTXPIYKhv+hu1bNMIHtHjXsjiEXgiFULg=; b=p9cYXRDuHxRK5R 0g3JEXwvACRrpF/3vK1/TCOj7tqBkV69/+km9Al186rjuCuOM3lrIuS7kRJuH1OvS17udRKNYUJLr uX29v3zZGRnfJtQNKQgpmjT+NYwunBeYWGlMqdKtzXl9/XJVKqXClw3K2vMydiFNXCDTSmOZzbynQ VtnoGEl3sFyAJI0qMyVTfXX4BledXH6CQcj186rLcXibN0x37vUO1aKONdQ81Jc5gyDpp5VkcU+rg /kvmuYVxr3zTkUYkgTLuQFD54OJxeJb/xDRxoyVrurphgLTqpYkAFRGwkEJHUjpk4eTsvd/J+Gxld hhGG1GlZvA6q6uzIJRhA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1tGMuG-0000000DqNO-0LHB; Wed, 27 Nov 2024 18:36:56 +0000 Received: from desiato.infradead.org ([2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tGLru-0000000DkgN-1aNu for linux-riscv@bombadil.infradead.org; Wed, 27 Nov 2024 17:30:26 +0000 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=Content-Transfer-Encoding:MIME-Version :References:In-Reply-To:Message-Id:Date:Subject:Cc:To:From:Sender:Reply-To: Content-Type:Content-ID:Content-Description; bh=bdAeGv9r70I9AKk6TiTHcOSKMA9eOfxpzpLIhOQV/Tg=; b=qon9q409VFeV5im5FRD6ngQEaY fNOLBSjqMd83Ouudm3WwwKrm0IpdB+0TO9zqdIES7ofbEN2q9ZN3DMJM+X9X3aMbIrH70AHlRsyzd Egh6E6CtWkETb+t+4Bp6jrLjUes/JvoW47H0nnscCkSXenb6c62IrAR3z7w7T99SeyyIhHvAccbEn vIGdc7IMIyqeKYhmPqhrkbv/Chac1QWb530iMnJTdQznt2YGTka0/wbAoqVMO2bKMC0MnT1CDg08m 7X7qFJt1udSfcpgTLe9G5u8/EBUmTQZRif80Xt8XVD6bTtCybIcVyjoKR92kpjubDjNPvgapw42TF a4PP6UiA==; Received: from mail-pg1-x529.google.com ([2607:f8b0:4864:20::529]) by desiato.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1tGLrr-00000001Rii-1Xmi for linux-riscv@lists.infradead.org; Wed, 27 Nov 2024 17:30:25 +0000 Received: by mail-pg1-x529.google.com with SMTP id 41be03b00d2f7-7e6cbf6cd1dso4788526a12.3 for ; Wed, 27 Nov 2024 09:30:21 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1732728620; x=1733333420; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=bdAeGv9r70I9AKk6TiTHcOSKMA9eOfxpzpLIhOQV/Tg=; b=ZTmJQWS6RtZqK2BIMTl+p/iUIb8ojTKC27moED1Th0h6cXKRbOd9qYB1sJBm01+DjX jXVRmfle/qaVXX1ePiWALk7B5le47xdnFFqy70UXe0qAqsoPTc4B4GcreSOXvOrdpspv fkOYrYWxzKC9qUmjyOQ4iYyoiPO6DxM2E61IgJWhUGNb3IvMOvCa2VQyop4QjKIVw7+u z4sWdgcm3tpJMptSSPizMIYJ1nChAUUwN3Fx9ZnRKuK2jXX13vqBhmr9l6vLsZEo/6L3 gOUhluP6ivwKVj5TuowqYl0lgJ6S9LeGdwCNjO7AdWdy2N+a0iuxQ6ZPztYMZ5tNaxzq zbsA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732728620; x=1733333420; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=bdAeGv9r70I9AKk6TiTHcOSKMA9eOfxpzpLIhOQV/Tg=; b=VDuEHPl5nq44a42r+UX+u9K2YLWZqGXyv7OAJkn3SmiZwtFq0x83YJX9my8GMLXSPh xoOflMTlICRKqXBOJKWNF4+T5vFYrXsvmdOtXB53BevpmTt7kESydCQyh2iTzBA44eRn wQq6uxUPqJ1kHzZzW8J0hRu4JBSDHkcWMS86/6L9CMF4CXIqywj0l/xfJXy+Ff2fFGg0 nxekzBo7D8wpBY0RHovqnImE60IwHrFXc+TLeMIxSIr/FiyUQeOu0LwEkotJ0M++A5Y6 4OR4N+1I2dnV38x5ZLHt2lF571/UfjmUYVJ0GzCjCvFxB2+DuRxcV3hVJHB0RaWxgoiT j0Pg== X-Gm-Message-State: AOJu0YxBT4EqfMzFCv+TS8Ce9zvMv1ORVmyCE70EPUDUOuLEbikNkP4u 7ekL1Me/yXly69E9TrVUDfb1eKpjLj2iOVDuSHLY152S1ik4wzDW X-Gm-Gg: ASbGncsJt7Fi57lukxEVTm9RU6tgCacflt3lmGVbeqMWjNLXjOMYy003+M5xHbGeOB9 Sjqk+0TMD4Fc8g5tAymEXyzF/TXYTzjJeGMHX04qwZYr6mWb4/1mddP0KXAqogJdtLAL42wxxND /3pfyfoZ1eOpxpPFoE4XaghIL3d23vY+mNYvlXjGNjdlgxQ4Lipd+cJk+fXU/2VzRzQCWjt0ykp z0Wws3pdGdQxpqV1yyf7A3eLA3eVrUd/IyMvMh3X3dSS2DX/oFIzslTNOkA0H+aO1DbjkKMjrbw 6GmlegEgxdAS2PxpfJKuxVzMwYyH X-Google-Smtp-Source: AGHT+IFBuiAY3Y5+T9rXhE/jtMcxIwZT/PZK882Xa9jD4xmTITkIiWYoAEUJevk5le6u0m/B/dND3Q== X-Received: by 2002:a05:6a20:6a09:b0:1e0:cb94:d3b4 with SMTP id adf61e73a8af0-1e0e0beb9b1mr7629311637.45.1732728620072; Wed, 27 Nov 2024 09:30:20 -0800 (PST) Received: from localhost.localdomain (1-171-29-17.dynamic-ip.hinet.net. [1.171.29.17]) by smtp.gmail.com with ESMTPSA id 41be03b00d2f7-7fbcc1e3fdbsm9359582a12.30.2024.11.27.09.30.17 (version=TLS1_3 cipher=TLS_CHACHA20_POLY1305_SHA256 bits=256/256); Wed, 27 Nov 2024 09:30:19 -0800 (PST) From: Andy Chiu To: Paul Walmsley , Palmer Dabbelt , Albert Ou Cc: linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org, bjorn@rivosinc.com, puranjay12@gmail.com, alexghiti@rivosinc.com, yongxuan.wang@sifive.com, greentime.hu@sifive.com, nick.hu@sifive.com, nylon.chen@sifive.com, tommy.wu@sifive.com, eric.lin@sifive.com, viccent.chen@sifive.com, zong.li@sifive.com, samuel.holland@sifive.com Subject: [PATCH v3 7/7] riscv: ftrace: support PREEMPT Date: Thu, 28 Nov 2024 01:29:08 +0800 Message-Id: <20241127172908.17149-8-andybnac@gmail.com> X-Mailer: git-send-email 2.39.3 (Apple Git-145) In-Reply-To: <20241127172908.17149-1-andybnac@gmail.com> References: <20241127172908.17149-1-andybnac@gmail.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241127_173023_499354_44B949AC X-CRM114-Status: UNSURE ( 9.56 ) X-CRM114-Notice: Please train this message. X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org From: Andy Chiu Now, we can safely enable dynamic ftrace with kernel preemption. Signed-off-by: Andy Chiu --- arch/riscv/Kconfig | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index 75a5ebde4427..554e4e363c54 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -148,7 +148,7 @@ config RISCV select HAVE_FTRACE_MCOUNT_RECORD if !XIP_KERNEL select HAVE_FUNCTION_GRAPH_TRACER select HAVE_FUNCTION_GRAPH_RETVAL if HAVE_FUNCTION_GRAPH_TRACER - select HAVE_FUNCTION_TRACER if !XIP_KERNEL && !PREEMPTION + select HAVE_FUNCTION_TRACER if !XIP_KERNEL select HAVE_EBPF_JIT if MMU select HAVE_GUP_FAST if MMU select HAVE_FUNCTION_ARG_ACCESS_API