| Message ID | 20240606-tracepoint-v1-3-6551627bf51b@google.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Tracepoints and static branch/call in Rust | expand |
On 2024-06-06 11:05, Alice Ryhl wrote: > Make it possible to have Rust code call into tracepoints defined by C > code. It is still required that the tracepoint is declared in a C > header, and that this header is included in the input to bindgen. > > Signed-off-by: Alice Ryhl <aliceryhl@google.com> [...] > diff --git a/rust/helpers.c b/rust/helpers.c > index 2c37a0f5d7a8..0560cc2a512a 100644 > --- a/rust/helpers.c > +++ b/rust/helpers.c > @@ -165,6 +165,30 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) > } > EXPORT_SYMBOL_GPL(rust_helper_krealloc); > > +void rust_helper_preempt_enable_notrace(void) > +{ > + preempt_enable_notrace(); > +} > +EXPORT_SYMBOL_GPL(rust_helper_preempt_enable_notrace); > + > +void rust_helper_preempt_disable_notrace(void) > +{ > + preempt_disable_notrace(); > +} > +EXPORT_SYMBOL_GPL(rust_helper_preempt_disable_notrace); > + > +bool rust_helper_current_cpu_online(void) > +{ > + return cpu_online(raw_smp_processor_id()); > +} > +EXPORT_SYMBOL_GPL(rust_helper_current_cpu_online); > + > +void *rust_helper___rcu_dereference_raw(void **p) > +{ > + return rcu_dereference_raw(p); > +} > +EXPORT_SYMBOL_GPL(rust_helper___rcu_dereference_raw); Ouch. Doing a function call for each of those small operations will have a rather large performance impact when tracing is active. If it is not possible to inline those in Rust, then implementing __DO_TRACE in a C function would at least allow Rust to only do a single call to C rather than go back and forth between Rust and C. What prevents inlining those helpers in Rust ? Thanks, Mathieu
On Thu, Jun 06, 2024 at 11:30:03AM -0400, Mathieu Desnoyers wrote: > On 2024-06-06 11:05, Alice Ryhl wrote: > > Make it possible to have Rust code call into tracepoints defined by C > > code. It is still required that the tracepoint is declared in a C > > header, and that this header is included in the input to bindgen. > > > > Signed-off-by: Alice Ryhl <aliceryhl@google.com> > > [...] > > > diff --git a/rust/helpers.c b/rust/helpers.c > > index 2c37a0f5d7a8..0560cc2a512a 100644 > > --- a/rust/helpers.c > > +++ b/rust/helpers.c > > @@ -165,6 +165,30 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) > > } > > EXPORT_SYMBOL_GPL(rust_helper_krealloc); > > +void rust_helper_preempt_enable_notrace(void) > > +{ > > + preempt_enable_notrace(); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper_preempt_enable_notrace); > > + > > +void rust_helper_preempt_disable_notrace(void) > > +{ > > + preempt_disable_notrace(); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper_preempt_disable_notrace); > > + > > +bool rust_helper_current_cpu_online(void) > > +{ > > + return cpu_online(raw_smp_processor_id()); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper_current_cpu_online); > > + > > +void *rust_helper___rcu_dereference_raw(void **p) > > +{ > > + return rcu_dereference_raw(p); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper___rcu_dereference_raw); > > Ouch. Doing a function call for each of those small operations will > have a rather large performance impact when tracing is active. If it is Long-term plan is to 1) compile the C helpers in some IR and 2) inline the helpers with Rust in IR-level, as what Gary has: https://lore.kernel.org/rust-for-linux/20240529202817.3641974-1-gary@garyguo.net/ and I use it for the upcoming atomic API support: https://github.com/fbq/linux/tree/dev/rust/atomic-rfc and it works very well. Regards, Boqun > not possible to inline those in Rust, then implementing __DO_TRACE in > a C function would at least allow Rust to only do a single call to C > rather than go back and forth between Rust and C. > > What prevents inlining those helpers in Rust ? > > Thanks, > > Mathieu > > -- > Mathieu Desnoyers > EfficiOS Inc. > https://www.efficios.com >
On Thu, Jun 6, 2024 at 5:29 PM Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote: > > On 2024-06-06 11:05, Alice Ryhl wrote: > > Make it possible to have Rust code call into tracepoints defined by C > > code. It is still required that the tracepoint is declared in a C > > header, and that this header is included in the input to bindgen. > > > > Signed-off-by: Alice Ryhl <aliceryhl@google.com> > > [...] > > > diff --git a/rust/helpers.c b/rust/helpers.c > > index 2c37a0f5d7a8..0560cc2a512a 100644 > > --- a/rust/helpers.c > > +++ b/rust/helpers.c > > @@ -165,6 +165,30 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) > > } > > EXPORT_SYMBOL_GPL(rust_helper_krealloc); > > > > +void rust_helper_preempt_enable_notrace(void) > > +{ > > + preempt_enable_notrace(); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper_preempt_enable_notrace); > > + > > +void rust_helper_preempt_disable_notrace(void) > > +{ > > + preempt_disable_notrace(); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper_preempt_disable_notrace); > > + > > +bool rust_helper_current_cpu_online(void) > > +{ > > + return cpu_online(raw_smp_processor_id()); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper_current_cpu_online); > > + > > +void *rust_helper___rcu_dereference_raw(void **p) > > +{ > > + return rcu_dereference_raw(p); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper___rcu_dereference_raw); > > Ouch. Doing a function call for each of those small operations will > have a rather large performance impact when tracing is active. If it is > not possible to inline those in Rust, then implementing __DO_TRACE in > a C function would at least allow Rust to only do a single call to C > rather than go back and forth between Rust and C. > > What prevents inlining those helpers in Rust ? There's nothing fundamental that prevents it. But they contain a large amount of architecture specific code, which makes them a significant amount of work to reimplement in Rust. For example, rcu_dereference_raw calls into READ_ONCE. READ_ONCE is usually a volatile load, but under arm64+LTO, you get a bunch of inline assembly that relies on ALTERNATIVE for feature detection: https://elixir.bootlin.com/linux/v6.9/source/arch/arm64/include/asm/rwonce.h#L36 And preempt_enable_notrace has a similar story. The solution that Boqun mentions is nice, but it relies on rustc and clang using the same version of LLVM. You are unlikely to have compilers with matching LLVMs unless you intentionally take steps to make that happen. But yes, perhaps these helpers are an argument to have a single call for __DO_TRACE instead. Alice
On 2024-06-06 11:49, Boqun Feng wrote: > On Thu, Jun 06, 2024 at 11:30:03AM -0400, Mathieu Desnoyers wrote: >> On 2024-06-06 11:05, Alice Ryhl wrote: >>> Make it possible to have Rust code call into tracepoints defined by C >>> code. It is still required that the tracepoint is declared in a C >>> header, and that this header is included in the input to bindgen. >>> >>> Signed-off-by: Alice Ryhl <aliceryhl@google.com> >> >> [...] >> >>> diff --git a/rust/helpers.c b/rust/helpers.c >>> index 2c37a0f5d7a8..0560cc2a512a 100644 >>> --- a/rust/helpers.c >>> +++ b/rust/helpers.c >>> @@ -165,6 +165,30 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) >>> } >>> EXPORT_SYMBOL_GPL(rust_helper_krealloc); >>> +void rust_helper_preempt_enable_notrace(void) >>> +{ >>> + preempt_enable_notrace(); >>> +} >>> +EXPORT_SYMBOL_GPL(rust_helper_preempt_enable_notrace); >>> + >>> +void rust_helper_preempt_disable_notrace(void) >>> +{ >>> + preempt_disable_notrace(); >>> +} >>> +EXPORT_SYMBOL_GPL(rust_helper_preempt_disable_notrace); >>> + >>> +bool rust_helper_current_cpu_online(void) >>> +{ >>> + return cpu_online(raw_smp_processor_id()); >>> +} >>> +EXPORT_SYMBOL_GPL(rust_helper_current_cpu_online); >>> + >>> +void *rust_helper___rcu_dereference_raw(void **p) >>> +{ >>> + return rcu_dereference_raw(p); >>> +} >>> +EXPORT_SYMBOL_GPL(rust_helper___rcu_dereference_raw); >> >> Ouch. Doing a function call for each of those small operations will >> have a rather large performance impact when tracing is active. If it is > > Long-term plan is to 1) compile the C helpers in some IR and 2) inline > the helpers with Rust in IR-level, as what Gary has: > > https://lore.kernel.org/rust-for-linux/20240529202817.3641974-1-gary@garyguo.net/ > > and I use it for the upcoming atomic API support: > > https://github.com/fbq/linux/tree/dev/rust/atomic-rfc > > and it works very well. Thanks for the pointers, it makes sense. Thanks, Mathieu > > Regards, > Boqun > >> not possible to inline those in Rust, then implementing __DO_TRACE in >> a C function would at least allow Rust to only do a single call to C >> rather than go back and forth between Rust and C. >> >> What prevents inlining those helpers in Rust ? >> >> Thanks, >> >> Mathieu >> >> -- >> Mathieu Desnoyers >> EfficiOS Inc. >> https://www.efficios.com >>
On 2024-06-06 12:16, Alice Ryhl wrote: > On Thu, Jun 6, 2024 at 5:29 PM Mathieu Desnoyers > <mathieu.desnoyers@efficios.com> wrote: >> >> On 2024-06-06 11:05, Alice Ryhl wrote: >>> Make it possible to have Rust code call into tracepoints defined by C >>> code. It is still required that the tracepoint is declared in a C >>> header, and that this header is included in the input to bindgen. >>> >>> Signed-off-by: Alice Ryhl <aliceryhl@google.com> >> >> [...] >> >>> diff --git a/rust/helpers.c b/rust/helpers.c >>> index 2c37a0f5d7a8..0560cc2a512a 100644 >>> --- a/rust/helpers.c >>> +++ b/rust/helpers.c >>> @@ -165,6 +165,30 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) >>> } >>> EXPORT_SYMBOL_GPL(rust_helper_krealloc); >>> >>> +void rust_helper_preempt_enable_notrace(void) >>> +{ >>> + preempt_enable_notrace(); >>> +} >>> +EXPORT_SYMBOL_GPL(rust_helper_preempt_enable_notrace); >>> + >>> +void rust_helper_preempt_disable_notrace(void) >>> +{ >>> + preempt_disable_notrace(); >>> +} >>> +EXPORT_SYMBOL_GPL(rust_helper_preempt_disable_notrace); >>> + >>> +bool rust_helper_current_cpu_online(void) >>> +{ >>> + return cpu_online(raw_smp_processor_id()); >>> +} >>> +EXPORT_SYMBOL_GPL(rust_helper_current_cpu_online); >>> + >>> +void *rust_helper___rcu_dereference_raw(void **p) >>> +{ >>> + return rcu_dereference_raw(p); >>> +} >>> +EXPORT_SYMBOL_GPL(rust_helper___rcu_dereference_raw); >> >> Ouch. Doing a function call for each of those small operations will >> have a rather large performance impact when tracing is active. If it is >> not possible to inline those in Rust, then implementing __DO_TRACE in >> a C function would at least allow Rust to only do a single call to C >> rather than go back and forth between Rust and C. >> >> What prevents inlining those helpers in Rust ? > > There's nothing fundamental that prevents it. But they contain a large > amount of architecture specific code, which makes them a significant > amount of work to reimplement in Rust. > > For example, rcu_dereference_raw calls into READ_ONCE. READ_ONCE is > usually a volatile load, but under arm64+LTO, you get a bunch of > inline assembly that relies on ALTERNATIVE for feature detection: > https://elixir.bootlin.com/linux/v6.9/source/arch/arm64/include/asm/rwonce.h#L36 > > And preempt_enable_notrace has a similar story. > > The solution that Boqun mentions is nice, but it relies on rustc and > clang using the same version of LLVM. You are unlikely to have > compilers with matching LLVMs unless you intentionally take steps to > make that happen. > > But yes, perhaps these helpers are an argument to have a single call > for __DO_TRACE instead. If those helpers end up being inlined into Rust with the solution pointed to by Boqun, then it makes sense to implement __DO_TRACE in Rust. Otherwise doing a single call to C would be more efficient than calling each of the helpers individually. Thanks, Mathieu > > Alice
On Thu, Jun 06, 2024 at 03:05:26PM +0000, Alice Ryhl wrote: > Make it possible to have Rust code call into tracepoints defined by C > code. It is still required that the tracepoint is declared in a C > header, and that this header is included in the input to bindgen. > > Signed-off-by: Alice Ryhl <aliceryhl@google.com> > --- > rust/bindings/bindings_helper.h | 1 + > rust/bindings/lib.rs | 15 +++++++ > rust/helpers.c | 24 +++++++++++ > rust/kernel/lib.rs | 1 + > rust/kernel/tracepoint.rs | 92 +++++++++++++++++++++++++++++++++++++++++ > 5 files changed, 133 insertions(+) > > diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h > index ddb5644d4fd9..d442f9ccfc2c 100644 > --- a/rust/bindings/bindings_helper.h > +++ b/rust/bindings/bindings_helper.h > @@ -15,6 +15,7 @@ > #include <linux/refcount.h> > #include <linux/sched.h> > #include <linux/slab.h> > +#include <linux/tracepoint.h> > #include <linux/wait.h> > #include <linux/workqueue.h> > > diff --git a/rust/bindings/lib.rs b/rust/bindings/lib.rs > index 40ddaee50d8b..48856761d682 100644 > --- a/rust/bindings/lib.rs > +++ b/rust/bindings/lib.rs > @@ -48,3 +48,18 @@ mod bindings_helper { > } > > pub use bindings_raw::*; > + > +/// Rust version of the C macro `rcu_dereference_raw`. > +/// > +/// The rust helper only works with void pointers, but this wrapper method makes it work with any > +/// pointer type using pointer casts. > +/// > +/// # Safety > +/// > +/// This method has the same safety requirements as the C macro of the same name. > +#[inline(always)] > +pub unsafe fn rcu_dereference_raw<T>(p: *const *mut T) -> *mut T { > + // SAFETY: This helper calls into the C macro, so the caller promises to uphold the safety > + // requirements. > + unsafe { __rcu_dereference_raw(p as *mut *mut _) as *mut T } > +} > diff --git a/rust/helpers.c b/rust/helpers.c > index 2c37a0f5d7a8..0560cc2a512a 100644 > --- a/rust/helpers.c > +++ b/rust/helpers.c > @@ -165,6 +165,30 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) > } > EXPORT_SYMBOL_GPL(rust_helper_krealloc); > > +void rust_helper_preempt_enable_notrace(void) > +{ > + preempt_enable_notrace(); > +} > +EXPORT_SYMBOL_GPL(rust_helper_preempt_enable_notrace); > + > +void rust_helper_preempt_disable_notrace(void) > +{ > + preempt_disable_notrace(); > +} > +EXPORT_SYMBOL_GPL(rust_helper_preempt_disable_notrace); A notrace wrapper that is tracable, lol. > +bool rust_helper_current_cpu_online(void) > +{ > + return cpu_online(raw_smp_processor_id()); > +} > +EXPORT_SYMBOL_GPL(rust_helper_current_cpu_online); > + > +void *rust_helper___rcu_dereference_raw(void **p) > +{ > + return rcu_dereference_raw(p); > +} > +EXPORT_SYMBOL_GPL(rust_helper___rcu_dereference_raw); I'm going to keep yelling and objecting to these wrappers. Fix bindgen already. Or whatever is needed to get it to interoperate with C inline / asm. NAK NAK NAK
On Thu, Jun 06, 2024 at 08:49:06AM -0700, Boqun Feng wrote: > Long-term plan is to 1) compile the C helpers in some IR and 2) inline > the helpers with Rust in IR-level, as what Gary has: > > https://lore.kernel.org/rust-for-linux/20240529202817.3641974-1-gary@garyguo.net/ Urgh, that still needs us to maintain that silly list of helpers :-/ Can't we pretty please have clang parse the actual header files into IR and munge that into rust? So that we don't get to manually duplicate everything+dog.
On Thu, Jun 06, 2024 at 07:35:44PM +0200, Peter Zijlstra wrote: > On Thu, Jun 06, 2024 at 08:49:06AM -0700, Boqun Feng wrote: > > > Long-term plan is to 1) compile the C helpers in some IR and 2) inline > > the helpers with Rust in IR-level, as what Gary has: > > > > https://lore.kernel.org/rust-for-linux/20240529202817.3641974-1-gary@garyguo.net/ > > Urgh, that still needs us to maintain that silly list of helpers :-/ > But it's an improvement from the current stage, right? ;-) > Can't we pretty please have clang parse the actual header files into IR > and munge that into rust? So that we don't get to manually duplicate > everything+dog. That won't always work, because some of our kernel APIs are defined as macros, and I don't think it's a trivial job to generate a macro definition to a function definition so that it can be translated to something in IR. We will have to do the macro -> function mapping ourselves somewhere, if we want to inline the API across languages. Regards, Boqun
On Thu, Jun 06, 2024 at 12:00:36PM -0700, Boqun Feng wrote: > On Thu, Jun 06, 2024 at 07:35:44PM +0200, Peter Zijlstra wrote: > > On Thu, Jun 06, 2024 at 08:49:06AM -0700, Boqun Feng wrote: > > > > > Long-term plan is to 1) compile the C helpers in some IR and 2) inline > > > the helpers with Rust in IR-level, as what Gary has: > > > > > > https://lore.kernel.org/rust-for-linux/20240529202817.3641974-1-gary@garyguo.net/ > > > > Urgh, that still needs us to maintain that silly list of helpers :-/ > > > > But it's an improvement from the current stage, right? ;-) Somewhat, but only marginal. > > Can't we pretty please have clang parse the actual header files into IR > > and munge that into rust? So that we don't get to manually duplicate > > everything+dog. > > That won't always work, because some of our kernel APIs are defined as > macros, and I don't think it's a trivial job to generate a macro > definition to a function definition so that it can be translated to > something in IR. We will have to do the macro -> function mapping > ourselves somewhere, if we want to inline the API across languages. We can try and see how far we can get with moving a bunch of stuff into inlines. There's quite a bit of simple CPP that could be inlines or const objects I suppose. Things like the tracepoints are of course glorious CPP abuse and are never going to work. But perhaps you can have an explicit 'eval-CPP on this here' construct or whatnot. If I squit I see this paste! thingy (WTF's up with that ! operator?) to munge function names in the static_call thing. So something like apply CPP from over there on this here can also be done :-)
On Thu, Jun 06, 2024 at 09:29:51PM +0200, Peter Zijlstra wrote: > On Thu, Jun 06, 2024 at 12:00:36PM -0700, Boqun Feng wrote: > > On Thu, Jun 06, 2024 at 07:35:44PM +0200, Peter Zijlstra wrote: > > > On Thu, Jun 06, 2024 at 08:49:06AM -0700, Boqun Feng wrote: > > > > > > > Long-term plan is to 1) compile the C helpers in some IR and 2) inline > > > > the helpers with Rust in IR-level, as what Gary has: > > > > > > > > https://lore.kernel.org/rust-for-linux/20240529202817.3641974-1-gary@garyguo.net/ > > > > > > Urgh, that still needs us to maintain that silly list of helpers :-/ > > > > > > > But it's an improvement from the current stage, right? ;-) > > Somewhat, but only marginal. > > > > Can't we pretty please have clang parse the actual header files into IR > > > and munge that into rust? So that we don't get to manually duplicate > > > everything+dog. > > > > That won't always work, because some of our kernel APIs are defined as > > macros, and I don't think it's a trivial job to generate a macro > > definition to a function definition so that it can be translated to > > something in IR. We will have to do the macro -> function mapping > > ourselves somewhere, if we want to inline the API across languages. > > We can try and see how far we can get with moving a bunch of stuff into > inlines. There's quite a bit of simple CPP that could be inlines or > const objects I suppose. > We can, but I'd first stick with what we have, improve it and make it stable until we go to the next stage. Plus, there's benefit of keeping an explicit helper list: it's clear what APIs are called by Rust, and moreover, it's easier to modify the helpers if you were to change an API, other than chasing where Rust code calls it. (Don't make me wrong, I'm happy if you want to do that ;-)) Regards, Boqun > Things like the tracepoints are of course glorious CPP abuse and are > never going to work. > > But perhaps you can have an explicit 'eval-CPP on this here' construct > or whatnot. If I squit I see this paste! thingy (WTF's up with that ! > operator?) to munge function names in the static_call thing. So > something like apply CPP from over there on this here can also be done > :-)
diff --git a/rust/bindings/bindings_helper.h b/rust/bindings/bindings_helper.h index ddb5644d4fd9..d442f9ccfc2c 100644 --- a/rust/bindings/bindings_helper.h +++ b/rust/bindings/bindings_helper.h @@ -15,6 +15,7 @@ #include <linux/refcount.h> #include <linux/sched.h> #include <linux/slab.h> +#include <linux/tracepoint.h> #include <linux/wait.h> #include <linux/workqueue.h> diff --git a/rust/bindings/lib.rs b/rust/bindings/lib.rs index 40ddaee50d8b..48856761d682 100644 --- a/rust/bindings/lib.rs +++ b/rust/bindings/lib.rs @@ -48,3 +48,18 @@ mod bindings_helper { } pub use bindings_raw::*; + +/// Rust version of the C macro `rcu_dereference_raw`. +/// +/// The rust helper only works with void pointers, but this wrapper method makes it work with any +/// pointer type using pointer casts. +/// +/// # Safety +/// +/// This method has the same safety requirements as the C macro of the same name. +#[inline(always)] +pub unsafe fn rcu_dereference_raw<T>(p: *const *mut T) -> *mut T { + // SAFETY: This helper calls into the C macro, so the caller promises to uphold the safety + // requirements. + unsafe { __rcu_dereference_raw(p as *mut *mut _) as *mut T } +} diff --git a/rust/helpers.c b/rust/helpers.c index 2c37a0f5d7a8..0560cc2a512a 100644 --- a/rust/helpers.c +++ b/rust/helpers.c @@ -165,6 +165,30 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) } EXPORT_SYMBOL_GPL(rust_helper_krealloc); +void rust_helper_preempt_enable_notrace(void) +{ + preempt_enable_notrace(); +} +EXPORT_SYMBOL_GPL(rust_helper_preempt_enable_notrace); + +void rust_helper_preempt_disable_notrace(void) +{ + preempt_disable_notrace(); +} +EXPORT_SYMBOL_GPL(rust_helper_preempt_disable_notrace); + +bool rust_helper_current_cpu_online(void) +{ + return cpu_online(raw_smp_processor_id()); +} +EXPORT_SYMBOL_GPL(rust_helper_current_cpu_online); + +void *rust_helper___rcu_dereference_raw(void **p) +{ + return rcu_dereference_raw(p); +} +EXPORT_SYMBOL_GPL(rust_helper___rcu_dereference_raw); + /* * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can * use it in contexts where Rust expects a `usize` like slice (array) indices. diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 22e1fedd0774..3f3b280bb437 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -46,6 +46,7 @@ pub mod sync; pub mod task; pub mod time; +pub mod tracepoint; pub mod types; pub mod workqueue; diff --git a/rust/kernel/tracepoint.rs b/rust/kernel/tracepoint.rs new file mode 100644 index 000000000000..d628ae71fc58 --- /dev/null +++ b/rust/kernel/tracepoint.rs @@ -0,0 +1,92 @@ +// SPDX-License-Identifier: GPL-2.0 + +// Copyright (C) 2024 Google LLC. + +//! Logic for tracepoints. + +/// Declare the Rust entry point for a tracepoint. +#[macro_export] +macro_rules! declare_trace { + ($($(#[$attr:meta])* $pub:vis fn $name:ident($($argname:ident : $argtyp:ty),* $(,)?);)*) => {$( + $( #[$attr] )* + #[inline(always)] + $pub unsafe fn $name($($argname : $argtyp),*) { + #[cfg(CONFIG_TRACEPOINTS)] + { + use $crate::bindings::*; + + // SAFETY: This macro only compiles if $name is a real tracepoint, and if it is a + // real tracepoint, then it is okay to query the static key. + let should_trace = unsafe { + $crate::macros::paste! { + $crate::static_key::static_key_false!( + [< __tracepoint_ $name >], + $crate::bindings::tracepoint, + key + ) + } + }; + + if should_trace { + $crate::tracepoint::do_trace!($name($($argname : $argtyp),*), cond); + } + } + + #[cfg(not(CONFIG_TRACEPOINTS))] + { + // If tracepoints are disabled, insert a trivial use of each argument + // to avoid unused argument warnings. + $( let _unused = $argname; )* + } + } + )*} +} + +#[doc(hidden)] +#[macro_export] +macro_rules! do_trace { + ($name:ident($($argname:ident : $argtyp:ty),* $(,)?), $cond:expr) => {{ + if !$crate::bindings::current_cpu_online() { + return; + } + + // SAFETY: This call is balanced with the call below. + unsafe { $crate::bindings::preempt_disable_notrace() }; + + // SAFETY: This calls the tracepoint with the provided arguments. The caller of the Rust + // wrapper guarantees that this is okay. + #[cfg(CONFIG_HAVE_STATIC_CALL)] + unsafe { + let it_func_ptr: *mut $crate::bindings::tracepoint_func = + $crate::bindings::rcu_dereference_raw( + ::core::ptr::addr_of!( + $crate::macros::concat_idents!(__tracepoint_, $name).funcs + ) + ); + + if !it_func_ptr.is_null() { + let __data = (*it_func_ptr).data; + $crate::macros::paste! { + $crate::static_call::static_call! { + [< tp_func_ $name >] (__data, $($argname),*) + }; + } + } + } + + // SAFETY: This calls the tracepoint with the provided arguments. The caller of the Rust + // wrapper guarantees that this is okay. + #[cfg(not(CONFIG_HAVE_STATIC_CALL))] + unsafe { + $crate::macros::concat_idents!(__traceiter_, $name)( + ::core::ptr::null_mut(), + $($argname),* + ); + } + + // SAFETY: This call is balanced with the call above. + unsafe { $crate::bindings::preempt_enable_notrace() }; + }} +} + +pub use {declare_trace, do_trace};
Make it possible to have Rust code call into tracepoints defined by C code. It is still required that the tracepoint is declared in a C header, and that this header is included in the input to bindgen. Signed-off-by: Alice Ryhl <aliceryhl@google.com> --- rust/bindings/bindings_helper.h | 1 + rust/bindings/lib.rs | 15 +++++++ rust/helpers.c | 24 +++++++++++ rust/kernel/lib.rs | 1 + rust/kernel/tracepoint.rs | 92 +++++++++++++++++++++++++++++++++++++++++ 5 files changed, 133 insertions(+)