Message ID | 20240618234025.15036-8-dakr@redhat.com (mailing list archive) |
---|---|
State | Handled Elsewhere |
Delegated to: | Bjorn Helgaas |
Headers | show |
Series | Device / Driver and PCI Rust abstractions | expand |
On Wed, Jun 19, 2024 at 01:39:53AM +0200, Danilo Krummrich wrote: > I/O memory is typically either mapped through direct calls to ioremap() > or subsystem / bus specific ones such as pci_iomap(). > > Even though subsystem / bus specific functions to map I/O memory are > based on ioremap() / iounmap() it is not desirable to re-implement them > in Rust. Why not? > Instead, implement a base type for I/O mapped memory, which generically > provides the corresponding accessors, such as `Io::readb` or > `Io:try_readb`. It provides a subset of the existing accessors, one you might want to trim down for now, see below... > +/* io.h */ > +u8 rust_helper_readb(const volatile void __iomem *addr) > +{ > + return readb(addr); > +} > +EXPORT_SYMBOL_GPL(rust_helper_readb); <snip> You provide wrappers for a subset of what io.h provides, why that specific subset? Why not just add what you need, when you need it? I doubt you need all of these, and odds are you will need more. > +u32 rust_helper_readl_relaxed(const volatile void __iomem *addr) > +{ > + return readl_relaxed(addr); > +} > +EXPORT_SYMBOL_GPL(rust_helper_readl_relaxed); I know everyone complains about wrapper functions around inline functions, so I'll just say it again, this is horrid. And it's going to hurt performance, so any rust code people write is not on a level playing field here. Your call, but ick... > +#ifdef CONFIG_64BIT > +u64 rust_helper_readq_relaxed(const volatile void __iomem *addr) > +{ > + return readq_relaxed(addr); > +} > +EXPORT_SYMBOL_GPL(rust_helper_readq_relaxed); > +#endif Rust works on 32bit targets in the kernel now? > +macro_rules! define_read { > + ($(#[$attr:meta])* $name:ident, $try_name:ident, $type_name:ty) => { > + /// Read IO data from a given offset known at compile time. > + /// > + /// Bound checks are performed on compile time, hence if the offset is not known at compile > + /// time, the build will fail. offsets aren't know at compile time for many implementations, as it could be a dynamically allocated memory range. How is this going to work for that? Heck, how does this work for DT-defined memory ranges today? thanks, greg k-h
On Thu, 2024-06-20 at 16:53 +0200, Greg KH wrote: > On Wed, Jun 19, 2024 at 01:39:53AM +0200, Danilo Krummrich wrote: > > I/O memory is typically either mapped through direct calls to > > ioremap() > > or subsystem / bus specific ones such as pci_iomap(). > > > > Even though subsystem / bus specific functions to map I/O memory > > are > > based on ioremap() / iounmap() it is not desirable to re-implement > > them > > in Rust. > > Why not? Because you'd then up reimplementing all that logic that the C code already provides. In the worst case that could lead to you effectively reimplemting the subsystem instead of wrapping it. And that's obviously uncool because you'd then have two of them (besides, the community in general rightfully pushes back against reimplementing stuff; see the attempts to provide redundant Rust drivers in the past). The C code already takes care of figuring out region ranges and all that, and it's battle hardened. The main point of Rust is to make things safer; so if that can be achieved without rewrite, as is the case with the presented container solution, that's the way to go. > > > Instead, implement a base type for I/O mapped memory, which > > generically > > provides the corresponding accessors, such as `Io::readb` or > > `Io:try_readb`. > > It provides a subset of the existing accessors, one you might want to > trim down for now, see below... > > > +/* io.h */ > > +u8 rust_helper_readb(const volatile void __iomem *addr) > > +{ > > + return readb(addr); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper_readb); > > <snip> > > You provide wrappers for a subset of what io.h provides, why that > specific subset? > > Why not just add what you need, when you need it? I doubt you need > all > of these, and odds are you will need more. > That was written by me as a first play set to test. Nova itself currently reads only 8 byte from a PCI BAR, so we could indeed drop everything but readq() for now and add things subsequently later, as you suggest. > > +u32 rust_helper_readl_relaxed(const volatile void __iomem *addr) > > +{ > > + return readl_relaxed(addr); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper_readl_relaxed); > > I know everyone complains about wrapper functions around inline > functions, so I'll just say it again, this is horrid. And it's going > to > hurt performance, so any rust code people write is not on a level > playing field here. > > Your call, but ick... Well, can anyone think of another way to do it? > > > +#ifdef CONFIG_64BIT > > +u64 rust_helper_readq_relaxed(const volatile void __iomem *addr) > > +{ > > + return readq_relaxed(addr); > > +} > > +EXPORT_SYMBOL_GPL(rust_helper_readq_relaxed); > > +#endif > > Rust works on 32bit targets in the kernel now? Ahm, afaik not. That's some relic. Let's address that with your subset comment from above. > > > +macro_rules! define_read { > > + ($(#[$attr:meta])* $name:ident, $try_name:ident, > > $type_name:ty) => { > > + /// Read IO data from a given offset known at compile > > time. > > + /// > > + /// Bound checks are performed on compile time, hence if > > the offset is not known at compile > > + /// time, the build will fail. > > offsets aren't know at compile time for many implementations, as it > could be a dynamically allocated memory range. How is this going to > work for that? Heck, how does this work for DT-defined memory ranges > today? The macro below will take care of those where it's only knowable at runtime I think. Rust has this feature (called "const generic") that can be used for APIs where ranges which are known at compile time, so the compiler can check all the parameters at that point. That has been judged to be positive because errors with the range handling become visible before the kernel runs and because it gives some performance advantages. P. > > thanks, > > greg k-h >
On Fri, Jun 21, 2024 at 11:43:34AM +0200, Philipp Stanner wrote: Please find a few additions below. But as mentioned, please let us sort out [1] first. [1] https://lore.kernel.org/lkml/ZnSeAZu3IMA4fR8P@cassiopeiae/ > On Thu, 2024-06-20 at 16:53 +0200, Greg KH wrote: > > On Wed, Jun 19, 2024 at 01:39:53AM +0200, Danilo Krummrich wrote: > > > I/O memory is typically either mapped through direct calls to > > > ioremap() > > > or subsystem / bus specific ones such as pci_iomap(). > > > > > > Even though subsystem / bus specific functions to map I/O memory > > > are > > > based on ioremap() / iounmap() it is not desirable to re-implement > > > them > > > in Rust. > > > > Why not? > > Because you'd then up reimplementing all that logic that the C code > already provides. In the worst case that could lead to you effectively > reimplemting the subsystem instead of wrapping it. And that's obviously > uncool because you'd then have two of them (besides, the community in > general rightfully pushes back against reimplementing stuff; see the > attempts to provide redundant Rust drivers in the past). > > The C code already takes care of figuring out region ranges and all > that, and it's battle hardened. To add an example, instead of reimplementing things like pci_iomap() we use `Io` as base type providing the accrssors like readl() and let the resource implement the mapping parts, such as `pci::Bar`. > > The main point of Rust is to make things safer; so if that can be > achieved without rewrite, as is the case with the presented container > solution, that's the way to go. > > > > > > Instead, implement a base type for I/O mapped memory, which > > > generically > > > provides the corresponding accessors, such as `Io::readb` or > > > `Io:try_readb`. > > > > It provides a subset of the existing accessors, one you might want to > > trim down for now, see below... > > > > > +/* io.h */ > > > +u8 rust_helper_readb(const volatile void __iomem *addr) > > > +{ > > > + return readb(addr); > > > +} > > > +EXPORT_SYMBOL_GPL(rust_helper_readb); > > > > <snip> > > > > You provide wrappers for a subset of what io.h provides, why that > > specific subset? > > > > Why not just add what you need, when you need it? I doubt you need > > all > > of these, and odds are you will need more. > > > > That was written by me as a first play set to test. Nova itself > currently reads only 8 byte from a PCI BAR, so we could indeed drop > everything but readq() for now and add things subsequently later, as > you suggest. I think it is reasonable to start with the most common accessors {read,write}{b,w,l,q and maybe their relaxed variants. We generate them through the `define_read!` and `define_write!` macros anyways and the only difference between all the variants is only the size type (u8, u16, etc.) we pass to the macro. > > > > > > +u32 rust_helper_readl_relaxed(const volatile void __iomem *addr) > > > +{ > > > + return readl_relaxed(addr); > > > +} > > > +EXPORT_SYMBOL_GPL(rust_helper_readl_relaxed); > > > > I know everyone complains about wrapper functions around inline > > functions, so I'll just say it again, this is horrid. And it's going > > to > > hurt performance, so any rust code people write is not on a level > > playing field here. > > > > Your call, but ick... > > Well, can anyone think of another way to do it? > > > > > > +#ifdef CONFIG_64BIT > > > +u64 rust_helper_readq_relaxed(const volatile void __iomem *addr) > > > +{ > > > + return readq_relaxed(addr); > > > +} > > > +EXPORT_SYMBOL_GPL(rust_helper_readq_relaxed); > > > +#endif > > > > Rust works on 32bit targets in the kernel now? > > Ahm, afaik not. That's some relic. Let's address that with your subset > comment from above. I think we should keep this guard; readq() implementations in the arch code have this guard as well. Should we ever add a 32bit target for Rust we also don't want this to break. > > > > > > +macro_rules! define_read { > > > + ($(#[$attr:meta])* $name:ident, $try_name:ident, > > > $type_name:ty) => { > > > + /// Read IO data from a given offset known at compile > > > time. > > > + /// > > > + /// Bound checks are performed on compile time, hence if > > > the offset is not known at compile > > > + /// time, the build will fail. > > > > offsets aren't know at compile time for many implementations, as it > > could be a dynamically allocated memory range. How is this going to > > work for that? Heck, how does this work for DT-defined memory ranges > > today? > > The macro below will take care of those where it's only knowable at > runtime I think. > > Rust has this feature (called "const generic") that can be used for > APIs where ranges which are known at compile time, so the compiler can > check all the parameters at that point. That has been judged to be > positive because errors with the range handling become visible before > the kernel runs and because it gives some performance advantages. Let's add an exammple based on `pci::Bar` here. As a driver you can optionally map a `pci::Bar` with an additional `SIZE` constant, e.g. ``` let bar = pdev.iomap_region_sized::<0x1000>(0)?; ``` This call only succeeds of the actual bar size is *at least* 4k. Subsequent calls to, let's say, `bar.readl(0x10)` can boundary check things on compile time, such that `bar.readl(0x1000)` would fail on compile time. This is useful when a driver knows the minum required / expected size of this memory region. Alternatively, a driver cann always fall back to a runtime check, e.g. ``` let bar = pdev.iomap_region(0)?; let val = bar.try_readl(0x1000)?; ``` - Danilo > > > P. > > > > > thanks, > > > > greg k-h > > >
Hi Danilo, Danilo Krummrich <dakr@redhat.com> writes: [...] > + > +macro_rules! define_write { > + ($(#[$attr:meta])* $name:ident, $try_name:ident, $type_name:ty) => { > + /// Write IO data from a given offset known at compile time. > + /// > + /// Bound checks are performed on compile time, hence if the offset is not known at compile > + /// time, the build will fail. > + $(#[$attr])* > + #[inline] > + pub fn $name(&self, value: $type_name, offset: usize) { > + let addr = self.io_addr_assert::<$type_name>(offset); > + > + unsafe { bindings::$name(value, addr as _, ) } > + } > + > + /// Write IO data from a given offset. > + /// > + /// Bound checks are performed on runtime, it fails if the offset (plus the type size) is > + /// out of bounds. > + $(#[$attr])* > + pub fn $try_name(&self, value: $type_name, offset: usize) -> Result { > + let addr = self.io_addr::<$type_name>(offset)?; > + > + unsafe { bindings::$name(value, addr as _) } > + Ok(()) > + } > + }; > +} > + I am curious why we do not need `&mut self` to write to this memory? Is it OK to race on these writes? Best regards, Andreas
On Tue, Jun 25, 2024 at 12:59:24PM +0200, Andreas Hindborg wrote: > Hi Danilo, > > Danilo Krummrich <dakr@redhat.com> writes: > > [...] > > > + > > +macro_rules! define_write { > > + ($(#[$attr:meta])* $name:ident, $try_name:ident, $type_name:ty) => { > > + /// Write IO data from a given offset known at compile time. > > + /// > > + /// Bound checks are performed on compile time, hence if the offset is not known at compile > > + /// time, the build will fail. > > + $(#[$attr])* > > + #[inline] > > + pub fn $name(&self, value: $type_name, offset: usize) { > > + let addr = self.io_addr_assert::<$type_name>(offset); > > + > > + unsafe { bindings::$name(value, addr as _, ) } > > + } > > + > > + /// Write IO data from a given offset. > > + /// > > + /// Bound checks are performed on runtime, it fails if the offset (plus the type size) is > > + /// out of bounds. > > + $(#[$attr])* > > + pub fn $try_name(&self, value: $type_name, offset: usize) -> Result { > > + let addr = self.io_addr::<$type_name>(offset)?; > > + > > + unsafe { bindings::$name(value, addr as _) } > > + Ok(()) > > + } > > + }; > > +} > > + > > I am curious why we do not need `&mut self` to write to this memory? Is > it OK to race on these writes? Yes, concurrent writes to the same I/O mapped memory region (within the same driver) should be totally fine. In the end it's only the driver that can know about (and has to ensure) the semantics, concurrency and ordering of those writes. > > > Best regards, > Andreas >
Hi Danilo,
From a Rust API point of view, this looks good to me.
Reviewed-by: Daniel Almeida <daniel.almeida@collabora.com>
Cheers,
— Daniel
diff --git a/rust/helpers.c b/rust/helpers.c index 0ce40ccb978b..824b7c0b98dc 100644 --- a/rust/helpers.c +++ b/rust/helpers.c @@ -26,6 +26,7 @@ #include <linux/device.h> #include <linux/err.h> #include <linux/errname.h> +#include <linux/io.h> #include <linux/mutex.h> #include <linux/rcupdate.h> #include <linux/refcount.h> @@ -181,6 +182,111 @@ void rust_helper_rcu_read_unlock(void) EXPORT_SYMBOL_GPL(rust_helper_rcu_read_unlock); /* end rcu */ +/* io.h */ +u8 rust_helper_readb(const volatile void __iomem *addr) +{ + return readb(addr); +} +EXPORT_SYMBOL_GPL(rust_helper_readb); + +u16 rust_helper_readw(const volatile void __iomem *addr) +{ + return readw(addr); +} +EXPORT_SYMBOL_GPL(rust_helper_readw); + +u32 rust_helper_readl(const volatile void __iomem *addr) +{ + return readl(addr); +} +EXPORT_SYMBOL_GPL(rust_helper_readl); + +#ifdef CONFIG_64BIT +u64 rust_helper_readq(const volatile void __iomem *addr) +{ + return readq(addr); +} +EXPORT_SYMBOL_GPL(rust_helper_readq); +#endif + +void rust_helper_writeb(u8 value, volatile void __iomem *addr) +{ + writeb(value, addr); +} +EXPORT_SYMBOL_GPL(rust_helper_writeb); + +void rust_helper_writew(u16 value, volatile void __iomem *addr) +{ + writew(value, addr); +} +EXPORT_SYMBOL_GPL(rust_helper_writew); + +void rust_helper_writel(u32 value, volatile void __iomem *addr) +{ + writel(value, addr); +} +EXPORT_SYMBOL_GPL(rust_helper_writel); + +#ifdef CONFIG_64BIT +void rust_helper_writeq(u64 value, volatile void __iomem *addr) +{ + writeq(value, addr); +} +EXPORT_SYMBOL_GPL(rust_helper_writeq); +#endif + +u8 rust_helper_readb_relaxed(const volatile void __iomem *addr) +{ + return readb_relaxed(addr); +} +EXPORT_SYMBOL_GPL(rust_helper_readb_relaxed); + +u16 rust_helper_readw_relaxed(const volatile void __iomem *addr) +{ + return readw_relaxed(addr); +} +EXPORT_SYMBOL_GPL(rust_helper_readw_relaxed); + +u32 rust_helper_readl_relaxed(const volatile void __iomem *addr) +{ + return readl_relaxed(addr); +} +EXPORT_SYMBOL_GPL(rust_helper_readl_relaxed); + +#ifdef CONFIG_64BIT +u64 rust_helper_readq_relaxed(const volatile void __iomem *addr) +{ + return readq_relaxed(addr); +} +EXPORT_SYMBOL_GPL(rust_helper_readq_relaxed); +#endif + +void rust_helper_writeb_relaxed(u8 value, volatile void __iomem *addr) +{ + writeb_relaxed(value, addr); +} +EXPORT_SYMBOL_GPL(rust_helper_writeb_relaxed); + +void rust_helper_writew_relaxed(u16 value, volatile void __iomem *addr) +{ + writew_relaxed(value, addr); +} +EXPORT_SYMBOL_GPL(rust_helper_writew_relaxed); + +void rust_helper_writel_relaxed(u32 value, volatile void __iomem *addr) +{ + writel_relaxed(value, addr); +} +EXPORT_SYMBOL_GPL(rust_helper_writel_relaxed); + +#ifdef CONFIG_64BIT +void rust_helper_writeq_relaxed(u64 value, volatile void __iomem *addr) +{ + writeq_relaxed(value, addr); +} +EXPORT_SYMBOL_GPL(rust_helper_writeq_relaxed); +#endif + /* * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can * use it in contexts where Rust expects a `usize` like slice (array) indices. diff --git a/rust/kernel/io.rs b/rust/kernel/io.rs new file mode 100644 index 000000000000..a19a1226181d --- /dev/null +++ b/rust/kernel/io.rs @@ -0,0 +1,219 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Memory-mapped IO. +//! +//! C header: [`include/asm-generic/io.h`](srctree/include/asm-generic/io.h) + +use crate::error::{code::EINVAL, Result}; +use crate::{bindings, build_assert}; + +/// IO-mapped memory, starting at the base address @addr and spanning @maxlen bytes. +/// +/// The creator (usually a subsystem such as PCI) is responsible for creating the +/// mapping, performing an additional region request etc. +/// +/// # Invariant +/// +/// `addr` is the start and `maxsize` the length of valid I/O remapped memory region. +/// +/// # Examples +/// +/// ``` +/// # use kernel::{bindings, io::Io}; +/// # use core::ops::Deref; +/// +/// // See also [`pci::Bar`] for a real example. +/// struct IoMem<const SIZE: usize>(Io<SIZE>); +/// +/// impl<const SIZE: usize> IoMem<SIZE> { +/// fn new(paddr: usize) -> Result<Self>{ +/// +/// // SAFETY: assert safety for this example +/// let addr = unsafe { bindings::ioremap(paddr as _, SIZE.try_into().unwrap()) }; +/// if addr.is_null() { +/// return Err(ENOMEM); +/// } +/// +/// // SAFETY: `addr` is guaranteed to be the start of a valid I/O mapped memory region of +/// // size `SIZE`. +/// let io = unsafe { Io::new(addr as _, SIZE)? }; +/// +/// Ok(IoMem(io)) +/// } +/// } +/// +/// impl<const SIZE: usize> Drop for IoMem<SIZE> { +/// fn drop(&mut self) { +/// // SAFETY: Safe as by the invariant of `Io`. +/// unsafe { bindings::iounmap(self.0.base_addr() as _); }; +/// } +/// } +/// +/// impl<const SIZE: usize> Deref for IoMem<SIZE> { +/// type Target = Io<SIZE>; +/// +/// fn deref(&self) -> &Self::Target { +/// &self.0 +/// } +/// } +/// +/// let iomem = IoMem::<{ core::mem::size_of::<u32>() }>::new(0xBAAAAAAD).unwrap(); +/// iomem.writel(0x42, 0x0); +/// assert!(iomem.try_writel(0x42, 0x0).is_ok()); +/// assert!(iomem.try_writel(0x42, 0x4).is_err()); +/// ``` +pub struct Io<const SIZE: usize = 0> { + addr: usize, + maxsize: usize, +} + +macro_rules! define_read { + ($(#[$attr:meta])* $name:ident, $try_name:ident, $type_name:ty) => { + /// Read IO data from a given offset known at compile time. + /// + /// Bound checks are performed on compile time, hence if the offset is not known at compile + /// time, the build will fail. + $(#[$attr])* + #[inline] + pub fn $name(&self, offset: usize) -> $type_name { + let addr = self.io_addr_assert::<$type_name>(offset); + + unsafe { bindings::$name(addr as _) } + } + + /// Read IO data from a given offset. + /// + /// Bound checks are performed on runtime, it fails if the offset (plus the type size) is + /// out of bounds. + $(#[$attr])* + pub fn $try_name(&self, offset: usize) -> Result<$type_name> { + let addr = self.io_addr::<$type_name>(offset)?; + + Ok(unsafe { bindings::$name(addr as _) }) + } + }; +} + +macro_rules! define_write { + ($(#[$attr:meta])* $name:ident, $try_name:ident, $type_name:ty) => { + /// Write IO data from a given offset known at compile time. + /// + /// Bound checks are performed on compile time, hence if the offset is not known at compile + /// time, the build will fail. + $(#[$attr])* + #[inline] + pub fn $name(&self, value: $type_name, offset: usize) { + let addr = self.io_addr_assert::<$type_name>(offset); + + unsafe { bindings::$name(value, addr as _, ) } + } + + /// Write IO data from a given offset. + /// + /// Bound checks are performed on runtime, it fails if the offset (plus the type size) is + /// out of bounds. + $(#[$attr])* + pub fn $try_name(&self, value: $type_name, offset: usize) -> Result { + let addr = self.io_addr::<$type_name>(offset)?; + + unsafe { bindings::$name(value, addr as _) } + Ok(()) + } + }; +} + +impl<const SIZE: usize> Io<SIZE> { + /// + /// + /// # Safety + /// + /// Callers must ensure that `addr` is the start of a valid I/O mapped memory region of size + /// `maxsize`. + pub unsafe fn new(addr: usize, maxsize: usize) -> Result<Self> { + if maxsize < SIZE { + return Err(EINVAL); + } + + Ok(Self { addr, maxsize }) + } + + /// Returns the base address of this mapping. + #[inline] + pub fn base_addr(&self) -> usize { + self.addr + } + + /// Returns the size of this mapping. + #[inline] + pub fn maxsize(&self) -> usize { + self.maxsize + } + + #[inline] + const fn offset_valid<U>(offset: usize, size: usize) -> bool { + let type_size = core::mem::size_of::<U>(); + if let Some(end) = offset.checked_add(type_size) { + end <= size && offset % type_size == 0 + } else { + false + } + } + + #[inline] + fn io_addr<U>(&self, offset: usize) -> Result<usize> { + if !Self::offset_valid::<U>(offset, self.maxsize()) { + return Err(EINVAL); + } + + // Probably no need to check, since the safety requirements of `Self::new` guarantee that + // this can't overflow. + self.base_addr().checked_add(offset).ok_or(EINVAL) + } + + #[inline] + fn io_addr_assert<U>(&self, offset: usize) -> usize { + build_assert!(Self::offset_valid::<U>(offset, SIZE)); + + self.base_addr() + offset + } + + define_read!(readb, try_readb, u8); + define_read!(readw, try_readw, u16); + define_read!(readl, try_readl, u32); + define_read!( + #[cfg(CONFIG_64BIT)] + readq, + try_readq, + u64 + ); + + define_read!(readb_relaxed, try_readb_relaxed, u8); + define_read!(readw_relaxed, try_readw_relaxed, u16); + define_read!(readl_relaxed, try_readl_relaxed, u32); + define_read!( + #[cfg(CONFIG_64BIT)] + readq_relaxed, + try_readq_relaxed, + u64 + ); + + define_write!(writeb, try_writeb, u8); + define_write!(writew, try_writew, u16); + define_write!(writel, try_writel, u32); + define_write!( + #[cfg(CONFIG_64BIT)] + writeq, + try_writeq, + u64 + ); + + define_write!(writeb_relaxed, try_writeb_relaxed, u8); + define_write!(writew_relaxed, try_writew_relaxed, u16); + define_write!(writel_relaxed, try_writel_relaxed, u32); + define_write!( + #[cfg(CONFIG_64BIT)] + writeq_relaxed, + try_writeq_relaxed, + u64 + ); +} diff --git a/rust/kernel/lib.rs b/rust/kernel/lib.rs index 601c3d3c9d54..f4dd11014a65 100644 --- a/rust/kernel/lib.rs +++ b/rust/kernel/lib.rs @@ -56,6 +56,7 @@ #[doc(hidden)] pub use bindings; +pub mod io; pub use macros; pub use uapi;