mbox series

[RFC,v2,0/20] fwctl/cxl: Add CXL feature commands support via fwctl

Message ID 20241115212745.869552-1-dave.jiang@intel.com
Headers show
Series fwctl/cxl: Add CXL feature commands support via fwctl | expand

Message

Dave Jiang Nov. 15, 2024, 9:25 p.m. UTC
v2:
- Dropped 1/13 and 2/13 from previous version. Merged upstream already.
- Combined changes from Shiju for "get supported features"
- Addressed comments from Jonathan and Jason
- See specific changes in individual patch revision history
- Added hardware command info to FWCTL
- Added filtering to set feature command
- Added documentation

This series add support for CXL feature commands using the FWCTL framework [1].
The code is untested and I'm looking for architectural and implementation feedback.
While CXL currently has a chardev for user ioctls to send some mailbox
commands to a memory device, the fwctl framework provides more security policies
that can be a potential vehicle to move CXL ioctl path to that.

For this RFC, the mailbox commands "Get Supported Features", "Get Feature", and
"Set Feature" commands are implemented. The "get" commands under the
FWCTL_RPC_DEBUG_READ_ONLY policy, the "set" command checks the policy depending
on the effect of the feature. All mailbox commands for CXL provides an effects
table that describes the effects of a command when performed on the device.
For CXL features, there is also an effects field that describes the effects
a feature write operation has on the device per feature. The security policy
is checked against this feature specific effects field. Looking for discussion
on matching the CXL spec defined effects with the FWCTL security policy.

The code is based off of v3 of FWCTL series [1] posted by Jason and rebased on top of
v6.12-rc6.

Jason,
Please see 11/20 and 13/20 in the series WRT the new hw info ioctl I introduced.
Let me know if that looks reasonable or if I should do that in a different way to
achieve what I want.

Jonathan and Shiju,
16/20 adds filtering of set feature command. Please check and see if that looks
reasonable for how we want to apply restrictions to the set feature command.

Shiju,
Please pick up 2/20 for your EDAC series. I made some corrections, including a pointer
math issue in cxl_get_supported_features() while looping through the feature
entries after saving them.

Jonathan,
I added documentation in 20/20. Maybe we can flesh out the usage policy there.

[1]: https://lore.kernel.org/linux-cxl/0-v3-960f17f90f17+516-fwctl_jgg@nvidia.com/#r

---

Dave Jiang (18):
      cxl: Refactor user ioctl command path from mds to mailbox
      cxl: Add Get Supported Features command for kernel usage
      cxl/test: Add Get Supported Features mailbox command support
      cxl: Add Get Feature command support for user submission
      cxl: Add Set Feature command support for user submission
      cxl: Move cxl_driver related bits to be usable by external drivers
      fwctl/cxl: Add driver for CXL mailbox for handling CXL features commands
      fwctl/cxl: Add support for get driver information
      fwctl: FWCTL_HW_INFO to return hardware information
      cxl: Save Command Effects Log (CEL) effects for enabled commands
      fwctl/cxl: Add hw_info callback
      cxl: Move defines and error codes from cxlmem.h to cxl/mailbox.h
      fwctl/cxl: Add support for fwctl RPC command to enable CXL feature commands
      fwctl/cxl: Add support to filter exclusive features
      cxl/test: Add Get Feature support to cxl_test
      cxl/test: Add Set Feature support to cxl_test
      fwctl: Move fwctl documentation to its own directory
      fwctl/cxl: Add documentation to FWCTL CXL

Shiju Jose (2):
      cxl/mbox: Add GET_FEATURE mailbox command
      cxl/mbox: Add SET_FEATURE mailbox command

 Documentation/userspace-api/fwctl/fwctl-cxl.rst   |  91 +++++
 Documentation/userspace-api/{ => fwctl}/fwctl.rst |   0
 Documentation/userspace-api/fwctl/index.rst       |  13 +
 Documentation/userspace-api/index.rst             |   2 +-
 MAINTAINERS                                       |  10 +-
 drivers/cxl/core/core.h                           |   9 +-
 drivers/cxl/core/mbox.c                           | 859 ++++++++++++++++++++++++++++++++++++++----
 drivers/cxl/core/memdev.c                         |  24 +-
 drivers/cxl/cxl.h                                 |  32 +-
 drivers/cxl/cxlmem.h                              | 184 ++++-----
 drivers/cxl/pci.c                                 |   4 +
 drivers/fwctl/Kconfig                             |   9 +
 drivers/fwctl/Makefile                            |   1 +
 drivers/fwctl/cxl/Makefile                        |   4 +
 drivers/fwctl/cxl/cxl.c                           | 237 ++++++++++++
 drivers/fwctl/main.c                              |  33 ++
 include/cxl/cxl.h                                 |  40 ++
 include/cxl/features.h                            |  52 +++
 include/cxl/mailbox.h                             | 183 ++++++++-
 include/linux/fwctl.h                             |   6 +
 include/uapi/fwctl/cxl.h                          |  51 +++
 include/uapi/fwctl/fwctl.h                        |  59 ++-
 include/uapi/linux/cxl_mem.h                      |  20 +-
 tools/testing/cxl/test/mem.c                      | 201 ++++++++++
 24 files changed, 1903 insertions(+), 221 deletions(-)

Comments

Jason Gunthorpe Nov. 20, 2024, 6:57 p.m. UTC | #1
On Fri, Nov 15, 2024 at 02:25:33PM -0700, Dave Jiang wrote:
> v2:
> - Dropped 1/13 and 2/13 from previous version. Merged upstream already.
> - Combined changes from Shiju for "get supported features"
> - Addressed comments from Jonathan and Jason
> - See specific changes in individual patch revision history
> - Added hardware command info to FWCTL
> - Added filtering to set feature command
> - Added documentation

I looked through it, and I think it is good enough that the next
posting should not be RFC from my perspective..

The amount of CXL core patches has reduced quite a lot, it would be good to
order the patches so CXL was grouped together at the start, that would
make it easier to create a shared branch down the road

For instance very roughly something like:

>       cxl: Refactor user ioctl command path from mds to mailbox
>       cxl: Add Get Supported Features command for kernel usage
>       cxl/mbox: Add GET_FEATURE mailbox command
>       cxl: Add Get Feature command support for user submission
>       cxl/mbox: Add SET_FEATURE mailbox command
>       cxl: Add Set Feature command support for user submission
>       cxl: Move cxl_driver related bits to be usable by external drivers
>       cxl: Save Command Effects Log (CEL) effects for enabled commands
>       cxl: Move defines and error codes from cxlmem.h to cxl/mailbox.h
>       cxl/test: Add Get Supported Features mailbox command support
>       cxl/test: Add Get Feature support to cxl_test
>       cxl/test: Add Set Feature support to cxl_test

>       fwctl: FWCTL_HW_INFO to return hardware information
>       fwctl/cxl: Add driver for CXL mailbox for handling CXL features commands
>       fwctl/cxl: Add support for get driver information
>       fwctl/cxl: Add hw_info callback
>       fwctl/cxl: Add support for fwctl RPC command to enable CXL feature commands
>       fwctl/cxl: Add support to filter exclusive features
>       fwctl/cxl: Add documentation to FWCTL CXL

Thanks,
Jason
Jonathan Cameron Nov. 21, 2024, 6:38 p.m. UTC | #2
On Wed, 20 Nov 2024 14:57:20 -0400
Jason Gunthorpe <jgg@nvidia.com> wrote:

> On Fri, Nov 15, 2024 at 02:25:33PM -0700, Dave Jiang wrote:
> > v2:
> > - Dropped 1/13 and 2/13 from previous version. Merged upstream already.
> > - Combined changes from Shiju for "get supported features"
> > - Addressed comments from Jonathan and Jason
> > - See specific changes in individual patch revision history
> > - Added hardware command info to FWCTL
> > - Added filtering to set feature command
> > - Added documentation  
> 
> I looked through it, and I think it is good enough that the next
> posting should not be RFC from my perspective..
> 
> The amount of CXL core patches has reduced quite a lot, it would be good to
> order the patches so CXL was grouped together at the start, that would
> make it easier to create a shared branch down the road
> 
> For instance very roughly something like:
> 
> >       cxl: Refactor user ioctl command path from mds to mailbox
> >       cxl: Add Get Supported Features command for kernel usage
> >       cxl/mbox: Add GET_FEATURE mailbox command
> >       cxl: Add Get Feature command support for user submission
> >       cxl/mbox: Add SET_FEATURE mailbox command

Some of these are common to the EDAC RAS features set which
I'm really hoping finally lands next cycle.

So we will need to be a little careful in how we merge things.

Going to need to be careful how we handle this.  Maybe Dave should
pick up the feature stuff as an immutable branch after rc1 and
both series get rebuilt on top of that.  Need to discuss if
Borislav is happy with that for potential merge of the RAS
features series (EDAC).

Anyhow, time up for me today. I'll look at the rest of the series
tomorrow.

Jonathan

> >       cxl: Add Set Feature command support for user submission
> >       cxl: Move cxl_driver related bits to be usable by external drivers
> >       cxl: Save Command Effects Log (CEL) effects for enabled commands
> >       cxl: Move defines and error codes from cxlmem.h to cxl/mailbox.h
> >       cxl/test: Add Get Supported Features mailbox command support
> >       cxl/test: Add Get Feature support to cxl_test
> >       cxl/test: Add Set Feature support to cxl_test  
> 
> >       fwctl: FWCTL_HW_INFO to return hardware information
> >       fwctl/cxl: Add driver for CXL mailbox for handling CXL features commands
> >       fwctl/cxl: Add support for get driver information
> >       fwctl/cxl: Add hw_info callback
> >       fwctl/cxl: Add support for fwctl RPC command to enable CXL feature commands
> >       fwctl/cxl: Add support to filter exclusive features
> >       fwctl/cxl: Add documentation to FWCTL CXL  
> 
> Thanks,
> Jason