[net-next,00/10] devlink: Control auxiliary devices

Message

Parav Pandit Aug. 10, 2021, 11:46 a.m. UTC

Hi Dave, Jakub,

Currently, for mlx5 multi-function device, a user is not able to control
which functionality to enable/disable. For example, each PCI
PF, VF, SF function by default has netdevice, RDMA and vdpa-net
devices always enabled.

Hence, enable user to control which device functionality to enable/disable.

This is achieved by using existing devlink params [1] to
enable/disable eth, rdma and vdpa net functionality control knob.

For example user interested in only vdpa device function: performs,

$ devlink dev param set pci/0000:06:00.0 name enable_rdma value false \
                   cmode driverinit
$ devlink dev param set pci/0000:06:00.0 name enable_eth value false \
                   cmode driverinit
$ devlink dev param set pci/0000:06:00.0 name enable_vnet value true \
                   cmode driverinit

$ devlink dev reload pci/0000:06:00.0

Reload command honors parameters set, initializes the device that user
has composed using devlink dev params and resources.
Devices before reload:

            mlx5_core.sf.4
         (subfunction device)
                  /\
                 /| \
                / |  \
               /  |   \
 mlx5_core.eth.4  |  mlx5_core.rdma.4
(SF eth aux dev)  |  (SF rdma aux dev)
    |             |        |
    |             |        |
 enp6s0f0s88      |      mlx5_0
 (SF netdev)      |  (SF rdma device)
                  |
         mlx5_core.vnet.4
         (SF vnet aux dev)
                 |
                 |
        auxiliary/mlx5_core.sf.4
        (vdpa net mgmt device)

Above example reconfigures the device with only VDPA functionality.
Devices after reload:

            mlx5_core.sf.4
         (subfunction device)
                  /\
                 /  \
                /    \
               /      \
 mlx5_core.vnet.4     no eth, no rdma aux devices
 (SF vnet aux dev) 

Above parameters enable user to compose the device as needed based
on the use case.

Since devlink params are done on the devlink instance, these
knobs are uniformly usable for PCI PF, VF and SF devices.

Patch summary:
patch-1 adds generic enable_eth devlink parameter to control Ethernet
        auxiliary device function
patch-2 adds generic enable_rdma devlink parameter to control RDMA
        auxiliary device function
patch-3 adds generic enable_vnet devlink parameter to control VDPA net
        auxilariy device function
patch-4 rework the code to register single device parameter
patch-5 added APIs to register, unregister single device parameter
patch-6 added APIs to publish, unpublishe single device parameter
patch-7 Fixed missing parameter unpublish call in mlx5 driver
patch-8 extends mlx5 driver to support enable_eth devlink parameter
patch-9 extends mlx5 driver to support enable_rdma devlink parameter
patch-10 extends mlx5 driver to support enable_vnet devlink parameter

Subsequent to this series, in future mlx5 driver will be updated to use
single device parameter API for metadata enable/disable knob which is
only applicable on the eswitch manager device.

[1] https://www.kernel.org/doc/html/latest/networking/devlink/devlink-params.html
 
Parav Pandit (10):
  devlink: Add new "enable_eth" generic device param
  devlink: Add new "enable_rdma" generic device param
  devlink: Add new "enable_vnet" generic device param
  devlink: Create a helper function for one parameter registration
  devlink: Add API to register and unregister single parameter
  devlink: Add APIs to publish, unpublish individual parameter
  net/mlx5: Fix unpublish devlink parameters
  net/mlx5: Support enable_eth devlink dev param
  net/mlx5: Support enable_rdma devlink dev param
  net/mlx5: Support enable_vnet devlink dev param

 .../networking/devlink/devlink-params.rst     |  12 ++
 drivers/net/ethernet/mellanox/mlx5/core/dev.c |  74 +++++++-
 .../net/ethernet/mellanox/mlx5/core/devlink.c | 159 ++++++++++++++++++
 .../ethernet/mellanox/mlx5/core/mlx5_core.h   |   5 +
 include/net/devlink.h                         |  20 +++
 net/core/devlink.c                            | 124 +++++++++++++-
 6 files changed, 382 insertions(+), 12 deletions(-)

Comments

Parav Pandit Aug. 10, 2021, 11:59 a.m. UTC | #1

> From: Parav Pandit <[e-mail]>
> Sent: Tuesday, August 10, 2021 5:16 PM


> Hi Dave, Jakub,
> 
> Currently, for mlx5 multi-function device, a user is not able to control which
> functionality to enable/disable. For example, each PCI PF, VF, SF function by
> default has netdevice, RDMA and vdpa-net devices always enabled.
> 
> Hence, enable user to control which device functionality to enable/disable.
> 
> This is achieved by using existing devlink params [1] to enable/disable eth,
> rdma and vdpa net functionality control knob.
> 
> For example user interested in only vdpa device function: performs,
> 
> $ devlink dev param set pci/0000:06:00.0 name enable_rdma value false \
>                    cmode driverinit
> $ devlink dev param set pci/0000:06:00.0 name enable_eth value false \
>                    cmode driverinit
> $ devlink dev param set pci/0000:06:00.0 name enable_vnet value true \
>                    cmode driverinit

I missed to CC rdma and virt mailing list.
I am resending the series to CC them.