mbox series

[v8,rdma-next,0/2] Dynamic rdma link creation

Message ID cover.1550257456.git.swise@opengridcomputing.com (mailing list archive)
Headers show
Series Dynamic rdma link creation | expand

Message

Steve Wise Feb. 15, 2019, 7:04 p.m. UTC 
This series enables creating soft rdma links dynamically using netlink
messages.  The user 'rdma' command [1] will be enhanced to send these new
messages.

Note there is at least one outstanding issue:  the rdma-core requires
RXE links to be named "rxe*" or it fails to associate an rxe uverbs
driver with the rxe provider.  This, however, can be addressed
in a subsequent series.

[1] rdma tool support (work-in-progress):
https://github.com/larrystevenwise/iproute2/tree/wip/newlink

Changes since V7:
- refactored on top of the core device register/unregister work by jason:
  https://www.spinics.net/lists/linux-rdma/msg75462.html

Changes since V6:
- added Jason's patch to refactor device unregisteration and
  rxe reference cleanup.
- added Yuval's patch as a dependency.
- remove dellink() rdma_link_ops method replaced with
  IB_DEVICE_ALLOW_USER_UNREG device caps flag.  If set, then the core
  will unregister the device in nldev_dellink() for the driver using the
  new ib_unregister_device_and_put().
- add r-b tags.

Changes since V4/V5:
- fixed device remove races in rxe
- fixed deadlock in NETDEV_UNREGISTER induced rxe device removal
- removed redundant string checks in nldev_newlink()
- added reviewed-by tag

Changes since V3:
- link_ops mutex replaced with rwsem
- save link_ops pointer in ib_device struct to be used when deleting
  a link
- only require the device index from userland for DELLINK operations
- pass struct ib_device pointer to drivers for dellink()

Changes since V2:
- removed unneeded enum value assignment
- removed gotos where there was no unwind logic
- protect get and use of link_ops with mutex

Changes since V1:
- rebased on for-next branch of linux-rdma repo
- removed 2 pr_debug()s
- validate string lengths in nldev_newlink()/nldev_dellink()
- don't allow '%' in ibdev names
- make all string parameters const char * in rdma_link_ops
- simplified the request_module logic in nldev_newlink()
- add leon's reviewed-by tag.

Changes since RFC:
- rebased on top of leon's and jason's device name work
- request module load if the link ops aren't found
- detect duplicate link ops adds
- const char * changes
- use the RDMA_NLDEV open enum values


Steve Wise (2):
  RDMA/core: add RDMA_NLDEV_CMD_NEWLINK/DELLINK support
  rdma_rxe: use netlink messages to add/delete links

 drivers/infiniband/core/nldev.c       | 122 ++++++++++++++++++++++++++++++++++
 drivers/infiniband/sw/rxe/rxe.c       |  36 +++++++++-
 drivers/infiniband/sw/rxe/rxe.h       |   2 +-
 drivers/infiniband/sw/rxe/rxe_net.c   |   4 +-
 drivers/infiniband/sw/rxe/rxe_net.h   |   2 +-
 drivers/infiniband/sw/rxe/rxe_param.h |   3 +-
 drivers/infiniband/sw/rxe/rxe_sysfs.c |   6 +-
 drivers/infiniband/sw/rxe/rxe_verbs.c |   4 +-
 drivers/infiniband/sw/rxe/rxe_verbs.h |   2 +-
 include/rdma/ib_verbs.h               |   3 +
 include/rdma/rdma_netlink.h           |  11 +++
 include/uapi/rdma/rdma_netlink.h      |  11 ++-
 12 files changed, 191 insertions(+), 15 deletions(-)

Comments

Jason Gunthorpe Feb. 19, 2019, 4:39 a.m. UTC | #1 
On Fri, Feb 15, 2019 at 11:04:16AM -0800, Steve Wise wrote:
> This series enables creating soft rdma links dynamically using netlink
> messages.  The user 'rdma' command [1] will be enhanced to send these new
> messages.
> 
> Note there is at least one outstanding issue:  the rdma-core requires
> RXE links to be named "rxe*" or it fails to associate an rxe uverbs
> driver with the rxe provider.  This, however, can be addressed
> in a subsequent series.
> 
> [1] rdma tool support (work-in-progress):
> https://github.com/larrystevenwise/iproute2/tree/wip/newlink
> 
> Changes since V7:
> - refactored on top of the core device register/unregister work by jason:
>   https://www.spinics.net/lists/linux-rdma/msg75462.html
> 
> Changes since V6:
> - added Jason's patch to refactor device unregisteration and
>   rxe reference cleanup.
> - added Yuval's patch as a dependency.
> - remove dellink() rdma_link_ops method replaced with
>   IB_DEVICE_ALLOW_USER_UNREG device caps flag.  If set, then the core
>   will unregister the device in nldev_dellink() for the driver using the
>   new ib_unregister_device_and_put().
> - add r-b tags.
> 
> Changes since V4/V5:
> - fixed device remove races in rxe
> - fixed deadlock in NETDEV_UNREGISTER induced rxe device removal
> - removed redundant string checks in nldev_newlink()
> - added reviewed-by tag
> 
> Changes since V3:
> - link_ops mutex replaced with rwsem
> - save link_ops pointer in ib_device struct to be used when deleting
>   a link
> - only require the device index from userland for DELLINK operations
> - pass struct ib_device pointer to drivers for dellink()
> 
> Changes since V2:
> - removed unneeded enum value assignment
> - removed gotos where there was no unwind logic
> - protect get and use of link_ops with mutex
> 
> Changes since V1:
> - rebased on for-next branch of linux-rdma repo
> - removed 2 pr_debug()s
> - validate string lengths in nldev_newlink()/nldev_dellink()
> - don't allow '%' in ibdev names
> - make all string parameters const char * in rdma_link_ops
> - simplified the request_module logic in nldev_newlink()
> - add leon's reviewed-by tag.
> 
> Changes since RFC:
> - rebased on top of leon's and jason's device name work
> - request module load if the link ops aren't found
> - detect duplicate link ops adds
> - const char * changes
> - use the RDMA_NLDEV open enum values
> 
> 
> Steve Wise (2):
>   RDMA/core: add RDMA_NLDEV_CMD_NEWLINK/DELLINK support
>   rdma_rxe: use netlink messages to add/delete links

Applied to for-next, please get the iproute side applied too, and lets
have a rdma-core man page patch saying rxe_cfg is obsolete, thanks

Jason