mbox series

[0/4] hbl: HabanaLabs RDMA provider

Message ID 20240613083802.1440904-1-oshpigelman@habana.ai (mailing list archive)
Headers show
Series hbl: HabanaLabs RDMA provider | expand

Message

Omer Shpigelman June 13, 2024, 8:37 a.m. UTC 
Introduce a provider which exposes hbl devices to user applications for
scaling of AI neural networks training through ROCEv2 protocol.
The hbl provider supports a DV API for fully operating the device.

The patches are also available at:
https://github.com/HabanaAI/rdma-core/tree/hbl_next

More info about the kernel driver and the HW can be found in the kernel
patch set:
https://lore.kernel.org/all/20240613082208.1439968-1-oshpigelman@habana.ai/

Omer Shpigelman (4):
  Update kernel headers
  hbl: HabanaLabs RDMA provider
  pyverbs/hbl: direct verbs support
  tests/hbl: direct verbs tests

 CMakeLists.txt                               |    2 +
 MAINTAINERS                                  |    5 +
 README.md                                    |    1 +
 buildlib/check-build                         |    1 +
 debian/control                               |    3 +-
 debian/copyright                             |    5 +
 debian/ibverbs-providers.install             |    1 +
 debian/ibverbs-providers.lintian-overrides   |    4 +-
 debian/ibverbs-providers.symbols             |   16 +
 debian/libibverbs-dev.install                |    7 +
 debian/rules                                 |    2 +-
 kernel-headers/CMakeLists.txt                |    4 +
 kernel-headers/rdma/hbl-abi.h                |  204 ++++
 kernel-headers/rdma/hbl_user_ioctl_cmds.h    |   66 ++
 kernel-headers/rdma/hbl_user_ioctl_verbs.h   |  106 ++
 kernel-headers/rdma/ib_user_ioctl_verbs.h    |    1 +
 libibverbs/verbs.h                           |    8 +-
 providers/hbl/CMakeLists.txt                 |   14 +
 providers/hbl/hbl-abi.h                      |   25 +
 providers/hbl/hbl.c                          |  225 ++++
 providers/hbl/hbl.h                          |  155 +++
 providers/hbl/hbldv.h                        |  433 +++++++
 providers/hbl/libhbl.map                     |   19 +
 providers/hbl/man/CMakeLists.txt             |   14 +
 providers/hbl/man/hbldv.7.md                 |   38 +
 providers/hbl/man/hbldv_create_cq.3.md       |   68 ++
 providers/hbl/man/hbldv_create_encap.3.md    |  116 ++
 providers/hbl/man/hbldv_create_usr_fifo.3.md |  121 ++
 providers/hbl/man/hbldv_is_supported.3.md    |   43 +
 providers/hbl/man/hbldv_modify_qp.3.md       |  102 ++
 providers/hbl/man/hbldv_open_device.3.md     |   59 +
 providers/hbl/man/hbldv_query_cq.3.md        |   84 ++
 providers/hbl/man/hbldv_query_device.3.md    |   61 +
 providers/hbl/man/hbldv_query_port.3.md      |  125 ++
 providers/hbl/man/hbldv_query_qp.3.md        |   63 +
 providers/hbl/man/hbldv_set_port_ex.3.md     |  106 ++
 providers/hbl/verbs.c                        | 1085 ++++++++++++++++++
 providers/hbl/verbs.h                        |   27 +
 pyverbs/CMakeLists.txt                       |    3 +-
 pyverbs/providers/hbl/CMakeLists.txt         |   10 +
 pyverbs/providers/hbl/__init__.pxd           |    0
 pyverbs/providers/hbl/__init__.py            |    0
 pyverbs/providers/hbl/hbl_enums.pxd          |   39 +
 pyverbs/providers/hbl/hbl_enums.pyx          |    0
 pyverbs/providers/hbl/hbldv.pxd              |   60 +
 pyverbs/providers/hbl/hbldv.pyx              |  394 +++++++
 pyverbs/providers/hbl/libhbl.pxd             |  142 +++
 pyverbs/providers/hbl/libhbl.pyx             |    0
 redhat/rdma-core.spec                        |    6 +
 suse/rdma-core.spec                          |   21 +
 tests/CMakeLists.txt                         |    2 +
 tests/hbl_base.py                            |   89 ++
 tests/test_hbldv.py                          |  226 ++++
 util/util.h                                  |   13 +
 54 files changed, 4416 insertions(+), 8 deletions(-)
 create mode 100644 kernel-headers/rdma/hbl-abi.h
 create mode 100644 kernel-headers/rdma/hbl_user_ioctl_cmds.h
 create mode 100644 kernel-headers/rdma/hbl_user_ioctl_verbs.h
 create mode 100644 providers/hbl/CMakeLists.txt
 create mode 100644 providers/hbl/hbl-abi.h
 create mode 100644 providers/hbl/hbl.c
 create mode 100644 providers/hbl/hbl.h
 create mode 100644 providers/hbl/hbldv.h
 create mode 100644 providers/hbl/libhbl.map
 create mode 100644 providers/hbl/man/CMakeLists.txt
 create mode 100644 providers/hbl/man/hbldv.7.md
 create mode 100644 providers/hbl/man/hbldv_create_cq.3.md
 create mode 100644 providers/hbl/man/hbldv_create_encap.3.md
 create mode 100644 providers/hbl/man/hbldv_create_usr_fifo.3.md
 create mode 100644 providers/hbl/man/hbldv_is_supported.3.md
 create mode 100644 providers/hbl/man/hbldv_modify_qp.3.md
 create mode 100644 providers/hbl/man/hbldv_open_device.3.md
 create mode 100644 providers/hbl/man/hbldv_query_cq.3.md
 create mode 100644 providers/hbl/man/hbldv_query_device.3.md
 create mode 100644 providers/hbl/man/hbldv_query_port.3.md
 create mode 100644 providers/hbl/man/hbldv_query_qp.3.md
 create mode 100644 providers/hbl/man/hbldv_set_port_ex.3.md
 create mode 100644 providers/hbl/verbs.c
 create mode 100644 providers/hbl/verbs.h
 create mode 100644 pyverbs/providers/hbl/CMakeLists.txt
 create mode 100644 pyverbs/providers/hbl/__init__.pxd
 create mode 100644 pyverbs/providers/hbl/__init__.py
 create mode 100644 pyverbs/providers/hbl/hbl_enums.pxd
 create mode 100644 pyverbs/providers/hbl/hbl_enums.pyx
 create mode 100644 pyverbs/providers/hbl/hbldv.pxd
 create mode 100644 pyverbs/providers/hbl/hbldv.pyx
 create mode 100644 pyverbs/providers/hbl/libhbl.pxd
 create mode 100644 pyverbs/providers/hbl/libhbl.pyx
 create mode 100644 tests/hbl_base.py
 create mode 100644 tests/test_hbldv.py