| Message ID | 20211007204051.10086-1-rpearsonhpe@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Replace AV by AH in UD sends | expand |
On Thu, Oct 07, 2021 at 03:40:46PM -0500, Bob Pearson wrote: > Currently the rdma_rxe driver and its user space provider exchange > addressing information for UD sends by having the provider compute an > address vector (AV) and send it with each WQE. This is not the way > that the RDMA verbs API was intended to operate. > > This series of patches modifies the way UD send WQEs work by exchanging > an index identifying the AH replacing the 88 byte AV by a 4 byte AH > index. In order to not break compatibility with the existing API the > rdma_rxe driver will recognise when an older version of the provider > is not sending an index (i.e. it is 0) and will use the AV instead. > > This series of patches is rebased to > commit: 286dba65a4a65a6d5de011767061f6ffa6e10389 (for-next) > > v6: > Rebase to 5.15.0-rc4 > Reduced the max number of AH objects from 1M to 32K. > > v5: > Rebase to 5.15.0-rc2+ > > v4: > Rebase to 5.15.0-rc1+ > > v3: > Split up commits into smaller steps. > > v2: > Rearranged AV in rxe_send_wqe to be in the ud struct but padded to the > same offset as the original preserving ABI compatibility. > > Bob Pearson (6): > RDMA/rxe: Move AV from rxe_send_wqe to rxe_send_wr > RDMA/rxe: Change AH objects to indexed > RDMA/rxe: Create AH index and return to user space > RDMA/rxe: Replace ah->pd by ah->ibah.pd > RDMA/rxe: Lookup kernel AH from ah index in UD WQEs > RDMA/rxe: Convert kernel UD post send to use ah_num Applied to for-next, thanks Jason
Currently the rdma_rxe driver and its user space provider exchange addressing information for UD sends by having the provider compute an address vector (AV) and send it with each WQE. This is not the way that the RDMA verbs API was intended to operate. This series of patches modifies the way UD send WQEs work by exchanging an index identifying the AH replacing the 88 byte AV by a 4 byte AH index. In order to not break compatibility with the existing API the rdma_rxe driver will recognise when an older version of the provider is not sending an index (i.e. it is 0) and will use the AV instead. This series of patches is rebased to commit: 286dba65a4a65a6d5de011767061f6ffa6e10389 (for-next) --- v6: Rebase to 5.15.0-rc4 Reduced the max number of AH objects from 1M to 32K. v5: Rebase to 5.15.0-rc2+ v4: Rebase to 5.15.0-rc1+ v3: Split up commits into smaller steps. v2: Rearranged AV in rxe_send_wqe to be in the ud struct but padded to the same offset as the original preserving ABI compatibility. Bob Pearson (6): RDMA/rxe: Move AV from rxe_send_wqe to rxe_send_wr RDMA/rxe: Change AH objects to indexed RDMA/rxe: Create AH index and return to user space RDMA/rxe: Replace ah->pd by ah->ibah.pd RDMA/rxe: Lookup kernel AH from ah index in UD WQEs RDMA/rxe: Convert kernel UD post send to use ah_num drivers/infiniband/sw/rxe/rxe_av.c | 20 +++++++++++++- drivers/infiniband/sw/rxe/rxe_param.h | 4 ++- drivers/infiniband/sw/rxe/rxe_pool.c | 4 ++- drivers/infiniband/sw/rxe/rxe_req.c | 8 +++--- drivers/infiniband/sw/rxe/rxe_verbs.c | 39 ++++++++++++++++++++++----- drivers/infiniband/sw/rxe/rxe_verbs.h | 8 +++++- include/uapi/rdma/rdma_user_rxe.h | 10 ++++++- 7 files changed, 79 insertions(+), 14 deletions(-)