| Message ID | 20210930042603.4318-1-rpearsonhpe@gmail.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Replace AV by AH in UD sends | expand |
On Wed, Sep 29, 2021 at 11:25:58PM -0500, Bob Pearson wrote: > Currently the rdma_rxe driver and its user space provider exchange > addressing information for UD sends by having the provider compute an > address vector (AV) and send it with each WQE. This is not the way > that the RDMA verbs API was intended to operate. > > This series of patches modifies the way UD send WQEs work by exchanging > an index identifying the AH replacing the 88 byte AV by a 4 byte AH > index. In order to not break compatibility with the existing API the > rdma_rxe driver will recognise when an older version of the provider > is not sending an index (i.e. it is 0) and will use the AV instead. > > This series of patches is almost identical to an earlier version > but rebased to 5.15.0-rc1+. It applies cleanly to > > 450f4f6aa1a369cc3ffadc1c7e27dfab3e90199f (for-next) It looks OK, but please rebase/resend it across Rao's patch to here: commit 0994a1bcd5f7c0bf7ca3b4938d4f0f6928ac7886 Author: Rao Shoaib <Rao.Shoaib@oracle.com> Date: Tue Sep 14 18:12:20 2021 -0700 RDMA/rxe: Bump up default maximum values used via uverbs In our internal testing we have found that default maximum values are too small. Ideally there should be no limits, but since maximum values are reported via ibv_query_device, we have to return some value. So, the default maximums have been changed to large values. Link: https://lore.kernel.org/r/20210915011220.307585-1-Rao.Shoaib@oracle.com Signed-off-by: Rao Shoaib <Rao.Shoaib@oracle.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> Jason
Currently the rdma_rxe driver and its user space provider exchange addressing information for UD sends by having the provider compute an address vector (AV) and send it with each WQE. This is not the way that the RDMA verbs API was intended to operate. This series of patches modifies the way UD send WQEs work by exchanging an index identifying the AH replacing the 88 byte AV by a 4 byte AH index. In order to not break compatibility with the existing API the rdma_rxe driver will recognise when an older version of the provider is not sending an index (i.e. it is 0) and will use the AV instead. This series of patches is almost identical to an earlier version but rebased to 5.15.0-rc1+. It applies cleanly to 450f4f6aa1a369cc3ffadc1c7e27dfab3e90199f (for-next) v4: Rebase to 5.15.0-rc1+ v3: Split up commits into smaller steps. v2: Rearranged AV in rxe_send_wqe to be in the ud struct but padded to the same offset as the original preserving ABI compatibility. Bob Pearson (6): RDMA/rxe: Move AV from rxe_send_wqe to rxe_send_wr RDMA/rxe: Change AH objects to indexed RDMA/rxe: Create AH index and return to user space RDMA/rxe: Replace ah->pd by ah->ibah.pd RDMA/rxe: Lookup kernel AH from ah index in UD WQEs RDMA/rxe: Convert kernel UD post send to use ah_num drivers/infiniband/sw/rxe/rxe_av.c | 20 +++++++++++++- drivers/infiniband/sw/rxe/rxe_param.h | 4 ++- drivers/infiniband/sw/rxe/rxe_pool.c | 4 ++- drivers/infiniband/sw/rxe/rxe_req.c | 8 +++--- drivers/infiniband/sw/rxe/rxe_verbs.c | 39 ++++++++++++++++++++++----- drivers/infiniband/sw/rxe/rxe_verbs.h | 8 +++++- include/uapi/rdma/rdma_user_rxe.h | 10 ++++++- 7 files changed, 79 insertions(+), 14 deletions(-)