| Message ID | 1551981764-26557-1-git-send-email-maxg@mellanox.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | Introduce new API for T10-PI offload | expand |
On Thu, Mar 07, 2019 at 08:02:26PM +0200, Max Gurtovoy wrote: > Hello Sagi, Christoph, Keith, Jason, Doug, Leon and Co > > This patchset adds a new verbs API for T10-PI offload and > implementation for iSER initiator (NVMe-oF/RDMA host side was completed > and will be sent on a different patchset). Where are we with this? Apply or no? CH/Bart/Sagi? Thanks, Jason
On Tue, 2019-03-26 at 11:12 -0300, Jason Gunthorpe wrote: > On Thu, Mar 07, 2019 at 08:02:26PM +0200, Max Gurtovoy wrote: > > Hello Sagi, Christoph, Keith, Jason, Doug, Leon and Co > > > > This patchset adds a new verbs API for T10-PI offload and > > implementation for iSER initiator (NVMe-oF/RDMA host side was completed > > and will be sent on a different patchset). > > Where are we with this? Apply or no? CH/Bart/Sagi? Hi Jason, If Christoph is fine with this series I'm fine with it too. Bart.
On Thu, Mar 07, 2019 at 08:02:26PM +0200, Max Gurtovoy wrote: > This series applies cleanly on top of kernel 5.0 tag and iser fixs > ("IB/iser: Pass the correct number of entries for dma mapped SGL") > and ("IB/iser: Fix dma_nents type definition"). I tried to apply it to 5.0 with those two patches cherry picked, but it fails to apply in mlx5 code: Applying: RDMA/mlx5: Implement mlx5_ib_map_mr_sg_pi and mlx5_ib_alloc_mr_integrity error: patch failed: drivers/infiniband/hw/mlx5/mlx5_ib.h:585 error: drivers/infiniband/hw/mlx5/mlx5_ib.h: patch does not apply Patch failed at 0009 RDMA/mlx5: Implement mlx5_ib_map_mr_sg_pi and mlx5_ib_alloc_mr_integrity hint: Use 'git am --show-current-patch' to see the failed patch When you have resolved this problem, run "git am --continue". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort". Please tell me how I can get an actually working tree with these patches, as it fails even more badly against 5.1-rc..
On 4/11/2019 8:36 PM, Christoph Hellwig wrote: > On Thu, Mar 07, 2019 at 08:02:26PM +0200, Max Gurtovoy wrote: >> This series applies cleanly on top of kernel 5.0 tag and iser fixs >> ("IB/iser: Pass the correct number of entries for dma mapped SGL") >> and ("IB/iser: Fix dma_nents type definition"). > I tried to apply it to 5.0 with those two patches cherry picked, but > it fails to apply in mlx5 code: > > Applying: RDMA/mlx5: Implement mlx5_ib_map_mr_sg_pi and > mlx5_ib_alloc_mr_integrity > error: patch failed: drivers/infiniband/hw/mlx5/mlx5_ib.h:585 > error: drivers/infiniband/hw/mlx5/mlx5_ib.h: patch does not apply > Patch failed at 0009 RDMA/mlx5: Implement mlx5_ib_map_mr_sg_pi and > mlx5_ib_alloc_mr_integrity > hint: Use 'git am --show-current-patch' to see the failed patch > When you have resolved this problem, run "git am --continue". > If you prefer to skip this patch, run "git am --skip" instead. > To restore the original branch and stop patching, run "git am --abort". > > Please tell me how I can get an actually working tree with these > patches, as it fails even more badly against 5.1-rc.. strange, I'll check it next week. I can send a pointer to a linux fork github repo with those patches...
Hello Sagi, Christoph, Keith, Jason, Doug, Leon and Co This patchset adds a new verbs API for T10-PI offload and implementation for iSER initiator (NVMe-oF/RDMA host side was completed and will be sent on a different patchset). This set starts with a few preparation commits to the RDMA/core layer. It continues with introducing a new MR type IB_MR_TYPE_INTEGRITY. Using this MR all the needed mappings will be done in the low level drivers and not be visible to the ULP. Later patches implement the needed functionality in the mlx5 layer. As suggested by Sagi, in the new API, the mlx5 driver will allocate a single internal memory region for the UMR operation to register both PI and data SG lists and it will look like: data start meta start | | ------------------------- |d1|d2|d3|d4|m1|m2|m3|m4| ------------------------- The sig_mr stride block would be using the same lkey but different offsets in it (offset 0 and offset d1+d2+d3+d4). The verbs layer will use a special mr type that will describe everything and will replace the old API, that enforce using 3 different memory regions (data_mr, protection_mr, sig_mr) and their local invalidations. This will ease the code in the ULP and will improve the abstraction of the HW (see iSER code changes). The patchset contains also iSER initator patches that using this new API. For iSER, the code was tested vs. LIO iSER target using Mellanox's ConnectX-4/ConnectX-5. This series applies cleanly on top of kernel 5.0 tag and iser fixs ("IB/iser: Pass the correct number of entries for dma mapped SGL") and ("IB/iser: Fix dma_nents type definition"). We should aim to push this code during 5.1 merge window. Next steps are: - use new API in iSER LIO target and remove the old API (this will cause changes in RDMA/core layer). - merge NVMe-oF/RDMA host side after merging this patchset - Implement metadata support for NVMe-oF/RDMA target side with new API --------- Changes since v2: - Rename IB_MR_TYPE_PI to IB_MR_TYPE_INTEGRITY (Sagi) - Rename IB_WR_REG_PI_MR to IB_WR_REG_MR_INTEGRITY (Sagi) - Refactor iser_login_rsp (Christoph) - Unwind WR union at iser_tx_desc (patch 16/18 - Christoph) - Rebase the code over kernel 5.0 plus 2 iser fixes - Added Reviewed-by signatures --------- Changes since v1: - Add a missing comma at patch 01/17 - Fix coding style at patches 03/17, 05/17 and 09/17 - Fix srp_map_finish_fr function at patch 04/17 - Rebase the code over 5.0-rc5 --------- Israel Rukshin (9): RDMA/core: Introduce IB_MR_TYPE_INTEGRITY and ib_alloc_mr_integrity API RMDA/core: Introduce ib_scatterlist structure IB/iser: Embed ib_scatterlist into iser_data_buf IB/srp: Embed ib_scatterlist into srp_map_state struct IB/iser: Refactor iscsi_iser_check_protection function IB/iser: Use IB_WR_REG_MR_INTEGRITY for PI handover IB/iser: Unwind WR union at iser_tx_desc IB/iser: Remove unused sig_attrs argument IB/isert: Remove unused sig_attrs argument Max Gurtovoy (9): RDMA/core: Introduce new header file for signature operations RDMA/core: Save the MR type in the ib_mr structure RDMA/core: Introduce ib_map_mr_sg_pi to map data/protection sgl's RDMA/core: Add signature attrs element for ib_mr structure RDMA/mlx5: Implement mlx5_ib_map_mr_sg_pi and mlx5_ib_alloc_mr_integrity RDMA/mlx5: Add attr for max number page list length for PI operation RDMA/mlx5: Pass UMR segment flags instead of boolean RDMA/mlx5: Update set_sig_data_segment attribute for new signature API RDMA/mlx5: Introduce and implement new IB_WR_REG_MR_INTEGRITY work request drivers/infiniband/core/device.c | 2 + drivers/infiniband/core/rw.c | 6 +- drivers/infiniband/core/uverbs_cmd.c | 2 + drivers/infiniband/core/verbs.c | 127 ++++++++++-- drivers/infiniband/hw/bnxt_re/ib_verbs.c | 5 +- drivers/infiniband/hw/bnxt_re/ib_verbs.h | 3 +- drivers/infiniband/hw/cxgb3/iwch_provider.c | 5 +- drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 3 +- drivers/infiniband/hw/cxgb4/mem.c | 5 +- drivers/infiniband/hw/hns/hns_roce_device.h | 3 +- drivers/infiniband/hw/hns/hns_roce_mr.c | 5 +- drivers/infiniband/hw/i40iw/i40iw_verbs.c | 8 +- drivers/infiniband/hw/mlx4/mlx4_ib.h | 3 +- drivers/infiniband/hw/mlx4/mr.c | 5 +- drivers/infiniband/hw/mlx5/main.c | 4 + drivers/infiniband/hw/mlx5/mlx5_ib.h | 12 +- drivers/infiniband/hw/mlx5/mr.c | 192 +++++++++++++++--- drivers/infiniband/hw/mlx5/qp.c | 248 ++++++++++++++++++++---- drivers/infiniband/hw/nes/nes_verbs.c | 5 +- drivers/infiniband/hw/ocrdma/ocrdma_verbs.c | 5 +- drivers/infiniband/hw/ocrdma/ocrdma_verbs.h | 3 +- drivers/infiniband/hw/qedr/verbs.c | 5 +- drivers/infiniband/hw/qedr/verbs.h | 3 +- drivers/infiniband/hw/vmw_pvrdma/pvrdma_mr.c | 5 +- drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h | 3 +- drivers/infiniband/sw/rdmavt/mr.c | 10 +- drivers/infiniband/sw/rdmavt/mr.h | 3 +- drivers/infiniband/sw/rxe/rxe_verbs.c | 5 +- drivers/infiniband/ulp/iser/iscsi_iser.c | 12 +- drivers/infiniband/ulp/iser/iscsi_iser.h | 78 ++------ drivers/infiniband/ulp/iser/iser_initiator.c | 18 +- drivers/infiniband/ulp/iser/iser_memory.c | 148 ++++++-------- drivers/infiniband/ulp/iser/iser_verbs.c | 103 ++++------ drivers/infiniband/ulp/isert/ib_isert.c | 11 +- drivers/infiniband/ulp/srp/ib_srp.c | 46 +++-- drivers/infiniband/ulp/srp/ib_srp.h | 2 +- drivers/nvme/host/rdma.c | 6 +- fs/cifs/smbdirect.c | 7 +- include/linux/mlx5/qp.h | 3 +- include/rdma/ib_verbs.h | 163 ++++------------ include/rdma/signature.h | 120 ++++++++++++ net/rds/ib_frmr.c | 9 +- net/smc/smc_ib.c | 9 +- net/sunrpc/xprtrdma/frwr_ops.c | 6 +- 44 files changed, 897 insertions(+), 529 deletions(-) create mode 100644 include/rdma/signature.h