| Message ID | 20210408170123.8788-15-logang@deltatee.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Add new DMA mapping operation for P2PDMA | expand |
On Thu, Apr 08, 2021 at 11:01:21AM -0600, Logan Gunthorpe wrote: > Ensure the dma operations support p2pdma before using the RDMA > device for P2PDMA. This allows switching the RDMA driver from > pci_p2pdma_map_sg() to dma_map_sg_p2pdma(). > > Signed-off-by: Logan Gunthorpe <logang@deltatee.com> > drivers/nvme/target/rdma.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c > index 6c1f3ab7649c..3ec7e77e5416 100644 > +++ b/drivers/nvme/target/rdma.c > @@ -414,7 +414,8 @@ static int nvmet_rdma_alloc_rsp(struct nvmet_rdma_device *ndev, > if (ib_dma_mapping_error(ndev->device, r->send_sge.addr)) > goto out_free_rsp; > > - if (!ib_uses_virt_dma(ndev->device)) > + if (!ib_uses_virt_dma(ndev->device) && > + dma_pci_p2pdma_supported(&ndev->device->dev)) ib_uses_virt_dma() should not be called by nvme and this is using the wrong device pointer to query for DMA related properties. I suspect this wants a ib_dma_pci_p2p_dma_supported() wrapper like everything else. Jason
On 2021-04-27 1:47 p.m., Jason Gunthorpe wrote: > On Thu, Apr 08, 2021 at 11:01:21AM -0600, Logan Gunthorpe wrote: >> Ensure the dma operations support p2pdma before using the RDMA >> device for P2PDMA. This allows switching the RDMA driver from >> pci_p2pdma_map_sg() to dma_map_sg_p2pdma(). >> >> Signed-off-by: Logan Gunthorpe <logang@deltatee.com> >> drivers/nvme/target/rdma.c | 3 ++- >> 1 file changed, 2 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c >> index 6c1f3ab7649c..3ec7e77e5416 100644 >> +++ b/drivers/nvme/target/rdma.c >> @@ -414,7 +414,8 @@ static int nvmet_rdma_alloc_rsp(struct nvmet_rdma_device *ndev, >> if (ib_dma_mapping_error(ndev->device, r->send_sge.addr)) >> goto out_free_rsp; >> >> - if (!ib_uses_virt_dma(ndev->device)) >> + if (!ib_uses_virt_dma(ndev->device) && >> + dma_pci_p2pdma_supported(&ndev->device->dev)) > > ib_uses_virt_dma() should not be called by nvme and this is using the > wrong device pointer to query for DMA related properties. > > I suspect this wants a ib_dma_pci_p2p_dma_supported() wrapper like > everything else. Makes sense. Will add for v2. Logan
On 4/8/21 10:01 AM, Logan Gunthorpe wrote: > Ensure the dma operations support p2pdma before using the RDMA > device for P2PDMA. This allows switching the RDMA driver from > pci_p2pdma_map_sg() to dma_map_sg_p2pdma(). Tentatively, this looks right, but it really should be combined with a following patch that uses it. Then you don't have to try to explain, above, why it's needed. :) > > Signed-off-by: Logan Gunthorpe <logang@deltatee.com> > --- > drivers/nvme/target/rdma.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c > index 6c1f3ab7649c..3ec7e77e5416 100644 > --- a/drivers/nvme/target/rdma.c > +++ b/drivers/nvme/target/rdma.c > @@ -414,7 +414,8 @@ static int nvmet_rdma_alloc_rsp(struct nvmet_rdma_device *ndev, > if (ib_dma_mapping_error(ndev->device, r->send_sge.addr)) > goto out_free_rsp; > > - if (!ib_uses_virt_dma(ndev->device)) > + if (!ib_uses_virt_dma(ndev->device) && > + dma_pci_p2pdma_supported(&ndev->device->dev)) > r->req.p2p_client = &ndev->device->dev; > r->send_sge.length = sizeof(*r->req.cqe); > r->send_sge.lkey = ndev->pd->local_dma_lkey; > thanks,
diff --git a/drivers/nvme/target/rdma.c b/drivers/nvme/target/rdma.c index 6c1f3ab7649c..3ec7e77e5416 100644 --- a/drivers/nvme/target/rdma.c +++ b/drivers/nvme/target/rdma.c @@ -414,7 +414,8 @@ static int nvmet_rdma_alloc_rsp(struct nvmet_rdma_device *ndev, if (ib_dma_mapping_error(ndev->device, r->send_sge.addr)) goto out_free_rsp; - if (!ib_uses_virt_dma(ndev->device)) + if (!ib_uses_virt_dma(ndev->device) && + dma_pci_p2pdma_supported(&ndev->device->dev)) r->req.p2p_client = &ndev->device->dev; r->send_sge.length = sizeof(*r->req.cqe); r->send_sge.lkey = ndev->pd->local_dma_lkey;
Ensure the dma operations support p2pdma before using the RDMA device for P2PDMA. This allows switching the RDMA driver from pci_p2pdma_map_sg() to dma_map_sg_p2pdma(). Signed-off-by: Logan Gunthorpe <logang@deltatee.com> --- drivers/nvme/target/rdma.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)