| Message ID | 20201015201750.8336-1-rpearson@hpe.com (mailing list archive) |
|---|---|
| State | Superseded |
| Headers | show |
| Series | Provider/rxe: Fix regression to UD traffic | expand |
On Thu, Oct 15, 2020 at 03:17:51PM -0500, Bob Pearson wrote: > Update enum rdma_network_type copy to match kernel version. > Without this change provider/rxe will send incorrect > network types to the kernel in send WQEs. > > This fix keeps rxe functional but should be replaced by a better > implementation. > > Signed-off-by: Bob Pearson <rpearson@hpe.com> > --- > providers/rxe/rxe.h | 1 + > 1 file changed, 1 insertion(+) Well, we can't just break user space so the kernel has to change in some way to accommodate this. Obviously rxe should not have uAPI stuff that is not in include/uapi/rdma, so lets just fix that directly. Please confirm I did this right. The PR for this merge window must be sent Friday. From 8b20e1ceea413c98cb98930cb549be390226320f Mon Sep 17 00:00:00 2001 From: Jason Gunthorpe <jgg@nvidia.com> Date: Thu, 15 Oct 2020 20:42:18 -0300 Subject: [PATCH] RDMA/rxe: Move the definitions for rxe_av.network_type to uAPI RXE was wrongly using an internal kernel enum as part of its uAPI, split this out into a dedicated uAPI enum just for RXE. It only uses the IPv4 and IPv6 values. This was exposed by changing the internal kernel enum definition which broke RXE. Fixes: 1c15b4f2a42f ("RDMA/core: Modify enum ib_gid_type and enum rdma_network_type") Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> --- drivers/infiniband/sw/rxe/rxe_net.c | 8 ++++---- include/uapi/rdma/rdma_user_rxe.h | 6 ++++++ 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/drivers/infiniband/sw/rxe/rxe_net.c b/drivers/infiniband/sw/rxe/rxe_net.c index 31b93e7e1e2f41..575e1a4ec82121 100644 --- a/drivers/infiniband/sw/rxe/rxe_net.c +++ b/drivers/infiniband/sw/rxe/rxe_net.c @@ -133,14 +133,14 @@ static struct dst_entry *rxe_find_route(struct net_device *ndev, if (dst) dst_release(dst); - if (av->network_type == RDMA_NETWORK_IPV4) { + if (av->network_type == RXE_NETWORK_TYPE_IPV4) { struct in_addr *saddr; struct in_addr *daddr; saddr = &av->sgid_addr._sockaddr_in.sin_addr; daddr = &av->dgid_addr._sockaddr_in.sin_addr; dst = rxe_find_route4(ndev, saddr, daddr); - } else if (av->network_type == RDMA_NETWORK_IPV6) { + } else if (av->network_type == RXE_NETWORK_TYPE_IPV6) { struct in6_addr *saddr6; struct in6_addr *daddr6; @@ -442,7 +442,7 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, if (IS_ERR(attr)) return NULL; - if (av->network_type == RDMA_NETWORK_IPV4) + if (av->network_type == RXE_NETWORK_TYPE_IPV6) hdr_len = ETH_HLEN + sizeof(struct udphdr) + sizeof(struct iphdr); else @@ -469,7 +469,7 @@ struct sk_buff *rxe_init_packet(struct rxe_dev *rxe, struct rxe_av *av, skb->dev = ndev; rcu_read_unlock(); - if (av->network_type == RDMA_NETWORK_IPV4) + if (av->network_type == RXE_NETWORK_TYPE_IPV4) skb->protocol = htons(ETH_P_IP); else skb->protocol = htons(ETH_P_IPV6); diff --git a/include/uapi/rdma/rdma_user_rxe.h b/include/uapi/rdma/rdma_user_rxe.h index d8f2e0e46daba7..e591d8c1f3cf10 100644 --- a/include/uapi/rdma/rdma_user_rxe.h +++ b/include/uapi/rdma/rdma_user_rxe.h @@ -39,6 +39,11 @@ #include <linux/in.h> #include <linux/in6.h> +enum { + RXE_NETWORK_TYPE_IPV4 = 1, + RXE_NETWORK_TYPE_IPV6 = 2, +}; + union rxe_gid { __u8 raw[16]; struct { @@ -57,6 +62,7 @@ struct rxe_global_route { struct rxe_av { __u8 port_num; + /* From RXE_NETWORK_TYPE_* */ __u8 network_type; __u8 dmac[6]; struct rxe_global_route grh;
On 10/15/20 6:47 PM, Jason Gunthorpe wrote: > On Thu, Oct 15, 2020 at 03:17:51PM -0500, Bob Pearson wrote: >> Update enum rdma_network_type copy to match kernel version. >> Without this change provider/rxe will send incorrect >> network types to the kernel in send WQEs. >> >> This fix keeps rxe functional but should be replaced by a better >> implementation. >> >> Signed-off-by: Bob Pearson <rpearson@hpe.com> >> --- >> providers/rxe/rxe.h | 1 + >> 1 file changed, 1 insertion(+) > > Well, we can't just break user space so the kernel has to change in > some way to accommodate this. > > Obviously rxe should not have uAPI stuff that is not in > include/uapi/rdma, so lets just fix that directly. > > Please confirm I did this right. The PR for this merge window must be > sent Friday. That works too. I am figuring out how to convert ah_handle to ib_ah in rdma_rxe. I think I've mostly got it. uobj_get_obj_read(type, id, attr) is close to what I need. The problem is that all the code in this area assumes it is getting called in a user verb API call so there is an attr to pass around. What we need is an API like rdma_get_obj(ucontext, type, id) since we need a way to get to the user api data structures. Better would be a pointer from qp to ucontext or ufile that would be set for user QPs. If we did this rxe just use handles from the core instead of creating its own databases of objects. Your patch will get us going for now. The above is a longer project that won't fit in your short term schedule. Bob
diff --git a/providers/rxe/rxe.h b/providers/rxe/rxe.h index 96f4ee9c..46e6ce72 100644 --- a/providers/rxe/rxe.h +++ b/providers/rxe/rxe.h @@ -44,6 +44,7 @@ enum rdma_network_type { RDMA_NETWORK_IB, + RDMA_NETWORK_ROCE_V1, RDMA_NETWORK_IPV4, RDMA_NETWORK_IPV6 };
Update enum rdma_network_type copy to match kernel version. Without this change provider/rxe will send incorrect network types to the kernel in send WQEs. This fix keeps rxe functional but should be replaced by a better implementation. Signed-off-by: Bob Pearson <rpearson@hpe.com> --- providers/rxe/rxe.h | 1 + 1 file changed, 1 insertion(+)