| Message ID | 20180316031814.GA15450@ziepe.ca (mailing list archive) |
|---|---|
| State | Accepted |
| Delegated to: | Jason Gunthorpe |
| Headers | show |
On Thu, Mar 15, 2018 at 09:18:14PM -0600, Jason Gunthorpe wrote: > What is going on here is a bit subtle, in the kernel there is no > problem because the struct is copied using copy_from_user, so it > can safely have an 8 byte alignment, however in userspace it must > be constructed by concatenation with the ib_uverbs_alloc_pd_resp > struct. This is due to the required memory layout to execute the > command. > > Since ibv_uverbs_alloc_pd_resp is only 4 bytes long, this causes > misalignment, and the user space will experience an unexpected padding. > Currently it works around this via pointer maths. > > Make everything more robust by having the compiler reduce the alignment > of the struct to 4. The userspace has assertions to ensure this > works properly in all situations. > > Signed-off-by: Jason Gunthorpe <jgg@mellanox.com> > --- > include/uapi/rdma/bnxt_re-abi.h | 7 ++++++- > 1 file changed, 6 insertions(+), 1 deletion(-) Applied to for-next Jason -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
diff --git a/include/uapi/rdma/bnxt_re-abi.h b/include/uapi/rdma/bnxt_re-abi.h index db54115be0447a..df6c8d5bf0dab8 100644 --- a/include/uapi/rdma/bnxt_re-abi.h +++ b/include/uapi/rdma/bnxt_re-abi.h @@ -53,11 +53,16 @@ struct bnxt_re_uctx_resp { __u32 rsvd; }; +/* + * This struct is placed after the ib_uverbs_alloc_pd_resp struct, which is + * not 8 byted aligned. To avoid undesired padding in various cases we have to + * set this struct to packed. + */ struct bnxt_re_pd_resp { __u32 pdid; __u32 dpi; __u64 dbr; -}; +} __attribute__((packed,aligned(4))); struct bnxt_re_cq_req { __u64 cq_va;
What is going on here is a bit subtle, in the kernel there is no problem because the struct is copied using copy_from_user, so it can safely have an 8 byte alignment, however in userspace it must be constructed by concatenation with the ib_uverbs_alloc_pd_resp struct. This is due to the required memory layout to execute the command. Since ibv_uverbs_alloc_pd_resp is only 4 bytes long, this causes misalignment, and the user space will experience an unexpected padding. Currently it works around this via pointer maths. Make everything more robust by having the compiler reduce the alignment of the struct to 4. The userspace has assertions to ensure this works properly in all situations. Signed-off-by: Jason Gunthorpe <jgg@mellanox.com> --- include/uapi/rdma/bnxt_re-abi.h | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) I almost had a stroke when I saw this in rdma-core: if (ibv_cmd_alloc_pd(ibvctx, &pd->ibvpd, &cmd, sizeof(cmd), &resp.resp, sizeof(resp))) goto out; pd->pdid = resp.pdid; dbr = *(uint64_t *)((uint32_t *)&resp + 3); Something that crazy didn't even deserve a comment?!? Anyhow, the new userspace support for the kabi structs detected this when I converted your driver, so at least that get tested.. Jason