| Message ID | 20250314081056.3496708-1-matsuda-daisuke@fujitsu.com (mailing list archive) |
|---|---|
| Headers | show |
| Series | RDMA/rxe: RDMA FLUSH and ATOMIC WRITE with ODP | expand |
在 2025/3/14 9:10, Daisuke Matsuda 写道: > RDMA FLUSH[1] and ATOMIC WRITE[2] were added to rxe, but they cannot run > in the ODP mode as of now. This series is for the kernel-side enablement. > > There are also minor changes in libibverbs and pyverbs. The rdma-core tests > are also added so that people can test the features. > PR: https://github.com/linux-rdma/rdma-core/pull/1580 > > You can try the patches with the tree below: > https://github.com/ddmatsu/linux/tree/odp-extension > > Note that the tree is a bit old (6.13-rc1), because there was an issue[3] > in the for-next tree that disabled ibv_query_device_ex(), which is used to > query ODP capabilities. However, there is already a fix[4], and it is to be > resolved in the next release. I will update the tree once it is ready. > > [1] [for-next PATCH 00/10] RDMA/rxe: Add RDMA FLUSH operation > https://lore.kernel.org/lkml/20221206130201.30986-1-lizhijian@fujitsu.com/ > > [2] [PATCH v7 0/8] RDMA/rxe: Add atomic write operation > https://lore.kernel.org/linux-rdma/1669905432-14-1-git-send-email-yangx.jy@fujitsu.com/ > > [3] [bug report] RDMA/rxe: Failure of ibv_query_device() and ibv_query_device_ex() tests in rdma-core > https://lore.kernel.org/all/1b9d6286-62fc-4b42-b304-0054c4ebee02@linux.dev/T/ > > [4] [PATCH rdma-rc 1/1] RDMA/rxe: Fix the failure of ibv_query_device() and ibv_query_device_ex() tests > https://lore.kernel.org/linux-rdma/174102882930.42565.11864314726635251412.b4-ty@kernel.org/T/#t Today I read these commits carefully. The 2 commits introduces ATOMIC_WRITE and ATOMIC_FLUSH operations with ODP enabled. In the rdma-core, the corresponding test cases are also added. I am fine with these 2 commits. But I notice that there are no perftest results with the 2 operations. Perftest is a stress-test tools. With this tool, it can test the 2 commits with some stress. Anyway, I am fine with the 2 commits. It is better if the perftest results are attached. Zhu Yanjun > > Daisuke Matsuda (2): > RDMA/rxe: Enable ODP in RDMA FLUSH operation > RDMA/rxe: Enable ODP in ATOMIC WRITE operation > > drivers/infiniband/sw/rxe/rxe.c | 2 + > drivers/infiniband/sw/rxe/rxe_loc.h | 12 +++ > drivers/infiniband/sw/rxe/rxe_mr.c | 4 - > drivers/infiniband/sw/rxe/rxe_odp.c | 132 ++++++++++++++++++++++++++- > drivers/infiniband/sw/rxe/rxe_resp.c | 18 ++-- > include/rdma/ib_verbs.h | 2 + > 6 files changed, 155 insertions(+), 15 deletions(-) >
On Sun, March 16, 2025 4:21 AM Zhu Yanjun wrote: > 在 2025/3/14 9:10, Daisuke Matsuda 写道: > > RDMA FLUSH[1] and ATOMIC WRITE[2] were added to rxe, but they cannot run > > in the ODP mode as of now. This series is for the kernel-side enablement. > > > > There are also minor changes in libibverbs and pyverbs. The rdma-core tests > > are also added so that people can test the features. > > PR: https://github.com/linux-rdma/rdma-core/pull/1580 > > > > You can try the patches with the tree below: > > https://github.com/ddmatsu/linux/tree/odp-extension > > > > Note that the tree is a bit old (6.13-rc1), because there was an issue[3] > > in the for-next tree that disabled ibv_query_device_ex(), which is used to > > query ODP capabilities. However, there is already a fix[4], and it is to be > > resolved in the next release. I will update the tree once it is ready. > > > > [1] [for-next PATCH 00/10] RDMA/rxe: Add RDMA FLUSH operation > > https://lore.kernel.org/lkml/20221206130201.30986-1-lizhijian@fujitsu.com/ > > > > [2] [PATCH v7 0/8] RDMA/rxe: Add atomic write operation > > https://lore.kernel.org/linux-rdma/1669905432-14-1-git-send-email-yangx.jy@fujitsu.com/ > > > > [3] [bug report] RDMA/rxe: Failure of ibv_query_device() and ibv_query_device_ex() tests in rdma-core > > https://lore.kernel.org/all/1b9d6286-62fc-4b42-b304-0054c4ebee02@linux.dev/T/ > > > > [4] [PATCH rdma-rc 1/1] RDMA/rxe: Fix the failure of ibv_query_device() and ibv_query_device_ex() tests > > https://lore.kernel.org/linux-rdma/174102882930.42565.11864314726635251412.b4-ty@kernel.org/T/#t > > Today I read these commits carefully. The 2 commits introduces > ATOMIC_WRITE and ATOMIC_FLUSH operations with ODP enabled. In the > rdma-core, the corresponding test cases are also added. I am fine with > these 2 commits. > > But I notice that there are no perftest results with the 2 operations. > Perftest is a stress-test tools. With this tool, it can test the 2 > commits with some stress. Hi Zhu, Thank you for the review. I cannot measure the 2 operations with perftest right now because the tool does not support them right now. However, they should ideally be supported before HW HCAs start to enable RDMA FLUSH and ATOMIC WRITE. After I complete my works for rxe ODP features, I think I can see to it. Using rxe will be very helpful in doing that. Thanks, Daisuke Matsuda > > Anyway, I am fine with the 2 commits. It is better if the perftest > results are attached. > > Zhu Yanjun