[v1,0/7] Switch NFS server CQs to use soft IRQ

Message

Chuck Lever Nov. 13, 2023, 1:42 p.m. UTC

I've found that using the ib_comp_wq for completion queues can add
microseconds of latency in the average case and extend tail
latencies significantly. Now that we've shaved a bunch of work out
of the server's completion handlers, it should be safe to go back to
using soft IRQ rather than ib_comp_wq, in order to gain better RTT
latencies.

---

Chuck Lever (7):
      svcrdma: Eliminate allocation of recv_ctxt objects in backchannel
      svcrdma: Clean up use of rdma->sc_pd->device in Receive paths
      svcrdma: Pre-allocate svc_rdma_recv_ctxt objects
      svcrdma: Switch Receive CQ to soft IRQ
      svcrdma: Clean up use of rdma->sc_pd->device
      svcrdma: Move the svcxprt_rdma::sc_pd field
      svcrdma: Move Send CQ to SOFTIRQ context


 include/linux/sunrpc/svc_rdma.h            |  6 +--
 net/sunrpc/xprtrdma/svc_rdma_backchannel.c | 11 ++--
 net/sunrpc/xprtrdma/svc_rdma_recvfrom.c    | 53 +++++++++++--------
 net/sunrpc/xprtrdma/svc_rdma_rw.c          |  4 +-
 net/sunrpc/xprtrdma/svc_rdma_sendto.c      | 59 ++++++++++++----------
 net/sunrpc/xprtrdma/svc_rdma_transport.c   |  6 +--
 6 files changed, 75 insertions(+), 64 deletions(-)

--
Chuck Lever