| Message ID | 20190605232348.6452-1-saeedm@mellanox.com (mailing list archive) |
|---|---|
| State | Changes Requested |
| Headers | show |
On Wed, Jun 05, 2019 at 11:24:31PM +0000, Saeed Mahameed wrote: > Hi Dave, Doug & Jason > > This series improves DIM - Dynamically-tuned Interrupt > Moderation- to be generic for netdev and RDMA use-cases. > > From Tal and Yamin: > The first 7 patches provide the necessary refactoring to current net_dim > library which affect some net drivers who are using the API. > > The last 2 patches provide the RDMA implementation for DIM. > > For more information please see tag log below. > > Once we are all happy with the series, please pull to net-next and > rdma-next trees. > > Thanks, > Saeed. > > --- > The following changes since commit cd6c84d8f0cdc911df435bb075ba22ce3c605b07: > > Linux 5.2-rc2 (2019-05-26 16:49:19 -0700) > > are available in the Git repository at: > > git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/dim-updates-2019-06-05 > > for you to fetch changes up to 1ec9974e75e7a58bff1ab17c4fcda17b180ed3bb: > > RDMA/core: Provide RDMA DIM support for ULPs (2019-06-05 16:09:02 -0700) > > ---------------------------------------------------------------- > dim-updates-2019-06-05 > > From: Tal Gilboa > > Implement net DIM over a generic DIM library > > net_dim.h lib exposes an implementation of the DIM algorithm for > dynamically-tuned interrupt moderation for networking interfaces. > > We want a similar functionality for other protocols, which might need to > optimize interrupts differently. Main motivation here is DIM for NVMf > storage protocol. > > Current DIM implementation prioritizes reducing interrupt overhead over > latency. Also, in order to reduce DIM's own overhead, the algorithm might > take some time to identify it needs to change profiles. While this is > acceptable for networking, it might not work well on other scenarios. > > Here I propose a new structure to DIM. The idea is to allow a slightly > modified functionality without the risk of breaking Net DIM behavior for > netdev. I verified there are no degradations in current DIM behavior with > the modified solution. > > Solution: > - Common logic is declared in include/linux/dim.h and implemented in > lib/dim/dim.c > - Net DIM (existing) logic is declared in include/linux/net_dim.h and > implemented in lib/dim/net_dim.c, which uses the common logic from dim.h > - Any new DIM logic will be declared in "/include/linux/new_dim.h" and > implemented in "lib/dim/new_dim.c". > - This new implementation will expose modified versions of profiles, > dim_step() and dim_decision(). > > Pros for this solution are: > - Zero impact on existing net_dim implementation and usage > - Relatively more code reuse (compared to two separate solutions) > - Increased extensibility > > ---------------------------------------------------------------- > Tal Gilboa (6): > linux/dim: Move logic to dim.h > linux/dim: Remove "net" prefix from internal DIM members > linux/dim: Rename externally exposed macros > linux/dim: Rename net_dim_sample() to net_dim_update_sample() > linux/dim: Rename externally used net_dim members > linux/dim: Move implementation to .c files > > Yamin Friedman (3): > linux/dim: Add completions count to dim_sample > linux/dim: Implement rdma_dim > RDMA/core: Provide RDMA DIM support for ULPs Saeed, No, for the RDMA patches. We need to see usage of those APIs before merging. Thanks
On 6/6/2019 10:14 AM, Leon Romanovsky wrote: > On Wed, Jun 05, 2019 at 11:24:31PM +0000, Saeed Mahameed wrote: >> Hi Dave, Doug & Jason >> >> This series improves DIM - Dynamically-tuned Interrupt >> Moderation- to be generic for netdev and RDMA use-cases. >> >> From Tal and Yamin: >> The first 7 patches provide the necessary refactoring to current net_dim >> library which affect some net drivers who are using the API. >> >> The last 2 patches provide the RDMA implementation for DIM. >> >> For more information please see tag log below. >> >> Once we are all happy with the series, please pull to net-next and >> rdma-next trees. >> >> Thanks, >> Saeed. >> >> --- >> The following changes since commit cd6c84d8f0cdc911df435bb075ba22ce3c605b07: >> >> Linux 5.2-rc2 (2019-05-26 16:49:19 -0700) >> >> are available in the Git repository at: >> >> git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/dim-updates-2019-06-05 >> >> for you to fetch changes up to 1ec9974e75e7a58bff1ab17c4fcda17b180ed3bb: >> >> RDMA/core: Provide RDMA DIM support for ULPs (2019-06-05 16:09:02 -0700) >> >> ---------------------------------------------------------------- >> dim-updates-2019-06-05 >> >> From: Tal Gilboa >> >> Implement net DIM over a generic DIM library >> >> net_dim.h lib exposes an implementation of the DIM algorithm for >> dynamically-tuned interrupt moderation for networking interfaces. >> >> We want a similar functionality for other protocols, which might need to >> optimize interrupts differently. Main motivation here is DIM for NVMf >> storage protocol. >> >> Current DIM implementation prioritizes reducing interrupt overhead over >> latency. Also, in order to reduce DIM's own overhead, the algorithm might >> take some time to identify it needs to change profiles. While this is >> acceptable for networking, it might not work well on other scenarios. >> >> Here I propose a new structure to DIM. The idea is to allow a slightly >> modified functionality without the risk of breaking Net DIM behavior for >> netdev. I verified there are no degradations in current DIM behavior with >> the modified solution. >> >> Solution: >> - Common logic is declared in include/linux/dim.h and implemented in >> lib/dim/dim.c >> - Net DIM (existing) logic is declared in include/linux/net_dim.h and >> implemented in lib/dim/net_dim.c, which uses the common logic from dim.h >> - Any new DIM logic will be declared in "/include/linux/new_dim.h" and >> implemented in "lib/dim/new_dim.c". >> - This new implementation will expose modified versions of profiles, >> dim_step() and dim_decision(). >> >> Pros for this solution are: >> - Zero impact on existing net_dim implementation and usage >> - Relatively more code reuse (compared to two separate solutions) >> - Increased extensibility >> >> ---------------------------------------------------------------- >> Tal Gilboa (6): >> linux/dim: Move logic to dim.h >> linux/dim: Remove "net" prefix from internal DIM members >> linux/dim: Rename externally exposed macros >> linux/dim: Rename net_dim_sample() to net_dim_update_sample() >> linux/dim: Rename externally used net_dim members >> linux/dim: Move implementation to .c files >> >> Yamin Friedman (3): >> linux/dim: Add completions count to dim_sample >> linux/dim: Implement rdma_dim >> RDMA/core: Provide RDMA DIM support for ULPs > Saeed, > > No, for the RDMA patches. > We need to see usage of those APIs before merging. I've asked Yamin to prepare patches for NVMeoF initiator and target for review, so I guess he has it on his plate (this is how he tested it..). It might cause conflict with NVMe/blk branch maintained by Sagi, Christoph and Jens. So we need a plan here. > > Thanks
On Thu, Jun 06, 2019 at 10:19:41AM +0300, Max Gurtovoy wrote: > > On 6/6/2019 10:14 AM, Leon Romanovsky wrote: > > On Wed, Jun 05, 2019 at 11:24:31PM +0000, Saeed Mahameed wrote: > > > Hi Dave, Doug & Jason > > > > > > This series improves DIM - Dynamically-tuned Interrupt > > > Moderation- to be generic for netdev and RDMA use-cases. > > > > > > From Tal and Yamin: > > > The first 7 patches provide the necessary refactoring to current net_dim > > > library which affect some net drivers who are using the API. > > > > > > The last 2 patches provide the RDMA implementation for DIM. > > > > > > For more information please see tag log below. > > > > > > Once we are all happy with the series, please pull to net-next and > > > rdma-next trees. > > > > > > Thanks, > > > Saeed. > > > > > > --- > > > The following changes since commit cd6c84d8f0cdc911df435bb075ba22ce3c605b07: > > > > > > Linux 5.2-rc2 (2019-05-26 16:49:19 -0700) > > > > > > are available in the Git repository at: > > > > > > git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/dim-updates-2019-06-05 > > > > > > for you to fetch changes up to 1ec9974e75e7a58bff1ab17c4fcda17b180ed3bb: > > > > > > RDMA/core: Provide RDMA DIM support for ULPs (2019-06-05 16:09:02 -0700) > > > > > > ---------------------------------------------------------------- > > > dim-updates-2019-06-05 > > > > > > From: Tal Gilboa > > > > > > Implement net DIM over a generic DIM library > > > > > > net_dim.h lib exposes an implementation of the DIM algorithm for > > > dynamically-tuned interrupt moderation for networking interfaces. > > > > > > We want a similar functionality for other protocols, which might need to > > > optimize interrupts differently. Main motivation here is DIM for NVMf > > > storage protocol. > > > > > > Current DIM implementation prioritizes reducing interrupt overhead over > > > latency. Also, in order to reduce DIM's own overhead, the algorithm might > > > take some time to identify it needs to change profiles. While this is > > > acceptable for networking, it might not work well on other scenarios. > > > > > > Here I propose a new structure to DIM. The idea is to allow a slightly > > > modified functionality without the risk of breaking Net DIM behavior for > > > netdev. I verified there are no degradations in current DIM behavior with > > > the modified solution. > > > > > > Solution: > > > - Common logic is declared in include/linux/dim.h and implemented in > > > lib/dim/dim.c > > > - Net DIM (existing) logic is declared in include/linux/net_dim.h and > > > implemented in lib/dim/net_dim.c, which uses the common logic from dim.h > > > - Any new DIM logic will be declared in "/include/linux/new_dim.h" and > > > implemented in "lib/dim/new_dim.c". > > > - This new implementation will expose modified versions of profiles, > > > dim_step() and dim_decision(). > > > > > > Pros for this solution are: > > > - Zero impact on existing net_dim implementation and usage > > > - Relatively more code reuse (compared to two separate solutions) > > > - Increased extensibility > > > > > > ---------------------------------------------------------------- > > > Tal Gilboa (6): > > > linux/dim: Move logic to dim.h > > > linux/dim: Remove "net" prefix from internal DIM members > > > linux/dim: Rename externally exposed macros > > > linux/dim: Rename net_dim_sample() to net_dim_update_sample() > > > linux/dim: Rename externally used net_dim members > > > linux/dim: Move implementation to .c files > > > > > > Yamin Friedman (3): > > > linux/dim: Add completions count to dim_sample > > > linux/dim: Implement rdma_dim > > > RDMA/core: Provide RDMA DIM support for ULPs > > Saeed, > > > > No, for the RDMA patches. > > We need to see usage of those APIs before merging. > > I've asked Yamin to prepare patches for NVMeoF initiator and target for > review, so I guess he has it on his plate (this is how he tested it..). > > It might cause conflict with NVMe/blk branch maintained by Sagi, Christoph > and Jens. > > So we need a plan here. We are on top of it and will handle. Thanks > > > > > > Thanks
On Thu, Jun 06, 2019 at 10:19:41AM +0300, Max Gurtovoy wrote: > > > Solution: > > > - Common logic is declared in include/linux/dim.h and implemented in > > > lib/dim/dim.c > > > - Net DIM (existing) logic is declared in include/linux/net_dim.h and > > > implemented in lib/dim/net_dim.c, which uses the common logic from dim.h > > > - Any new DIM logic will be declared in "/include/linux/new_dim.h" and > > > implemented in "lib/dim/new_dim.c". > > > - This new implementation will expose modified versions of profiles, > > > dim_step() and dim_decision(). > > > > > > Pros for this solution are: > > > - Zero impact on existing net_dim implementation and usage > > > - Relatively more code reuse (compared to two separate solutions) > > > - Increased extensibility > > > > > > Tal Gilboa (6): > > > linux/dim: Move logic to dim.h > > > linux/dim: Remove "net" prefix from internal DIM members > > > linux/dim: Rename externally exposed macros > > > linux/dim: Rename net_dim_sample() to net_dim_update_sample() > > > linux/dim: Rename externally used net_dim members > > > linux/dim: Move implementation to .c files > > > > > > Yamin Friedman (3): > > > linux/dim: Add completions count to dim_sample > > > linux/dim: Implement rdma_dim > > > RDMA/core: Provide RDMA DIM support for ULPs > > Saeed, > > > > No, for the RDMA patches. > > We need to see usage of those APIs before merging. > > I've asked Yamin to prepare patches for NVMeoF initiator and target for > review, so I guess he has it on his plate (this is how he tested it..). > > It might cause conflict with NVMe/blk branch maintained by Sagi, Christoph > and Jens. It looks like nvme could pull this series + the RDMA patches into the nvme tree via PR? I'm not familiar with how that tree works. But we need to get the patches posted right away.. Jason
On Thu, 2019-06-06 at 13:07 +0000, Jason Gunthorpe wrote: > On Thu, Jun 06, 2019 at 10:19:41AM +0300, Max Gurtovoy wrote: > > > > Solution: > > > > - Common logic is declared in include/linux/dim.h and > > > > implemented in > > > > lib/dim/dim.c > > > > - Net DIM (existing) logic is declared in > > > > include/linux/net_dim.h and > > > > implemented in lib/dim/net_dim.c, which uses the common > > > > logic from dim.h > > > > - Any new DIM logic will be declared in > > > > "/include/linux/new_dim.h" and > > > > implemented in "lib/dim/new_dim.c". > > > > - This new implementation will expose modified versions of > > > > profiles, > > > > dim_step() and dim_decision(). > > > > > > > > Pros for this solution are: > > > > - Zero impact on existing net_dim implementation and usage > > > > - Relatively more code reuse (compared to two separate > > > > solutions) > > > > - Increased extensibility > > > > > > > > Tal Gilboa (6): > > > > linux/dim: Move logic to dim.h > > > > linux/dim: Remove "net" prefix from internal DIM members > > > > linux/dim: Rename externally exposed macros > > > > linux/dim: Rename net_dim_sample() to > > > > net_dim_update_sample() > > > > linux/dim: Rename externally used net_dim members > > > > linux/dim: Move implementation to .c files > > > > > > > > Yamin Friedman (3): > > > > linux/dim: Add completions count to dim_sample > > > > linux/dim: Implement rdma_dim > > > > RDMA/core: Provide RDMA DIM support for ULPs > > > Saeed, > > > > > > No, for the RDMA patches. > > > We need to see usage of those APIs before merging. > > > > I've asked Yamin to prepare patches for NVMeoF initiator and target > > for > > review, so I guess he has it on his plate (this is how he tested > > it..). > > > > It might cause conflict with NVMe/blk branch maintained by Sagi, > > Christoph > > and Jens. > > It looks like nvme could pull this series + the RDMA patches into the > nvme tree via PR? I'm not familiar with how that tree works. > > But we need to get the patches posted right away.. > What do you suggest here ? I think the netdev community also deserve to see the rdma patches, at least with an external link, I can drop the last patch (or two ) ? but i need an external rdma link for people who are going to review this series. > Jason
On Fri, Jun 07, 2019 at 06:14:11PM +0000, Saeed Mahameed wrote: > On Thu, 2019-06-06 at 13:07 +0000, Jason Gunthorpe wrote: > > On Thu, Jun 06, 2019 at 10:19:41AM +0300, Max Gurtovoy wrote: > > > > > Solution: > > > > > - Common logic is declared in include/linux/dim.h and > > > > > implemented in > > > > > lib/dim/dim.c > > > > > - Net DIM (existing) logic is declared in > > > > > include/linux/net_dim.h and > > > > > implemented in lib/dim/net_dim.c, which uses the common > > > > > logic from dim.h > > > > > - Any new DIM logic will be declared in > > > > > "/include/linux/new_dim.h" and > > > > > implemented in "lib/dim/new_dim.c". > > > > > - This new implementation will expose modified versions of > > > > > profiles, > > > > > dim_step() and dim_decision(). > > > > > > > > > > Pros for this solution are: > > > > > - Zero impact on existing net_dim implementation and usage > > > > > - Relatively more code reuse (compared to two separate > > > > > solutions) > > > > > - Increased extensibility > > > > > > > > > > Tal Gilboa (6): > > > > > linux/dim: Move logic to dim.h > > > > > linux/dim: Remove "net" prefix from internal DIM members > > > > > linux/dim: Rename externally exposed macros > > > > > linux/dim: Rename net_dim_sample() to > > > > > net_dim_update_sample() > > > > > linux/dim: Rename externally used net_dim members > > > > > linux/dim: Move implementation to .c files > > > > > > > > > > Yamin Friedman (3): > > > > > linux/dim: Add completions count to dim_sample > > > > > linux/dim: Implement rdma_dim > > > > > RDMA/core: Provide RDMA DIM support for ULPs > > > > Saeed, > > > > > > > > No, for the RDMA patches. > > > > We need to see usage of those APIs before merging. > > > > > > I've asked Yamin to prepare patches for NVMeoF initiator and target > > > for > > > review, so I guess he has it on his plate (this is how he tested > > > it..). > > > > > > It might cause conflict with NVMe/blk branch maintained by Sagi, > > > Christoph > > > and Jens. > > > > It looks like nvme could pull this series + the RDMA patches into the > > nvme tree via PR? I'm not familiar with how that tree works. > > > > But we need to get the patches posted right away.. > > > > What do you suggest here ? > I think the netdev community also deserve to see the rdma patches, at > least with an external link, I can drop the last patch (or two ) ? but > i need an external rdma link for people who are going to review this > series. Yes, all the patches need to be posted. We should have a 'double branch' where you send the linux/dim & net stuff to net and then we add the RDMA stuff on top and send to nvme & rdma with the ULP patches Assuming nvme takes pull requests. But the whole thing should be posted as a single series on the list to get acks before the PRs are generated. Similar to how we've run the mlx5 shared branch Jason
Hi Dave, Doug & Jason This series improves DIM - Dynamically-tuned Interrupt Moderation- to be generic for netdev and RDMA use-cases. From Tal and Yamin: The first 7 patches provide the necessary refactoring to current net_dim library which affect some net drivers who are using the API. The last 2 patches provide the RDMA implementation for DIM. For more information please see tag log below. Once we are all happy with the series, please pull to net-next and rdma-next trees. Thanks, Saeed. --- The following changes since commit cd6c84d8f0cdc911df435bb075ba22ce3c605b07: Linux 5.2-rc2 (2019-05-26 16:49:19 -0700) are available in the Git repository at: git://git.kernel.org/pub/scm/linux/kernel/git/saeed/linux.git tags/dim-updates-2019-06-05 for you to fetch changes up to 1ec9974e75e7a58bff1ab17c4fcda17b180ed3bb: RDMA/core: Provide RDMA DIM support for ULPs (2019-06-05 16:09:02 -0700) ---------------------------------------------------------------- dim-updates-2019-06-05 From: Tal Gilboa Implement net DIM over a generic DIM library net_dim.h lib exposes an implementation of the DIM algorithm for dynamically-tuned interrupt moderation for networking interfaces. We want a similar functionality for other protocols, which might need to optimize interrupts differently. Main motivation here is DIM for NVMf storage protocol. Current DIM implementation prioritizes reducing interrupt overhead over latency. Also, in order to reduce DIM's own overhead, the algorithm might take some time to identify it needs to change profiles. While this is acceptable for networking, it might not work well on other scenarios. Here I propose a new structure to DIM. The idea is to allow a slightly modified functionality without the risk of breaking Net DIM behavior for netdev. I verified there are no degradations in current DIM behavior with the modified solution. Solution: - Common logic is declared in include/linux/dim.h and implemented in lib/dim/dim.c - Net DIM (existing) logic is declared in include/linux/net_dim.h and implemented in lib/dim/net_dim.c, which uses the common logic from dim.h - Any new DIM logic will be declared in "/include/linux/new_dim.h" and implemented in "lib/dim/new_dim.c". - This new implementation will expose modified versions of profiles, dim_step() and dim_decision(). Pros for this solution are: - Zero impact on existing net_dim implementation and usage - Relatively more code reuse (compared to two separate solutions) - Increased extensibility ---------------------------------------------------------------- Tal Gilboa (6): linux/dim: Move logic to dim.h linux/dim: Remove "net" prefix from internal DIM members linux/dim: Rename externally exposed macros linux/dim: Rename net_dim_sample() to net_dim_update_sample() linux/dim: Rename externally used net_dim members linux/dim: Move implementation to .c files Yamin Friedman (3): linux/dim: Add completions count to dim_sample linux/dim: Implement rdma_dim RDMA/core: Provide RDMA DIM support for ULPs MAINTAINERS | 3 + drivers/infiniband/core/cq.c | 78 ++++- drivers/net/ethernet/broadcom/Kconfig | 1 + drivers/net/ethernet/broadcom/bcmsysport.c | 20 +- drivers/net/ethernet/broadcom/bcmsysport.h | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 12 +- drivers/net/ethernet/broadcom/bnxt/bnxt.h | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt_debugfs.c | 4 +- drivers/net/ethernet/broadcom/bnxt/bnxt_dim.c | 7 +- drivers/net/ethernet/broadcom/genet/bcmgenet.c | 18 +- drivers/net/ethernet/broadcom/genet/bcmgenet.h | 2 +- drivers/net/ethernet/mellanox/mlx4/Kconfig | 1 + drivers/net/ethernet/mellanox/mlx5/core/Kconfig | 2 + drivers/net/ethernet/mellanox/mlx5/core/en.h | 8 +- drivers/net/ethernet/mellanox/mlx5/core/en_dim.c | 12 +- .../net/ethernet/mellanox/mlx5/core/en_ethtool.c | 4 +- drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 22 +- drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c | 10 +- include/linux/dim.h | 271 +++++++++++++++ include/linux/net_dim.h | 371 +++------------------ include/linux/rdma_dim.h | 28 ++ include/rdma/ib_verbs.h | 27 +- lib/Kconfig | 8 + lib/Makefile | 1 + lib/dim/Makefile | 14 + lib/dim/dim.c | 83 +++++ lib/dim/net_dim.c | 191 +++++++++++ lib/dim/rdma_dim.c | 162 +++++++++ 28 files changed, 968 insertions(+), 396 deletions(-) create mode 100644 include/linux/dim.h create mode 100644 include/linux/rdma_dim.h create mode 100644 lib/dim/Makefile create mode 100644 lib/dim/dim.c create mode 100644 lib/dim/net_dim.c create mode 100644 lib/dim/rdma_dim.c