| Message ID | 20241021223551.508030-1-jdamato@fastly.com (mailing list archive) |
|---|---|
| State | Awaiting Upstream |
| Delegated to: | Netdev Maintainers |
| Headers | show |
| Series | [intel-next] ice: Add support for persistent NAPI config | expand |
On Mon, Oct 21, 2024 at 10:35:51PM +0000, Joe Damato wrote: > Use netif_napi_add_config to assign persistent per-NAPI config when > initializing NAPIs. This preserves NAPI config settings when queue > counts are adjusted. > > Tested with an E810-2CQDA2 NIC. ... > Signed-off-by: Joe Damato <jdamato@fastly.com> Reviewed-by: Simon Horman <horms@kernel.org>
> -----Original Message----- > From: Intel-wired-lan <intel-wired-lan-bounces@osuosl.org> On Behalf Of Joe Damato > Sent: 22 October 2024 04:06 > To: netdev@vger.kernel.org > Cc: Damato, Joe <jdamato@fastly.com>; Nguyen, Anthony L <anthony.l.nguyen@intel.com>; Kitszel, Przemyslaw <przemyslaw.kitszel@intel.com>; Andrew Lunn <andrew+netdev@lunn.ch>; David S. Miller <davem@davemloft.net>; Eric Dumazet <edumazet@google.com>; Jakub Kicinski <kuba@kernel.org>; Paolo Abeni <pabeni@redhat.com>; moderated list:INTEL ETHERNET DRIVERS <intel-wired-lan@lists.osuosl.org>; open list <linux-kernel@vger.kernel.org> > Subject: [Intel-wired-lan] [PATCH intel-next] ice: Add support for persistent NAPI config > > Use netif_napi_add_config to assign persistent per-NAPI config when initializing NAPIs. This preserves NAPI config settings when queue counts are adjusted. > > Tested with an E810-2CQDA2 NIC. > > Begin by setting the queue count to 4: > > $ sudo ethtool -L eth4 combined 4 > > Check the queue settings: > > $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ > --dump napi-get --json='{"ifindex": 4}' > [{'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8452, > 'ifindex': 4, > 'irq': 2782}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8451, > 'ifindex': 4, > 'irq': 2781}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8450, > 'ifindex': 4, > 'irq': 2780}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8449, > 'ifindex': 4, > 'irq': 2779}] > > Now, set the queue with NAPI ID 8451 to have a gro-flush-timeout of > 1111: > > $ sudo ./tools/net/ynl/cli.py \ > --spec Documentation/netlink/specs/netdev.yaml \ > --do napi-set --json='{"id": 8451, "gro-flush-timeout": 1111}' > None > > Check that worked: > > $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ > --dump napi-get --json='{"ifindex": 4}' > [{'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8452, > 'ifindex': 4, > 'irq': 2782}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 1111, > 'id': 8451, > 'ifindex': 4, > 'irq': 2781}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8450, > 'ifindex': 4, > 'irq': 2780}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8449, > 'ifindex': 4, > 'irq': 2779}] > > Now reduce the queue count to 2, which would destroy the queue with NAPI ID 8451: > > $ sudo ethtool -L eth4 combined 2 > > Check the queue settings, noting that NAPI ID 8451 is gone: > > $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ > --dump napi-get --json='{"ifindex": 4}' > [{'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8450, > 'ifindex': 4, > 'irq': 2780}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8449, > 'ifindex': 4, > 'irq': 2779}] > > Now, increase the number of queues back to 4: > > $ sudo ethtool -L eth4 combined 4 > > Dump the settings, expecting to see the same NAPI IDs as above and for NAPI ID 8451 to have its gro-flush-timeout set to 1111: > > $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ > --dump napi-get --json='{"ifindex": 4}' > [{'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8452, > 'ifindex': 4, > 'irq': 2782}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 1111, > 'id': 8451, > 'ifindex': 4, > 'irq': 2781}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8450, > 'ifindex': 4, > 'irq': 2780}, > {'defer-hard-irqs': 0, > 'gro-flush-timeout': 0, > 'id': 8449, > 'ifindex': 4, > 'irq': 2779}] > > Signed-off-by: Joe Damato <jdamato@fastly.com> > --- > drivers/net/ethernet/intel/ice/ice_base.c | 3 ++- drivers/net/ethernet/intel/ice/ice_lib.c | 6 ++++-- > 2 files changed, 6 insertions(+), 3 deletions(-) > Tested-by: Pucha Himasekhar Reddy <himasekharx.reddy.pucha@intel.com> (A Contingent worker at Intel)
diff --git a/drivers/net/ethernet/intel/ice/ice_base.c b/drivers/net/ethernet/intel/ice/ice_base.c index 3a8e156d7d86..82a9cd4ec7ae 100644 --- a/drivers/net/ethernet/intel/ice/ice_base.c +++ b/drivers/net/ethernet/intel/ice/ice_base.c @@ -156,7 +156,8 @@ static int ice_vsi_alloc_q_vector(struct ice_vsi *vsi, u16 v_idx) * handler here (i.e. resume, reset/rebuild, etc.) */ if (vsi->netdev) - netif_napi_add(vsi->netdev, &q_vector->napi, ice_napi_poll); + netif_napi_add_config(vsi->netdev, &q_vector->napi, + ice_napi_poll, v_idx); out: /* tie q_vector and VSI together */ diff --git a/drivers/net/ethernet/intel/ice/ice_lib.c b/drivers/net/ethernet/intel/ice/ice_lib.c index d4e74f96a8ad..a7d45a8ce7ac 100644 --- a/drivers/net/ethernet/intel/ice/ice_lib.c +++ b/drivers/net/ethernet/intel/ice/ice_lib.c @@ -2777,8 +2777,10 @@ void ice_napi_add(struct ice_vsi *vsi) return; ice_for_each_q_vector(vsi, v_idx) - netif_napi_add(vsi->netdev, &vsi->q_vectors[v_idx]->napi, - ice_napi_poll); + netif_napi_add_config(vsi->netdev, + &vsi->q_vectors[v_idx]->napi, + ice_napi_poll, + v_idx); } /**
Use netif_napi_add_config to assign persistent per-NAPI config when initializing NAPIs. This preserves NAPI config settings when queue counts are adjusted. Tested with an E810-2CQDA2 NIC. Begin by setting the queue count to 4: $ sudo ethtool -L eth4 combined 4 Check the queue settings: $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ --dump napi-get --json='{"ifindex": 4}' [{'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8452, 'ifindex': 4, 'irq': 2782}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8451, 'ifindex': 4, 'irq': 2781}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8450, 'ifindex': 4, 'irq': 2780}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8449, 'ifindex': 4, 'irq': 2779}] Now, set the queue with NAPI ID 8451 to have a gro-flush-timeout of 1111: $ sudo ./tools/net/ynl/cli.py \ --spec Documentation/netlink/specs/netdev.yaml \ --do napi-set --json='{"id": 8451, "gro-flush-timeout": 1111}' None Check that worked: $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ --dump napi-get --json='{"ifindex": 4}' [{'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8452, 'ifindex': 4, 'irq': 2782}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 1111, 'id': 8451, 'ifindex': 4, 'irq': 2781}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8450, 'ifindex': 4, 'irq': 2780}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8449, 'ifindex': 4, 'irq': 2779}] Now reduce the queue count to 2, which would destroy the queue with NAPI ID 8451: $ sudo ethtool -L eth4 combined 2 Check the queue settings, noting that NAPI ID 8451 is gone: $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ --dump napi-get --json='{"ifindex": 4}' [{'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8450, 'ifindex': 4, 'irq': 2780}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8449, 'ifindex': 4, 'irq': 2779}] Now, increase the number of queues back to 4: $ sudo ethtool -L eth4 combined 4 Dump the settings, expecting to see the same NAPI IDs as above and for NAPI ID 8451 to have its gro-flush-timeout set to 1111: $ ./tools/net/ynl/cli.py --spec Documentation/netlink/specs/netdev.yaml \ --dump napi-get --json='{"ifindex": 4}' [{'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8452, 'ifindex': 4, 'irq': 2782}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 1111, 'id': 8451, 'ifindex': 4, 'irq': 2781}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8450, 'ifindex': 4, 'irq': 2780}, {'defer-hard-irqs': 0, 'gro-flush-timeout': 0, 'id': 8449, 'ifindex': 4, 'irq': 2779}] Signed-off-by: Joe Damato <jdamato@fastly.com> --- drivers/net/ethernet/intel/ice/ice_base.c | 3 ++- drivers/net/ethernet/intel/ice/ice_lib.c | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-) base-commit: 6f07cd8301706b661776074ddc97c991d107cc91