diff mbox series

[net-next,5/5] net/mlx5e: do not create xdp_redirect for non-uplink rep

Message ID 20241031125856.530927-6-tariqt@nvidia.com (mailing list archive)
State Accepted
Commit 355cf2749769ce7ada9afcaad8802f5ed37e88d5
Delegated to: Netdev Maintainers
Headers show
Series mlx5 misc patches 2024-10-31 | expand

Checks

Context Check Description
netdev/series_format success Posting correctly formatted
netdev/tree_selection success Clearly marked for net-next
netdev/ynl success Generated files up to date; no warnings/errors; no diff in generated;
netdev/fixes_present success Fixes tag not required for -next series
netdev/header_inline success No static functions without inline keyword in header files
netdev/build_32bit success Errors and warnings before: 5 this patch: 5
netdev/build_tools success No tools touched, skip
netdev/cc_maintainers warning 6 maintainers not CCed: daniel@iogearbox.net ast@kernel.org john.fastabend@gmail.com hawk@kernel.org bpf@vger.kernel.org linux-rdma@vger.kernel.org
netdev/build_clang success Errors and warnings before: 3 this patch: 3
netdev/verify_signedoff success Signed-off-by tag matches author and committer
netdev/deprecated_api success None detected
netdev/check_selftest success No net selftest shell script
netdev/verify_fixes success No Fixes tag
netdev/build_allmodconfig_warn success Errors and warnings before: 4 this patch: 4
netdev/checkpatch success total: 0 errors, 0 warnings, 0 checks, 41 lines checked
netdev/build_clang_rust success No Rust files in patch. Skipping build
netdev/kdoc success Errors and warnings before: 0 this patch: 0
netdev/source_inline success Was 0 now: 0
netdev/contest success net-next-2024-11-03--21-00 (tests: 781)

Commit Message

Tariq Toukan Oct. 31, 2024, 12:58 p.m. UTC 
From: William Tu <witu@nvidia.com>

XDP and XDP socket require extra SQ/RQ/CQs. Most of these resources
are dynamically created: no XDP program loaded, no resources are
created. One exception is the SQ/CQ created for XDP_REDRIECT, used
for other netdev to forward packet to mlx5 for transmit. The patch
disables creation of SQ and CQ used for egress XDP_REDIRECT, by
checking whether ndo_xdp_xmit is set or not.

For netdev without XDP support such as non-uplink representor, this
saves around 0.35MB of memory, per representor netdevice per channel.

Signed-off-by: William Tu <witu@nvidia.com>
Reviewed-by: Parav Pandit <parav@nvidia.com>
Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
---
 .../net/ethernet/mellanox/mlx5/core/en_main.c   | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

Comments

Daniel Machon Nov. 1, 2024, 1:22 p.m. UTC | #1 
Hi Tariq, William

> From: William Tu <witu@nvidia.com>
> 
> XDP and XDP socket require extra SQ/RQ/CQs. Most of these resources
> are dynamically created: no XDP program loaded, no resources are
> created. One exception is the SQ/CQ created for XDP_REDRIECT, used
> for other netdev to forward packet to mlx5 for transmit. The patch
> disables creation of SQ and CQ used for egress XDP_REDIRECT, by
> checking whether ndo_xdp_xmit is set or not.
> 
> For netdev without XDP support such as non-uplink representor, this
> saves around 0.35MB of memory, per representor netdevice per channel.
> 
> Signed-off-by: William Tu <witu@nvidia.com>
> Reviewed-by: Parav Pandit <parav@nvidia.com>
> Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
> ---
>  .../net/ethernet/mellanox/mlx5/core/en_main.c   | 17 +++++++++++------
>  1 file changed, 11 insertions(+), 6 deletions(-)
> 
> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> index 2f609b92d29b..59d7a0e28f24 100644
> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
> @@ -2514,6 +2514,7 @@ static int mlx5e_open_queues(struct mlx5e_channel *c,
>                              struct mlx5e_params *params,
>                              struct mlx5e_channel_param *cparam)
>  {
> +       const struct net_device_ops *netdev_ops = c->netdev->netdev_ops;
>         struct dim_cq_moder icocq_moder = {0, 0};
>         struct mlx5e_create_cq_param ccp;
>         int err;
> @@ -2534,10 +2535,12 @@ static int mlx5e_open_queues(struct mlx5e_channel *c,
>         if (err)
>                 goto err_close_icosq_cq;
> 
> -       c->xdpsq = mlx5e_open_xdpredirect_sq(c, params, cparam, &ccp);
> -       if (IS_ERR(c->xdpsq)) {
> -               err = PTR_ERR(c->xdpsq);
> -               goto err_close_tx_cqs;
> +       if (netdev_ops->ndo_xdp_xmit) {

Is it possible to have ndo_xdp_xmit() set, but *not* have an XDP prog attached
to the netdevice? I see that c->xdp = !!params->xdp_prog - could that be
used instead?

/Daniel

> +               c->xdpsq = mlx5e_open_xdpredirect_sq(c, params, cparam, &ccp);
> +               if (IS_ERR(c->xdpsq)) {
> +                       err = PTR_ERR(c->xdpsq);
> +                       goto err_close_tx_cqs;
> +               }
>         }
> 
>         err = mlx5e_open_cq(c->mdev, params->rx_cq_moderation, &cparam->rq.cqp, &ccp,
> @@ -2601,7 +2604,8 @@ static int mlx5e_open_queues(struct mlx5e_channel *c,
>         mlx5e_close_cq(&c->rq.cq);
> 
>  err_close_xdpredirect_sq:
> -       mlx5e_close_xdpredirect_sq(c->xdpsq);
> +       if (c->xdpsq)
> +               mlx5e_close_xdpredirect_sq(c->xdpsq);
> 
>  err_close_tx_cqs:
>         mlx5e_close_tx_cqs(c);
> @@ -2629,7 +2633,8 @@ static void mlx5e_close_queues(struct mlx5e_channel *c)
>         if (c->xdp)
>                 mlx5e_close_cq(&c->rq_xdpsq.cq);
>         mlx5e_close_cq(&c->rq.cq);
> -       mlx5e_close_xdpredirect_sq(c->xdpsq);
> +       if (c->xdpsq)
> +               mlx5e_close_xdpredirect_sq(c->xdpsq);
>         mlx5e_close_tx_cqs(c);
>         mlx5e_close_cq(&c->icosq.cq);
>         mlx5e_close_cq(&c->async_icosq.cq);
> --
> 2.44.0
> 
>
Tariq Toukan Nov. 3, 2024, 7:33 a.m. UTC | #2 
On 01/11/2024 15:22, Daniel Machon wrote:
> Hi Tariq, William
> 
>> From: William Tu <witu@nvidia.com>
>>
>> XDP and XDP socket require extra SQ/RQ/CQs. Most of these resources
>> are dynamically created: no XDP program loaded, no resources are
>> created. One exception is the SQ/CQ created for XDP_REDRIECT, used
>> for other netdev to forward packet to mlx5 for transmit. The patch
>> disables creation of SQ and CQ used for egress XDP_REDIRECT, by
>> checking whether ndo_xdp_xmit is set or not.
>>
>> For netdev without XDP support such as non-uplink representor, this
>> saves around 0.35MB of memory, per representor netdevice per channel.
>>
>> Signed-off-by: William Tu <witu@nvidia.com>
>> Reviewed-by: Parav Pandit <parav@nvidia.com>
>> Signed-off-by: Tariq Toukan <tariqt@nvidia.com>
>> ---
>>   .../net/ethernet/mellanox/mlx5/core/en_main.c   | 17 +++++++++++------
>>   1 file changed, 11 insertions(+), 6 deletions(-)
>>
>> diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> index 2f609b92d29b..59d7a0e28f24 100644
>> --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
>> @@ -2514,6 +2514,7 @@ static int mlx5e_open_queues(struct mlx5e_channel *c,
>>                               struct mlx5e_params *params,
>>                               struct mlx5e_channel_param *cparam)
>>   {
>> +       const struct net_device_ops *netdev_ops = c->netdev->netdev_ops;
>>          struct dim_cq_moder icocq_moder = {0, 0};
>>          struct mlx5e_create_cq_param ccp;
>>          int err;
>> @@ -2534,10 +2535,12 @@ static int mlx5e_open_queues(struct mlx5e_channel *c,
>>          if (err)
>>                  goto err_close_icosq_cq;
>>
>> -       c->xdpsq = mlx5e_open_xdpredirect_sq(c, params, cparam, &ccp);
>> -       if (IS_ERR(c->xdpsq)) {
>> -               err = PTR_ERR(c->xdpsq);
>> -               goto err_close_tx_cqs;
>> +       if (netdev_ops->ndo_xdp_xmit) {
> 
> Is it possible to have ndo_xdp_xmit() set, but *not* have an XDP prog attached
> to the netdevice? I see that c->xdp = !!params->xdp_prog - could that be
> used instead?
> 
> /Daniel
> 

Hi Daniel,

Unlike drivers of other vendors, in mlx5e we have the 
egress-xdp-redirect feature enabled without the need to load a dummy xdp 
program in rx.
diff mbox series

Patch

diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
index 2f609b92d29b..59d7a0e28f24 100644
--- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
+++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c
@@ -2514,6 +2514,7 @@  static int mlx5e_open_queues(struct mlx5e_channel *c,
 			     struct mlx5e_params *params,
 			     struct mlx5e_channel_param *cparam)
 {
+	const struct net_device_ops *netdev_ops = c->netdev->netdev_ops;
 	struct dim_cq_moder icocq_moder = {0, 0};
 	struct mlx5e_create_cq_param ccp;
 	int err;
@@ -2534,10 +2535,12 @@  static int mlx5e_open_queues(struct mlx5e_channel *c,
 	if (err)
 		goto err_close_icosq_cq;
 
-	c->xdpsq = mlx5e_open_xdpredirect_sq(c, params, cparam, &ccp);
-	if (IS_ERR(c->xdpsq)) {
-		err = PTR_ERR(c->xdpsq);
-		goto err_close_tx_cqs;
+	if (netdev_ops->ndo_xdp_xmit) {
+		c->xdpsq = mlx5e_open_xdpredirect_sq(c, params, cparam, &ccp);
+		if (IS_ERR(c->xdpsq)) {
+			err = PTR_ERR(c->xdpsq);
+			goto err_close_tx_cqs;
+		}
 	}
 
 	err = mlx5e_open_cq(c->mdev, params->rx_cq_moderation, &cparam->rq.cqp, &ccp,
@@ -2601,7 +2604,8 @@  static int mlx5e_open_queues(struct mlx5e_channel *c,
 	mlx5e_close_cq(&c->rq.cq);
 
 err_close_xdpredirect_sq:
-	mlx5e_close_xdpredirect_sq(c->xdpsq);
+	if (c->xdpsq)
+		mlx5e_close_xdpredirect_sq(c->xdpsq);
 
 err_close_tx_cqs:
 	mlx5e_close_tx_cqs(c);
@@ -2629,7 +2633,8 @@  static void mlx5e_close_queues(struct mlx5e_channel *c)
 	if (c->xdp)
 		mlx5e_close_cq(&c->rq_xdpsq.cq);
 	mlx5e_close_cq(&c->rq.cq);
-	mlx5e_close_xdpredirect_sq(c->xdpsq);
+	if (c->xdpsq)
+		mlx5e_close_xdpredirect_sq(c->xdpsq);
 	mlx5e_close_tx_cqs(c);
 	mlx5e_close_cq(&c->icosq.cq);
 	mlx5e_close_cq(&c->async_icosq.cq);