From patchwork Tue Mar 26 14:46:37 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604296 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 38EF413C698 for ; Tue, 26 Mar 2024 14:46:54 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464415; cv=none; b=QA2rTaQDW8f8Sdq0q+sBt0VTE9fHhmhCQSC+50fQpPkY6ulc54keCWbRnEYuaTx5UsAELRxL20ZB+1w3CqANNR1LBAfSfQIhZKDvG4WKKls9AS95UU3UyKtmff5RdZQIFsajTQjo0vPg3CXP/+nf16xuQtH0b7gYBw0eMXg5CzY= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464415; c=relaxed/simple; bh=fY4th8eIAv/dBU8j4SI3m7hueZqAPUNG3ZqRyIbZKcg=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=oYI1rAl93SokuK0wBBQDvbIINQetADLGVX4U+DGfJDMqXwqEoLBhIIUrHo/QgkQIWb4XjrR33+XAgy/4QBSomZZc74CzHIVcvqM33+7p/CFrpXlMyUWB1Qq1Trnq0xKYLZMladElgl2CXB2Sgh4+VOaf3z51OV77/7S054ANUy8= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=sCN39AM2; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="sCN39AM2" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 6FA4DC43394; Tue, 26 Mar 2024 14:46:54 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464414; bh=fY4th8eIAv/dBU8j4SI3m7hueZqAPUNG3ZqRyIbZKcg=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=sCN39AM2dGIyI1pkpZ2jZZcovtOSo/vyYygXNHytbGahbsGciEQd/ROYuDRpQ7CB4 ygcgvS0nymuMvp4wJg3xXi8NxHJtWV/uULRRlrdPvuemHoqImUtFTJgAHfyee3B7RP GPXgudkwpK+xn19QEoExbMVxGIKDlOUNkI9nGPrGlaOFt7CEJIIbQSVvs4R5SpWom9 AbUVQWwEzQ6eIgdktjoRt01BBvwTiAV01UrfgsrdVIMamcT5PBeCPGXG8kihRZ7Zeu 47eBKomygeK9E+CvfAyckBImuQGevi8r7pAcpBioDGCTbaF88rLmO/ryyrecwY20M3 biWJOlzDWWbeg== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Shay Drory , Moshe Shemesh Subject: [net 01/10] net/mlx5: E-switch, store eswitch pointer before registering devlink_param Date: Tue, 26 Mar 2024 07:46:37 -0700 Message-ID: <20240326144646.2078893-2-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Shay Drory Next patch will move devlink register to be first. Therefore, whenever mlx5 will register a param, the user will be notified. In order to notify the user, devlink is using the get() callback of the param. Hence, resources that are being used by the get() callback must be set before the devlink param is registered. Therefore, store eswitch pointer inside mdev before registering the param. Signed-off-by: Shay Drory Reviewed-by: Moshe Shemesh Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/eswitch.c | 9 +++------ .../net/ethernet/mellanox/mlx5/core/eswitch_offloads.c | 4 ++++ 2 files changed, 7 insertions(+), 6 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 3047d7015c52..1789800faaeb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -1868,6 +1868,7 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev) if (err) goto abort; + dev->priv.eswitch = esw; err = esw_offloads_init(esw); if (err) goto reps_err; @@ -1892,11 +1893,6 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev) esw->offloads.encap = DEVLINK_ESWITCH_ENCAP_MODE_BASIC; else esw->offloads.encap = DEVLINK_ESWITCH_ENCAP_MODE_NONE; - if (MLX5_ESWITCH_MANAGER(dev) && - mlx5_esw_vport_match_metadata_supported(esw)) - esw->flags |= MLX5_ESWITCH_VPORT_MATCH_METADATA; - - dev->priv.eswitch = esw; BLOCKING_INIT_NOTIFIER_HEAD(&esw->n_head); esw_info(dev, @@ -1908,6 +1904,7 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev) reps_err: mlx5_esw_vports_cleanup(esw); + dev->priv.eswitch = NULL; abort: if (esw->work_queue) destroy_workqueue(esw->work_queue); @@ -1926,7 +1923,6 @@ void mlx5_eswitch_cleanup(struct mlx5_eswitch *esw) esw_info(esw->dev, "cleanup\n"); - esw->dev->priv.eswitch = NULL; destroy_workqueue(esw->work_queue); WARN_ON(refcount_read(&esw->qos.refcnt)); mutex_destroy(&esw->state_lock); @@ -1937,6 +1933,7 @@ void mlx5_eswitch_cleanup(struct mlx5_eswitch *esw) mutex_destroy(&esw->offloads.encap_tbl_lock); mutex_destroy(&esw->offloads.decap_tbl_lock); esw_offloads_cleanup(esw); + esw->dev->priv.eswitch = NULL; mlx5_esw_vports_cleanup(esw); debugfs_remove_recursive(esw->debugfs_root); devl_params_unregister(priv_to_devlink(esw->dev), mlx5_eswitch_params, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index baaae628b0a0..e3cce110e52f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -2476,6 +2476,10 @@ int esw_offloads_init(struct mlx5_eswitch *esw) if (err) return err; + if (MLX5_ESWITCH_MANAGER(esw->dev) && + mlx5_esw_vport_match_metadata_supported(esw)) + esw->flags |= MLX5_ESWITCH_VPORT_MATCH_METADATA; + err = devl_params_register(priv_to_devlink(esw->dev), esw_devlink_params, ARRAY_SIZE(esw_devlink_params)); From patchwork Tue Mar 26 14:46:38 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604297 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 547DB13C818 for ; Tue, 26 Mar 2024 14:46:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464416; cv=none; b=jdZhp6gnNFBXbBekdqPWJTnYHhJneUStyeuJ9i37Rm+kIWUhKFQJ1cYDmXYUBtCbNM4FNoEas7x6bieTCMQYqyPoGt+kqVj1ibVvslM5BlOgxzWK5X/6hySF2mDa00hursVJ8XdXnad1kpHgawWAt4Bo8kIKEefGIjvJ+uMS9/8= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464416; c=relaxed/simple; bh=hh/0H6cX+4dK1VYYrl1Rc4KG3n2vekJZLfZJSfQ0UmY=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=fYn6MgfGuBLTw1HIC+GBP45GcW3WT7B+MTUywMdFYT71wHlbTvjp6hzEVBTlFAEB/GdSlDdKNA35U/9upQWSf/S8BVFeLukElS4E1FgXRF6Nkz87s0aVu7kXq8CFda83Qh7JDUb9ci2IFyfUWB6/Ca7b4DBoObZurDEmAfRXwWY= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=rTiKCXIN; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="rTiKCXIN" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0DD66C433F1; Tue, 26 Mar 2024 14:46:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464416; bh=hh/0H6cX+4dK1VYYrl1Rc4KG3n2vekJZLfZJSfQ0UmY=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=rTiKCXINf5jfIm9NGV0KsSTic6m7TbSzXCZ16oW6sRy7l/2BaDXpSDQGaQ4UbQt/8 fl0A7/85gF7mR4GS1Affuu1kfPAvF8m5LyuEvZD9IC0ve81yHbyM7y2fXLuu6Z2Emt 8Io59FHn0ErB7wjHz76VEruYyrrDkFyrDuXZhAxALRjsDB5qS6KXi+P6YAgwGFgxRo PpZGfW0ZrnHX41TuKlXUBSbtV7S24X6LTecH06pzPg8O9ocbFXl3DtGxwNl9eL+3LQ UPbqG/+/UXTux2LvzV2e1Ez5rYhTsfr2OqAtef6rVz11Ya8VIvoWz7wfyQk3n8Gpnu WP0MjKe+oT6+A== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Shay Drory , Moshe Shemesh Subject: [net 02/10] net/mlx5: Register devlink first under devlink lock Date: Tue, 26 Mar 2024 07:46:38 -0700 Message-ID: <20240326144646.2078893-3-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Shay Drory In case device is having a non fatal FW error during probe, the driver will report the error to user via devlink. This will trigger a WARN_ON, since mlx5 is calling devlink_register() last. In order to avoid the WARN_ON[1], change mlx5 to invoke devl_register() first under devlink lock. [1] WARNING: CPU: 5 PID: 227 at net/devlink/health.c:483 devlink_recover_notify.constprop.0+0xb8/0xc0 CPU: 5 PID: 227 Comm: kworker/u16:3 Not tainted 6.4.0-rc5_for_upstream_min_debug_2023_06_12_12_38 #1 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 Workqueue: mlx5_health0000:08:00.0 mlx5_fw_reporter_err_work [mlx5_core] RIP: 0010:devlink_recover_notify.constprop.0+0xb8/0xc0 Call Trace: ? __warn+0x79/0x120 ? devlink_recover_notify.constprop.0+0xb8/0xc0 ? report_bug+0x17c/0x190 ? handle_bug+0x3c/0x60 ? exc_invalid_op+0x14/0x70 ? asm_exc_invalid_op+0x16/0x20 ? devlink_recover_notify.constprop.0+0xb8/0xc0 devlink_health_report+0x4a/0x1c0 mlx5_fw_reporter_err_work+0xa4/0xd0 [mlx5_core] process_one_work+0x1bb/0x3c0 ? process_one_work+0x3c0/0x3c0 worker_thread+0x4d/0x3c0 ? process_one_work+0x3c0/0x3c0 kthread+0xc6/0xf0 ? kthread_complete_and_exit+0x20/0x20 ret_from_fork+0x1f/0x30 Fixes: cf530217408e ("devlink: Notify users when objects are accessible") Signed-off-by: Shay Drory Reviewed-by: Moshe Shemesh Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/main.c | 37 ++++++++++--------- .../mellanox/mlx5/core/sf/dev/driver.c | 1 - 2 files changed, 20 insertions(+), 18 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index c2593625c09a..59806553889e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -1480,6 +1480,14 @@ int mlx5_init_one_devl_locked(struct mlx5_core_dev *dev) if (err) goto err_register; + err = mlx5_crdump_enable(dev); + if (err) + mlx5_core_err(dev, "mlx5_crdump_enable failed with error code %d\n", err); + + err = mlx5_hwmon_dev_register(dev); + if (err) + mlx5_core_err(dev, "mlx5_hwmon_dev_register failed with error code %d\n", err); + mutex_unlock(&dev->intf_state_mutex); return 0; @@ -1505,7 +1513,10 @@ int mlx5_init_one(struct mlx5_core_dev *dev) int err; devl_lock(devlink); + devl_register(devlink); err = mlx5_init_one_devl_locked(dev); + if (err) + devl_unregister(devlink); devl_unlock(devlink); return err; } @@ -1517,6 +1528,8 @@ void mlx5_uninit_one(struct mlx5_core_dev *dev) devl_lock(devlink); mutex_lock(&dev->intf_state_mutex); + mlx5_hwmon_dev_unregister(dev); + mlx5_crdump_disable(dev); mlx5_unregister_device(dev); if (!test_bit(MLX5_INTERFACE_STATE_UP, &dev->intf_state)) { @@ -1534,6 +1547,7 @@ void mlx5_uninit_one(struct mlx5_core_dev *dev) mlx5_function_teardown(dev, true); out: mutex_unlock(&dev->intf_state_mutex); + devl_unregister(devlink); devl_unlock(devlink); } @@ -1680,16 +1694,20 @@ int mlx5_init_one_light(struct mlx5_core_dev *dev) } devl_lock(devlink); + devl_register(devlink); + err = mlx5_devlink_params_register(priv_to_devlink(dev)); - devl_unlock(devlink); if (err) { mlx5_core_warn(dev, "mlx5_devlink_param_reg err = %d\n", err); goto query_hca_caps_err; } + devl_unlock(devlink); return 0; query_hca_caps_err: + devl_unregister(devlink); + devl_unlock(devlink); mlx5_function_disable(dev, true); out: dev->state = MLX5_DEVICE_STATE_INTERNAL_ERROR; @@ -1702,6 +1720,7 @@ void mlx5_uninit_one_light(struct mlx5_core_dev *dev) devl_lock(devlink); mlx5_devlink_params_unregister(priv_to_devlink(dev)); + devl_unregister(devlink); devl_unlock(devlink); if (dev->state != MLX5_DEVICE_STATE_UP) return; @@ -1943,16 +1962,7 @@ static int probe_one(struct pci_dev *pdev, const struct pci_device_id *id) goto err_init_one; } - err = mlx5_crdump_enable(dev); - if (err) - dev_err(&pdev->dev, "mlx5_crdump_enable failed with error code %d\n", err); - - err = mlx5_hwmon_dev_register(dev); - if (err) - mlx5_core_err(dev, "mlx5_hwmon_dev_register failed with error code %d\n", err); - pci_save_state(pdev); - devlink_register(devlink); return 0; err_init_one: @@ -1973,16 +1983,9 @@ static void remove_one(struct pci_dev *pdev) struct devlink *devlink = priv_to_devlink(dev); set_bit(MLX5_BREAK_FW_WAIT, &dev->intf_state); - /* mlx5_drain_fw_reset() and mlx5_drain_health_wq() are using - * devlink notify APIs. - * Hence, we must drain them before unregistering the devlink. - */ mlx5_drain_fw_reset(dev); mlx5_drain_health_wq(dev); - devlink_unregister(devlink); mlx5_sriov_disable(pdev, false); - mlx5_hwmon_dev_unregister(dev); - mlx5_crdump_disable(dev); mlx5_uninit_one(dev); mlx5_pci_close(dev); mlx5_mdev_uninit(dev); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c index bc863e1f062e..e3bf8c7e4baa 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c @@ -101,7 +101,6 @@ static void mlx5_sf_dev_remove(struct auxiliary_device *adev) devlink = priv_to_devlink(mdev); set_bit(MLX5_BREAK_FW_WAIT, &mdev->intf_state); mlx5_drain_health_wq(mdev); - devlink_unregister(devlink); if (mlx5_dev_is_lightweight(mdev)) mlx5_uninit_one_light(mdev); else From patchwork Tue Mar 26 14:46:39 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604298 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 2A05613C808 for ; Tue, 26 Mar 2024 14:46:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464418; cv=none; b=t7Bo4/P9nf0BYTvlNfzdRH3qbwAkt5gmFxAiBYfxOgqvovHcfbeMYnloVbQp88SIsQOKbhCLMU+B3GBnuuX4dNCFTzySiOBYw/UwklToqCGZ26QrgzLaIj2Flgec3fevhqkH6Pb/neA4w2AW+RmJcS9dVEtFd47GN1BACygNpDU= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464418; c=relaxed/simple; bh=eHfJye8/AU2XdbG2KjGZM/VlHjmoaSA4lCel2jPGFdw=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dFETfVpWAwA3DBuJ8bTlu7tLyHadRo7e0dtSwnBYky4mNjdiqf7k6GR0DMncOde4xCH6dAdTpJMLSUNmXY7hZdsQ+QThsDuibHTQHCiyVa/s27Ek4PytsaNHO6RKfG4+J13Pyzqm7Ai9aNuKk5gHp2lzR2kP8lGle/qcL4MK/Bk= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=j4o74JKi; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="j4o74JKi" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 94EFFC433B2; Tue, 26 Mar 2024 14:46:57 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464417; bh=eHfJye8/AU2XdbG2KjGZM/VlHjmoaSA4lCel2jPGFdw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=j4o74JKikJxd54nYmDQo3rvOhjCXbY6vtVyhZjf5nWL9j98Tbp3UgCqbMzbmT/muj dZ+nLSeb408xC5A1+IE+ge6rjyxXzq/RvjkwZ7XN8jE95f635M943ZQUsPciC9gDJq D5zHQcFUQjJ4wdPm1/W8fNQHoNQetAqDCmw4uErIPkHqyREhvcnMJmAq52nnppuPyJ 8tRfVN4vL/GCySt/Ksdki4FsWHoJirLh1VbEvHHPnAYt50C8tYfC4wvu9C6BFi2C+1 ixJRSG4pisTBhRp4eiPUf0vOmCplbO6pfUFSjrfI/MQIZzRKhx8iPcLfcYgwl7y3eG VtxbQP4nMpxqA== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Michael Liang , Mohamed Khalfella , Yuanyuan Zhong , Shay Drory Subject: [net 03/10] net/mlx5: offset comp irq index in name by one Date: Tue, 26 Mar 2024 07:46:39 -0700 Message-ID: <20240326144646.2078893-4-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Michael Liang The mlx5 comp irq name scheme is changed a little bit between commit 3663ad34bc70 ("net/mlx5: Shift control IRQ to the last index") and commit 3354822cde5a ("net/mlx5: Use dynamic msix vectors allocation"). The index in the comp irq name used to start from 0 but now it starts from 1. There is nothing critical here, but it's harmless to change back to the old behavior, a.k.a starting from 0. Fixes: 3354822cde5a ("net/mlx5: Use dynamic msix vectors allocation") Reviewed-by: Mohamed Khalfella Reviewed-by: Yuanyuan Zhong Signed-off-by: Michael Liang Reviewed-by: Shay Drory Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c index 4dcf995cb1a2..6bac8ad70ba6 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c @@ -19,6 +19,7 @@ #define MLX5_IRQ_CTRL_SF_MAX 8 /* min num of vectors for SFs to be enabled */ #define MLX5_IRQ_VEC_COMP_BASE_SF 2 +#define MLX5_IRQ_VEC_COMP_BASE 1 #define MLX5_EQ_SHARE_IRQ_MAX_COMP (8) #define MLX5_EQ_SHARE_IRQ_MAX_CTRL (UINT_MAX) @@ -246,6 +247,7 @@ static void irq_set_name(struct mlx5_irq_pool *pool, char *name, int vecidx) return; } + vecidx -= MLX5_IRQ_VEC_COMP_BASE; snprintf(name, MLX5_MAX_IRQ_NAME, "mlx5_comp%d", vecidx); } @@ -585,7 +587,7 @@ struct mlx5_irq *mlx5_irq_request_vector(struct mlx5_core_dev *dev, u16 cpu, struct mlx5_irq_table *table = mlx5_irq_table_get(dev); struct mlx5_irq_pool *pool = table->pcif_pool; struct irq_affinity_desc af_desc; - int offset = 1; + int offset = MLX5_IRQ_VEC_COMP_BASE; if (!pool->xa_num_irqs.max) offset = 0; From patchwork Tue Mar 26 14:46:40 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604299 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id ADA6513C3EB for ; Tue, 26 Mar 2024 14:46:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464419; cv=none; b=LazMzGY0jbMFpWKFvkBjdROGCxBnvM8gW0PO1qJzF8UXajwF3oGKhYACpMVqRwyg+HTOSXSduJ1ZQPuClasK8MDqSshzBQP0VvsS7u4osydr/kopyYWwoS52k8Qc7tPc0w3x5aZiWoCeThj9kOl0UlqZaLeS/BiSs0slaxrwepI= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464419; c=relaxed/simple; bh=lmaJt2nwjSsfM3LN46WStqi6lUZAUyBVUkbOtZppF+w=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=qf6YD4HCFNCYFwjJq57KE6i1v5ikDQFi+yACokfG7vIVImMBWJYSR5vYp3O07T7b6/G4WxgocKTzngdtBlJRV0U4kOZkzwzXu+omPaD6wq9oTTcBkTZ2RDAwTCmhua4uUfMhXZZCACSblrVR05zEizgO4nENZVsc4d3GTkaJZ6M= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=CqLPRi9+; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="CqLPRi9+" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1D867C43394; Tue, 26 Mar 2024 14:46:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464419; bh=lmaJt2nwjSsfM3LN46WStqi6lUZAUyBVUkbOtZppF+w=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=CqLPRi9+/mFhu/E4B1qJwWxn78b6cXsdJkPDfjWLEMlOUo8hpK6hufp2gpkG3Ls1o LhwR+u+8waYzTSRqK9UkyBCiHSd7guNw8Pbr+YtrTrx9c8NJ7DuVNu1vIs8kjMhwY9 M2gNnLc/uNJ6JdNYaW/ALMYB8zH0W785u7AASUM9nFY4oqo/eEFfXuInMr4CX3BfDb m9wTmlY0vuREyVdyci8tvTBKnoMdQksmGPSxpJCyfomjTPoq0Iu9TTLSZ2fCBl7vXs X4h07X0Cb1meLvieL31esbkdzwSg/Ph6woiL0kCpGvSE061qHUv+WN1qsIsOs12J/v cArVTQMQsjkKg== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Cosmin Ratiu , Mark Bloch Subject: [net 04/10] net/mlx5: Properly link new fs rules into the tree Date: Tue, 26 Mar 2024 07:46:40 -0700 Message-ID: <20240326144646.2078893-5-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Cosmin Ratiu Previously, add_rule_fg would only add newly created rules from the handle into the tree when they had a refcount of 1. On the other hand, create_flow_handle tries hard to find and reference already existing identical rules instead of creating new ones. These two behaviors can result in a situation where create_flow_handle 1) creates a new rule and references it, then 2) in a subsequent step during the same handle creation references it again, resulting in a rule with a refcount of 2 that is not linked into the tree, will have a NULL parent and root and will result in a crash when the flow group is deleted because del_sw_hw_rule, invoked on rule deletion, assumes node->parent is != NULL. This happened in the wild, due to another bug related to incorrect handling of duplicate pkt_reformat ids, which lead to the code in create_flow_handle incorrectly referencing a just-added rule in the same flow handle, resulting in the problem described above. Full details are at [1]. This patch changes add_rule_fg to add new rules without parents into the tree, properly initializing them and avoiding the crash. This makes it more consistent with how rules are added to an FTE in create_flow_handle. Fixes: 74491de93712 ("net/mlx5: Add multi dest support") Link: https://lore.kernel.org/netdev/ea5264d6-6b55-4449-a602-214c6f509c1e@163.com/T/#u [1] Signed-off-by: Cosmin Ratiu Reviewed-by: Tariq Toukan Reviewed-by: Mark Bloch Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/fs_core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c index e6bfa7e4f146..2a9421342a50 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c @@ -1808,8 +1808,9 @@ static struct mlx5_flow_handle *add_rule_fg(struct mlx5_flow_group *fg, } trace_mlx5_fs_set_fte(fte, false); + /* Link newly added rules into the tree. */ for (i = 0; i < handle->num_rules; i++) { - if (refcount_read(&handle->rule[i]->node.refcount) == 1) { + if (!handle->rule[i]->node.parent) { tree_add_node(&handle->rule[i]->node, &fte->node); trace_mlx5_fs_add_rule(handle->rule[i]); } From patchwork Tue Mar 26 14:46:41 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604300 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D89E613C83C for ; Tue, 26 Mar 2024 14:47:00 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464420; cv=none; b=PgrS1XgHwlgPe6KHESZs1RukVeaKWmaYJacnAHLwfxG/zTd5a0mJ49yY+D/Fo0T+1EpSO1Xu5RhdgpHWwZljuDvP/6kpiDMOeOOOzetF7o5Kl+3G2EiixqF1BwbhR/y3fm/YbpkMOLLoazYQPjHHwpGjleh2o4w2kZB4IoyuDY0= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464420; c=relaxed/simple; bh=ZXZFe9RdKJFei25adHKLISho6XjQaaSBDcfXhrq1uEc=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=cJLavgkBUnEKJSkxzmY44JjmcrPh1zJEAFt+vhKi6HzyeWpr/QtWrqREkvOU1J/+QIftBd3t8QUvoMEj+fFc0Qn3Sr3x9WtF/ACQ/NIrbDedvXkgUuGbw77O2+EfzrNwqasN4pOYYut8hY0ASZkoljRjdG8s+84vgAve2PwNbCc= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=q/7cv9qU; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="q/7cv9qU" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A5851C43399; Tue, 26 Mar 2024 14:47:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464420; bh=ZXZFe9RdKJFei25adHKLISho6XjQaaSBDcfXhrq1uEc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=q/7cv9qUoVx2PkyGQqBxghWokOzDSjsVu3s8bPaaCO0RQQO07PyoQk4wsSzssU5dU oLEHA80Wf7LH5xm0i9+1Jus3Kq+gobisgnH2o8S9G5tx086+6KRNSlN2Czwd0D3RFM L7SuwxT6r/MIkVOGGm6zkB2d6z4YTwhp9xGMLYTGEj0XRiHbXXfmjDcPW34yGT1+Sg tyTjOKHZw9IvCTjwSHObZPYHu3v/Jw+b3rJcbXScPwOvyxicuETr2TBhjV2fiLnCxw zvfeukppVHgJF7FFvz6SGEv/zcOxFu+jFfGDIkjqGVKIKP9KNu9wHb9O5Zsiy6VZBh xOtg7SjZZGDPA== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Cosmin Ratiu , Mark Bloch Subject: [net 05/10] net/mlx5: Correctly compare pkt reformat ids Date: Tue, 26 Mar 2024 07:46:41 -0700 Message-ID: <20240326144646.2078893-6-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Cosmin Ratiu struct mlx5_pkt_reformat contains a naked union of a u32 id and a dr_action pointer which is used when the action is SW-managed (when pkt_reformat.owner is set to MLX5_FLOW_RESOURCE_OWNER_SW). Using id directly in that case is incorrect, as it maps to the least significant 32 bits of the 64-bit pointer in mlx5_fs_dr_action and not to the pkt reformat id allocated in firmware. For the purpose of comparing whether two rules are identical, interpreting the least significant 32 bits of the mlx5_fs_dr_action pointer as an id mostly works... until it breaks horribly and produces the outcome described in [1]. This patch fixes mlx5_flow_dests_cmp to correctly compare ids using mlx5_fs_dr_action_get_pkt_reformat_id for the SW-managed rules. Link: https://lore.kernel.org/netdev/ea5264d6-6b55-4449-a602-214c6f509c1e@163.com/T/#u [1] Fixes: 6a48faeeca10 ("net/mlx5: Add direct rule fs_cmd implementation") Signed-off-by: Cosmin Ratiu Reviewed-by: Mark Bloch Reviewed-by: Tariq Toukan Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/fs_core.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c index 2a9421342a50..cf085a478e3e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fs_core.c @@ -1664,6 +1664,16 @@ static int create_auto_flow_group(struct mlx5_flow_table *ft, return err; } +static bool mlx5_pkt_reformat_cmp(struct mlx5_pkt_reformat *p1, + struct mlx5_pkt_reformat *p2) +{ + return p1->owner == p2->owner && + (p1->owner == MLX5_FLOW_RESOURCE_OWNER_FW ? + p1->id == p2->id : + mlx5_fs_dr_action_get_pkt_reformat_id(p1) == + mlx5_fs_dr_action_get_pkt_reformat_id(p2)); +} + static bool mlx5_flow_dests_cmp(struct mlx5_flow_destination *d1, struct mlx5_flow_destination *d2) { @@ -1675,8 +1685,8 @@ static bool mlx5_flow_dests_cmp(struct mlx5_flow_destination *d1, ((d1->vport.flags & MLX5_FLOW_DEST_VPORT_VHCA_ID) ? (d1->vport.vhca_id == d2->vport.vhca_id) : true) && ((d1->vport.flags & MLX5_FLOW_DEST_VPORT_REFORMAT_ID) ? - (d1->vport.pkt_reformat->id == - d2->vport.pkt_reformat->id) : true)) || + mlx5_pkt_reformat_cmp(d1->vport.pkt_reformat, + d2->vport.pkt_reformat) : true)) || (d1->type == MLX5_FLOW_DESTINATION_TYPE_FLOW_TABLE && d1->ft == d2->ft) || (d1->type == MLX5_FLOW_DESTINATION_TYPE_TIR && From patchwork Tue Mar 26 14:46:42 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604301 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B5BA313C80D for ; Tue, 26 Mar 2024 14:47:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464422; cv=none; b=h016myHZA/gVYS1uUkvv/xRm3MtnGsMGCAUa7zn0y6wW/owb1lF2HIfpAbN5EVsIaphVvbHY8pO6Gw3/9M34jSwJRFe2NfpojYl0TRVEJLNFvyz43IvgriZZfaMaXFYZExcNMglkVbWnuErrPrpHKkaWaTXZ4jhw58+Uyrx+7Pw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464422; c=relaxed/simple; bh=ercE3vCJHsDyP0t7JhZIg9P42vJZI0+O+T+p666bpic=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=dxSNlWFx68OsnUvNwp6ykAckRWxC3EstCT3IKsizw7FjNkOQZsfY67IIFHcBFQr3+d6bro+pJb0yphCEsLde8x5mOAP7qh8zOMbK0ymg4lmzid43CspTVY2WZ9P4hLbBNmwqcNcowFLHcUOn4QM0IavqNPvxFdMDJFhIIwWbDiw= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=B0xYgd05; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="B0xYgd05" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 1F9F6C433F1; Tue, 26 Mar 2024 14:47:02 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464422; bh=ercE3vCJHsDyP0t7JhZIg9P42vJZI0+O+T+p666bpic=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=B0xYgd05I1llFXLY6FwWWKmyX1faIVMZpauOTl+27CImtX14FPI19pAz8xBmYo8wn T5CpiTGYCLwOYnFx+PRBprfPcgZI7dDIaLPWrNqRajICamWkMsHyIF1bLm4JpzIFdU jdXxX5PaCk+0TKi+m6Ekpy9LfWn0M0JLjI/mWN9GSUN4RIJ6K6M2p5K06XFEW4Lcza zFFypDKrIb4svGQicPSrfMFT1Whg8ep9GXbb8qlixtHNgo0W5PEwnGmEhngUjQCCJr sLotnKsF3+o+cX6NxA36hXWwr/YVOPk5cZ37S4t4B4HYqQ5zDPJhcXNJJzlimzqMQf JhBrzYsymIyKg== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Carolina Jubran Subject: [net 06/10] net/mlx5: RSS, Block changing channels number when RXFH is configured Date: Tue, 26 Mar 2024 07:46:42 -0700 Message-ID: <20240326144646.2078893-7-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Carolina Jubran Changing the channels number after configuring the receive flow hash indirection table may alter the RSS table size. The previous configuration may no longer be compatible with the new receive flow hash indirection table. Block changing the channels number when RXFH is configured. Fixes: 74a8dadac17e ("net/mlx5e: Preparations for supporting larger number of channels") Signed-off-by: Carolina Jubran Reviewed-by: Tariq Toukan Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c index cc51ce16df14..c5a203b5119d 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c @@ -451,6 +451,17 @@ int mlx5e_ethtool_set_channels(struct mlx5e_priv *priv, mutex_lock(&priv->state_lock); + /* Don't allow changing the number of channels if RXFH was previously configured. + * Changing the channels number after configuring the RXFH may alter the RSS table size, + * the previous configuration may no longer be compatible with the new RSS table. + */ + if (netif_is_rxfh_configured(priv->netdev)) { + err = -EINVAL; + netdev_err(priv->netdev, "%s: RXFH is configured, cannot change the number of channels\n", + __func__); + goto out; + } + /* Don't allow changing the number of channels if HTB offload is active, * because the numeration of the QoS SQs will change, while per-queue * qdiscs are attached. From patchwork Tue Mar 26 14:46:43 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604302 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 4482813C666 for ; Tue, 26 Mar 2024 14:47:03 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464424; cv=none; b=BB21N5SDXtBc4rBoKEQ80O9AdWhqsQz5Agr+9Otbx5jbDGkqyFkYY76YSVNY9jW3V5Bxc4IaUYPrsE74LHkuS101E4+xpJXl1wrJz6Cze13WA9eIolAx8QPa9kWLxcZ0PUDFAw5CSZwzFLBtk+ckzz5LfsgIFa7h3TvmDfmjNQw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464424; c=relaxed/simple; bh=oblu9EvmQTQpj1KFzD7wrNRmOXsl3wbhZkrpxMtHhDM=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=iHRlydmjP8es2YjVhATQHMgZHZKsACt/Uye8OqAHizipsHlQ1acVznh4wdW5e+9BlsR26e0vIwZ1kTFIRMz18tNP7ll6JHotanJzym1v5E6KILaQ0/RhMBH+Xztw3xl8T62JfXmHkPvG5RECqGvMjAifL5KYxFxaq/Bm5DtWl3U= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=ZICLlkJ8; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="ZICLlkJ8" Received: by smtp.kernel.org (Postfix) with ESMTPSA id A9A6CC433F1; Tue, 26 Mar 2024 14:47:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464423; bh=oblu9EvmQTQpj1KFzD7wrNRmOXsl3wbhZkrpxMtHhDM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=ZICLlkJ8P0VGyESitSCOGlLFMVaYGjBJcaI6/yBf3wRuxDkOBJXszH2wduJqC0J7B MMLuxdUwPfOcpk8DVGzmJsVak2VOXnCz1QL7dKH2VbHuzUPCFg6L4GJtiGpWEZbu0c LFjTNYeS1ouxP07I6XlbXJEqsTPMMdZZMAtAy3FGPdHT4LSwLtEjjZ2BKyouRtI34v HbuIqRG0eSCFRoFqChQLJ7f8iWTC4Zuf/b1ezo0Q4+08KAVlpLiOKBPJ/AG7z7h57r O1fJKpeNyBrLRDDNyauRdUqN7GU+T67L5PrruH3bt+Abno3xh+sApIWXvIFnLmj9LY EM1PNwkV02uOQ== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Carolina Jubran , Dragos Tatulea Subject: [net 07/10] net/mlx5e: Fix mlx5e_priv_init() cleanup flow Date: Tue, 26 Mar 2024 07:46:43 -0700 Message-ID: <20240326144646.2078893-8-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Carolina Jubran When mlx5e_priv_init() fails, the cleanup flow calls mlx5e_selq_cleanup which calls mlx5e_selq_apply() that assures that the `priv->state_lock` is held using lockdep_is_held(). Acquire the state_lock in mlx5e_selq_cleanup(). Kernel log: ============================= WARNING: suspicious RCU usage 6.8.0-rc3_net_next_841a9b5 #1 Not tainted ----------------------------- drivers/net/ethernet/mellanox/mlx5/core/en/selq.c:124 suspicious rcu_dereference_protected() usage! other info that might help us debug this: rcu_scheduler_active = 2, debug_locks = 1 2 locks held by systemd-modules/293: #0: ffffffffa05067b0 (devices_rwsem){++++}-{3:3}, at: ib_register_client+0x109/0x1b0 [ib_core] #1: ffff8881096c65c0 (&device->client_data_rwsem){++++}-{3:3}, at: add_client_context+0x104/0x1c0 [ib_core] stack backtrace: CPU: 4 PID: 293 Comm: systemd-modules Not tainted 6.8.0-rc3_net_next_841a9b5 #1 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-0-gf21b5a4aeb02-prebuilt.qemu.org 04/01/2014 Call Trace: dump_stack_lvl+0x8a/0xa0 lockdep_rcu_suspicious+0x154/0x1a0 mlx5e_selq_apply+0x94/0xa0 [mlx5_core] mlx5e_selq_cleanup+0x3a/0x60 [mlx5_core] mlx5e_priv_init+0x2be/0x2f0 [mlx5_core] mlx5_rdma_setup_rn+0x7c/0x1a0 [mlx5_core] rdma_init_netdev+0x4e/0x80 [ib_core] ? mlx5_rdma_netdev_free+0x70/0x70 [mlx5_core] ipoib_intf_init+0x64/0x550 [ib_ipoib] ipoib_intf_alloc+0x4e/0xc0 [ib_ipoib] ipoib_add_one+0xb0/0x360 [ib_ipoib] add_client_context+0x112/0x1c0 [ib_core] ib_register_client+0x166/0x1b0 [ib_core] ? 0xffffffffa0573000 ipoib_init_module+0xeb/0x1a0 [ib_ipoib] do_one_initcall+0x61/0x250 do_init_module+0x8a/0x270 init_module_from_file+0x8b/0xd0 idempotent_init_module+0x17d/0x230 __x64_sys_finit_module+0x61/0xb0 do_syscall_64+0x71/0x140 entry_SYSCALL_64_after_hwframe+0x46/0x4e Fixes: 8bf30be75069 ("net/mlx5e: Introduce select queue parameters") Signed-off-by: Carolina Jubran Reviewed-by: Tariq Toukan Reviewed-by: Dragos Tatulea Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/en/selq.c | 2 ++ drivers/net/ethernet/mellanox/mlx5/core/en_main.c | 2 -- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/selq.c b/drivers/net/ethernet/mellanox/mlx5/core/en/selq.c index f675b1926340..f66bbc846464 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/selq.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/selq.c @@ -57,6 +57,7 @@ int mlx5e_selq_init(struct mlx5e_selq *selq, struct mutex *state_lock) void mlx5e_selq_cleanup(struct mlx5e_selq *selq) { + mutex_lock(selq->state_lock); WARN_ON_ONCE(selq->is_prepared); kvfree(selq->standby); @@ -67,6 +68,7 @@ void mlx5e_selq_cleanup(struct mlx5e_selq *selq) kvfree(selq->standby); selq->standby = NULL; + mutex_unlock(selq->state_lock); } void mlx5e_selq_prepare_params(struct mlx5e_selq *selq, struct mlx5e_params *params) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 91848eae4565..b375ef268671 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -5726,9 +5726,7 @@ void mlx5e_priv_cleanup(struct mlx5e_priv *priv) kfree(priv->tx_rates); kfree(priv->txq2sq); destroy_workqueue(priv->wq); - mutex_lock(&priv->state_lock); mlx5e_selq_cleanup(&priv->selq); - mutex_unlock(&priv->state_lock); free_cpumask_var(priv->scratchpad.cpumask); for (i = 0; i < priv->htb_max_qos_sqs; i++) From patchwork Tue Mar 26 14:46:44 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604303 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id CA4EF13C666 for ; Tue, 26 Mar 2024 14:47:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464425; cv=none; b=nQLnzmbg190TJqcwPw232ACSwTYEeXjXTDB855jXYby/WhY5IdzHJzcg/TLZJuySIUXLnrSmk/dPcbhcG+cABFrpsx7qX3S6UJ8REoKG7rOvoFUBP/P8HLnK2w7ecs4H8WbmNwBCC/z06OL60trs/vX1NCo3lFz5CnVd6DD8Slw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464425; c=relaxed/simple; bh=WDSMVBdK4khOmOZqIo19jACc2yX49nOcGyxD1syaniE=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=QazQTPMyrK9br09Iqs7Fq8iKo1rvePbsrSWtW/7gFa9btW4NzO0+CIbV/5Efpd0K1fAj1ldG/MzGJ/aUhqZpfI/iXGeEX84E0zltgCVo0tdseSkcI29Tn6qKod3mhUgH6Ym/yNxqGev/aOp2sCbWtypBTaoCUiDzL7/T7XcngVM= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=IM36hn3y; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="IM36hn3y" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 41A69C433F1; Tue, 26 Mar 2024 14:47:05 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464425; bh=WDSMVBdK4khOmOZqIo19jACc2yX49nOcGyxD1syaniE=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IM36hn3y1YcE5i/KX+/ITawzmW1Z+vxdwQpNIjou9GJinm7WsZJPuv3OcI3LG8ZP5 M9GtnlIqcRq47ogDTP7TDeWFT+xP6nSSqtw6g7UfRz+HHrx18B/5QIdqrYmIP2OIDn mbMTdxyJJ1xZCEMpifrf1fjEPUMeS55jIzbPbTMFsJdG4cdNTzHdQTi0YAz4WvNt6E diEfM1v8QmKzHc3DMtLBLa1VYvEO4FPANbT1wsalzOcQm+Vs/JLFiBKzykhhUC9Kr5 5QmsBBt9wUeJ+bb7KfnrIxs8y/UVl8Ca862dHY1mVtI9bjNvOiuHQtDpO9XHputf3F uBnurKvc7VfXA== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Carolina Jubran , Dragos Tatulea Subject: [net 08/10] net/mlx5e: HTB, Fix inconsistencies with QoS SQs number Date: Tue, 26 Mar 2024 07:46:44 -0700 Message-ID: <20240326144646.2078893-9-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Carolina Jubran When creating a new HTB class while the interface is down, the variable that follows the number of QoS SQs (htb_max_qos_sqs) may not be consistent with the number of HTB classes. Previously, we compared these two values to ensure that the node_qid is lower than the number of QoS SQs, and we allocated stats for that SQ when they are equal. Change the check to compare the node_qid with the current number of leaf nodes and fix the checking conditions to ensure allocation of stats_list and stats for each node. Fixes: 214baf22870c ("net/mlx5e: Support HTB offload") Signed-off-by: Carolina Jubran Reviewed-by: Tariq Toukan Reviewed-by: Dragos Tatulea Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/en/qos.c | 33 ++++++++++--------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c b/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c index e87e26f2c669..6743806b8480 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/qos.c @@ -83,24 +83,25 @@ int mlx5e_open_qos_sq(struct mlx5e_priv *priv, struct mlx5e_channels *chs, txq_ix = mlx5e_qid_from_qos(chs, node_qid); - WARN_ON(node_qid > priv->htb_max_qos_sqs); - if (node_qid == priv->htb_max_qos_sqs) { - struct mlx5e_sq_stats *stats, **stats_list = NULL; - - if (priv->htb_max_qos_sqs == 0) { - stats_list = kvcalloc(mlx5e_qos_max_leaf_nodes(priv->mdev), - sizeof(*stats_list), - GFP_KERNEL); - if (!stats_list) - return -ENOMEM; - } + WARN_ON(node_qid >= mlx5e_htb_cur_leaf_nodes(priv->htb)); + if (!priv->htb_qos_sq_stats) { + struct mlx5e_sq_stats **stats_list; + + stats_list = kvcalloc(mlx5e_qos_max_leaf_nodes(priv->mdev), + sizeof(*stats_list), GFP_KERNEL); + if (!stats_list) + return -ENOMEM; + + WRITE_ONCE(priv->htb_qos_sq_stats, stats_list); + } + + if (!priv->htb_qos_sq_stats[node_qid]) { + struct mlx5e_sq_stats *stats; + stats = kzalloc(sizeof(*stats), GFP_KERNEL); - if (!stats) { - kvfree(stats_list); + if (!stats) return -ENOMEM; - } - if (stats_list) - WRITE_ONCE(priv->htb_qos_sq_stats, stats_list); + WRITE_ONCE(priv->htb_qos_sq_stats[node_qid], stats); /* Order htb_max_qos_sqs increment after writing the array pointer. * Pairs with smp_load_acquire in en_stats.c. From patchwork Tue Mar 26 14:46:45 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604304 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5716613C666 for ; Tue, 26 Mar 2024 14:47:07 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464427; cv=none; b=SNWn4mavLjyWz6JlWQ7xG78bF9bSBR42dcWZPShDxz9KRj9O5DO7+W96WyIMXfk53VWSpXDukMeoNC8pkCfhtUWAh6MElWh+41IkmxAg7E7S3ee+RmNqLjO5VJxzPtSDQnLG91E9ZBpLLg62UJGQyOxsaY40p9UqKzUSFlyF7kM= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464427; c=relaxed/simple; bh=FPCDULxX2JZK+MYnvtoUHQRsvHjfLGwPqNFWembX4lo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=uaTzg2v2a6TIwA3eQA1dtMLBokx0Gwa3Y0k/5IjJLeGes31Xd4j5o1HElfQ6yX7BrPsjwutzW8z/JEYf2vafggKoj3B86SPPEYrFZWrxXOfNl59gJBtrBFOWxSS1/MoY3ZKs66oMKgF6qfdX6XK5dCrRhB60D8LjsNIj5MZkwwQ= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=itW+ttUo; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="itW+ttUo" Received: by smtp.kernel.org (Postfix) with ESMTPSA id CBB4EC433F1; Tue, 26 Mar 2024 14:47:06 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464427; bh=FPCDULxX2JZK+MYnvtoUHQRsvHjfLGwPqNFWembX4lo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=itW+ttUotex+vgHenyTdCrsT++gGs8Idjx5KuIW8nTAWLhgGSZcqRj2BucOU0AcBe eic5M3BTi0/9ZQuFr3oB17E7s9aAA4CAkWpzyzB3+D43kn2YlF8X8XgxeHRUMppomh djRPNyJt7WWxmjPMRHkwyhw7e9s9QoCCukIXuTGzQZCI5XiVZxMeIIKcn09GuCbIPx VZ5EzVD3x8uXHWAoNNZH3/kKJjUh1YRNr0m+McuIii+rWKxSWFfrLi2GBAXL4Fy8x3 Ix7KxfD4/WtNqrpSziaPH9qAE84NJ4I9NW1VAshVikRiCxmcCTCr+3Y1ziTDe32laY V8m8mD67JkrHA== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Rahul Rameshbabu Subject: [net 09/10] net/mlx5e: Do not produce metadata freelist entries in Tx port ts WQE xmit Date: Tue, 26 Mar 2024 07:46:45 -0700 Message-ID: <20240326144646.2078893-10-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Rahul Rameshbabu Free Tx port timestamping metadata entries in the NAPI poll context and consume metadata enties in the WQE xmit path. Do not free a Tx port timestamping metadata entry in the WQE xmit path even in the error path to avoid a race between two metadata entry producers. Fixes: 3178308ad4ca ("net/mlx5e: Make tx_port_ts logic resilient to out-of-order CQEs") Signed-off-by: Rahul Rameshbabu Reviewed-by: Tariq Toukan Signed-off-by: Saeed Mahameed --- drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h | 8 +++++++- drivers/net/ethernet/mellanox/mlx5/core/en_tx.c | 7 +++---- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h index 86f1854698b4..883c044852f1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h @@ -95,9 +95,15 @@ static inline void mlx5e_ptp_metadata_fifo_push(struct mlx5e_ptp_metadata_fifo * } static inline u8 +mlx5e_ptp_metadata_fifo_peek(struct mlx5e_ptp_metadata_fifo *fifo) +{ + return fifo->data[fifo->mask & fifo->cc]; +} + +static inline void mlx5e_ptp_metadata_fifo_pop(struct mlx5e_ptp_metadata_fifo *fifo) { - return fifo->data[fifo->mask & fifo->cc++]; + fifo->cc++; } static inline void diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c index 2fa076b23fbe..e21a3b4128ce 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tx.c @@ -398,6 +398,8 @@ mlx5e_txwqe_complete(struct mlx5e_txqsq *sq, struct sk_buff *skb, (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))) { u8 metadata_index = be32_to_cpu(eseg->flow_table_metadata); + mlx5e_ptp_metadata_fifo_pop(&sq->ptpsq->metadata_freelist); + mlx5e_skb_cb_hwtstamp_init(skb); mlx5e_ptp_metadata_map_put(&sq->ptpsq->metadata_map, skb, metadata_index); @@ -496,9 +498,6 @@ mlx5e_sq_xmit_wqe(struct mlx5e_txqsq *sq, struct sk_buff *skb, err_drop: stats->dropped++; - if (unlikely(sq->ptpsq && (skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP))) - mlx5e_ptp_metadata_fifo_push(&sq->ptpsq->metadata_freelist, - be32_to_cpu(eseg->flow_table_metadata)); dev_kfree_skb_any(skb); mlx5e_tx_flush(sq); } @@ -657,7 +656,7 @@ static void mlx5e_cqe_ts_id_eseg(struct mlx5e_ptpsq *ptpsq, struct sk_buff *skb, { if (unlikely(skb_shinfo(skb)->tx_flags & SKBTX_HW_TSTAMP)) eseg->flow_table_metadata = - cpu_to_be32(mlx5e_ptp_metadata_fifo_pop(&ptpsq->metadata_freelist)); + cpu_to_be32(mlx5e_ptp_metadata_fifo_peek(&ptpsq->metadata_freelist)); } static void mlx5e_txwqe_build_eseg(struct mlx5e_priv *priv, struct mlx5e_txqsq *sq, From patchwork Tue Mar 26 14:46:46 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Saeed Mahameed X-Patchwork-Id: 13604313 X-Patchwork-Delegate: kuba@kernel.org Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A653013C90F for ; Tue, 26 Mar 2024 14:47:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=10.30.226.201 ARC-Seal: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464428; cv=none; b=cQI+etEmmVT+YTVC29HWdL9SoC4p5Ld4z90liBycbF7jQ82UPt8LAZfuHW4jrQS4aIKep1V8a+aCi2+0f1z3mrt6uZeujjfIEgm85+8au99udu7Mv9STkZWOb44Pe9LC6qH4olHFrtNj3qbger5gfk0fTyFnSL/uluzwjyISmtw= ARC-Message-Signature: i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711464428; c=relaxed/simple; bh=R42jsIDG/t+XNrhIdId5YWvl61mgtbTcHWQRtcC7afo=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=UKgN3zwYvaCGwLhqiMz1AwiH7W7CQncpw+mDq78fMbCfZPIoqnhPNwpVu5R8qxSkvtNemdp4vbGZEGgUqVGO9NZh1aY24fY94GkDBMGl+HnieOBlOZ+GaKqIk3A+B5dE0J2DJJ31nxEN+wEr8XVDkmkuVBVd25jbWxq8+IkiZVo= ARC-Authentication-Results: i=1; smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b=VtFCZ0Zh; arc=none smtp.client-ip=10.30.226.201 Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="VtFCZ0Zh" Received: by smtp.kernel.org (Postfix) with ESMTPSA id 62810C433B2; Tue, 26 Mar 2024 14:47:08 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1711464428; bh=R42jsIDG/t+XNrhIdId5YWvl61mgtbTcHWQRtcC7afo=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=VtFCZ0ZhGt8HIPfE78eJ0NutkTPs7aUUEUHeUAHrDaDBsLqVqT7yTn2oLkVV/pfxY dCuIf5f65a9ZkN76Jw3+WOza90xcVvZBb5o7EPZpHyCKvAl/fAYBoNnSh3GPLNYebr Qdh2kDVqlvlL/aFzAfxlWkuDGhIXviuoLz3xEGrOttvug8BIRp5WxC+XOx6ChU/DES sYBAgtiOJeNhXlREwYNxOv/WP6HXqIUza16hUmfPbSG/nKU+W9Kig7ar4vXHvtNEYu FS0019YupmS1ywWvxbENM5BQEwyZf6Bp2dHp/vXSFdWZ1WXoR9B3juzs8b9410XTcd b7P/EeDAVeV+g== From: Saeed Mahameed To: "David S. Miller" , Jakub Kicinski , Paolo Abeni , Eric Dumazet Cc: Saeed Mahameed , netdev@vger.kernel.org, Tariq Toukan , Gal Pressman , Leon Romanovsky , Carolina Jubran Subject: [net 10/10] net/mlx5e: RSS, Block XOR hash with over 128 channels Date: Tue, 26 Mar 2024 07:46:46 -0700 Message-ID: <20240326144646.2078893-11-saeed@kernel.org> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240326144646.2078893-1-saeed@kernel.org> References: <20240326144646.2078893-1-saeed@kernel.org> Precedence: bulk X-Mailing-List: netdev@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 X-Patchwork-Delegate: kuba@kernel.org From: Carolina Jubran When supporting more than 128 channels, the RQT size is calculated by multiplying the number of channels by 2 and rounding up to the nearest power of 2. The index of the RQT is derived from the RSS hash calculations. If XOR8 is used as the RSS hash function, there are only 256 possible hash results, and therefore, only 256 indexes can be reached in the RQT. Block setting the RSS hash function to XOR when the number of channels exceeds 128. Fixes: 74a8dadac17e ("net/mlx5e: Preparations for supporting larger number of channels") Signed-off-by: Carolina Jubran Signed-off-by: Saeed Mahameed --- .../net/ethernet/mellanox/mlx5/core/en/rqt.c | 7 +++++ .../net/ethernet/mellanox/mlx5/core/en/rqt.h | 1 + .../ethernet/mellanox/mlx5/core/en_ethtool.c | 28 +++++++++++++++++-- 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.c b/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.c index bcafb4bf9415..8d9a3b5ec973 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.c @@ -179,6 +179,13 @@ u32 mlx5e_rqt_size(struct mlx5_core_dev *mdev, unsigned int num_channels) return min_t(u32, rqt_size, max_cap_rqt_size); } +#define MLX5E_MAX_RQT_SIZE_ALLOWED_WITH_XOR8_HASH 256 + +unsigned int mlx5e_rqt_max_num_channels_allowed_for_xor8(void) +{ + return MLX5E_MAX_RQT_SIZE_ALLOWED_WITH_XOR8_HASH / MLX5E_UNIFORM_SPREAD_RQT_FACTOR; +} + void mlx5e_rqt_destroy(struct mlx5e_rqt *rqt) { mlx5_core_destroy_rqt(rqt->mdev, rqt->rqtn); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.h b/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.h index e0bc30308c77..2f9e04a8418f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/rqt.h @@ -38,6 +38,7 @@ static inline u32 mlx5e_rqt_get_rqtn(struct mlx5e_rqt *rqt) } u32 mlx5e_rqt_size(struct mlx5_core_dev *mdev, unsigned int num_channels); +unsigned int mlx5e_rqt_max_num_channels_allowed_for_xor8(void); int mlx5e_rqt_redirect_direct(struct mlx5e_rqt *rqt, u32 rqn, u32 *vhca_id); int mlx5e_rqt_redirect_indir(struct mlx5e_rqt *rqt, u32 *rqns, u32 *vhca_ids, unsigned int num_rqns, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c index c5a203b5119d..d60cb6d47d26 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c @@ -451,6 +451,17 @@ int mlx5e_ethtool_set_channels(struct mlx5e_priv *priv, mutex_lock(&priv->state_lock); + if (mlx5e_rx_res_get_current_hash(priv->rx_res).hfunc == ETH_RSS_HASH_XOR) { + unsigned int xor8_max_channels = mlx5e_rqt_max_num_channels_allowed_for_xor8(); + + if (count > xor8_max_channels) { + err = -EINVAL; + netdev_err(priv->netdev, "%s: Requested number of channels (%d) exceeds the maximum allowed by the XOR8 RSS hfunc (%d)\n", + __func__, count, xor8_max_channels); + goto out; + } + } + /* Don't allow changing the number of channels if RXFH was previously configured. * Changing the channels number after configuring the RXFH may alter the RSS table size, * the previous configuration may no longer be compatible with the new RSS table. @@ -1292,17 +1303,30 @@ int mlx5e_set_rxfh(struct net_device *dev, struct ethtool_rxfh_param *rxfh, struct mlx5e_priv *priv = netdev_priv(dev); u32 *rss_context = &rxfh->rss_context; u8 hfunc = rxfh->hfunc; + unsigned int count; int err; mutex_lock(&priv->state_lock); + + count = priv->channels.params.num_channels; + + if (hfunc == ETH_RSS_HASH_XOR) { + unsigned int xor8_max_channels = mlx5e_rqt_max_num_channels_allowed_for_xor8(); + + if (count > xor8_max_channels) { + err = -EINVAL; + netdev_err(priv->netdev, "%s: Cannot set RSS hash function to XOR, current number of channels (%d) exceeds the maximum allowed for XOR8 RSS hfunc (%d)\n", + __func__, count, xor8_max_channels); + goto unlock; + } + } + if (*rss_context && rxfh->rss_delete) { err = mlx5e_rx_res_rss_destroy(priv->rx_res, *rss_context); goto unlock; } if (*rss_context == ETH_RXFH_CONTEXT_ALLOC) { - unsigned int count = priv->channels.params.num_channels; - err = mlx5e_rx_res_rss_init(priv->rx_res, rss_context, count); if (err) goto unlock;