From patchwork Fri Feb 11 09:20:18 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 12743050 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 06B70C433EF for ; Fri, 11 Feb 2022 09:21:10 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348497AbiBKJVJ (ORCPT ); Fri, 11 Feb 2022 04:21:09 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33880 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234834AbiBKJVE (ORCPT ); Fri, 11 Feb 2022 04:21:04 -0500 Received: from NAM12-DM6-obe.outbound.protection.outlook.com (mail-dm6nam12on2079.outbound.protection.outlook.com [40.107.243.79]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2CD7A337; Fri, 11 Feb 2022 01:21:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=TEbxkgZcT0A3HzBYwlJyrBEhy+T9fqjUXYhDwUfM4UF4KVvxp1Y5Vuc7JFy+0olIKeP8lB9pgPQeupS7ufwnNkgexUjcjRNK2txn8p8sGdhHWhxsiF/55HHA9U0gQBDB/Xgcrpwp32WipaTjXOTVayO4hlSiheRdMOll0+FdN9TpYlG0Lv60+xgPJrrKBVJnBdqA2jEuH+3ILf3ShQtF2ojQ8SHMd1UxL4VxZ1Qm90IKUoetA2IaYU5Hh0YP0nFBtuQIoqazG0b43bvDVCt0dLJvGPG1Mtv+GAn5rQGBrNrWxLgeLpb5G2e4p2Zs0hIeZ5j32bh0CZJSoKRn4C/tFA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=ZUgTWgtUpSUYxncI0pCBf1y6AddDiImbe/TPYjlutNA=; b=HfQDkJA1fuumMK8JOgJtt/OWock0meOke3AwOnxsXmIW/h2WLftQXxI2xeolZciVS7uaIKDvw4y28OBEkoSbEaFct+tsNh1lGIA8eMufsVHiFcu++IM0LXk46uZ04t+wydPB98YZLyM5Ml49/22AfLvY1gvUGP+xdz8ICXHz3M/WhSu/na6lJapFSzacYBuvMfrMOmApUzWfrdjtE/PhG1URjMlWiN3agNJqa7TOgA2TjxMPDPKfT2+2BDlx7ULUIbQj7I9qqHxHoWQ7NQlu22THdZ7dNtgjwjswMjTIlHoM3HkZSuYWDCQ9LFMO5qnTKljWRktc6VGUJobUTCsexw== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=ZUgTWgtUpSUYxncI0pCBf1y6AddDiImbe/TPYjlutNA=; b=bfdTL+MqheSwdYB5phTeczdm+9zjFRVsf1jBddPuPWRj3VxNlXIfrk8Yom+/yYwkQJgRU0Ls4tE+sxAVcb0CifxzpC0bT/Ye/gwWdo1LSRYm/xmeVoVDG3A/gJiqGXjTaHVpU9Gp6p+scsUDeonlTzwjd2ZScaJXau2CnunAE1aEgQKkANmLINC4K1f2x+whvMLUJOxY0D5sbKaSx1yQ4NV4vcPLklyBONKKGofMxB7QyaWeJFdc9IL9fZXY2pXFXxKAI2EAaLXZ73Tgs0D/Xur+5ph/74WaFvvZ0mq05ZLJSaAcHalseu3iW5myF2CwuN1Hc4+FeVtgOt/lIjY+vQ== Received: from MW4PR03CA0013.namprd03.prod.outlook.com (2603:10b6:303:8f::18) by SN1PR12MB2560.namprd12.prod.outlook.com (2603:10b6:802:26::19) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.12; Fri, 11 Feb 2022 09:21:02 +0000 Received: from CO1NAM11FT018.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8f:cafe::ca) by MW4PR03CA0013.outlook.office365.com (2603:10b6:303:8f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.14 via Frontend Transport; Fri, 11 Feb 2022 09:21:02 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT018.mail.protection.outlook.com (10.13.175.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4975.11 via Frontend Transport; Fri, 11 Feb 2022 09:21:01 +0000 Received: from rnnvmail202.nvidia.com (10.129.68.7) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 11 Feb 2022 09:20:56 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail202.nvidia.com (10.129.68.7) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 11 Feb 2022 01:20:55 -0800 Received: from vdi.nvidia.com (10.127.8.14) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 11 Feb 2022 01:20:53 -0800 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski , Parav Pandit CC: Jiri Pirko , Saeed Mahameed , , , Shay Drory Subject: [PATCH net-next v2 1/4] net/mlx5: Split function_setup() to enable and open functions Date: Fri, 11 Feb 2022 11:20:18 +0200 Message-ID: <1644571221-237302-2-git-send-email-moshe@nvidia.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1644571221-237302-1-git-send-email-moshe@nvidia.com> References: <1644571221-237302-1-git-send-email-moshe@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1f6d8bff-185a-41c1-d036-08d9ed3fd28c X-MS-TrafficTypeDiagnostic: SN1PR12MB2560:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:6108; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: qwed99iTzLJcTghGPc+eeAuKyUCtalZkgbynuKwBntXpbnT0RDLxcSS+x5TW0h3eN6yj5/tyJrCBL92fMVVh+MdBjbpg9Ofyw+jKkYv/Jpga/evokLgrkW1HQLBwo/VyRa24TOFuFD85uUvkPNuu/axatEpGKFMSUcoQA/pKqAkMrTz22FuLOQjkhf3sOe2gEoq1rJWiVj802Qfjga98WJbW7xvRO4Ee2O/tQN1jzsFZiwTgOGU4gdSkiaLtei2XWGVvQntvMQb5vuvSZ7CLSOpbO9l1DrjtwSCTtUWe3UIBQG2LJzu47qdQxwfwT2Cr3oBq2gmq/K48K90wmFaaPgoNrW+wdX4fynnBRVwVGBebWr3G/VHEvhiGfOiZEZpWA6+yUKmHNFLuHlxhrss3S5S6qJ26xbZTc4J7XpVHsnUz3bryaETyfmWRA8RJjcaF/F23kyEqBlJ6KN2mZGuAoRFMncySc3fGo7He9i9SJkH+bQmXy3fa2ZO5/RNdZTiegmYG7xx/zuWPQXPUPh7UOi4loDkAJMLzXen7AEGdxnsXDk4+49s1l9Oh+H0d3deCZG9t1T9gcp1ngGWmbeYZhyC1SxinIm55quuIL4G7Ll75w2ME1awxvV45ct6BQGvee5/VZlIIa0H4q/mleLoVtuYa8ekvWWgF9WObqBOihLpCsM5XjtbDi1DyexSZqJEMNZo/S9r37lERfkjeq30+WQ== X-Forefront-Antispam-Report: CIP:12.22.5.236;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(36840700001)(46966006)(40470700004)(6666004)(83380400001)(47076005)(508600001)(5660300002)(2906002)(40460700003)(36756003)(36860700001)(110136005)(8936002)(70586007)(86362001)(70206006)(8676002)(4326008)(316002)(82310400004)(6636002)(54906003)(107886003)(336012)(426003)(81166007)(2616005)(186003)(26005)(356005)(7696005)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Feb 2022 09:21:01.9708 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1f6d8bff-185a-41c1-d036-08d9ed3fd28c X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT018.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN1PR12MB2560 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Shay Drory mlx5_cmd_init_hca() is taking ~0.2 seconds. In case of a user who desire to disable some of the SF aux devices, and with large scale-1K SFs for example, this user will waste more than 3 minutes on mlx5_cmd_init_hca() which isn't needed at this stage. Therefore, split function_setup() in order to avoid executing mlx5_cmd_init_hca() in case user disables SFs aux dev via the devlink param which will be introduced in downstream patch. Signed-off-by: Shay Drory Reviewed-by: Saeed Mahameed Reviewed-by: Moshe Shemesh --- .../net/ethernet/mellanox/mlx5/core/main.c | 81 +++++++++++++------ 1 file changed, 56 insertions(+), 25 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 2c774f367199..73d2cec01ead 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -1006,7 +1006,7 @@ static void mlx5_cleanup_once(struct mlx5_core_dev *dev) mlx5_devcom_unregister_device(dev->priv.devcom); } -static int mlx5_function_setup(struct mlx5_core_dev *dev, bool boot) +static int mlx5_function_enable(struct mlx5_core_dev *dev) { int err; @@ -1070,28 +1070,53 @@ static int mlx5_function_setup(struct mlx5_core_dev *dev, bool boot) goto reclaim_boot_pages; } + return 0; + +reclaim_boot_pages: + mlx5_reclaim_startup_pages(dev); +err_disable_hca: + mlx5_core_disable_hca(dev, 0); +err_cmd_cleanup: + mlx5_cmd_set_state(dev, MLX5_CMDIF_STATE_DOWN); + mlx5_cmd_cleanup(dev); + + return err; +} + +static void mlx5_function_disable(struct mlx5_core_dev *dev) +{ + mlx5_reclaim_startup_pages(dev); + mlx5_core_disable_hca(dev, 0); + mlx5_cmd_set_state(dev, MLX5_CMDIF_STATE_DOWN); + mlx5_cmd_cleanup(dev); +} + +static int mlx5_function_open(struct mlx5_core_dev *dev, bool boot) +{ + int err; + err = set_hca_ctrl(dev); if (err) { mlx5_core_err(dev, "set_hca_ctrl failed\n"); - goto reclaim_boot_pages; + return err; } err = set_hca_cap(dev); if (err) { mlx5_core_err(dev, "set_hca_cap failed\n"); - goto reclaim_boot_pages; + return err; } err = mlx5_satisfy_startup_pages(dev, 0); if (err) { mlx5_core_err(dev, "failed to allocate init pages\n"); - goto reclaim_boot_pages; + return err; } err = mlx5_cmd_init_hca(dev, sw_owner_id); if (err) { mlx5_core_err(dev, "init hca failed\n"); - goto reclaim_boot_pages; + return err; } mlx5_set_driver_version(dev); @@ -1099,25 +1124,15 @@ static int mlx5_function_setup(struct mlx5_core_dev *dev, bool boot) err = mlx5_query_hca_caps(dev); if (err) { mlx5_core_err(dev, "query hca failed\n"); - goto reclaim_boot_pages; + return err; } mlx5_start_health_poll(dev); return 0; - -reclaim_boot_pages: - mlx5_reclaim_startup_pages(dev); -err_disable_hca: - mlx5_core_disable_hca(dev, 0); -err_cmd_cleanup: - mlx5_cmd_set_state(dev, MLX5_CMDIF_STATE_DOWN); - mlx5_cmd_cleanup(dev); - - return err; } -static int mlx5_function_teardown(struct mlx5_core_dev *dev, bool boot) +static int mlx5_function_close(struct mlx5_core_dev *dev, bool boot) { int err; @@ -1127,14 +1142,30 @@ static int mlx5_function_teardown(struct mlx5_core_dev *dev, bool boot) mlx5_core_err(dev, "tear_down_hca failed, skip cleanup\n"); return err; } - mlx5_reclaim_startup_pages(dev); - mlx5_core_disable_hca(dev, 0); - mlx5_cmd_set_state(dev, MLX5_CMDIF_STATE_DOWN); - mlx5_cmd_cleanup(dev); return 0; } +static int mlx5_function_setup(struct mlx5_core_dev *dev, bool boot) +{ + int err; + + err = mlx5_function_enable(dev); + if (err) + return err; + + err = mlx5_function_open(dev, boot); + if (err) + mlx5_function_disable(dev); + return err; +} + +static void mlx5_function_teardown(struct mlx5_core_dev *dev, bool boot) +{ + if (!mlx5_function_close(dev, boot)) + mlx5_function_disable(dev); +} + static int mlx5_load(struct mlx5_core_dev *dev) { int err; @@ -1290,7 +1321,7 @@ int mlx5_init_one(struct mlx5_core_dev *dev) err = mlx5_function_setup(dev, true); if (err) - goto err_function; + goto err_function_setup; err = mlx5_init_once(dev); if (err) { @@ -1324,7 +1355,7 @@ int mlx5_init_one(struct mlx5_core_dev *dev) mlx5_cleanup_once(dev); function_teardown: mlx5_function_teardown(dev, true); -err_function: +err_function_setup: dev->state = MLX5_DEVICE_STATE_INTERNAL_ERROR; mutex_unlock(&dev->intf_state_mutex); return err; @@ -1366,7 +1397,7 @@ int mlx5_load_one(struct mlx5_core_dev *dev) err = mlx5_function_setup(dev, false); if (err) - goto err_function; + goto err_function_setup; err = mlx5_load(dev); if (err) @@ -1386,7 +1417,7 @@ int mlx5_load_one(struct mlx5_core_dev *dev) mlx5_unload(dev); err_load: mlx5_function_teardown(dev, false); -err_function: +err_function_setup: dev->state = MLX5_DEVICE_STATE_INTERNAL_ERROR; out: mutex_unlock(&dev->intf_state_mutex); From patchwork Fri Feb 11 09:20:19 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 12743049 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 13304C433FE for ; Fri, 11 Feb 2022 09:21:05 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1346668AbiBKJVE (ORCPT ); Fri, 11 Feb 2022 04:21:04 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:33870 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234834AbiBKJVD (ORCPT ); Fri, 11 Feb 2022 04:21:03 -0500 Received: from NAM10-MW2-obe.outbound.protection.outlook.com (mail-mw2nam10on2040.outbound.protection.outlook.com [40.107.94.40]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 465D8102D; Fri, 11 Feb 2022 01:21:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=fo5f9o29h+Pbrpi992+giUcvNIJymC3vcKzeShRILiSCqwBLi+TJSRvmvSbL/mY0qf9Pxx3IpMFucJgZtiaiPrFT+ICtBaW9Fh0nUQiZsbC4zQ+/uLGDhSADzVQlxm+gEbBi5MSZ3Jl+lKmTgFRFyCTS8RVUTqX+fOmv2HeIpms31lBARPTIAG5MYAHEPDTIjJmKeXBwS6Sorq7zxwktQ5WPLeEZAiIB79jLWGMIb8TAcwNonnh+hd2OUWX6bPl7D25Nd5ceIAtScZZHYUd8Kc+zRn148Nmt7lHyVkT5U/cWHOlkoRXleXjRZsgE1r024tqsYHn1+8Z/2UivOKi8cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=O9SpnAZFq6PFMtygOpdxQylPAuS6Us4gXIQSvjUc1O8=; b=DQHxvzDmPfr/5Sn4A2z0cjL8jMFDf6cXSCHAXml1txlU3gVv4WK4HQHJ51ABsfSUQavXS5CNXfGqUmjpKxkeGqnVhzgAq749AuUw5mRs5/hHlaUNPFGstNDXtiPAzWh0RuRazXaiYB/VQ8ieMne/EWckJs4UcAFEtD97aeCbC1IRxgHkNNL+rDBYP2txWZRRA9ddPGLTs6grRe+VAwwaSt7cWuGKRHByh+YEQYIOIWsWJYOZhGteOCBPmuhr5AFes/WTAgkoYMQyHlXCKbkdzE/B6iHTN/VQ7JPRplmGmltIDVKNl/xVzkbGTKMCuvp3rLEHxRt/D+JGakKrU4xwUQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.234) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=O9SpnAZFq6PFMtygOpdxQylPAuS6Us4gXIQSvjUc1O8=; b=CiX5NSj+aAzb6En2E9zWDZYNIcD97EIGAKjzGDG7LjeGTvBkGtMaz2bZfrJ1tMOzwuH2lnexKP36YoRL2gDFdTT9yjOa24Bq++b6AdGvzxUV6SGPP61DFAV92pSqz0a8NXbC/5kC2YA7vuaH0bt+fmPCpEhriKCYS4dbjtE7MQ+8S4eEzvYbZ4hZyz8FGh3w0D8Mb3maDQNMB7KMT8lLUmfBthRKTSOFg4Ha/HdtPUzPfo8g76YSYMfHgDDksGl7FfCImkcywwzENKpRvSGbUwPLqDuI/nmeEfn6dHaA7sFmp8sl/dru8wKg2zjR73c5k559iOlv/r6HMtnTQziN0g== Received: from DM5PR13CA0048.namprd13.prod.outlook.com (2603:10b6:3:7b::34) by CH2PR12MB3655.namprd12.prod.outlook.com (2603:10b6:610:25::23) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11; Fri, 11 Feb 2022 09:20:59 +0000 Received: from DM6NAM11FT018.eop-nam11.prod.protection.outlook.com (2603:10b6:3:7b:cafe::5e) by DM5PR13CA0048.outlook.office365.com (2603:10b6:3:7b::34) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4995.6 via Frontend Transport; Fri, 11 Feb 2022 09:20:59 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.234) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.234 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.234; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.234) by DM6NAM11FT018.mail.protection.outlook.com (10.13.172.110) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4975.11 via Frontend Transport; Fri, 11 Feb 2022 09:20:59 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by DRHQMAIL101.nvidia.com (10.27.9.10) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 11 Feb 2022 09:20:58 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 11 Feb 2022 01:20:58 -0800 Received: from vdi.nvidia.com (10.127.8.14) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 11 Feb 2022 01:20:56 -0800 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski , Parav Pandit CC: Jiri Pirko , Saeed Mahameed , , , Shay Drory Subject: [PATCH net-next v2 2/4] net/mlx5: Delete redundant default assignment of runtime devlink params Date: Fri, 11 Feb 2022 11:20:19 +0200 Message-ID: <1644571221-237302-3-git-send-email-moshe@nvidia.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1644571221-237302-1-git-send-email-moshe@nvidia.com> References: <1644571221-237302-1-git-send-email-moshe@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: b83ad22f-0e79-482b-da1f-08d9ed3fd119 X-MS-TrafficTypeDiagnostic: CH2PR12MB3655:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:3173; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: PvO39q9WN3Ac7CzBDYjsd43m+Gz78EnEaQWcWfGkW1szLRgZICn9KyztgL01fF51MtJb2OuMbYF4Sm3VCmaubBNHpPpTYLHauJKhqwxZ6Xkee4V2AThYfRx1ICZgUwbF2Kl64TRID4yJZ2Cgu0uWBHnQ8Na2DMWTlRGWmPrQGkjptUTkvtCy8MRzwRLQtACx68Dsf4sH5YHl5jHqjv6qsJysURzN6i4NkG2+4HJzBKveYV6FWLjjuIKyEjqoj+MH3W3KjZufVmtNsrIRJk91tvz3S+/aJbsCVjUSrmreFMJBt9/mqVKFXN8AHDsJtZvgdiJQ41lG6PhBKhMCoANt0GfAgZWpCWOYYWe98wxeFfPF5CkpoCcUnSJdVUjejaFT+GcwpeMnUk3m2a1aX6wcOePHFU8oz80GPtMf5kaWVI/kIOf1zhJZyq0rcxeGe5C5V7nHnULw7QrcJL/bpUS1+RDXuFZN5hsjLEuY+/BPsXAlNxZgiy+PhWs1J0LyLzvNJJ/Q+VGQtu7YJfwTL5y8iHpIh3WYPLpCNW22EJD6UjgJIm6+s9rY/psiFoz3r+ne2Gg6WRUk9pkvpi3I6LkLhPMD8DIFHviZPmOXOZrRfMKGwUH+/BhDV3F39GF3AIN6XIn4gadFXZH+DFDrruv3555XurRyKGtH/0JQzZ7Q6IId5cwinTba9ogfuitI56ZAwgjvUPjawfflwZXIOpRCUQ== X-Forefront-Antispam-Report: CIP:12.22.5.234;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(46966006)(40470700004)(36840700001)(110136005)(316002)(54906003)(8936002)(4326008)(86362001)(6636002)(6666004)(7696005)(36860700001)(47076005)(36756003)(356005)(81166007)(8676002)(2616005)(82310400004)(83380400001)(508600001)(107886003)(26005)(70206006)(5660300002)(70586007)(336012)(426003)(2906002)(40460700003)(186003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Feb 2022 09:20:59.5240 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: b83ad22f-0e79-482b-da1f-08d9ed3fd119 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.234];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: DM6NAM11FT018.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB3655 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Shay Drory Runtime devlink params always read their values from the get() callbacks. Also, it is an error to set driverinit_value for params which don't support driverinit cmode. Delete such assignments. In addition, move the set of default matching mode inside eswitch code. Signed-off-by: Shay Drory Reviewed-by: Saeed Mahameed Reviewed-by: Moshe Shemesh --- .../net/ethernet/mellanox/mlx5/core/devlink.c | 20 ------------------- .../net/ethernet/mellanox/mlx5/core/eswitch.c | 3 +++ 2 files changed, 3 insertions(+), 20 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index d1093bb2d436..e832a3f4c18a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -588,14 +588,6 @@ static void mlx5_devlink_set_params_init_values(struct devlink *devlink) struct mlx5_core_dev *dev = devlink_priv(devlink); union devlink_param_value value; - if (dev->priv.steering->mode == MLX5_FLOW_STEERING_MODE_DMFS) - strcpy(value.vstr, "dmfs"); - else - strcpy(value.vstr, "smfs"); - devlink_param_driverinit_value_set(devlink, - MLX5_DEVLINK_PARAM_ID_FLOW_STEERING_MODE, - value); - value.vbool = MLX5_CAP_GEN(dev, roce); devlink_param_driverinit_value_set(devlink, DEVLINK_PARAM_GENERIC_ID_ENABLE_ROCE, @@ -606,18 +598,6 @@ static void mlx5_devlink_set_params_init_values(struct devlink *devlink) devlink_param_driverinit_value_set(devlink, MLX5_DEVLINK_PARAM_ID_ESW_LARGE_GROUP_NUM, value); - - if (MLX5_ESWITCH_MANAGER(dev)) { - if (mlx5_esw_vport_match_metadata_supported(dev->priv.eswitch)) { - dev->priv.eswitch->flags |= MLX5_ESWITCH_VPORT_MATCH_METADATA; - value.vbool = true; - } else { - value.vbool = false; - } - devlink_param_driverinit_value_set(devlink, - MLX5_DEVLINK_PARAM_ID_ESW_PORT_METADATA, - value); - } #endif value.vu32 = MLX5_COMP_EQ_SIZE; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c index 458ec0bca1b8..25f2d2717aaa 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch.c @@ -1582,6 +1582,9 @@ int mlx5_eswitch_init(struct mlx5_core_dev *dev) esw->offloads.encap = DEVLINK_ESWITCH_ENCAP_MODE_BASIC; else esw->offloads.encap = DEVLINK_ESWITCH_ENCAP_MODE_NONE; + if (MLX5_ESWITCH_MANAGER(dev) && + mlx5_esw_vport_match_metadata_supported(esw)) + esw->flags |= MLX5_ESWITCH_VPORT_MATCH_METADATA; dev->priv.eswitch = esw; BLOCKING_INIT_NOTIFIER_HEAD(&esw->n_head); From patchwork Fri Feb 11 09:20:20 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 12743053 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 07087C433EF for ; Fri, 11 Feb 2022 09:21:23 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348547AbiBKJVU (ORCPT ); Fri, 11 Feb 2022 04:21:20 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:34122 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237798AbiBKJVR (ORCPT ); Fri, 11 Feb 2022 04:21:17 -0500 Received: from NAM12-BN8-obe.outbound.protection.outlook.com (mail-bn8nam12on2065.outbound.protection.outlook.com [40.107.237.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id D974A1034; Fri, 11 Feb 2022 01:21:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=jJmszvxl7c32Y7Cl0n/fFhyBIPWt+YmeoTKTMhUM/X3ykZIceADOHt3VGZqPMW9xoI7zd2SGWXZTRdM1orHz88ykvFIkyMmyEGcBKg1MSCoRWXAKA+AXetmWA9za/8katF91UR12M96uqZT2nF60sB6YQTMe8f+dJSMqsdyXcGub3O2m9QUhRBPmdt9h3e/N8uQilMz/d4nZY4BpvNPoPKGSe5PFu8wOjKtr5mVLF1aXeb0kusHiX5yuxU4bUeu58sloTFCRHr1zDF7t3UowYiCSh8Bqkc5W3AWK+jaZZPY3F31ameLJy+Q9HPRQKVomxIHrkZyTQIxwqXhQrcx8fA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=5k8t1DLrIa7Lpc6/8FPOEDHil4UH20t5yO0nrHsCcio=; b=CuwUbO10A5pXMkY64SH2AsXV38pDQ6LMwv3CrA5U1k1UahLdcMVTYhtxRcI23LvzqioKrz7IJs7qGNy+0FFMBslkYKwyBxWVICHSN7a8OXV0m0+mdk2E7dVBGqqGJYdQ38DsVHKX/OkYB+z7IRYKjXT6HDgnaSVAOCnf+CX3bJAySRNK87pmQk/l3X72TFxHfShhRM1I4dSafw/fwqvvRGsrFlKKPwk33L5FPM8YDZw2McIT0972mYX0UMG6uG9M04L3FGbTf3AgKf0h584pQCR0RlrmcJ3lebEhfSfh5KTwDd5T8+wmovZwCzRi+xzeRfxB3cmpAOvBZ1IuewWfLQ== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.236) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=5k8t1DLrIa7Lpc6/8FPOEDHil4UH20t5yO0nrHsCcio=; b=ANh/PKtpZZ4pSvzzi9Wnf6vaQ1g0dQjEj7X/atsQqgKnOpXBu4myfvFV4mGE73qOIHUevwdJfajp+CbcA8YDkK2YyYXHacV6YSY09Cz1TqVgOvMVAzGGxcfB0NSwUt/pe40zYLlG9lvwJ7YNPiOYtBJ48u7Rqh/9PHqZVRCeZwtn7JgoyFSKm9b3Fa2SS+uPnVAFcScabPLWHD5/vJT1/uTDfMEX9fOA1yKn3f5tUxOlzFTG6gPPEZOYtozLaSTkv3RSzcmpNJA3v/6Ds1lVcTZan8ww1yKm6eMZvIvlzDqtwbel/tzMx0N0BYR/fmbdb2MIWKMLGvBOI7Z5/I6Xjg== Received: from MW4PR03CA0013.namprd03.prod.outlook.com (2603:10b6:303:8f::18) by MN2PR12MB2989.namprd12.prod.outlook.com (2603:10b6:208:c4::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.14; Fri, 11 Feb 2022 09:21:06 +0000 Received: from CO1NAM11FT018.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8f:cafe::d8) by MW4PR03CA0013.outlook.office365.com (2603:10b6:303:8f::18) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.14 via Frontend Transport; Fri, 11 Feb 2022 09:21:06 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.236) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.236 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.236; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.236) by CO1NAM11FT018.mail.protection.outlook.com (10.13.175.16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4975.11 via Frontend Transport; Fri, 11 Feb 2022 09:21:06 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by DRHQMAIL109.nvidia.com (10.27.9.19) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 11 Feb 2022 09:21:01 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail203.nvidia.com (10.129.68.9) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 11 Feb 2022 01:21:00 -0800 Received: from vdi.nvidia.com (10.127.8.14) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 11 Feb 2022 01:20:58 -0800 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski , Parav Pandit CC: Jiri Pirko , Saeed Mahameed , , , Shay Drory Subject: [PATCH net-next v2 3/4] devlink: Add new "enable_sfs_aux_devs" generic device param Date: Fri, 11 Feb 2022 11:20:20 +0200 Message-ID: <1644571221-237302-4-git-send-email-moshe@nvidia.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1644571221-237302-1-git-send-email-moshe@nvidia.com> References: <1644571221-237302-1-git-send-email-moshe@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 63418489-9feb-4f8d-477e-08d9ed3fd4f6 X-MS-TrafficTypeDiagnostic: MN2PR12MB2989:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:8273; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: eEJS62tcvAjHjG0xJNklSQW6A2Bzgdgd+L8n512IVLuRED1ctdFHhGDBVhI1qG+dUFkWCYEcInkBXb2ChIgrrU3xkC+hjSX3BUTldpS3roakBn93xRFrZ5kukYE1gZiGlRB+t+oFUb7Labhk8AM615gmJ9iJOhfyZjKWJz8YU21Ga4TspIVOTtTs4ASgotbsVJ7v3Q2ihgfcprz60hsb221ZXckCgI8tJJCHnqcwhfR9F2R0gY+BHb9eXF/7jIVvfd18FVjr6UtbFxWlECQ3gSJyReWzZha1wkMHgpM6kVpZFarK5yJolPN/zOgoA0qlaioVVd6ydC8ZwYlH8QRSFN+BNnjmkfhnR/IRxWc5ZL4BXCH3Zs3+5zml2TKzb8UgGJ4pAmUF3KB6p5Qcd49Hee9SwB/W9wa96XC/ujrFx4mi0IB4DeAHOXKgVq4GLfjlDNNa1giSdpzOKpXzgexXF6x6BcCfF+S8u1xsYEaIeayutg3AmWaaX2npn3ZIkoVjOTySiHxamQW+e21HU7va5uPtnDRXjKvuYo0NORTnznLB7u7+b/KbH6FLpFc9oXymyTimzhBvz4ewq3OzRxQZvpTV+rabLcc38ATTYYaq5FRsHu3zM/wWeuRbsGTY7Nc1RycdK9hdJa3hZos+zk4B1Ahojmtan4lQusMvKNV3IZozu5xCT6o75efLKxscXCRirT8sIKolvdeG8KmfVBaBDHIrwAoABGohEVCfnDK2kFQ= X-Forefront-Antispam-Report: CIP:12.22.5.236;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(6636002)(316002)(7696005)(82310400004)(6666004)(70586007)(70206006)(54906003)(508600001)(110136005)(86362001)(47076005)(36860700001)(40460700003)(426003)(336012)(186003)(107886003)(26005)(81166007)(83380400001)(356005)(36756003)(8936002)(2906002)(2616005)(8676002)(4326008)(5660300002)(41533002)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Feb 2022 09:21:06.0174 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 63418489-9feb-4f8d-477e-08d9ed3fd4f6 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.236];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT018.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB2989 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Shay Drory Add new device generic parameter to enable/disable creation of Sub-Functions auxiliary devices of a certain Physical-Function. User who wants to use specific SFs auxiliary devices, can disable the creation of all SF auxiliary devices upon SF creation. After the SF is created, the user can enable only the requested auxiliary devices. The new parameter can be used also in multi-host setup where ESW and PF are on different hosts, for example: Disable SF's children auxiliary device probing for the specified PF on host: $ devlink dev param set pci/0000:0b:00.0 \ name enable_sfs_aux_devs value false cmode runtime Create SF on ESW side: $ devlink port add pci/0000:08:00.0 flavour pcisf pfnum 0 sfnum 11 \ controller 1 $ devlink port function set pci/0000:08:00.0/32768 \ hw_addr 00:00:00:00:00:11 state active Enable ETH auxiliary device on SF on host: $ devlink dev param set auxiliary/mlx5_core.sf.1 \ name enable_eth value true cmode driverinit $ devlink dev reload auxiliary/mlx5_core.sf.1 At this point the user have SF devlink instance with auxiliary device for the Ethernet functionality only. Signed-off-by: Shay Drory Reviewed-by: Jiri Pirko Reviewed-by: Moshe Shemesh Reviewed-by: Parav Pandit --- changelog: v1->v2: - updated example to make clear SF port and device creation PF can be on different hosts. --- Documentation/networking/devlink/devlink-params.rst | 5 +++++ include/net/devlink.h | 4 ++++ net/core/devlink.c | 5 +++++ 3 files changed, 14 insertions(+) diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst index 4e01dc32bc08..aa0edb915f88 100644 --- a/Documentation/networking/devlink/devlink-params.rst +++ b/Documentation/networking/devlink/devlink-params.rst @@ -137,3 +137,8 @@ own name. * - ``event_eq_size`` - u32 - Control the size of asynchronous control events EQ. + * - ``enable_sfs_aux_devs`` + - Boolean + - When enabled, the device driver will instantiate all auxiliary devices of + SFs of the devlink device. When clear, the SFs of the devlink device will + not instantiate any auxiliary devices. diff --git a/include/net/devlink.h b/include/net/devlink.h index 8d5349d2fb68..8013252790bf 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -461,6 +461,7 @@ enum devlink_param_generic_id { DEVLINK_PARAM_GENERIC_ID_ENABLE_IWARP, DEVLINK_PARAM_GENERIC_ID_IO_EQ_SIZE, DEVLINK_PARAM_GENERIC_ID_EVENT_EQ_SIZE, + DEVLINK_PARAM_GENERIC_ID_ENABLE_SFS_AUX_DEVS, /* add new param generic ids above here*/ __DEVLINK_PARAM_GENERIC_ID_MAX, @@ -519,6 +520,9 @@ enum devlink_param_generic_id { #define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME "event_eq_size" #define DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE DEVLINK_PARAM_TYPE_U32 +#define DEVLINK_PARAM_GENERIC_ENABLE_SFS_AUX_DEVS_NAME "enable_sfs_aux_devs" +#define DEVLINK_PARAM_GENERIC_ENABLE_SFS_AUX_DEVS_TYPE DEVLINK_PARAM_TYPE_BOOL + #define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \ { \ .id = DEVLINK_PARAM_GENERIC_ID_##_id, \ diff --git a/net/core/devlink.c b/net/core/devlink.c index fcd9f6d85cf1..b5368024ac18 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -4477,6 +4477,11 @@ static const struct devlink_param devlink_param_generic[] = { .name = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_NAME, .type = DEVLINK_PARAM_GENERIC_EVENT_EQ_SIZE_TYPE, }, + { + .id = DEVLINK_PARAM_GENERIC_ID_ENABLE_SFS_AUX_DEVS, + .name = DEVLINK_PARAM_GENERIC_ENABLE_SFS_AUX_DEVS_NAME, + .type = DEVLINK_PARAM_GENERIC_ENABLE_SFS_AUX_DEVS_TYPE, + }, }; static int devlink_param_generic_verify(const struct devlink_param *param) From patchwork Fri Feb 11 09:20:21 2022 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Moshe Shemesh X-Patchwork-Id: 12743052 X-Patchwork-Delegate: kuba@kernel.org Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 1466FC433F5 for ; Fri, 11 Feb 2022 09:21:21 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348518AbiBKJVT (ORCPT ); Fri, 11 Feb 2022 04:21:19 -0500 Received: from mxb-00190b01.gslb.pphosted.com ([23.128.96.19]:34004 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348511AbiBKJVK (ORCPT ); Fri, 11 Feb 2022 04:21:10 -0500 Received: from NAM11-BN8-obe.outbound.protection.outlook.com (mail-bn8nam11on2073.outbound.protection.outlook.com [40.107.236.73]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 66DC01035; Fri, 11 Feb 2022 01:21:08 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=VgYpgOHNvJzAJ3+nIpewQw9vI366R6AzS7X9wfqL1cYL7W3RQ/ptimr00hHugPQTW58CXIGqWP6K8vI4I79Kfak/3NwRfNMsxJQISS8ArbhMCE0b2NDs2oOZ4RJAZANtqRsr8BNjd1xEi4A4eYF2Rp4XiGmnksiSF0bkNsf0HfHFKDBO3X83vRTSeyL61sM4BGvLPn3+wVFLrCDnP8nWhlS83cYS0DZRpuWIJluCfXRuCG3gcabZSK/547NkncminuXLp8e4PROmTQREjOzD1i14hgQWdcH1h0HWLg7Vgh7poo0YCUDIS7fk7gsF+3rnki0inKdAqtVtRZeAOJFcdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=2su7j5LSn9RrQIbcd9clW/GVTQS04MUfZbmHQWIEq6g=; b=Tpx8Lx+dhziijB47oF6kvGmtjNCdNKxXOTZP4btvQY7cxIaRPgn1IF24UfWWffwRGZZxzyvP2P20NDJpKpKrZZ8b9SfG0Amm5Z5BGrp6M7k+EPXOg2PPUQ5ZknV6kEkxRALKB+nOXNEE9j/4BwSEe5EXuncznzXVC39+eyhH/wRaqUltEuYQA9BTwyjBY8+yG2oQ+F1hz5OTkmOP53JTXX23rq7PcYIVRw7izxSHCrwHFc7Aa2qYYtdCwRmh+t5nbuM8MtToRGMki4OW7mgN7qCs60x4cy/LlERHoTKz0fapfsRaDrbpw+RsxkyhMbeZHPuM6ZvNxftF/HEIPYoKBA== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 12.22.5.235) smtp.rcpttodomain=vger.kernel.org smtp.mailfrom=nvidia.com; dmarc=pass (p=reject sp=reject pct=100) action=none header.from=nvidia.com; dkim=none (message not signed); arc=none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=Nvidia.com; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=2su7j5LSn9RrQIbcd9clW/GVTQS04MUfZbmHQWIEq6g=; b=YHMQRBQzbnrPDg2wcVHMKP2hGTc9NZtuAbcRVVAHHvDfvWp4KrOsKVsv14tBm+86u1AkxFtIUum7W29IM6ogIh+0zhlTpKhyf52qLhimiQOQPgSkHwXVa2vjgZsEgMZJD4XkKDnHGC1sZ/R6T/ofOmqayyPGI/57n95ClH11Cl8Cowy5ldfJ1HQR5n1FrIAC2xePo2MyJIuoQJILuiV07jW6XVHb7G5fGpIVYa+mEAulU7HsRY6GpYMarmURzoFw3U0/4mCn1G+zLx3m/a5gwyK0l//SrbKTeqA7nENJ/bQkwTqcRClzowd6TTEABFNubso9u8pZzF4UvfWgb++ZQA== Received: from MW4PR04CA0360.namprd04.prod.outlook.com (2603:10b6:303:8a::35) by BN9PR12MB5257.namprd12.prod.outlook.com (2603:10b6:408:11e::16) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.11; Fri, 11 Feb 2022 09:21:06 +0000 Received: from CO1NAM11FT057.eop-nam11.prod.protection.outlook.com (2603:10b6:303:8a:cafe::77) by MW4PR04CA0360.outlook.office365.com (2603:10b6:303:8a::35) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.4975.13 via Frontend Transport; Fri, 11 Feb 2022 09:21:05 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 12.22.5.235) smtp.mailfrom=nvidia.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=nvidia.com; Received-SPF: Pass (protection.outlook.com: domain of nvidia.com designates 12.22.5.235 as permitted sender) receiver=protection.outlook.com; client-ip=12.22.5.235; helo=mail.nvidia.com; Received: from mail.nvidia.com (12.22.5.235) by CO1NAM11FT057.mail.protection.outlook.com (10.13.174.205) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384) id 15.20.4975.11 via Frontend Transport; Fri, 11 Feb 2022 09:21:05 +0000 Received: from rnnvmail201.nvidia.com (10.129.68.8) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1497.18; Fri, 11 Feb 2022 09:21:04 +0000 Received: from rnnvmail203.nvidia.com (10.129.68.9) by rnnvmail201.nvidia.com (10.129.68.8) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.986.9; Fri, 11 Feb 2022 01:21:03 -0800 Received: from vdi.nvidia.com (10.127.8.14) by mail.nvidia.com (10.129.68.9) with Microsoft SMTP Server id 15.2.986.9 via Frontend Transport; Fri, 11 Feb 2022 01:21:01 -0800 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski , Parav Pandit CC: Jiri Pirko , Saeed Mahameed , , , Shay Drory Subject: [PATCH net-next v2 4/4] net/mlx5: Support enable_sfs_aux_devs devlink param Date: Fri, 11 Feb 2022 11:20:21 +0200 Message-ID: <1644571221-237302-5-git-send-email-moshe@nvidia.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1644571221-237302-1-git-send-email-moshe@nvidia.com> References: <1644571221-237302-1-git-send-email-moshe@nvidia.com> MIME-Version: 1.0 X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: 1e5b1056-3521-42cc-959f-08d9ed3fd496 X-MS-TrafficTypeDiagnostic: BN9PR12MB5257:EE_ X-Microsoft-Antispam-PRVS: X-MS-Oob-TLC-OOBClassifiers: OLM:1443; X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0; X-Microsoft-Antispam-Message-Info: +JBJoHUw1QXVfKrVp0M0m8vMkVfXz5Wkwy+svLfe9Hmvbg18kyvjcxnnoEF87NG5os2ElEvPSGU/cujDlei/ReSZUIgwWkhqcgJ1D8/dxbJtP+Bz4AxKfZvJlmOzBl6LEZ7EcHDj3XO29m8zeUS4luqyoFJNUkXaDhcGDQI3TPWLDXJT2w4M1zri6ynPutCOe21R9hrGkcEepo6jx8ZbWZdDY23AyIXtZp518LmAyudogRja9ZQhGOVon7VO8lLiBKvh7uplXlPVmX7kttGdXv3/Z2SS/rwKnFO0ZSU/sQBP2P0ezFPb/lU2yxaKisjSUOAy5LTvlaATCwiI8or+F9U2/cCGuXNIek1nZUGLFu7ki0C2pUyvEhl6+Ulig/2+tOktlJihonD46XOBf3aK7SPXzZ2MYQWWo1eORRDIyH2QmBXi60o3RL/vdDugxjU5C6e+YqMueyh/2YgsFKZYKGmuH4XYSxRHn6Ld2Oqw2YMBIi1lpZn7+DpQxCg8gcAiJPytT7AI13Bn+ckiLfz2M/8BZljAQnSCNt2z0Rqn7bQv+1tr6wYNrlpj9g/KiKl2Fd27tDWZY46CX37P3kokheZRoC6KTN+xp1YY0fkuxGxnesPbHNq8jiWjQ7u3ZZalP4BlNq7R1ahxmo2QsL5T6fbzqC7L1S5sR1P9RWbWQhLiNCmK2DITz+ORFrgvQMLddbR3PivxiMD9dyZqxhtzOo8Ogz5RlUVEQB9pT+21FB8= X-Forefront-Antispam-Report: CIP:12.22.5.235;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:mail.nvidia.com;PTR:InfoNoRecords;CAT:NONE;SFS:(13230001)(4636009)(40470700004)(46966006)(36840700001)(6636002)(8936002)(54906003)(110136005)(336012)(426003)(316002)(26005)(4326008)(8676002)(82310400004)(86362001)(70586007)(356005)(7696005)(70206006)(186003)(107886003)(81166007)(2616005)(36756003)(36860700001)(508600001)(83380400001)(47076005)(40460700003)(2906002)(5660300002)(6666004)(30864003)(21314003)(36900700001);DIR:OUT;SFP:1101; X-OriginatorOrg: Nvidia.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Feb 2022 09:21:05.3590 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: 1e5b1056-3521-42cc-959f-08d9ed3fd496 X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=43083d15-7273-40c1-b7db-39efd9ccc17a;Ip=[12.22.5.235];Helo=[mail.nvidia.com] X-MS-Exchange-CrossTenant-AuthSource: CO1NAM11FT057.eop-nam11.prod.protection.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5257 Precedence: bulk List-ID: X-Mailing-List: netdev@vger.kernel.org X-Patchwork-Delegate: kuba@kernel.org From: Shay Drory In case user wants to configure the SFs, for example: to use only vdpa functionality, he needs to fully probe a SF, configure what he wants, and afterward reload the SF. In order to save the time of the reload, exposing a devlink runtime param on the PF, that when set to off, the SFs won't create any auxiliary sub-device, so that the SF can be configured prior to its full probe. The new supported settings here should suit either if ESW and PF are on same host or not. Usage example, when ESW and SF hosting PF are the same: Disable SF aux dev probe: $ devlink dev param set pci/0000:08:00.0 name enable_sfs_aux_devs \ value false cmode runtime Create SF: $ devlink port add pci/0000:08:00.0 flavour pcisf pfnum 0 sfnum 11 $ devlink port function set pci/0000:08:00.0/32768 \ hw_addr 00:00:00:00:00:11 state active Enable ETH auxiliary device: $ devlink dev param set auxiliary/mlx5_core.sf.1 \ name enable_eth value true cmode driverinit Now, in order to fully probe the SF, use devlink reload: $ devlink dev reload auxiliary/mlx5_core.sf.1 At this point the user have SF devlink instance with auxiliary device for the Ethernet functionality only. Signed-off-by: Shay Drory Reviewed-by: Moshe Shemesh --- changelog: v1->v2: - Clarify the example in commit message suits the case of ESW and SF hosting PFs are the same. --- drivers/net/ethernet/mellanox/mlx5/core/dev.c | 16 +++ .../net/ethernet/mellanox/mlx5/core/devlink.c | 31 +++++- .../net/ethernet/mellanox/mlx5/core/health.c | 5 +- .../net/ethernet/mellanox/mlx5/core/main.c | 102 +++++++++++++++++- .../ethernet/mellanox/mlx5/core/mlx5_core.h | 6 ++ .../mellanox/mlx5/core/sf/dev/driver.c | 13 ++- .../ethernet/mellanox/mlx5/core/sf/devlink.c | 40 +++++++ .../ethernet/mellanox/mlx5/core/sf/hw_table.c | 7 ++ .../net/ethernet/mellanox/mlx5/core/sf/priv.h | 2 + include/linux/mlx5/driver.h | 1 + 10 files changed, 211 insertions(+), 12 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c index ba6dad97e308..89ead37b98ad 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c @@ -333,6 +333,18 @@ static void del_adev(struct auxiliary_device *adev) auxiliary_device_uninit(adev); } +void mlx5_dev_mark_unregistered(struct mlx5_core_dev *dev) +{ + mutex_lock(&mlx5_intf_mutex); + dev->priv.flags |= MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV; + mutex_unlock(&mlx5_intf_mutex); +} + +bool mlx5_dev_is_unregistered(struct mlx5_core_dev *dev) +{ + return dev->priv.flags & MLX5_PRIV_FLAGS_DISABLE_ALL_ADEV; +} + int mlx5_attach_device(struct mlx5_core_dev *dev) { struct mlx5_priv *priv = &dev->priv; @@ -473,6 +485,10 @@ static int add_drivers(struct mlx5_core_dev *dev) if (!is_supported) continue; + if (mlx5_adev_devices[i].is_enabled && + !(mlx5_adev_devices[i].is_enabled(dev))) + continue; + priv->adev[i] = add_adev(dev, i); if (IS_ERR(priv->adev[i])) { mlx5_core_warn(dev, "Device[%d] (%s) failed to load\n", diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index e832a3f4c18a..1b769b5da577 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -139,6 +139,13 @@ static int mlx5_devlink_reload_down(struct devlink *devlink, bool netns_change, struct pci_dev *pdev = dev->pdev; bool sf_dev_allocated; + if (mlx5_dev_is_unregistered(dev)) { + if (action != DEVLINK_RELOAD_ACTION_DRIVER_REINIT) + return -EOPNOTSUPP; + mlx5_unload_one_light(dev); + return 0; + } + sf_dev_allocated = mlx5_sf_dev_allocated(dev); if (sf_dev_allocated) { /* Reload results in deleting SF device which further results in @@ -182,6 +189,10 @@ static int mlx5_devlink_reload_up(struct devlink *devlink, enum devlink_reload_a *actions_performed = BIT(action); switch (action) { case DEVLINK_RELOAD_ACTION_DRIVER_REINIT: + if (mlx5_dev_is_unregistered(dev)) { + mlx5_fw_reporters_create(dev); + return mlx5_init_one(dev); + } return mlx5_load_one(dev); case DEVLINK_RELOAD_ACTION_FW_ACTIVATE: if (limit == DEVLINK_RELOAD_LIMIT_NO_RESET) @@ -258,6 +269,9 @@ static int mlx5_devlink_trap_action_set(struct devlink *devlink, struct mlx5_devlink_trap *dl_trap; int err = 0; + if (mlx5_dev_is_unregistered(dev)) + return -EOPNOTSUPP; + if (is_mdev_switchdev_mode(dev)) { NL_SET_ERR_MSG_MOD(extack, "Devlink traps can't be set in switchdev mode"); return -EOPNOTSUPP; @@ -440,6 +454,9 @@ static int mlx5_devlink_fs_mode_get(struct devlink *devlink, u32 id, { struct mlx5_core_dev *dev = devlink_priv(devlink); + if (mlx5_dev_is_unregistered(dev)) + return -EOPNOTSUPP; + if (dev->priv.steering->mode == MLX5_FLOW_STEERING_MODE_SMFS) strcpy(ctx->val.vstr, "smfs"); else @@ -499,6 +516,9 @@ static int mlx5_devlink_esw_port_metadata_get(struct devlink *devlink, u32 id, { struct mlx5_core_dev *dev = devlink_priv(devlink); + if (mlx5_dev_is_unregistered(dev)) + return -EOPNOTSUPP; + if (!MLX5_ESWITCH_MANAGER(dev)) return -EOPNOTSUPP; @@ -542,6 +562,9 @@ static int mlx5_devlink_enable_remote_dev_reset_get(struct devlink *devlink, u32 { struct mlx5_core_dev *dev = devlink_priv(devlink); + if (mlx5_dev_is_unregistered(dev)) + return -EOPNOTSUPP; + ctx->val.vbool = mlx5_fw_reset_enable_remote_dev_reset_get(dev); return 0; } @@ -588,7 +611,7 @@ static void mlx5_devlink_set_params_init_values(struct devlink *devlink) struct mlx5_core_dev *dev = devlink_priv(devlink); union devlink_param_value value; - value.vbool = MLX5_CAP_GEN(dev, roce); + value.vbool = MLX5_CAP_GEN(dev, roce) && !mlx5_dev_is_unregistered(dev); devlink_param_driverinit_value_set(devlink, DEVLINK_PARAM_GENERIC_ID_ENABLE_ROCE, value); @@ -628,7 +651,7 @@ static int mlx5_devlink_eth_param_register(struct devlink *devlink) if (err) return err; - value.vbool = true; + value.vbool = !mlx5_dev_is_unregistered(dev); devlink_param_driverinit_value_set(devlink, DEVLINK_PARAM_GENERIC_ID_ENABLE_ETH, value); @@ -673,7 +696,7 @@ static int mlx5_devlink_rdma_param_register(struct devlink *devlink) if (err) return err; - value.vbool = true; + value.vbool = !mlx5_dev_is_unregistered(devlink_priv(devlink)); devlink_param_driverinit_value_set(devlink, DEVLINK_PARAM_GENERIC_ID_ENABLE_RDMA, value); @@ -705,7 +728,7 @@ static int mlx5_devlink_vnet_param_register(struct devlink *devlink) if (err) return err; - value.vbool = true; + value.vbool = !mlx5_dev_is_unregistered(dev); devlink_param_driverinit_value_set(devlink, DEVLINK_PARAM_GENERIC_ID_ENABLE_VNET, value); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/health.c b/drivers/net/ethernet/mellanox/mlx5/core/health.c index 737df402c927..1b40ada0e9c2 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/health.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/health.c @@ -700,7 +700,7 @@ static const struct devlink_health_reporter_ops mlx5_fw_fatal_reporter_ops = { }; #define MLX5_REPORTER_FW_GRACEFUL_PERIOD 1200000 -static void mlx5_fw_reporters_create(struct mlx5_core_dev *dev) +void mlx5_fw_reporters_create(struct mlx5_core_dev *dev) { struct mlx5_core_health *health = &dev->priv.health; struct devlink *devlink = priv_to_devlink(dev); @@ -893,7 +893,8 @@ int mlx5_health_init(struct mlx5_core_dev *dev) struct mlx5_core_health *health; char *name; - mlx5_fw_reporters_create(dev); + if (!mlx5_dev_is_unregistered(dev)) + mlx5_fw_reporters_create(dev); health = &dev->priv.health; name = kmalloc(64, GFP_KERNEL); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/main.c b/drivers/net/ethernet/mellanox/mlx5/core/main.c index 73d2cec01ead..b5e40ad0b5fb 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/main.c @@ -1314,6 +1314,7 @@ static void mlx5_unload(struct mlx5_core_dev *dev) int mlx5_init_one(struct mlx5_core_dev *dev) { + bool light_probe = mlx5_dev_is_unregistered(dev); int err = 0; mutex_lock(&dev->intf_state_mutex); @@ -1335,9 +1336,14 @@ int mlx5_init_one(struct mlx5_core_dev *dev) set_bit(MLX5_INTERFACE_STATE_UP, &dev->intf_state); - err = mlx5_devlink_register(priv_to_devlink(dev)); - if (err) - goto err_devlink_reg; + /* In case of light_probe, mlx5_devlink is already registered. + * Hence, don't register devlink again. + */ + if (!light_probe) { + err = mlx5_devlink_register(priv_to_devlink(dev)); + if (err) + goto err_devlink_reg; + } err = mlx5_register_device(dev); if (err) @@ -1347,7 +1353,8 @@ int mlx5_init_one(struct mlx5_core_dev *dev) return 0; err_register: - mlx5_devlink_unregister(priv_to_devlink(dev)); + if (!light_probe) + mlx5_devlink_unregister(priv_to_devlink(dev)); err_devlink_reg: clear_bit(MLX5_INTERFACE_STATE_UP, &dev->intf_state); mlx5_unload(dev); @@ -1443,6 +1450,93 @@ void mlx5_unload_one(struct mlx5_core_dev *dev) mutex_unlock(&dev->intf_state_mutex); } +/* In case of light probe, we don't need a full query of hca_caps, but only the bellow caps. + * A full query of hca_caps will be done when the device will reload. + */ +static int mlx5_query_hca_caps_light(struct mlx5_core_dev *dev) +{ + int err; + + err = mlx5_core_get_caps(dev, MLX5_CAP_GENERAL); + if (err) + return err; + + if (MLX5_CAP_GEN(dev, eth_net_offloads)) { + err = mlx5_core_get_caps(dev, MLX5_CAP_ETHERNET_OFFLOADS); + if (err) + return err; + } + + if (MLX5_CAP_GEN(dev, nic_flow_table) || + MLX5_CAP_GEN(dev, ipoib_enhanced_offloads)) { + err = mlx5_core_get_caps(dev, MLX5_CAP_FLOW_TABLE); + if (err) + return err; + } + + if (MLX5_CAP_GEN_64(dev, general_obj_types) & + MLX5_GENERAL_OBJ_TYPES_CAP_VIRTIO_NET_Q) { + err = mlx5_core_get_caps(dev, MLX5_CAP_VDPA_EMULATION); + if (err) + return err; + } + + return 0; +} + +int mlx5_init_one_light(struct mlx5_core_dev *dev) +{ + int err; + + dev->state = MLX5_DEVICE_STATE_UP; + err = mlx5_function_enable(dev); + if (err) { + mlx5_core_warn(dev, "mlx5_function_enable err=%d\n", err); + goto out; + } + + err = mlx5_query_hca_caps_light(dev); + if (err) { + mlx5_core_warn(dev, "mlx5_query_hca_caps_light err=%d\n", err); + goto query_hca_caps_err; + } + + err = mlx5_devlink_register(priv_to_devlink(dev)); + if (err) { + mlx5_core_warn(dev, "mlx5_devlink_reg err = %d\n", err); + goto query_hca_caps_err; + } + + return 0; + +query_hca_caps_err: + mlx5_function_disable(dev); +out: + dev->state = MLX5_DEVICE_STATE_INTERNAL_ERROR; + return err; +} + +void mlx5_uninit_one_light(struct mlx5_core_dev *dev) +{ + mlx5_devlink_unregister(priv_to_devlink(dev)); + if (dev->state != MLX5_DEVICE_STATE_UP) + return; + mlx5_function_disable(dev); +} + +/* xxx_ligth() function are used in order to configure the device without full + * init (light init). e.g.: There isn't a point in reload a device to light state. + * Hence, mlx5_load_one_light() isn't needed. + */ + +void mlx5_unload_one_light(struct mlx5_core_dev *dev) +{ + if (dev->state != MLX5_DEVICE_STATE_UP) + return; + mlx5_function_disable(dev); + dev->state = MLX5_DEVICE_STATE_INTERNAL_ERROR; +} + static const int types[] = { MLX5_CAP_GENERAL, MLX5_CAP_GENERAL_2, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index 6f8baa0f2a73..e2d827474552 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -209,11 +209,14 @@ int mlx5_attach_device(struct mlx5_core_dev *dev); void mlx5_detach_device(struct mlx5_core_dev *dev); int mlx5_register_device(struct mlx5_core_dev *dev); void mlx5_unregister_device(struct mlx5_core_dev *dev); +void mlx5_dev_mark_unregistered(struct mlx5_core_dev *dev); +bool mlx5_dev_is_unregistered(struct mlx5_core_dev *dev); struct mlx5_core_dev *mlx5_get_next_phys_dev(struct mlx5_core_dev *dev); void mlx5_dev_list_lock(void); void mlx5_dev_list_unlock(void); int mlx5_dev_list_trylock(void); +void mlx5_fw_reporters_create(struct mlx5_core_dev *dev); int mlx5_query_mtpps(struct mlx5_core_dev *dev, u32 *mtpps, u32 mtpps_size); int mlx5_set_mtpps(struct mlx5_core_dev *mdev, u32 *mtpps, u32 mtpps_size); int mlx5_query_mtppse(struct mlx5_core_dev *mdev, u8 pin, u8 *arm, u8 *mode); @@ -291,6 +294,9 @@ int mlx5_init_one(struct mlx5_core_dev *dev); void mlx5_uninit_one(struct mlx5_core_dev *dev); void mlx5_unload_one(struct mlx5_core_dev *dev); int mlx5_load_one(struct mlx5_core_dev *dev); +int mlx5_init_one_light(struct mlx5_core_dev *dev); +void mlx5_uninit_one_light(struct mlx5_core_dev *dev); +void mlx5_unload_one_light(struct mlx5_core_dev *dev); int mlx5_vport_get_other_func_cap(struct mlx5_core_dev *dev, u16 function_id, void *out); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c index 7b4783ce213e..8d6e0e44e614 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c @@ -28,6 +28,9 @@ static int mlx5_sf_dev_probe(struct auxiliary_device *adev, const struct auxilia mdev->priv.adev_idx = adev->id; sf_dev->mdev = mdev; + if (sf_dev->parent_mdev->priv.sfs_light_probe) + mlx5_dev_mark_unregistered(mdev); + err = mlx5_mdev_init(mdev, MLX5_DEFAULT_PROF); if (err) { mlx5_core_warn(mdev, "mlx5_mdev_init on err=%d\n", err); @@ -41,7 +44,10 @@ static int mlx5_sf_dev_probe(struct auxiliary_device *adev, const struct auxilia goto remap_err; } - err = mlx5_init_one(mdev); + if (sf_dev->parent_mdev->priv.sfs_light_probe) + err = mlx5_init_one_light(mdev); + else + err = mlx5_init_one(mdev); if (err) { mlx5_core_warn(mdev, "mlx5_init_one err=%d\n", err); goto init_one_err; @@ -64,7 +70,10 @@ static void mlx5_sf_dev_remove(struct auxiliary_device *adev) struct devlink *devlink = priv_to_devlink(sf_dev->mdev); devlink_unregister(devlink); - mlx5_uninit_one(sf_dev->mdev); + if (mlx5_dev_is_unregistered(sf_dev->mdev)) + mlx5_uninit_one_light(sf_dev->mdev); + else + mlx5_uninit_one(sf_dev->mdev); iounmap(sf_dev->mdev->iseg); mlx5_mdev_uninit(sf_dev->mdev); mlx5_devlink_free(devlink); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c index 3be659cd91f1..e3f6066ab7af 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c @@ -10,6 +10,7 @@ #include "ecpf.h" #define CREATE_TRACE_POINTS #include "diag/sf_tracepoint.h" +#include "devlink.h" struct mlx5_sf { struct devlink_port dl_port; @@ -456,6 +457,44 @@ static int mlx5_sf_vhca_event(struct notifier_block *nb, unsigned long opcode, v return 0; } +static int mlx5_devlink_sfs_light_probe_get(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + + ctx->val.vbool = !dev->priv.sfs_light_probe; + return 0; +} + +static int mlx5_devlink_sfs_light_probe_set(struct devlink *devlink, u32 id, + struct devlink_param_gset_ctx *ctx) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + + dev->priv.sfs_light_probe = !ctx->val.vbool; + return 0; +} + +static const struct devlink_param sfs_light_probe_param = + DEVLINK_PARAM_GENERIC(ENABLE_SFS_AUX_DEVS, BIT(DEVLINK_PARAM_CMODE_RUNTIME), + mlx5_devlink_sfs_light_probe_get, + mlx5_devlink_sfs_light_probe_set, NULL); + +int mlx5_devlink_sfs_light_probe_param_register(struct devlink *devlink) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + + if (!mlx5_core_is_pf(dev) || !mlx5_sf_max_functions(dev)) + return 0; + + return devlink_param_register(devlink, &sfs_light_probe_param); +} + +void mlx5_devlink_sfs_light_probe_param_unregister(struct devlink *devlink) +{ + devlink_param_unregister(devlink, &sfs_light_probe_param); +} + static void mlx5_sf_table_enable(struct mlx5_sf_table *table) { init_completion(&table->disable_complete); @@ -533,6 +572,7 @@ int mlx5_sf_table_init(struct mlx5_core_dev *dev) table->dev = dev; xa_init(&table->port_indices); dev->priv.sf_table = table; + dev->priv.sfs_light_probe = false; refcount_set(&table->refcount, 0); table->esw_nb.notifier_call = mlx5_sf_esw_event; err = mlx5_esw_event_notifier_register(dev->priv.eswitch, &table->esw_nb); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c index 17aa348989cb..664704598b76 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c @@ -283,9 +283,15 @@ int mlx5_sf_hw_table_init(struct mlx5_core_dev *dev) if (err) goto ext_err; + err = mlx5_devlink_sfs_light_probe_param_register(priv_to_devlink(dev)); + if (err) + goto sfs_reg_err; + mlx5_core_dbg(dev, "SF HW table: max sfs = %d, ext sfs = %d\n", max_fn, max_ext_fn); return 0; +sfs_reg_err: + mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_EXTERNAL]); ext_err: mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_LOCAL]); table_err: @@ -301,6 +307,7 @@ void mlx5_sf_hw_table_cleanup(struct mlx5_core_dev *dev) if (!table) return; + mlx5_devlink_sfs_light_probe_param_unregister(priv_to_devlink(dev)); mutex_destroy(&table->table_lock); mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_EXTERNAL]); mlx5_sf_hw_table_hwc_cleanup(&table->hwc[MLX5_SF_HWC_LOCAL]); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h b/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h index 7114f3fc335f..c228edc8e43c 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h @@ -19,4 +19,6 @@ void mlx5_sf_hw_table_sf_free(struct mlx5_core_dev *dev, u32 controller, u16 id) void mlx5_sf_hw_table_sf_deferred_free(struct mlx5_core_dev *dev, u32 controller, u16 id); bool mlx5_sf_hw_table_supported(const struct mlx5_core_dev *dev); +int mlx5_devlink_sfs_light_probe_param_register(struct devlink *devlink); +void mlx5_devlink_sfs_light_probe_param_unregister(struct devlink *devlink); #endif diff --git a/include/linux/mlx5/driver.h b/include/linux/mlx5/driver.h index 78655d8d13a7..fd07175896ff 100644 --- a/include/linux/mlx5/driver.h +++ b/include/linux/mlx5/driver.h @@ -604,6 +604,7 @@ struct mlx5_priv { struct mlx5_vhca_state_notifier *vhca_state_notifier; struct mlx5_sf_dev_table *sf_dev_table; struct mlx5_core_dev *parent_mdev; + u8 sfs_light_probe:1; #endif #ifdef CONFIG_MLX5_SF_MANAGER struct mlx5_sf_hw_table *sf_hw_table;